CN102111610A - Panoramic image generating method for assisting driving - Google Patents
Panoramic image generating method for assisting driving Download PDFInfo
- Publication number
- CN102111610A CN102111610A CN201010180178.0A CN201010180178A CN102111610A CN 102111610 A CN102111610 A CN 102111610A CN 201010180178 A CN201010180178 A CN 201010180178A CN 102111610 A CN102111610 A CN 102111610A
- Authority
- CN
- China
- Prior art keywords
- camera
- panoramic picture
- coordinate system
- coordinate
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Closed-Circuit Television Systems (AREA)
- Image Processing (AREA)
Abstract
A panoramic image generating method for assisting driving comprises the following steps: cameras are respectively arranged on all sides of an auto body; the panoramic picture including icons on the auto body and four areas (a front area, a rear area, a left area and a right area) surrounding the icons on the auto body is defined; frame pictures are synchronously extracted from the video stream output by the four cameras through a video acquisition unit; the mapping chart of a world coordinate system-camera image coordinate system is used for calculation, and the synchronously extracted four frames of pictures are mapped to corresponding areas of the panoramic picture to form one frame of panoramic picture; and video coding is performed to the synthetic panoramic picture, the video-coded synthetic panoramic picture is output to a vehicle-mounted display screen, so as to provide a panoramic image within 360 degrees around the auto body for a driver in real time. The method provided by the invention can achieve real non-blind area driving. The generation method of the mapping chart is simple, convenient to operate, and is suitable for commercialization.
Description
Technical field
The present invention relates to vehicle electric field, particularly a kind of panorama type image generation method that is used for driver assistance.
Background technology
There are a lot of blind areas in car steering person in driving procedure, as the place that is blocked, vehicle body the place ahead, blind area behind sided mirror unit blind area and the car.These automobile blind areas potential danger, tend to bring the heavy losses and the spiritual grievous injury of lives and properties to people.In addition, along with the sharp increase of automobile quantity, when private car was come into huge numbers of families, it is nervous all the more that each parking stall, incity, big city becomes, more and more higher to the requirement of driver's parking technical ability.And the existence of automobile blind area has strengthened the difficulty of stopping especially, slightly because of carelessness will scratch or with adjacent vehicle collision.Therefore, blind zone problem has received concern more and more.
Common in the market drive assist system mainly is divided into two classes: a kind of is device such as sensor installation around the vehicle body, when its near and report to the police when having detected barrier.With the radar for backing car is main representative, whether barrier and the barrier distance apart from car is arranged after the employing ultrasonic ranging principle probe vehicles, and send alarm signal.This series products reduces to a great extent or has remedied the automobile blind area, reminds the existence of driver's car and obstruction.Yet radar for backing car is not what be perfectly safe yet, and when low excessively, too high or angle surpassed the scope of probe detecting when barrier, radar for backing car just can lose efficacy.Another kind is the image backing system, utilize rearview camera with the things image definition at car rear be transferred on the console LCDs.And this image system has only solved blind area behind the car, and vehicle body the place ahead and rear view mirror blind zone still exist.Therefore, non-blind area is driven the problem to be solved that remains.
Summary of the invention
The purpose of this invention is to provide a kind of panorama type image generation method that is used for driver assistance, it is by handling in real time the video of the multichannel camera collection around the vehicle body and showing, offer driver's vehicle body 360 degree full-view images on every side, realize real non-blind area driving.
The panorama type image generation method that is used for driver assistance provided by the invention may further comprise the steps:
In the vehicle body front, rear, left and right camera is set respectively;
The definition panoramic picture is formed by the vehicle body icon with around four zones, the front, rear, left and right of vehicle body icon, and every bit x in the panoramic picture ", " with corresponding points X in the world coordinate system, satisfy x "=aX, y "=aY between the Y, a is a zoom factor to y;
Video acquisition unit is extracted two field picture synchronously from the video flowing of four-way CCD camera output;
Calculate with the mapping chart of world coordinate system-camera image coordinate system, four two field pictures that extract synchronously are mapped to the corresponding region of panoramic picture, form a frame panoramic picture;
Synthetic panoramic picture is carried out video coding, output to vehicle-carrying display screen, provide the vehicle body full-view images of 360 degree on every side for the driver in real time;
Wherein, the mapping chart of every road camera corresponding world coordinate system-camera image coordinate system is generated by following steps:
1) on this ground, camera the place ahead, road, arranges at least four gauge points, obtains their image coordinate in this camera image coordinate system, and the image coordinate of obtaining is carried out distortion correction that the distortion correction model is
Δx＝x′-x，Δy＝y′-y (2)
Wherein, n is a positive integer, x, and y is the coordinate before proofreading and correct, x ', y ' is the coordinate after proofreading and correct, x
0, y
0Be selected photocentre point coordinates,
2) with the image coordinate after each gauge point correction and the weight parameter matrix L of its world coordinate system coordinate Calculation linear transformation formula (3)
-v＝XL-b (3)
Wherein, v is the correction matrix of measured value; X is a coordinates matrix, by world coordinate system coordinate variable X, and the coordinate variable x ' of Y and correction back single camera image, y ' formation; B is the coordinate x ' of single camera image, the matrix of y ' formation;
3) every bit of this camera corresponding region, road in the traversal panoramic picture, calculate image coordinate x ' behind the corresponding distortion correction with formula (3), y ' obtains their corresponding coordinate x in this road camera image coordinate system, y by formula (1), (2) inverse then.
The actual conditions of 360 degree convert panoramic video in real time to and offer the driver around the vehicle body that the inventive method can be taken the multichannel camera, thereby can realize real non-blind area driving.Its mapping graph table generating method is simple, is convenient to operation, is fit to commercialization.
Description of drawings
Fig. 1 is the system framework figure of panorama type image generation system of the present invention;
Fig. 2 is a panoramic picture area dividing schematic diagram;
The schematic diagram that Fig. 3 generates for panoramic picture;
Fig. 4 is the camera calibration flow chart;
Fig. 5 a, b are the camera calibration schematic diagram, and wherein a figure is ground six gauge points; B figure is the gauge point that shows in the camera image.
Embodiment
Below in conjunction with accompanying drawing the present invention is further specified.
With reference to Fig. 1, panorama type image generation system of the present invention comprises master control borad 2, multichannel camera 1, reversing trigger control unit 3, control input unit 4 and vehicle-mounted display unit 5.Camera 1 adopts the wide-angle imaging head, and is the rarest four the tunnel, is respectively applied for the video that obtains the vehicle body front, rear, left and right.Reversing trigger control unit 3 is connected with master control borad 2, and generation+12V is as the triggering signal of master control borad 2 work after automobile is hung reverse gear.Control input unit 4 can be touch-screen or button, is used for carrying out alternately with master control borad 2, in the process that automobile positive travels, also can or quit work by 4 control master control borads, 2 beginnings of control input unit.The demonstration triggering signal of output+12V was given vehicle-mounted display unit 5 after master control borad 2 was started working, and notified vehicle-mounted display unit 5 to receive from the video of master control borad 2 and exported.Vehicle-mounted display unit 5 can adopt the display screen of the original audio-video system of automobile, as vehicle-carrying DVD etc.
Generate the method for panorama type image, may further comprise the steps:
In the vehicle body front, rear, left and right camera 1 is set respectively;
The definition panoramic picture is formed (as shown in Figure 2) by the vehicle body icon with around four zones, the front, rear, left and right of vehicle body icon, and every bit x in the panoramic picture ", y is " with corresponding points X in the world coordinate system, it between the Y simple scaling relation, "=aX, the y "=aY that is x, a is a zoom factor;
When the automobile gear is hung into reverse gear, or in the automobile positive driving process by the mode of button, to system's transmission work triggering signal, start camera 1 and master control borad 2, video acquisition unit on the master control borad 2 is extracted two field picture to the video flowing synchronous acquisition of four-way CCD camera 1 output; The mapping chart of master control borad 2 usefulness world coordinate system-camera image coordinate system calculates, with extract synchronously from four two field picture correspondence mappings of four cameras in vehicle body front, rear, left and right four zones, front, rear, left and right to panoramic picture, form a frame panoramic picture; Synthetic panoramic picture is carried out video coding, output to vehicle-mounted display unit 5, provide the vehicle body full-view images of 360 degree on every side for the driver in real time.
Fig. 3 is the schematic diagram that panoramic picture of the present invention generates, and with reference to Fig. 3, for each pixel in the panoramic picture, corresponds to respective pixel in the corresponding camera image by the mapping chart, and the information with this pixel is filled in the panoramic picture then.The mapping chart is set up one-to-one relationship with image coordinate system and world coordinate system that camera obtains.The mapping chart is that the demarcation to camera 1 generates during by first install, and is stored in the Flash memory of master control borad 2, is used for finishing in follow-up use the conversion of video input, output.
Fig. 4 is the camera calibration flow chart, and each camera 1 need be demarcated respectively, obtains its mapping chart.Wherein, the mapping chart of every road camera 1 corresponding world coordinate system-camera image coordinate system is obtained by following steps:
1) on this ground, road camera 1 the place ahead, arranges six gauge points (shown in Fig. 5 a), the record coordinate position, enter calibration interface then, with reference to Fig. 5 b, spider on the screen is adjusted to the center of six gauge point images successively, obtain the image coordinate of six gauge points in this camera image coordinate system; Because what adopt is the wide-angle imaging head, picture distortion is bigger, need at first carry out distortion correction, and the distortion correction model is
Δx＝x′-x，Δy＝y′-y (2)
Wherein, n is a positive integer, x, and y is the coordinate before proofreading and correct, x ', y ' is the coordinate after proofreading and correct, x
0, y
0Be selected photocentre point coordinates,
2), calculate the weight parameter matrix L of linear transformation formula (3) with the image coordinate after the correction of each gauge point and its world coordinate system coordinate substitution linear transformation formula (3)
-v＝XL-b (3)
Wherein, v is the correction matrix of measured value, and X is a coordinates matrix, by world coordinate system coordinate variable X, and the coordinate variable x ' of Y and correction back single camera image, y ' formation, b is the coordinate x ' of single camera image, the matrix of y ' formation;
3) every bit of this camera corresponding region, road in the traversal panoramic picture, calculate image coordinate x ' behind the corresponding distortion correction with formula (3), y ', obtain the coordinate x of their correspondences in this road camera image coordinate system then by formula (1), (2) inverse, y, thus the mapping chart of this road camera corresponding world coordinate system-camera image coordinate system generated.The inverse process is used the method that is similar to Newton iteration.Iterative convergence speed is very fast, usually the result that can obtain expecting with interior iteration for 10 times.
Above-mentioned steps 1) in, can determine photocentre point coordinates x by the following method
0, y
0: adopt special cam device, make and select the imaging region of camera lens to be completely contained in the photosensitive region of sensitive chip.In the image that obtains, the imaging region of camera lens is circular, is the black background zone beyond the imaging region.By calculating the circular position of the center of circle in image, obtain photocentre point coordinates x
0, y
0
Parameter n adopts artificial method to choose, and by attempting using different parameter n that lens distortion is proofreaied and correct, artificially selectes a n value of visual effect the best.Than the method for setting up iterative behind the calibration model, process is simple, and does not have the problem that does not restrain.
Above-mentioned steps 2) in, each element among the correction matrix v of hypothetic observation is that desired value is 0 stochastic variable.
Claims (4)
1. panorama type image generation method that is used for driver assistance may further comprise the steps:
In the vehicle body front, rear, left and right camera is set respectively;
The definition panoramic picture is formed by the vehicle body icon with around four zones, the front, rear, left and right of vehicle body icon, and every bit x in the panoramic picture ", " with corresponding points X in the world coordinate system, satisfy x "=aX, y "=aY between the Y, a is a zoom factor to y;
Video acquisition unit is extracted two field picture synchronously from the video flowing of four-way CCD camera output;
Calculate with the mapping chart of world coordinate system one camera image coordinate system, four two field pictures that extract synchronously are mapped to the corresponding region of panoramic picture, form a frame panoramic picture;
Synthetic panoramic picture is carried out video coding, output to vehicle-carrying display screen, provide the vehicle body full-view images of 360 degree on every side for the driver in real time;
Wherein, every road camera corresponding world coordinate is that the mapping chart of a camera image coordinate system is generated by following steps:
1) on this ground, camera the place ahead, road, arranges at least four gauge points, obtains their image coordinate in this camera image coordinate system, and the image coordinate of obtaining is carried out distortion correction that the distortion correction model is
Δx＝x′-x，Δy＝y′-y (2)
Wherein, n is a positive integer, x, and y is the coordinate before proofreading and correct, x ', y ' is the coordinate after proofreading and correct, x
0, y
0Be selected photocentre point coordinates,
2) with the image coordinate after each gauge point correction and the weight parameter matrix L of its world coordinate system coordinate Calculation linear transformation formula (3)
-v＝XL-b (3)
Wherein, v is the correction matrix of measured value; X is a coordinates matrix, by world coordinate system coordinate variable X, and the coordinate variable x ' of Y and correction back single camera image, y ' formation; B is the coordinate x ' of single camera image, the matrix of y ' formation;
3) every bit of this camera corresponding region, road in the traversal panoramic picture, calculate image coordinate x ' behind the corresponding distortion correction with formula (3), y ' obtains their corresponding coordinate x in this road camera image coordinate system, y by formula (1), (2) inverse then.
2. panorama type image generation method according to claim 1 is characterized in that, in the described step 1), determines photocentre point coordinates x
0, y
0Method be: adopt special cam device, make to select the imaging region of camera lens be completely contained in the photosensitive region of sensitive chip; In the image that obtains, the imaging region of camera lens is circular, is the black background zone beyond the imaging region; By calculating the circular position of the center of circle in image, obtain photocentre point coordinates x
0, y
0
3. panorama type image generation method according to claim 1 is characterized in that: start or start with pressing key control by the reverse gear shift triggering.
4. panorama type image generation method according to claim 1 is characterized in that: described camera is the wide-angle imaging head.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010180178.0A CN102111610B (en) | 2010-05-21 | 2010-05-21 | Panoramic image generating method for assisting driving |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010180178.0A CN102111610B (en) | 2010-05-21 | 2010-05-21 | Panoramic image generating method for assisting driving |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102111610A true CN102111610A (en) | 2011-06-29 |
| CN102111610B CN102111610B (en) | 2012-08-29 |
Family
ID=44175616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010180178.0A Expired - Fee Related CN102111610B (en) | 2010-05-21 | 2010-05-21 | Panoramic image generating method for assisting driving |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102111610B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024428A (en) * | 2012-12-04 | 2013-04-03 | 同致电子科技（厦门）有限公司 | Automatic correction device of panorama image system and correction method |
| CN103253194A (en) * | 2013-05-17 | 2013-08-21 | 黄梅县云泰时代光学仪器有限公司 | Traveling vehicle auxiliary system |
| CN103253193A (en) * | 2013-04-23 | 2013-08-21 | 上海纵目科技有限公司 | Method and system of calibration of panoramic parking based on touch screen operation |
| CN104163135A (en) * | 2014-07-30 | 2014-11-26 | 麦特汽车服务股份有限公司 | Wide-visual-angle displaying method and device for automobile rearview mirror information |
| WO2015043507A1 (en) * | 2013-09-27 | 2015-04-02 | 比亚迪股份有限公司 | Image processing method and apparatus for cars, method for generating car surround view image, and car surround view system |
| CN105015418A (en) * | 2015-07-17 | 2015-11-04 | 惠州华阳通用电子有限公司 | Video switching method for vehicle-mounted display screen |
| CN105825475A (en) * | 2016-04-01 | 2016-08-03 | 西安电子科技大学 | 360-degree panorama image generation method based on single pick-up head |
| CN106355546A (en) * | 2015-07-13 | 2017-01-25 | 比亚迪股份有限公司 | Vehicle panorama generating method and apparatus |
| CN108008596A (en) * | 2017-12-29 | 2018-05-08 | 百维雅（东莞）网络技术有限公司 | A kind of 360 degree of full-view image systems of solid and implementation method |
| CN108335273A (en) * | 2018-02-06 | 2018-07-27 | 大唐终端技术有限公司 | The real-time removing method of the distortion of big wide-angle flake full shot camera |
| WO2019192359A1 (en) * | 2018-04-02 | 2019-10-10 | 杭州海康威视数字技术股份有限公司 | Vehicle panoramic video display system and method, and vehicle controller |
| CN112026496A (en) * | 2019-06-04 | 2020-12-04 | 上海擎感智能科技有限公司 | Vehicle window display method, front end, rear end, medium and equipment |
| CN112565590A (en) * | 2020-11-16 | 2021-03-26 | 李诚专 | Object 360-degree all-round-looking image generation method |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104506764A (en) * | 2014-11-17 | 2015-04-08 | 南京泓众电子科技有限公司 | An automobile traveling recording system based on a spliced video image |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101425181A (en) * | 2008-12-15 | 2009-05-06 | 浙江大学 | Panoramic view vision auxiliary parking system demarcating method |
| CN101442618A (en) * | 2008-12-31 | 2009-05-27 | 葛晨阳 | Method for synthesizing 360 DEG ring-shaped video of vehicle assistant drive |
| WO2009066506A1 (en) * | 2007-11-20 | 2009-05-28 | Sanyo Electric Co., Ltd. | Operation support system, vehicle, and method for estimating three-dimensional object area |
-
2010
- 2010-05-21 CN CN201010180178.0A patent/CN102111610B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009066506A1 (en) * | 2007-11-20 | 2009-05-28 | Sanyo Electric Co., Ltd. | Operation support system, vehicle, and method for estimating three-dimensional object area |
| JP2009129001A (en) * | 2007-11-20 | 2009-06-11 | Sanyo Electric Co Ltd | Operation support system, vehicle, and method for estimating three-dimensional object area |
| CN101425181A (en) * | 2008-12-15 | 2009-05-06 | 浙江大学 | Panoramic view vision auxiliary parking system demarcating method |
| CN101442618A (en) * | 2008-12-31 | 2009-05-27 | 葛晨阳 | Method for synthesizing 360 DEG ring-shaped video of vehicle assistant drive |
Non-Patent Citations (2)
| Title |
|---|
| 《Hitachi Review》 20001231 Jiro Takezaki Support System for Safe Driving 第29卷, 第3期 * |
| 《计算机辅助设计与图形学学报》 20091031 王德鑫，等 四路摄像头协同的多重触控系统 第21卷, 第10期 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103024428A (en) * | 2012-12-04 | 2013-04-03 | 同致电子科技（厦门）有限公司 | Automatic correction device of panorama image system and correction method |
| CN103253193A (en) * | 2013-04-23 | 2013-08-21 | 上海纵目科技有限公司 | Method and system of calibration of panoramic parking based on touch screen operation |
| CN103253194A (en) * | 2013-05-17 | 2013-08-21 | 黄梅县云泰时代光学仪器有限公司 | Traveling vehicle auxiliary system |
| CN104512328B (en) * | 2013-09-27 | 2016-11-09 | 比亚迪股份有限公司 | Automobile looks around image generating method and automobile viewing system |
| WO2015043507A1 (en) * | 2013-09-27 | 2015-04-02 | 比亚迪股份有限公司 | Image processing method and apparatus for cars, method for generating car surround view image, and car surround view system |
| CN104163135A (en) * | 2014-07-30 | 2014-11-26 | 麦特汽车服务股份有限公司 | Wide-visual-angle displaying method and device for automobile rearview mirror information |
| CN106355546A (en) * | 2015-07-13 | 2017-01-25 | 比亚迪股份有限公司 | Vehicle panorama generating method and apparatus |
| CN106355546B (en) * | 2015-07-13 | 2019-11-05 | 比亚迪股份有限公司 | The generation method and device of vehicle panoramic image |
| CN105015418A (en) * | 2015-07-17 | 2015-11-04 | 惠州华阳通用电子有限公司 | Video switching method for vehicle-mounted display screen |
| CN105825475A (en) * | 2016-04-01 | 2016-08-03 | 西安电子科技大学 | 360-degree panorama image generation method based on single pick-up head |
| CN105825475B (en) * | 2016-04-01 | 2019-01-29 | 西安电子科技大学 | 360 degree of full-view image generation methods based on single camera |
| CN108008596A (en) * | 2017-12-29 | 2018-05-08 | 百维雅（东莞）网络技术有限公司 | A kind of 360 degree of full-view image systems of solid and implementation method |
| CN108335273A (en) * | 2018-02-06 | 2018-07-27 | 大唐终端技术有限公司 | The real-time removing method of the distortion of big wide-angle flake full shot camera |
| CN108335273B (en) * | 2018-02-06 | 2022-02-08 | 大唐终端技术有限公司 | Real-time distortion elimination method for large wide-angle fisheye panoramic lens camera |
| WO2019192359A1 (en) * | 2018-04-02 | 2019-10-10 | 杭州海康威视数字技术股份有限公司 | Vehicle panoramic video display system and method, and vehicle controller |
| CN112026496A (en) * | 2019-06-04 | 2020-12-04 | 上海擎感智能科技有限公司 | Vehicle window display method, front end, rear end, medium and equipment |
| CN112026496B (en) * | 2019-06-04 | 2022-02-22 | 上海擎感智能科技有限公司 | Vehicle window display method, front end, rear end, medium and equipment |
| CN112565590A (en) * | 2020-11-16 | 2021-03-26 | 李诚专 | Object 360-degree all-round-looking image generation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102111610B (en) | 2012-08-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102111610B (en) | Panoramic image generating method for assisting driving | |
| TWI600558B (en) | Dynamic lane detection system and method | |
| JP4707109B2 (en) | Multi-camera image processing method and apparatus | |
| JP4412380B2 (en) | Driving support device, driving support method, and computer program | |
| CN103987582B (en) | Drive assistance device | |
| JP5922866B2 (en) | System and method for providing guidance information to a vehicle driver | |
| US20150042799A1 (en) | Object highlighting and sensing in vehicle image display systems | |
| CN109552315B (en) | Full-view camera host control system | |
| KR101438892B1 (en) | Apparatus and method for displaying dead zone of vehicle | |
| CN103434448B (en) | A kind of elimination car post blind area system and using method thereof | |
| KR20190047027A (en) | How to provide a rearview mirror view of the vehicle's surroundings in the vehicle | |
| CN104185010A (en) | Enhanced perspective view generation in a front curb viewing system | |
| CN102774324A (en) | Panoramic parking assist system and implementation method thereof | |
| CN105966312A (en) | Multi-functional advanced driving assistance system for vehicle and method | |
| KR102045088B1 (en) | Image displaying Method and Apparatus therefor | |
| CN103863192A (en) | Method and system for vehicle-mounted panoramic imaging assistance | |
| JP2010215027A (en) | Driving assistant device for vehicle | |
| CN105083123B (en) | Electron assistant back-sight visual system based on Image Communication | |
| CN103802725A (en) | New method for generating vehicle-mounted driving assisting image | |
| TW201623050A (en) | Vehicle-reversing display system capable of automatically switching multiple field-of-view modes by image estimation and vehicle-reversing image capture device thereof | |
| CN105711497A (en) | Backup camera system for automatically switching tri-state view angle and backup camera device of backup camera system | |
| CN105270249A (en) | Vehicle sight glass system using deformable display to display image of camera | |
| JP2019110448A (en) | Display control device and display system | |
| WO2023284748A1 (en) | Auxiliary driving system and vehicle | |
| CN205916033U (en) | Visual supplementary driving equipment of automobile panorama |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120829 Termination date: 20140521 |