CN104822020A - Automatic follow focus device based on video target tracking - Google Patents
Automatic follow focus device based on video target tracking Download PDFInfo
- Publication number
- CN104822020A CN104822020A CN201510147454.6A CN201510147454A CN104822020A CN 104822020 A CN104822020 A CN 104822020A CN 201510147454 A CN201510147454 A CN 201510147454A CN 104822020 A CN104822020 A CN 104822020A
- Authority
- CN
- China
- Prior art keywords
- video
- camera lens
- camera
- measuring sensor
- lens motor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Accessories Of Cameras (AREA)
- Studio Devices (AREA)
Abstract
The present invention discloses an automatic follow focus device based on video target tracking, which comprises: a cradle head, a cradle head motor, a cradle head motor driver, a distance measuring sensor used for measuring the distance from a movable target in real time and installed on the cradle head, a video collecting card used for receiving a video signal of a camera and outputting the video signal to a video terminal, the video terminal provided with video target tracking software in advance and used for receiving the video signal of the video collecting card and outputting coordinate values of a followed target relative to an image, a first lens motor and a second lens motor used for driving a focusing ring and a zooming ring of a lens, a lens motor driver used for driving the first lens motor and the second lens motor, and a main controller used for receiving a target distance value outputted by the distance measuring sensor and target coordinate values outputted by the video terminal and controlling the cradle head motor driver and the lens motor driver. The automatic follow focus device of the present invention has the advantages of automatic computation of field angles of the camera, no need of calibration and simple operation.
Description
Technical field
The present invention relates to radio and television film projection apparatus field, particularly a kind of based on video frequency object tracking automatically with coke installation.
Background technology
In existing technology, having utilizes distance measuring sensor and video frequency object tracking to carry out the equipment of auto-focusing, but need the manual calibration camera angle of visual field, lens focus cannot be utilized to calculate the angle of visual field of video camera, therefore cannot be applied on zoom lens, cause rangefinder to follow target accurate not.In general, the installation site of The Cloud Terrace and the optical axis of video camera in the horizontal direction or vertical direction have certain position deviation, existing based on video frequency object tracking automatically with burnt cannot according to target and the vertical range auto modification level of focal plane or the position deviation of vertical direction.
Therefore, need badly a kind of have automatically calculate camera coverage angle, without the need to calibration, simple to operate based on video frequency object tracking automatically with coke installation.
Summary of the invention
The object of this invention is to provide a kind of have automatically calculate camera coverage angle, without the need to calibration, simple to operate based on video frequency object tracking automatically with coke installation.
To achieve these goals, technical scheme provided by the invention is: provide a kind of and follow coke installation based on the automatic of video frequency object tracking, comprising:
The The Cloud Terrace that can rotate around horizontal direction and vertical direction;
For the horizontal stage electric machine driving described The Cloud Terrace to rotate around horizontal direction and vertical direction, described horizontal stage electric machine is generally stepping motor or servomotor;
For measuring the distance measuring sensor of moving target distance in real time, described distance measuring sensor is installed on described The Cloud Terrace; Described distance measuring sensor refers to laser range sensor or ultrasonic distance-measuring sensor, especially measuring frequency 50Hz and more than, measure error 5cm and within high speed ranging instrument;
For driving the horizontal stage electric machine driver of horizontal stage electric machine;
For receiving the vision signal of video camera, and export to the video frequency collection card of video terminal, especially SDI or HDMI video signal turn the video frequency collection card of USB vision signal;
Be preinstalled with video frequency object tracking software and for the vision signal of receiver, video capture card, and export the target of the following the tracks of video terminal relative to the coordinate figure of image;
Camera lens motor, comprises two camera lens motors, is respectively used to the first camera lens motor and the second camera lens motor that drive lens focusing circle and zoom circle;
For driving the camera lens motor driver of described first camera lens motor and the second camera lens motor;
For the coordinates of targets value that the distance value and described video terminal that receive the output of described distance measuring sensor export, control the main controller of described horizontal stage electric machine driver and described camera lens motor driver.
The horizontal field of view angle θ of described video camera can by the following derivation of equation: tan (θ/2)=(L/2)/f
θ＝2·arctan((L/2)/f)
Wherein θ represents the horizontal field of view angle of described video camera, and L represents described video camera sensitive chip horizontal width, and f represents described camera lens focal length, in like manner, can calculate video camera vertical field of view angle.
Described second camera lens motor is the servomotor of band closed-loop system, described camera lens motor driver can detect the rotational angle of described second camera lens motor gear, described second camera lens motor gear and camera lens gears meshing, also the angle that zoom circle rotates can just be detected, and according to being pre-stored within camera lens rotational angle and the lens zoom circle scale corresponding relation of described main controller, the focal length that camera lens is current can be calculated.
Described video frequency object tracking software exports coordinates of targets value, and in conjunction with described horizontal field of view angle and vertical field of view angle, described main controller can control described distance measuring sensor and accurately follow target travel.
Described The Cloud Terrace is installed on described video camera top or below, the optical axis of described video camera and described distance measuring sensor point to the optical axis meeting shape ψ 1 at a certain angle during video coordinates central point, and derivation formula is:
ψ1＝arctan(d2/d1)
Wherein d2 is the vertical range of initial point to described video camera optical axis of described distance measuring sensor, d1 be described distance measuring sensor initial point with perpendicular to described video camera optical axis and the vertical range of the plane at target place.
D2 is the eigenvalue with coke installation, and d1 can by the following derivation of equation:
H1＝L1·sinθ
1，H2≈H1，
L2＝L1/cosβ
1，
Wherein L1 represents the distance value that stadia surveying arrives
θ
1represent the angle of the targeted straight line of distance measuring sensor and horizontal plane
β
1represent the angle of the targeted straight line of distance measuring sensor and vertical plane
Described video frequency object tracking software can follow the tracks of multiple target simultaneously, and can switch the tracking target pointed by distance measuring sensor fast.
Compared with prior art, the present invention is based on the automatic with in coke installation of video frequency object tracking, owing to comprising described horizontal stage electric machine, horizontal stage electric machine driver, video frequency collection card, camera lens motor and main controller, described horizontal stage electric machine can drive described The Cloud Terrace to rotate around horizontal direction and vertical direction, and described horizontal stage electric machine is driven by described horizontal stage electric machine driver, described camera lens motor is driven by described camera lens motor driver, drive the camera lens motor driver of described first camera lens motor and the second camera lens motor, and described horizontal stage electric machine driver, described camera lens motor and driver control by described main controller, therefore described main controller can be unified to control, and can automatically calculate camera coverage angle, without the need to calibration, operate also fairly simple.
By following description also by reference to the accompanying drawings, the present invention will become more clear, and these accompanying drawings are for explaining embodiments of the invention.
Accompanying drawing explanation
Fig. 1 is the circuit theory module map of automatically following an embodiment of coke installation that the present invention is based on video frequency object tracking.
Fig. 2 is the schematic diagram of the horizontal field of view angle θ of video camera of the present invention.
Optical axis institute shape ψ 1 at a certain angle when the optical axis that Fig. 3 is video camera and described distance measuring sensor point to video coordinates central point derives schematic diagram.
Embodiment
With reference now to accompanying drawing, describe embodiments of the invention, element numbers similar in accompanying drawing represents similar element.As mentioned above, as Figure 1-3, the invention provides a kind of based on video frequency object tracking automatically with coke installation 100, comprising:
The The Cloud Terrace 2 that can rotate around horizontal direction and vertical direction;
For the horizontal stage electric machine 3 driving described The Cloud Terrace 2 to rotate around horizontal direction and vertical direction, described horizontal stage electric machine 3 is generally stepping motor or servomotor;
For measuring the distance measuring sensor 1 of moving target distance in real time, described distance measuring sensor 1 is installed on described The Cloud Terrace 2; Described distance measuring sensor 1 refers to laser range sensor or ultrasonic distance-measuring sensor, especially measuring frequency 50Hz and more than, measure error 5cm and within high speed ranging instrument;
For driving the horizontal stage electric machine driver 4 of horizontal stage electric machine 3;
For receiving the vision signal of video camera 5, and export to the video frequency collection card 7 of video terminal 6, especially SDI or HDMI video signal turn the video frequency collection card of USB vision signal;
Be preinstalled with video frequency object tracking software and for the vision signal of receiver, video capture card 7, and export the target of the following the tracks of video terminal 6 relative to the coordinate figure of image; This video terminal can be the equipment such as panel computer, smart mobile phone.
Camera lens motor 8, comprises two camera lens motors, is respectively used to the first camera lens motor 81 and the second camera lens motor 82 driving lens focusing circle and zoom circle;
For driving the camera lens motor driver 9 of described first camera lens motor 81 and the second camera lens motor 82;
For the coordinates of targets value that the distance value and described video terminal 6 that receive the output of described distance measuring sensor 1 export, control the main controller 10 of described horizontal stage electric machine driver 4 and described camera lens motor driver 9.
As shown in Figure 2, the horizontal field of view angle θ of described video camera 5 can by the following derivation of equation: tan (θ/2)=(L/2)/f
θ＝2·arctan((L/2)/f)
Wherein θ represents the horizontal field of view angle of described video camera 5, and L represents described video camera 5 sensitive chip horizontal width, and f represents described video camera 5 lens focus, in like manner, can calculate video camera 5 vertical field of view angle.
In an embodiment, described second camera lens motor 82 is the servomotor of band closed-loop system, described camera lens motor driver 9 can detect the rotational angle of described second camera lens motor 82 gear, described second camera lens motor 82 gear and camera lens gears meshing, also the angle that zoom circle rotates can just be detected, and according to being pre-stored within camera lens rotational angle and the lens zoom circle scale corresponding relation of described main controller 9, the focal length that camera lens is current can be calculated.
In an embodiment, described video frequency object tracking software exports coordinates of targets value, and in conjunction with described camera coverage angle, described main controller 10 can control described distance measuring sensor 1 and accurately follow target travel.
In an embodiment, described The Cloud Terrace 2 is installed on described video camera 5 top or below, the optical axis of described video camera 5 and described distance measuring sensor 1 point to the optical axis meeting shape ψ 1 at a certain angle during video coordinates central point, and derivation formula is:
ψ1＝arctan(d2/d1)
Wherein d2 is the vertical range of initial point to described video camera 5 optical axis of described distance measuring sensor 1, d1 be described distance measuring sensor 1 initial point with perpendicular to described video camera 5 optical axis and the vertical range of the plane at target place.
D2 is the eigenvalue with coke installation, and d1 can by the following derivation of equation:
H1＝L1·sinθ
1，H2≈H1，
L2＝L1/cosβ
1，
Wherein L1 represents the distance value that stadia surveying arrives
θ
1represent the angle of the targeted straight line of distance measuring sensor and horizontal plane
β
1represent the angle of the targeted straight line of distance measuring sensor and vertical plane
In an embodiment, described video frequency object tracking software can follow the tracks of multiple target simultaneously, and can switch the tracking target pointed by described distance measuring sensor 1 fast.
Compared with prior art, composition graphs 1-3, what the present invention is based on video frequency object tracking follows coke installation 100 automatically, owing to comprising described horizontal stage electric machine 3, horizontal stage electric machine driver 4, video frequency collection card 7, camera lens motor 8 and main controller 10, described horizontal stage electric machine 3 can drive described The Cloud Terrace 2 to rotate around horizontal direction and vertical direction, and described horizontal stage electric machine 3 is driven by described horizontal stage electric machine driver 4, described camera lens motor 8 is driven by described camera lens motor driver 9, drive the camera lens motor driver 9 of described first camera lens motor 81 and the second camera lens motor 82, and described horizontal stage electric machine driver 4 controls by described main controller 10 with described camera lens motor driver 9, therefore described main controller 10 can be unified to control, and can automatically calculate camera coverage angle, without the need to calibration, operate also fairly simple.
Above disclosedly be only the preferred embodiments of the present invention, certainly can not limit the interest field of the present invention with this, therefore according to the equivalent variations that the present patent application the scope of the claims is done, still belong to the scope that the present invention is contained.
Claims (6)
1. follow a coke installation based on the automatic of video frequency object tracking, it is characterized in that, comprising:
The The Cloud Terrace that can rotate around horizontal direction and vertical direction;
For the horizontal stage electric machine driving described The Cloud Terrace to rotate around horizontal direction and vertical direction; For driving the horizontal stage electric machine driver of horizontal stage electric machine;
For measuring the distance measuring sensor of moving target distance in real time, described distance measuring sensor is installed on described The Cloud Terrace;
For receiving the vision signal of video camera, and export to the video frequency collection card of video terminal;
Be preinstalled with video frequency object tracking software and for the vision signal of receiver, video capture card, and export the target of the following the tracks of video terminal relative to the coordinate figure of image;
Camera lens motor, comprises two camera lens motors, is respectively used to the first camera lens motor and the second camera lens motor that drive lens focusing circle and zoom circle;
For driving the camera lens motor driver of described first camera lens motor and the second camera lens motor;
For the coordinates of targets value that the target range value and described video terminal that receive the output of described distance measuring sensor export, control the main controller of described horizontal stage electric machine driver and described camera lens motor driver.
2. follow coke installation based on the automatic of video frequency object tracking as claimed in claim 1, it is characterized in that, the horizontal field of view angle θ of described video camera can by the following derivation of equation: tan (θ/2)=(L/2)/f
θ＝2·arctan((L/2)/f)
Wherein θ represents the horizontal field of view angle of described video camera, and L represents described video camera sensitive chip horizontal width, and f represents described camera lens focal length.
As claimed in claim 2, according to same method, the vertical field of view angle of described video camera can by the following derivation of equation:
tan(β/2)＝(H/2)/f
β＝2·arctan((H/2)/f)
Wherein β represents the vertical field of view angle of described video camera, and H represents described video camera sensitive chip vertical height, and f represents described camera lens focal length.
3. follow coke installation based on the automatic of video frequency object tracking as claimed in claim 2, it is characterized in that, described second camera lens motor is the servomotor of band closed-loop system, described camera lens motor driver can detect the rotational angle of described second camera lens motor gear, described second camera lens motor gear and camera lens gears meshing, also the angle that zoom circle rotates can just be detected, and according to being pre-stored within camera lens rotational angle and the lens zoom circle scale corresponding relation of described main controller, the focal length that camera lens is current can be calculated.
4. follow coke installation based on the automatic of video frequency object tracking as claimed in claim 3, it is characterized in that, described video frequency object tracking software exports coordinates of targets value, and in conjunction with described horizontal field of view angle θ and vertical field of view angle β, described main controller can control described distance measuring sensor and accurately follow target travel.
5. follow coke installation based on the automatic of video frequency object tracking as claimed in claim 4, it is characterized in that, described The Cloud Terrace is installed on described video camera top or below, the optical axis of described video camera and described distance measuring sensor point to the optical axis meeting shape ψ 1 at a certain angle during video coordinates central point, and derivation formula is:
ψ1＝arctan(d2/d1)
Wherein d2 is the vertical range of initial point to described video camera optical axis of described distance measuring sensor, d1 be described distance measuring sensor initial point with perpendicular to described video camera optical axis and the vertical range of the plane at target place.
D2 is the eigenvalue with coke installation, and d1 can by the following derivation of equation:
H1＝L1·sinθ
1，H2≈H1，
L2＝L1/cosβ
1
Wherein L1 represents the distance value that stadia surveying arrives
θ
1represent the angle of the targeted straight line of distance measuring sensor and horizontal plane
β
1represent the angle of the targeted straight line of distance measuring sensor and vertical plane.
6. follow coke installation based on the automatic of video frequency object tracking as claimed in claim 5, it is characterized in that, described video frequency object tracking software can follow the tracks of multiple target simultaneously.And the tracking target that can switch fast pointed by distance measuring sensor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510147454.6A CN104822020B (en) | 2015-03-31 | 2015-03-31 | It is automatic with coke installation based on video frequency object tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510147454.6A CN104822020B (en) | 2015-03-31 | 2015-03-31 | It is automatic with coke installation based on video frequency object tracking |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104822020A true CN104822020A (en) | 2015-08-05 |
| CN104822020B CN104822020B (en) | 2019-02-12 |
Family
ID=53732175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510147454.6A Active CN104822020B (en) | 2015-03-31 | 2015-03-31 | It is automatic with coke installation based on video frequency object tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104822020B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105681666A (en) * | 2016-02-29 | 2016-06-15 | 广东欧珀移动通信有限公司 | Control method, control device and electronic device |
| CN105763804A (en) * | 2016-02-29 | 2016-07-13 | 广东欧珀移动通信有限公司 | Control method and device and electronic device |
| CN105874384A (en) * | 2016-01-06 | 2016-08-17 | 深圳市莫孚康技术有限公司 | Focus following system and method based on multiple distance measuring modes, and shooting system |
| CN105917262A (en) * | 2016-01-06 | 2016-08-31 | 深圳市莫孚康技术有限公司 | Focus following system with simple multiple distance measurement manners and shooting system |
| CN106101558A (en) * | 2016-07-29 | 2016-11-09 | 广东欧珀移动通信有限公司 | camera focusing method, device and terminal |
| CN106559617A (en) * | 2015-11-25 | 2017-04-05 | 北京索斯克科技开发有限公司 | A kind of telephoto lens autofocus |
| CN111712667A (en) * | 2019-07-16 | 2020-09-25 | 深圳市大疆创新科技有限公司 | Control method and device of shooting equipment, handheld cloud deck and storage medium |
| CN112188089A (en) * | 2020-09-22 | 2021-01-05 | 北京小米移动软件有限公司 | Distance acquisition method and device, focal length adjustment method and device, and distance measurement assembly |
| CN113473020A (en) * | 2015-11-25 | 2021-10-01 | 深圳市大疆灵眸科技有限公司 | Focus following device control method, device and system |
| WO2022061534A1 (en) * | 2020-09-22 | 2022-03-31 | 深圳市大疆创新科技有限公司 | Photographing control method and apparatus, gimbal, follow focus electric motor, and storage medium |
| WO2022077748A1 (en) * | 2020-10-15 | 2022-04-21 | 深圳市大疆创新科技有限公司 | Photography assistance structure and photography apparatus |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102073193A (en) * | 2010-11-17 | 2011-05-25 | 泉州市东南光电有限公司 | Method for testing focal distance of camera lens by utilizing field angle |
| CN104010131B (en) * | 2014-05-05 | 2019-05-24 | 深圳市莫孚康技术有限公司 | A kind of video camera is automatically with coke installation and method |
| CN203984544U (en) * | 2014-05-30 | 2014-12-03 | 深圳市莫孚康技术有限公司 | A kind of video camera is automatically with burnt target hunting gear |
| CN204859351U (en) * | 2015-03-31 | 2015-12-09 | 深圳市莫孚康技术有限公司 | It is automatic with burnt device based on video target following |
-
2015
- 2015-03-31 CN CN201510147454.6A patent/CN104822020B/en active Active
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113473020A (en) * | 2015-11-25 | 2021-10-01 | 深圳市大疆灵眸科技有限公司 | Focus following device control method, device and system |
| CN106559617A (en) * | 2015-11-25 | 2017-04-05 | 北京索斯克科技开发有限公司 | A kind of telephoto lens autofocus |
| CN106559617B (en) * | 2015-11-25 | 2022-08-05 | 北京索斯克科技开发有限公司 | Automatic focusing device of telephoto lens |
| CN105874384A (en) * | 2016-01-06 | 2016-08-17 | 深圳市莫孚康技术有限公司 | Focus following system and method based on multiple distance measuring modes, and shooting system |
| CN105917262A (en) * | 2016-01-06 | 2016-08-31 | 深圳市莫孚康技术有限公司 | Focus following system with simple multiple distance measurement manners and shooting system |
| WO2017117749A1 (en) * | 2016-01-06 | 2017-07-13 | 深圳市莫孚康技术有限公司 | Follow focus system and method based on multiple ranging approaches, and photographing system |
| CN105874384B (en) * | 2016-01-06 | 2019-07-16 | 深圳市莫孚康技术有限公司 | Based on a variety of distance measuring methods with burnt system, method and camera system |
| CN105763804A (en) * | 2016-02-29 | 2016-07-13 | 广东欧珀移动通信有限公司 | Control method and device and electronic device |
| CN105681666A (en) * | 2016-02-29 | 2016-06-15 | 广东欧珀移动通信有限公司 | Control method, control device and electronic device |
| CN106101558A (en) * | 2016-07-29 | 2016-11-09 | 广东欧珀移动通信有限公司 | camera focusing method, device and terminal |
| CN106101558B (en) * | 2016-07-29 | 2017-09-29 | 广东欧珀移动通信有限公司 | camera focusing method, device and terminal |
| CN111712667A (en) * | 2019-07-16 | 2020-09-25 | 深圳市大疆创新科技有限公司 | Control method and device of shooting equipment, handheld cloud deck and storage medium |
| CN112188089A (en) * | 2020-09-22 | 2021-01-05 | 北京小米移动软件有限公司 | Distance acquisition method and device, focal length adjustment method and device, and distance measurement assembly |
| WO2022061534A1 (en) * | 2020-09-22 | 2022-03-31 | 深圳市大疆创新科技有限公司 | Photographing control method and apparatus, gimbal, follow focus electric motor, and storage medium |
| WO2022077748A1 (en) * | 2020-10-15 | 2022-04-21 | 深圳市大疆创新科技有限公司 | Photography assistance structure and photography apparatus |
| CN114503542A (en) * | 2020-10-15 | 2022-05-13 | 深圳市大疆创新科技有限公司 | Shooting auxiliary structure and shooting equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104822020B (en) | 2019-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104822020A (en) | Automatic follow focus device based on video target tracking | |
| CN104822019B (en) | The method for calculating camera coverage angle | |
| CN204859351U (en) | It is automatic with burnt device based on video target following | |
| EP3393120B1 (en) | Method for automatic focus and ptz camera | |
| US9959626B2 (en) | Three-dimensional depth perception method and apparatus with an adjustable working range | |
| EP3742200B1 (en) | Detection apparatus and parameter adjustment method thereof | |
| CN109120883B (en) | Far and near scene-based video monitoring method and device and computer-readable storage medium | |
| US9092677B2 (en) | Apparatus and method for recognizing location of vehicle | |
| KR101758735B1 (en) | Method for acquiring horizontal distance between camera and target, camera and surveillance system adopting the method | |
| US20140232859A1 (en) | Measuring system and method for marking a known target point in a coordinate system | |
| CN103024276A (en) | Positioning and focusing method of pan-tilt camera | |
| CN104469168A (en) | Shooting module and automatic focusing method thereof | |
| CN105225241A (en) | The acquisition methods of unmanned plane depth image and unmanned plane | |
| WO2019208101A1 (en) | Position estimating device | |
| CN105874384B (en) | Based on a variety of distance measuring methods with burnt system, method and camera system | |
| US8861948B2 (en) | Image pickup apparatus | |
| CN102445183A (en) | Apparatus of ranging laser point of remote ranging system and positioning method based on paralleling of laser and camera | |
| CN108007426B (en) | Camera ranging method | |
| US20140022636A1 (en) | Optical apparatus used in 3d image pickup system | |
| CN104010131A (en) | Automatic follow-focus device and method of cameras | |
| KR100948947B1 (en) | Localization apparatus of autonomous vehicle and method thereof | |
| CN104931011A (en) | Passive distance measuring method of infrared thermal imager | |
| US11233961B2 (en) | Image processing system for measuring depth and operating method of the same | |
| CN108061902A (en) | A kind of method and device of detecting object | |
| US20130120585A1 (en) | Automatic tracking camera system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |