CN104539914B - A kind of autosynchronous multiband monitor of band and its working method - Google Patents
A kind of autosynchronous multiband monitor of band and its working method Download PDFInfo
- Publication number
- CN104539914B CN104539914B CN201510034544.4A CN201510034544A CN104539914B CN 104539914 B CN104539914 B CN 104539914B CN 201510034544 A CN201510034544 A CN 201510034544A CN 104539914 B CN104539914 B CN 104539914B
- Authority
- CN
- China
- Prior art keywords
- lens
- visible light
- infrared
- camera
- multiband
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Camera Bodies And Camera Details Or Accessories (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a kind of autosynchronous multiband monitors of band, including visible light lens, infrared lens and MCU controllers, it is seen that is fixed with direct current generator and precision potentiator on light microscopic head and infrared lens；MCU controllers pass sequentially through direct current generator, precision potentiator control visible light lens and infrared lens rotation；Further include infrared detector and visible light camera, visible light camera is fixed on the rear end of visible light lens, infrared detector is fixed on the rear end of infrared lens, and MCU controllers are connect with infrared detector and visible light camera respectively, configures infrared detector and visible light camera running parameter；The invention also discloses a kind of autosynchronous multiband monitor working methods of band.When the present invention adjusts a lens focus, another camera lens can reach to be followed automatically, effectively solves the problems, such as individually to adjust the problems such as each wave band leads to target loss and visual field mismatch, convenience and reliability after finding monitoring objective.
Description
Technical field
The present invention relates to field of video monitoring more particularly to a kind of autosynchronous multiband monitor of band and its work sides
Method.
Background technology
With the development of security protection industry, video monitoring equipment has been made remote and two waveband very by more and more producers
To three wave bands, it is applied to the monitoring of highway, oil field, forest, safe city etc..But whole two wavebands monitoring currently on the market
Equipment or be the independent focal length for adjusting two camera lenses, makes the two reach essentially identical field angle, by the observation of human eye
Visible light uses zoom lens or all-in-one machine, thermal imagery that tight shot, field angle is directly used both to mismatch, and thermal imagery can not yet
It is accurately positioned target, What is more directly selects two matched tight shots of field angle.Such design makes user
It is very inconvenient for, such as one automobile run at high speed of monitoring, it either first adjusts visible light lens focal length and adjusts again
Thermal imaging system lens focus, or first adjust thermal imaging system lens focus and adjust visible light lens focal length again, it is arrived if both adjusted
Essentially identical field angle, it is possible to which automobile has travelled out the visible range of some wave band, will lose to target
Tracking needs to readjust, and is possible to lose the tracking to target under serious conditions, this is exhausted for monitoring field
To what is do not allowed.Therefore, for multiband monitoring device, by taking two waveband as an example, target is lost during monitoring in order to prevent,
Meet adjust lens focus another camera lens can achieve the purpose that follow automatically, the convenience that equipment uses and monitoring
Reliability, it is necessary to designing a can realize autosynchronous multiband monitoring device.
Invention content
In order to overcome the deficiencies in the prior art, the present invention provide a kind of autosynchronous multiband monitor of band and
Its working method, adjusts lens focus another camera lens and can reach and follow automatically, effectively solves discovery monitoring objective
The problems such as each wave band causes target to lose problem and visual field mismatch, convenience and reliability is individually adjusted afterwards.
To achieve the above object, the present invention adopts the following technical scheme that：
A kind of autosynchronous multiband monitor of band, including multiple optical lens and MCU controllers, each optical lens
On be respectively and fixedly provided with direct current generator and precision potentiator；The MCU controllers are communicated with console, pass sequentially through direct current
Machine, precision potentiator control visible light lens and infrared lens rotation.
Preferably, multiple optical lens include visible light lens, infrared lens or laser lens, and each optical lens can be right
Ying Yuyi wave band.The spectral region of visible light is 380-780nm；Laser belongs to near-infrared, spectral region 780-1000nm；
Infrared i.e. thermal imaging, infrared in belonging to, spectral region 7.5-14um（7500-14000nm）, the multiwave meaning of the present invention is just
It is divided according to spectral region, with applied to different use environments.
Further, this system further includes infrared detector and visible light camera, and the visible light camera is fixed
In the rear end of visible light lens, infrared detector is fixed on the rear end of infrared lens, the MCU controllers respectively with infrared acquisition
Device is connected with visible light camera, configures infrared detector and visible light camera running parameter.
Further, this system further includes shield and duckbilled, the visible light lens, infrared lens, visible image capturing
Machine, infrared detector and MCU controllers are respectively positioned in shield, and the shield is fixed on duckbilled, duckbilled be screwed in
On monitor stand.
Further, MCU controllers are communicated by RS232 interface with visible light camera, and MCU controllers pass through
RS485 interfaces are communicated with console.
Further, MCU controllers are communicated by wired or wireless way and console.
Further, it is seen that light video camera and the video information of infrared detector acquisition are delivered to exterior with double
In the monitor of road input function.
Further, direct current generator and precision potentiator are replaced by stepper motor, and the MCU controllers pass through stepping
Motor controls visible light lens and infrared lens rotation.
A kind of autosynchronous multiband monitor working method of band, includes the following steps：
（1）System power-on reset and initialization；
（2）Visible light lens, infrared lens, infrared detector and visible light camera self-test；
（3）Console data are come from etc. to be received；If receiving data, the data that will be received into interrupt service routine
It is read from data buffer zone, jumps into step（4）；
（4）Judge the correctness of reception data；If receiving correct data, judgement is change camera parameters order
Or lens control order jumps into step if what is received is the order for changing camera parameters（5）If received
It is lens control order, jumps into step（6）；Otherwise return to step（3）；
（5）The order of control camera parameters change is sent by RS232 interface；It returns to step later（3）；
（6）Judge lens control order control is visible light lens or infrared lens, if infrared lens, then can
Light-exposed camera lens is servo-actuated, otherwise infrared lens are servo-actuated；And judge whether two during visible light lens and infrared lens act
Person reaches identical field angle, if it is return to step（3）, otherwise, continue whether both detections reach identical field angle.
Further, the step（6）Middle judgement visible light lens and infrared lens reach sentencing for identical field angle
The method of determining is：
The focal length data of visible light lens potentiometer and infrared lens potentiometer is acquired and stored in MCU controllers in advance,
The selection of the focal length data of above-mentioned visible light lens potentiometer gathered in advance and infrared lens potentiometer be respectively uniformly distributed in
In camera lens focal length to the entire focal length section of short focus and the focal length data of above-mentioned visible light lens potentiometer gathered in advance and red
The focal length data of outer camera lens potentiometer corresponds, and corresponding method is the field angle size based on camera lens, and two kinds of camera lenses regard
Rink corner is visually substantially essentially identical；During adjusting two camera lenses, MCU controllers receive the order for controlling corresponding camera lens
Afterwards, the corresponding camera lens direct current generator rotation is controlled, while another camera lens direct current generator being driven to rotate, while acquiring another camera lens
The position data of potentiometer feedback, and the numerical value that collected another camera lens potentiometer is fed back and above-mentioned pre-stored institute
The focal length data for stating corresponding camera lens compares, if the corresponding camera lens of the numerical value and above-mentioned storage of collected potentiometer feedback
Focal length data some numerical value it is identical or essentially identical, then it is assumed that visible light lens and infrared lens have identical visual field
Angle.
Advantageous effect：The present invention in order to prevent during monitoring lose by target, is realized by MCU controllers and adjusts one
Lens focus, another camera lens can reach to be followed automatically, effectively solves individually to adjust each wave after finding monitoring objective
The problems such as section causes target to lose problem and visual field mismatch, convenience and reliability.
Description of the drawings
Fig. 1 is multiband monitor principle assumption diagram provided by the invention.
Fig. 2 is multiband monitor front view provided by the invention.
Fig. 3 is multiband monitor left view provided by the invention.
Fig. 4 is multiband monitor internal view provided by the invention.
Fig. 5 is multiband monitor work flow diagram provided by the invention.
Fig. 6 is that multiband monitor provided by the invention interrupts flow chart.
Wherein, 1, visible light lens, 2, visible light camera, 3, precision potentiator, 4, direct current generator, 5, infrared lens,
6, infrared detector, 7, MCU controllers, 8, shield, 9, duckbilled.
Specific implementation mode
The present invention is further described below in conjunction with the accompanying drawings.
The present invention provides a kind of autosynchronous multiband monitor of band, including multiple optical lens and MCU controllers, often
It is respectively and fixedly provided with direct current generator and precision potentiator on a optical lens；The MCU controllers are communicated with console, are led to successively
Cross direct current generator, precision potentiator control visible light lens and infrared lens rotation.Multiple optical lens include visible light lens,
Infrared lens or laser lens, each optical lens can correspond to a wave band.
It describes in detail below to the present invention by taking two waveband monitor as an example, as shown in Figure 1, two waveband monitor includes
Following components：Visible light camera 2, it is seen that light microscopic head 1, infrared detector 6, infrared lens 5, circuit control module, directly
Galvanic electricity machine 4, precision potentiator 3, MCU controllers 7, shield 8 and duckbilled 9.Composition portion of the external control devices not as this system
Point, it can be that upper computer software turns RS485 controls by USB, can also be monitoring control keyboard, one is intended only as in figure
It is monitor that communication interface, which facilitates explanation, external display device,.
As shown in Figure 2,3, 4, it is seen that light video camera 2 is fixed on the rear end of visible light lens 1, it is seen that fixed on light microscopic head 1
There are direct current generator 4 and precision potentiator 3, infrared detector 6 to be fixed on the rear end of infrared lens 5, is fixed on infrared lens 5 straight
Galvanic electricity machine 4 and precision potentiator 3, are both fixed in shield 8, there is one piece of circuit board, circuit board and video camera inside shield 8
The interface of control is RS232 interface, and MCU controllers 7 control camera parameters by program and the driving of direct current generator 4 is set and controlled
Camera lens rotates and correspondence with foreign country, external communication interface RS485, matches USB by the USB interface of monitoring keyboard or computer
Turn the control of RS485 modules；Shield 8 is fixed on duckbilled 9, and duckbilled 9 is screwed on holder.2 He of visible light camera
The video that infrared detector 6 exports is connected on the monitor with dual input function.
MCU controllers 7：The core of whole system, for sending or receiving control command setting video camera and thermal imaging
Parameter and read lens location feedback etc. functions.
Visible light camera 2：SD or high-definition camera；
Infrared detector 6：Using the 5th generation hyperfine miniaturization Detection Techniques；
Visible light lens 1：Realize the zoom function of visible light lens 1；
Infrared lens 5：Realize zoom, the focusing function of infrared lens 5；
Direct current generator 4：Control can be by the movement of light microscopic piece and infrared eyeglass；
Precision potentiator 3：The high-precision potentiometer for selecting ± 1% precision, is separately fixed at infrared lens 5 and visible light microscopic
On first 1, it is driven by the motor gear on camera lens.
Implement description：Acquire each the 20 of 5 potentiometer of 1 potentiometer of visible light lens and infrared lens respectively by program first
20 data are converted into digital quantity by a data by analog-digital conversion by MCU controllers 7；The selection of 20 data is wanted
It is uniformly distributed in camera lens focal length to the entire focal length section of short focus, and this two groups 20 data will correspond, it is corresponding
Method is the field angle size based on camera lens, and the field angle of visual two kinds of camera lenses is substantially essentially identical, such as is all every group
The 2nd data this 2 groups of data are stored in MCU controllers 7 then the field angle of their corresponding camera lenses is answered about the same.
During gathered data, in order to ensure the accuracy of data, it should adopt and the data of acquisition are screened, such as in order to standard
True gathered data, it should right to the data progress digital filtering of acquisition, such as middle position value filtering, Glitch Filter, multi collect
It is averaged or cry afterwards the mode that arithmetic mean filters to realize.Position if necessary to camera lens feedback is more accurate, may be used
16 even 24 high-resolution specialized simulation-number conversion chips coordinate special hardware filtering circuit to realize.
By taking MCU controllers 7 receive the order of control infrared lens 5 as an example, the order of control infrared lens 5 is received
Afterwards, MCU controllers 7 control the rotation of 5 direct current generator 4 of infrared lens, while MCU controllers 7 drive 1 direct current generator of visible light lens
4 rotations, MCU controllers 7 adopt the position data for reading visible light camera switching potentiometer feedback simultaneously, and MCU controllers 7 will be collected
The numerical value of potentiometer feedback is compared with 20 numerical value of the infrared lens 5 of storage, if the numerical value and visible light of infrared lens 5
Some numerical value of camera lens 1 is identical or essentially identical（Analog-to-digital has small mistake because of reasons such as conversion accuracy, resolution ratio
Difference）, then it is assumed that visible light lens 1 have identical field angle, MCU controllers 7 to control visible light lens direct current with infrared lens 5
Machine 4 stops operating.Same MCU controllers 7 control visible light lens 1, and it is same principle that the realization of infrared lens 5, which synchronizes, no longer
Repeated explanation.
This monitor specific work process includes the following steps：
（1）System power-on reset and initialization；
（2）Visible light lens 1,2 self-test of infrared lens 5, infrared detector 6 and visible light camera；
（3）Console data are come from etc. to be received；If receiving data, the data that will be received into interrupt service routine
It is read from data buffer zone, jumps into step（4）；
（4）Judge the correctness of reception data；If receiving correct data, judgement is change camera parameters order
Or lens control order jumps into step if what is received is the order for changing camera parameters（5）If received
It is lens control order, jumps into step（6）；Otherwise return to step（3）；
（5）The order of control camera parameters change is sent by RS232 interface；
（6）Judge lens control order control is visible light lens or infrared lens 5, if infrared lens 5, then
Visible light lens 1 are servo-actuated, otherwise infrared lens 5 are servo-actuated；And judge whether two during light microscopic head and infrared lens 5 act
Person reaches identical field angle, if it is return to step（3）, otherwise, continue whether both detections reach identical field angle.
Technical characteristic of the present invention without description can by or using the prior art realize, such as high-precision potentiometer and
Direct current generator 4 is merged into one, and three can be made into or with upper ripple by using stepper motor instead the same function, two waveband may be implemented
Section etc., details are not described herein, and above description is not limitation of the present invention, and the present invention is also not limited to the example above, this
The variations, modifications, additions or substitutions that the those of ordinary skill of technical field is made in the essential scope of the present invention, should also belong to
In protection scope of the present invention.
Claims (6)
1. a kind of autosynchronous multiband monitor working method of band, which is characterized in that the autosynchronous more waves of the band
Section monitor, including multiple optical lens and MCU controllers are respectively and fixedly provided with direct current generator and accurate current potential on each optical lens
Device；The MCU controllers are communicated with console, pass sequentially through direct current generator, precision potentiator control visible light lens and
Infrared lens rotate；The multiple optical lens includes visible light lens, infrared lens or laser lens；The band is automatically same
The multiband monitor of step further includes infrared detector and visible light camera, and the visible light camera is fixed on visible light microscopic
The rear end of head, infrared detector are fixed on the rear end of infrared lens, the MCU controllers respectively with infrared detector and visible light
Video camera connects, and configures infrared detector and visible light camera running parameter；
The autosynchronous multiband monitor working method of the band, includes the following steps：
（1）System power-on reset and initialization；
（2）Visible light lens, infrared lens, infrared detector and visible light camera self-test；
（3）Console data are come from etc. to be received；If receiving data, into interrupt service routine by the data received from number
According to being read in buffering area, step is jumped into（4）；
（4）Judge the correctness of reception data；If receiving correct data, judgement be change camera parameters order or
Lens control order jumps into step if what is received is the order for changing camera parameters（5）If what is received is mirror
Head control command, jumps into step（6）；Otherwise return to step（3）；
（5）The order of control camera parameters change is sent by RS232 interface；It returns to step later（3）；
（6）Judge lens control order control is visible light lens or infrared lens, if infrared lens, it can be seen that light
Camera lens is servo-actuated, otherwise infrared lens are servo-actuated；And judge whether that the two reaches during visible light lens and infrared lens act
To identical field angle, if it is return to step（3）, otherwise, continue whether both detections reach identical field angle；
The step（6）The middle determination method for judging that visible light lens and infrared lens reach identical field angle is：
The focal length data of visible light lens potentiometer and infrared lens potentiometer is acquired and stored in MCU controllers in advance, it is above-mentioned
The selection of the focal length data of visible light lens potentiometer and infrared lens potentiometer gathered in advance is respectively uniformly distributed in camera lens
In focal length to the entire focal length section of short focus and the focal length data of above-mentioned visible light lens potentiometer gathered in advance and infrared mirror
The focal length data of head potentiometer corresponds, and corresponding method is the field angle size based on camera lens, the field angle of two kinds of camera lenses
It is visually substantially essentially identical；During adjusting two camera lenses, after MCU controllers receive the order for controlling corresponding camera lens,
The corresponding camera lens direct current generator rotation is controlled, while another camera lens direct current generator being driven to rotate, while acquiring another camera lens electricity
The position data of position device feedback, and by the numerical value of collected another camera lens potentiometer feedback with it is above-mentioned pre-stored described
The focal length data of corresponding camera lens compares, if the numerical value and the corresponding camera lens of above-mentioned storage of collected potentiometer feedback
Some numerical value of focal length data is identical or essentially identical, then it is assumed that visible light lens have identical visual field with infrared lens
Angle.
2. the autosynchronous multiband monitor working method of band according to claim 1, it is characterised in that：The band is certainly
It further includes shield and duckbilled to move synchronous multiband monitor, the visible light lens, infrared lens, visible light camera, red
External detector and MCU controllers are respectively positioned in shield, and the shield is fixed on duckbilled, and duckbilled is screwed in monitor
On holder.
3. the autosynchronous multiband monitor working method of band according to claim 1, it is characterised in that：The MCU
Controller is communicated by RS232 interface with visible light camera, and MCU controllers are carried out by RS485 interfaces and console
Communication.
4. the autosynchronous multiband monitor working method of band according to claim 3, it is characterised in that：The MCU
Controller is communicated by wired or wireless way and console.
5. the autosynchronous multiband monitor working method of band according to claim 1, it is characterised in that：It is described visible
Light video camera and the video information of infrared detector acquisition are delivered in monitor of the exterior with dual input function.
6. the autosynchronous multiband monitor working method of band according to claim 1, it is characterised in that：The direct current
Motor and precision potentiator are replaced by stepper motor, and the MCU controllers pass through step motor control visible light lens and red
Outer rotation of lens.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510034544.4A CN104539914B (en) | 2015-01-23 | 2015-01-23 | A kind of autosynchronous multiband monitor of band and its working method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510034544.4A CN104539914B (en) | 2015-01-23 | 2015-01-23 | A kind of autosynchronous multiband monitor of band and its working method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104539914A CN104539914A (en) | 2015-04-22 |
| CN104539914B true CN104539914B (en) | 2018-11-02 |
Family
ID=52855373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510034544.4A Active CN104539914B (en) | 2015-01-23 | 2015-01-23 | A kind of autosynchronous multiband monitor of band and its working method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104539914B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105737989A (en) * | 2016-02-01 | 2016-07-06 | 北京理工大学 | Visible light auxiliary multiband infrared real-time imaging analysis system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101175202A (en) * | 2007-10-31 | 2008-05-07 | 天津市亚安科技电子有限公司 | Video monitoring apparatus based on double optical band |
| CN101762327A (en) * | 2010-01-14 | 2010-06-30 | 广州科易光电技术有限公司 | Infrared temperature monitoring method and system of electrified railway contact network |
| CN102305664A (en) * | 2011-05-19 | 2012-01-04 | 中国农业大学 | Thermal imaging temperature measurement and fault location inspection system |
| CN102693603A (en) * | 2012-06-26 | 2012-09-26 | 山东神戎电子股份有限公司 | Dual spectrum based intelligent monitoring system for forest fire prevention |
| CN203104667U (en) * | 2013-03-05 | 2013-07-31 | 山东神戎电子股份有限公司 | Multi-spectral monitoring system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20130117411A (en) * | 2012-04-17 | 2013-10-28 | 한국전자통신연구원 | Apparatus and method for recognizing a user |
-
2015
- 2015-01-23 CN CN201510034544.4A patent/CN104539914B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101175202A (en) * | 2007-10-31 | 2008-05-07 | 天津市亚安科技电子有限公司 | Video monitoring apparatus based on double optical band |
| CN101762327A (en) * | 2010-01-14 | 2010-06-30 | 广州科易光电技术有限公司 | Infrared temperature monitoring method and system of electrified railway contact network |
| CN102305664A (en) * | 2011-05-19 | 2012-01-04 | 中国农业大学 | Thermal imaging temperature measurement and fault location inspection system |
| CN102693603A (en) * | 2012-06-26 | 2012-09-26 | 山东神戎电子股份有限公司 | Dual spectrum based intelligent monitoring system for forest fire prevention |
| CN203104667U (en) * | 2013-03-05 | 2013-07-31 | 山东神戎电子股份有限公司 | Multi-spectral monitoring system |
Non-Patent Citations (1)
| Title |
|---|
| 双谱段成像探测系统的多线程设计与实现;闫丰,于晓,盛任农,隋永新,杨怀江;《吉林大学学报（理学版）》;20060531;第44卷(第3期);正文第427页第2.3节，图1 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104539914A (en) | 2015-04-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103278139B (en) | A kind of varifocal single binocular vision sensing device | |
| CN104535186B (en) | A kind of moving platform infared spectrum association detection system and method | |
| EP2737278B1 (en) | Measuring device that can be operated without contact and control method for such a measuring device | |
| CN106952252B (en) | Skin imaging system based on three spectrums | |
| CN104296694B (en) | The detection means and method of a kind of laser tracker optical axis and mechanical rotating shaft angle | |
| EP2860550A1 (en) | Scanner for spatial measurement | |
| CN205071145U (en) | Many image sensor camera of abundant sensitization | |
| CN102982518A (en) | Fusion method of infrared image and visible light dynamic image and fusion device of infrared image and visible light dynamic image | |
| CN103063410B (en) | Automatic detection system and detection method for ultraviolet or visible light optical system parameters | |
| CN104280866A (en) | Confocal microscope for imaging of selected locations of the body of a patient | |
| CN102261986B (en) | Zoom-stereo microscope detection device and method | |
| CN109632102A (en) | Infrared polarization image measuring device based on rotatory polarization piece | |
| CN103126654A (en) | Detecting system for near-infared body surface blood vessel detector | |
| CN100543413C (en) | A kind of digital airborne photography system | |
| CN101340562A (en) | Monitoring system and method | |
| CN105516688A (en) | Resolution-transforming type eagle eye-mimic visual imaging device and imaging method thereof | |
| CN104539914B (en) | A kind of autosynchronous multiband monitor of band and its working method | |
| CN103076091B (en) | For the camera installation that Illuminant chromaticity is measured | |
| CN104614803A (en) | ARM-based integrated polarization maintaining fiber axis positioning instrument | |
| CN208598368U (en) | Retina digital imaging system and retina digital imaging instrument | |
| CN212410025U (en) | Laser display speckle measuring device for simulating visual perception | |
| CN110673313B (en) | Zoom fisheye lens system and design method | |
| CN108416281A (en) | A kind of video camera applied to iris recognition | |
| CN104297884B (en) | A kind of portable video otoscope CMOS camera focal length fine-adjustment mechanism | |
| CN201207703Y (en) | Monitoring system and view line tracking device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |