CN103974039A - Intelligent infrared thermal imaging and high-definition visible light dual-waveband video monitoring system - Google Patents
Intelligent infrared thermal imaging and high-definition visible light dual-waveband video monitoring system Download PDFInfo
- Publication number
- CN103974039A CN103974039A CN201410179327.XA CN201410179327A CN103974039A CN 103974039 A CN103974039 A CN 103974039A CN 201410179327 A CN201410179327 A CN 201410179327A CN 103974039 A CN103974039 A CN 103974039A
- Authority
- CN
- China
- Prior art keywords
- video
- thermal imaging
- high definition
- infrared thermal
- visible light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Closed-Circuit Television Systems (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Studio Devices (AREA)
Abstract
The invention provides an intelligent infrared thermal imaging and high-definition visible light dual-waveband video monitoring system which is characterized by comprising a master control circuit unit, a high-definition visible light imaging unit, a thermal imaging unit, an HD-SDI high-definition video processing circuit, an HD-SDI high-definition video output interface, a standard-definition video switching processing circuit, a standard-definition video processing circuit, a CVBS standard-definition video output interface, a control interface circuit and a light intensity sensor processing circuit. The master control circuit unit is responsible for receiving instruction information from the control interface circuit or receiving an environment light intensity signal from the light intensity sensor processing circuit, controlling the high-definition visible light imaging unit to carry out switching on a high-definition visible light digital video and a standard-definition visible light video, and controlling the standard-definition video switching processing circuit to carry out switching on the standard-definition visible light video and an infrared thermal imaging standard-definition video. According to the intelligent infrared thermal imaging and high-definition visible light dual-waveband video monitoring system, all-weather video monitoring can be achieved, and the visible light high-definition video monitoring function and the infrared thermal imaging monitoring function are achieved.
Description
Technical field
The present invention relates to video monitoring system, the supervisory control system that especially a kind of infrared thermal imaging is combined with visual light imaging.
Background technology
Current video monitoring system mainly adopts visible light camera and the artificial mode combining that monitors, records a video to carry out daily security protection, but because visible light camera is under bad weather or the lower condition of illumination, very difficult filtering interfering obtains useful video image, therefore the prevention ability of whole safety-protection system under night or severe weather conditions is had a greatly reduced quality.In this case, infrared thermal imaging technique is far away with its operating distance, penetration capacity strong, can identify the advantages such as concealed target is introduced into safety-security area, becomes an one's share of expenses for a joint undertaking in monitoring field.
Then, the resolution of infrared thermal imaging camera is the highest at present can only arrive 300,000 pixel left and right, can not reach the requirement of current field of video monitoring to high-definition monitoring far away.On the other hand, high-definition video monitoring field is several years ago monopolized all over the world by network high definition substantially, yet due to reasons such as network bandwidth deficiency, encoding and decoding technique bottlenecks, has also highlighted gradually some places not fully up to expectations of high-definition network video monitoring product.
To sum up it seems, the maximum effect of performance that how effectively high-definition camera and infrared thermal imaging technique combined is the key point that realizes high-quality video monitoring.
Summary of the invention
The dual-band infrared night vision device system that the object of the present invention is to provide a kind of high definition visible light camera based on high-definition digital component serial line interface HD-SDI and infrared thermal imaging unit to become one, can carry out the remote control management, and can switch easily and flexibly high definition visible light camera video and infrared thermal imaging video, and switch high definition visible light camera video and SD analog video.The technical solution used in the present invention is:
A kind of Intelligent infrared thermal imaging and high definition visible ray two-waveband video supervisory control system, it is characterized in that, comprising: main control circuit unit, high definition visual light imaging unit, thermal imaging unit, HD-SDI HD video treatment circuit, HD-SDI HD video output interface, SD video switch treatment circuit, SD video processing circuits, CVBS SD video output interface, control interface and light intensity sensor treatment circuit;
Described main control circuit unit connects high definition visual light imaging unit, SD video switch treatment circuit, SD video processing circuits, control interface and light intensity sensor treatment circuit; Be responsible for accepting command information or from light intensity sensor treatment circuit reception environment light intensity signal from control interface, and control high definition visual light imaging unit and carry out the switching of high definition visible ray digital video and SD visible light video, and control SD video switch treatment circuit and carry out the switching of SD visible light video and infrared thermal imaging SD video;
Described high definition visual light imaging unit is for generating high definition visible light video signal or SD visible light video signal according to different control commands;
Described HD-SDI HD video treatment circuit is responsible for receiving the high definition video steaming of high definition visual light imaging unit output, and generates the high-definition digital video signal of HD-SDI interface standard;
Described thermal imaging unit is used for generating infrared thermal imaging SD vision signal;
Described SD video switch treatment circuit is responsible for receiving SD visible light video signal and infrared thermal imaging SD vision signal, and exports a wherein road vision signal according to different instruction;
Described SD video processing circuits is responsible for receiving the SD visible light video signal of high definition visual light imaging unit output or the infrared thermal imaging SD vision signal that thermal imaging unit produces, and generates the CVBS analog video signal of standard.
Further, control interface adopts RS485 interface circuit.
The important process process of Intelligent infrared thermal imaging and high definition visible ray two-waveband video supervisory control system is as described below:
1) strength signal of the illuminance that main control circuit unit receives according to the transducer from light intensity sensor treatment circuit, sends switching signal to SD video switch treatment circuit; Or according to the different switching commands that receive from RS485 interface circuit, to SD video switch treatment circuit, send switching signal or send switching signal to high definition visual light imaging unit.
2) high definition visual light imaging unit receives after corresponding switching controls instruction, can realize fast and switching between high definition visible ray digital video and SD visible ray analog video.
3) SD video switch treatment circuit receives after corresponding switching controls instruction, can realize fast and switching between SD infrared thermal imaging SD analog video and SD visible ray analog video.
Tool of the present invention has the following advantages:
1. adopt HD-SDI high definition visible light camera, can realize high-definition image without postponing real-time Transmission, high-definition image free of losses is undistorted;
2. can realize round-the-clock video monitoring, and visible ray high-definition video monitoring function and better monitoring effect are provided.
3 adopt video frequency switching module, the seamlessly switching between visible ray and infrared thermal imaging video, visible ray HD video and visible ray SD video, and can selectively to monitor video, transmit and store, reduce transmission bandwidth and save video storage space, reduce system cost.
Accompanying drawing explanation
Fig. 1 is that structure of the present invention forms schematic diagram.
Fig. 2 is the circuit diagram of main control circuit unit of the present invention.
Fig. 3 is the circuit diagram of HD-SDI HD video treatment circuit of the present invention.
Fig. 4 is the circuit diagram of SD video processing circuits of the present invention.
Fig. 5 is the circuit diagram of SD video switch treatment circuit of the present invention.
Fig. 6 is the circuit diagram of light intensity sensor treatment circuit of the present invention.
Fig. 7 is workflow diagram of the present invention.
Embodiment
Below in conjunction with concrete drawings and Examples, the invention will be further described.
As shown in Figure 1, Intelligent infrared thermal imaging proposed by the invention and high definition visible ray two-waveband video supervisory control system, comprising: main control circuit unit, high definition visual light imaging unit, thermal imaging unit, HD-SDI HD video treatment circuit, HD-SDI HD video output interface, SD video switch treatment circuit, SD video processing circuits, CVBS SD video output interface, control interface and light intensity sensor treatment circuit.
Wherein control interface can adopt the RS485 interface circuit shown in Fig. 1.Thermal imaging unit adopts thermal infrared imager.High definition visual light imaging unit can adopt highest resolution at 720p or high definition visible light camera more than 1080i, and its resolution can, according to external control signal adjustment, be carried out SD shooting such as being also adjusted into 480p resolution.
Main control circuit unit connects high definition visual light imaging unit, SD video switch treatment circuit, SD video processing circuits, RS485 interface circuit and light intensity sensor treatment circuit; Be responsible for accepting command information or from light intensity sensor treatment circuit reception environment light intensity signal from RS485 interface circuit, and control high definition visual light imaging unit and carry out the switching of high definition visible ray digital video and SD visible light video, and control SD video switch treatment circuit and carry out the switching of SD visible light video and infrared thermal imaging SD video.
As shown in Figure 2, main control circuit unit mainly consists of the STM32F103 microprocessor of ST Microelectronics.U104 is the master chip STM32F103 of main control circuit unit, is the microprocessor based on Cortex-M3 kernel, completes the realization of all control commands and the conversion of agreement.Wherein RXD/TXD interface connects serial ports conversion chip, UART1 interface connects high definition visual light imaging unit, SPI interface connects SD video processing circuits, and I2C interface connects light intensity sensor treatment circuit etc., and V_SEL signal connects SD video switch treatment circuit etc.U101 is low dropout voltage regulator, adopts AMS1177-3.3, and producer is ams-semitech.U102(CAT706S) provide reset control function.U103 is RS485 interface conversion chip RSM3485CHT, external RS485 interface.
HD-SDI HD video treatment circuit is responsible for receiving the high definition video steaming of high definition visual light imaging unit output, and generates the high-definition digital video signal of HD-SDI interface standard.As shown in Figure 3, U201 mainly realizes the HD-SDI high-definition digital video signal that serial video signal is converted to standard to its internal circuit.U201 adopts the deserializer LMH0341 of National Semiconductor's (purchase of Xian Yibei Texas Instrument).LVDS interface (RXOUTP/N[0:4], RXCLKOUT_P/N) be high definition serial video signal input interface, this interface connects high definition visual light imaging unit video output interface.SDA/SDC interface is the control interface of deserializer U201, and this interface connects the I2C interface of the U104 of main control circuit unit.TXDOUT_P/N is standard HD-SDI high-definition video signal output interface.
SD video processing circuits is responsible for receiving the SD visible light video signal of high definition visual light imaging unit output or the infrared thermal imaging SD vision signal that thermal imaging unit produces, and generates the CVBS analog video signal of standard.Its circuit as shown in Figure 4.U301 is vision signal Sync Separator Chip LM1881, U302 is the OSD special driving chip SD1000 that Zhou Ligong company produces, realize video character superposition function, U303 mainly realizes the control function of the black balance of U302, U304 is binary channels video amplification chip, YouJRC company produces, herein only need to be with a passage.
SD video switch treatment circuit is responsible for receiving SD visible light video signal and infrared thermal imaging SD vision signal, and exports a wherein road vision signal according to different instruction.Its circuit as shown in Figure 5.VIDEO_NOMAL signal is the SD visible light video signal that high definition visual light imaging unit produces; VIDEO_INFRARED is the SD Infrared Video Signal that thermal imaging unit produces; VIEDO_SELECT is video switch control signal, is connected with the main control chip U104 of main control circuit unit.U401 is binary channels single-pole double-throw switch (SPDT) chip ADG736, by ANALOG DEVICES company, is produced.Herein only need to be with a passage.
Light intensity sensor treatment circuit is mainly to detect light intensity signal, and is connected with main control circuit unit by I2C interface, and wherein U501 is light intensity sensor, realizes the detection of light intensity.Its circuit diagram as shown in Figure 6.U501 adopts ambient light intensity sensor IC BH1750FVI, by Japanese ROHM semiconductor group, is produced.SCL/SDA is the signal of I2C interface, connects the I2C interface of main control circuit unit U104.
The summary of the present invention's two kinds of video switch patterns while working below, pattern one: automatically switch; Pattern two: manual switchover.As shown in Figure 7, below for operation specifically describes:
1: when pattern is automatic mode, by light intensity sensor, determine whether daytime, when being judged as daytime, enter the 4th step, otherwise, enter the 3rd step;
2: when pattern is manual mode, by RS485 interface, receive control command, judge whether to start thermal imager, if definite, need to start, enter the 3rd step, otherwise, enter the 4th step;
3: open thermal infrared imager, close visible light camera, and enter the 5th step;
4: close thermal infrared imager, open visible light camera, and enter the 6th step;
5: switch to infrared photography pattern, output SD analog video signal;
6: judge whether to start HD video output mode, if so, enter the 7th step, otherwise, enter the 8th step;
7: switch to HD video output mode, output high-definition digital video signal;
8: switch to SD video output mode, output SD analog video signal.
Being explained as follows of related some terms in the present invention:
CVBS:Composite Video Broadcast Signal, composite video signal.
HD-SDI:HD-serial digital interface, high-definition digital component serial line interface.
HD video refers to 720p or vision signal more than 1080i definition.SD video be resolution lower than the vision signal of above-mentioned standard, resolution is in 400 about lines (being generally 480p).
OSD is the abbreviation of On Screen Display, and OSD core is to utilize character generation chip on the screen of display, to show the character needing.
Claims (7)
1. an Intelligent infrared thermal imaging and high definition visible ray two-waveband video supervisory control system, it is characterized in that, comprising: main control circuit unit, high definition visual light imaging unit, thermal imaging unit, HD-SDI HD video treatment circuit, HD-SDI HD video output interface, SD video switch treatment circuit, SD video processing circuits, CVBS SD video output interface, control interface and light intensity sensor treatment circuit;
Described main control circuit unit connects high definition visual light imaging unit, SD video switch treatment circuit, SD video processing circuits, control interface and light intensity sensor treatment circuit; Be responsible for accepting command information or from light intensity sensor treatment circuit reception environment light intensity signal from control interface, and control high definition visual light imaging unit and carry out the switching of high definition visible ray digital video and SD visible light video, and control SD video switch treatment circuit and carry out the switching of SD visible light video and infrared thermal imaging SD video;
Described high definition visual light imaging unit is for generating high definition visible light video signal or SD visible light video signal according to different control commands;
Described HD-SDI HD video treatment circuit is responsible for receiving the high definition video steaming of high definition visual light imaging unit output, and generates the high-definition digital video signal of HD-SDI interface standard;
Described thermal imaging unit is used for generating infrared thermal imaging SD vision signal;
Described SD video switch treatment circuit is responsible for receiving SD visible light video signal and infrared thermal imaging SD vision signal, and exports a wherein road vision signal according to different instruction;
Described SD video processing circuits is responsible for receiving the SD visible light video signal of high definition visual light imaging unit output or the infrared thermal imaging SD vision signal that thermal imaging unit produces, and generates the CVBS analog video signal of standard.
2. Intelligent infrared thermal imaging as claimed in claim 1 and high definition visible ray two-waveband video supervisory control system, is characterized in that: described control interface adopts RS485 interface circuit.
3. Intelligent infrared thermal imaging as claimed in claim 1 or 2 and high definition visible ray two-waveband video supervisory control system, is characterized in that: main control circuit unit mainly consists of STM32F103 microprocessor.
4. Intelligent infrared thermal imaging as claimed in claim 1 or 2 and high definition visible ray two-waveband video supervisory control system, is characterized in that: HD-SDI HD video treatment circuit mainly consists of deserializer LMH0341.
5. Intelligent infrared thermal imaging as claimed in claim 1 or 2 and high definition visible ray two-waveband video supervisory control system, is characterized in that: SD video processing circuits mainly consists of OSD special driving chip SD1000.
6. Intelligent infrared thermal imaging as claimed in claim 1 or 2 and high definition visible ray two-waveband video supervisory control system, is characterized in that: SD video switch treatment circuit mainly consists of single-pole double-throw switch (SPDT) chip ADG736.
7. Intelligent infrared thermal imaging as claimed in claim 1 or 2 and high definition visible ray two-waveband video supervisory control system, is characterized in that: light intensity sensor treatment circuit mainly consists of ambient light intensity sensor IC BH1750FVI.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410179327.XA CN103974039B (en) | 2014-04-29 | 2014-04-29 | Intelligent infrared thermal imaging and high definition visible ray two-waveband video monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410179327.XA CN103974039B (en) | 2014-04-29 | 2014-04-29 | Intelligent infrared thermal imaging and high definition visible ray two-waveband video monitoring system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103974039A true CN103974039A (en) | 2014-08-06 |
| CN103974039B CN103974039B (en) | 2017-10-13 |
Family
ID=51243018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410179327.XA Active CN103974039B (en) | 2014-04-29 | 2014-04-29 | Intelligent infrared thermal imaging and high definition visible ray two-waveband video monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103974039B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106488195A (en) * | 2016-12-13 | 2017-03-08 | 合肥英威晟光电科技有限公司 | A kind of monitoring system based on thermal imaging |
| CN106657925A (en) * | 2017-01-18 | 2017-05-10 | 沈阳上博智像科技有限公司 | Video output method and thermal infrared imager |
| CN109618082A (en) * | 2018-12-17 | 2019-04-12 | 珠海市国宏网络科技有限公司 | A kind of high-definition intelligent ball machine and data transmission method |
| CN109756706A (en) * | 2018-12-17 | 2019-05-14 | 珠海市国宏网络科技有限公司 | A kind of distributed video monitoring analysis system and data transmission method |
| CN113408506A (en) * | 2021-08-20 | 2021-09-17 | 航天宏图信息技术股份有限公司 | All-time cloud detection method and device for geostationary satellite |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201504278U (en) * | 2009-06-19 | 2010-06-09 | 深圳市汉锐科技有限公司 | Video circuit used for camera |
| CN101923212A (en) * | 2010-07-06 | 2010-12-22 | 山东神戎电子股份有限公司 | High-definition laser night-vision viewer |
| CN101998111A (en) * | 2009-08-28 | 2011-03-30 | 杨占坤 | Video acquisition mode automatic conversion system and method of high-definition (HD) camera |
| US20110199489A1 (en) * | 2010-02-18 | 2011-08-18 | Testo Ag | Method for the extraction of an ir-image and thermal imaging camera |
| CN102694977A (en) * | 2011-03-25 | 2012-09-26 | 杨占昆 | Method for automatically converting between high definition mode and standard definition mode of high-definition camera |
| CN102752502A (en) * | 2012-04-28 | 2012-10-24 | 北京汉王智通科技有限公司 | Intelligent camera and intelligent camera shooting method |
| CN102833511A (en) * | 2012-09-18 | 2012-12-19 | 浙江红苹果电子有限公司 | Method and device for storing high-definition videos based on serial digital interface (SDI) |
| CN103051887A (en) * | 2013-01-23 | 2013-04-17 | 河海大学常州校区 | Eagle eye-imitated intelligent visual sensing node and work method thereof |
| CN103220487A (en) * | 2012-01-20 | 2013-07-24 | 北京凯新创达科技发展有限公司 | Universal converter capable of converting high definition into standard definition |
| CN203231819U (en) * | 2013-05-17 | 2013-10-09 | 成都市晶林科技有限公司 | Switchable visible light and infrared integrated imaging system |
| CN103578097A (en) * | 2012-07-30 | 2014-02-12 | 中国科学院光电研究院 | Dual-waveband high-low resolution collaborative target identification device |
-
2014
- 2014-04-29 CN CN201410179327.XA patent/CN103974039B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201504278U (en) * | 2009-06-19 | 2010-06-09 | 深圳市汉锐科技有限公司 | Video circuit used for camera |
| CN101998111A (en) * | 2009-08-28 | 2011-03-30 | 杨占坤 | Video acquisition mode automatic conversion system and method of high-definition (HD) camera |
| US20110199489A1 (en) * | 2010-02-18 | 2011-08-18 | Testo Ag | Method for the extraction of an ir-image and thermal imaging camera |
| CN101923212A (en) * | 2010-07-06 | 2010-12-22 | 山东神戎电子股份有限公司 | High-definition laser night-vision viewer |
| CN102694977A (en) * | 2011-03-25 | 2012-09-26 | 杨占昆 | Method for automatically converting between high definition mode and standard definition mode of high-definition camera |
| CN103220487A (en) * | 2012-01-20 | 2013-07-24 | 北京凯新创达科技发展有限公司 | Universal converter capable of converting high definition into standard definition |
| CN102752502A (en) * | 2012-04-28 | 2012-10-24 | 北京汉王智通科技有限公司 | Intelligent camera and intelligent camera shooting method |
| CN103578097A (en) * | 2012-07-30 | 2014-02-12 | 中国科学院光电研究院 | Dual-waveband high-low resolution collaborative target identification device |
| CN102833511A (en) * | 2012-09-18 | 2012-12-19 | 浙江红苹果电子有限公司 | Method and device for storing high-definition videos based on serial digital interface (SDI) |
| CN103051887A (en) * | 2013-01-23 | 2013-04-17 | 河海大学常州校区 | Eagle eye-imitated intelligent visual sensing node and work method thereof |
| CN203231819U (en) * | 2013-05-17 | 2013-10-09 | 成都市晶林科技有限公司 | Switchable visible light and infrared integrated imaging system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106488195A (en) * | 2016-12-13 | 2017-03-08 | 合肥英威晟光电科技有限公司 | A kind of monitoring system based on thermal imaging |
| CN106657925A (en) * | 2017-01-18 | 2017-05-10 | 沈阳上博智像科技有限公司 | Video output method and thermal infrared imager |
| CN109618082A (en) * | 2018-12-17 | 2019-04-12 | 珠海市国宏网络科技有限公司 | A kind of high-definition intelligent ball machine and data transmission method |
| CN109756706A (en) * | 2018-12-17 | 2019-05-14 | 珠海市国宏网络科技有限公司 | A kind of distributed video monitoring analysis system and data transmission method |
| CN113408506A (en) * | 2021-08-20 | 2021-09-17 | 航天宏图信息技术股份有限公司 | All-time cloud detection method and device for geostationary satellite |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103974039B (en) | 2017-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103974039A (en) | Intelligent infrared thermal imaging and high-definition visible light dual-waveband video monitoring system | |
| US9628750B2 (en) | Transmission scheme and image quality adaptive security camera and DVR system | |
| US10509614B2 (en) | Video display apparatus-apparatus communication | |
| CN105704445B (en) | A kind of upgrade method of video camera | |
| US8350898B2 (en) | Head-mounted visual display device with stereo vision and its system | |
| CN102223512B (en) | System and device for supporting conversion of multi-format audio-video signal and serial data | |
| CN104394349A (en) | Warm backup multi-format switching system and method for video conference | |
| JP2004357029A (en) | Device and method for selecting signal | |
| CN204206329U (en) | A kind of multi-format Hot Spare switched system for video conference | |
| KR20080020305A (en) | Method for automatically selecting resolution and video receving apparatus thereof | |
| CN104168445A (en) | dynamic video storage method and network security monitoring device | |
| CN104243853A (en) | High-definition multimedia interface (HDMI) double-display device and display method | |
| CN104702906A (en) | High-resolution monitoring video system and control method thereof | |
| CN113612938A (en) | Multi-type adaptive resolution image conversion method and device | |
| CN207251800U (en) | A kind of intelligent SDI video switching boxs based on FPGA | |
| CN102075719B (en) | Audio and video equipment and startup and shutdown method thereof | |
| CN112383732A (en) | Signal transmission system and method with adaptive resolution | |
| KR20130096439A (en) | Remote monitoring and controlling system of semiconductor equipment | |
| CN107707829A (en) | A kind of method that multiplex roles intelligence SDI video switching boxs are realized based on FPGA | |
| US20110292035A1 (en) | Image output control apparatus and image output control method | |
| CN107846588B (en) | Method and device for acquiring serial port record information in television | |
| CN102655574A (en) | Device for controlling display equipment, method for controlling television and television | |
| CN110418079A (en) | Image signal conversion equipment | |
| KR100990357B1 (en) | Matrix switcher and operating method thereof | |
| US7925106B2 (en) | Apparatus and method for detecting motion in continuous images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |