CN111221762B - Image controller and image system - Google Patents
Image controller and image system Download PDFInfo
- Publication number
- CN111221762B CN111221762B CN202010117392.5A CN202010117392A CN111221762B CN 111221762 B CN111221762 B CN 111221762B CN 202010117392 A CN202010117392 A CN 202010117392A CN 111221762 B CN111221762 B CN 111221762B
- Authority
- CN
- China
- Prior art keywords
- image
- data
- image controller
- interface
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/382—Information transfer, e.g. on bus using universal interface adapter
- G06F13/385—Information transfer, e.g. on bus using universal interface adapter for adaptation of a particular data processing system to different peripheral devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
Abstract
The invention relates to an image controller and an image system, the image controller comprises: the image controller interface comprises a first data interface and a plurality of second data interfaces which are arranged in parallel, the plurality of second data interfaces comprise at least one third data interface, and when the third data interface is in a first gear, the third data interface and the rest second data interfaces are used for transmitting data signals which are sent by the same image sensor and are related to clock signals to the image controller; when the third data interface is in the second gear, the third data interface is used for transmitting the clock signal sent by another image sensor to the image controller, and the remaining second data interfaces are used for transmitting the data signal which is sent by each image sensor and is associated with the corresponding clock signal to the image controller, so that the image controller is simultaneously connected with at least two image sensors through the image controller interfaces, and the design complexity and the manufacturing cost of the image controller can be reduced.
Description
Technical Field
The present invention relates to the field of integrated circuit technologies, and in particular, to an image controller and an image system.
Background
The existing image controller has certain limitation, usually, only one image sensor can be connected through an image controller interface, if the function that the image controller is simultaneously connected with at least two image sensors is to be realized through the image controller interface, the number of data interfaces arranged on the image controller interface needs to be increased, and thus, the design complexity and the manufacturing cost of the image controller can be greatly increased.
Disclosure of Invention
Based on this, it is necessary to provide an image controller and an image system capable of realizing connection with at least two image sensors without changing the number of data interfaces provided for an image controller interface.
An image controller comprising: the image controller interface comprises a first data interface and a plurality of second data interfaces which are arranged in parallel, the first data interface is used for transmitting a clock signal sent by an image sensor to the image controller, the plurality of second data interfaces comprise at least one third data interface, and the third data interface can be switched between a first gear and a second gear; when the third data interface is in the first gear, the third data interface and the remaining second data interfaces are used for transmitting data signals which are sent by the same image sensor and are related to the clock signals to the image controller, so that the image controller is connected with the same image sensor through the image controller interface; when the third data interface is in the second gear, the third data interface is used for transmitting a clock signal sent by another image sensor to the image controller, and the remaining second data interfaces are used for transmitting the data signal sent by each image sensor and associated with the corresponding clock signal to the image controller, so that the image controller is simultaneously connected with at least two image sensors through the image controller interface.
In one embodiment, the number of the image sensors simultaneously connected to the image controller through the image controller interface when the third data interface is switched to the second gear is adjustable by changing the number of the third data interfaces.
In one embodiment, the image controller further comprises a PLL circuit connected to the corresponding image sensor through a data interface corresponding to the image controller interface, and the PLL circuit is configured to multiply the clock signal received by the PLL circuit and sent by the corresponding image sensor.
In one embodiment, the image controller further includes a serial-to-parallel conversion module, where the serial-to-parallel conversion module is connected to the corresponding image sensor through a data interface corresponding to the image controller interface, and the serial-to-parallel conversion module is configured to perform serial-to-parallel conversion with different bit widths on the data signal received by the serial-to-parallel conversion module and sent by the corresponding image sensor.
In one embodiment, the image processing device further includes a buffer module, the buffer module is connected to the corresponding image sensor through a data interface corresponding to the image controller interface, and the buffer module is configured to buffer the clock signal sent by the corresponding image sensor and the data signal associated with the clock signal.
In one embodiment, the image analysis and integration device further comprises an image analysis and integration module, the image analysis and integration module is connected with the corresponding image sensor through a data interface corresponding to the image controller interface, and the image analysis and integration module is used for analyzing the received data signals sent by the image sensors and outputting the image data independently, and integrating the received data signals sent by the image sensors and outputting the synthesized image data.
An image system, comprising: the image controller and the image sensor are provided.
In one embodiment, the image sensor is a CMOS image sensor or a CCD image sensor.
In one embodiment, the clock signal and the data signal sent by the image sensor are a differential clock signal and a differential data signal, respectively.
In one embodiment, the method further comprises at least one of the following steps:
the image processing module is connected with the image controller and is used for processing the image data sent by the image controller; and
and the image compression module is connected with the image controller and is used for compressing the image data sent by the image controller.
When the third data interface is in the first gear, the third data interface and the remaining second data interfaces are used for transmitting data signals which are sent by the same image sensor and are related to clock signals to the image controller, so that the image controller is connected with the same image sensor through the image controller interface; when the third data interface is in the second gear, the third data interface is used for transmitting the clock signal sent by another image sensor to the image controller, and the remaining second data interfaces are used for transmitting the data signal which is sent by each image sensor and is associated with the corresponding clock signal to the image controller, so that the image controller is simultaneously connected with at least two image sensors through the image controller interface, and therefore the image controller can be connected with the at least two image sensors on the premise that the number of the data interfaces arranged on the image controller interface is not changed, the design complexity and the manufacturing cost of the image controller are effectively reduced, and the adaptation performance of the image controller is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a block diagram of an image system in an embodiment.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and the like as used herein are for illustrative purposes only and do not represent the only embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
As shown in fig. 1, the image controller 100 in an embodiment includes an image controller interface 110, where the image controller interface 110 includes a first data interface 112 and a plurality of second data interfaces 114 arranged in parallel, the first data interface 112 is used for transmitting a clock signal sent by the image sensor 200 to the image controller 100, the plurality of second data interfaces 114 includes at least one third data interface 116, and the third data interface 116 is capable of switching between a first gear and a second gear; when the third data interface 116 is in the first gear position, the third data interface 116 and the remaining second data interface 114 are used for transmitting data signals associated with clock signals, which are sent by the same image sensor 200, to the image controller 100, so that the image controller 100 is connected with the same image sensor 200 through the image controller interface 110; when the third data interface 116 is in the second gear position, the third data interface 116 is used for transmitting the clock signal transmitted by another image sensor 200 to the image controller 100, and the remaining second data interface 114 is used for transmitting the data signal transmitted by each image sensor 200 and associated with the corresponding clock signal to the image controller 100, so that the image controller 100 is simultaneously connected with at least two image sensors 200 through the image controller interface 110.
The image controller 100 includes at least one third data interface 116, where the plurality of second data interfaces 114 of the image controller interface 110 include the third data interface 116, the third data interface 116 can be switched between the first gear and the second gear, and when the third data interface 116 is in the first gear, the third data interface 116 and the remaining second data interfaces 114 are used for transmitting data signals associated with clock signals, which are sent by the same image sensor 200, to the image controller 100, so that the image controller 100 is connected to the same image sensor 200 through the image controller interface 110; when the third data interface 116 is in the second gear, the third data interface 116 is used for transmitting the clock signal sent by another image sensor 200 to the image controller 100, and the remaining second data interface 114 is used for transmitting the data signal associated with the corresponding clock signal sent by each image sensor 200 to the image controller 100, so that the image controller 100 is simultaneously connected with at least two image sensors 200 through the image controller interface 110, thereby enabling the image controller 100 to realize the connection between the image controller 100 and at least two image sensors 200 without changing the number of the data interfaces arranged on the image controller interface 110, effectively reducing the design complexity and the manufacturing cost of the image controller 100, and improving the adaptability of the image controller 100.
In one embodiment, the image controller 100 may be a GPU (Graphics Processing Unit) or a CPU (Central Processing Unit), and the image controller Interface 110 may be, but is not limited to, a MIPI (Mobile Industry Processor) Interface.
In one embodiment, the number of the image sensors 200 simultaneously connected to the image controller 100 through the image controller interface 110 can be adjusted by changing the number of the third data interfaces 116 when the third data interfaces 116 are switched to the second gear. Specifically, when the number of the third data interfaces 116 is increased, the number of the image sensors 200 simultaneously connected to the image controller 100 through the image controller interface 110 may be increased synchronously when the third data interface 116 is switched to the second gear; when the number of the third data interface 116 is reduced, the number of the image sensors 200 simultaneously connected by the image controller 100 through the image controller interface 110 can be synchronously reduced when the third data interface 116 is switched to the second gear position, so as to further enhance the adaptation performance of the image controller 100.
In an embodiment, the image controller 100 further includes a PLL (Phase Lock Loop) circuit 120, the PLL circuit 120 is connected to the corresponding image sensor 200 through a data interface corresponding to the image controller interface 110, and the PLL circuit 120 is configured to multiply the clock signal received by the PLL circuit and sent by the corresponding image sensor 200, so as to meet the use requirement of the clock signal with the transmission speed required by the image controller 100. In one embodiment, the specific multiple of the frequency multiplication of the clock signal transmitted by the image sensor 200 by the PLL circuit 120 is adjustable, depending on the transmission speed requirement of the image controller 100 for the accessed clock signal.
Specifically, the PLL circuits 120 are connected to the corresponding image sensors 200 through the first data interfaces 112 and the third data interfaces 116, and further, the number of the PLL circuits 120 corresponds to the sum of the number of the first data interfaces 112 and the third data interfaces 116.
In an embodiment, the image controller 100 further includes a serial-to-parallel conversion module 130, the serial-to-parallel conversion module 130 is connected to the corresponding image sensor 200 through a data interface corresponding to the image controller interface 110, and the serial-to-parallel conversion module 130 is configured to perform serial-to-parallel conversion with different bit widths on the data signals received by the serial-to-parallel conversion module and sent by the corresponding image sensor 200, so as to meet a usage requirement of the data signals in the parallel transmission format required by the image controller 100. In one embodiment, the serial-to-parallel conversion module 130 is connected to the corresponding image sensor 200 through the second data interface 114 and the third data interface 116.
In an embodiment, the image controller 100 further includes a buffer module 140, the buffer module 140 is connected to the corresponding image sensor 200 through a data interface corresponding to the image controller interface 110, and the buffer module 140 is configured to buffer a clock signal sent by the corresponding image sensor 200 and a data signal associated with the clock signal. Specifically, the buffer module 140 is connected to the serial-to-parallel conversion module 130, and the buffer module 140 is configured to buffer the clock signal and the data signal associated with the clock signal, which are converted by the serial-to-parallel conversion module 130 in a serial-to-parallel manner and transmitted by the corresponding image sensor 200. In one implementation, the buffer module 140 is connected to the serial-to-parallel conversion module 130 through a data transmission interface. Preferably, when the serial-parallel conversion module 130 is connected to the corresponding image sensor 200 through the third data interface 116, a control switch may be disposed between the buffer module 140 and the serial-parallel conversion module 130, and the control switch is used to control the data transmission interface between the buffer module 140 and the serial-parallel conversion module 130.
In an embodiment, the image controller 100 further includes an image decomposition and integration module, the image decomposition and integration module is connected to the corresponding image sensor 200 through a data interface corresponding to the image controller interface 110, and the image decomposition and integration module is configured to decompose the received data signals sent by the image sensors 200 and output the image data separately, and integrate the received data signals sent by the image sensors 200 and output the synthesized image data, so as to meet actual requirements of users for obtaining image frames in different forms. In one embodiment, the image decomposition and integration module is connected to each of the cache modules 140.
As shown in fig. 1, it should be noted that the present invention also provides an image system, which includes an image controller 100 and an image sensor 200.
In one embodiment, the clock signal and the data signal transmitted by the image sensor 200 are a differential clock signal and a differential data signal, respectively. Compared with the traditional data signal, the differential data signal has the advantages of high speed, low power consumption, low noise, low electromagnetic interference and the like, and the data signal output by the image system has stronger environmental adaptability by carrying out differential processing on the clock signal and the data signal sent by the image sensor 200, so that the data signal output by the image system is prevented from being received wrongly at a data receiving end due to the change of external environmental temperature and signal voltage fluctuation.
In an embodiment, the image sensor 200 may be a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge-coupled Device) image sensor.
In one embodiment, the image sensor 200 is a 4K image sensor, where a 4K image is an image with 3840 × 2160 pixels arranged in an array to provide a preview of a higher resolution image at which each detail in the image frame will be visible to the user; it is understood that in other embodiments, the image sensor 200 may be a conventional FHD image sensor, where an FHD image refers to an image with a pixel array of 1920 × 1080 arranged to provide a preview of a lower resolution image, and the specific model of the image sensor 200 may be flexibly selected according to the visual requirements of the user.
In an embodiment, the image system further includes an image processing module, where the image processing module is connected to the image controller 100, and the image processing module is configured to process image data sent by the image controller 100. Specifically, the image processing module is used for performing operations related to amplification, reduction, gain adjustment, color processing, and the like on the image data sent by the image controller 100.
In an embodiment, the image system further includes an image compression module, where the image compression module is connected to the image controller 100, and the image compression module is configured to perform compression processing on the image data sent by the image controller 100 to reduce a space occupied by the image data, so as to effectively reduce a transmission data amount of the image data.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An image controller, comprising: the image controller interface comprises a first data interface and a plurality of second data interfaces which are arranged in parallel, the first data interface is used for transmitting a clock signal sent by an image sensor to the image controller, the plurality of second data interfaces comprise at least one third data interface, and the third data interface can be switched between a first gear and a second gear; when the third data interface is in the first gear, the third data interface and the remaining second data interfaces are used for transmitting data signals which are sent by the same image sensor and are related to the clock signals to the image controller, so that the image controller is connected with the same image sensor through the image controller interface; when the third data interface is in the second gear, the third data interface is used for transmitting a clock signal sent by another image sensor to the image controller, and the remaining second data interfaces are used for transmitting the data signal sent by each image sensor and associated with the corresponding clock signal to the image controller, so that the image controller is simultaneously connected with at least two image sensors through the image controller interface.
2. The image controller according to claim 1, wherein the number of the image sensors simultaneously connected to the image controller through the image controller interface is adjustable by changing the number of the third data interfaces when the third data interface is switched to the second position.
3. The image controller according to claim 1, further comprising a PLL circuit connected to the corresponding image sensor through a corresponding data interface of the image controller interface, the PLL circuit configured to multiply the clock signal received by the PLL circuit and transmitted by the corresponding image sensor.
4. The image controller according to claim 1, further comprising a serial-to-parallel conversion module, wherein the serial-to-parallel conversion module is connected to the corresponding image sensor through a data interface corresponding to the image controller interface, and the serial-to-parallel conversion module is configured to perform serial-to-parallel conversion with different bit widths on the data signal received by the serial-to-parallel conversion module and sent by the corresponding image sensor.
5. The image controller according to claim 1, further comprising a buffer module, wherein the buffer module is connected to the corresponding image sensor through a data interface corresponding to the image controller interface, and the buffer module is configured to buffer the clock signal sent by the corresponding image sensor and the data signal associated with the clock signal.
6. The image controller according to claim 1, further comprising an image decomposition and integration module, wherein the image decomposition and integration module is connected to the corresponding image sensor through a data interface corresponding to the image controller interface, and the image decomposition and integration module is configured to decompose the data signal received and transmitted by each image sensor and output each image data separately, and integrate the data signal received and transmitted by each image sensor and output the synthesized image data.
7. An imaging system, comprising: the image controller and the image sensor as claimed in any one of claims 1 to 6.
8. The imaging system of claim 7, wherein the image sensor is a CMOS image sensor or a CCD image sensor.
9. The imaging system of claim 7, wherein the clock signal and the data signal sent by the image sensor are a differential clock signal and a differential data signal, respectively.
10. The imaging system of claim 7, further comprising at least one of:
the image processing module is connected with the image controller and is used for processing the image data sent by the image controller; and
and the image compression module is connected with the image controller and is used for compressing the image data sent by the image controller.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010117392.5A CN111221762B (en) | 2020-02-25 | 2020-02-25 | Image controller and image system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010117392.5A CN111221762B (en) | 2020-02-25 | 2020-02-25 | Image controller and image system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111221762A CN111221762A (en) | 2020-06-02 |
| CN111221762B true CN111221762B (en) | 2021-05-25 |
Family
ID=70831348
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010117392.5A Active CN111221762B (en) | 2020-02-25 | 2020-02-25 | Image controller and image system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111221762B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101827219A (en) * | 2010-01-22 | 2010-09-08 | 中兴通讯股份有限公司 | Method and device for controlling two cameras in master/slave mode in wireless terminal |
| CN102186015A (en) * | 2011-04-26 | 2011-09-14 | 北京思比科微电子技术股份有限公司 | Multi-image sensor image processing equipment and image processing method |
| CN104052934A (en) * | 2013-03-13 | 2014-09-17 | 三星电子株式会社 | Electronic Device And Method For Processing Image |
| CN104880330A (en) * | 2014-08-14 | 2015-09-02 | 深圳市亚泰光电技术有限公司 | Mechanical failure monitoring system and method |
| CN108959135A (en) * | 2017-05-24 | 2018-12-07 | 三星电子株式会社 | Data storage and processing system and its operating method |
| CN208707751U (en) * | 2018-08-15 | 2019-04-05 | 成都睿云物联科技有限公司 | A kind of data terminal serial expanded circuit based on video camera |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPWO2017149591A1 (en) * | 2016-02-29 | 2018-12-20 | オリンパス株式会社 | Image processing device |
| US10852877B2 (en) * | 2017-03-17 | 2020-12-01 | Intel Corporation | System, apparatus and method for communicating touch data |
| EP3707572B1 (en) * | 2017-11-10 | 2023-08-23 | Nvidia Corporation | Systems and methods for safe and reliable autonomous vehicles |
| CN209375765U (en) * | 2019-01-18 | 2019-09-10 | 中控智慧科技股份有限公司 | A kind of Image Acquisition mould group and face identification device |
-
2020
- 2020-02-25 CN CN202010117392.5A patent/CN111221762B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101827219A (en) * | 2010-01-22 | 2010-09-08 | 中兴通讯股份有限公司 | Method and device for controlling two cameras in master/slave mode in wireless terminal |
| CN102186015A (en) * | 2011-04-26 | 2011-09-14 | 北京思比科微电子技术股份有限公司 | Multi-image sensor image processing equipment and image processing method |
| CN104052934A (en) * | 2013-03-13 | 2014-09-17 | 三星电子株式会社 | Electronic Device And Method For Processing Image |
| CN104880330A (en) * | 2014-08-14 | 2015-09-02 | 深圳市亚泰光电技术有限公司 | Mechanical failure monitoring system and method |
| CN108959135A (en) * | 2017-05-24 | 2018-12-07 | 三星电子株式会社 | Data storage and processing system and its operating method |
| CN208707751U (en) * | 2018-08-15 | 2019-04-05 | 成都睿云物联科技有限公司 | A kind of data terminal serial expanded circuit based on video camera |
Non-Patent Citations (1)
| Title |
|---|
| 《新颖CCD图像传感器最新发展及应用》;程开富;《集成电路通讯》;20060930;第24卷(第3期);30-38 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111221762A (en) | 2020-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107509033B (en) | Remote sensing camera image real-time acquisition and processing system | |
| KR100382015B1 (en) | Moving picture transmitting apparatus and moving picture communicating apparatus | |
| CN107249101A (en) | A kind of sample of high-resolution image and processing unit | |
| JP2007151174A (en) | Method and device for data transmission | |
| CN111601019B (en) | Image data processing module and electronic equipment | |
| CN111314710B (en) | Video compression processing method and device for airborne multi-sensor multiprocessor of unmanned aerial vehicle | |
| US20210211585A1 (en) | Image processing system and image processing method | |
| US7350091B2 (en) | Control apparatus for controlling a plurality of computers | |
| CN111447409B (en) | Video compression processing method and device for airborne multi-sensor single processor of unmanned aerial vehicle | |
| CN111221762B (en) | Image controller and image system | |
| CN101562748B (en) | Method for optimizing scheduling of multidigital signal processor facing video processing | |
| CN109660746B (en) | MIPI signal distance transmission device and method | |
| CN106412384A (en) | Video processing system | |
| US20080117314A1 (en) | Device and method for controlling a camera module in a mobile terminal to reduce power consumption | |
| Sarkar et al. | FPGACam: A FPGA based efficient camera interfacing architecture for real time video processing | |
| US7268334B2 (en) | Image sensor device and a method for outputting digital signals | |
| CN211557366U (en) | Ultra-high definition four-way image divider | |
| WO2008057870A1 (en) | High-performance bipolar tuner solution systems and methods | |
| CN113612938A (en) | Multi-type adaptive resolution image conversion method and device | |
| Alzubi et al. | A top-up design for pal to vga conversion in real time video processing system | |
| CN101500094A (en) | Delay apparatus for regulating phase under standard moving video specification | |
| CN217116233U (en) | Video output circuit and FPGA chip | |
| CN101247474B (en) | Image processing device and method | |
| CN103414898A (en) | Method and system for collecting high-resolution video | |
| CN201044472Y (en) | Image processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |