CN113156642B - Transmission type image differential device and image edge extraction system formed by same - Google Patents
Transmission type image differential device and image edge extraction system formed by same Download PDFInfo
- Publication number
- CN113156642B CN113156642B CN202110410651.8A CN202110410651A CN113156642B CN 113156642 B CN113156642 B CN 113156642B CN 202110410651 A CN202110410651 A CN 202110410651A CN 113156642 B CN113156642 B CN 113156642B
- Authority
- CN
- China
- Prior art keywords
- image
- differential device
- films
- transmission type
- edge extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0012—Optical design, e.g. procedures, algorithms, optimisation routines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/28—Design optimisation, verification or simulation using fluid dynamics, e.g. using Navier-Stokes equations or computational fluid dynamics [CFD]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/06—Multi-objective optimisation, e.g. Pareto optimisation using simulated annealing [SA], ant colony algorithms or genetic algorithms [GA]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
- Y02E60/00—Enabling technologies; Technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Geometry (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Optimization (AREA)
- Fluid Mechanics (AREA)
- Pure & Applied Mathematics (AREA)
- Computer Hardware Design (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Algebra (AREA)
- Mathematical Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention provides a transmission type image differential device and an image edge extraction system formed by the same, wherein the transmission type image differential device comprises a plurality of layers of films which are arranged on a substrate in a sequentially overlapped mode, and the dielectric constants of two adjacent layers of films are different; when incident light waves generate multiple interference effects among the multilayer films, the transmission coefficient of the multilayer films and the transfer function between the incident angles of the light waves conform to a second-order differential function by optimally designing the physical parameters of the multilayer films, and image differentiation is realized. The image edge extraction system comprises an image loading system, a transmission type image differential device and an image receiving system which are sequentially connected. The invention can realize two-dimensional space filtering with large numerical aperture and insensitive polarization; and compared with structures such as super-surface photonic crystals and the like, the thin film structure is easier to process, and the large-scale manufacturing can be easily realized by utilizing mature processing technology.
Description
Technical Field
The invention can be applied to the fields of satellite remote sensing, computer vision and the like, and relates to an image differential device for realizing image edge extraction and an image edge extraction system formed by the same.
Background
The high-frequency spatial information of the image can be selectively extracted by carrying out spatial differentiation on the image, and the enhancement of the image edge information is realized. The edge information contains the most critical information of the geometric features of the target object in the image, and is widely applied to image processing technologies such as target identification, target reconstruction, image enhancement, information compression and the like. The most common method for extracting edge information is to convert image information into digital signals through a computer by using an electrical method and then process the images, but the method is limited by the computing speed of the computer, needs a large amount of power consumption, and cannot meet the requirements of emerging applications such as automatic driving and the like on low power consumption and real-time performance. The optical simulation method can directly process optical signals, usually an amplitude mask is placed in the center of a Fourier plane of a 4-f optical imaging system, low-wave vector components of incident light are filtered, image differentiation can be achieved, and high-resolution images can be processed in real time and in parallel with the minimum power consumption. However, the 4-f system requires many bulky optical element fits and requires the amplitude reticle to be placed accurately in the fourier plane.
Based on Green's function
(where k isxAnd kyThe components of the spatial wave vector of the Brillouin zone in the x direction and the y direction respectively, and the wave number represents Fourier transform), the spatial filtering of the image can be directly realized without the Fourier transform by designing the response of an optical filter in a 4-f optical imaging system related to the incident light angle, and the method has great potential in the aspect of miniaturization of an image differential device, does not need the precise positioning of the optical filter, and has attracted great attention in recent years. A multilayer thin film composed of a metamaterial having a dielectric constant of-2.12 to 13.85 can realize a second order spatial differential, but the processing of such a multilayer thin film having a continuously varying dielectric constant is difficult to realize. On the basis, the multilayer film made of two dielectric materials also realizes the spatial differentiation of images, but the multilayer film aims at the image differentiation in a reflection mode, needs other reflection elements to be matched, and has poor integration. In recent years, one-dimensional (1D) differentiation of a p-polarized image in the reflection mode is also realized by the surface plasmon resonance of a metal thin film. In addition, there is work to achieve transmission-type image differentiation using photonic crystals and metamorphic surfaces, some of which are designed to enable two-dimensional (2D) image differentiation and are polarization insensitive. However, the fabrication of large area photonic crystal/metal surface samples with sub-wavelength dimensions typically requires expensive and time consuming lithographic processes, which may limit their application.
Disclosure of Invention
The problems that the conventional image differential device is large in size, difficult to integrate, high in processing difficulty, capable of only working in a reflection mode, sensitive to polarization and the like are solved. The invention realizes the two-dimensional transmission type image differentiator which is easy to process, has high numerical aperture and is insensitive to polarization and the image edge extraction system formed by the differentiator by designing the material, the layer number and the thickness of the multilayer film structure.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a transmission type image differential device which is characterized by comprising a plurality of layers of films which are sequentially stacked on a substrate, wherein the dielectric constants of two adjacent layers of films are different; when incident light waves generate multiple interference effects among the multilayer films, the transmission coefficient of the multilayer films and the transfer function between the incident angles of the light waves are enabled to accord with a second-order differential function by optimizing and designing physical parameters of the multilayer films, and image differentiation is achieved.
Further, the physical parameters of the multilayer film include the total number of layers of the multilayer film, and the dielectric constant and thickness of each layer of the film.
The invention also provides an image edge extraction system, which comprises an image loading system, a space light field differential device and an image receiving system which are sequentially connected, and is characterized in that the space light field differential device adopts the transmission type image differential device.
Compared with the prior art, the invention has the following characteristics and beneficial effects:
the invention provides a transmission type image differential device, which generates strong transmission depending on the incident angle of light waves according to the multiple interference effect among multilayer thin films, and makes the transmission coefficient of the multilayer thin films and the transfer function between the incident angles of the light waves accord with a second order differential function by designing the dielectric constant, the number of layers and the thickness of the multilayer thin films, thereby realizing the differentiation of images. The optimization algorithm is adopted to optimize the multilayer film structure, so that two-dimensional spatial filtering with large numerical aperture and insensitive polarization can be realized; the thin film structure is easier to process compared with the structures such as the super-surface photonic crystal and the like, and the image differential device can be easily manufactured on a large scale by utilizing a mature processing technology; the numerical aperture is large, and the image resolution is high; the device is insensitive to the polarization direction of incident light, and a polarizing plate is not required to be added into an image edge extraction system formed by the transmission type image differentiator; two-dimensional spatial filtering can be directly realized, and image information is not lost in single imaging; further, the transmission image differentiator of the present invention may be conveniently integrated with a compact imaging system for applications including, but not limited to, microscopy, consumer electronics, and autopilot.
Drawings
Fig. 1 is a schematic structural view of a transmission type image differentiating device according to an embodiment of the present invention;
FIG. 2 is a simulation of the transmission coefficient transfer function of the multilayer thin film structure of the embodiment of FIG. 1;
fig. 3 is a schematic configuration diagram of an image edge extraction system constituted by the transmissive image differentiating device shown in fig. 1.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Fig. 1 is a schematic structural view of a transmission type image differentiating device according to an embodiment of the present invention, the transmission type image differentiating device including a plurality of films 1-1 stacked in sequence on a substrate 1-2, the adjacent films having different dielectric constants; when the incident light wave 3 generates multiple interference effect between the multiple thin films (interference generated by meeting reflected or refracted light beams by the upper and lower surfaces of the thin films, different optical path differences are generated by the thin films with different thicknesses and dielectric constants, and the thin films are overlapped to form multiple interference), namely strong transmission depending on the incident angle can be generated between the multiple thin films 1-1, and the transfer function between the transmission coefficient t of the multiple thin films and the light wave incident angle theta conforms to a second order differential function by optimally designing the physical parameters of the multiple thin films 1-1, so that image differentiation is realized. Specifically, the change of the transmission coefficient of the multilayer film with the incident angle of the light wave, namely the transfer function between the transmission coefficient of the multilayer film and the incident angle of the light wave, is related to the physical parameters of the multilayer film, including the total number of layers of the multilayer film, the thickness of each layer of the film and the dielectric constant of each layer of the film; on the basis, the physical parameters of the multilayer film are optimized by using an optimization algorithm, and a target transfer function with the characteristics of high numerical aperture, polarization insensitivity and the like can be determined. According to the required transfer function and the working wave band applied by the transmission type image differential device, the thin film material can be made of materials capable of generating multiple interference, such as medium or metal, such as amorphous silicon, silicon dioxide, titanium dioxide, gold, silver and the like, and the substrate material can be made of materials with high transmission rate (the transmission rate is more than or equal to 80 percent) in the required working wave band, such as silicon dioxide, fused silica and the like.
The total layer number, the thickness and the dielectric constant of the film are optimized by adopting an optimization algorithm, the difference value between a target transfer function and an actual transfer function generated by the multilayer film is used as a loss function of the optimization algorithm, the loss function is minimized to be the target function of the optimization algorithm, the physical parameters of the film are optimized, the actual transfer function of the multilayer film can be modulated to be the most in line with the condition of the target transfer function, and the optimization algorithm can be selected from a particle swarm algorithm, a simulated annealing algorithm, a genetic algorithm, a convex optimization algorithm, a deep learning algorithm and the like. Loss function O (k)x,ky) Is represented by the general formula:
wherein Htarget(. h) is the target transfer function; h (·); k is a radical ofxAnd kyThe components of the spatial wave vector of the brillouin zone in the x and y directions, respectively.
In the embodiment, in order to reduce the processing difficulty, the multilayer film is prepared by selecting silicon dioxide films 1-1-1 and titanium dioxide films 1-1-2 which are alternately stacked and have mature processing technology, wherein the substrate is fused quartz 1-2, and the working waveband is 532 nm. The method comprises the following steps of optimizing the number and thickness of layers of the film by adopting a particle swarm optimization algorithm, setting parameters such as the number of group particles, learning factors, an inertia weight range and a searching speed, and setting an upper limit value of the total thickness of the multilayer film 1-1 and upper and lower limit values of the thickness of a single layer of film according to actual processing capacity, wherein in the embodiment, the upper limit value of the total thickness is set to be 15 mu m, the upper limit value of the single layer of film is 1 mu m, the difference value between a transmission coefficient generated by the multilayer film and a green function is taken as a target function of the optimization algorithm, and a loss function is as follows:
target transfer function H adopted in the present embodimenttarget(kx,ky) Comprises the following steps:
wherein, alpha is a constant and NA is the numerical aperture; hs(·)、Hp(. cndot.) is the actual transfer function of the incident light wave in s and p polarization directions, respectively, as the quantity to be solved.
In this example, a structure consisting of 13 pairs of the silica thin film 1-1-1 and the titania thin film 1-1-2 and the fused silica substrate 1-2 was obtained by optimization, and the transmission coefficient with angle change curve thereof is shown in fig. 2, and a transfer function curve conforming to an ideal green function was realized for both s-and p-polarized light within a numerical aperture range of 0.31.
The invention also provides an image edge extraction system formed based on the transmission type image differential device, and referring to fig. 3, the system comprises an image loading system, a spatial light field differential device and an image receiving system which are connected in sequence.
The image loading system comprises a light source 2 and an object to be measured 3 which are arranged on a common optical axis, and light waves generated by the light source 2 carry image information of the object to be measured and are scattered after passing through the object to be measured 3.
The spatial light field differential device is arranged on a light path between an object to be detected 3 and an image receiving system, is a transmission type image differential device 1 and comprises a plurality of layers of films 1-2 which are sequentially stacked and arranged on a substrate 1-1, and can generate multiple interference effects among the plurality of layers of films by changing physical parameters (such as dielectric constant, thickness, layer number and the like of the films) of the films, so that a transfer function of incident light angle change is realized, the transfer function of the multilayer film differentiator near a certain working waveband meets a Green function, equivalent spatial differential operation can be realized, and edge extraction of the image of the object to be detected is realized.
The image receiving system comprises a focusing lens 4 and a photoelectric detector 5, the focusing lens 4 focuses an image of the object to be measured 3 extracted by the transmission type image differential device on the photoelectric detector 5, and the photoelectric detector 5 converts a received image optical signal into an electric signal to be displayed on a display.
Under the irradiation of the light source 2, the image of the object 3 to be detected after being processed by the space light field differential device is input to an image receiving system for receiving and detecting. Depending on the desired application, the light source may be selected from a laser or the like operating at a wavelength from visible to infrared, in this example 532 nm. The system has no specific requirement on the position of the space light field differential device, the space light field differential device can move freely along the X, Y and Z axial directions, does not need to be placed accurately and can be directly integrated with an image receiving system.
Finally, it should be noted that: the present invention is not described in detail and belongs to the known technology in the field, the above embodiments are only used to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (7)
1. A transmission type image differential device is characterized by comprising a plurality of films which are sequentially stacked on a substrate, wherein the dielectric constants of two adjacent films are different; when incident light waves generate multiple interference effects among the multilayer films, the transmission coefficient of the multilayer films and the transfer function between the incident angles of the light waves conform to a second-order differential function by optimally designing the physical parameters of the multilayer films, and image differentiation is realized; the physical parameters of the multilayer film comprise the total number of layers of the multilayer film, and the dielectric constant and the thickness of each layer of the film;
optimizing the total number of layers, the thickness and the dielectric constant of the multilayer film by adopting an optimization algorithm, and taking the difference value between a target transfer function and an actual transfer function between the transmission coefficient and the light wave incident angle of the multilayer film as a loss function O (k) of the optimization algorithmx,ky) Minimizing the loss function as an objective function of the optimization algorithm; wherein the loss function O (k)x,ky) The expression of (c) is as follows:
wherein, alpha is a constant, and NA is a numerical aperture; hs(·)、Hp(. h) is the actual transfer function of the incident light wave in the s and p polarization directions, respectively; htarget(. h) is the target transfer function; k is a radical of formulax,kyThe components of the spatial wave vector of the brillouin zone in the x and y directions, respectively.
2. The device according to claim 1, wherein the thin film is made of a material selected to generate multiple interference, and includes a dielectric and a metal; the medium comprises amorphous silicon, silicon dioxide and titanium dioxide; the metal includes gold and silver.
3. The device of claim 1, wherein the substrate is made of a material having a high transmittance in a desired operating band, and comprises silica and fused silica.
4. The transmissive image differentiating device according to claim 1, wherein the optimization algorithm comprises a particle swarm algorithm, a simulated annealing algorithm, a genetic algorithm, a convex optimization algorithm, and a deep learning algorithm.
5. An image edge extraction system comprises an image loading system, a spatial light field differential device and an image receiving system which are connected in sequence, and is characterized in that the spatial light field differential device adopts a transmission type image differential device according to any one of claims 1 to 4.
6. The image edge extraction system of claim 5, wherein the image loading system comprises a light source and an object to be measured, which are arranged coaxially, and the light waves generated by the light source carry image information of the object to be measured and are scattered after passing through the object to be measured.
7. The image edge extraction system of claim 6, wherein the image receiving system comprises a focusing lens and a photodetector, the focusing lens focuses an image of the object to be measured extracted by the transmissive image differentiating device on the photodetector, and the photodetector converts the received image optical signal into an electrical signal to be displayed on the display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110410651.8A CN113156642B (en) | 2021-04-14 | 2021-04-14 | Transmission type image differential device and image edge extraction system formed by same |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110410651.8A CN113156642B (en) | 2021-04-14 | 2021-04-14 | Transmission type image differential device and image edge extraction system formed by same |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113156642A CN113156642A (en) | 2021-07-23 |
| CN113156642B true CN113156642B (en) | 2022-07-05 |
Family
ID=76868309
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110410651.8A Active CN113156642B (en) | 2021-04-14 | 2021-04-14 | Transmission type image differential device and image edge extraction system formed by same |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113156642B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116719111B (en) * | 2023-08-03 | 2023-10-20 | 中国科学院长春光学精密机械与物理研究所 | Planar micro-nano optical analog computing device |
| CN117109643B (en) * | 2023-10-24 | 2024-01-02 | 中国科学院长春光学精密机械与物理研究所 | Test method and system using transmission type angle sensor |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013031051A (en) * | 2011-07-29 | 2013-02-07 | Ricoh Co Ltd | Image pickup device and object detection device incorporating the same and polarization filter |
| CN108629787A (en) * | 2018-04-02 | 2018-10-09 | 浙江大学 | A kind of image edge extraction method and system based on optical spin Hall effect spatial light field differentiator |
| CN109477924A (en) * | 2016-07-13 | 2019-03-15 | 3M创新有限公司 | Optical film |
| CN110412667A (en) * | 2019-07-17 | 2019-11-05 | 深圳市隆利科技股份有限公司 | Multilayered structure optical thin film |
-
2021
- 2021-04-14 CN CN202110410651.8A patent/CN113156642B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013031051A (en) * | 2011-07-29 | 2013-02-07 | Ricoh Co Ltd | Image pickup device and object detection device incorporating the same and polarization filter |
| CN109477924A (en) * | 2016-07-13 | 2019-03-15 | 3M创新有限公司 | Optical film |
| CN108629787A (en) * | 2018-04-02 | 2018-10-09 | 浙江大学 | A kind of image edge extraction method and system based on optical spin Hall effect spatial light field differentiator |
| CN110412667A (en) * | 2019-07-17 | 2019-11-05 | 深圳市隆利科技股份有限公司 | Multilayered structure optical thin film |
Non-Patent Citations (1)
| Title |
|---|
| Extreme Nonlinear Optics in Epsilon-near-zero Materials;Yuanmu Yang;《24th Microoptics Conference》;20191120;第54-55页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113156642A (en) | 2021-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11543653B2 (en) | Device components formed of geometric structures | |
| CN113156642B (en) | Transmission type image differential device and image edge extraction system formed by same | |
| CN109799611B (en) | Design method of achromatic super-structure lens and achromatic super-structure lens | |
| Bai et al. | Chip-integrated plasmonic flat optics for mid-infrared full-Stokes polarization detection | |
| JP6164212B2 (en) | Imaging optical system, imaging device | |
| EP1789829A1 (en) | Low height imaging system and associated methods | |
| US9366877B2 (en) | Planar diffractive optical element lens and method for producing same | |
| KR20210043590A (en) | Meta-surface main lens, secondary lens, manufacturing method and optical system thereof | |
| Zhao et al. | Synthetic aperture metalens | |
| JP4916187B2 (en) | Lens system | |
| CN114280707A (en) | Full-polarization medium super-structured lens and use method thereof | |
| Song et al. | Topological differential microscopy based on the spin–orbit interaction of light in a natural crystal | |
| Peng et al. | Metalens in improving imaging quality: advancements, challenges, and prospects for future display | |
| CN114942527B (en) | Super-surface-based first-order second-order multiplexing optical field differential device and design method thereof | |
| Wang et al. | Metalens for accelerated optoelectronic edge detection under ambient illumination | |
| Feng et al. | Insect-inspired nanofibrous polyaniline multi-scale films for hybrid polarimetric imaging with scattered light | |
| WO2022198736A1 (en) | Fabry-perot cavity array and spectrum detector | |
| Zheng et al. | Intelligent multi-channel meta-imagers for accelerating machine vision | |
| CN116719111B (en) | Planar micro-nano optical analog computing device | |
| CN114740689A (en) | Mosaic snapshot type micro-optical filter array based on nano-imprinting and preparation method thereof | |
| JP2015115329A (en) | Imaging element and imaging apparatus comprising the same | |
| Xie et al. | Efficient fabrication method for non-periodic microstructures using one-step two-photon lithography and a metal lift-off process | |
| Macé et al. | Progress in thin film coating on space IR detectors | |
| JP4825053B2 (en) | Optical device | |
| Colburn et al. | Single-shot three-dimensional imaging with a metasurface depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |