CN110196232B - Confocal imaging device and confocal imaging method - Google Patents
Confocal imaging device and confocal imaging method Download PDFInfo
- Publication number
- CN110196232B CN110196232B CN201910476670.3A CN201910476670A CN110196232B CN 110196232 B CN110196232 B CN 110196232B CN 201910476670 A CN201910476670 A CN 201910476670A CN 110196232 B CN110196232 B CN 110196232B
- Authority
- CN
- China
- Prior art keywords
- plane
- sample
- confocal
- confocal imaging
- spectroscope
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/21—Polarisation-affecting properties
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/0816—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements
Abstract
The embodiment of the invention adopts an aperture imaging method to solve the problem of complexity in the scanning illumination process, and an adjustable plane reflector array is arranged, so that the problem that the confocal illumination cannot be moved is solved by adjusting the plane reflector array, the confocal imaging under the application scenes such as underwater and the like is realized, and the application scene range is expanded.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a confocal imaging device and a confocal imaging method.
Background
Confocal means that the optical path comprising excitation and emission is focused at two locations. During confocal imaging, excitation light is focused on the sample surface and emission light is focused on the illumination pinhole. The pinhole limits the focusing depth of the light path on the surface of the sample, and effectively prevents the background noise interference generated by impurity signals, thereby reducing the intensity of the background signals and improving the imaging quality of the sample. Confocal imaging methods were first applied to confocal microscopes, mainly using focus mode illumination and simultaneous imaging to create cross-sections of three-dimensional biological specimens. The method is an optical imaging means for removing scattered light of a sample non-focal plane by light focusing illumination and illumination pinhole modulation, and can improve optical resolution and visual contrast compared with other imaging methods. The confocal microscope makes great contribution in the aspect of biomedicine, has a plurality of results, and has great significance for researching tiny substances such as cells or brains.
However, confocal imaging has been limited to the design of microscopes, and the scope of application has been limited to biomedical applications. In order to adapt confocal imaging techniques to large-scale scenes, it has been proposed to use projectors and camera arrays instead of the optical apertures used by microscopes. In particular implementations, confocal imaging techniques fall into two categories: confocal scanning mode and confocal encoding mode. The confocal scanning mode is to realize point illumination and point detection by using an illumination pinhole arranged behind a light source unit and a detection pinhole arranged in front of a detector, light from the light source unit passes through the illumination pinhole to obtain emitted light and focuses the emitted light on a certain point of a focal plane of a sample, the point is imaged in the detection pinhole, and the detector acquires an image of the sample through the detection pinhole. In a confocal coding mode, a random mask is arranged in front of a set projector, a spectroscope is arranged in front of the random mask, the projector and a camera are coaxial by using the spectroscope, and an emergent projection light source emitted by the projector is reflected to a scene where a sample is located through the set plane mirror array after passing through the set random mask and the spectroscope; emergent light of the sample is reflected by the plane mirror array and then enters the camera through the spectroscope, and the camera acquires an image of the sample.
However, confocal imaging using both of the above methods has disadvantages: the confocal scanning mode has a small imaging range due to the small aperture of the illumination pinhole and the detection pinhole, and the scanning mode is adopted to complete confocal illumination and imaging, so that the scanning process is complicated and slow, and once the confocal scanning mode is applied to a slightly larger scene, a large amount of time and energy are consumed. The confocal coding mode overcomes the defects of the confocal scanning mode, but the plane mirror array is fixedly arranged, cannot be adjusted, is not simple, convenient and flexible enough, and limits the application scene range of confocal imaging.
Disclosure of Invention
In view of this, embodiments of the present invention provide a confocal imaging apparatus, which can simply and flexibly adjust the illumination depth of confocal imaging, so as to expand the application scene range of confocal imaging.
The embodiment of the invention also provides a confocal imaging method, which can simply, conveniently and flexibly adjust the illumination depth of confocal imaging and enlarge the application scene range of confocal imaging.
The embodiment of the invention is realized as follows:
a confocal imaging apparatus comprising: the device comprises a light source unit, a spectroscope, a polarizing film, a plane reflector array and an imaging unit, wherein the plane reflector array is obtained by splicing a plurality of rotatable plane reflectors, no gap exists between every two plane reflectors, and the spliced plane reflector array has a set radian;
the white light projected by the light source unit is transmitted to the polarizer by the spectroscope; the polaroid adjusts the polarization information of the projection light, and the adjusted white light forms an illumination beam to be focused on a focusing surface of the sample after being reflected by the plane mirror array;
the focus plane of the sample reflects the illumination light beam, and the imaging unit acquires the image of the focus plane of the sample after the reflected illumination light beam is reflected by the plane mirror array, transmitted by the polaroid and reflected by the spectroscope in sequence.
A confocal imaging method of providing the confocal imaging apparatus of claim 1, the method comprising:
white light projected by a light source unit in the confocal imaging device is transmitted to a polarizer in the confocal imaging device by a spectroscope in the confocal imaging device;
after the polarizing film in the confocal imaging device adjusts the polarization information of the projection light, the adjusted white light is reflected by a plane mirror array in the confocal imaging device to form an illumination beam and focuses on a focusing surface of a sample;
the focus plane of the sample reflects the illumination light beam, and the imaging unit acquires the image of the focus plane of the sample after the reflected illumination light beam is reflected by the plane mirror array, transmitted by the polaroid and reflected by the spectroscope in sequence.
As can be seen from the above, the embodiment of the present invention adopts the aperture imaging method to solve the problem of complexity in the scanning illumination process, and the adjustable planar mirror array is provided, so that the problem that the confocal illumination cannot be shifted is overcome by adjusting the planar mirror array, thereby realizing confocal imaging in application scenes such as underwater, and expanding the application scene range.
Drawings
Fig. 1 is a schematic structural diagram of a confocal imaging apparatus provided in an embodiment of the present invention;
fig. 2 is a flowchart of a confocal imaging method provided by an embodiment of the invention;
fig. 3 is a schematic structural diagram of a specific example of a confocal imaging apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of illumination and imaging optical paths provided by an embodiment of the present invention;
fig. 5 is a schematic view of a confocal shift concept provided by an embodiment of the invention.
Description of the reference symbols
11-light source unit
12-spectroscope
13-polarizing plate
14-plane mirror array
15-imaging unit
31-projector
32-camera
33-sample
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and examples.
Although the confocal coding method in the background art can expand the imaging range, the confocal illumination is completed without scanning, so that the confocal imaging process is simple and fast. However, the plane mirror array in the confocal encoding mode is fixedly arranged, cannot be adjusted, is not simple and flexible enough, and limits the application scene of confocal imaging. Therefore, in order to solve the problem, the embodiment of the invention adopts an aperture imaging method to solve the problem of complexity in the scanning illumination process, and the adjustable plane reflector array is arranged, so that the problem that confocal illumination cannot be moved is solved by adjusting the plane reflector array, confocal imaging in application scenes such as underwater and the like is realized, and the application scene range is expanded.
Fig. 1 is a schematic structural diagram of a confocal imaging apparatus provided in an embodiment of the present invention, including: the device comprises a light source unit 11, a spectroscope 12, a polarizing plate 13, a plane reflector array 14 and an imaging unit 15, wherein the plane reflector array 14 is obtained by splicing a plurality of rotatable plane reflectors, no gap exists between every two plane reflectors, and the spliced plane reflector array 14 has a set radian.
The white light projected by the light source unit 11 is transmitted to the polarizing plate 13 by the dichroic mirror 12; after the polarization information of the projection light is adjusted by the polarizer 13, the adjusted white light is reflected by the plane mirror array 14 to form an illumination beam and is focused on the focusing surface of the sample;
the illumination light beam is reflected by the focusing surface of the sample, and the imaging unit 15 acquires the image of the focusing surface of the sample after the reflected illumination light beam is reflected by the plane mirror array 14, transmitted by the polarizing plate 13 and reflected by the spectroscope 14 in sequence.
In the embodiment of the present invention, the white light projected by the light source unit 11 is encoded by the set projection image and then received by the beam splitter 12.
In the embodiment of the present invention, the polarizer 13 adjusts the polarization information of the projection light into the horizontal polarization and the vertical polarization; the set projection image includes: and calculating depth distribution information of the sample application scene according to the adjustment information of the horizontal polarization and the vertical polarization of the polarizing plate 13, and setting a projection image according to the depth distribution information of the sample application scene.
In an embodiment of the invention, the set projection image is adjusted or updated according to the application scene change of the sample.
Thus, when the device is used for the first confocal imaging, the depth distribution information of the application scene of the sample can be obtained according to the polarization information of the polarizing film 13, and then a projection image is obtained, so that when the device is used for the second confocal imaging, the projection image is used for projecting white light. The invalid foreground and the interesting part of the sample are judged and determined according to the depth distribution information of the sample application scene, and the set projection image can reduce the illumination of the foreground of the sample, so that the illumination contrast of the interesting part of the sample and the invalid foreground is improved.
In the embodiment of the invention, if the application scene is changed, the device is used for adjusting the brightness distribution of the projected image to realize the self-adaptive confocal imaging. Specifically, the projection image is adjusted or replaced according to the application scene, and the self-adaptive confocal lapse imaging is realized.
In the embodiment of the present invention, the light source unit 11 may be a projector.
In the embodiment of the present invention, the radian set by the plane mirror array 14 realizes the reflection of the adjusted white light onto the focus plane of the sample. The angle and the set arc of the plane mirror array 14 can be automatically adjusted.
In the embodiment of the invention, the spectroscope 12 is a spectroscope with a light transmission ratio of 50:50 and is arranged at an angle of 45 degrees with the projection optical axis of the device; the polarizing plate 13 is a color transflective polarizing plate, and has a function of shielding and transmitting the projection light.
In the embodiment of the present invention, the imaging unit 15 is a Charge Coupled Device (CCD) plane provided for the camera.
In the embodiment of the present invention, the imaging unit 15 further includes a computer, and the image of the focal plane of the sample is acquired by the imaging unit 15 and is output to the computer for processing, and the sample is identified and divided.
In the embodiment of the invention, the application scene of the device is underwater.
In the embodiment of the present invention, the position of the light source unit 11 is perpendicular to the position of the imaging unit 15, and coaxial imaging is realized.
Fig. 2 is a flowchart of a confocal imaging method provided in an embodiment of the present invention, and a confocal imaging apparatus is provided, which includes the following specific steps:
In the embodiment of the invention, the plane mirror array is obtained by splicing a plurality of rotatable plane mirrors, no gap exists between every two plane mirrors, and the spliced plane mirror array has a set radian. The angle and the set radian of the plane mirror array are controlled and adjusted.
In the method, before the white light is transmitted to a polarizer in the confocal imaging device by a spectroscope in the confocal imaging device, the method further comprises: the projected white light is encoded according to a set projection image and then received by the spectroscope;
the adjustment of the polarization information of the projection light is the adjustment of horizontal polarization and vertical polarization;
the set projection image includes: calculating depth distribution information of the sample application scene according to the adjustment information of the horizontal polarization and the vertical polarization of the projection light, and setting a projection image according to the depth distribution information of the sample application scene;
the set projection image is adjusted or updated according to the application scene change of the sample.
In the embodiment of the invention, if the application scene is changed, the device is used for adjusting the brightness distribution of the projected image to realize the self-adaptive confocal imaging.
It can be seen that the embodiment of the invention adopts the controllable plane mirror array, and the angle and the set radian of the plane mirror array can be adjusted through external control, so that the illumination light beam is focused on the focusing surface of the sample, the problem of complicated confocal scanning illumination in the background technology is solved, and the multi-depth confocal illumination transition is realized. The embodiment of the invention combines an aperture imaging mode and a confocal illumination moving mode, thereby realizing multi-view confocal imaging and solving the problems of weak astigmatism and shielding under severe application scenes, particularly underwater. The embodiment of the invention adopts the method for setting the projection image codes, and the white light projected by the light source unit is coded by the projection image, so that the coded white light can irradiate the focus plane of the sample more accurately, and the reflection of the foreground targets (relative to the interested target) with different depths to the illumination light is effectively reduced, thereby realizing the high-contrast illumination distribution of the sample and the invalid foreground, and better obtaining the sample focus plane image.
Fig. 3 is a schematic structural diagram of a specific example of a confocal imaging apparatus according to an embodiment of the present invention. The apparatus consists of a projector 31, a spectroscope 12, a polarizing plate 13, a plane mirror array 14, a camera 32, and a sample 33. Wherein the spectroscope 12 is arranged at an angle of 45 degrees with the horizontal plane, and the selected transmittance inverse ratio is 50: 50. The planar mirror array 14 for coaxial imaging is composed of 4 × 4 or 5 × 5, or even more planar mirrors with adjustable size of 50 mm × 50 mm, which is realized by the semi-transparent and semi-reflective function of the beam splitter 12 and the placement position of the projector 31 is perpendicular to that of the camera 32. By adjusting the rotation angle of each plane mirror, the plane mirror can be reflected to form an illumination beam focused on the same focal plane of the sample 33.
In the confocal imaging process, the light source of the projector 31 is emitted from the point A and is encoded by the projection pattern in front of the light source; the coded light source is projected by a spectroscope 12, and a polarizing plate 13 changes the polarization information of the projected light source to a plane mirror array 14; then reflected by the plane mirror array 14 to the application scene of the sample 33;
the reflected light of the application scene of the sample 33 returns to the plane mirror array 14, the plane mirror array 14 reflects the reflected light, passes through the polarizing plate 13 and the spectroscope 12, and is reflected again to the camera 32 to be acquired by the camera 32; the camera transmits the acquired image to the computer for subsequent image processing, thereby completing the identification, segmentation and the like of the sample 33.
Fig. 4 is a schematic diagram of illumination and imaging optical paths provided by the embodiment of the present invention. In the illumination light path, a light source is emitted from a projector, and the surface light source is encoded by a projection image, completely transmits to the spectroscope and irradiates on the plane mirror array through the spectroscope. In the plane mirror array, each plane mirror receives a part of the light source and reflects the part of the light source to the application scene of the sample, and the plane mirror array illuminates the application scene of the sample in total. In the imaging optical path, the sample reflected light returns to each plane reflector in the plane reflector array, each plane reflector can receive a sample image, and the position angle of each plane reflector is different, so that the image formed by each plane reflector has a certain parallax error; the planar reflector array transmits sample images of multiple viewing angles through the polaroid, reflects the sample images onto the spectroscope, and reflects the sample images again through the spectroscope, so that the sample images are acquired by a CCD plane arranged on the camera.
Fig. 5 is a schematic view of a confocal shift concept provided by an embodiment of the invention. The embodiment of the invention changes the position of the illumination light beam focused on the application scene of the sample by changing the angle of each plane reflector in the plane reflector array. As shown in the figure, the illumination beam is irradiated on the focal plane 1 of the sample and on the focal plane 2 of the sample, and the radian and angle of the plane mirror array forming the illumination beam are different.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. A confocal imaging apparatus, comprising: the device comprises a light source unit (11), a spectroscope (12), a polarizing plate (13), a plane reflector array (14) and an imaging unit (15), wherein the plane reflector array (14) is obtained by splicing a plurality of rotatable plane reflectors, no gap exists between every two plane reflectors, the spliced plane reflector array (14) has a set radian, and the confocal plane of the device is pushed by adjusting the plane reflector array (14);
white light projected by the light source unit (11) is transmitted to the polarizing plate (13) by the spectroscope (12); the polaroid (13) adjusts the polarization information of the projection light, and the adjusted white light forms an illumination light beam and focuses on the focus plane of the sample after being reflected by the plane mirror array (14);
the focus plane of the sample reflects the illumination light beam, and the imaging unit (15) acquires a sample focus plane image after the reflected illumination light beam is reflected by the plane mirror array (14), transmitted by the polaroid (13) and reflected by the spectroscope (12) in sequence;
the white light projected by the light source unit (11) is coded by the set projection image and then received by the spectroscope (12);
the polaroid (13) adjusts the polarization information of the projection light into horizontal polarization and vertical polarization;
the set projection image includes: calculating depth distribution information of the sample application scene according to the adjustment information of the horizontal polarization and the vertical polarization of the polarizing plate (13), and setting a projection image according to the depth distribution information of the sample application scene;
the set projection image is adjusted or updated according to the application scene change of the sample.
2. The apparatus according to claim 1, wherein the light source unit (11) is a projector;
the imaging unit (15) is a Charge Coupled Device (CCD) plane provided for a camera.
3. The apparatus according to claim 1, wherein the angle and the set arc of the plane mirror array (14) are automatically adjusted.
4. The device according to claim 1, characterized in that the beam splitter (12) is a 50:50 transmission ratio beam splitter plate arranged at 45 degrees to the projection axis of the device.
5. The apparatus according to claim 1, wherein the position of the light source unit (11) is perpendicular to the position of the imaging unit (15), enabling coaxial imaging.
6. The apparatus of claim 1, further comprising a computer, wherein the imaging unit (15) acquires an image of the sample focal plane and outputs the image to the computer for processing.
7. The apparatus of claim 1, wherein the application scenario of the apparatus is underwater.
8. A confocal imaging method, wherein the confocal imaging apparatus of claim 1 is provided, the method comprising:
white light projected by a light source unit in the confocal imaging device is transmitted to a polarizer in the confocal imaging device by a spectroscope in the confocal imaging device;
after the polarizing film in the confocal imaging device adjusts the polarization information of the projection light, the adjusted white light is reflected by a plane reflector array in the confocal imaging device to form an illumination beam to be focused on a focusing surface of a sample, and the plane reflector array realizes the transition of a confocal plane of the confocal imaging device through adjustment;
the focus plane of the sample reflects the illumination light beam, and the reflected illumination light beam sequentially passes through the reflection of the plane mirror array, the transmission of the polaroid and the reflection of the spectroscope, and then the imaging unit acquires a sample focus plane image;
before the white light is transmitted to a polarizer in the confocal imaging device by a spectroscope in the confocal imaging device, the method further comprises the following steps: the projected white light is encoded according to a set projection image and then received by the spectroscope;
the adjustment of the polarization information of the projection light is the adjustment of horizontal polarization and vertical polarization;
the set projection image includes: calculating depth distribution information of the sample application scene according to the adjustment information of the horizontal polarization and the vertical polarization of the projection light, and setting a projection image according to the depth distribution information of the sample application scene;
the set projection image is adjusted or updated according to the application scene change of the sample.
9. The method of claim 8, wherein the plane mirror array is obtained by splicing a plurality of rotatable plane mirrors, the plane mirrors have no gap between each two and the spliced plane mirror array has a set arc, and the angle and the set arc of the plane mirror array are controlled and adjusted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910476670.3A CN110196232B (en) | 2019-06-03 | 2019-06-03 | Confocal imaging device and confocal imaging method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910476670.3A CN110196232B (en) | 2019-06-03 | 2019-06-03 | Confocal imaging device and confocal imaging method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110196232A CN110196232A (en) | 2019-09-03 |
| CN110196232B true CN110196232B (en) | 2021-11-26 |
Family
ID=67753795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910476670.3A Expired - Fee Related CN110196232B (en) | 2019-06-03 | 2019-06-03 | Confocal imaging device and confocal imaging method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110196232B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826423B (en) * | 2019-10-18 | 2022-10-04 | 中北大学 | Method, device and system for detecting interested target in group target |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101750754B (en) * | 2008-12-17 | 2011-05-25 | 中国科学院西安光学精密机械研究所 | Field cut-off optical synthesis aperture imaging system |
| CN101482706A (en) * | 2009-01-23 | 2009-07-15 | 上海微电子装备有限公司 | Focusing leveling device based on confocal measuring technique |
| US8237835B1 (en) * | 2011-05-19 | 2012-08-07 | Aeon Imaging, LLC | Confocal imaging device using spatially modulated illumination with electronic rolling shutter detection |
| US9386222B2 (en) * | 2014-06-20 | 2016-07-05 | Qualcomm Incorporated | Multi-camera system using folded optics free from parallax artifacts |
| CN107343130B (en) * | 2017-08-17 | 2019-07-05 | 中国人民解放军国防科技大学 | High dynamic imaging module based on DMD dynamic light splitting |
-
2019
- 2019-06-03 CN CN201910476670.3A patent/CN110196232B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN110196232A (en) | 2019-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105807412B (en) | A kind of total internal reflection microscopic method and device based on free form surface shaping | |
| CN108982452B (en) | Multi-focus scanning three-dimensional imaging method and system based on double-spiral point spread function | |
| US7977625B2 (en) | Method and assembly for optical reproduction with depth discrimination | |
| US5043570A (en) | High resolution microscoping system using convolution integration process | |
| CN108718406B (en) | Variable-focus 3D depth camera and imaging method thereof | |
| JP2003121749A (en) | Microscope | |
| KR100958478B1 (en) | Projection display | |
| CN106226895B (en) | A kind of the rotation total internal reflection microscopic method and device of band feedback | |
| CN110320654A (en) | Based on the microscopical quick three-dimensional body imaging system of multi-angle 4Pi and method | |
| CN108292084A (en) | Image projection device | |
| CN110196232B (en) | Confocal imaging device and confocal imaging method | |
| TWI452453B (en) | System of forming color holographic image | |
| JPH10513287A (en) | Method and apparatus for recording and imaging images of objects, images | |
| TWI452451B (en) | System of forming holographic image | |
| KR101121673B1 (en) | Projection display apparatus | |
| CN106647045B (en) | Light-operated orientation device and method for liquid crystal selected area | |
| US20200358946A1 (en) | Methods and Systems for Single Frame Autofocusing Based on Color-Multiplexed Illumination | |
| CN110596996B (en) | Projector and automatic geometric correction method | |
| CN110230995A (en) | A kind of area-of-interest imaging device based on ghost imaging | |
| CN114047619B (en) | Three-dimensional microscopic imaging method and imaging light path structure thereof | |
| CN214066973U (en) | Large-depth-of-field display screen defect detection device based on multi-camera structure | |
| CN111736162B (en) | Laser illumination echo detection device and method for complex target | |
| CN109936692B (en) | Through scattering medium image acquisition system and method based on epipolar constraint | |
| US10841564B2 (en) | Three dimensional scan system that can increase the effective scan depth | |
| CN114442297A (en) | Microscope and method for light field microscopy and confocal microscopy with light sheet excitation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20211126 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |