CN107024275B - Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time - Google Patents

Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time Download PDF

Info

Publication number
CN107024275B
CN107024275B CN201710217823.3A CN201710217823A CN107024275B CN 107024275 B CN107024275 B CN 107024275B CN 201710217823 A CN201710217823 A CN 201710217823A CN 107024275 B CN107024275 B CN 107024275B
Authority
CN
China
Prior art keywords
spectral coverage
polarization
array
positioning
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710217823.3A
Other languages
Chinese (zh)
Other versions
CN107024275A (en
Inventor
刘智颖
朱启凡
付跃刚
丁秋伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changchun University of Science and Technology
Original Assignee
Changchun University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changchun University of Science and Technology filed Critical Changchun University of Science and Technology
Priority to CN201710217823.3A priority Critical patent/CN107024275B/en
Publication of CN107024275A publication Critical patent/CN107024275A/en
Application granted granted Critical
Publication of CN107024275B publication Critical patent/CN107024275B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J4/00Measuring polarisation of light
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J1/00Photometry, e.g. photographic exposure meter
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J4/00Measuring polarisation of light
    • G01J4/04Polarimeters using electric detection means

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Spectrometry And Color Measurement (AREA)

Abstract

More spectral coverage polarization imaging detection devices belong to the optical detector technology field of target to imitative mantis shrimp eye multichannel in real time.Existing detection device is loosely organized, volume is big, using flexible is low.Change with signal and be connected with process part by fibre bundle in signal detection part of the present invention;Focal imaging array, spectral coverage polarization arrays, positioning coupling array, positioning output array are identical with arrangement mode with the quantity of the array element of photoelectric conversion array, position correspondence;Spectral coverage polarization arrays substrate it is positive it is each array element is coated with spectral coverage filter membrane, each array element at the back side scribbles gold nanorods masking liquid polarizing coating, in each polarizing coating, field word is distributed 0 °, 45 °, 90 ° and 135 ° channel of polarization;Each spectral coverage λ i of collection target polarization image is transmitted same computer by photoelectric conversion array, it is four that each spectral coverage λ i target polarization image is distributed into one point by field word by computer, obtains each spectral coverage λ i of target 0 °, 45 °, 90 °, 135 ° of printing opacity direction polarization images.

Description

Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time
Technical field
The present invention relates to a kind of imitative mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time, belong to the optics of target Detection technology field.
Background technology
The reflected light of body surface in nature all has the spectral characteristic and polarization characteristic of oneself.For detection, The polarization information of target, such as the angle of polarization and degree of polarization, turn into reflection target effective information.Mantis shrimp eye has complicated structure, Compound eye is formed by thousands of individual ommatidiums, these ommatidiums are divided into three veutro, dorsal part and midband regions.Mantis shrimp eye can efficiently, it is accurate Really identification polarised light, because thus different ommatidiums produce antagonism to the polarization sensitive of different directions.Veutro and the back of the body Side ommatidium is respectively to orthogonal polarization sensitive;5 rows, 6 row's ommatidiums in midband to the row of being parallel and perpendicular to it is inclined Shake photaesthesia.Described ommatidium is made up of cornea and rhabdom.The rhabdom is in strip and is connected on retina, as Light guide will be as being presented on retina.
The polarization detecting device of imitative mantis shrimp eye rhabdom structure design has polarization imaging, polarization identification and degree of polarization meter Calculation ability, it is possible to increase target detection and identification probability and precision, the volume of detection device can reduce.
Application publication number be CN103900696A, it is entitled " a kind of imitative squill vision polarize antagonism perceive it is underwater In one Chinese invention patent application of polarization imaging method ", by gathering antagonism to polarization signal, obtain degree of polarization parameter and Light intensity parameter is synthesized, is achieved in submarine target imaging.The program can improve submarine target imaging detection precision.One by Springer Berlin Heidelberg in 2016 publish title be《Multi-band Polarization Imaging and Applications》Publication in include an entitled " Bio-inspired Multi-band Polarization Imaging " article, disclose a kind of more spectral coverage polarization vision systems of big visual field of imitative mantis shrimp eye, such as Fig. 1 Shown, the system is made up of 9 cameras by 3 × 3 array arrangements, and 5 cameras positioned at array corner and center belong to spectrum and led to Road, spectral matching factor is realized, other 4 cameras are the channel of polarization in 4 directions, and 4 directions are respectively 0 °, 45 °, 90 °, 135 ° Four printing opacity directions, the image input computer that 9 cameras obtain, polarization image and spectrum through the big visual field of image procossing acquisition Image.
For the prior art in terms of spectral coverage selected section and channel of polarization, the two is relatively independent, and the volume of device is larger, Flexibility is poor in use.
The content of the invention
It is an object of the present invention to improve the degree of integration of polarization imaging detection device so that compact-sized, volume subtracts It is small, using flexible is improved, therefore, more spectral coverage polarization imaging detections fill in real time we have invented a kind of imitative mantis shrimp eye multichannel Put.
The imitative mantis shrimp eye multichannel of the present invention in real time more spectral coverage polarization imaging detection devices characterized in that, as shown in Fig. 2 Change with signal and be connected with process part by fibre bundle 1 in signal detection part;In signal detection part, focal imaging array, Spectral coverage polarization arrays and positioning coupling array are arranged in order, in signal conversion with process part, positioning output array and photoelectricity Conversion array is arranged in order;The focal imaging array, spectral coverage polarization arrays, positioning coupling array, positioning output array and light The quantity of the array element of electric conversion array is identical with arrangement mode, position correspondence;In each battle array of focal imaging array In column unit, field word is distributed 4 imaging lens 2, as shown in Figure 3;In positive each array of the substrate 3 of spectral coverage polarization arrays Unit is coated with spectral coverage filter membrane 4, as shown in figure 4, each spectral coverage filter membrane 4 has respective transmission spectral coverage λ i, as shown in Figure 5;In spectral coverage Each array element at the back side of polarization arrays substrate 3 scribbles gold nanorods masking liquid polarizing coating 5, as shown in figure 4, by gold nano The draw ratio of rod determines the polarization spectral coverage λ i of each polarizing coating 5, as shown in fig. 6, in each polarizing coating 5, the distribution of field word 0 °, 45 °, 90 ° and 135 ° channel of polarization, the division of channel of polarization is determined by the brushing direction of gold nanorods masking liquid;Each spectral coverage Filter membrane 4 corresponds through the polarization spectral coverage λ i of spectral coverage λ i and each polarizing coating 5;In the coupling positioning of positioning coupling array Each array element of plate 6 is provided with 4 taper holes 7 of field word distribution, as shown in fig. 7, taper hole 7 is open towards the polarizing coating 5, The input of optical fiber 8 is fixed by the positioning hole on coupling location-plate 6, and is docked with the cone angle of taper hole 7, as shown in Figure 8, Figure 9;Positioning 4 beam-expanding collimation mirrors 10 of each array element arrangement field word distribution of the output location-plate 9 of output array, as shown in Figure 10, The output end of optical fiber 8 is fixed by the positioning hole on output location-plate 9, and towards beam-expanding collimation mirror 10, as shown in Fig. 8, Figure 11;Connection The boundling of the optical fiber 8 of every group of taper hole 7 and beam-expanding collimation mirror 10 is fibre bundle 1;In each array element of photoelectric conversion array One electrooptical device 11 is set, as shown in figure 12, each electrooptical device 11 connects same computer, as shown in Fig. 2 Each electrooptical device 11 gathers spectral coverage λ i target polarization image, by computer that each spectral coverage λ i target is inclined It is four that the image that shakes is distributed one point by field word, as shown in figure 13, obtains each spectral coverage λ i of target 0 °, 45 °, 90 °, 135 ° of printing opacities Direction polarization image.
The present invention it has technical effect that, some imaging lens 2 in focal imaging array imitate the ommatidium in mantis shrimp eye Cornea, complete target imaging.On the one hand spectral coverage polarization arrays imitate the characteristic that mantis shrimp eye difference ommatidium identifies different spectrum, in the future Spectral coverage division is carried out from the optical signal of same target, so as to realize more spectral coverage polarization imaging detections in real time, on the other hand imitates shrimp More channel of polarization structures of mqb eye, the optical signal from same target different spectral coverage λ i is subjected to polarization state division, so as to visit Measure each spectral coverage λ i 0 °, 45 °, 90 °, 135 ° of printing opacity direction polarization images.Spectral coverage polarization arrays in same substrate 3 just Anti- two sides makes spectral coverage filter membrane 4 and polarizing coating 5, greatly improves the degree of integration of detection device so that detection device structure is more Compact, volume is obviously reduced.Signal detection part and signal conversion and process part, signal conversion and place are connected by fibre bundle 1 Reason part is fixed, and there is necessary length of fibre beam 1 make it that the locus of signal detection part can be arbitrarily variable, also, The use of coupling array warp and positioning output array is positioned, no matter how the locus of signal detection part changes, is detected To target optical signal as excellent as before can send photoelectric conversion array to, this causes the using flexible of detection device significantly Improve.
Brief description of the drawings
Fig. 1 is the more spectral coverage polarization vision system schematics of big visual field of existing imitative mantis shrimp eye.
Fig. 2 is imitative mantis shrimp eye multichannel more spectral coverage polarization imaging detection device overall structure diagrams in real time of the present invention, The figure is used as Figure of abstract simultaneously.
Fig. 3 is the focal imaging array structure schematic diagram in the detection device of the present invention.
Fig. 4 is the spectral coverage polarization arrays structural representation in the detection device of the present invention.
Fig. 5 is the spectral coverage filter membrane dividing elements schematic diagram in the spectral coverage polarization arrays in the detection device of the present invention.
Fig. 6 is polarizing coating dividing elements and channel of polarization stroke in the spectral coverage polarization arrays in the detection device of the present invention Divide schematic diagram.
Fig. 7 is the positioning coupling array structural representation in the detection device of the present invention.
Fig. 8 is taper hole, optical fiber and beam-expanding collimation mirror three's structural relation schematic diagram in the detection device of the present invention.
Fig. 9 is taper hole and optical fiber structure relation partial enlargement schematic cross-sectional view in the detection device of the present invention.
Figure 10 is the positioning output array structural representation in the detection device of the present invention.
Figure 11 is optical fiber and beam-expanding collimation mirror structural relation partial enlargement schematic cross-sectional view in the detection device of the present invention.
Figure 12 is the photoelectric conversion array structural representation in the detection device of the present invention.
Figure 13 is that the computer in the detection device of the present invention is divided each spectral coverage λ i target polarization image by field word Cloth one divides for four dividing condition schematic diagrames.
Embodiment
The present invention's imitates mantis shrimp eye multichannel more following institutes of its embodiment of spectral coverage polarization imaging detection device in real time State.
It is connected as shown in Fig. 2 signal detection part is changed with signal with process part by fibre bundle 1.
In signal detection part, focal imaging array, spectral coverage polarization arrays and positioning coupling array are arranged in order, and are being believed Number conversion is arranged in order with process part, positioning output array and photoelectric conversion array.
The focal imaging array, spectral coverage polarization arrays, positioning coupling array, positioning output array and photoelectric conversion array Array element quantity is identical with arrangement mode, position correspondence;Array element arrangement mode include 2 × 2,2 × 3 or 3×3。
In each array element of focal imaging array, field word is distributed 4 imaging lens 2, as shown in Figure 3.
Spectral coverage filter membrane 4 is coated with positive each array element of the substrate 3 of spectral coverage polarization arrays, as shown in figure 4, each Spectral coverage filter membrane 4 has respective transmission spectral coverage λ i, as shown in figure 5, the spectral coverage number through spectral coverage λ i is 4,6 or 9.
Each array element at the back side of spectral coverage polarization arrays substrate 3 scribbles gold nanorods masking liquid polarizing coating 5, such as schemes Shown in 4, using gold nanorods draw ratio and the corresponding relation through spectral coverage λ i extinction ratios, determined respectively by the draw ratio of gold nanorods The polarization spectral coverage λ i of individual polarizing coating 5, as shown in fig. 6, in each polarizing coating 5, field word is distributed 0 °, 45 °, 90 ° and 135 ° Channel of polarization, the division of channel of polarization is determined by the brushing direction of gold nanorods masking liquid;Substrate 3 is each transparent through spectral coverage λ i Planar opticses glass, such as K9 trade mark optical glass;Draw ratio about gold nanorods polarizes spectral coverage λ i pass with polarizing coating 5 One concrete scheme of system see the table below.
Each spectral coverage filter membrane 4 corresponds through the polarization spectral coverage λ i of spectral coverage λ i and each polarizing coating 5.
4 taper holes 7 of field word distribution are provided with each array element of the coupling location-plate 6 of positioning coupling array, such as Fig. 7 Shown, taper hole 7 is open to be fixed towards the polarizing coating 5, the input of optical fiber 8 by the positioning hole on coupling location-plate 6, and with cone The cone angle docking of hole 7, as shown in Figure 8, Figure 9, taper hole 7 is used as a light funnel, can farthest realize incident light and optical fiber Coupling.
In 4 beam-expanding collimation mirrors of each array element arrangement field word distribution of the output location-plate 9 of positioning output array 10, as shown in Figure 10, the output end of optical fiber 8 is fixed by the positioning hole on output location-plate 9, and towards beam-expanding collimation mirror 10, is such as schemed 8th, shown in Figure 11.
The boundling for connecting the optical fiber 8 of every group of taper hole 7 and beam-expanding collimation mirror 10 is fibre bundle 1;Optical fiber 8 is polarization maintaining optical fibre.
One electrooptical device 11 is set in each array element of photoelectric conversion array, it is as shown in figure 12, each Electrooptical device 11 connects same computer, as shown in Fig. 2 each electrooptical device 11 gathers spectral coverage λ i target Polarization image, it is four that each spectral coverage λ i target polarization image is distributed into one point by field word by computer, as shown in figure 13, is obtained 0 °, 45 °, 90 °, 135 ° of printing opacity direction polarization images to each spectral coverage λ i of target;0 ° of each spectral coverage λ i, 45 °, 90 °, 135 ° of printing opacity direction polarization image intensity are followed successively by I0°,λi、I45°,λi、I90°,λi、I135°,λi, according to Stokes vector method, each Spectral coverage λ i polarization image strength component is:
The degree of polarization P of polarization imageλiWith angle of polarization αλiRespectively:
Each spectral coverage λ i polarization image intensity IλiFor:
The photosurface of electrooptical device 11 in each array element of photoelectric conversion array is made up of n × m pixel, As shown in figure 13, the target polarization image for sending the spectral coverage λ i of computer to by electrooptical device 11 is then by n × m pixel The analog image of composition, with computer program image processing techniques, by computer with 2 points of pixel (n/2,0), (n/2, m) Line, and pixel (0, m/2), the mode of (n, m/2) 2 lines, field is pressed by each spectral coverage λ i target polarization image It is four that word, which is distributed one point,.

Claims (4)

  1. A kind of 1. imitative mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time, it is characterised in that signal detection part with Signal conversion is connected with process part by fibre bundle (1);In signal detection part, focal imaging array, spectral coverage polarization arrays It is arranged in order with positioning coupling array, in signal conversion with process part, positioning output array and photoelectric conversion array successively Arrangement;The focal imaging array, spectral coverage polarization arrays, position coupling array, positioning output array and photoelectric conversion array The quantity of array element is identical with arrangement mode, position correspondence;In each array element of focal imaging array, field word It is distributed 4 imaging lens (2);Spectral coverage filter membrane is coated with positive each array element of the substrate (3) of spectral coverage polarization arrays (4), each spectral coverage filter membrane (4) has respective through spectral coverage λ i;Each array list at the back side of spectral coverage polarization arrays substrate (3) Member scribbles gold nanorods masking liquid polarizing coating (5), and the polarization spectral coverage of each polarizing coating (5) is determined by the draw ratio of gold nanorods λ i, in each polarizing coating (5), field word is distributed 0 °, 45 °, 90 ° and 135 ° channel of polarization, by the brushing of gold nanorods masking liquid Direction determines the division of channel of polarization;The polarization spectral coverage through spectral coverage λ i and each polarizing coating (5) of each spectral coverage filter membrane (4) λ i are corresponded;4 taper holes of field word distribution are provided with each array element of the coupling location-plate (6) of positioning coupling array (7), taper hole (7) opening is consolidated towards the polarizing coating (5), optical fiber (8) input by the positioning hole on coupling location-plate (6) It is fixed, and docked with taper hole (7) cone angle;In each array element arrangement field word point of the output location-plate (9) of positioning output array 4 beam-expanding collimation mirrors (10) of cloth, optical fiber (8) output end are fixed by the positioning hole on output location-plate (9), and direction expands standard Straight mirror (10);The boundling for connecting the optical fiber (8) of every group of taper hole (7) and beam-expanding collimation mirror (10) is fibre bundle (1);In opto-electronic conversion One electrooptical device (11) is set in each array element of array, and each electrooptical device (11) connects same calculating Machine, each electrooptical device (11) gathers spectral coverage λ i target polarization image, by computer by each spectral coverage λ i's It is four that target polarization image is distributed one point by field word, obtains each spectral coverage λ i of target 0 °, 45 °, 90 °, 135 ° of printing opacity directions Polarization image.
  2. 2. imitative mantis shrimp eye multichannel according to claim 1 more spectral coverage polarization imaging detection devices in real time, it is characterised in that Array element arrangement mode includes 2 × 2,2 × 3 or 3 × 3;Spectral coverage number through spectral coverage λ i is 4,6 or 9.
  3. 3. imitative mantis shrimp eye multichannel according to claim 1 more spectral coverage polarization imaging detection devices in real time, it is characterised in that Substrate (3) is each planar opticses glass transparent through spectral coverage λ i.
  4. 4. imitative mantis shrimp eye multichannel according to claim 1 more spectral coverage polarization imaging detection devices in real time, it is characterised in that Optical fiber (8) is polarization maintaining optical fibre.
CN201710217823.3A 2017-04-05 2017-04-05 Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time Active CN107024275B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710217823.3A CN107024275B (en) 2017-04-05 2017-04-05 Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710217823.3A CN107024275B (en) 2017-04-05 2017-04-05 Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time

Publications (2)

Publication Number Publication Date
CN107024275A CN107024275A (en) 2017-08-08
CN107024275B true CN107024275B (en) 2018-04-06

Family

ID=59526634

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710217823.3A Active CN107024275B (en) 2017-04-05 2017-04-05 Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time

Country Status (1)

Country Link
CN (1) CN107024275B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107741274B (en) * 2017-10-19 2023-12-08 中国科学院西安光学精密机械研究所 Miniature polarization spectrum imaging detection system and method
WO2021035090A1 (en) * 2019-08-21 2021-02-25 Arizona Board Of Regents On Behalf Of The University Of Arizona Reconfigurable polarization imaging system
CN110849479A (en) * 2019-10-28 2020-02-28 西北工业大学 Array type polarization color composite filtering imaging method
CN111256820A (en) * 2020-01-10 2020-06-09 南京理工大学 Polarization multiplexing-based transient multispectral imaging device and method
CN111735539A (en) * 2020-06-29 2020-10-02 中国科学院西安光学精密机械研究所 Spectrum imaging detection device based on polarization synchronous modulation and adjustable filtering

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103900696B (en) * 2014-03-07 2015-09-30 河海大学 The method of polarization imaging under water of a kind of imitative squill vision polarization antagonism perception
CN106191796B (en) * 2016-07-07 2018-08-14 首都师范大学 A kind of color printing method of ultra-thin semiconductor nano coating additional polarization information
CN106504222B (en) * 2016-11-21 2019-09-06 河海大学常州校区 A kind of underwater Polarization Image Fusion system based on bionic visual mechanism

Also Published As

Publication number Publication date
CN107024275A (en) 2017-08-08

Similar Documents

Publication Publication Date Title
CN107024275B (en) Imitate mantis shrimp eye multichannel more spectral coverage polarization imaging detection devices in real time
CN1156795C (en) Image processing method for image transmission system and its optical fibre endoscope
DE102006031114B4 (en) 3D combination meter from digital camera and laser scanner
US11163169B2 (en) Endoscope and imaging arrangement providing improved depth of field and resolution
CN211824967U (en) System for measuring virtual image distance of binocular near-eye display
US11965775B2 (en) Method and apparatus for focus correction of multi-image laser beam quality measurements
CN103271721A (en) Method and system for detecting parallel OCT based on spectrum coding and orthogonal light splitting
DE102017100904A1 (en) Image conversion module for a microscope and microscope
EP4068747A1 (en) Compound eye camera device and compound eye system
JP2020076717A (en) Optical inspection device and optical inspection method
CN107843343A (en) Polarization multi-spectrum aerial surveying camera based on monolithic face battle array cmos detector
CN106989822B (en) Imitate the optional polarization state detection device of mantis shrimp eye spectral coverage and method
US11092489B2 (en) Wide-angle computational imaging spectroscopy method and apparatus
CN106597422B (en) Miniaturization photo-electricity passive ranging device
CN103558160A (en) Method and system for improving resolution ratio of spectral imaging space
WO2019223090A1 (en) Intelligent mixed reality telescope
CN206038152U (en) Polarized interference spectral imaging system
CN108955640A (en) The restructural remote sensing camera in space based on synthetic aperture technique
US11422264B2 (en) Optical remote sensing
CN108937842A (en) A kind of OCT system integrating confocal fundus imaging
WO2021149916A1 (en) Machine vision inspection device using multi-line beam
WO2022167393A1 (en) Measuring apparatus and method for measuring a modulation transfer function of an afocal optical system
CN208076758U (en) High-power optical fiber laser receiving and transmitting integrated end cap
JPH01197716A (en) Enlargement photographing mechanism for electronic endoscope
CN110274689A (en) A kind of EO-1 hyperion camera based on microlens array

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant