CN113727028A - Modular night vision imaging camera - Google Patents

Modular night vision imaging camera Download PDF

Info

Publication number
CN113727028A
CN113727028A CN202111032853.XA CN202111032853A CN113727028A CN 113727028 A CN113727028 A CN 113727028A CN 202111032853 A CN202111032853 A CN 202111032853A CN 113727028 A CN113727028 A CN 113727028A
Authority
CN
China
Prior art keywords
image
module
imaging
output
interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111032853.XA
Other languages
Chinese (zh)
Other versions
CN113727028B (en
Inventor
刘阳
齐朝祥
侯进永
臧晴
魏斌斌
韩兴斌
刘春恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
32802 Troops Of People's Liberation Army Of China
Original Assignee
32802 Troops Of People's Liberation Army Of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 32802 Troops Of People's Liberation Army Of China filed Critical 32802 Troops Of People's Liberation Army Of China
Priority to CN202111032853.XA priority Critical patent/CN113727028B/en
Publication of CN113727028A publication Critical patent/CN113727028A/en
Application granted granted Critical
Publication of CN113727028B publication Critical patent/CN113727028B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95Computational photography systems, e.g. light-field imaging systems
    • H04N23/951Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
    • G06T5/80
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise

Abstract

The invention discloses a modularized night vision imaging camera, which comprises a plurality of identical imaging modules, a central image processing unit and a synchronous control module, wherein each imaging module comprises a lens, an image sensor, a synchronous interface, a distortion correction module and an image output module, the imaging modules adopt a plurality of imaging modules for shooting, photons of each imaging module are converted into electric charges and are converted into digital signals, and the digital signals of each channel are added to obtain enhanced signals. The distortion correction module corrects the digital image signals in real time so as to eliminate image inconsistency caused by the difference of lens distortion among the imaging modules. The invention continuously improves the detection capability of the imaging camera by increasing the number of units, avoids the risk that the multiplication technology may cause the burning of the camera under stronger light, and breaks through the limit caused by photon noise in a multiplication type low-light-level shooting system.

Description

Modular night vision imaging camera
Technical Field
The invention belongs to the technical field of photoelectric imaging, and particularly relates to a modularized night vision imaging camera.
Background
The existing low-light night vision cameras are divided into an image enhancement type low-light night vision camera and a charge multiplication type low-light night vision camera. The image enhancement type low-light night vision camera multiplies photons collected by an optical system through a microchannel plate, and then couples the multiplied photons to an image sensor to realize the enhancement of light intensity signals. The charge multiplication type low-light night vision camera firstly converts light into charges and then completes the multiplication of the charges on an image sensor, thereby realizing the enhancement of signals.
The existing low-light night vision camera has some limitations in use, including that light is too strong and cannot work in the daytime, the existing low-light night vision camera needs to be combined with a conventional sensor of a visible light wave band for use, meanwhile, the weak light detection capability of the existing low-light night vision camera is limited, and the noise generated by photons is an important factor limiting the signal-to-noise ratio of a system under the condition of less photon number. For example, for N photons, the self-generated photon noise is sqr (N). As the number of photons approaches a single photon, the signal-to-noise ratio will tend to 1. In the case of multiplication-type micro-light photographing systems, the photographing limit is the photographing of a single photon. Furthermore, the application of multiplication techniques may also lead to a risk of burning out the camera under strong light.
Disclosure of Invention
Aiming at the problems faced by the existing low-light night vision camera technology, the invention discloses a modular night vision imaging camera which adopts a plurality of shooting units for shooting, converts photons of each shooting unit into electric charges and then into digital signals, and adds the digital signals of each shooting unit in a calculation mode to obtain enhanced signals.
The invention discloses a modularized night vision imaging camera, which comprises a plurality of identical imaging modules, a central image processing unit and a synchronous control module, wherein each imaging module is connected with the central image processing unit and the synchronous control module.
The imaging module comprises a lens, an image sensor, a synchronous interface, a distortion correction module and an image output module.
The image sensor is located on the focal plane of the lens, all the imaging modules are placed in parallel, and the axes of the lenses of all the imaging modules point to the same direction, so that the non-directional light rays emitted by the same target are received by the lens of each imaging module and are imaged on the focal plane of the lens. The image sensor at the focal plane converts the light received by the image sensor into electric charge and further converts the electric charge into a digital image signal, and the image sensor outputs the digital image signal to the distortion correction module from a data interface of the image sensor.
And a data interface of the image sensor is connected with a distortion correction module, and the distortion correction module is connected with an image output module. The distortion correction module is realized by a digital signal processor or a field programmable array, digital image signals acquired by the image sensor are corrected in real time by utilizing methods such as polynomial geometric correction and the like so as to eliminate the inconsistency of the acquired digital image signals caused by the difference of lens distortion among the imaging modules, and the distortion correction module sends the digital image signals after real-time correction to the image output module.
The image output module is used for packaging the digital image signals after real-time correction and sending the packaged images to the central image processing unit. The image output module is realized by a USB transmitter or an Ethernet transmitter.
The image sensor is connected with a synchronous interface, the synchronous interface is used for receiving a trigger signal or a line-field synchronous signal of the image sensor, the synchronous interfaces of all the imaging modules are connected with the synchronous control module and are uniformly controlled by the synchronous control module, and synchronous shooting of all the imaging modules is achieved.
The central image processing unit is realized by a field programmable array or a GPU processor and comprises an image registration unit, an image superposition module and an image data interface. The image registration unit is connected with the image output module of each imaging module, and is used for carrying out image registration on the digital image signals output by the image output module of each imaging module and outputting the registered result to the image superposition module, and the image superposition module is connected with the image data interface.
The image registration unit performs image registration calculation by using image processing methods such as a reference star matching method or a corner matching method, and acquires registration information of each imaging module by extracting image features of a digital image signal output by an image output module of each imaging module to complete image registration.
The reference star matching method is characterized in that the star images shot by the imaging modules are subjected to feature recognition matching with the reference star images in the star catalogue, and the registration information of the imaging modules is obtained and comprises the translation amount and the rotation amount of digital image signals output by the image output module of each imaging module.
The corner matching method comprises the steps of firstly extracting corner features of digital image signals output by each imaging module, extracting corners, then extracting descriptors from the extracted corners, describing the corners by using mathematical features such as local random two-dimensional features, and the like, and then judging the corresponding relation of the corners in different images through the descriptors corresponding to the corners, so that registration information of each imaging module is obtained, wherein the registration information comprises the translation amount and the rotation amount of the digital image signals output by the image output module of each imaging module.
The method comprises the following steps of extracting the characteristic of an angular point of a digital image signal output by each imaging module, and extracting the angular point, and specifically comprises the following steps:
the gray value of a pixel point with coordinates (x, y) of the digital image under an image plane coordinate system is I (x, y), and the gradient I of the gray value in the directions of the x axis and the y axis is calculatedxAnd IyAnd gradient product
Figure BDA0003246033890000031
And IxIyAnd calculating a matrix M, wherein the calculation formula is as follows:
Figure BDA0003246033890000032
wherein, the two-dimensional gaussian function w (x, y) represents a window function of the pixel point (x, y). And calculating a Harris response value R corresponding to each pixel point, wherein the calculation formula is as follows:
R=det(M)-k(trace(M))2
where k is a constant term. And setting a discrimination threshold t to obtain a pixel point corresponding to the R value larger than the threshold t, namely the extracted corner point.
The image superposition module translates and rotates the images output by the image registration unit to realize image alignment, and then performs addition calculation on each translated and rotated image to obtain a sum image after alignment, wherein the sum image is used as the output of the image superposition module.
The addition calculation is to add the images after the translation and the rotation according to the addition principle of random noise, for N unit images output by the image registration unit, if the signal intensity of each unit image is s, the readout noise of each unit image is r, after the addition calculation, the signal intensity of the sum image output by the image superposition module is N & lts & gt, and the corresponding total readout noise is
Figure BDA0003246033890000033
Image signal-to-noise ratio improvement over that before superposition
Figure BDA0003246033890000034
And (4) doubling.
The image superposition module is connected with the image output interface, and the image output interface converts the image after the addition calculation into image data for output. The image output interface is an HDMI interface, or a USB interface, or a network interface.
The invention has the beneficial effects that:
the detection capability of the weak light is related to the number of units of the system, so that the detection capability can be continuously improved by increasing the number of units. The invention does not require multiplication techniques and therefore there is no risk that multiplication may lead to burnout under stronger light. The invention is not limited by photon noise. In the case of a small number of photons, the noise generated by the photons itself is an important factor limiting the signal-to-noise ratio of the system. For example, for N photons, the self-generated photon noise is sqr (N). As the number of photons approaches a single photon, the signal-to-noise ratio will tend to 1. For multiplication-type micro-light photographing systems, the photographing limit is the photographing of a single photon. The computational imaging mode of the invention utilizes a plurality of units to increase the capability of acquiring photons, which is equivalent to increase the total number of photons, so the mode can break through the theoretical limit caused by photon noise in a multiplication type micro-light shooting system.
Drawings
Fig. 1 is a block diagram of the components of the modular night vision imaging camera of the present invention.
Detailed Description
For a better understanding of the present disclosure, an example is given here.
The invention discloses a modularized night vision imaging camera, which comprises a plurality of identical imaging modules, a central image processing unit and a synchronous control module, wherein each imaging module is connected with the central image processing unit and the synchronous control module, and the system structure is shown as the attached figure 1.
The imaging module comprises a lens, an image sensor, a synchronous interface, a distortion correction module and an image output module.
The image sensor is located on the focal plane of the lens, all the imaging modules are placed in parallel, and the axes of the lenses of all the imaging modules point to the same direction, so that the non-directional light rays emitted by the same target are received by the lens of each imaging module and are imaged on the focal plane of the lens. The image sensor at the focal plane converts the light received by the image sensor into electric charge and further converts the electric charge into a digital image signal, and the image sensor outputs the digital image signal to the distortion correction module from a data interface of the image sensor.
And a data interface of the image sensor is connected with a distortion correction module, and the distortion correction module is connected with an image output module. The distortion correction module is realized by a digital signal processor or a field programmable array, digital image signals acquired by the image sensor are corrected in real time by utilizing methods such as polynomial geometric correction and the like so as to eliminate the inconsistency of the acquired digital image signals caused by the difference of lens distortion among the imaging modules, and the distortion correction module sends the digital image signals after real-time correction to the image output module.
The image output module is used for packaging the digital image signals after real-time correction and sending the packaged images to the central image processing unit. The image output module is realized by a USB transmitter or an Ethernet transmitter.
The image sensor is connected with a synchronous interface, the synchronous interface is used for receiving a trigger signal or a line-field synchronous signal of the image sensor, the synchronous interfaces of all the imaging modules are connected with the synchronous control module and are uniformly controlled by the synchronous control module, and synchronous shooting of all the imaging modules is achieved.
The central image processing unit is realized by a field programmable array or a GPU processor and comprises an image registration unit, an image superposition module and an image data interface. The image registration unit is connected with the image output module of each imaging module, and is used for carrying out image registration on the digital image signals output by the image output module of each imaging module and outputting the registered result to the image superposition module, and the image superposition module is connected with the image data interface.
The image registration unit performs image registration calculation by using image processing methods such as a reference star matching method or a corner matching method, and acquires registration information of each imaging module by extracting image features of a digital image signal output by an image output module of each imaging module to complete image registration.
The reference star matching method is characterized in that the star images shot by the imaging modules are subjected to feature recognition matching with the reference star images in the star catalogue, and the registration information of the imaging modules is obtained and comprises the translation amount and the rotation amount of digital image signals output by the image output module of each imaging module.
The corner matching method comprises the steps of firstly extracting corner features of digital image signals output by each imaging module, extracting corners, then extracting descriptors from the extracted corners, describing the corners by using mathematical features such as local random two-dimensional features, and the like, and then judging the corresponding relation of the corners in different images through the descriptors corresponding to the corners, so that registration information of each imaging module is obtained, wherein the registration information comprises the translation amount and the rotation amount of the digital image signals output by the image output module of each imaging module.
The method comprises the following steps of extracting the characteristic of an angular point of a digital image signal output by each imaging module, and extracting the angular point, and specifically comprises the following steps:
the gray value of a pixel point with coordinates (x, y) of the digital image under an image plane coordinate system is I (x, y), and the gradient I of the gray value in the directions of the x axis and the y axis is calculatedxAnd IyAnd gradient product
Figure BDA0003246033890000051
And IxIyAnd calculating a matrix M, wherein the calculation formula is as follows:
Figure BDA0003246033890000061
wherein, the two-dimensional gaussian function w (x, y) represents a window function of the pixel point (x, y). And calculating a Harris response value R corresponding to each pixel point, wherein the calculation formula is as follows:
R=det(M)-k(trace(M))2
wherein k is a constant term, the value of k is set according to the characteristics of the digital image, and the value range of k is usually between 0.04 and 0.06. And setting a discrimination threshold t to obtain a pixel point corresponding to the R value larger than the threshold t, namely the extracted corner point.
The image superposition module translates and rotates the images output by the image registration unit to realize image alignment, and then performs addition calculation on each translated and rotated image to obtain a sum image after alignment, wherein the sum image is used as the output of the image superposition module.
The addition calculation is to add the images after the translation and the rotation according to the addition principle of random noise, for N unit images output by the image registration unit, if the signal intensity of each unit image is s, the readout noise of each unit image is r, after the addition calculation, the signal intensity of the sum image output by the image superposition module is N & lts & gt, and the corresponding total readout noise is
Figure BDA0003246033890000062
The signal-to-noise ratio of the target image of each unit is s/r, the signal-to-noise ratio of the image superposition module output and the image target is N s/r SQR (N) s/r, and the improvement of the image signal-to-noise ratio is realized.
The image superposition module is connected with the image output interface, and the image output interface converts the image after the addition calculation into image data for output. The image output interface is an HDMI interface, or a USB interface, or a network interface.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (6)

1. A modularized night vision imaging camera is characterized by comprising a plurality of identical imaging modules, a central image processing unit and a synchronous control module, wherein each imaging module is connected with the central image processing unit and the synchronous control module;
the imaging module comprises a lens, an image sensor, a synchronous interface, a distortion correction module and an image output module;
the image sensor is positioned on the focal plane of the lens, all the imaging modules are placed in parallel, and the lens axes of all the imaging modules point to the same direction so as to ensure that the non-directional light rays emitted by the same target are received by the lens of each imaging module and imaged on the focal plane of the lens; the image sensor at the focal plane converts the light received by the image sensor into electric charge and further converts the electric charge into a digital image signal, and the image sensor outputs the digital image signal to the distortion correction module from a data interface of the image sensor.
2. The modular night vision imaging camera of claim 1,
the data interface of the image sensor is connected with the distortion correction module, and the distortion correction module is connected with the image output module; the distortion correction module is realized by a digital signal processor or a field programmable array, digital image signals acquired by the image sensor are corrected in real time by utilizing a polynomial geometric correction method so as to eliminate the inconsistency of the acquired digital image signals caused by the difference of lens distortion among the imaging modules, and the digital image signals after real-time correction are sent to the image output module by the distortion correction module;
the image output module is used for packaging the digital image signals corrected in real time and sending the packaged images to the central image processing unit; the image output module is realized by a USB transmitter or an Ethernet transmitter;
the image sensor is connected with a synchronous interface, the synchronous interface is used for receiving a trigger signal or a line-field synchronous signal of the image sensor, and the synchronous interfaces of all the imaging modules are connected with the synchronous control module and are uniformly controlled by the synchronous control module, so that synchronous shooting of all the imaging modules is realized;
the central image processing unit is realized by a field programmable array or a GPU processor and comprises an image registration unit, an image superposition module and an image data interface; the image registration unit is connected with the image output module of each imaging module, is used for carrying out image registration on the digital image signals output by the image output module of each imaging module and outputting the registered result to the image superposition module, and the image superposition module is connected with the image data interface;
the image registration unit performs image registration calculation by using a reference star matching method or a corner matching method, and acquires registration information of each imaging module by extracting image features of a digital image signal output by an image output module of each imaging module;
the image superposition module translates and rotates the images output by the image registration unit to realize image alignment, and then performs addition calculation on each translated and rotated image to obtain a sum image after alignment, wherein the sum image is used as the output of the image superposition module;
the image superposition module is connected with the image output interface, and the image output interface converts the image subjected to the addition calculation into image data for output; the image output interface is an HDMI interface, or a USB interface, or a network interface.
3. The modular night vision imaging camera of claim 2,
the reference star matching method is characterized in that the star images shot by the imaging modules are subjected to feature recognition matching with the reference star images in the star catalogue, and the registration information of the imaging modules is obtained and comprises the translation amount and the rotation amount of digital image signals output by the image output module of each imaging module.
4. The modular night vision imaging camera of claim 2,
the corner matching method comprises the steps of firstly extracting the feature of a digital image signal output by each imaging module, extracting corners, then extracting descriptors from the extracted corners, describing the corners by using local random two-dimensional features, and then judging the corresponding relation of the corners in different images by the descriptors corresponding to the corners, so as to obtain the registration information of each imaging module, including the translation amount and the rotation amount of the digital image signal output by the image output module of each imaging module.
5. The modular night vision imaging camera of claim 4,
the method comprises the following steps of extracting the characteristic of an angular point of a digital image signal output by each imaging module, and extracting the angular point, and specifically comprises the following steps:
the gray value of a pixel point with coordinates (x, y) of the digital image under an image plane coordinate system is I (x, y), and the gradient I of the gray value in the directions of the x axis and the y axis is calculatedxAnd IyAnd gradient product
Figure FDA0003246033880000021
And IxIyAnd calculating a matrix M, wherein the calculation formula is as follows:
Figure FDA0003246033880000022
wherein, the two-dimensional Gaussian function w (x, y) represents a window function of the pixel point (x, y); and calculating a Harris response value R corresponding to each pixel point, wherein the calculation formula is as follows:
R=det(M)-k(trace(M))2
wherein k is a constant term; and setting a discrimination threshold t to obtain a pixel point corresponding to the R value larger than the threshold t, namely the extracted corner point.
6. The modular night vision imaging camera of claim 2,
the addition calculation is to add the images after the translation and the rotation according to the addition principle of random noise, for N unit images output by the image registration unit, if the signal intensity of each unit image is s, the readout noise of each unit image is r, after the addition calculation, the signal intensity of the sum image output by the image superposition module is N & lts & gt, and the corresponding total readout noise is
Figure FDA0003246033880000031
CN202111032853.XA 2021-09-03 2021-09-03 Modular night vision imaging camera Active CN113727028B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111032853.XA CN113727028B (en) 2021-09-03 2021-09-03 Modular night vision imaging camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111032853.XA CN113727028B (en) 2021-09-03 2021-09-03 Modular night vision imaging camera

Publications (2)

Publication Number Publication Date
CN113727028A true CN113727028A (en) 2021-11-30
CN113727028B CN113727028B (en) 2022-03-25

Family

ID=78681533

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111032853.XA Active CN113727028B (en) 2021-09-03 2021-09-03 Modular night vision imaging camera

Country Status (1)

Country Link
CN (1) CN113727028B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105744137A (en) * 2016-04-22 2016-07-06 王俊懿 System for real-time correction and splicing of image formed by fish-eye lens through using hardware circuit
US20170176139A1 (en) * 2015-12-22 2017-06-22 Huntercraft Limited Infrared-light and low-light two-phase fusion night-vision sighting device
CN107203987A (en) * 2017-06-07 2017-09-26 云南师范大学 A kind of infrared image and low-light (level) image real time fusion system
CN108154494A (en) * 2017-12-25 2018-06-12 北京航空航天大学 A kind of image fusion system based on low-light and infrared sensor
CN110620885A (en) * 2019-10-18 2019-12-27 中国科学院深圳先进技术研究院 Infrared low-light-level image fusion system and method and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170176139A1 (en) * 2015-12-22 2017-06-22 Huntercraft Limited Infrared-light and low-light two-phase fusion night-vision sighting device
CN105744137A (en) * 2016-04-22 2016-07-06 王俊懿 System for real-time correction and splicing of image formed by fish-eye lens through using hardware circuit
CN107203987A (en) * 2017-06-07 2017-09-26 云南师范大学 A kind of infrared image and low-light (level) image real time fusion system
CN108154494A (en) * 2017-12-25 2018-06-12 北京航空航天大学 A kind of image fusion system based on low-light and infrared sensor
CN110620885A (en) * 2019-10-18 2019-12-27 中国科学院深圳先进技术研究院 Infrared low-light-level image fusion system and method and electronic equipment

Also Published As

Publication number Publication date
CN113727028B (en) 2022-03-25

Similar Documents

Publication Publication Date Title
CN111145238B (en) Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment
CN109003311B (en) Calibration method of fisheye lens
JP6087947B2 (en) Method for 3D reconstruction of scenes that rely on asynchronous sensors
CN106611430A (en) An RGB-D image generation method, apparatus and a video camera
US11940348B1 (en) System and method for detecting centroid of complementary single pixel
US11265530B2 (en) Stereoscopic camera
CN109559353A (en) Camera module scaling method, device, electronic equipment and computer readable storage medium
CN112004029A (en) Exposure processing method, exposure processing device, electronic apparatus, and computer-readable storage medium
CN109584312A (en) Camera calibration method, device, electronic equipment and computer readable storage medium
CN112700502A (en) Binocular camera system and binocular camera space calibration method
US11734877B2 (en) Method and device for restoring image obtained from array camera
CN114092647A (en) Three-dimensional reconstruction system and method based on panoramic binocular stereo vision
CN113727028B (en) Modular night vision imaging camera
CN113945921A (en) Multi-mode data acquisition system and synchronous acquisition method
CN109495694B (en) RGB-D-based environment sensing method and device
CN116580107A (en) Cross-view multi-target real-time track tracking method and system
CN115598744A (en) High-dimensional light field event camera based on micro-lens array and extraction method
CN109658459A (en) Camera calibration method, device, electronic equipment and computer readable storage medium
CN105628622A (en) Polarization imaging system based on three cameras
CN112308972A (en) Large-scale cable tunnel environment model reconstruction method
CN111445507A (en) Data processing method for non-visual field imaging
CN112396687A (en) Binocular stereoscopic vision three-dimensional reconstruction system and method based on infrared micro-polarizer array
CN204479182U (en) Low-light (level) multiband spectrum imager
US11448483B1 (en) Projectile tracking and 3D traceback method
EP3972244A1 (en) System and method for processing image related to depth

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant