CN116295873A - Pyramid wavefront sensor wavefront reconstruction method based on artificial neural network - Google Patents
Pyramid wavefront sensor wavefront reconstruction method based on artificial neural network Download PDFInfo
- Publication number
- CN116295873A CN116295873A CN202310404779.2A CN202310404779A CN116295873A CN 116295873 A CN116295873 A CN 116295873A CN 202310404779 A CN202310404779 A CN 202310404779A CN 116295873 A CN116295873 A CN 116295873A
- Authority
- CN
- China
- Prior art keywords
- wavefront
- neural network
- pyramid
- wavefront sensor
- artificial neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 13
- 238000003062 neural network model Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 238000010587 phase diagram Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 claims description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 210000001747 pupil Anatomy 0.000 claims description 2
- 238000005096 rolling process Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 abstract description 8
- 230000003287 optical effect Effects 0.000 abstract description 7
- 230000035945 sensitivity Effects 0.000 abstract description 5
- 238000013135 deep learning Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 abstract description 3
- 230000004075 alteration Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J9/00—Measuring optical phase difference; Determining degree of coherence; Measuring optical wavelength
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Photometry And Measurement Of Optical Pulse Characteristics (AREA)
Abstract
The invention relates to a pyramid wavefront sensor wavefront reconstruction method based on an artificial neural network. Compared with a shack-Hartmann wavefront sensor, the pyramid wavefront sensor has the advantages of adjustable sampling rate, high sensitivity, strong spatial aliasing robustness, suitability for weak light detection application and the like. The traditional pyramid wavefront sensor wavefront reconstruction algorithm has weak fitting nonlinearity, so that the reconstruction accuracy is limited. The kernel for deep learning is the capability of the artificial neural network to approach the nonlinear function with arbitrary precision, can effectively learn the physical and optical process, and is very suitable for solving the problem of reconstruction of pyramid nonlinear wave fronts. Based on the method, the invention provides a pyramid wavefront sensor wavefront reconstruction method based on an artificial neural network, and the improved neural network architecture based on U-Net is used for fitting the nonlinear input-output relationship of the pyramid wavefront sensor, so that the noise robustness and reconstruction precision can be effectively improved.
Description
Technical Field
The invention belongs to the field of optical technology wavefront sensing, and particularly relates to a pyramid wavefront sensor wavefront reconstruction method based on an artificial neural network, which is a pyramid wavefront sensor wavefront reconstruction method based on an improved U-Net neural network architecture.
Background
The large-caliber foundation telescope plays a significant role in astronomy development history, and can help astronomists to detect more remote stars and know the transition of universe. When the ground telescope is used for ground observation, the observation image is blurred due to the interference of atmospheric turbulence. The self-adaptive optical system can compensate wavefront distortion caused by atmospheric turbulence in real time and correct wavefront aberration. The wavefront sensor is mainly used for detecting the distortion information of the wavefront, and further processing and correcting the wavefront information, and the accuracy of the wavefront sensor is the basis of the work of the integral self-adaptive optical system.
Three important indexes of the wavefront sensor are the relation of sensitivity, dynamic range and input and output, respectively. A wavefront sensor with high sensitivity, wide dynamic range and linear input-output relationship is ideal, and can accurately sense weak phase change, measure larger aberration and reduce the influence of noise on the system. In recent years, the pyramid wavefront sensor has the unique advantages of adjustable sampling rate, high sensitivity, strong spatial aliasing robustness, suitability for weak light detection application and the like, and has wide development prospect.
Although the theory of wavefront reconstruction algorithm based on pyramid wavefront sensor is applied in practice and good results are obtained, the traditional wavefront reconstruction algorithm such as singular value decomposition, linear iteration and the like has weak fitting nonlinear capability. In the future, the deep learning of the great variety in the fields of computer vision, natural language processing and the like is realized, the kernel of the deep learning method is the capability of approximating a nonlinear function with arbitrary precision by an artificial neural network, programming according to specific physical rules is not needed, a physical optical process can be effectively learned, and the deep learning method is very suitable for solving the nonlinear wavefront reconstruction problem of a pyramid sensor.
Aiming at the problems, the invention provides a wavefront reconstruction method of a pyramid wavefront sensor based on an artificial neural network, which uses an improved neural network architecture based on U-Net to fit a nonlinear input-output relationship of the pyramid wavefront sensor, and realizes the wavefront reconstruction of three indexes of comprehensively improving the operation speed, the reconstruction precision and the noise robustness of the pyramid wavefront sensor so as to control the optimal shape of a deformable mirror.
Disclosure of Invention
In order to overcome the defects of the existing wavefront reconstruction technology, the invention provides a pyramid wavefront sensor wavefront reconstruction method based on an artificial neural network.
The invention adopts the technical scheme that: a pyramid wavefront sensor wavefront reconstruction method based on an artificial neural network realizes wavefront reconstruction by the following steps:
step 1: obtaining a simulated atmospheric turbulence image and corresponding Zernike coefficients of each order as a tag data set, wherein each order Zernike coefficient is expressed as a= [ a ] 1 ,a 2 ,...,a n ];
Step 2: inputting distorted wavefront caused by atmospheric turbulence into a pyramid wavefront sensor, acquiring sensing image data of the pyramid wavefront sensor, and calculating corresponding x-direction wavefront slope and y-direction wavefront slope S of each pixel point x 、S y As a sample dataset;
step 3: randomly extracting and dividing the corresponding data sets obtained in the step 1 and the step 2, wherein 80% is used as a training data set, and 20% is used as a test data set;
step 4: constructing a pyramid wavefront sensor wavefront reconstruction neural network model based on a U-Net architecture, wherein the model comprises an encoder, a decoder and jump connection;
step 5: and inputting the training sample data set into the model, comparing the output data with training label data by using a loss function, and returning the result to the neural network model to optimize the parameters. Repeatedly repeating the steps to obtain an Epoch-Loss graph, and determining optimal neural network model parameters;
step 6: and confirming the training effect of the neural network by using the test data set.
Further, the method for acquiring the simulated atmospheric turbulence data comprises the following steps: based on the Kolmogorov turbulence statistical theory, the telescope pupil diameter D, fried parameter r is set 0 Zernike order n, by different turbulence intensities (D/r 0 ) Generating corresponding Zernike coefficients of each order byForming a simulated atmospheric turbulence screen, wherein z i Representing the Zernike polynomials of each order.
Further, the wavefront sensor adopts a rectangular pyramid wavefront sensor, and incident light passes through a pyramid to generate four sub-pupil images, and the wavefront slope S x 、S y The calculation formula is as follows:
wherein I is 00 (x,y)、I 01 (x,y)、I 10 (x,y)、I 11 (x, y) represents the intensity of light at (x, y) in each quadrant, I 0 The average light intensity at the detector plane is expressed as follows:
the wavefront slope S x 、S y Normalization was performed to bring all slopes RMS to 1.
Further, the neural network comprises two branch encoders, a wavefront slope S x 、S y Respectively inputting two encoders, wherein the encoders comprise a rolling stage and a downsampling stage, and data obtained by double convolution are spliced to an upsampling stage of a decoder; the reconstructed wavefront phase is the output of a decoder that includes a convolution and up-sampling stage.
Further, the loss function is the mean square error MSE of the target wavefront and the reconstructed wavefront:
wherein N represents the turbulence plane sampling number, T (i) represents the ith phase of a neural network predicted turbulence phase diagram, T 0 (i) Representing the ith phase of the simulated turbulent phase diagram。
Further, the neural network weight optimization algorithm employs an adaptive gradient descent algorithm (Adam).
Compared with the prior art, the invention has the advantages that:
(1) The invention provides a pyramid wavefront sensor wavefront reconstruction method based on an artificial neural network, which adopts a neural network architecture of a U-Net encoder-decoder to extract wavefront slope data characteristics, and the neural network architecture is in jump connection with complementary lost data information, so as to finally realize distorted wavefront reconstruction.
(2) Compared with the traditional shack-Hartmann wavefront sensor, the wavefront sensor provided by the invention has the advantages of adjustable sampling rate, high sensitivity, strong spatial aliasing robustness, suitability for weak light detection application and the like.
(3) Compared with the traditional wavefront reconstruction algorithm, the artificial neural network has relatively strong capability of fitting the nonlinear relation of the pyramid wavefront sensor, effectively improves noise robustness and reconstruction accuracy, and develops a new thought for high-accuracy distorted wavefront reconstruction of the pyramid wavefront sensor in the future.
Drawings
Fig. 1 is a classical optical path propagation schematic diagram of a pyramid wavefront sensor, wherein 1 is a laser, 2 is distortion turbulence, 3 is a focusing lens, 4 is a quadrangular pyramid, 5 is an imaging lens, and 6 is a CCD camera;
FIG. 2 is a flow chart of wavefront reconstruction of a pyramid wavefront sensor based on an artificial neural network;
FIG. 3 is a simulated atmospheric turbulence image and corresponding Zernike coefficients, wherein (a) is a simulated atmospheric turbulence image and (b) is a Zernike coefficient corresponding to the simulated atmospheric turbulence image;
FIG. 4 is a sensed image of a pyramid wavefront sensor;
fig. 5 is a model of a wavefront reconstruction neural network based on the U-net architecture.
Detailed Description
The present invention will be further described with reference to the drawings and the specific embodiments, in order to make the objects, technical solutions and advantages of the present invention more apparent.
The optical system for realizing wavefront sensing based on the pyramid wavefront sensor of the invention is shown in fig. 1, and comprises a laser 1, a focusing lens 3, a rectangular pyramid 4, an imaging lens 5 and a CCD camera 6. In this embodiment, the resolution of imaging the pyramid wavefront sensor at the CCD is 200×200, the pixel size is 10 μm, the laser wavelength is 635nm, the resolution of the distorted turbulence 2 plane is 160×160, and the neural network architecture is improved based on the U-Net architecture.
Fig. 2 is a workflow diagram of a wavefront reconstruction method of a pyramid wavefront sensor based on an artificial neural network, and the specific implementation process is as follows:
step 1: a simulated atmospheric turbulence image (fig. 3 (a)) and corresponding first 65 th order Zernike coefficients (excluding the 0 th order piston term) (fig. 3 (b)) were acquired as a label dataset, a schematic diagram being shown in fig. 3. Wherein the Zernike coefficients are expressed as a= [ a ] 1 ,a 2 ,...,a 65 ]The atmospheric turbulence aberration wavefront is expressed asWherein z is i Representing the Zernike polynomials of each order. Turbulence intensity (D/r) 0 ) 100 groups were sampled uniformly from 1-100, and the same turbulence intensity was run for 110 time steps, generating 11000 turbulence planes. Removing the first 10 time step turbulence plane data in each group of operation, avoiding the initial similar turbulence from interfering with the neural network training process, and obtaining 10000 turbulence planes as a label data set;
step 2: distorted wavefront caused by atmospheric turbulence is input into the pyramid wavefront sensors, 10000 pyramid wavefront sensors are obtained to sense image data, and a schematic diagram is shown in fig. 4. Next, calculating the corresponding x-direction wavefront slope S and y-direction wavefront slope S of each pixel point x 、S y As a sample dataset, wavefront slope S x 、S y The calculation formula is as follows:
wherein I is 00 (x,y)、I 01 (x,y)、I 10 (x,y)、I 11 (x, y) represents the intensity of light at (x, y) in each quadrant, I 0 The average light intensity at the detector plane is expressed as follows:
the wavefront slope S x 、S y Normalizing to make all slopes RMS 1;
step 3: randomly extracting and dividing the corresponding data sets obtained in the step 1 and the step 2, wherein 8000 groups are used as training data sets, and 2000 groups are used as test data sets;
step 4: a pyramid wavefront sensor wavefront reconstruction neural network model based on a U-Net architecture is built, and a schematic diagram is shown in fig. 5, and comprises an encoder, a decoder, jump connection and the like. Input layer: the x and y slopes are used as inputs to the two encoders, each of 200 x 200. An encoder: including 4 double convolution and downsampling, the convolution kernel size is 3×3, and the pooling kernel size is 2×2. Each convolution follows the BatchNorm and ReLu activation functions to extract the valid data features, and the data from the double convolution is clipped and spliced to the up-sampling stage of the decoder. A decoder: including 4 double convolution and upsampling, the convolution kernel size is 3×3, the deconvolution kernel size is 2×2, each convolution follows the BatchNorm and ReLu activation functions. Output layer: distorted turbulent phase T of 160×160 magnitude;
step 5: inputting training sample data set into model, and using loss function for output data and training label dataBy comparison, T (i) represents the ith phase of the predicted turbulent phase map of the neural network, T 0 (i) The i-th phase of the simulated turbulent phase diagram is shown. Then, willReturning the result to the neural network model to optimize the parameters by using an adaptive gradient descent algorithm, wherein the learning rate is respectively set to be 1 multiplied by 10 -3 、5×10 -4 And 1X 10 -4 . Repeatedly repeating the steps to obtain an Epoch-Loss graph, and determining optimal neural network model parameters according to the change trend of the Loss function;
step 6: and confirming the training effect of the neural network by using the test data set.
The foregoing is a preferred embodiment of the present invention, and the scope of the present invention is not limited to the foregoing examples, but all designs falling under the concept of the present invention fall within the scope of the present invention.
Claims (6)
1. The wavefront reconstruction method of the pyramid wavefront sensor based on the artificial neural network is characterized by comprising the following steps of:
step 1: obtaining a simulated atmospheric turbulence image and corresponding Zernike coefficients of each order as a tag data set, wherein each order Zernike coefficient is expressed as a= [ a ] 1 ,a 2 ,...,a n ];
Step 2: inputting distorted wavefront caused by atmospheric turbulence into a pyramid wavefront sensor, acquiring sensing image data of the pyramid wavefront sensor, and calculating corresponding x-direction wavefront slope and y-direction wavefront slope S of each pixel point x 、S y As a sample dataset;
step 3: randomly extracting and dividing the corresponding data sets obtained in the step 1 and the step 2, wherein 80% is used as a training data set, and 20% is used as a test data set;
step 4: the method comprises the steps of constructing a wavefront reconstruction neural network model of a pyramid wavefront sensor based on a U-Net architecture, wherein the model comprises an encoder, a decoder and jump connection;
step 5: inputting a training sample data set into a model, comparing output data with training label data by using a Loss function, returning a result to a neural network model to optimize parameters, repeatedly repeating the steps to obtain an Epoch-Loss diagram, and determining optimal neural network model parameters;
step 6: and confirming the training effect of the neural network by using the test data set.
2. The method for reconstructing a pyramid wavefront sensor wavefront based on an artificial neural network according to claim 1, wherein the method for acquiring simulated atmospheric turbulence data is as follows: based on the Kolmogorov turbulence statistical theory, the telescope pupil diameter D, fried parameter r is set 0 Zernike order n, by different turbulence intensities (D/r 0 ) Generating corresponding Zernike coefficients of each order byForming a simulated atmospheric turbulence screen, wherein z i Representing the Zernike polynomials of each order.
3. The method for reconstructing wavefront of pyramid wavefront sensor based on artificial neural network as claimed in claim 1, wherein the wavefront sensor is a rectangular pyramid wavefront sensor, the incident light is pyramid-shaped to generate four sub-pupil images, and the wavefront slope S x 、S y The calculation formula is as follows:
wherein I is 00 (x,y)、I 01 (x,y)、I 10 (x,y)、I 11 (x, y) represents the intensity of light at (x, y) in each quadrant, I 0 The average light intensity at the detector plane is expressed as follows:
then, the wavefront slope S x 、S y Performing normalizationAll slopes RMS were made 1.
4. The method for reconstructing a pyramid wavefront sensor wavefront based on an artificial neural network according to claim 1, wherein the neural network comprises two branch encoders, a wavefront slope S x 、S y Respectively inputting two encoders, wherein the encoders comprise a rolling stage and a downsampling stage, and data obtained by double convolution are spliced to an upsampling stage of a decoder; the reconstructed wavefront phase is the output of a decoder that includes a convolution and up-sampling stage.
5. The method for reconstructing a pyramid wavefront sensor wavefront based on an artificial neural network according to claim 1, wherein the loss function is a mean square error MSE of the target wavefront and the reconstructed wavefront:
wherein N represents the turbulence plane sampling number, T (i) represents the ith phase of a neural network predicted turbulence phase diagram, T 0 (i) The i-th phase of the simulated turbulent phase diagram is shown.
6. The method for reconstructing a pyramid wavefront sensor wavefront based on an artificial neural network according to claim 1, wherein the neural network weight optimization algorithm adopts an adaptive gradient descent algorithm (Adam).
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310404779.2A CN116295873A (en) | 2023-04-17 | 2023-04-17 | Pyramid wavefront sensor wavefront reconstruction method based on artificial neural network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310404779.2A CN116295873A (en) | 2023-04-17 | 2023-04-17 | Pyramid wavefront sensor wavefront reconstruction method based on artificial neural network |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116295873A true CN116295873A (en) | 2023-06-23 |
Family
ID=86822408
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310404779.2A Pending CN116295873A (en) | 2023-04-17 | 2023-04-17 | Pyramid wavefront sensor wavefront reconstruction method based on artificial neural network |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116295873A (en) |
-
2023
- 2023-04-17 CN CN202310404779.2A patent/CN116295873A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111579097B (en) | High-precision optical scattering compensation method based on neural network | |
CN113158487B (en) | Wavefront phase difference detection method based on long-short term memory depth network | |
Suárez Gómez et al. | Improving adaptive optics reconstructions with a deep learning approach | |
CN111932452B (en) | Infrared image convolution neural network super-resolution method based on visible image enhancement | |
CN112598683A (en) | Frequency sweep OCT (optical coherence tomography) human eye image segmentation method based on frequency sweep optical coherence tomography | |
Sweere et al. | Deep learning-based super-resolution and de-noising for XMM-newton images | |
CN117195960A (en) | Atmospheric turbulence wave front detection method based on shack Hartmann UT conversion model | |
CN117760571B (en) | Unsupervised learning wavefront detection method based on Hartmann detector | |
CN116295873A (en) | Pyramid wavefront sensor wavefront reconstruction method based on artificial neural network | |
WO2023144519A1 (en) | Determining optical aberration | |
Dai et al. | Intelligent ammeter reading recognition method based on deep learning | |
WO2023023961A1 (en) | Piv image calibration apparatus and method based on laser linear array | |
Weddell et al. | Reservoir computing for prediction of the spatially-variant point spread function | |
Norouzi et al. | CNN to mitigate atmospheric turbulence effect on Shack-Hartmann Wavefront Sensing: A case study on the Magdalena Ridge Observatory Interferometer. | |
CN114463192A (en) | Infrared video distortion correction method based on deep learning | |
CN112179503A (en) | Deep learning wavefront restoration method based on sparse subaperture shack-Hartmann wavefront sensor | |
CN113192146B (en) | Method for eliminating defocusing error in grid line projection phase shift technology based on deep learning | |
Li et al. | Spectral-Learning-Based Transformer Network for the Spectral Super-Resolution of Remote-Sensing Degraded Images | |
CN116071268B (en) | Image illumination removal model based on contrast learning and training method thereof | |
CN116862800B (en) | Large-view-field single-lens space-variant blurred image restoration method and device | |
Yu et al. | Microscopy image reconstruction method based on convolution network feature fusion | |
CN118225255A (en) | Atmospheric degradation wave front phase direct extraction method in space target optical image | |
Gu et al. | In situ fast measurement of the radius of curvature for the plano-convex lens with deep-learning based on the point spread functions | |
CN116051372A (en) | Self-adaptive optical high-resolution imaging method based on RESNET network | |
CN116929570A (en) | Compressed wavefront detection method based on deep learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |