CN110514137A - Phase- un- wrapping method, apparatus, system, computer equipment and storage medium - Google Patents
Phase- un- wrapping method, apparatus, system, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110514137A CN110514137A CN201910633802.9A CN201910633802A CN110514137A CN 110514137 A CN110514137 A CN 110514137A CN 201910633802 A CN201910633802 A CN 201910633802A CN 110514137 A CN110514137 A CN 110514137A
- Authority
- CN
- China
- Prior art keywords
- phase
- matrix
- threshold value
- wrapped
- phase matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 239000011159 matrix material Substances 0.000 claims abstract description 212
- 238000012545 processing Methods 0.000 claims abstract description 15
- 238000009499 grossing Methods 0.000 claims abstract description 14
- 230000000994 depressogenic effect Effects 0.000 claims abstract description 4
- 238000001914 filtration Methods 0.000 claims description 14
- 238000001228 spectrum Methods 0.000 claims description 8
- 239000004065 semiconductor Substances 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 abstract description 24
- 230000008569 process Effects 0.000 description 14
- 230000003044 adaptive effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000003384 imaging method Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000007781 pre-processing Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000010363 phase shift Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000010008 shearing Methods 0.000 description 2
- 238000002834 transmittance Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000005305 interferometry Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000001314 profilometry Methods 0.000 description 1
- 238000007430 reference method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 239000012780 transparent material Substances 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
- G01S13/90—Radar or analogous systems specially adapted for specific applications for mapping or imaging using synthetic aperture techniques, e.g. synthetic aperture radar [SAR] techniques
- G01S13/9021—SAR image post-processing techniques
- G01S13/9023—SAR image post-processing techniques combined with interferometric techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/2433—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures for measuring outlines by shadow casting
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
- G01S13/90—Radar or analogous systems specially adapted for specific applications for mapping or imaging using synthetic aperture techniques, e.g. synthetic aperture radar [SAR] techniques
- G01S13/9004—SAR image acquisition techniques
- G01S13/9005—SAR image acquisition techniques with optical processing of the SAR signals
Landscapes
- Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of Phase- un- wrapping method, apparatus, system, computer equipment and storage mediums, which comprises obtains wrapped phase interference pattern；Threshold value；The recess boundary of wrapped phase interference pattern is determined by threshold value, forms overturning critical line；Automatic turning replacement is carried out to the depressed section of wrapped phase interference pattern, obtains the first Phase Unwrapping Algorithm matrix；Smoothing processing is filtered to the first Phase Unwrapping Algorithm matrix, obtains the second Phase Unwrapping Algorithm matrix；Calculate and save the relative error of the first Phase Unwrapping Algorithm matrix and the second Phase Unwrapping Algorithm matrix；When threshold value is not up to preset value, return redefines threshold value and executes subsequent operation；When threshold value reaches preset value, the corresponding threshold value of minimum relative error and the second Phase Unwrapping Algorithm matrix are exported；According to the corresponding second Phase Unwrapping Algorithm matrix of minimum relative error, the true phase distribution of reconstruct is exported.The present invention improves the efficiency and accuracy of the true phase reduction of wrapped phase interference pattern.
Description
Technical Field
The invention relates to a phase unwrapping method, a phase unwrapping device, a phase unwrapping system, computer equipment and a storage medium, and belongs to the field of image processing research.
Background
The phase unwrapping calculation is widely applied to interferometric synthetic aperture radars, three-dimensional measurement based on structured light, optical interferometry, imaging and other measurement methods. In the process of acquiring the phase from the wrapped interferogram, the phase is generally extracted by using an arc tangent norm, so that the continuous phase is truncated, a value range is limited to (-pi, pi) to form a wrapped phase map.
The existing phase unwrapping algorithms and methods are numerous and mainly include two types: path tracking algorithms and path independent algorithms. The path tracking algorithm mainly comprises a branch cutting method, a quality diagram guiding method, a mask secant method and the like; the path-independent algorithm mainly seeks a solution satisfying a minimum norm, such as a least square method based on discrete cosine transform, a least square method based on fast fourier transform, and the like. Severe undersampling and strong noise are two problems for unwrapping, with qiangfang et al proposing the transverse shearing least squares method (LSBLS), goutiao et al proposing the four-way transverse shearing least squares method (FSLBS). These algorithms can recover the undersampled phase map but still do not effectively solve the severely undersampled wrapped phase map.
In recent years, adaptive thresholds have been widely used in image segmentation, denoising, defogging, phase conversion, and the like. An Eser Sert et al foreign scholars provides a three-level calibration algorithm of self-adaptive threshold value phase conversion based on a structured light system to measure three-dimensional objects. In image processing, researchers have proposed edge projection profilometry based on adaptive threshold removal of background, and adaptive thresholds have also been used for cell image processing, image denoising, and the like. Currently, the adaptive threshold has not been widely applied in the aspect of image unwrapping.
Disclosure of Invention
In view of the above, the present invention provides a phase unwrapping method, device, system, computer device and storage medium, which have high phase unwrapping precision, improve the efficiency and accuracy of true phase restoration of wrapped phase interferograms, and solve the problems of the existing phase unwrapping algorithm.
A first object of the present invention is to provide a phase unwrapping method.
A second object of the present invention is to provide a phase unwrapping device.
It is a third object of the present invention to provide a phase unwrapping system.
It is a fourth object of the invention to provide a computer apparatus.
A fifth object of the present invention is to provide a storage medium.
The first purpose of the invention can be achieved by adopting the following technical scheme:
a phase unwrapping method, the method comprising:
acquiring a wrapped phase interferogram;
determining a threshold value;
determining a sunken boundary wrapping the phase matrix through a threshold value to form a turning critical line;
automatically overturning and replacing the concave part of the wrapped phase matrix to obtain a first unwrapped phase matrix;
carrying out filtering smoothing treatment on the first unwrapped phase matrix to obtain a second unwrapped phase matrix;
calculating and storing the relative error of the first unwrapped phase matrix and the second unwrapped phase matrix;
when the threshold value does not reach the preset value, returning to re-determine the threshold value, and executing subsequent operation;
when the threshold value reaches a preset value, outputting the threshold value corresponding to the minimum relative error and a second unwrapped phase matrix;
and outputting the reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
Further, the threshold is a value between 0.9 and 1, where 0.9 is the first threshold and 1 is the last threshold; the step of re-determining the threshold value is to add 0.01 to the current threshold value; the preset value is 1.
Further, the determining a concave boundary wrapping the phase matrix by the threshold to form a critical line of inversion specifically includes:
taking the threshold value W as a basis for acquiring the concave boundary of the wrapped phase matrix, extracting the pixel with the phase value larger than W x phi (max) in the wrapped phase matrix, and setting the pixel area to be 1, wherein phi (max) is the maximum value of the phase of the input phase matrix, namely the pixel area set to be 1 is a concave part, and the rest pixels are set to be 0.
Further, the automatic flip replacement is performed on the recessed portion of the wrapped phase matrix to obtain a first unwrapped phase matrix, which specifically includes:
overturning the sunken part of the wrapped phase matrix, and replacing the position of the sunken part in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix;
when the recessed matrix is a non-zero matrix, the recessed matrix is turned, and the position of a recessed position in the current phase matrix is replaced by the recessed matrix obtained after turning to obtain a new phase matrix;
and when the depression matrix is a zero matrix, taking the current phase matrix as a first unwrapped phase matrix and outputting.
Further, after acquiring the wrapped phase interferogram, the method further includes:
the wrapped phase interferogram is preprocessed, and the arccosine wrapped phase interferogram can be converted into an arctangent wrapped phase interferogram when needed in the actual process.
The second purpose of the invention can be achieved by adopting the following technical scheme:
a phase unwrapping device, the device comprising:
the acquisition module is used for acquiring a wrapped phase interferogram;
a first determining module for determining a threshold;
the second determining module is used for determining a sunken boundary wrapping the phase matrix through a threshold value to form a turning critical line;
the overturning replacement module is used for automatically overturning and replacing the sunken part of the wrapped phase matrix to obtain a first unwrapped phase matrix;
the processing module is used for carrying out filtering smoothing processing on the first unwrapped phase matrix to obtain a second unwrapped phase matrix;
the calculation module is used for calculating and storing the relative error of the first unwrapped phase matrix and the second unwrapped phase matrix;
the returning module is used for returning to re-determine the threshold value and executing subsequent operation when the threshold value does not reach the preset value;
the first output module is used for outputting a threshold value corresponding to the minimum relative error and a second unwrapped phase matrix when the threshold value reaches a preset value;
and the second output module is used for outputting the reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
Further, the flipping replacement module specifically includes:
the first overturning and replacing unit is used for overturning the sunken part of the wrapped phase matrix and replacing the sunken position in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix;
the second overturning and replacing unit is used for overturning the sunken matrix when the sunken matrix is a non-zero matrix, and replacing the position of a sunken part in the current phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix;
and the output unit is used for taking the current phase matrix as a first unwrapped phase matrix and outputting the first unwrapped phase matrix when the depression matrix is a zero matrix.
Further, after the obtaining module, the method further includes:
and the preprocessing module is used for preprocessing the wrapped phase interference pattern, and can convert the inverse cosine wrapped phase interference pattern into an arc tangent wrapped phase interference pattern when needed in the actual process.
The third purpose of the invention can be achieved by adopting the following technical scheme:
a phase unwrapping system comprises a semiconductor laser, an adjustable attenuator, a beam expander, a pinhole filter, a first collimating lens, a second collimating lens, a phase plate, a third collimating lens, a camera and a computer which are sequentially connected, wherein a target object is placed between the first collimating lens and the second collimating lens, the target object is located at a front focal plane of the second collimating lens, the phase plate is located at a rear focal plane of the second collimating lens, and the camera is located at a rear focal plane of the third collimating lens;
laser beams emitted by the semiconductor laser sequentially pass through the attenuator, the beam expander, the pinhole filter and the first collimating lens to form planar light waves so as to irradiate a target object; obtaining the frequency spectrum of the object light wave on the back focal plane of the second collimating lens, enabling the phase modulation region to be aligned to the zero-frequency region of the frequency spectrum of the object light wave by the phase plate, enabling the light wave passing through the zero-frequency region and the light wave passing through the outside of the zero-frequency region to undergo different phase modulations, generating a wrapped phase interference pattern on the back focal plane of the third collimating lens, collecting the wrapped phase interference pattern by a camera, and inputting the wrapped phase interference pattern into a computer;
the computer is used for executing the phase unwrapping method.
The fourth purpose of the invention can be achieved by adopting the following technical scheme:
a computer device comprising a processor and a memory for storing a program executable by the processor, the processor implementing the phase unwrapping method when executing the program stored in the memory.
The fifth purpose of the invention can be achieved by adopting the following technical scheme:
a storage medium stores a program which, when executed by a processor, implements the phase unwrapping method described above.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention determines the threshold value after obtaining the wrapping phase interferogram, determines the depressed boundary of the wrapping phase matrix through the threshold value to form a turning critical line, automatically turns and replaces the depressed part of the wrapping phase matrix, filters and smoothes the phase matrix after automatic turning and replacing, calculates and stores the relative errors of the two phase matrices before and after filtering and smoothes, re-determines the threshold value, executes the subsequent operation of determining the threshold value until the threshold value reaches the preset value, and then terminates the cycle by comparing the relative errors stored in the threshold value each time to obtain the minimum relative error, outputs the threshold value corresponding to the minimum relative error and the phase matrix after filtering and smoothing, adopts a phase turning phase recovery method in phase unwrapping, and has the greatest advantages that the turning critical line is determined by using the threshold value, and the complete real phase can be solved by the minimum relative error, therefore, the accuracy of phase unwrapping is improved, manual threshold setting can be effectively avoided by automatically searching for the threshold, correct threshold selection is provided, and the efficiency and accuracy of phase restoration are improved.
2. The invention adopts an automatic overturning algorithm to process the sunken part of the wrapped phase matrix, overturns the sunken part of the wrapped phase matrix, replaces the sunken position in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix, judges the sunken matrix at the moment, overturns the sunken matrix if the sunken matrix is a non-zero matrix, replaces the sunken position in the current phase matrix with the sunken matrix obtained after overturning to obtain the new phase matrix, stops circulation until the sunken matrix is a zero matrix, and enables the sunken part in the unwrapped phase matrix not to be found in an automatic overturning and replacing mode.
3. The invention can accurately recover the real phase under the condition of strong noise, has certain anti-noise performance and solves the problem that the noise can influence the unwrapping precision.
4. The error between the reconstructed phase and the real phase obtained by the method is small, the phase unwrapping can be realized on the wrapped phase interferogram (such as 1501 multiplied by 1501) with large grid size, the phase unwrapping precision is improved, and a new reference method is provided for unwrapping with high precision and large calculation amount.
5. The invention is suitable for unwrapping various wrapped phase interferograms with wrapping periods of pi, 2 pi, 3 pi and the like, and has wide application range and field.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
Fig. 1 is an optical path diagram of a phase unwrapping system according to embodiment 1 of the present invention.
FIG. 2 is a background intensity plot for example 1 of the present invention.
FIG. 3 is a wrapped phase interferogram of embodiment 1 of the invention.
Fig. 4 is a flowchart of the phase unwrapping method in embodiment 1 of the present invention.
Fig. 5 is a complete flow chart of the phase unwrapping method of embodiment 1 of the present invention.
Fig. 6 is a flowchart of an automatic rollover replacement algorithm according to embodiment 1 of the present invention.
Fig. 7 is a true phase distribution diagram obtained by unwrapping by using the adaptive threshold flipping algorithm in embodiment 1 of the present invention.
Fig. 8 is a diagram of a true phase distribution obtained by unwrapping using a conventional transverse shear least squares solution.
Fig. 9 is a block diagram of a phase unwrapping device according to embodiment 3 of the present invention.
Fig. 10 is a block diagram of a flip replacement module according to embodiment 3 of the present invention.
Fig. 11 is a block diagram of a computer device according to embodiment 4 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Example 1:
the phase contrast method is a common quantitative phase distribution imaging method and is widely applied to the fields of transparent biomedical cell microscopic imaging, transparent material or microlens detection, interference measurement, flow field density distribution measurement and the like. In the reconstruction of the true phase distribution by the full-field phase contrast method, the unwrapping process of the phase is often involved.
If the sample of the target object is a pure phase transparent object, referred to as a phase object for short, and the parallel light with amplitude a is normally incident on the object plane, the complex amplitude of the object light field can be expressed as:
if the influence of the finite aperture of the coherent optical system is neglected, the spectrum is:
if the Zernike phase plate is applied to the spectrum plane, the spatial filter function is:
wherein t andthe amplitude transmittance and the phase shift amount of the phase plate are respectively. After adding the phase plate, image planeThe mathematical expression of the light intensity distribution of (a) can be written as:
require solving for the phase distributionIt is also necessary to know the image surface background light intensity I when the sample and the phase plate are not placed0The size of (2). Considering that moving the phase plate multiple times when imaging an actual sample adds much wasted work, the sample is directly removed or the light field of the region without object information in the sample is passed through without moving the phase plate. At this time, the image plane background light intensity can be expressed as:
I0＝A2t2；
from I (x, y) and I0The phase principal value distribution of the measured sample can be calculated as follows:
wherein,the phase values in the range of 0 to pi can be uniquely determined according to the above formula. Therefore, when the phase change of the measured object is small, namely the phase distribution change range is within pi,the value of (a) can directly reflect the real phase distribution of the object; if the object phase is not exceeded, two phase contrast images are shot, namely a background light intensity image I0And wrapping the phase interferogram I (x, y), because the phase is wrapped, the solution can not be directly carried out, and the true phase distribution of the measured phase object can be reconstructed by unwrapping by using the above-mentioned inverse cosine relation.
FIG. 1 is an optical path diagram of a phase unwrapping system according to an embodiment, which includes a semiconductor laser 101, a phase unwrapper, and a phase unwrapperThe phase contrast method can be performed by the attenuator 102, the beam expander 103, the pinhole filter 104, the first collimating lens 105, the second collimating lens 106, the phase plate 107, the third collimating lens 108, the camera 109 and the computer 110, the camera 109 can adopt a CCD camera, a wrapped phase interferogram of the sample 111 can be obtained by the camera 109, if the phase distribution of the sample 111 exceeds pi, the unwrapping algorithm processing is required to obtain the real phase distribution or imaging of the sample, the sample 111 selected in the experiment is a microlens array group, wherein the actual standard maximum thickness of a single microlens sphere is about 3.52 μm, and when the microlens array is irradiated by a laser beam with a wavelength of 635nm, the maximum phase difference from the top to the bottom of the sphere is about 15.88rad (5.055 pi); the phase plate 107 has an amplitude transmittance t of 0.3 and a phase shift amountA home-made phase plate of 1.1 pi is used for experiments with this phase plate 107, placed strictly on the spectral plane (focal plane) of the imaging lens by means of a fine-tuning support.
The experiment for acquiring the background light intensity image and the wrapped phase interference image by using the phase unwrapping system comprises the following steps:
1) a laser beam emitted from a semiconductor laser 101 passes through an adjustable attenuator 102, a beam expander 103, a pinhole filter 104 and a first collimating lens 105 in sequence to form a planar lightwave to irradiate a sample 111; the wavelength of the laser beam emitted by the semiconductor laser 101 is 635nm, and the focal length of the first collimating lens 105 is 150 mm.
2) Adjusting the position of the second collimating lens 106 to make the sample 111 be located at the front focal plane of the second collimating lens 106, and obtaining the frequency spectrum of the object light wave on the rear focal plane of the second collimating lens 106; wherein the focal length of the second collimating lens 106 is 180 mm.
3) The phase plate 107 is positioned at the back focal plane of the second collimating lens 106, the position of the phase plate 107 is adjusted, the phase modulation area of the phase plate 107 is aligned to the zero-frequency area of the object optical wave spectrum, the optical wave passing through the zero-frequency area (zero-frequency component) and the optical wave passing outside the zero-frequency area (high-frequency component) undergo different phase modulation, and a wrapped phase interference pattern is generated on the back focal plane of the third collimating lens 108; wherein the focal length of the third collimating lens 108 is 180 mm.
3) The camera 109 is located at the back focal plane of the third collimating lens, the background light intensity map and the wrapped phase interference map are collected by the camera 109, the collected images can be input into the computer 110 through a data line, or the collected images can be input into the computer 110 through a wireless transmission mode, fig. 2 is a background light intensity map containing noise, and fig. 3 is a wrapped phase interference map containing noise.
As shown in fig. 4 and fig. 5, the embodiment further provides a phase unwrapping method, which uses an adaptive threshold flipping algorithm (SATR) for unwrapping, and is applied in the computer 110, and includes the following steps:
s501, acquiring a wrapped phase interference pattern.
The camera 109 collects the background light intensity image and the wrapped phase interferogram and inputs the background light intensity image and the wrapped phase interferogram into the computer 110, and the computer 110 acquires the wrapped phase interferogram and unwrapps the wrapped phase interferogram through subsequent steps.
And S502, preprocessing the wrapped phase interference pattern.
The wrapped phase interferogram is preprocessed, and the arccosine wrapped phase interferogram can be converted into an arctangent wrapped phase interferogram when needed in the actual process.
S503, determining a threshold value.
Specifically, the threshold is denoted as W, W is greater than or equal to 0.9 and less than or equal to 1, and the threshold determined for the first time in this embodiment is 0.9.
And S504, determining a sunken boundary wrapping the phase matrix through a threshold value to form a turnover critical line.
Specifically, taking a threshold value W as a basis for acquiring a recessed boundary of a wrapped phase matrix, extracting pixels with phase values greater than W × phi (max) in the wrapped phase matrix, and setting a pixel region to be 1, where phi (max) is a maximum value of a phase of an input phase matrix, that is, the pixel region set to be 1 is a recessed portion, and the remaining pixels are set to be 0, forming a "wall" with only a recessed boundary value of 1, where the "wall" is a boundary line for automatic flipping of a flipping critical line, and the wrapped phase matrix is denoted as phi00, and the implementation code is as follows:
and S505, automatically overturning and replacing the concave part wrapped with the phase matrix to obtain a first unwrapped phase matrix.
The implementation of step S505 adopts an automatic roll-over replacement algorithm, as shown in fig. 6, which specifically includes:
s5051, overturning the sunken part wrapped with the phase matrix, and replacing the sunken position in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix.
S5052, judging the depression matrix.
If the hollow matrix is a non-zero matrix, the process proceeds to step S5053, and if the hollow matrix is a zero matrix, the process proceeds to step S5054.
S5053, overturning the depression matrix, and replacing the depression position in the current phase matrix with the depression matrix obtained after overturning to obtain a new phase matrix;
after the execution of this step is completed, the process returns to step S5052, and the loop is terminated to step S5054 until it is determined in step S5052 that the recessed matrix is a zero matrix.
S5054, taking the current phase matrix as a first unwrapped phase matrix, and outputting.
The phase matrixes in the step S505 are all marked as phi00, the symmetry axis of the overturn is pi for the first time, 2 pi for the second time, 4 pi for the third time, and so on; the code to implement the first flip is as follows:
s506, filtering and smoothing the first unwrapped phase matrix to obtain a second unwrapped phase matrix which is recorded as phi 00T.
And S507, calculating and storing the relative error between the first unwrapped phase matrix and the second unwrapped phase matrix.
And S508, judging the threshold value.
If the threshold W does not reach the preset value, that is, the threshold W is smaller than the preset value, the process returns to step S503 to re-determine the threshold W, and if the threshold W reaches the preset value, that is, equal to the preset value, the process proceeds to step S509.
The preset value of this embodiment is 1, and the re-determining the threshold W means that the current threshold W is added to 0.01, that is, the threshold W determined for the second time is 0.91, the threshold W determined for the third time is 0.92, the threshold W determined for the fourth time is 0.93 … …, and so on, and the loop is terminated until the threshold W is determined to be 1, that is, the threshold W determined for the last time is 1, and 11 thresholds W are calculated in total, that is, 11 times of automatic inversion replacement are performed.
And S509, outputting a threshold corresponding to the minimum relative error and the second unwrapped phase matrix.
And comparing the relative errors stored in the threshold W determined each time to obtain the minimum relative error, and outputting the threshold W corresponding to the minimum relative error and the second unwrapped phase matrix phi00T, wherein the threshold W is the optimal threshold.
The steps S502 to S509 are implemented by using a findsunnken function, which is a core function of the adaptive threshold flipping algorithm, a variable W is defined when a threshold is set in the function, the variable W is a key for implementing subsequent automatic flipping and replacing operations, each time the threshold W corresponds to one automatic flipping and replacing, relative errors between the first unwrapped phase matrix phi00 after each automatic flipping and replacing and the second unwrapped phase matrix phi00T after filtering and smoothing are stored, the threshold W corresponding to the minimum relative error and the second unwrapped phase matrix phi00T are found and output, the threshold W is continuously updated, and the accuracy of the operation is ensured.
And S510, outputting reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
The true phase distribution map obtained by unwrapping with the adaptive threshold inversion algorithm (SATR) of the present embodiment is shown in fig. 7, and the true phase distribution map obtained by unwrapping with the conventional transverse shear least squares algorithm (LSBLS) is shown in fig. 8, where the maximum phase value of a single microlens obtained by SATR inversion is about 16.35rad, the thickness is about 3.62 μm, and compared with the microlens array parameters, there is an absolute error of about 0.10 μm, and the relative error is 2.84%; the phase maximum of a single microlens obtained by inversion with LSBLS is about 15.22rad and the thickness is about 3.37 μm, with an absolute error of about 0.15 μm and a relative error of 4.16% compared to the microlens array parameters. It is shown that the error between the reconstructed phase and the true phase obtained by the inversion by the method of the embodiment is small, and the accuracy is high.
Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be implemented by a program to instruct associated hardware, and the corresponding program may be stored in a computer-readable storage medium.
It should be noted that although the method operations of the above-described embodiments are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Rather, the depicted steps may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
Example 3:
as shown in fig. 9, the present embodiment provides a phase unwrapping apparatus, which may be applied in a computer, and includes an obtaining module 901, a preprocessing module 902, a first determining module 903, a second determining module 904, a flipping module 905, a processing module 906, a calculating module 907, a returning module 908, a first outputting module 909, and a second outputting module 910, where specific functions of the modules are as follows:
the acquiring module 901 is configured to acquire a wrapped phase interferogram.
The preprocessing module 902 is configured to preprocess the wrapped phase interferogram, and may convert the arccosine wrapped phase interferogram into an arctangent wrapped phase interferogram if necessary in an actual process.
The first determining module 903 is configured to determine a threshold.
The second determining module 904 is configured to determine a concave boundary wrapping the phase matrix by using a threshold value, so as to form a flip critical line.
And the overturning replacement module 905 is used for automatically overturning and replacing the concave part of the wrapped phase matrix to obtain a first unwrapped phase matrix.
The processing module 906 is configured to perform filtering smoothing processing on the first unwrapped phase matrix to obtain a second unwrapped phase matrix.
The calculating module 907 is configured to calculate and store a relative error between the first unwrapped phase matrix and the second unwrapped phase matrix.
And the returning module 908 is configured to return to re-determine the threshold value and perform subsequent operations when the threshold value does not reach the preset value.
The first output module 909 is configured to output the threshold corresponding to the minimum relative error and the second unwrapped phase matrix when the threshold reaches a preset value.
The second output module 910 is configured to output the reconstructed true phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
Further, the flipping replacement module 905 specifically includes, as shown in fig. 10:
and the first overturning and replacing unit 9051 is configured to overturn the sunken part of the wrapped phase matrix, and replace the sunken position in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix.
And a second flipping and replacing unit 9052, configured to flip the recessed matrix when the recessed matrix is a non-zero matrix, and replace a position of a recessed position in the current phase matrix with the flipped and obtained recessed matrix to obtain a new phase matrix.
And the output unit 9053 is configured to, when the recess matrix is a zero matrix, take the current phase matrix as a first unwrapped phase matrix, and output the first unwrapped phase matrix.
For specific implementation of each module in this embodiment, reference may be made to the phase unwrapping method in embodiment 1, which is not described herein any more; it should be noted that, the apparatus provided in this embodiment is only illustrated by dividing the functional modules, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above.
It is to be understood that the terms "first", "second", and the like, as used in the apparatus of the present embodiment, may be used to describe various units, but the units are not limited by these terms. These terms are only used to distinguish one module from another. For example, a first output module may be referred to as a second output module, and similarly, a second output module may be referred to as a first output module, both of which are output modules, but which are not the same output module, without departing from the scope of the present invention.
Example 4:
the present embodiment provides a computer device, which may be a computer, as shown in fig. 11, and includes a processor 902, a memory, an input device 1103, a display 1104 and a network interface 1105 connected by a system bus 1101, the processor is used for providing computing and controlling capability, the memory includes a nonvolatile storage medium 1106 and an internal memory 1107, the nonvolatile storage medium 906 stores an operating system, computer programs and a database, the internal memory 1107 provides an environment for the operation of the operating system and the computer programs in the nonvolatile storage medium, and when the processor 1102 executes the computer programs stored in the memory, the phase unwrapping method of the above embodiment 1 is implemented as follows:
acquiring a wrapped phase interferogram;
determining a threshold value;
determining a sunken boundary wrapping the phase matrix through a threshold value to form a turning critical line;
automatically overturning and replacing the concave part of the wrapped phase matrix to obtain a first unwrapped phase matrix;
carrying out filtering smoothing treatment on the first unwrapped phase matrix to obtain a second unwrapped phase matrix;
calculating and storing the relative error of the first unwrapped phase matrix and the second unwrapped phase matrix;
when the threshold value does not reach the preset value, returning to re-determine the threshold value, and executing subsequent operation;
when the threshold value reaches a preset value, outputting the threshold value corresponding to the minimum relative error and a second unwrapped phase matrix;
and outputting the reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
Further, after acquiring the wrapped phase interferogram, the method further includes: the wrapped phase interferogram is preprocessed, and the arccosine wrapped phase interferogram can be converted into an arctangent wrapped phase interferogram when needed in the actual process.
Example 5:
the present embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program, and when the program is executed by a processor, and the processor executes the computer program stored in the memory, the phase unwrapping method of the above embodiment 1 is implemented as follows:
acquiring a wrapped phase interferogram;
determining a threshold value;
determining a sunken boundary wrapping the phase matrix through a threshold value to form a turning critical line;
automatically overturning and replacing the concave part of the wrapped phase matrix to obtain a first unwrapped phase matrix;
carrying out filtering smoothing treatment on the first unwrapped phase matrix to obtain a second unwrapped phase matrix;
calculating and storing the relative error of the first unwrapped phase matrix and the second unwrapped phase matrix;
when the threshold value does not reach the preset value, returning to re-determine the threshold value, and executing subsequent operation;
when the threshold value reaches a preset value, outputting the threshold value corresponding to the minimum relative error and a second unwrapped phase matrix;
and outputting the reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
Further, after acquiring the wrapped phase interferogram, the method further includes: the wrapped phase interferogram is preprocessed, and the arccosine wrapped phase interferogram can be converted into an arctangent wrapped phase interferogram when needed in the actual process.
The storage medium in this embodiment may be a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), a usb disk, a removable hard disk, or other media.
In summary, after acquiring the wrapped phase interferogram, the invention determines the threshold, determines the recessed boundary of the wrapped phase matrix through the threshold to form the flip critical line, performs automatic flip replacement on the recessed part of the wrapped phase matrix, performs filtering smoothing processing on the phase matrix after the automatic flip replacement, calculates and stores the relative errors of the two phase matrices before and after the filtering smoothing processing, re-determines the threshold, executes the subsequent operation of determining the threshold, and terminates the cycle until the threshold reaches the preset value, at this time, obtains the minimum relative error by comparing the relative errors stored in each time of determining the threshold, outputs the threshold corresponding to the minimum relative error and the phase matrix after the filtering smoothing processing, and adopts the phase flip phase recovery method in phase unwrapping, which has the greatest advantage that the flip critical line is determined by using the threshold, and can solve the complete real phase with the minimum relative error, therefore, the accuracy of phase unwrapping is improved, manual threshold setting can be effectively avoided by automatically searching for the threshold, correct threshold selection is provided, and the efficiency and accuracy of phase restoration are improved.
The above description is only for the preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the scope of the present invention.
Claims (10)
1. A method of phase unwrapping, the method comprising:
acquiring a wrapped phase interferogram;
determining a threshold value;
determining a sunken boundary wrapping the phase matrix through a threshold value to form a turning critical line;
automatically overturning and replacing the concave part of the wrapped phase matrix to obtain a first unwrapped phase matrix;
carrying out filtering smoothing treatment on the first unwrapped phase matrix to obtain a second unwrapped phase matrix;
calculating and storing the relative error of the first unwrapped phase matrix and the second unwrapped phase matrix;
when the threshold value does not reach the preset value, returning to re-determine the threshold value, and executing subsequent operation;
when the threshold value reaches a preset value, outputting the threshold value corresponding to the minimum relative error and a second unwrapped phase matrix;
and outputting the reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
2. The phase unwrapping method according to claim 1, wherein the threshold is a value between 0.9 and 1, where 0.9 is a first threshold and 1 is a last threshold; the step of re-determining the threshold value is to add 0.01 to the current threshold value; the preset value is 1.
3. The phase unwrapping method according to any one of claims 1-2, wherein the determining the boundary of the dip wrapping the phase matrix by a threshold value to form a critical line of inversion includes:
taking the threshold value W as a basis for acquiring the concave boundary of the wrapped phase matrix, extracting the pixel with the phase value larger than W x phi (max) in the wrapped phase matrix, and setting the pixel area to be 1, wherein phi (max) is the maximum value of the phase of the input phase matrix, namely the pixel area set to be 1 is a concave part, and the rest pixels are set to be 0.
4. The phase unwrapping method according to any one of claims 1-2, wherein the automatically flipping and replacing the depressed portion of the wrapped phase matrix to obtain the first unwrapped phase matrix specifically comprises:
overturning the sunken part of the wrapped phase matrix, and replacing the position of the sunken part in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix;
when the recessed matrix is a non-zero matrix, the recessed matrix is turned, and the position of a recessed position in the current phase matrix is replaced by the recessed matrix obtained after turning to obtain a new phase matrix;
and when the depression matrix is a zero matrix, taking the current phase matrix as a first unwrapped phase matrix and outputting.
5. The phase unwrapping method according to any one of claims 1-2, further comprising, after said acquiring the wrapped phase interferogram:
the wrapped phase interferogram is preprocessed.
6. A phase unwrapping device, comprising:
the acquisition module is used for acquiring a wrapped phase interferogram;
a first determining module for determining a threshold;
the second determining module is used for determining a sunken boundary wrapping the phase matrix through a threshold value to form a turning critical line;
the overturning replacement module is used for automatically overturning and replacing the sunken part of the wrapped phase matrix to obtain a first unwrapped phase matrix;
the processing module is used for carrying out filtering smoothing processing on the first unwrapped phase matrix to obtain a second unwrapped phase matrix;
the calculation module is used for calculating and storing the relative error of the first unwrapped phase matrix and the second unwrapped phase matrix;
the returning module is used for returning to re-determine the threshold value and executing subsequent operation when the threshold value does not reach the preset value;
the first output module is used for outputting a threshold value corresponding to the minimum relative error and a second unwrapped phase matrix when the threshold value reaches a preset value;
and the second output module is used for outputting the reconstructed real phase distribution according to the second unwrapped phase matrix corresponding to the minimum relative error.
7. The phase unwrapping device of claim 6, wherein the flipping module specifically includes:
the first overturning and replacing unit is used for overturning the sunken part of the wrapped phase matrix and replacing the sunken position in the wrapped phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix;
the second overturning and replacing unit is used for overturning the sunken matrix when the sunken matrix is a non-zero matrix, and replacing the position of a sunken part in the current phase matrix with the sunken matrix obtained after overturning to obtain a new phase matrix;
and the output unit is used for taking the current phase matrix as a first unwrapped phase matrix and outputting the first unwrapped phase matrix when the depression matrix is a zero matrix.
8. A phase unwrapping system is characterized by comprising a semiconductor laser, an adjustable attenuator, a beam expander, a pinhole filter, a first collimating lens, a second collimating lens, a phase plate, a third collimating lens, a camera and a computer which are sequentially connected, wherein a target object is placed between the first collimating lens and the second collimating lens, the target object is located at the front focal plane of the second collimating lens, the phase plate is located at the rear focal plane of the second collimating lens, and the camera is located at the rear focal plane of the third collimating lens;
laser beams emitted by the semiconductor laser sequentially pass through the attenuator, the beam expander, the pinhole filter and the first collimating lens to form planar light waves so as to irradiate a target object; obtaining the frequency spectrum of the object light wave on the back focal plane of the second collimating lens, enabling the phase modulation region to be aligned to the zero-frequency region of the frequency spectrum of the object light wave by the phase plate, enabling the light wave passing through the zero-frequency region and the light wave passing through the outside of the zero-frequency region to undergo different phase modulations, generating a wrapped phase interference pattern on the back focal plane of the third collimating lens, collecting the wrapped phase interference pattern by a camera, and inputting the wrapped phase interference pattern into a computer;
the computer configured to perform the phase unwrapping method of any one of claims 1-5.
9. A computer device comprising a processor and a memory for storing processor-executable programs, wherein the processor, when executing a program stored in the memory, implements the phase unwrapping method of any one of claims 1-5.
10. A storage medium storing a program, wherein the program, when executed by a processor, implements the phase unwrapping method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910633802.9A CN110514137B (en) | 2019-07-15 | 2019-07-15 | Phase unwrapping method, device, system, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910633802.9A CN110514137B (en) | 2019-07-15 | 2019-07-15 | Phase unwrapping method, device, system, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110514137A true CN110514137A (en) | 2019-11-29 |
| CN110514137B CN110514137B (en) | 2021-01-08 |
Family
ID=68623209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910633802.9A Expired - Fee Related CN110514137B (en) | 2019-07-15 | 2019-07-15 | Phase unwrapping method, device, system, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110514137B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461224A (en) * | 2020-04-01 | 2020-07-28 | 西安交通大学 | Phase data unwrapping method based on residual self-coding neural network |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000058686A3 (en) * | 1999-03-10 | 2001-01-04 | Veridian Erim International In | Minimum-gradient-path phase unwrapping |
| CN1912541A (en) * | 2006-08-30 | 2007-02-14 | 天津大学 | Phase solution package method base on template in micro-nano structure 3-D contour measuring |
| CN101276323A (en) * | 2008-05-05 | 2008-10-01 | 南京大学 | Minimum discontinuousness two-dimension phase unwrapping method based on phase discontinuousness area detection |
| CN101691997A (en) * | 2009-09-08 | 2010-04-07 | 南昌航空大学 | Method for measuring three-dimensional profile of double-frequency optical grating projection based on rotary splicing |
| CN107202550A (en) * | 2017-06-09 | 2017-09-26 | 北京工业大学 | A kind of method based on least square method Phase- un- wrapping figure |
| CN108955575A (en) * | 2018-08-22 | 2018-12-07 | 江南大学 | A method of corrugated is accurately restored based on single width interference fringe |
-
2019
- 2019-07-15 CN CN201910633802.9A patent/CN110514137B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000058686A3 (en) * | 1999-03-10 | 2001-01-04 | Veridian Erim International In | Minimum-gradient-path phase unwrapping |
| CN1912541A (en) * | 2006-08-30 | 2007-02-14 | 天津大学 | Phase solution package method base on template in micro-nano structure 3-D contour measuring |
| CN101276323A (en) * | 2008-05-05 | 2008-10-01 | 南京大学 | Minimum discontinuousness two-dimension phase unwrapping method based on phase discontinuousness area detection |
| CN101691997A (en) * | 2009-09-08 | 2010-04-07 | 南昌航空大学 | Method for measuring three-dimensional profile of double-frequency optical grating projection based on rotary splicing |
| CN107202550A (en) * | 2017-06-09 | 2017-09-26 | 北京工业大学 | A kind of method based on least square method Phase- un- wrapping figure |
| CN108955575A (en) * | 2018-08-22 | 2018-12-07 | 江南大学 | A method of corrugated is accurately restored based on single width interference fringe |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461224A (en) * | 2020-04-01 | 2020-07-28 | 西安交通大学 | Phase data unwrapping method based on residual self-coding neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110514137B (en) | 2021-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| DE69924129T2 (en) | METHOD AND DEVICE FOR THE SIMULTANEOUS PICTURE OF AMPLITUDE AND QUANTITATIVE PHASE CONTRASTING BY NUMERICAL RECONSTRUCTION OF DIGITAL HOLOGRAMS | |
| Trusiak et al. | Advanced processing of optical fringe patterns by automated selective reconstruction and enhanced fast empirical mode decomposition | |
| US7127109B1 (en) | Digital interference holographic microscope and methods | |
| EP2572157B1 (en) | Method and device for high resolution full field interference microscopy | |
| WO2003050472A1 (en) | Systems and methods for wavefront measurement | |
| WO2004052189A1 (en) | Optical wavefront analyzer | |
| Katkovnik | Phase retrieval from noisy data based on sparse approximation of object phase and amplitude | |
| Huang et al. | Continuous-wave off-axis and in-line terahertz digital holography with phase unwrapping and phase autofocusing | |
| CN110514137B (en) | Phase unwrapping method, device, system, computer equipment and storage medium | |
| CN113393472B (en) | Zoning composite phase unwrapping method based on digital holographic microscopic imaging | |
| CN110619680A (en) | Three-dimensional fault phase microscope reconstruction method based on figure variation | |
| EP3535623B1 (en) | System and method for reconstruction of holographic lens-free images by multi-depth sparse phase recovery | |
| CN210924139U (en) | Image edge enhancement device based on spatial light modulator | |
| Obando-Vásquez et al. | Semi-heuristic phase compensation in digital holographic microscopy for stable and accurate quantitative phase imaging of moving objects | |
| Katkovnik et al. | Computational wavelength resolution for in-line lensless holography: phase-coded diffraction patterns and wavefront group-sparsity | |
| Weng et al. | Design of adaptive spatial filter at uniform standard for automatic analysis of digital holographic microscopy | |
| JP2019078635A (en) | Measuring apparatus, data processor, method for processing data, and program | |
| CN112114501B (en) | Method for calculating Langerhans shearing interference image in photoetching projection objective | |
| Liebling et al. | Nonlinear Fresnelet approximation for interference term suppression in digital holography | |
| WO2021011637A1 (en) | Complex diversity for accurate phase retrieval with single-shot acquisition | |
| US20240029253A1 (en) | Method for acquiring evaluation value | |
| Migukin et al. | Advanced multi-plane phase retrieval using graphic processing unit: augmented Lagrangian technique with sparse regularization | |
| CN115619684B (en) | Microscopic scattering imaging method based on coded camera shooting and non-invasive laser scanning | |
| CN108051184A (en) | A kind of beam-expanding collimation light detection method | |
| CN118672102A (en) | Sample image acquisition method and device and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210108 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |