CN113702567B - Optical layered imaging method and system for high dynamic combustion field - Google Patents
Optical layered imaging method and system for high dynamic combustion field Download PDFInfo
- Publication number
- CN113702567B CN113702567B CN202111003253.0A CN202111003253A CN113702567B CN 113702567 B CN113702567 B CN 113702567B CN 202111003253 A CN202111003253 A CN 202111003253A CN 113702567 B CN113702567 B CN 113702567B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- images
- function
- image
- illuminant
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000002485 combustion reaction Methods 0.000 title claims abstract description 106
- 230000003287 optical effect Effects 0.000 title claims abstract description 84
- 238000003384 imaging method Methods 0.000 title claims abstract description 48
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims description 99
- 238000000034 method Methods 0.000 claims description 72
- 238000001228 spectrum Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 23
- 238000010586 diagram Methods 0.000 claims description 21
- 230000028161 membrane depolarization Effects 0.000 claims description 15
- 238000005070 sampling Methods 0.000 claims description 14
- 230000001360 synchronised effect Effects 0.000 claims description 12
- 239000012535 impurity Substances 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000012634 optical imaging Methods 0.000 abstract description 3
- 230000002999 depolarising effect Effects 0.000 description 13
- 238000009529 body temperature measurement Methods 0.000 description 11
- 230000005855 radiation Effects 0.000 description 4
- 239000007787 solid Substances 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000001069 Raman spectroscopy Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 239000003380 propellant Substances 0.000 description 2
- 239000004449 solid propellant Substances 0.000 description 2
- 230000002269 spontaneous effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000001237 Raman spectrum Methods 0.000 description 1
- 238000002679 ablation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000002844 melting Methods 0.000 description 1
- 230000008018 melting Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000004861 thermometry Methods 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N31/00—Investigating or analysing non-biological materials by the use of the chemical methods specified in the subgroup; Apparatus specially adapted for such methods
- G01N31/12—Investigating or analysing non-biological materials by the use of the chemical methods specified in the subgroup; Apparatus specially adapted for such methods using combustion
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
Abstract
The application relates to an optical layered imaging method and an optical layered imaging system for a high dynamic combustion field, and mainly relates to the field of optical imaging. The application provides an optical layered imaging method of a high dynamic combustion field, which is characterized in that an image of a three-dimensional flame combustion illuminant is divided into images of a plurality of parts, then the images of each part are respectively processed, so that the three-dimensional image of the three-dimensional flame combustion illuminant is converted into a plurality of two-dimensional images, the two-dimensional images are respectively processed through a preset algorithm, marking points are respectively marked in the two-dimensional images, flame combustion images of different parts in the two-dimensional images are obtained through the marking points, and then the flame combustion images are overlapped according to a preset sequence, so that the three-dimensional image of the three-dimensional flame combustion illuminant is obtained, namely, the three-dimensional temperature field of the flame combustion illuminant is reconstructed through the two-dimensional section image of the flame combustion illuminant.
Description
Technical Field
The application relates to the field of optical imaging, in particular to an optical layered imaging method and system of a high dynamic combustion field.
Background
Solid rocket engines are also called solid propellant rocket engines, and the gas temperature of the solid propellant is an important index parameter for rocket engine structural design, ablation prevention design and propellant combustion mechanism research. The propellant has the characteristics of high temperature, quick change and the like when being combusted in the combustion chamber of the engine. The measurement of the flame temperature field of the combustion chamber has important significance for researching the structure and the generation mechanism of the flame, the mixing and burning mode of the fuel and the stable operation and control of the combustion equipment. Therefore, accurate acquisition of the three-dimensional temperature field of the combustion chamber of a solid rocket engine is an urgent need for development of high-temperature-rise combustion chambers.
The current temperature measuring methods can be basically divided into two main types, namely a contact temperature measuring method and a non-contact temperature measuring method. The contact temperature measuring method, such as thermocouple method, has the advantages of high temperature measuring precision and relatively simple experimental system. The non-contact gas temperature measurement method comprises an infrared temperature measurement method, a Raman spectrum temperature measurement method, a multispectral temperature measurement method and the like. The infrared temperature measurement method mainly depends on the infrared temperature measurement principle, and can greatly improve the accuracy of infrared radiation temperature measurement. The multispectral radiation temperature measurement method can be used for simultaneously measuring the real temperature of a target and the spectral emissivity of a material, and can also be used as an important means for multi-parameter dynamic thermophysical property test for measuring the emissivity, the melting point, the specific heat, the thermal expansion and the like of a sample.
However, thermocouple methods are generally only used to measure temperatures below 2200K, and the accuracy of infrared thermometry measurements remains to be detected. The application of spontaneous raman scattering is also limited to many practical systems with bright background and fluorescence interference due to weak and incoherent signals. The multispectral radiation temperature measurement method must assume a functional relation between emissivity and wavelength, otherwise, the functional relation cannot be solved, so a method or device capable of accurately acquiring a three-dimensional temperature field of a combustion chamber of a solid rocket engine is urgently needed.
Disclosure of Invention
The application aims to overcome the defects in the prior art, and provides an optical layered imaging method and system of a high dynamic combustion field, so as to solve the problems that a thermocouple method in the prior art can only be used for measuring the temperature below 2200K, and the accuracy of a measurement result of an infrared temperature measurement method is still to be detected. The application of spontaneous raman scattering is also limited to many practical systems with bright background and fluorescence interference due to weak and incoherent signals. The multispectral radiation temperature measurement method must assume a functional relation between emissivity and wavelength, otherwise, the method cannot be solved, and therefore, a problem of a method or a device capable of accurately acquiring a three-dimensional temperature field of a combustion chamber of a solid rocket engine is urgently needed.
In order to achieve the above purpose, the technical scheme adopted by the embodiment of the application is as follows:
in a first aspect, the present application provides a method of optical layered imaging of a high dynamic combustion field, the method comprising: splitting the light signals emitted by the obtained three-dimensional flame combustion illuminant by using a plurality of broadband depolarization prisms; shooting the split light beams by using a plurality of high-speed cameras respectively; processing the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of a two-dimensional section of each image; and superposing the brightness functions of the two-dimensional sections of each image according to a preset sequence to obtain the image of the three-dimensional flame combustion illuminant.
Optionally, the step of processing the images captured by the plurality of high-speed cameras by using a preset algorithm to obtain a luminance function of the two-dimensional section of each image further includes: calibrating the defocusing degree of a preset calibration plate arranged at a preset position by using a Gaussian defocusing model through a blade edge method; acquiring gray values of pixel points in each calibration image, performing discretization analysis, performing curve fitting of the gray values by using a least square method, and calculating differences between adjacent pixel points to obtain discrete points in a plurality of images; and fitting the discrete points by using a Gaussian curve fitting method to obtain a point spread function in each image.
Optionally, the step of processing the images captured by the plurality of high-speed cameras by using a preset algorithm to obtain a luminance function of the two-dimensional section of each image specifically includes: converting images shot by a plurality of high-speed cameras into gray level images, up-sampling the gray level images, and down-sampling the same scale according to a point spread function corresponding to the gray level images, wherein the expression forms of the images are m multiplied by n matrixes; performing Fourier transform on the gray level diagram by using a two-dimensional Fourier transform formula to obtain a spectrum expression form of the gray level diagram, translating the spectrum, and moving a zero spectrum component to the center of the spectrum; calculating a frequency domain expression function of the two-dimensional section brightness function according to the point spread function, the gray map function and the frequency domain expression equation; and carrying out inverse Fourier transform on the frequency domain expression function of the two-dimensional section brightness function by using a two-dimensional Fourier transform formula to obtain the two-dimensional section brightness function.
Optionally, the step of splitting the acquired light signal emitted by the three-dimensional flame combustion illuminant using a plurality of broadband depolarizing prisms further comprises: burning the obtained three-dimensional flame into a luminous body; the three-dimensional flame combustion illuminant is arranged on a preset optical axis.
Optionally, the step of disposing the three-dimensional flame combustion illuminant on the preset optical axis further includes: the aperture device is arranged on the optical axis, filters natural light and impurity light and is used for transmitting light signals generated by the three-dimensional flame combustion illuminant along a preset direction, wherein the aperture device is provided with a fixed field angle.
Optionally, the step of photographing the split light beams with a plurality of high-speed cameras respectively specifically includes: setting controllers, wherein the controllers are synchronous controllers, and the synchronous signal precision of each controller is set to be less than 20 nanoseconds; the controller controls the plurality of high-speed cameras to shoot the split light beams at the same time respectively.
In a second aspect, the present application provides an optical layered imaging system for a high dynamic combustion field, the system for implementing the optical layered imaging method for a high dynamic combustion field of any one of the first aspects, the system comprising: the system comprises a beam splitting module, a shooting module, a processing module and a superposition module; the beam splitting module uses a plurality of broadband depolarization prisms to split the light signals emitted by the obtained three-dimensional flame combustion illuminant; the shooting module shoots the split light beams respectively by using a plurality of high-speed cameras; the processing module processes the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of a two-dimensional section of each image; and the superposition module is used for superposing the brightness functions of the two-dimensional sections of each image according to a preset sequence to obtain the images of the three-dimensional flame combustion illuminant.
Optionally, the system further comprises a calibration module, wherein the Gaussian defocus model is used for calibrating the defocus degree of a preset calibration plate arranged at a preset position through a blade edge method; obtaining gray values of pixel points in each image, performing curve fitting of the gray values by using a least square method, and calculating differences between adjacent pixel points to obtain discrete points in a plurality of images; and fitting the discrete points by using a Gaussian curve fitting method to obtain the point spread function number in each image.
Optionally, the processing module is specifically configured to: converting images shot by a plurality of high-speed cameras into gray level images, up-sampling the gray level images, and down-sampling the same scale according to a point spread function corresponding to the gray level images, wherein the expression forms of the images are m multiplied by n matrixes; performing Fourier transform on the gray level diagram by using a two-dimensional Fourier transform formula to obtain a spectrum expression form of the gray level diagram, translating the spectrum, and moving a zero spectrum component to the center of the spectrum; calculating a frequency domain expression function of the two-dimensional section brightness function according to the point spread function, the gray map function and the frequency domain expression equation; and carrying out inverse Fourier transform on the frequency domain expression function of the two-dimensional section brightness function by using a two-dimensional Fourier transform formula to obtain the two-dimensional section brightness function.
Optionally, the system further comprises an acquisition module for burning the acquired three-dimensional flame with a luminous body; the three-dimensional flame combustion illuminant is arranged on a preset optical axis.
Optionally, the obtaining module is further configured to use a diaphragm device to set on the optical axis, where the diaphragm device filters natural light and impurity light, and is configured to propagate an optical signal generated by burning the illuminant by three-dimensional flame along a preset direction, and the diaphragm device sets a fixed angle of view.
Optionally, the shooting module is specifically configured to set a controller, where the controller is a synchronous controller, and the synchronous signal precision of each controller is set to be less than 20 nanoseconds; the controller controls the plurality of high-speed cameras to shoot the split light beams at the same time respectively.
The beneficial effects of the application are as follows:
the application provides an optical layered imaging method of a high dynamic combustion field, which comprises the following steps: splitting the light signals emitted by the obtained three-dimensional flame combustion illuminant by using a plurality of broadband depolarization prisms; shooting the split light beams by using a plurality of high-speed cameras respectively; processing the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of a two-dimensional section of each image; according to the method, the images of the three-dimensional flame combustion luminous body are divided into images of a plurality of parts, then the images of the parts are processed respectively, so that the three-dimensional images of the three-dimensional flame combustion luminous body are converted into a plurality of two-dimensional images, the two-dimensional images are processed respectively through a preset algorithm, marking points are marked in the two-dimensional images respectively, flame combustion images of different parts in the two-dimensional images are obtained through the marking points, and then the flame combustion images are overlapped according to the preset sequence, so that the three-dimensional images of the three-dimensional flame combustion luminous body are obtained.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an application scenario of the optical layered imaging method of a high dynamic combustion field provided by the present application;
FIG. 2 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application;
FIG. 3 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application;
FIG. 4 is a graph of an edge diffusion function fit of another high dynamic combustion field optical layered imaging method provided by an embodiment of the application;
FIG. 5 is a plot of the point spread function of another high dynamic combustion field optical layered imaging method provided by an embodiment of the application;
FIG. 6 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application;
FIG. 7 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application;
FIG. 8 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application;
FIG. 9 is a block diagram of an optical layered imaging system for a high dynamic combustion field provided by an embodiment of the application;
FIG. 10 is a block diagram of another high dynamic combustion field optical layered imaging system provided in accordance with an embodiment of the present application;
FIG. 11 is a block diagram of another high dynamic combustion field optical layered imaging system provided in an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made more apparent and fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the application are shown. The components of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. Embodiments of the present application all other embodiments that can be obtained by a person skilled in the art without making any inventive effort fall within the scope of protection of the present application.
FIG. 1 is a schematic view of an application scenario of the optical layered imaging method of a high dynamic combustion field provided by the present application; as shown in fig. 1, the method of the present application can be applied to the electronic device 10, the optical device 20, and the object 30 to be measured shown in fig. 1. As shown in fig. 1, the electronic device 10 may include: a high-speed camera 11, a controller 12, an image processor 13; the optical device 20 may include: an optical axis, a principal axis filter 21, and a broadband depolarizing prism 22; the controller 12 is used for controlling the high-speed camera 11 to shoot, the number of the controller 12 and the high-speed camera 11 are set according to actual needs, generally, if the accuracy of the finally obtained three-dimensional images of the three-dimensional flame combustion illuminant is required to be higher, the number of the controller 12 and the high-speed camera 11 is higher, the image processor 13 is generally set as a computer, software for processing the images is built in the computer, the images are processed according to a preset method, and the controller 12 is respectively in communication connection with the high-speed camera 11 and the image processor 13, and the components can be electrically connected through one or more communication buses or signal wires.
The spindle filter 21 in the optical device 20 is arranged on the optical axis, the broadband depolarizing prism 22 is also arranged on the optical axis, the three-dimensional flame combustion illuminant is equivalent to a light source, and the three-dimensional flame combustion illuminant is also arranged on the light source, wherein the light source, the spindle filter 21 and the broadband depolarizing prism 22 are sequentially arranged from left to right, and the broadband depolarizing prism 22 is arranged according to actual needs.
In addition, the signal transmission sequence of the application is as follows: the optical signals of the three-dimensional flame combustion illuminant are transmitted to the main shaft filtering device 21, the main shaft filtering device 21 is used for filtering other impurity light of the optical signals of the three-dimensional flame combustion illuminant, the filtered impurity light is transmitted to the broadband depolarization prism 22, the plurality of broadband depolarization prisms 22 respectively divide the optical signals into beams, the optical signals passing through the broadband depolarization prism 22 are divided into a plurality of beams of optical signals, the controller 12 controls the plurality of high-speed cameras 11 to acquire image information of a plurality of positions after the beams are divided, the high-speed cameras 11 transmit the acquired image information to the image processor 13, the image processor 13 processes a plurality of images according to a preset function, and the brightness functions of two-dimensional sections of each image are overlapped according to a preset sequence to obtain the images of the three-dimensional flame combustion illuminant.
It is to be understood that the configuration shown in fig. 1 is merely illustrative and that electronic device 10 may also include more or fewer components than those shown in fig. 1 or have a different configuration than that shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
FIG. 2 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application; as shown in fig. 2, the present application provides an optical layered imaging method of a high dynamic combustion field, the method comprising:
s101, using a plurality of broadband depolarization prisms to split the light signals emitted by the obtained three-dimensional flame combustion illuminant.
After the obtained three-dimensional flame burning illuminant is subjected to light beam splitting, the light beam is split by the broadband depolarizing prisms, the broadband depolarizing prisms enable the light generated by the three-dimensional flame burning illuminant to be imaged in different directions, and the quantity of the broadband depolarizing prisms is set according to actual needs without specific limitation. Generally, the more the number of the broadband depolarization prisms is, the clearer and more accurate the three-dimensional image of the three-dimensional flame combustion illuminant is finally obtained, and the broadband depolarization prisms can change the propagation direction of the optical fiber, so that a part of optical signals propagate along the original direction, and the other part of optical signals propagate according to the refraction angle of the broadband depolarization prisms. The specific setting position and setting angle of the broadband depolarization prism are set according to actual needs, and are not particularly limited herein. For convenience of explanation, the broadband depolarizing prisms are disposed at the optical axis, and the broadband depolarizing prisms are parallel to each other, so that a portion of light rays emitted by the three-dimensional flame burning illuminant are refracted while the light rays propagate along the original direction, that is, the three-dimensional flame burning illuminant forms images in the direction of the optical axis and the reflection direction of the broadband depolarizing prisms, and each image represents an image of a different fault of the three-dimensional flame burning illuminant.
S102, shooting the split light beams by using a plurality of high-speed cameras.
The number of the high-speed cameras is equal to the number of light beams split by the broadband depolarization prism, after the broadband depolarization prism splits the light, images of different faults of the three-dimensional flame combustion illuminant exist in a plurality of directions, the controller controls the high-speed cameras to acquire a plurality of fault images, and in practical application, the more the number of the light beams is split, the more the number of the high-speed cameras is, and the clearer and more accurate the three-dimensional images of the three-dimensional flame combustion illuminant are finally obtained; in addition, the synchronization accuracy of the synchronization signals between the controller and the plurality of high-speed cameras is less than 20 nanoseconds, namely, the synchronization signals are equivalent to images of different faults of the three-dimensional flame combustion illuminant shot by the plurality of high-speed cameras, the exposure time, the frame rate and the shooting time of the high-speed cameras are controlled by the synchronization signals generated by the controller through the external trigger interface on the high-speed cameras, and in practical application, the exposure time, the frame rate and the shooting time of the high-speed cameras are set according to practical needs, and are not particularly limited herein.
S103, processing the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of the two-dimensional section of each image.
Because an ideal linear light source or point light source is very difficult to obtain, but for an imaging system, the imaging system is very sensitive to the acquisition of straight edges, an image processing tool is used for image processing by using a straight edge image shot by a computer to obtain gray values of pixels in each row of the image, a least square method is used for curve fitting, the difference between adjacent pixel points is calculated, the obtained discrete points conform to the characteristic of Gaussian distribution, and the distribution of a Point Spread Function (PSF) can be obtained by using Gaussian curve fitting.
FIG. 3 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application; as shown in fig. 3, optionally, the step of processing the images captured by the plurality of high-speed cameras by using a preset algorithm to obtain a luminance function of the two-dimensional section of each image further includes:
s201, calibrating the defocus degree of a preset calibration plate arranged at a preset position by using a Gaussian defocus model through a blade edge method.
Using Gaussian defocus models, i.e.Calibrating the defocus degree in the images shot by each high-speed camera, wherein the middle parameters of the model are respectively represented, x and y are respectively represented by an abscissa, sigma represents the sharpness of the image, sigma is larger, and definition is lower, namely calibrating a preset calibration plate at a preset position by a blade edge method by using a Gaussian defocus model, wherein the preset position is the position where the calibration plate is arranged, the calibration plate is arranged at the position where an image corresponding object is generated, and the preset calibration plate is used for calibrating the image corresponding objectThe degree to which the image is defocused at the time of shooting.
S202, acquiring gray values of pixel points in each image, performing curve fitting of the gray values by using a least square method, and calculating differences between adjacent pixel points to obtain discrete points in a plurality of images.
Dividing each image into rows and columns according to pixels, obtaining pixels in an L multiplied by W area in each image as a target to be detected, wherein L and W are positive integers greater than or equal to 1, and using an algorithmWherein Px is j Average value of gray values of all pixel points in j-th column, and Px ji The j-th column and i-th pixel point are represented, and L and W represent the number of rows and columns of the pixels of the taken image of the region to be detected. Weighting all pixel points of the j-th row of pixels, then taking an average value to obtain a gray value of the row of pixels, and then performing curve fitting on each row of weighted pixel values by using a least square method, wherein fig. 4 is a fitting curve diagram of another optical layered imaging method of a high dynamic combustion field provided by the embodiment of the application; as shown in fig. 4, the graph is an edge diffusion function of the image, and differences between adjacent pixel points are calculated to obtain discrete points in a plurality of images, and the obtained discrete point distribution characteristics conform to gaussian distribution.
The term "gray value" means the depth of color at the point in a black-and-white image, typically ranging from 0 to 255, with 255 being white and 0 being black, so that a black-and-white image is also referred to as a gray image,
and S203, fitting discrete points by using a Gaussian curve fitting method to obtain a point spread function in each image.
Fitting the discrete points after difference by using a Gaussian curve fitting method to obtain a Gaussian model expression of a point spread function conforming to the focal length of the current optical system, wherein FIG. 5 is a diagram of the point spread function of another optical layered imaging method of a high dynamic combustion field provided by the embodiment of the application; as shown in fig. 5, the point spread function of the image is shown, and the function represents the defocus degree of the picture, and the image shot by the high-speed camera is obtained by convolving the point spread function with the object image.
For convenience of explanation, the number of images is exemplified here as two;
setting two layers of images to be reconstructed by optical tomography, the following equation can be obtained according to the obtained image g (x, y, z) and the point spread function h (x, y, z), wherein f (x, y, z) represents the surface to be reconstructed:
a certain optical imaging system can measure the point spread function, and the above formula only has two unknowns f (x 1 ,y 1 ,z 1 ) And f (x) 2 ,y 2 ,z 2 ) I.e. the equation is close-solvable. According to fourier optics theory, the above is transformed into the frequency domain in the form of:
from the above, F (x) 1 ,y 1 ,z 1 ) And F (x) 2 ,y 2 ,z 2 ) Then Fourier transform is carried out to obtain f (x) 1 ,y 1 ,z 1 ) And f (x) 2 ,y 2 ,z 2 ) I.e. the two images yield two-dimensional sections with luminance functions f (x 1 ,y 1 ,z 1 ) And f (x) 2 ,y 2 ,z 2 ) I.e. f (x) 1 ,y 1 ,z 1 ) And f (x) 2 ,y 2 ,z 2 ) Converted into an image.
Because the picture can not be focused perfectly, the picture is defocused to a certain extent, the defocusing accords with a Gaussian defocusing theoretical model, and the point spread function describes the defocusing degree of the picture. The method comprises the following steps: the edge method is used for calibrating the point spread function, because the camera is sensitive to the straight edge, the image obtained by shooting the calibration plate is black and white straight edge (explaining that in the shot calibration image, because defocus is carried out, a gray transition area is arranged between black and white and is from a black area to a white area, the greater the defocus degree is, the wider the transition area is), the gray value of a pixel point in each image is obtained, discretization analysis is carried out, curve fitting is carried out through a least square method, discrete points which accord with Gaussian model distribution are obtained through differentiation, and the point spread function of the image can be obtained through Gaussian curve fitting.
FIG. 6 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application; as shown in fig. 6, specifically, the step of processing the images shot by the plurality of high-speed cameras by using a preset algorithm to obtain a luminance function of a two-dimensional section of each image specifically includes:
s301, converting images shot by a plurality of high-speed cameras into gray level images, up-sampling the gray level images, and down-sampling the same scale according to a point spread function corresponding to the gray level images.
All the images shot by the high-speed cameras are converted into a gray scale image, the gray scale image is up-sampled by an up-sampling method, and down-sampling with the same size is performed on the gray scale image according to the point spread function calculated in the step S203. Specifically, the gray-scale patterns of the plurality of images are each expressed in an m×n matrix, that is, after the images are gray-scale-expressed, the images are each in m rows and n columns of gray-scale patterns.
S302, carrying out Fourier transform on the gray level diagram by using a two-dimensional Fourier transform formula to obtain a spectrum expression form of the gray level diagram, translating the spectrum, and moving a zero spectrum component to the center of the spectrum.
Processing the gray map using a two-dimensional fourier transform formula defining a discrete fourier transform Y of an mxn matrix X;
wherein omega m And omega n Is complex unit root and ω m ＝e -2πi/m ,ω n ＝e -2πi/n ；
i is an imaginary unit. p and j are indexes ranging from 0 to m-1, q and k are indexes ranging from 0 to n-1, a spectrum expression form corresponding to a gray level diagram is obtained through the Fourier transform, and the spectrum is shifted so that a zero spectrum component of the spectrum moves to the center of the spectrum.
S303, calculating a frequency domain expression function of the two-dimensional section brightness function according to the point spread function, the gray map function and the frequency domain expression equation.
And using a frequency domain expression equation, taking the point spread function and the gray map function as known quantities into the frequency domain expression equation for calculation, wherein the obtained calculation structure is the frequency domain expression function of the two-dimensional section brightness function.
S304, performing inverse Fourier transform on the frequency domain expression function of the two-dimensional section brightness function by using a two-dimensional Fourier transform formula to obtain the two-dimensional section brightness function.
The discrete inverse fourier transform X defining the mxn matrix Y is applied using the following formula;
wherein ωm and ωn are complex unit roots:
ω m ＝e -2πi/m ,ω n ＝e -2πi/n
i is an imaginary unit. p and j are indices having values ranging from 0 to m-1, q and k are indices having values ranging from 0 to n-1, and a two-dimensional cross-sectional luminance function is calculated, i.e., the two-dimensional cross-sectional luminance function can represent an image of a three-dimensional flame combustion illuminant at that location or in that direction.
S104, superposing the brightness functions of the two-dimensional sections of each image according to a preset sequence to obtain the images of the three-dimensional flame combustion illuminant.
And converting the brightness function of the two-dimensional section of each image into images, and sequentially superposing the obtained images obtained by converting the brightness function of the two-dimensional section corresponding to each image according to the light transmission sequence and the placement sequence of the high-speed camera, so that the image of the three-dimensional flame combustion illuminant can be obtained.
FIG. 7 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application; as shown in fig. 7, optionally, before the step of splitting the optical signal emitted by the obtained three-dimensional flame combustion illuminant by using a plurality of broadband depolarizing prisms, the method further includes:
s401, burning the obtained three-dimensional flame into a luminous body.
S402, setting the three-dimensional flame combustion illuminant on a preset optical axis.
The flame is ignited, the flame core of the flame is stabilized, the center position of the flame is set on the axis of the optical axis, and a part of light generated by the flame is transmitted according to the direction of the optical axis.
Optionally, the step of disposing the three-dimensional flame combustion illuminant on the preset optical axis further includes:
the aperture device is arranged on the optical axis, filters natural light and impurity light and is used for transmitting light signals generated by the three-dimensional flame combustion illuminant along a preset direction, wherein the aperture device is provided with a fixed field angle.
The diaphragm device is used for limiting the field visual angle, isolating natural light and impurity light in other directions, enabling light generated by the three-dimensional flame combustion illuminant to propagate in the direction of an optical axis, setting the size of the diaphragm and the setting position of the diaphragm according to actual needs, and only needing to filter the natural light and the impurity light by the diaphragm and enabling light signals generated by the three-dimensional flame combustion illuminant to propagate in the preset direction.
FIG. 8 is a flow chart of another method for optical layered imaging of a high dynamic combustion field provided by an embodiment of the application; as shown in fig. 8, optionally, the step of photographing the split light beams by using a plurality of high-speed cameras respectively specifically includes:
s501, setting controllers, wherein the controllers are synchronous controllers, and the synchronous signal precision of each controller is set to be less than 20 nanoseconds.
S502, the controller respectively controls the plurality of high-speed cameras to shoot the split light beams at the same time.
The controller is used for controlling the high-speed cameras to take pictures, the number of the controllers is set according to actual needs, all the high-speed cameras can be controlled by one controller, a plurality of controllers can also be used for controlling a plurality of high-speed cameras, each controller in the plurality of controllers is used for controlling all the high-speed cameras in order to ensure the stability of shooting, a synchronous controller is arranged among the plurality of controllers and used for controlling the shooting time of the high-speed cameras to be less than 20 nanoseconds, and if the number of the controllers is a plurality of controllers, the synchronous controller is also arranged among the plurality of controllers and further controlling the shooting time of the plurality of high-speed cameras to be less than 20 nanoseconds.
FIG. 9 is a block diagram of an optical layered imaging system for a high dynamic combustion field provided by an embodiment of the application; as shown in fig. 9, the present application provides an optical layered imaging system of a high dynamic combustion field, which is used for implementing the optical layered imaging method of the high dynamic combustion field of any one of the above, and the system includes: a beam splitting module 61, a photographing module 62, a processing module 63, and a superimposing module 64; the beam splitting module 61 uses a plurality of broadband depolarizing prisms to split the acquired light signals emitted by the three-dimensional flame combustion illuminant; the photographing module 62 photographs the split light beams using a plurality of high-speed cameras, respectively; the processing module 63 processes the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of a two-dimensional section of each image; the superposition module 64 superimposes the luminance functions of the two-dimensional sections of each image in a preset order to obtain an image of the three-dimensional flame combustion illuminant.
FIG. 10 is a block diagram of another high dynamic combustion field optical layered imaging system provided in accordance with an embodiment of the present application; as shown in fig. 10, the system optionally further includes a calibration module 65 for calibrating the defocus level of the preset calibration plate set at the preset position by the blade edge method using the gaussian defocus model; obtaining gray values of pixel points in each image, performing curve fitting of the gray values by using a least square method, and calculating differences between adjacent pixel points to obtain discrete points in a plurality of images; and fitting the discrete points by using a Gaussian curve fitting method to obtain the point spread function number in each image.
Optionally, the processing module 63 is specifically configured to: converting images shot by a plurality of high-speed cameras into gray level images, up-sampling the gray level images, and down-sampling the same scale according to a point spread function corresponding to the gray level images, wherein the expression forms of the images are m multiplied by n matrixes; performing Fourier transform on the gray level diagram by using a two-dimensional Fourier transform formula to obtain a spectrum expression form of the gray level diagram, translating the spectrum, and moving a zero spectrum component to the center of the spectrum; calculating a frequency domain expression function of the two-dimensional section brightness function according to the point spread function, the gray map function and the frequency domain expression equation; and carrying out inverse Fourier transform on the frequency domain expression function of the two-dimensional section brightness function by using a two-dimensional Fourier transform formula to obtain the two-dimensional section brightness function.
FIG. 11 is a block diagram of another high dynamic combustion field optical layered imaging system provided in accordance with an embodiment of the present application; as shown in fig. 11, the system optionally further comprises an acquisition module 66 for combusting the acquired three-dimensional flame with a light; the three-dimensional flame combustion illuminant is arranged on a preset optical axis.
Optionally, the obtaining module 66 is further configured to use a diaphragm device to set a fixed angle of view, where the diaphragm device is disposed on the optical axis, and the diaphragm device filters out natural light and impurity light and is configured to propagate an optical signal generated by burning the illuminant by the three-dimensional flame along a preset direction.
Optionally, the shooting module 62 is specifically configured to set a plurality of controllers, where each controller is a synchronous controller, and the synchronous signal precision of each controller is set to be less than 20 nanoseconds; the controller controls the plurality of high-speed cameras to shoot the split light beams simultaneously
The present application provides an electronic device including: a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed implements the above-described method of optical layered imaging of a high dynamic combustion field.
The application provides a computer readable storage medium, which comprises a computer program, wherein the computer program controls an electronic device where the computer readable storage medium is located to execute the optical layered imaging method of the high dynamic combustion field when running.
The above is only a preferred embodiment of the present application, and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (8)
1. A method of optical layered imaging of a highly dynamic combustion field, the method comprising:
acquiring a three-dimensional flame combustion illuminant;
setting the three-dimensional flame combustion illuminant on a preset optical axis;
setting a diaphragm device on the optical axis, filtering natural light and impurity light by the diaphragm device, and transmitting an optical signal generated by the three-dimensional flame combustion illuminant along a preset direction, wherein the diaphragm device is provided with a fixed field angle;
splitting the acquired optical signals emitted by the three-dimensional flame combustion illuminant by using a plurality of broadband depolarization prisms;
shooting the split light beams by using a plurality of high-speed cameras respectively;
processing the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of a two-dimensional section of each image;
and superposing the brightness functions of the two-dimensional sections of each image according to a preset sequence to obtain the images of the three-dimensional flame combustion illuminant.
2. The method of claim 1, wherein the step of processing the images captured by the plurality of high-speed cameras using a predetermined algorithm to obtain a luminance function of a two-dimensional cross section of each of the images further comprises:
calibrating the defocusing degree of a preset calibration plate arranged at a preset position by using a Gaussian defocusing model through a blade edge method;
acquiring gray values of pixel points in each calibration image, performing discretization analysis, performing curve fitting of the gray values by using a least square method, and calculating differences between adjacent pixel points to obtain discrete points in a plurality of images;
and fitting the discrete points by using a Gaussian curve fitting method to obtain a point spread function in each image.
3. The method for optical layered imaging of a high dynamic combustion field according to claim 2, wherein the step of processing the images captured by the plurality of high speed cameras using a preset algorithm to obtain a luminance function of a two-dimensional section of each of the images specifically comprises:
converting images shot by a plurality of high-speed cameras into gray level images, up-sampling the gray level images, and down-sampling the same scale according to the point spread function corresponding to the gray level images, wherein the expression forms of the images are m multiplied by n matrixes;
performing Fourier transform on the gray level diagram by using a two-dimensional Fourier transform formula to obtain a spectrum expression form of the gray level diagram, translating the spectrum, and moving a zero spectrum component to the center of the spectrum;
calculating to obtain a frequency domain expression function of a two-dimensional section brightness function according to the point spread function, the gray map function and the frequency domain expression equation;
and carrying out inverse Fourier transform on the frequency domain expression function of the two-dimensional section brightness function by using a two-dimensional Fourier transform formula to obtain the two-dimensional section brightness function.
4. The method for optical layered imaging of a high dynamic combustion field according to claim 3, wherein the step of photographing the split light beams using a plurality of high-speed cameras, respectively, specifically comprises:
setting a controller, wherein the controller is a synchronous controller, and the synchronous signal precision of each controller is set to be less than 20 nanoseconds;
the controller controls a plurality of the high-speed cameras to simultaneously photograph the split light beams.
5. An optical layered imaging system for a high dynamic combustion field for implementing the optical layered imaging method for a high dynamic combustion field of any one of claims 1-4, the system comprising: the system comprises a beam splitting module, a shooting module, a processing module and a superposition module; the beam splitting module uses a plurality of broadband depolarization prisms to split the acquired light signals emitted by the three-dimensional flame combustion illuminant; the shooting module shoots the split light beams respectively by using a plurality of high-speed cameras; the processing module processes the images shot by the high-speed cameras by using a preset algorithm to obtain a brightness function of a two-dimensional section of each image; and the superposition module superposes the brightness functions of the two-dimensional sections of each image according to a preset sequence to obtain the images of the three-dimensional flame combustion illuminant.
6. The optical layered imaging system of a high dynamic combustion field of claim 5, further comprising a calibration module for calibrating the magnitude of defocus of a preset calibration plate disposed at a preset position by a knife-edge method using a gaussian defocus model; obtaining gray values of pixel points in each image, performing curve fitting of the gray values by using a least square method, and calculating differences between adjacent pixel points to obtain a plurality of discrete points in the image; and fitting the discrete points by using a Gaussian curve fitting method to obtain a point spread function in each image.
7. The high dynamic combustion field optical layered imaging system of claim 6, wherein the processing module is specifically configured to: converting images shot by a plurality of high-speed cameras into gray level images, up-sampling the gray level images, and down-sampling the same scale according to the point spread function corresponding to the gray level images, wherein the expression forms of the images are m multiplied by n matrixes; performing Fourier transform on the gray level diagram by using a two-dimensional Fourier transform formula to obtain a spectrum expression form of the gray level diagram, translating the spectrum, and moving a zero spectrum component to the center of the spectrum; calculating to obtain a frequency domain expression function of a two-dimensional section brightness function according to the point spread function, the gray map function and the frequency domain expression equation; and carrying out inverse Fourier transform on the frequency domain expression function of the two-dimensional section brightness function by using a two-dimensional Fourier transform formula to obtain the two-dimensional section brightness function.
8. The high dynamic combustion field optical layered imaging system of claim 7, further comprising an acquisition module for combusting the acquired three-dimensional flame with a luminary; and setting the three-dimensional flame combustion illuminant on a preset optical axis.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111003253.0A CN113702567B (en) | 2021-08-30 | 2021-08-30 | Optical layered imaging method and system for high dynamic combustion field |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111003253.0A CN113702567B (en) | 2021-08-30 | 2021-08-30 | Optical layered imaging method and system for high dynamic combustion field |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113702567A CN113702567A (en) | 2021-11-26 |
| CN113702567B true CN113702567B (en) | 2023-12-15 |
Family
ID=78656685
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111003253.0A Active CN113702567B (en) | 2021-08-30 | 2021-08-30 | Optical layered imaging method and system for high dynamic combustion field |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113702567B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015108917A (en) * | 2013-12-04 | 2015-06-11 | 能美防災株式会社 | Flame detector, and method for determining flame detection candidate area |
| CN105608738A (en) * | 2016-03-04 | 2016-05-25 | 华北电力大学（保定） | Light field camera-based flame three-dimensional photometric field reconstruction method |
| CN105784121A (en) * | 2016-04-01 | 2016-07-20 | 华北电力大学 | Nine-channel adaptive large-range two-dimensional temperature field measuring device and measuring method thereof |
| CN107084794A (en) * | 2017-04-10 | 2017-08-22 | 东南大学 | Flame three-dimensional temperature field measuring system and method based on light field Stratified Imaging technology |
| CN109115348A (en) * | 2018-07-24 | 2019-01-01 | 哈尔滨工业大学 | A kind of three dimensional temperature reconstruction integrated processes based on flame light field refocusing image |
| CN110199190A (en) * | 2017-01-23 | 2019-09-03 | 霍尼韦尔国际公司 | For opening the three-dimensional radiation in burning situation and the device and method of gas material field estimation |
| JP2019196845A (en) * | 2018-05-07 | 2019-11-14 | 一般財団法人電力中央研究所 | Method for observing combustion field, observing device and observing program |
| CN111257497A (en) * | 2020-02-17 | 2020-06-09 | 上海交通大学 | Flame temperature field measuring device and method |
| CN112082672A (en) * | 2020-08-28 | 2020-12-15 | 上海交通大学 | Three-dimensional combustion field temperature measuring method based on extinction correction time analysis |
| JP2021038871A (en) * | 2019-09-02 | 2021-03-11 | 一般財団法人電力中央研究所 | Method of observing combustion field, observation device, and observation program |
-
2021
- 2021-08-30 CN CN202111003253.0A patent/CN113702567B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015108917A (en) * | 2013-12-04 | 2015-06-11 | 能美防災株式会社 | Flame detector, and method for determining flame detection candidate area |
| CN105608738A (en) * | 2016-03-04 | 2016-05-25 | 华北电力大学（保定） | Light field camera-based flame three-dimensional photometric field reconstruction method |
| CN105784121A (en) * | 2016-04-01 | 2016-07-20 | 华北电力大学 | Nine-channel adaptive large-range two-dimensional temperature field measuring device and measuring method thereof |
| CN110199190A (en) * | 2017-01-23 | 2019-09-03 | 霍尼韦尔国际公司 | For opening the three-dimensional radiation in burning situation and the device and method of gas material field estimation |
| CN107084794A (en) * | 2017-04-10 | 2017-08-22 | 东南大学 | Flame three-dimensional temperature field measuring system and method based on light field Stratified Imaging technology |
| JP2019196845A (en) * | 2018-05-07 | 2019-11-14 | 一般財団法人電力中央研究所 | Method for observing combustion field, observing device and observing program |
| CN109115348A (en) * | 2018-07-24 | 2019-01-01 | 哈尔滨工业大学 | A kind of three dimensional temperature reconstruction integrated processes based on flame light field refocusing image |
| JP2021038871A (en) * | 2019-09-02 | 2021-03-11 | 一般財団法人電力中央研究所 | Method of observing combustion field, observation device, and observation program |
| CN111257497A (en) * | 2020-02-17 | 2020-06-09 | 上海交通大学 | Flame temperature field measuring device and method |
| CN112082672A (en) * | 2020-08-28 | 2020-12-15 | 上海交通大学 | Three-dimensional combustion field temperature measuring method based on extinction correction time analysis |
Non-Patent Citations (2)
| Title |
|---|
| Dynamic flame temperature tomography utilizing multi-camera common optical path;Kun Chen 等;Optics and Lasers in Engineering(第162期);第1-10页 * |
| 一种基于立体视觉技术的燃烧诊断方法;熊月飞;黄希桥;李墨祺;郑龙席;;推进技术(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113702567A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107084794B (en) | Flame three-dimensional temperature field measuring system and method based on light field layered imaging technology | |
| US10127687B2 (en) | Calibration device, calibration method, optical device, image-capturing device, projection device, measuring system, and measuring method | |
| CN102494609B (en) | Three-dimensional photographing process based on laser probe array and device utilizing same | |
| CN109115348B (en) | Three-dimensional temperature reconstruction combination method based on flame light field refocusing image | |
| CN112067147B (en) | Method and device for synchronously measuring temperature and deformation | |
| KR20080108424A (en) | Wavefront aberration and distance measurement phase camera | |
| CN111458051B (en) | Three-dimensional temperature field measuring system and method based on pixel-level spectral photodetector | |
| CN104748678A (en) | Method of compensating image quality during high-temperature object measurement | |
| CN109780992A (en) | Interferometer measuration system error calibrating method based on the processing of optical flat stripe pattern | |
| CN113865717B (en) | Transient high-temperature colorimetric temperature measuring device based on high-speed camera | |
| JP4189925B2 (en) | Surface displacement measuring method and surface displacement measuring apparatus | |
| Kelly et al. | A novel multi-band plenoptic pyrometer for high-temperature applications | |
| CN113702567B (en) | Optical layered imaging method and system for high dynamic combustion field | |
| CN113552125B (en) | Ablation rate testing device and method for materials in high-temperature wind tunnel environment | |
| EP3220185B1 (en) | Device and process for the plenoptic capture of images | |
| Wei et al. | 3-D reconstruction algorithm of flame based on inversion calculation of thermal radiation | |
| Hall et al. | Tomographic time resolved laser induced incandescence | |
| Kelly et al. | A novel multi-band plenoptic pyrometer used for temperature measurements of strand burner plumes | |
| CN115077714A (en) | Flame multispectral imaging measurement device, system and method based on optical fiber image transmission bundle | |
| CN114155289A (en) | Electric spark contour dimension measuring method of electric ignition system based on binocular vision | |
| CN106716991B (en) | Infrared pick-up device, image processing method and recording medium | |
| Willers et al. | Optronics sensor development using an imaging simulation system | |
| RU2693532C1 (en) | Method for increasing accuracy of geometrical measurements carried out using a stereoscopic device based on a prism lens optical system | |
| Min et al. | Pattern-projected schlieren imaging method using a diffractive optics element | |
| Hodges et al. | Thermographic Digital Image Correlation (TDIC) Measurements of Mechanically-Loaded Structures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |