CN112346323B - Laminated phase microscope device and method based on DMD digital addressing - Google Patents

Laminated phase microscope device and method based on DMD digital addressing Download PDF

Info

Publication number
CN112346323B
CN112346323B CN202011205799.XA CN202011205799A CN112346323B CN 112346323 B CN112346323 B CN 112346323B CN 202011205799 A CN202011205799 A CN 202011205799A CN 112346323 B CN112346323 B CN 112346323B
Authority
CN
China
Prior art keywords
sample
camera
light
thin lens
diffraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011205799.XA
Other languages
Chinese (zh)
Other versions
CN112346323A (en
Inventor
郑娟娟
雷云泽
郜鹏
温凯
马英
刘旻
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shaanxi Faner Photoelectric Technology Co ltd
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202011205799.XA priority Critical patent/CN112346323B/en
Publication of CN112346323A publication Critical patent/CN112346323A/en
Application granted granted Critical
Publication of CN112346323B publication Critical patent/CN112346323B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03HHOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/0005Adaptation of holography to specific applications
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0032Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/06Means for illuminating specimens
    • G02B21/08Condensers
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03HHOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/0005Adaptation of holography to specific applications
    • G03H2001/005Adaptation of holography to specific applications in microscopy, e.g. digital holographic microscope [DHM]

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Optics & Photonics (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

The invention discloses a laminated phase microscope device and a method based on DMD digital addressing, wherein the device comprises an illumination light source, an attenuation sheet, a first thin lens, a second thin lens, a first reflector, a digital micromirror device, a third thin lens, a fourth thin lens, a second reflector, a fifth thin lens, a first microscope objective, a second microscope objective, a sixth thin lens and a camera which are sequentially arranged along the direction of a light path, wherein a sample is placed between the first microscope objective and the second microscope objective and is positioned on the back focal plane of the first microscope objective, and the camera is arranged at a preset distance from the image plane of the sample; the digital micromirror device is configured to sequentially gate different sub-regions of the target surface to locally illuminate the sample; the camera is used to record a plurality of diffraction patterns of the sample under a plurality of localized illuminations. The device and the method can realize the rapid and high-resolution quantitative phase microscopic imaging of the transparent living biological sample by digitally addressing and illuminating the sample and recording the generated diffraction pattern.

Description

Laminated phase microscope device and method based on DMD digital addressing
Technical Field
The invention belongs to the technical field of microscopic imaging, and particularly relates to a laminated phase microscopic device and method based on DMD digital addressing.
Background
In the biomedical field, most biological samples are transparent or semitransparent under visible light, and the imaging contrast is extremely low under a traditional microscope, so that the biological samples cannot be effectively observed. The quantitative phase microscopy technology utilizes the phase information of object light waves after passing through a transparent sample, not only can improve the imaging contrast, but also can quantitatively obtain the three-dimensional appearance and the refractive index distribution of a microscopic object. Therefore, the research on the novel quantitative phase imaging technology has important value in the fields of biomedicine, industrial detection, gas fluid visualization, self-adaptive imaging and the like.
The phase imaging technology based on optical interference can reproduce the amplitude and phase distribution of a sample by recording a hologram formed by the interference of object light and reference light. The technology has extremely high phase measurement precision. However, in most of the technologies, an optical path structure with separated object and reference light is adopted, and disturbance of the external environment causes different influences on the object light and the reference light respectively, so that a measurement result is very sensitive to environmental vibration. Therefore, it is desirable to study single-beam quantization phase microscopy to overcome the effect of environmental perturbations on phase imaging. Currently, the commonly used single-beam phase imaging techniques are roughly classified into the following categories:
wavefront sensing technology based on microlens arrays or pyramids: the wavefront sensor mainly comprises a two-dimensional micro-lens array and an area array CCD. The microlens array on the sensor divides the wave surface of the incident light into a plurality of sub-wave surfaces. The detection of the sub-wavefronts is achieved by detecting the amount of lateral movement of the focal point of the microlens caused by each sub-wavefront. The wavefront detectors have the advantages of simple structure, good flexibility, large dynamic range, high optical efficiency, no moving part, low requirement on environmental conditions, strong adaptability and the like, and are widely applied to the fields of adaptive optics and quantitative phase microscopy. However, these wavefront sensing methods have the disadvantage that the spatial resolution is limited by the aperture of the micro-lens (typically around 100 μm), which cannot meet the phase imaging requirements for biological samples.
Differential interference microscopy: the object light after microscopic amplification is divided into two parallel parts, and the two parts of light are staggered by a certain distance in a certain direction so as to generate interference. The interference pattern reflects the derivative of the measured phase in the shear direction. Furthermore, by integrating the phase derivatives in two orthogonal directions, the phase distribution of the sample can be obtained. This technique can only measure the phase distribution of a continuous phase object, but cannot measure a step-by-step phase object.
Phase contrast imaging: the sample is illuminated by adopting an annular light source, and the zero-frequency component of the object light wave is subjected to phase delay by adopting an annular phase plate, so that the phase information of the sample is converted into intensity information, and phase contrast microscopy is realized. Because of the nonlinearity between the phase of the measured object and the intensity of the interference pattern, conventional zernike phase contrast imaging can only be used to qualitatively observe the measured object, since it has only a single interferogram.
Single beam phase imaging technique based on diffraction spot recording and iterative reconstruction: the Pedrini reproduces the phase distribution of the sample by combining the intensity images of different defocused planes with an iterative algorithm. Abalone peng quantitatively obtained the phase distribution of the sample by recording the diffraction patterns of the sample under different wavelength illumination. In addition, phase information can be obtained from diffraction images by moving the sub-aperture on a sample plane, turning over the sample, carrying out different phase modulation on object light waves, adopting different illumination directions and structured light illumination to record the obtained diffraction patterns, and combining a similar iterative algorithm.
The technology of the laminated diffraction imaging (PIE) is characterized in that different areas of a sample are irradiated by incident light by transversely moving a light-transmitting small hole (probe) or moving the sample, and meanwhile, the overlapping of a certain area of the irradiated part between every two times of movement is ensured, so that the diffraction patterns of the sample at different illumination areas are recorded; and then, the amplitude and phase distribution of the sample can be reproduced through a corresponding reconstruction algorithm. The technology has the advantages of simple structure, low requirement on light source coherence, large imaging view field, high resolution, high convergence speed and the like. Meanwhile, compared with the traditional iteration method, the method can overcome the problems of stagnation, local convergence, low-frequency component loss and the like of the traditional iteration method.
In recent years, the laminated diffraction imaging technique has been favored and widely used in a plurality of fields such as X-ray imaging, optical microscopy, life science, and the like. However, the existing stacked diffraction imaging technology needs to mechanically move the light-transmitting aperture (probe) laterally or move the sample itself to record diffraction patterns formed when different parts of the sample are illuminated, and often hundreds of diffraction patterns need to be recorded to reproduce a phase image of the sample. Therefore, this technique has not been able to phase image dynamic samples.
Disclosure of Invention
In order to solve the problems in the traditional laminated phase imaging technology, the invention provides a laminated phase microscope device and a method based on DMD digital addressing, which can effectively improve the imaging speed. The technical problem to be solved by the invention is realized by the following technical scheme:
one aspect of the present invention provides a DMD digital addressing-based stacked phase microscope device, comprising an illumination light source, an attenuator, a first thin lens, a second thin lens, a first reflector, a digital micromirror device, a third thin lens, a fourth thin lens, a second reflector, a fifth thin lens, a first microscope objective, a second microscope objective, a sixth thin lens and a camera, which are sequentially arranged along an optical path direction, wherein,
the sample is placed between the first microscope objective and the second microscope objective and is positioned on the back focal plane of the first microscope objective, and the camera is arranged at a preset distance from the sample image plane;
the digital micro-mirror device comprises a two-dimensional array target surface consisting of a plurality of micro-mirrors, the two-dimensional array target surface is pre-divided into a plurality of target surface sub-regions arranged in an array, the adjacent target surface sub-regions have preset overlapping areas, and the digital micro-mirror device is configured to sequentially gate different target surface sub-regions to locally illuminate the sample;
the camera is used for recording a plurality of diffraction patterns of the sample under a plurality of local illuminations.
In one embodiment of the present invention, the stacked phase microscope apparatus further comprises a data processing module for obtaining phase information of the sample by calculation using a plurality of diffraction patterns recorded by a CCD.
In one embodiment of the invention, the target surface sub-regions arranged in the plurality of arrays are circular regions with the same size, and the adjacent circular regions have an overlapping area of more than or equal to 60 percent.
In one embodiment of the invention, the digital micromirror device is further configured to gate a circular area with a binary intensity grating arranged in the center of the two-dimensional array target surface, and record diffraction spots of different diffraction orders by a camera; using the lateral distance between the diffracted spots of different diffraction orders, and the propagation direction of the + -1 st order diffracted light, the defocus distance between the sample image plane and the camera can be calculated.
In one embodiment of the invention, the illumination source is a laser or a partially coherent light source.
Another aspect of the present invention provides a stacked phase microscopy method based on DMD digital addressing, comprising:
s1: obtaining a plurality of diffraction patterns of a sample according to the DMD digitally addressed laminated phase microscopy apparatus of any one of the preceding embodiments;
s2: and calculating and obtaining phase information of the sample according to the plurality of diffraction patterns.
In an embodiment of the present invention, the S1 includes:
s11: turning on an illumination light source, and adjusting an attenuation sheet to enable the light intensity to be suitable for an imaging light path;
s12: the method comprises the following steps of collimating an imaging light path, placing a sample on a back focal plane of a first microscope objective, finding an image plane position through an axial adjusting camera, and moving the camera for a certain preset distance in a direction away from the sample;
s13: sequentially gating different target surface sub-regions of the digital micromirror device to locally illuminate the sample for multiple times;
s14: recording a plurality of diffraction patterns of the sample under the plurality of localized illuminations with the camera.
In an embodiment of the present invention, the S2 includes:
s21: initializing a complex transmittance function of the sample at the image plane:
O1(x,y,z=0)=1,
wherein z represents the optical axis direction between the sample image plane and the camera, z is 0 to represent the position of the image plane, and x and y represent two directions perpendicular to each other in a plane perpendicular to the optical axis direction respectively;
s22: obtaining a first partial illumination P1(x, y) complex amplitude of object light wave at camera position for sample:
Figure BDA0002757052140000051
wherein E is1(x,y,0)=P1(x,y)·O1(x, y,0) denotes the first partial illumination P1(x, y) complex amplitude of the light field at the image plane, z { [ d ] denotes the position of the camera, FT { [ fourier ] denotes the fourier transform, IFT { [ inverse fourier ] denotes the inverse fourier transform operator, ξ and η denote the spatial coordinates of the frequency domain, λ denotes the incident light wavelength, k ═ 2 π/λ denotes the wavenumber of the light wave;
s23: first diffraction pattern I obtained by means of a stacked phase microscopy apparatus1(x, y, d) updating the object wave E at the camera position1(x, y, d), obtaining a new light-field complex amplitude:
Figure BDA0002757052140000052
wherein, I1(x, y, d) is expressed as the intensity distribution of the sample diffraction pattern under the first local illumination;
s24: the light field complex amplitude E obtained in step S231' (x, y, d) is transmitted back to the sample image plane, resulting in a new image plane light field complex amplitude:
Figure BDA0002757052140000053
s25: updating the complex transmittance function of the sample by using a laminated diffraction imaging algorithm:
Figure BDA0002757052140000061
wherein, the upper mark indicates P1The complex conjugate of (x, y), α is the update weight, and ε is a small amount (to avoidZero denominator);
s26: let O be1(x,y,0)=O2(x, y,0) using the second local illumination function P2(x, y) instead of P1(x, y), repeating steps S22 to S25, and sequentially replacing m in the above formula by subscripts m 2,3, …, until all diffraction patterns obtained are updated, and completing one iteration;
s27: continuing to repeat steps S22-S26 using the plurality of diffraction patterns, making a plurality of iterations and calculating a mean square error fSSEWhen f isSSEWhen f is more than 0.01, the iteration is continued when fSSEWhen the complex transmittance is less than or equal to 0.01, completing iteration, and obtaining a current complex transmittance function of the sample as a final complex transmittance function, wherein,
Figure BDA0002757052140000062
wherein E ism(x, y, d) represents the iteratively calculated complex amplitude of the light field on the CCD plane, Im(x, y, d) represents the corresponding diffraction pattern recorded by the camera;
s28: and obtaining phase information of the sample according to the final complex transmittance function of the sample.
In an embodiment of the present invention, before the S2, the method further includes:
a circular area with a binary intensity grating is gated at the center of a two-dimensional array target surface of a digital micromirror device, light spots of different diffraction orders are obtained by a camera, and the defocusing distance d between a sample image surface and the camera is calculated by the light spots of the different diffraction orders.
In one embodiment of the invention, calculating the defocus distance d between the sample image plane and the camera by using the light spots of different diffraction orders comprises:
obtaining the distance l between the center of the plus or minus 1-order diffraction light and the center of the 0-order diffraction light by utilizing light spots of different diffraction orders on a camera;
calculating the period of the grating on the image surface of the sample: the method comprises the following steps of A, obtaining a digital micromirror device, wherein A is the period of loading a grating on the digital micromirror device, and M is the magnification of an imaging system between the digital micromirror device and a sample image surface;
calculating the angle between the center of +1 order diffraction light and the optical axis by utilizing the period of the grating on the image surface of the sample: θ ═ arcsin (λ/Λ');
calculating by using a geometric relation to obtain a defocus distance: d is l/tan (θ) l/tan [ arcsin (λ/Λ') ].
Compared with the prior art, the invention has the beneficial effects that:
1. the laminated phase microscopic device and the method can realize rapid and high-resolution quantitative phase microscopic imaging of the transparent living biological sample by digitally addressing and illuminating the sample and recording the generated diffraction pattern, and can be widely applied to a plurality of fields such as biomedical imaging, industrial detection and the like.
2. The laminated phase microscopic method based on the DMD digital addressing replaces the traditional method for mechanically moving the illumination diaphragm to address by means of the digital micromirror device, realizes digital addressing and illumination of different sub-areas of the sample, and can effectively improve the imaging speed.
3. By loading a circular area with a binary intensity grating on the DMD and recording a diffraction pattern formed by the circular area, the defocus distance of the sample can be digitally obtained, a procedure that the defocus distance needs to be measured in advance in the traditional method is avoided, and convenience of phase imaging is improved.
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Drawings
Fig. 1 is a schematic structural diagram of a stacked phase microscope device based on DMD digital addressing according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of gating different target surface sub-areas of a DMD according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of stacked micro-imaging based on DMD digital addressing provided by the embodiment of the invention;
FIG. 4 is a schematic diagram of calculating a defocus distance between an image plane of a sample and a camera according to an embodiment of the present invention;
fig. 5 is a graph of imaging results obtained using a stacked phase microscopy device based on DMD digital addressing.
Description of reference numerals:
1-an illumination light source; 2-an attenuation sheet; 3-a first thin lens; 4-a second thin lens; 5-a first mirror; 6-digital micromirror device; 7-a third thin lens; 8-a fourth thin lens; 9-a second mirror; 10-a fifth thin lens; 11-a first microscope objective; 12-the sample; 13-a second microscope objective; 14-a sixth thin lens; 15-camera.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, a detailed description is given below of a stacked phase microscope apparatus and a method based on DMD digital addressing according to the present invention with reference to the accompanying drawings and the detailed description thereof.
The foregoing and other technical matters, features and effects of the present invention will be apparent from the following detailed description of the embodiments, which is to be read in connection with the accompanying drawings. The technical means and effects of the present invention adopted to achieve the predetermined purpose can be more deeply and specifically understood through the description of the specific embodiments, however, the attached drawings are provided for reference and description only and are not used for limiting the technical scheme of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or device that comprises a list of elements does not include only those elements but may include other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of additional like elements in the article or device comprising the element.
Example one
Referring to fig. 1 and fig. 2, fig. 1 is a schematic structural diagram of a stacked phase microscope device based on DMD digital addressing according to an embodiment of the present invention; fig. 2 is a schematic diagram of gating different target surface sub-areas of a DMD according to an embodiment of the present invention. The laminated phase microscope Device based on the DMD Digital addressing in this embodiment includes an illumination light source 1, an attenuator 2, a first thin lens 3, a second thin lens 4, a first reflector 5, a DMD (Digital Micromirror Device)6, a third thin lens 7, a fourth thin lens 8, a second reflector 9, a fifth thin lens 10, a first microscope objective 11, a second microscope objective 13, a sixth thin lens 14, and a camera 15, which are sequentially arranged along an optical path direction, wherein a sample 12 is placed between the first microscope objective 11 and the second microscope objective 13 and located at a back focal plane of the first microscope objective 11, and the camera 15 is set to be a predetermined distance from an image plane of the sample. In the present embodiment, the predetermined distance is 40.7 mm. The illumination source 1 may be a laser or a partially coherent light source (e.g. an LED).
In the present embodiment, the illumination light source 1 is a laser, the wavelength is in the visible light range, the output laser power is stable, and the output laser has an appropriate coherence length. The attenuation sheet 2 is a continuously adjustable attenuation sheet. The first thin lens 3 and the second thin lens 4 form a telescope system, and the third thin lens 7 and the fourth thin lens 8 form a telescope system to expand the light beam. The first microscope objective 11 and the fifth thin lens 10 form a beam-shrinking system, and the DMD is imaged on the sample 12 according to a certain beam-shrinking ratio. The second microscope objective 13 and the sixth thin lens 14 form a telescope system, and the telescope system expands and amplifies light waves from the sample 12 to realize microscopic imaging.
The DMD6 includes a two-dimensional array target surface comprised of a plurality of micromirrors pre-divided into a plurality of arrayed target surface sub-regions with adjacent target surface sub-regions having a predetermined overlap area, the DMD6 configured to sequentially gate different target surface sub-regions to locally illuminate the sample 12 a plurality of times. The camera 15 is a CCD camera with appropriate gray scale, pixel size and number of pixels for recording multiple diffraction patterns of the sample 12 under multiple localized illuminations.
Specifically, as shown in fig. 1, laser light emitted from a laser passes through a continuously adjustable attenuator and is used as illumination light, which is expanded and collimated into parallel light by a telescope system composed of a first thin lens 3 and a second thin lens 4. The parallel light is reflected by the reflecting mirror 5 and then irradiates on the target surface of the two-dimensional array of the DMD6, and then the light reflected by the DMD6 is imaged on the surface of the sample 12 by a telescope system consisting of a third thin lens 7 and a fourth thin lens 8 and a beam reduction system consisting of a fifth thin lens 10 and a first microscope objective 11. It should be noted that the DMD is a two-dimensional array of millions of micromirrors arranged on a semiconductor chip, each micromirror forming a pixel, and capable of deflecting under the control of a driving circuit, thereby implementing the function of an optical switch. In the present embodiment, the pixel size of the DMD6 is 1920 × 1080, the size of the picture element is 7.56 μm, and the frame rate is 9253 fps. That is, the DMD of the present embodiment is a two-dimensional array of 1920 × 1080 micromirrors each having a side of 7.56 μm.
When a voltage is applied to a pixel (or micromirror) on the DMD6, the micromirror deflects at an angle (preferably 24 °) (in this case, "on" state), and the incident light reflected by the micromirror exits perpendicularly into the imaging optical path. The mirror with no applied voltage is not deflected (in this case, the "off" state) and reflects incident light in a direction other than the imaging path. The gray scale value of the image loaded on the DMD6 determines the time that the micromirror stays in the "on" state. By sequentially gating the sub-regions at different locations on the DMD6 (making the gray scale value of the region 255 and the other regions 0), the corresponding regions on the sample can be "lit" to achieve the addressing illumination. The sample 12 is magnified and imaged by a telescope system consisting of a second objective 13 and a lens 14 under the illumination of subareas at different positions, and an image surface of the sample appears on a back focal plane of the lens 14. In this example, a CCD was placed 40.7mm from the sample image plane to record the diffraction pattern of the sample under the localized illumination described above.
The target surface of the DMD6 of this embodiment is divided into 19 × 9 circular areas arranged in an array, and every two adjacent circular areas form a certain intersection in the x and y directions, so as to ensure that the two adjacent circular areas overlap by more than or equal to 60%. This ratio is a compromise between the speed of the laminated phase microscopy imaging and the convergence of the algorithm. In a specific imaging process, according to the scanning sequence of the microscope device, a circular area is loaded each time, and the sample is sequentially illuminated at a selected address, as shown in fig. 2, that is, all the micromirrors in the circular area are deflected by 24 ° at the same time, and the incident light reflected by the micromirrors in the circular area is emitted and enters the imaging light path. The CCD recorded the diffraction pattern produced by the sample under illumination in 19 x 9 circular areas. It should be noted that this process uses DMD digital addressing (9253 frames/second) instead of mechanically moving the illumination aperture in conventional stacked diffraction imaging, so that the imaging speed is limited only by the exposure time of the CCD. The CCD of this example recorded the 19 × 9 diffraction patterns using an industrial camera DMK 33UX226 (pixel 4000 × 3000, 30 frames/sec) produced by Imaging Source corporation, and the Imaging took 5.7 seconds.
Further, the stacked phase microscope device further comprises a data processing module for calculating and obtaining phase information of the sample 12 by using a plurality of diffraction patterns recorded by the CCD.
For convenience, the local illumination function corresponding to the mth circular region is denoted as Pm(x, y) and the intensity distribution of the diffraction pattern of the sample under the local illumination of the circular area is recorded as Im(x, y, d), where m is 1,2,3, …,171, indicating the number of circular areas, and d indicates the distance between the sample image plane and the CCD.
In particular, Pm(x, y) can be measured by: when the sample 12 is not placed, the CCD is moved to the image surface of the sample, and the intensity distribution of the local illumination light corresponding to different circular areas of the DMD is recorded one by one, namely the local illumination function P corresponding to different circular areasm(x,y)。
How these 19 x 9 illumination functions P are utilized will be described belowm(x, y) and corresponding diffraction Pattern Im(x, y) to obtain a complex amplitude distribution of the sample.
Referring to fig. 3, fig. 3 is a schematic flow chart of stacked micro-imaging based on DMD digital addressing according to an embodiment of the present invention. The phase calculation method of the present embodiment specifically includes the following steps:
step 1: initializing a complex transmittance function of the sample at the image plane:
O1(x,y,z=0)=1,
wherein z represents the optical axis direction between the sample image plane and the camera, z is 0 to represent the position of the image plane, and x and y represent two directions perpendicular to each other in a plane perpendicular to the optical axis direction respectively;
step 2: obtaining a first partial illumination P1(x, y) complex amplitude of object light wave at camera position for sample:
Figure BDA0002757052140000121
wherein E is1(x,y,0)=P1(x,y)·O1(x, y,0) denotes the first partial illumination P1(x, y) complex amplitude of the light field at the image plane, z ═ d denotes the position of the camera, FT { } denotes fourier transform, IFT { } denotes an inverse fourier transform operator, ξ and η denote the spatial coordinates of the frequency domain, λ denotes the incident light wavelength, and k ═ 2 pi/λ denotes the light wave number;
and step 3: first diffraction pattern I obtained by means of a stacked phase microscopy apparatus1(x, y, d) updating the complex amplitude E of the object light wave at the camera position1(x, y, d), obtaining a new light field complex amplitude:
Figure BDA0002757052140000122
wherein, I1(x, y, d) is expressed as the intensity distribution of the sample diffraction pattern under the first local illumination;
and 4, step 4: the light field complex amplitude E obtained in the step 31' (x, y, d) is transmitted back to the sample image plane, resulting in a new image plane light field complex amplitude:
Figure BDA0002757052140000123
and 5: updating the complex transmittance function of the sample by using a laminated diffraction imaging algorithm:
Figure BDA0002757052140000124
wherein, the upper mark indicates P1Complex conjugate of (x, y), α is the update weight, ε is a minor amount (to avoid denominator being zero);
step 6: let O be1(x,y,0)=O2(x, y,0) using the second local illumination function P2(x, y) instead of P1(x, y), repeating steps S22 to S25 (and sequentially replacing m in the above formula with subscript m 2,3, …) until all 19 × 9 diffraction patterns obtained on the CCD are updated, completing one iteration;
that is, one iteration can be completed by repeating m to 19 × 9, and the current complex transmittance function O of the sample is obtainedm(x,y,0)。
And 7: continuing to repeat steps S22-S26 using the plurality of diffraction patterns, making a plurality of iterations and calculating a mean square error fSSEWhen f isSSEWhen f is more than 0.01, the iteration is continued when fSSEWhen the complex transmittance is less than or equal to 0.01, completing iteration, and obtaining a current complex transmittance function of the sample as a final complex transmittance function, wherein,
Figure BDA0002757052140000131
here, Em(x, y, d) is the iteratively calculated complex amplitude of the light field on the CCD plane, Im(x, y, d) are the corresponding diffraction patterns recorded by the CCD.
It should be noted that: the above iteration-based stack phase retrieval algorithm converges quickly, which is determined by its recording mechanism: the two adjacent circular areas have more than or equal to 60 percent of overlap. Compared with the traditional single-beam phase recovery algorithm, the algorithm is not easy to enter iteration stagnation or converge to a local minimum value. In addition, the algorithm is also different from the conventional PIE reproduction method. Conventional PIE reconstruction methods utilize fourier transforms to compute the propagation of light waves between the object plane and the CCD plane. This requires the defocus distance d to be large enough to satisfy the fraunhofer diffraction condition, which is disadvantageous for miniaturization of the imaging apparatus and also reduces the imaging resolution. The method of the embodiment utilizes the angular spectrum theory to calculate the back-and-forth propagation of the object light wave between the sample image surface and the CCD surface, can greatly reduce the requirement on the defocusing distance, and is favorable for realizing the miniaturization of equipment and high-resolution quantitative phase imaging.
And 8: and obtaining phase information of the sample according to the final complex transmittance function of the sample.
It should be noted that, in the conventional method, the defocus distance d between the image plane of the sample and the CCD plane needs to be measured in advance before the phase information of the sample is reproduced through the above steps. The method of the patent can obtain the defocus distance d by the following operations.
Specifically, the digital micromirror device 6 of this embodiment is further configured to gate a circular area in which a binary intensity grating is arranged at the center of the two-dimensional array target surface to obtain diffraction spots of different diffraction orders with the camera 15, which can be used to calculate the defocus distance d between the sample image plane and the camera 15.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating calculation of a defocus distance between an image plane of a sample and a camera according to an embodiment of the present invention. In order to obtain an accurate defocus distance d, a circular area with binary intensity gratings having gray values of 0 and 255, respectively, and a period of Λ (which can be set to 2-5 times the resolution of the camera) is loaded in the center of the DMD. The circular area is imaged on the surface of the sample, and then is imaged on the image surface of the sample by a telescope system consisting of the second microscope objective 13 and the sixth thin lens 14. Due to the diffraction effect of the grating, the light wave in the circular area is diffracted into a plurality of light beams and propagates along the direction of different diffraction orders. And finally, after the defocusing distance d, the light spots of different diffraction orders are transversely displaced and recorded by the CCD. The distance between the center of the +1 order diffracted light and the center of the 0 order diffracted light, denoted as l, can be obtained from the image of the CCD.
The defocus distance d can be obtained by the following calculation method: firstly, calculating the period Λ' ═ M Λ of the grating on the sample image surface, wherein Λ is the period of loading the circular area grating on the DMD, and M is the period from the DMD to the sample image surfaceMagnification of the image system, which magnification can be measured bym(x, y) are measured in a similar manner; next, the angle between the +1 st order diffracted light and the optical axis is calculated: θ ═ arcsin (λ/Λ'); finally, by using the triangular relationship in fig. 4, the defocus distance can be obtained: d ═ l/tan (θ) ═ l/tan [ arcsin (λ/Λ')]. The accurately obtained d is used for calculating the phase of the sample, so that the phase measurement precision can be improved.
The laminated phase microscope device of the embodiment performs multiple local illumination on a sample through DMD digital addressing and records the generated diffraction pattern, can realize rapid and high-resolution quantitative phase microscope imaging on a transparent living biological sample, has the advantages of high stability, real-time amplitude/phase imaging and the like, and can be widely applied to multiple fields of biomedical imaging, industrial detection and the like.
Example two
On the basis of the above embodiments, the present embodiment provides a stacked phase microscopy method based on DMD digital addressing. The method comprises the following steps:
s1: obtaining a plurality of diffraction patterns of a sample by using the DMD digital addressing-based laminated phase microscope device in the first embodiment;
specifically, an illumination light source is turned on, and the attenuation sheet is adjusted to make the light intensity suitable for an imaging light path, so that an image in the CCD is clear but not saturated; the method comprises the following steps of collimating an imaging light path, placing a sample on a back focal plane of a first microscope objective, finding an image plane position through an axial adjusting camera and moving the camera for a preset distance in a direction far away from the sample; sequentially gating different target surface sub-regions of the digital micromirror device to locally illuminate the sample for multiple times; recording a plurality of diffraction patterns of the sample under the plurality of localized illuminations with the camera.
S2: and calculating and obtaining phase information of the sample according to the plurality of diffraction patterns.
Specifically, the S2 includes:
s21: initializing a complex transmittance function of the sample at the image plane:
O1(x,y,z=0)=1,
wherein z represents the optical axis direction between the sample image plane and the camera, z is 0 to represent the position of the image plane, and x and y represent two directions perpendicular to each other in a plane perpendicular to the optical axis direction respectively;
s22: obtaining a first partial illumination P1(x, y) complex amplitude of object light wave at camera position for sample:
Figure BDA0002757052140000151
wherein E is1(x,y,0)=P1(x,y)·O1(x, y,0) denotes the first partial illumination P1(x, y) complex amplitude of the light field at the image plane, z ═ d denotes the position of the camera, FT { } denotes fourier transform, IFT { } denotes an inverse fourier transform operator, ξ and η denote the spatial coordinates of the frequency domain, λ denotes the incident light wavelength, and k ═ 2 pi/λ denotes the light wave number;
s23: first diffraction pattern I obtained by means of a stacked phase microscopy apparatus1(x, y, d) updating the complex amplitude E of the object light wave at the camera position1(x, y, d), obtaining a new light field complex amplitude:
Figure BDA0002757052140000161
wherein, I1(x, y, d) is expressed as the intensity distribution of the sample diffraction pattern under the first local illumination;
s24: the light field complex amplitude E obtained in step S231' (x, y, d) is transmitted back to the sample image plane, resulting in a new image plane light field complex amplitude:
Figure BDA0002757052140000162
s25: updating the complex transmittance function of the sample by using a laminated diffraction imaging algorithm:
Figure BDA0002757052140000163
wherein, the upper mark indicates P1Complex conjugate of (x, y), α is the update weight, and ε is a constant;
s26: let O be1(x,y,0)=O2(x, y,0) using the second local illumination function P2(x, y) instead of P1(x, y), repeating steps S22 to S25, and sequentially replacing m in the above formula with subscripts m 2,3, … until all 19 × 9 diffraction patterns obtained on the CCD are updated, completing one iteration;
that is, one iteration can be completed by repeating m to 19 × 9, and the current complex transmittance function O of the sample is obtainedm(x,y,0)。
S27: continuing to repeat steps S22-S26 using the plurality of diffraction patterns, making a plurality of iterations and calculating a mean square error fSSEWhen f isSSEWhen f is more than 0.01, the iteration is continued when fSSEWhen the complex transmittance is less than or equal to 0.01, completing iteration, and obtaining a current complex transmittance function of the sample as a final complex transmittance function, wherein,
Figure BDA0002757052140000171
wherein E ism(x, y, d) represents the iteratively calculated complex amplitude of the light field on the CCD plane, Im(x, y, d) represents the corresponding diffraction pattern recorded by the CCD;
s28: and obtaining phase information of the sample according to the final complex transmittance function of the sample.
Further, before the S2, the method further includes:
a circular area with a binary intensity grating is gated at the center of a two-dimensional array target surface of a digital micromirror device, light spots of different diffraction orders are obtained by a camera, and the defocusing distance d between a sample image surface and the camera is calculated by the light spots of the different diffraction orders.
Specifically, calculating the defocus distance d between the sample image plane and the camera by using the light spots of different diffraction orders comprises:
obtaining the distance l between the center of the plus or minus 1-order diffraction light and the center of the 0-order diffraction light by utilizing light spots of different diffraction orders on a camera;
calculating the period of the grating on the image surface of the sample: the method comprises the following steps of A, obtaining a digital micromirror device, wherein A is the period of loading a grating on the digital micromirror device, and M is the magnification of an imaging system between the digital micromirror device and a sample image surface;
calculating the angle between the center of +1 order diffraction light and the optical axis by utilizing the period of the grating on the image surface of the sample: θ ═ arcsin (λ/Λ');
calculating by using a geometric relation to obtain a defocus distance: d is l/tan (θ) l/tan [ arcsin (λ/Λ') ].
Next, the effects of the laminated phase microscope apparatus and method according to the embodiments of the present invention were verified through experiments. In the experiment, the illumination light source 1 was a semiconductor laser, and the wavelength range of light was 473nm ± 5 nm. The pixels of the DMD6 are 1920 × 1080, the pixel size is 7.56 μm, and the frame rate is 9253 fps. The first microscope objective 11 has a magnification of 20X and a numerical aperture NA of 0.4. The second microscope objective had a magnification of 10X and a numerical aperture NA of 0.32. The first thin lens 3 has a focal length f150mm, the second thin lens 4 has a focal length f2150mm, focal length f of the third thin lens 7375mm, the fourth thin lens 8 has a focal length f475mm, the focal length f of the fifth thin lens 105200mm, the sixth thin lens 14 has a focal length f6200 mm. The CCD was an industrial camera DMK 33UX226 (pixels 4000X 3000, 30 frames/sec) from Imaging Source.
In the experiment, phase steps (with the size of 20 μm × 70 μm) are taken as a sample to be measured, and DMD digital addressing illumination is adopted. Referring to fig. 5, fig. 5 is a graph of imaging results obtained using a stacked phase microscope device based on digital addressing of a DMD, in which 171(19 × 8) circular areas (as shown in fig. 5 (a)) are sequentially loaded on the DMD and imaged onto a sample surface. The CCD was synchronized with the DMD, and the diffraction pattern of the sample under different area illumination generated by the DMD was recorded sequentially at a frequency of 30 frames per second (as shown in FIG. 5 (b)). By using a numerical reproduction algorithm of the laminated phase microscope, a light intensity distribution (as shown in fig. 5 (c)) and a phase distribution (as shown in fig. 5 (d)) of the sample can be obtained. By comparing the intensity image and the phase image, we sendNow for this transparent sample, the phase image can clearly reveal the structure of the sample. In addition, the phase can be passed
Figure BDA0002757052140000181
And a thickness dsThe height of the step to be measured is further calculated by the relation of (x, y)
Figure BDA0002757052140000182
Where n issRefractive index of 1.53 sample (glass). It should be noted that the lamination phase microscopic speed of the phase reproduction process is greatly improved by adopting the method of the embodiment of the invention. The laminated phase microscopy method based on the DMD digital addressing in the embodiment replaces mechanical moving of an illumination diaphragm with the aid of a digital micro-mirror device for addressing, realizes digital addressing and illumination of different sub-regions of a sample, and can effectively improve imaging speed.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.

Claims (7)

1. A laminated phase microscope device based on DMD digital addressing is characterized by comprising an illumination light source (1), an attenuation sheet (2), a first thin lens (3), a second thin lens (4), a first reflector (5), a digital micromirror device (6), a third thin lens (7), a fourth thin lens (8), a second reflector (9), a fifth thin lens (10), a first microscope objective (11), a second microscope objective (13), a sixth thin lens (14) and a camera (15), wherein the illumination light source, the attenuation sheet, the first thin lens (3), the second thin lens (4), the first reflector (5), the digital micromirror device, the third thin lens (7), the fourth thin lens (8), the second reflector (9), the fifth thin lens (10), the first microscope objective (11), the second microscope objective (13), the sixth thin lens (14) and the camera (15) are sequentially arranged along the optical path direction, a sample (12) is placed between the first microscope objective (11) and the second microscope objective (13) and is located on a back focal plane of the first microscope objective (11), and the camera (15) is arranged at a predetermined distance from a sample image plane;
the digital micromirror device (6) comprises a two-dimensional array target surface consisting of a plurality of micromirrors, the two-dimensional array target surface is pre-divided into a plurality of array target surface sub-regions, the plurality of array target surface sub-regions are circular regions with the same size, the adjacent circular regions have an overlapping area of more than or equal to 60%, and the digital micromirror device (6) is configured to sequentially gate different target surface sub-regions to locally illuminate the sample (12); the light reflected by the digital micromirror device (6) is imaged on the surface of a sample (12) by a telescope system consisting of the third thin lens (7) and the fourth thin lens (8) and a beam reduction system consisting of the fifth thin lens (10) and the first microscope objective (11);
the camera (15) is used for recording a plurality of diffraction patterns of the sample (12) under a plurality of local illuminations;
the digital micromirror device (6) is further configured to gate a circular area with a binary intensity grating arranged in the center of the two-dimensional array target surface to obtain diffraction spots of different diffraction orders with the camera (15); using the lateral distance between the diffracted spots of different diffraction orders, and the propagation direction of the + -1 st order diffracted light, the defocus distance between the sample image plane and the camera (15) can be calculated.
2. The DMD digitally addressed laminated phase microscopy apparatus as claimed in claim 1, further comprising a data processing module for computationally obtaining phase information of said sample (12) using a plurality of diffraction patterns recorded by a camera.
3. The DMD digital addressing-based stacked phase microscopy device according to claim 1, characterized in that the illumination light source (1) is a laser or a partially coherent light source.
4. A laminated phase microscopy method based on DMD digital addressing is characterized by comprising the following steps:
s1: obtaining a plurality of diffraction patterns of a sample according to the DMD digital addressing-based stacked phase microscopy device of any one of claims 1 to 3;
s2: calculating phase information of the sample according to the plurality of diffraction patterns,
the S2 includes:
s21: initializing a complex transmittance function of the sample at the image plane:
O1(x,y,z=0)=1,
wherein z represents the optical axis direction between the sample image plane and the camera, z is 0 to represent the position of the image plane, and x and y represent two directions perpendicular to each other in a plane perpendicular to the optical axis direction respectively;
s22: obtaining a first partial illumination P1(x, y) complex amplitude of object light wave at camera position for sample:
Figure FDA0003227776860000021
wherein E is1(x,y,0)=P1(x,y)·O1(x, y,0) denotes the first partial illumination P1(x, y) complex amplitude of the light field at the image plane, z ═ d denotes the position of the camera, FT { } denotes fourier transform, IFT { } denotes an inverse fourier transform operator, ξ and η denote the spatial coordinates of the frequency domain, λ denotes the incident light wavelength, and k ═ 2 pi/λ denotes the light wave number;
s23: first diffraction pattern I obtained by means of a stacked phase microscopy apparatus1(x, y, d) updating the complex amplitude E of the object light wave at the camera position1(x, y, d), obtaining a new light field complex amplitude:
Figure FDA0003227776860000022
wherein, I1(x, y, d) is expressed as the intensity distribution of the sample diffraction pattern under the first local illumination;
s24: the light field complex amplitude E obtained in step S231' (x, y, d) is transmitted back to the sample image plane, resulting in a new image plane light field complex amplitude:
Figure FDA0003227776860000031
s25: updating the complex transmittance function of the sample by using a laminated diffraction imaging algorithm:
Figure FDA0003227776860000032
wherein, the upper mark indicates P1Complex conjugate of (x, y), α is the update weight, and ε is a constant;
s26: let O be1(x,y,0)=O2(x, y,0) using the second local illumination function P2(x, y) instead of P1(x, y), repeating the steps S22 to S25 until all the obtained diffraction patterns are updated, and completing one iteration;
s27: continuing to repeat steps S22-S26 using the plurality of diffraction patterns, making a plurality of iterations and calculating a mean square error fSSEWhen f isSSEWhen f is more than 0.01, the iteration is continued when fSSEWhen the complex transmittance is less than or equal to 0.01, completing iteration, and obtaining a current complex transmittance function of the sample as a final complex transmittance function, wherein,
Figure FDA0003227776860000033
wherein E ism(x, y, d) denotes the iteratively calculated complex amplitude of the light field in the camera plane, Im(x, y, d) represents the corresponding diffraction pattern recorded by the camera;
s28: and obtaining phase information of the sample according to the final complex transmittance function of the sample.
5. The DMD digital addressing-based stacked phase microscopy method of claim 4, wherein the S1 comprises:
s11: turning on an illumination light source, and adjusting an attenuation sheet to enable the light intensity to be suitable for an imaging light path;
s12: the method comprises the following steps of collimating an imaging light path, placing a sample on a back focal plane of a first microscope objective, finding an image plane position through an axial adjusting camera, and moving the camera for a certain preset distance in a direction away from the sample;
s13: sequentially gating different target surface sub-regions of the digital micromirror device to locally illuminate the sample for multiple times;
s14: recording a plurality of diffraction patterns of the sample under the plurality of localized illuminations with the camera.
6. The DMD digital addressing-based stacked phase microscopy method of claim 4, further comprising, prior to said S2:
a circular area with a binary intensity grating is gated at the center of a two-dimensional array target surface of a digital micromirror device, light spots of different diffraction orders are obtained by a camera, and the defocusing distance d between a sample image surface and the camera is calculated by the light spots of the different diffraction orders.
7. The DMD digital addressing-based stacked phase microscopy method of claim 6, wherein calculating a defocus distance d between a sample image plane and the camera using the light spots of different diffraction orders comprises:
obtaining the distance l between the center of the plus or minus 1-order diffraction light and the center of the 0-order diffraction light by utilizing light spots of different diffraction orders on a camera;
calculating the period of the grating on the image surface of the sample: the method comprises the following steps of A, obtaining a digital micromirror device, wherein A is the period of loading a grating on the digital micromirror device, and M is the magnification of an imaging system between the digital micromirror device and a sample image surface;
calculating the angle between the center of +1 order diffraction light and the optical axis by utilizing the period of the grating on the image surface of the sample: θ ═ arcsin (λ/Λ');
calculating by using a geometric relation to obtain a defocus distance: d is l/tan (θ) l/tan [ arcsin (λ/Λ') ].
CN202011205799.XA 2020-11-02 2020-11-02 Laminated phase microscope device and method based on DMD digital addressing Active CN112346323B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011205799.XA CN112346323B (en) 2020-11-02 2020-11-02 Laminated phase microscope device and method based on DMD digital addressing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011205799.XA CN112346323B (en) 2020-11-02 2020-11-02 Laminated phase microscope device and method based on DMD digital addressing

Publications (2)

Publication Number Publication Date
CN112346323A CN112346323A (en) 2021-02-09
CN112346323B true CN112346323B (en) 2021-09-28

Family

ID=74355466

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011205799.XA Active CN112346323B (en) 2020-11-02 2020-11-02 Laminated phase microscope device and method based on DMD digital addressing

Country Status (1)

Country Link
CN (1) CN112346323B (en)

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
《快速扫描PIE成像技术研究》;孙瑷蕙;《中国优秀硕士学位论文全文数据库(电子期刊)工程科技Ⅱ辑》;20190131;第2.1节,3.1-3.2节 *
《连续太赫兹波双物距叠层定量相衬成像》;王大勇;《物理学报》;20200208;第2节 *
孙瑷蕙.《快速扫描PIE成像技术研究》.《中国优秀硕士学位论文全文数据库(电子期刊)工程科技Ⅱ辑》.2019,第2.1节,3.1-3.2节. *

Also Published As

Publication number Publication date
CN112346323A (en) 2021-02-09

Similar Documents

Publication Publication Date Title
US5291012A (en) High resolution optical microscope and irradiation spot beam-forming mask
US7812959B1 (en) Total internal reflection holographic microscope
KR100704136B1 (en) Three-dimensional imaging system
US9719859B2 (en) Interferometer and phase shift amount measuring apparatus with diffraction gratings to produce two diffraction beams
US20100135547A1 (en) Optical sectioning microscopy
US9316536B2 (en) Spatial frequency reproducing apparatus and optical distance measuring apparatus
JP5951451B2 (en) Light irradiation device, microscope device, and laser processing device
US20120082943A1 (en) Diffractive photo masks and methods of using and fabricating the same
JPWO2019044336A1 (en) Holographic imaging device and data processing method used for the device
US7445873B2 (en) Lithograph with beam guidance and method of producing digital holograms in a storage medium
US11561389B2 (en) High spatial and temporal resolution synthetic aperture phase microscopy (HISTR-SAPM)
CN114076670A (en) Splicing main mirror common-phase error detection method and system and storage medium
US7471430B1 (en) Holographic image corrector
CN112346323B (en) Laminated phase microscope device and method based on DMD digital addressing
CN110927116B (en) Method, device and system for measuring mark structure
JP6230358B2 (en) Optical distance measuring device
CN114324245B (en) Quantitative phase microscopic device and method based on partially coherent structured light illumination
CN113654656B (en) Light beam drift detection device and method based on three-light-beam interference
JPH08313205A (en) Oblique-incidence interferometer equipment
US20120038911A1 (en) Defect detection apparatus
Emery et al. Digital holography microscopy (DHM): fast and robust systems for industrial inspection with interferometer resolution
CN112925091A (en) Apparatus and method for estimating value from image
TWI673516B (en) Illumination apparatus and system for multiplexed multi-depth confocal microscopy
JP3958099B2 (en) Holographic device
JP6259491B2 (en) Light irradiation apparatus, microscope apparatus, laser processing apparatus, and light irradiation method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20211230

Address after: 710000 room F2001, 20 / F, block 4-A, Xixian financial port, Fengdong new city, Xixian New District, Xi'an City, Shaanxi Province

Patentee after: Shaanxi faner Photoelectric Technology Co.,Ltd.

Address before: No.2, Taibai South Road, Yanta District, Xi'an City, Shaanxi Province

Patentee before: XIDIAN University

TR01 Transfer of patent right