WO2020192235A1

WO2020192235A1 - Two-photon fluorescence imaging method and system, and image processing device

Info

Publication number: WO2020192235A1
Application number: PCT/CN2019/130895
Authority: WO
Inventors: 高玉峰; 郑炜
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2019-03-28
Filing date: 2019-12-31
Publication date: 2020-10-01
Also published as: CN110006861B; CN110006861A

Abstract

Provided are a two-photon fluorescence imaging method and system, and an image processing device, which are applicable to the technical field of two-photon fluorescence. The method comprises: performing wavefront shaping on two incident pulsed light beams in sequence to obtain two light beams, light fields of which at a focal point of an objective lens are gradient light fields, and making light intensity distributions of the gradient light fields of the two light beams be opposite; and then scanning a sample to be tested through the two light beams sequentially to obtain two images, and performing three-dimensional data reconstruction according to the two images to obtain a three-dimensional depth image of the sample to be tested. According to the method, the imaging speed is high, the resolution is high, and the algorithm is simple, and the present application is easily implemented; furthermore, since the imaging speed is high, light damage and a photobleaching effect on the sample to be tested are effectively reduced, and the present application is especially applicable to performing three-dimensional imaging on embryonic development samples and neural activity samples.

Description

Two-photon fluorescence imaging method, system and image processing equipment

Technical field

The invention belongs to the technical field of two-photon fluorescence, and in particular relates to a two-photon fluorescence imaging method, system and image processing equipment.

Background technique

The existing two-photon fluorescence microscope mainly excites the fluorescence signal only at the focal point with the highest energy through the nonlinear effect, provides optical sectioning capability, and can only image a certain depth position of the sample at a time. To achieve imaging of a large-volume three-dimensional area of a sample, the following three technologies are usually used:

Technique 1: With the help of a stepper motor or a zoom lens, the focus can be moved axially along the depth direction of the sample. This method has a slower imaging speed;

Technique 2: Use the characteristic of axial elongated focus of Bessel beam to scan the large-volume three-dimensional area of the sample, this method lacks axial position information;

Technology 3: Design the focus of the two incident beams into a V shape, so that the same fluorescent signal has two corresponding positions in the image. The distance between these two corresponding positions is related to the axial position of the fluorescent signal, so that the The axial position information of the fluorescence signal is converted into the lateral position information of two corresponding positions to realize three-dimensional imaging. In this way, in order to realize the V-shaped focus, each incident beam only uses less than 1/4 of the numerical aperture. The rate is very poor, and complex algorithms are needed for location matching.

Summary of the invention

In view of this, the embodiments of the present invention provide a two-photon fluorescence imaging method, system and image processing equipment to solve the problems of slow imaging speed, poor resolution, and complex algorithms in the prior art for imaging a large-volume three-dimensional area of a sample .

The first aspect of the embodiments of the present invention provides a two-photon fluorescence imaging method, which is applied to a two-photon fluorescence imaging system, and the two-photon fluorescence imaging method includes:

Wavefront shaping is performed on the incident first pulsed light beam to obtain the first light beam; wherein the light field of the first light beam at the focal point of the objective lens of the two-photon fluorescence imaging system is a first gradient light field, and The light intensity distribution of a gradient light field changes in a gradient along the z-axis, and the z-axis direction is the depth direction of the sample to be tested;

Scan the sample to be tested by the first light beam to obtain a first image;

Wavefront shaping is performed on the incident second pulsed light beam to obtain a second light beam; wherein the light field of the second light beam at the focal point of the objective lens is the first light intensity distribution opposite to the light intensity distribution of the first gradient light field Two gradient light fields;

Scan the sample to be tested by the second light beam to obtain a second image;

Perform three-dimensional data reconstruction according to the first image and the second image to obtain a three-dimensional depth image of the sample to be tested.

In an embodiment, performing wavefront shaping on the incident first pulse beam to obtain the first beam includes:

Rotating the polarization direction of the incident first pulsed beam to obtain a first polarized beam with a preset polarization direction;

Performing spatial phase modulation on the first polarized light beam to obtain a first light beam with a first preset phase;

The wavefront shaping of the incident second pulse beam to obtain the second beam includes:

Rotating the polarization direction of the incident second pulse beam to obtain a second polarization beam with a preset polarization direction;

Performing spatial phase modulation on the second polarized light beam to obtain a second light beam with a second preset phase.

In one embodiment, before the wavefront shaping of the incident first pulse beam to obtain the first beam, the method includes:

Design a first focal point with light intensity distributed along the z axis; wherein the light field at the first focal point is a first gradient light field;

Design a second focal point with light intensity distributed along the z-axis; wherein the light field at the second focal point is a second gradient light field;

Dividing the entrance pupil of the objective lens into a preset number of rings of equal area;

Combining the phases of the preset number of rings with equal areas as the phase function of the entrance pupil;

Calculating the light field at the focal point corresponding to the phase function of each ring according to the phase function of the entrance pupil;

Using a genetic algorithm to find the phase corresponding to the light field most similar to the first gradient light field as the first preset phase;

A genetic algorithm is used to find the phase corresponding to the light field most similar to the second gradient light field as the second preset phase.

In an embodiment, according to the phase function, the calculation formula for calculating the light field at the focal point corresponding to each phase function is:

Where is the convergence angle of light, P() is the phase function of the entrance pupil, which is the maximum value,

k=2π/λ is the wave number, NA is the numerical aperture of the objective lens, n is the immersion medium of the objective lens, λ is the wavelength of the light, and z is the z-axis position information.

In an embodiment, performing three-dimensional data reconstruction on the sample to be tested based on the first image and the second image to obtain a three-dimensional depth image of the sample to be tested includes:

Performing maximum normalization processing on the first image and the second image respectively;

Performing intensity information extraction on the first image and the second image after the maximum value normalization processing is performed to obtain an intensity image;

Setting an intensity threshold to extract intensity information from the intensity image to obtain an effective information area;

Extracting the z-axis position information of the effective information area to obtain a position matrix with the same pixel size as the intensity image;

Sorting the z-axis position information in the position matrix, and selecting a value range according to the sorting result to normalize the position matrix;

The intensity information in the intensity image and the z-axis position information in the normalized position matrix are encoded into the same image to obtain a three-dimensional depth image of the sample to be tested; wherein, the Z-axis The position information is coded as different colors, and the intensity information is coded as color saturation.

The second aspect of the embodiments of the present invention provides a two-photon fluorescence imaging system, including:

The wavefront shaping module is used to shape the wavefront of the incident first pulse beam to obtain the first beam;

The photon fluorescence microscope is used to scan the sample to be tested through the first light beam to obtain a first image; wherein the light field of the first light beam at the focal point of the objective lens of the two-photon fluorescence microscope is a first gradient light field , The light intensity distribution of the first gradient light field changes in a gradient along the z-axis, and the z-axis direction is the depth direction of the sample to be tested;

The wavefront shaping module is also used to perform wavefront shaping on the incident second pulsed light beam to obtain a second light beam; wherein the light field of the second light beam at the focal point of the objective lens is the same as that of the first A second gradient light field with an opposite light intensity distribution of the gradient light field;

The two-photon fluorescence microscope is also used to scan the sample to be tested through the second beam to obtain a second image;

The image processing device is used to perform three-dimensional data reconstruction according to the first image and the second image to obtain a three-dimensional depth image of the sample to be tested.

In an embodiment, the wavefront shaping module includes:

The half-wave plate is used to rotate the polarization direction of the incident first pulsed beam to obtain the first polarized beam with the preset polarization direction;

A spatial light modulator, configured to perform spatial phase modulation on the first polarized light beam to obtain a first light beam with a first preset phase;

The half-wave plate is also used to rotate the polarization direction of the incident second pulsed light beam to obtain a second polarized light beam with a preset polarization direction;

The spatial light modulator is also used to perform spatial phase modulation on the second polarized light beam to obtain a second light beam with a second preset phase.

In an embodiment, the image processing device is in communication connection with the spatial light modulator, and the image processing device includes:

The first design module is used to design a first focal point with light intensity distributed along the z axis; wherein the light field at the first focal point is a first gradient light field;

The second design module is used to design a second focal point where the light intensity is distributed along the z axis; wherein the light field at the second focal point is a second gradient light field;

A dividing module, configured to divide the entrance pupil of the objective lens into a preset number of rings of equal area;

A combination module, configured to combine the phases of the preset number of rings with the same area as the phase function of the entrance pupil;

A calculation module, configured to calculate the light field at the focal point corresponding to the phase function of each ring according to the phase function of the entrance pupil;

The first searching module is configured to use a genetic algorithm to find the phase corresponding to the light field most similar to the first gradient light field, as the first preset phase and output to the spatial light modulator;

The second searching module is used to search for the phase corresponding to the light field most similar to the second gradient light field by using a genetic algorithm, as a second preset phase and output to the spatial light modulator.

In an embodiment, the image processing device includes:

A normalization processing module, configured to perform maximum normalization processing on the first image and the second image respectively;

The first extraction module is configured to extract intensity information of the first image and the second image after the maximum value normalization processing is performed to obtain an intensity image;

The second extraction module is configured to set an intensity threshold to extract intensity information from the intensity image to obtain an effective information area;

The third extraction module is configured to extract the z-axis position information of the effective information area to obtain a position matrix with the same pixel size as the intensity image;

A sorting module, configured to sort the z-axis position information in the position matrix, and select a value range according to the sorting result to normalize the position matrix;

An encoding module for encoding the intensity information in the intensity image and the z-axis position information in the normalized position matrix into the same image to obtain a three-dimensional depth image of the sample to be tested; wherein The z-axis position information is coded as different colors, and the intensity information is coded as color saturation.

The third aspect of the embodiments of the present invention provides an image processing device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor executes the computer program. Realize the functions of the above-mentioned modules when programming.

A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the functions of the foregoing modules are realized.

In the embodiment of the present invention, the wavefront shaping of the two incident pulsed light beams is performed sequentially to obtain two light beams with a gradient light field at the focal point of the objective lens, and the light intensity distributions of the gradient light fields of the two light beams are opposite. , And then scan the sample under test through two light beams to obtain two images, and reconstruct the three-dimensional data according to the two images to obtain the three-dimensional depth image of the sample under test. The imaging speed is fast, the resolution is high, the algorithm is simple, and it is easy to implement. And because of the fast imaging speed, the photodamage and photobleaching effects of the sample to be tested are effectively reduced, and it is especially suitable for three-dimensional imaging of embryonic development samples and neural activity samples.

Description of the drawings

In order to explain the technical solutions in the embodiments of the present invention more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present invention. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a two-photon fluorescence imaging method according to Embodiment 1 of the present invention;

2 is a schematic diagram of gradient light field imaging according to Embodiment 1 of the present invention;

3 is a schematic flow chart of a two-photon fluorescence imaging method provided by Embodiment 2 of the present invention;

4 is a schematic flowchart of a method for calculating the phase of a gradient light field according to the second embodiment of the present invention;

5-7 are the light intensity distributions of the first gradient light field and the second gradient light field provided by the second embodiment of the present invention;

FIG. 8 is a schematic flowchart of a genetic algorithm provided by Embodiment 2 of the present invention;

9 and 10 are schematic flowcharts of a three-dimensional data reconstruction method provided by Embodiment 3 of the present invention;

FIG. 11 is a schematic structural diagram of a two-photon fluorescence imaging system provided by Embodiment 4 of the present invention;

FIG. 12 is a schematic structural diagram of a wavefront shaping module provided by Embodiment 4 of the present invention;

FIG. 13 is a schematic structural diagram of an image processing device provided by Embodiment 5 of the present invention.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are the present invention. Part of the embodiment, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

The term "comprising" in the description and claims of the present invention and the above-mentioned drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally includes Other steps or units inherent in these processes, methods, products or equipment. In addition, the terms "first", "second", and "third" are used to distinguish different objects, rather than describing a specific order.

Example one

This embodiment provides a two-photon fluorescence imaging method, which is applied to a two-photon fluorescence imaging system. The two-photon fluorescence imaging system includes a wavefront shaping module, a two-photon fluorescence microscope, and image processing equipment.

In specific applications, the wavefront shaping module can include a spatial light modulator that modulates the phase of the wavefront, and the image processing equipment can be a fully automatic or semi-automatic microscope photography device that comes with a two-photon fluorescence microscope, or it can include an image sensor and Computing equipment such as desktop computers, notebooks, palmtop computers and cloud servers.

As shown in Fig. 1, the two-photon fluorescence imaging method provided by this embodiment includes:

Step S101: Perform wavefront shaping on the incident first pulsed light beam to obtain the first light beam; wherein the light field of the first light beam at the focal point of the objective lens of the two-photon fluorescence imaging system is a first gradient light field, The light intensity distribution of the first gradient light field changes in a gradient along the z-axis, and the z-axis direction is the depth direction of the sample to be tested.

In specific applications, a laser can be used to generate the first pulse beam, and the wavefront shaping module can perform wavefront shaping on the incident first pulse beam to modulate the phase of the first pulse beam to obtain a first pulse beam with a specific phase. The light beam, when the first light beam reaches the focal point of the objective lens of the two-photon fluorescence imaging system, forms a first gradient light field with a gradient of light intensity distribution along the z-axis. Define the depth direction of the sample to be tested as the z-axis direction, the width direction of the sample to be tested as the x-axis direction, and the length direction of the sample to be tested as the y-axis direction. The x-axis, y-axis and z-axis directions can also be customized according to actual needs. Definition, as long as it follows the definition rules of the Cartesian coordinate system.

Step S102: Scan the sample to be tested by the first light beam to obtain a first image.

In a specific application, the first light beam can be emitted to the sample under test by a two-photon fluorescence microscope, and then the fluorescence excited by the area at the focus of the objective lens in the sample under test and the beam reflected by the sample under test can be obtained by the image processing device. Perform imaging to get the first image. The light intensity distribution of the pixels in the first image in the z-axis direction is the same as the first gradient light field.

Step S103, performing wavefront shaping on the incident second pulsed light beam to obtain a second light beam; wherein the light field of the second light beam at the focal point of the objective lens is the same as the light intensity distribution of the first gradient light field Opposite the second gradient light field.

In specific applications, the second pulse beam can be generated by a laser, and the incident second pulse beam can be wavefront-shaped by the wavefront shaping module to modulate the phase of the second pulse beam to obtain a second pulse beam with a specific phase. Two beams, so that when the second beam reaches the focal point of the double objective lens, it forms a second gradient light field with a gradient of light intensity distribution along the z-axis, and the light intensity distribution of the second gradient light field is opposite to that of the first gradient light field , That is, the gradient change of the first light field in the positive z-axis direction is the same as the gradient change of the second light field in the negative z-axis direction. For example, assuming that the light intensity distribution of the first gradient light field changes from strong to weak along the positive z-axis direction, the light intensity distribution of the second gradient light field changes from weak to strong along the positive z-axis direction, and vice versa.

Step S104: Scan the sample to be tested by the second light beam to obtain a second image.

In specific applications, the second light beam can be emitted to the sample under test through a two-photon fluorescence microscope, and then the fluorescence excited by the area at the focal point of the objective lens in the sample under test and the beam reflected by the sample under test can be obtained by the image processing device. Perform imaging to obtain a second image. The light intensity distribution of the pixels in the second image in the z-axis direction is the same as the second gradient light field.

Step S105: Perform three-dimensional data reconstruction according to the first image and the second image to obtain a three-dimensional depth image of the sample to be tested.

In specific applications, three-dimensional data reconstruction of the first image and the second image can be performed by the image processing equipment, mainly through the ratio R of the first image and the second image, which reflects the position of each position point in the sample to be tested. Then through the corresponding relationship between R and the z-axis position information z, the z-axis position information of each position point in the sample to be tested is obtained, so as to know the position of each position point in the sample to be tested in the z-axis direction, and then obtain the sample Three-dimensional depth image.

As shown in FIG. 2, it exemplarily shows a gradient light field imaging principle diagram that uses the first gradient light field and the second gradient light field to obtain a three-dimensional gradient image of the sample to be tested.

Fig. 2 exemplarily shows a sample to be tested in a rectangular shape, and the positions of two points in the sample to be tested in the z-axis direction are d1 and d2;

Scan the sample to be tested by the first beam Scan1 to form a first gradient light field with a gradient distribution of light intensity in the x-z plane at the focal point of the objective lens, and obtain the first image Im1 in the x-y plane;

Scan the sample to be tested by the second beam Scan2 to form a second gradient light field with a gradient distribution of light intensity in the x-z plane at the focal point of the objective lens, and obtain a second image Im2 in the x-y plane;

The ratio R between the first image and the second image reflects the positions of the two points in the sample to be tested; among them,

or

(Exemplary shown in Figure 2

); Among them, 0～1 represents the numerical range of R;

Then through the corresponding relationship between R and z-axis position information z, the z-axis position information of each position in the sample to be tested is obtained, and the three-dimensional depth image of the sample to be tested is finally obtained; among them, 0-12 represents the value range of z, and the unit is μm.

In this embodiment, the wavefront shaping of the two incident pulse beams is performed sequentially to obtain two beams with a gradient light field at the focal point of the objective lens, and the light intensity distributions of the gradient light fields of the two beams are opposite. Then scan the sample to be tested by two light beams to obtain two images, and reconstruct the three-dimensional data based on the two images to obtain the three-dimensional depth image of the sample to be tested. The imaging speed is fast, the resolution is high, the algorithm is simple, and it is easy to implement. Due to the fast imaging speed (6-10 times of ordinary two-photon fluorescence microscope), the photodamage and photobleaching effects of the sample to be tested are effectively reduced, and it is especially suitable for three-dimensional imaging of embryonic development samples and neural activity samples.

Example two

As shown in FIG. 3, in this embodiment, step S101 in the first embodiment includes:

Step S301: Rotate the polarization direction of the incident first pulse beam to obtain a first polarization beam with a preset polarization direction;

Step S302: Perform spatial phase modulation on the first polarized light beam to obtain a first light beam with a first preset phase.

In specific applications, the wavefront shaping module may include a half-wave plate and a spatial light modulator. The polarization direction of the incident first pulsed beam can be adjusted by the half-wave plate so that the polarization direction of the first polarized beam is the same as that of the spatial light. The polarization direction of the modulator is consistent.

In specific applications, the first polarized light beam can be spatially phase modulated by the spatial light modulator, so that when the first light beam reaches the focal point of the objective lens of the two-photon fluorescence imaging system, the light intensity distribution changes along the z axis with a gradient. The first gradient light field.

Step S103 includes:

Step S303: Rotate the polarization direction of the incident second pulse beam to obtain a second polarization beam with a preset polarization direction;

Step S304: Perform spatial phase modulation on the second polarized light beam to obtain a second light beam with a second preset phase.

In specific applications, the polarization direction of the incident second pulsed beam can be adjusted by the half-wave plate, so that the polarization direction of the second polarized beam is consistent with the polarization direction of the spatial light modulator.

In specific applications, the second polarized light beam can be spatially phase modulated by the spatial light modulator, so that when the second light beam reaches the focal point of the objective lens of the two-photon fluorescence imaging system, the light intensity distribution changes along the z axis with a gradient. The second gradient light field.

As shown in Fig. 4, in this embodiment, before step S301, the method further includes:

Step S401: Design a first focal point with light intensity distributed along the z-axis; wherein the light field at the first focal point is a first gradient light field;

Step S402: Design a second focal point with light intensity distributed along the z axis; wherein the light field at the second focal point is a second gradient light field;

Step S403: Divide the entrance pupil of the objective lens into a preset number of rings with equal areas.

In specific applications, the light intensity distribution (ie, gradient change) of the first gradient light field and the second gradient light field can be set according to actual needs, as long as the light intensity distributions of the two light fields are opposite.

In specific applications, the preset number can be set according to actual needs, and the preset number is positively correlated with the designed light intensity distribution accuracy of the gradient light field corresponding to the first focus and the second focus. For example, the preset number can be set to any value from 20 to 80, specifically 40.

As shown in FIGS. 5-7, exemplary light intensity distributions of the first gradient light field and the second gradient light field;

Wherein, FIG. 5 exemplarily shows the phase image of the entrance pupil divided into a preset number of rings of equal area, and the first gradient light field and the second gradient light field corresponding to the phase image of the entrance pupil; wherein, GrandF1 means that the light intensity of the first gradient light field changes from strong to weak along the Z axis, GrandF2 means that the light intensity of the second gradient light field changes from weak to strong along the Z axis, 0～1 means the range of light intensity, -π～ π represents the phase angle;

Fig. 6 exemplarily shows the light intensity distribution of the first gradient light field and the second gradient light field along the z axis;

Fig. 7 exemplarily shows the light intensity distribution along the z-axis of the light field obtained by adding the first gradient light field and the second gradient light field.

Step S404: Combining the phases of the preset number of rings with the same area as the phase function of the entrance pupil.

In specific applications, the phase function of each ring can be regarded as a combination of the phases of each ring, and the phase function of the entrance pupil can be regarded as a combination of the phase functions of a preset number of rings. Therefore, you can use The combination of the phases of the preset number of rings represents the phase function of the entrance pupil.

Step S405: Calculate the light field at the focal point corresponding to the phase function of each ring according to the phase function of the entrance pupil.

In one embodiment, the calculation formula of step S405 is Richards-Wolf (Richards-Wolf vector diffraction integral) calculation formula, which is specifically expressed as follows:

Where is the convergence angle of the light, P() is the phase function of the entrance pupil, which is the maximum value,

k=2π/λ is the wave number, NA is the numerical aperture of the objective lens, n is the immersion medium of the objective lens, λ is the wavelength of the light, z is the position information of the z-axis, z-axis is the optical axis of the objective lens, and the position of z=0 is the objective lens The position where the beam is focused.

In specific applications, the numerical aperture and immersion medium of the objective lens are different depending on the type of objective lens selected; the wavelength of the light is also different according to the type of laser selected.

In an embodiment, the numerical aperture of the objective lens is 1, the wetting medium of the objective lens is 1.33, and the wavelength of the light is 920 nm.

Step S406: Use a genetic algorithm to find the phase corresponding to the light field most similar to the first gradient light field as the first preset phase;

Step S407: Use a genetic algorithm to find the phase corresponding to the light field most similar to the second gradient light field as the second preset phase.

As shown in FIG. 8, in an embodiment, the implementation steps of step S406 and step 407 are as follows:

Step S801, the initial population, go to step S802; wherein, the initial population is a combination of the phases of the preset number of rings of equal area;

Step S802: Calculate the light intensity distribution I(z) at the focal point of each individual in the population, and proceed to step S803; wherein I(z) is calculated by the above-mentioned Richards-Wolf calculation formula using the phase of each ring get;

Step S803, use the formula F _obj =∑ _z (I(z)-I _t (z)) ^{2 to} evaluate I(z), and proceed to step S804; where I _t (z) is the pre-designed first focus or second focus The light intensity distribution at the focal point, t: target (target);

Step S804, judge whether the current iteration number is less than the maximum iteration number; if yes, go to step S805; if not, go to step S808;

Step S805, select, and go to step S806;

Step S806, crossover and mutation, go to step S807;

Step S807: For a new population, go to step S802; wherein the number of individuals in the new population is less than the number of individuals in the initial population;

Step S808: Output the individual with the smallest F _obj , and proceed to step S809; wherein the individual with the smallest F _{obj is} the individual with the light intensity distribution most similar to the target gradient light field.

In specific applications, when I _t (z) is the light intensity distribution at the first focus, the target gradient light field is the first gradient light field; when I _t (z) is the light intensity distribution at the second focus, The target gradient light field is the second gradient light field.

In specific applications, the maximum number of iterations and the initial population number can be set according to actual needs.

In an embodiment, before step S406, it includes:

The maximum number of iterations of the genetic algorithm is 4000, the initial population is set to 8000, and the variable accuracy of each individual is set to ²⁸ .

In specific applications, the genetic algorithm is used to calculate and the population number is gradually reduced to speed up the number of iterations. After the iteration, the phase corresponding to the light field most similar to the target gradient light field is output and loaded on the spatial light modulator as the optimal phase. Then the corresponding gradient light field can be obtained behind the objective lens.

Example three

As shown in Figure 9, in this embodiment, step S105 in the first embodiment includes:

Step S901: Perform maximum normalization processing on the first image and the second image respectively.

In a specific application, the light intensity of each pixel in the first image is divided by the maximum light intensity in the first image to realize the normalization of the maximum value of the first image; each pixel in the second image The light intensity of is divided by the maximum light intensity in the second image to achieve normalization of the maximum value of the second image.

Step S902: Perform intensity information extraction on the first image and the second image after the maximum value normalization process is performed to obtain an intensity image.

In an embodiment, the implementation formula of step S902 is:

Among them, Im is the intensity image, Im ₁ is the first image, and Im ₂ is the second image.

In specific applications, the intensity distribution in the intensity image reflects the strength of the fluorescence signal excited by the sample to be tested.

Step S903: Set an intensity threshold to perform intensity information extraction on the intensity image to obtain an effective information area.

In specific applications, due to the influence of noise, if the z-axis position calculation is performed on the noise area of the image, a large error will occur, which will affect the calculation accuracy. Therefore, it is necessary to use an intensity threshold to select a mask, and the following data processing flow is only effective for areas with signals. The light intensity distribution of the light field obtained by adding the first gradient light field and the second gradient light field shown in FIG. 7 can be used for intensity information extraction to obtain an effective information area.

In one embodiment, after step S903, the method includes:

Binarize the intensity image to isolate the effective information area.

Step S904: Extract z-axis position information of the effective information area to obtain a position matrix with the same pixel size as the intensity image.

In an embodiment, the implementation formula of step S904 is:

Wherein, z denotes a z-axis position information, l _axial focal length of the first or the second focal point, I _Im1 gradation value of a first _pixel in the image Im, I _Im2 second grayscale pixels in image Im ₂ Value, the position matrix includes the z-axis position information of all pixels in the effective information area, that is, the position matrix is equivalent to a set of z. In an embodiment, after step S904, the method further includes:

Gaussian filtering is performed on the intensity image to perform noise reduction processing on the intensity image.

In specific applications, the window size of the Gaussian filtering operation can be set according to actual needs. The smaller the window, the better the noise reduction processing effect.

In one embodiment, the window size of the Gaussian filtering operation is 1 pixel.

Step S905: Sort the z-axis position information in the position matrix, and select a value range according to the sorting result to normalize the position matrix.

In specific applications, the z-axis position information in the position matrix can be sorted from small to large or from large to small. Since the effective ratio of noise influence is distributed in a specific interval, it is necessary to select the appropriate minimum z and maximum z. The position matrix is normalized.

In one embodiment, the value range of z is 0-12.

Step S906: Encode the intensity information in the intensity image and the z-axis position information in the position matrix after the normalization process into the same image to obtain a three-dimensional depth image of the sample to be tested; where The Z-axis position information is coded as different colors, and the intensity information is coded as color saturation.

In specific applications, step S906 can be implemented according to the colormap function used to set and obtain the current color map in MATLAB.

As shown in FIG. 10, the images at each stage in step S901 to step S906 are exemplarily shown. The maximum value of z shown in Figure 2, Figure 6 or Figure 10 is 12um, which is l _axial . It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

Example four

As shown in FIG. 11, this embodiment provides a two-photon fluorescence imaging system 100 for performing the two-photon fluorescence imaging method in any one of Embodiment 1 to Embodiment 3, including:

The wavefront shaping module 1 is used for wavefront shaping of the incident first pulse beam to obtain the first beam;

The photon fluorescence microscope 2 is used to scan the sample to be tested through the first beam to obtain the first image; wherein, the light field of the first beam at the focal point of the objective lens of the two-photon fluorescence microscope 2 is a first gradient light field, and the first gradient The light intensity distribution of the light field changes in a gradient along the z-axis, and the z-axis direction is the depth direction of the sample to be tested;

The wavefront shaping module 2 is also used to perform wavefront shaping on the incident second pulsed light beam to obtain a second light beam; wherein the light field of the second light beam at the focal point of the objective lens is the same as the light intensity distribution of the first gradient light field The opposite second gradient light field;

The two-photon fluorescence microscope 2 is also used to scan the sample to be tested with a second beam to obtain a second image;

The image processing device 3 is used to perform three-dimensional data reconstruction based on the first image and the second image to obtain a three-dimensional depth image of the sample to be tested.

In specific applications, the image processing equipment can be a fully automatic or semi-automatic microscope photography device that comes with a two-photon fluorescence microscope, or it can include image sensors and computing devices such as desktop computers, notebooks, palmtops, and cloud servers.

As shown in FIG. 11, in this embodiment, the image processing device 3 is in communication connection with the spatial light modulator 1, and the image processing device is used to control the spatial light modulator to change the phases of the first pulse beam and the second pulse beam in order to The light field at the focal point of the objective lens is the first light beam of the first gradient light field, and the light field at the focal point of the objective lens is the second light beam of the second gradient light field opposite to the light intensity distribution of the first gradient light field.

As shown in FIG. 11, in this embodiment, the two-photon fluorescence imaging system 100 further includes a laser 4 for generating a first pulse beam and a second pulse beam and emitting them to the wavefront shaping module 1.

In specific applications, the laser can be selected according to actual needs to be suitable for two-photon fluorescence imaging, for example, a femtosecond pulsed laser.

As shown in FIG. 12, in one embodiment, the wavefront shaping module 1 includes:

The half-wave plate 11 is used to rotate the polarization direction of the incident first pulsed beam to obtain the first polarized beam with a preset polarization direction;

The spatial light modulator 12 is configured to perform spatial phase modulation on the first polarized light beam to obtain the first light beam of the first preset phase;

The half-wave plate 11 is also used to rotate the polarization direction of the incident second pulsed light beam to obtain a second polarized light beam with a preset polarization direction;

The spatial light modulator 12 is also used to perform spatial phase modulation on the second polarized light beam to obtain a second light beam with a second preset phase.

In specific applications, the entrance surface of the half-wave plate faces the exit surface of the laser, the exit surface of the half-wave plate faces the entrance surface of the spatial light modulator, and the exit surface of the spatial light modulator faces the entrance surface of the scanning galvanometer of the two-photon fluorescence microscope .

As shown in FIG. 12, in one embodiment, the wavefront shaping module 1 further includes:

The reflecting mirror 13 is used to reflect the first pulse beam and the second pulse beam emitted by the laser to the incident surface of the half-wave plate 12;

The 4f system 14 is used to make the exit surface of the spatial light modulator 12 and the entrance surface of the scanning galvanometer of the two-photon fluorescence microscope 2 conjugate.

In specific applications, the reflective surface of the mirror faces the exit surface of the laser and the entrance surface of the half-wave plate respectively, the exit surface of the spatial light modulator is located at the objective focal plane of the 4f system, and the entrance surface of the scanning galvanometer is located at the 4f system Image square focal plane.

In one embodiment, the image processing device includes:

In an embodiment, the image processing device includes:

In specific applications, the modules included in the image processing device may be software program modules stored in the memory of the image processing device and executed by the processor of the image processing device. The processor implements the corresponding computer program when it executes The function of each module.

Example five

As shown in FIG. 13, this embodiment provides an image processing device 3, which includes: an image sensor 30, a processor 31, a memory 32, and a computer program stored in the memory 32 and running on the processor 31 33, such as a three-dimensional data reconstruction program. Alternatively, when the processor 31 executes the computer program 33, the function of each module in the fourth embodiment is realized.

Exemplarily, the computer program 33 may be divided into one or more modules, and the one or more modules are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 33 in the image processing device. For example, the computer program 33 can be divided into the following modules:

Alternatively, the computer program 33 can also be divided into the following modules:

The image processing device may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server with an image sensor. The image processing device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art can understand that FIG. 13 is only an example of an image processing device, and does not constitute a limitation on the image processing device. It may include more or less components than shown in the figure, or a combination of certain components, or different components. For example, the image processing device may also include input and output devices, network access devices, buses, and the like.

The so-called processor 31 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 32 may be an internal storage unit of the image processing device, such as a hard disk or memory of the image processing device. The memory 32 may also be an external storage device of the image processing device, such as a plug-in hard disk equipped on the image processing device, a smart memory card (Smart Media Card, SMC), or a Secure Digital (SD). Card, Flash Card, etc. Further, the memory 32 may also include both an internal storage unit of the image processing device and an external storage device. The memory 32 is used to store the computer program and other programs and data required by the image processing device. The memory 32 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned functions can be allocated to different functional units and modules as required. Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Form realization can also be realized in the form of software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed device/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted in accordance with the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in Within the protection scope of the present invention.

Claims

A two-photon fluorescence imaging method is characterized in that it is applied to a two-photon fluorescence imaging system, and the two-photon fluorescence imaging method includes:

Wavefront shaping is performed on the incident first pulsed light beam to obtain the first light beam; wherein the light field of the first light beam at the focal point of the objective lens of the two-photon fluorescence imaging system is a first gradient light field, and The light intensity distribution of a gradient light field changes in a gradient along the z-axis, and the z-axis direction is the depth direction of the sample to be tested;

Scan the sample to be tested by the first light beam to obtain a first image;

Wavefront shaping is performed on the incident second pulsed light beam to obtain a second light beam; wherein the light field of the second light beam at the focal point of the objective lens is the first light intensity distribution opposite to the light intensity distribution of the first gradient light field Two gradient light fields;

Scan the sample to be tested by the second light beam to obtain a second image;

Perform three-dimensional data reconstruction according to the first image and the second image to obtain a three-dimensional depth image of the sample to be tested.
The two-photon fluorescence imaging method of claim 1, wherein the wavefront shaping of the incident first pulsed light beam to obtain the first light beam comprises:

Rotating the polarization direction of the incident first pulsed beam to obtain a first polarized beam with a preset polarization direction;

Performing spatial phase modulation on the first polarized light beam to obtain a first light beam with a first preset phase;

The wavefront shaping of the incident second pulse beam to obtain the second beam includes:

Rotating the polarization direction of the incident second pulse beam to obtain a second polarization beam with a preset polarization direction;

Performing spatial phase modulation on the second polarized light beam to obtain a second light beam with a second preset phase.
3. The two-photon fluorescence imaging method of claim 2, wherein the wavefront shaping of the incident first pulsed beam to obtain the first beam comprises:

Design a first focal point with light intensity distributed along the z axis; wherein the light field at the first focal point is a first gradient light field;

Design a second focal point with light intensity distributed along the z-axis; wherein the light field at the second focal point is a second gradient light field;

Dividing the entrance pupil of the objective lens into a preset number of rings of equal area;

Combining the phases of the preset number of rings with equal areas as the phase function of the entrance pupil;

Calculating the light field at the focal point corresponding to the phase function of each ring according to the phase function of the entrance pupil;

Using a genetic algorithm to find the phase corresponding to the light field most similar to the first gradient light field as the first preset phase;

A genetic algorithm is used to find the phase corresponding to the light field most similar to the second gradient light field as the second preset phase.
The two-photon fluorescence imaging method according to claim 3, wherein, according to the phase function, the calculation formula for calculating the light field at the focal point corresponding to each phase function is:

Where is the convergence angle of light, P() is the phase function of the entrance pupil, which is the maximum value,
k=2/λ is the wave number, NA is the numerical aperture of the objective lens, n is the immersion medium of the objective lens, hand is the wavelength of the light, and z is the z-axis position information.
The two-photon fluorescence imaging method according to any one of claims 1 to 4, characterized in that, according to the first image and the second image, three-dimensional data reconstruction of the sample to be tested is performed to obtain the The three-dimensional depth image of the test sample, including:

Performing maximum normalization processing on the first image and the second image respectively;

Performing intensity information extraction on the first image and the second image after the maximum value normalization processing is performed to obtain an intensity image;

Setting an intensity threshold to extract intensity information from the intensity image to obtain an effective information area;

Extracting the z-axis position information of the effective information area to obtain a position matrix with the same pixel size as the intensity image;

Sorting the z-axis position information in the position matrix, and selecting a value range according to the sorting result to normalize the position matrix;

The intensity information in the intensity image and the z-axis position information in the normalized position matrix are encoded into the same image to obtain a three-dimensional depth image of the sample to be tested; wherein, the Z-axis The position information is coded as different colors, and the intensity information is coded as color saturation.
A two-photon fluorescence imaging system is characterized in that it comprises:

The wavefront shaping module is used to shape the wavefront of the incident first pulse beam to obtain the first beam;

The photon fluorescence microscope is used to scan the sample to be tested through the first light beam to obtain a first image; wherein the light field of the first light beam at the focal point of the objective lens of the two-photon fluorescence microscope is a first gradient light field , The light intensity distribution of the first gradient light field changes in a gradient along the z-axis, and the z-axis direction is the depth direction of the sample to be tested;

The wavefront shaping module is also used to perform wavefront shaping on the incident second pulsed light beam to obtain a second light beam; wherein the light field of the second light beam at the focal point of the objective lens is the same as that of the first A second gradient light field with an opposite light intensity distribution of the gradient light field;

The two-photon fluorescence microscope is also used to scan the sample to be tested through the second beam to obtain a second image;

The image processing device is used to perform three-dimensional data reconstruction according to the first image and the second image to obtain a three-dimensional depth image of the sample to be tested.
8. The two-photon fluorescence imaging system of claim 6, wherein the wavefront shaping module comprises:

The half-wave plate is used to rotate the polarization direction of the incident first pulsed beam to obtain the first polarized beam with the preset polarization direction;

A spatial light modulator, configured to perform spatial phase modulation on the first polarized light beam to obtain a first light beam with a first preset phase;

The half-wave plate is also used to rotate the polarization direction of the incident second pulsed light beam to obtain a second polarized light beam with a preset polarization direction;

The spatial light modulator is also used to perform spatial phase modulation on the second polarized light beam to obtain a second light beam with a second preset phase.
8. The two-photon fluorescence imaging system of claim 7, wherein the image processing device is in communication connection with the spatial light modulator, and the image processing device comprises:

The first design module is used to design a first focal point with light intensity distributed along the z axis; wherein the light field at the first focal point is a first gradient light field;

The second design module is used to design a second focal point where the light intensity is distributed along the z axis; wherein the light field at the second focal point is a second gradient light field;

A dividing module, configured to divide the entrance pupil of the objective lens into a preset number of rings of equal area;

A combination module, configured to combine the phases of the preset number of rings with the same area as the phase function of the entrance pupil;

A calculation module, configured to calculate the light field at the focal point corresponding to the phase function of each ring according to the phase function of the entrance pupil;

The first searching module is configured to use a genetic algorithm to find the phase corresponding to the light field most similar to the first gradient light field, as the first preset phase and output to the spatial light modulator;

The second searching module is used to search for the phase corresponding to the light field most similar to the second gradient light field by using a genetic algorithm, as a second preset phase and output to the spatial light modulator.
The two-photon fluorescence imaging system according to any one of claims 6 to 8, wherein the image processing device comprises:

A normalization processing module, configured to perform maximum normalization processing on the first image and the second image respectively;

The first extraction module is configured to extract intensity information of the first image and the second image after the maximum value normalization processing is performed to obtain an intensity image;

The second extraction module is configured to set an intensity threshold to extract intensity information from the intensity image to obtain an effective information area;

The third extraction module is configured to extract the z-axis position information of the effective information area to obtain a position matrix with the same pixel size as the intensity image;

A sorting module, configured to sort the z-axis position information in the position matrix, and select a value range according to the sorting result to normalize the position matrix;

An encoding module for encoding the intensity information in the intensity image and the z-axis position information in the normalized position matrix into the same image to obtain a three-dimensional depth image of the sample to be tested; wherein The z-axis position information is coded as different colors, and the intensity information is coded as color saturation.
An image processing device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claim 3 Steps of any one of the methods to 5.