JP2016133748A - Calibration creation method and pattern irradiation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology that highly accurately calibrates a pattern irradiation device.SOLUTION: A pattern irradiation device 100 is configured to: irradiate a sample S with a calibration pattern (step S10); acquire an image of the sample S having the calibration pattern irradiated for each depth position (step S20); specify an irradiation depth location having the calibration pattern irradiated from a plurality of acquired images (step S30); and create calibration data on the pattern irradiation device 100 for forming a desired pattern at the specified irradiation depth location on the basis of image data on the irradiation depth location and the calibration pattern (step S40).SELECTED DRAWING: Figure 2

Description

  The present invention relates to a method for generating calibration data of a pattern irradiation apparatus including a wavefront modulator, and a pattern irradiation apparatus.

  Conventionally, pattern irradiation that uses a wavefront modulator such as a spatial light modulator (SLM) placed at a position optically conjugate with the pupil position of the objective lens to perform pattern irradiation of light The device is known.

  In general, when performing pattern irradiation of light with a pattern irradiation apparatus, the shape and irradiation position of a light pattern to be formed on a specimen are specified on a screen on which an image of the specimen is displayed. However, in practice, pattern irradiation may not be performed on the shape and position as specified on the screen due to insufficient adjustment of the optical system or errors. For this reason, a technique for calibrating the pattern irradiation apparatus is required so that the pattern irradiation is performed as specified on the screen.

  Japanese Patent Laid-Open No. 2004-151867 corrects the modulation amount of a phase modulation type SLM by using a correction parameter calculated based on a light pattern designated by a user and a light pattern actually irradiated. Techniques for calibrating the device are described.

JP 2014-112122 A

  Patent Document 1 shows an example in which a correction parameter is calculated by irradiating a light pattern on a flat plate such as a fluorescent plate. The pattern in the optical axis direction of the objective lens (hereinafter referred to as the depth direction) is shown. Shift is not considered. However, in practice, the light pattern is not always formed at the specified depth position, and the pattern may shift in the depth direction.

  In light of the above situation, an object of the present invention is to provide a technique for accurately calibrating a pattern irradiation apparatus.

  According to a first aspect of the present invention, there is provided a calibration data generation method for a pattern irradiation apparatus including a wavefront modulator, wherein the wavefront modulation is performed based on a calibration pattern including a two-dimensional target pattern orthogonal to a depth direction of a specimen. The sample is irradiated with light whose wavefront is modulated by a vessel, and image data of the sample irradiated with the light is generated for each depth position. Based on the generated plurality of image data, two-dimensional An irradiation depth position, which is a depth position where the irradiation pattern is formed, is specified, and calibration data for forming a pattern at the irradiation depth position is determined based on the image data of the irradiation depth position and the target pattern. A method for generating calibration data is provided.

  According to a second aspect of the present invention, in the calibration data generation method according to the first aspect, generating the calibration data for forming a pattern at the irradiation depth position is an image of the irradiation depth position. Based on the data, acquiring the three-dimensional coordinate information of the irradiation pattern formed at the irradiation depth position, and based on the three-dimensional coordinate information of the irradiation pattern and the three-dimensional coordinate information of the target pattern, Generating a calibration data for forming a pattern at an irradiation depth position.

  According to a third aspect of the present invention, in the calibration data generation method according to the second aspect, the wavefront is modulated by the wavefront modulator based on the calibration pattern including the target pattern at a plurality of different depth positions. Irradiating the specimen with light, identifying a plurality of irradiation depth positions on which a two-dimensional irradiation pattern is formed based on the plurality of image data, and a plurality of images of the plurality of irradiation depth positions Based on the data, three-dimensional coordinate information of a plurality of irradiation patterns formed at the plurality of irradiation depth positions is acquired, and a plurality of target patterns having depth positions different from the three-dimensional coordinate information of the plurality of irradiation patterns. Based on the three-dimensional coordinate information, a plurality of calibration data for forming a pattern at the plurality of irradiation depth positions is generated, the plurality of calibration data are interpolated, and a plurality of calibration data other than the plurality of irradiation depth positions are generated. 1 depth Providing calibration data generation method for generating calibration data for forming a pattern on location.

  According to a fourth aspect of the present invention, in the calibration data generation method according to the second aspect, the wavefront is modulated by the wavefront modulator based on the calibration pattern including the target pattern at a plurality of different depth positions. Irradiating the specimen with light, identifying a plurality of irradiation depth positions on which a two-dimensional irradiation pattern is formed based on the plurality of image data, and a plurality of images of the plurality of irradiation depth positions Based on the data, three-dimensional coordinate information of a plurality of irradiation patterns formed at the plurality of irradiation depth positions is acquired, and a plurality of target patterns having depth positions different from the three-dimensional coordinate information of the plurality of irradiation patterns. The plurality of irradiations generated by generating a plurality of calibration data for forming a pattern at the plurality of irradiation depth positions based on the three-dimensional coordinate information, and interpolating the three-dimensional coordinate information of the plurality of irradiation patterns. Provided is a calibration data generation method for generating calibration data for forming a pattern at the first depth position based on three-dimensional coordinate information of an irradiation pattern formed at a first depth position other than the vertical position. To do.

  According to a fifth aspect of the present invention, in the calibration data generation method according to the third aspect or the fourth aspect, the calibration pattern includes the plurality of target patterns having the same two-dimensional coordinate information and different intensity distributions. Provide a data generation method.

  According to a sixth aspect of the present invention, in the calibration data generation method according to any one of the first to fifth aspects, the calibration data for forming a pattern at the irradiation depth position is the calibration data generation method. Provided is a calibration data generation method including two-dimensional coordinate conversion information of an irradiation depth position and a depth position correction amount for forming a pattern at the irradiation depth position.

  According to a seventh aspect of the present invention, in the calibration data generation method according to any one of the first to sixth aspects, the target pattern is a two-dimensional pattern including two or more condensing points. A calibration data generation method is provided.

  According to an eighth aspect of the present invention, in the calibration data generation method according to any one of the first to seventh aspects, a contrast value of each of the plurality of image data is calculated, and the calculated plurality of Provided is a calibration data generation method for specifying the irradiation depth position based on a contrast value.

  According to a ninth aspect of the present invention, in the calibration data generation method according to any one of the first to eighth aspects, the wavefront modulator is a spatial light modulator provided with an LCOS, or a deformer. A calibration data generation method that is a bull mirror is provided.

  A tenth aspect of the present invention provides the calibration data generating method according to any one of the first to ninth aspects, wherein the specimen is a three-dimensional fluorescent specimen.

  An eleventh aspect of the present invention includes a wavefront modulator, and the light whose wavefront is modulated by the wavefront modulator based on a calibration pattern including a two-dimensional target pattern orthogonal to the depth direction of the sample is applied to the sample. Irradiating means for irradiating, image data generating means for generating image data of the specimen irradiated with the light by the irradiating means for each depth position, and a plurality of image data generated by the image data generating means The irradiation depth position, which is the depth position where the two-dimensional irradiation pattern is formed, is specified, and a pattern is formed at the irradiation depth position based on the image data of the specified irradiation depth position and the target pattern. There is provided a pattern irradiation apparatus including calibration data generation means for generating calibration data for performing the above.

  ADVANTAGE OF THE INVENTION According to this invention, the calibration data which calibrate a pattern irradiation apparatus accurately can be produced | generated.

It is the figure which showed the structure of the pattern irradiation apparatus 100 which concerns on Example 1. FIG. 3 is a flowchart showing a flow of processing performed in the pattern irradiation apparatus 100. It is a flowchart which shows the flow of a calibration pattern irradiation process. It is the figure which illustrated the target pattern set in calibration pattern irradiation processing. It is a figure for demonstrating an example of the irradiation method of a calibration pattern. It is a figure for demonstrating another example of the irradiation method of a calibration pattern. It is a flowchart which shows the flow of an image acquisition process. It is a figure for demonstrating the acquisition method of image data. It is a flowchart which shows the flow of an irradiation depth position specific process. It is the figure which showed an example of the relationship between the depth position of image data, and a contrast value. It is a flowchart which shows the flow of a calibration data production | generation process. It is a figure for demonstrating the interpolation method of the three-dimensional coordinate information of an irradiation pattern. It is the figure which showed an example of the relationship between the depth position of a target pattern, and the depth position of image data. It is the figure which showed the structure of the pattern irradiation apparatus 200 which concerns on Example 2. FIG.

  FIG. 1 is a diagram illustrating a configuration of a pattern irradiation apparatus 100 according to the present embodiment. A pattern irradiation apparatus 100 shown in FIG. 1 is an apparatus having a pattern irradiation function for irradiating a specimen S with light to form an arbitrary light pattern and a calibration function for correcting the light pattern to calibrate the apparatus. is there. The pattern irradiation apparatus 100 includes a microscope, a computer 15 that controls the microscope, and a monitor 16 and an input device 17 connected to the computer 15.

  The microscope constituting the pattern irradiation apparatus 100 is a microscope having a laser 1, an LCOS-SLM 4, and an objective lens 11, and an arbitrary laser beam is applied to the specimen S by modulating the wavefront of the laser beam with the LCOS-SLM 4. The pattern is formed. The LCOS-SLM 4 is a phase modulation spatial light modulator provided with LCOS (Liquid Crystal On Silicon), and is a wavefront modulator that modulates the wavefront of laser light.

  More specifically, as shown in FIG. 1, the microscope includes a laser 1, a beam expander 2, a prism 3, an LCOS-SLM 4, a relay lens 5, a galvano mirror 6, and a pupil relay lens 7. , An imaging lens 8, a mirror 9, a dichroic mirror 10, an objective lens 11, a stage 12, a relay lens 13, and a photomultiplier tube (PMT) 14. It is desirable that the LCOS-SLM 4 and the galvanometer mirror 6 are respectively disposed at positions optically conjugate with the pupil position of the objective lens 11. It is also desirable that the PMT 14 is disposed at a position optically conjugate with the pupil position of the objective lens 11.

  The computer 15 includes an arithmetic control unit 15a that is a processor, for example, and a storage unit 15b that is a hard disk or a memory, for example, and includes at least the LCOS-SLM 4, the galvano mirror 6, the stage 12, and the like among the components of the microscope. It is connected to PMT14. The computer 15 is configured to control these components and to receive and process signals from these components.

  In the pattern irradiation apparatus 100 configured as described above, the laser beam emitted from the laser 1 is adjusted in the beam diameter by the beam expander 2 and enters the LCOS-SLM 4 via the prism 3. The LCOS-SLM 4 includes a plurality of pixel elements arranged in two dimensions, each controlled independently, and the phase of the laser light incident on the LCOS-SLM 4 is modulated by the incident pixel elements. As a result, the wavefront of the laser light is modulated by the LCOS-SLM 4. This wavefront modulation is performed by the computer 15 calculating the wavefront modulation amount of the laser light and setting it in the LCOS-SLM 4 based on the pattern input from the input device 17 as the pattern to be formed on the sample S. Thereafter, the laser light is incident on the mirror 9 via the relay lens 5, the galvano mirror 6, the pupil relay lens 7, and the imaging lens 8. The laser light is reflected by the mirror 9 in the direction of the optical axis of the objective lens 11, passes through the dichroic mirror 10 having the characteristics of transmitting the laser light and reflecting the fluorescence, and enters the objective lens 11. Then, the objective lens 11 irradiates the sample S arranged on the stage 12 with laser light, and a laser light pattern corresponding to wavefront modulation in the LCOS-SLM 4 is formed on the sample S.

  That is, in the pattern irradiation apparatus 100, the microscope is a pattern irradiation unit that irradiates the sample S with light modulated by the LCOS-SLM 4.

  In addition, when generating the image data of the sample S, the pattern irradiation apparatus 100 irradiates the sample S with the laser light without modulating the wavefront of the laser light with the LCOS-SLM 4. Then, the galvanometer mirror 6 disposed between the objective lens 11 and the LCOS-SLM 4 scans the specimen S with laser light in the XY directions orthogonal to the optical axis of the objective lens 11. When the two-dimensional scanning is completed, the stage 12 moves in the depth direction (Z-axis direction) of the sample S, and the focal plane of the objective lens 11 moves to a different depth position (Z position) of the sample S. Thereafter, the galvanometer mirror 6 scans the sample S again. These processes are repeated within a predetermined depth range. The fluorescence generated from the specimen S by the irradiation of the laser light is reflected by the dichroic mirror 10 that enters through the objective lens 11 and enters the PMT 14 through the relay lens 13. The PMT 14 that has detected the fluorescence outputs an electrical signal to the computer 15. The computer 15 receives the electrical signal from the PMT 14 and generates image data of the sample S for each Z position from the received electrical signal and control information of the galvano mirror 6 indicating the scanning position.

  That is, in the pattern irradiation apparatus 100, the galvanometer mirror 6 is a scanning unit that scans the sample S with laser light to generate image data of the sample S, and the PMT 14 detects fluorescence that is observation light from the sample S. The computer 15 is an image data generating unit that generates image data of the specimen S based on the signal from the PMT 13 and the control information of the galvanometer mirror 6. The computer 15 is a wavefront modulation amount control unit that calculates a wavefront modulation amount according to a pattern to be formed on the sample S and transmits the wavefront modulation amount to an SLM controller (not shown), and a calibration data generation unit that generates calibration data described later. is there.

  FIG. 2 is a flowchart showing the flow of processing performed by the pattern irradiation apparatus 100. As shown in FIG. 2, the pattern irradiation apparatus 100 first irradiates the specimen S with a pattern set for calibration (hereinafter referred to as a calibration pattern) (step S10: calibration pattern irradiation step), and the calibration pattern is An image of the irradiated specimen S is acquired for each depth position to generate image data (step S20: image acquisition process). Furthermore, the irradiation depth position irradiated with the calibration pattern is specified from the plurality of generated image data (step S30: irradiation depth position specifying step), and a desired pattern is formed at the specified irradiation depth position. Calibration data of the pattern irradiation apparatus 100 is generated based on the image data of the irradiation depth position and the calibration pattern (step S40: calibration data generation step).

Hereafter, each process shown in FIG. 2 is demonstrated in detail, referring FIGS. 3-13.
FIG. 3 is a flowchart showing the flow of the calibration pattern irradiation process. FIG. 4 is a diagram illustrating a target pattern set in the calibration pattern irradiation process. 5 and 6 are diagrams for explaining an example of a calibration pattern irradiation method. The calibration pattern irradiation process will be described with reference to FIGS.

  In the calibration pattern irradiation step, first, the user of the microscope system 100 installs the specimen S on the stage 12 (step S11). The sample S is, for example, a three-dimensional fluorescent sample.

  Thereafter, the user sets a calibration pattern including the target pattern in the pattern irradiation apparatus 100 (step S12). The target pattern is, for example, a two-dimensional pattern orthogonal to the depth direction (Z-axis direction) of the sample S as shown in FIG. 4 and a two-dimensional pattern including two or more condensing points. desirable. The target pattern shown in FIG. 4 is a two-dimensional pattern composed of four points A, B, C, and D. The calibration pattern may be the target pattern itself or a three-dimensional pattern including the target pattern at a plurality of different depth positions. Hereinafter, a case where the calibration pattern is a pattern including target patterns at different depth positions Z1, Z2, Z3, and Z4 will be described as an example.

  In step S12, a calibration pattern may be set by setting a target pattern and specifying one or more depth positions of the target pattern. In this case, a two-dimensional image of the sample S is acquired without performing wavefront modulation with the LCOS-SLM 4, and a plurality of points (for example, points A, B, and B in FIG. 4 are displayed on the monitor 16 displaying the acquired two-dimensional image. The target pattern may be set by designating (C, D). The target pattern may be set by reading a pattern stored in advance in the storage unit 15b. In addition, a three-dimensional image of the sample S is acquired without performing wavefront modulation with the LCOS-SLM 4, and one or more target patterns are designated on the monitor 16 displaying the acquired three-dimensional image. May be set.

  When the calibration pattern is set, the pattern irradiation apparatus 100 calculates a wavefront modulation amount based on the calibration pattern set in step S12 and sets it in the LCOS-SLM4 (wavefront modulator) (step S13). More specifically, a phase modulation amount in each pixel element is calculated, and the phase modulation amount is set in each pixel element.

  Thereafter, the pattern irradiation apparatus 100 oscillates laser light from the laser 1 and irradiates the sample S with laser light whose wavefront is modulated by the LCOS-SLM 4 (step S14). That is, the sample S is irradiated with light whose wavefront is modulated by the LCOS-SLM 4 based on the calibration pattern. As a result, a pattern corresponding to the calibration pattern is formed on the specimen S. Here, the pattern corresponding to the calibration pattern is generally a pattern approximated to the calibration pattern, and is a pattern including an error due to the fact that the pattern irradiation apparatus 100 is not calibrated.

  In step S13, a wavefront modulation amount for simultaneously (at once) forming a calibration pattern including a plurality of target patterns on the sample S may be calculated. In this case, in step S14, as shown in FIG. A three-dimensional pattern corresponding to the pattern is formed on the specimen S. In step S13, a plurality of wavefront modulation amounts for forming each of a plurality of target patterns having different depth positions on the sample S are calculated, and these wavefront modulation amounts are sequentially set in the LCOS-SLM4. May be. In that case, in step S14, as shown in FIGS. 6A to 6D, patterns (irradiation patterns) corresponding to each of the plurality of target patterns are sequentially formed on the specimen S, and the entire calibration pattern is formed. A corresponding three-dimensional pattern is formed.

  When the pattern irradiation apparatus 100 executes the process shown in FIG. 3, a three-dimensional pattern corresponding to the calibration pattern is formed on the sample S, and as a result, the portion of the sample S irradiated with light is faded.

  FIG. 7 is a flowchart showing the flow of the image acquisition process. FIG. 8 is a diagram for explaining a method for acquiring image data. The image acquisition process will be described with reference to FIGS.

  In the image acquisition process, the pattern irradiation apparatus 100 moves the focal plane of the objective lens 11 to the initial depth position Z0 (step S21), and generates two-dimensional image data of the sample S (step S22). Here, without performing phase modulation by the LCOS-SLM 4, as shown in FIG. 8, the sample S is scanned with the laser light by the galvano mirror 6, and the signal from the PMT 14 that detects the fluorescence, the control information of the galvano mirror 6, Based on the two-dimensional image data of the sample S is generated.

  When the image data is generated, the pattern irradiation apparatus 100 determines whether or not the focal plane is within a predetermined range (step S23). If it is determined that the focal plane is within the predetermined range, the focal plane is moved by a predetermined distance. The sample is moved in the depth direction (Z-axis direction) (step S24), and two-dimensional image data of the sample S is generated again (step S22). The pattern irradiation apparatus 100 repeatedly executes the above processing until the focal plane is located outside the predetermined range. The predetermined range includes a plurality of different depth positions where a plurality of target patterns included in the calibration pattern are formed.

  When the pattern irradiation apparatus 100 executes the process shown in FIG. 7, image data of the specimen S in which the part irradiated with light in the calibration pattern irradiation process is faded is generated for each depth position.

  FIG. 9 is a flowchart showing the flow of the irradiation depth position specifying process. FIG. 10 is a diagram illustrating an example of the relationship between the depth position of the image data and the contrast value. The irradiation depth position specifying step will be described with reference to FIGS. 9 and 10.

  In the image acquisition process, first, the pattern irradiation apparatus 100 calculates the contrast value of each of the plurality of image data generated in the image acquisition process (step S31). Here, the contrast value of the image data is calculated based on a difference in luminance value between pixels constituting the image data using a known contrast evaluation method. For example, a value obtained by integrating the square of the difference between the luminance values of two pixels located at positions shifted by n pixels in the x direction over the entire image data is calculated as the contrast value.

  Thereafter, the irradiation depth position is specified based on the plurality of contrast values calculated in step S31 (step S32). When a plurality of target patterns having different depth positions are included in the calibration pattern, a plurality of irradiation depth positions are specified. The irradiation depth position is a depth position where a two-dimensional irradiation pattern is formed. Since there is a faint part of the specimen S at the irradiation depth position, the contrast value of the image data at the irradiation depth position is higher than the contrast value of the image data at a position other than the irradiation depth position. Using this feature, the pattern irradiation apparatus 100 identifies the irradiation depth position. For example, if the depth position of the image data and the contrast value have a relationship as shown in FIG. 10, the irradiation depth position is a depth position Z1 ′, Z2 ′, Z3 where the contrast value is a maximum (peak). Specify ', Z4'.

  The pattern irradiation apparatus 100 executes the process shown in FIG. 9, so that the irradiation depth position where the irradiation pattern is formed is reliably specified.

  FIG. 11 is a flowchart showing the flow of calibration data generation processing. FIG. 12 is a diagram for explaining an interpolation method of the three-dimensional coordinate information of the irradiation pattern. FIG. 13 is a diagram showing an example of the relationship between the depth position of the target pattern and the depth position of the image data. The calibration data generation process will be described with reference to FIGS.

  In the calibration data generation step, first, the pattern irradiation apparatus 100 acquires three-dimensional coordinate information of the irradiation pattern formed at the irradiation depth position based on the image data at the irradiation depth position (step S41). When a plurality of irradiation depth positions are specified, three-dimensional coordinate information of a plurality of irradiation patterns is acquired based on a plurality of image data at a plurality of irradiation depth positions.

  The three-dimensional coordinate information of the irradiation pattern is acquired by identifying a fading part from the image data and calculating the coordinates of the part. For example, if the target pattern is composed of 4 points as shown in FIG. 4, 4 discolored parts (for example, A ′, B ′, C ′, D ′) are identified from the image data, and the 4 Three-dimensional coordinate information including the two-dimensional coordinate information (XY coordinate information) of the point and the irradiation depth position (Z coordinate information) is acquired.

  Next, the pattern irradiation apparatus 100 generates calibration data of the irradiation depth position based on the three-dimensional coordinate information of the irradiation pattern and the three-dimensional coordinate information of the target pattern (step S42). When there are a plurality of irradiation patterns and target patterns, calibration data of a plurality of irradiation depth positions is generated based on the three-dimensional coordinate information of the plurality of irradiation patterns and the three-dimensional coordinate information of the plurality of target patterns.

  The three-dimensional coordinate information of the target pattern is specified based on information set in the calibration pattern irradiation process. For example, if the target pattern is composed of 4 points as shown in FIG. 4, the 3D coordinate information of the target pattern is 2D coordinate information (XY coordinate information) of 4 points (A, B, C, D). And the depth position (Z coordinate information) of the target pattern.

  The calibration data of the irradiation depth position generated in step S42 is calibration data for forming a desired pattern at the irradiation depth position, the two-dimensional coordinate conversion information of the irradiation depth position, and the irradiation depth. A depth correction amount for forming a two-dimensional pattern at the vertical position is included.

  The correction amount in the depth direction is the difference between the depth position (Z coordinate information) of the target pattern and the depth position (Z coordinate information) of the irradiation pattern. The two-dimensional coordinate conversion information is a conversion matrix calculated based on the two-dimensional coordinate information of the target pattern and the two-dimensional coordinate information of the irradiation pattern.

If the target pattern and the irradiation pattern each include three condensing points, the six-parameter two-dimensional affine transformation shown in Equation (1) can correct translation, rotation, enlargement / reduction, and shear strain. A matrix T can be calculated.

In this case, the relationship between the two-dimensional coordinates (x ′, y ′) of the irradiation pattern and the two-dimensional coordinates (x, y) of the target pattern is expressed by Expression (2) using the transformation matrix T, and Expression (3) ) And (4).

Note that elements d ₁ and d ₂ of the transformation matrix T indicate the amount of parallel movement. A 2 × 2 matrix composed of elements a ₁ , b ₁ , a ₂ , and b ₂ represents a deformation in which rotation, enlargement / reduction, and shear strain are combined. This point will be further described.
Any 2 × 2 matrix S can be transformed as shown in Equation (5).

Further, the rotation can be expressed by a matrix X shown in Expression (6), the enlargement / reduction can be expressed by a matrix Y shown in Expression (7), and the shear strain can be expressed by a matrix Z shown in Expression (8). Here, α, β, and γ are expressed by the equations (9), (10), and (11), respectively. Θ satisfies the expressions (12) and (13).

Furthermore, the matrix S and the matrices X, Y, and Z have the relationship shown in Expression (14).

From the above, it can be seen that the 2 × 2 matrix composed of the elements a ₁ , b ₁ , a ₂ , and b ₂ included in the transformation matrix T shows a deformation in which rotation, enlargement / reduction, and shear distortion are combined. I can confirm.

Specifically, the transformation matrix T shown in Expression (1) is calculated as follows. First, a matrix P shown in Expression (15) is defined from two-dimensional coordinates of three points (A, B, C) included in the target pattern. Here, the two-dimensional coordinates of the points A, B, and C are as follows. The superscript T indicates transposition.

Furthermore, a matrix Q shown in Expression (16) is defined from two-dimensional coordinates of three points (A ′, B ′, C ′) included in the irradiation pattern.

The relationship between the matrix P and the matrix Q shown in Expressions (15) and (16) is expressed by Expression (17) using the transformation matrix T.

Since the matrix P is a regular matrix and there is an inverse matrix, the transformation matrix T is calculated by Expression (18).

If the target pattern and the irradiation pattern each include four condensing points, the eight-parameter two-dimensional affine transformation shown in Expression (19) can correct translation, rotation, enlargement / reduction, and trapezoidal distortion. A matrix T can be calculated.

In this case, the relationship between the two-dimensional coordinates (x ′, y ′) of the irradiation pattern and the two-dimensional coordinates (x, y) of the target pattern is expressed by Expression (20) using the transformation matrix T, and Expression (21) ) And (22).

Specifically, the transformation matrix T shown in Expression (18) is calculated as follows. First, a matrix P shown in Expression (23) is defined from the two-dimensional coordinates of four points (A, B, C, D) included in the target pattern. Here, the two-dimensional coordinates of the points A, B, C, and D are as follows.

Further, a matrix Q shown in Expression (24) is defined from the two-dimensional coordinates of four points (A ′, B ′, C ′, D ′) included in the irradiation pattern.

  The relationship between the matrix P and the matrix Q shown in the equations (23) and (24) is also expressed by the above equation (17) using the transformation matrix T. Further, since an inverse matrix exists in the matrix P, the transformation matrix T is calculated by the above equation (18).

  When the calibration data of the irradiation depth position is generated, the pattern irradiation apparatus 100 calculates the calibration data of the depth position (first depth position) other than the irradiation depth position by interpolation (step S43). Note that the interpolation here may include extrapolation (so-called extrapolation).

  The first depth position calibration data may be generated by interpolating the plurality of irradiation depth position calibration data generated in step S42. In this case, the two-dimensional coordinate conversion information of the first depth position is calculated by interpolating the two-dimensional coordinate conversion information of a plurality of irradiation depth positions. More specifically, it is calculated by performing interpolation for each element (component) of the transformation matrix. Further, the correction amount in the depth direction for the first depth position is calculated by interpolating the correction amounts in the depth direction for the plurality of irradiation depth positions.

  Further, the calibration data of the first depth position may be generated based on the three-dimensional coordinate information of the irradiation pattern formed at the first depth position. The three-dimensional coordinate information of the irradiation pattern formed at the first depth position is generated by interpolating the three-dimensional coordinate information of the plurality of irradiation patterns acquired in step S41. Hereinafter, a procedure for generating calibration data of a depth position Z5 ′ (first depth position) between depth positions Z2 ′ and Z3 ′, which are irradiation depth positions, with reference to FIGS. explain.

  First, as shown in FIG. 12, the three-dimensional coordinate information of the irradiation pattern formed at the depth position Z2 ′ and the three-dimensional coordinate information of the irradiation pattern formed at the depth position Z3 ′ are interpolated to obtain the depth position. The three-dimensional coordinate information of the irradiation pattern formed on Z5 ′ is calculated. Thereafter, two-dimensional coordinate information regarding the XY directions is acquired from the three-dimensional coordinate information of the depth position Z5 '. Based on the acquired two-dimensional coordinate information and the two-dimensional coordinate information of the target pattern, the two-dimensional coordinate conversion information of the depth position Z5 'is calculated. Furthermore, from the depth positions (Z1, Z2, Z3, Z4) of the plurality of target patterns and the plurality of irradiation depth positions (Z1 ′, Z2 ′, Z3 ′, Z4 ′), a target as shown in FIG. A function indicating the relationship between the depth position of the pattern and the depth position of the image data is calculated. Based on the calculated function, the depth position Z5 of the target pattern corresponding to the depth position Z5 'of the image data is specified, and the correction amount in the depth direction for the depth position Z5' is calculated. Thereby, calibration data of the depth position Z5 'including the two-dimensional coordinate conversion information and the correction amount in the depth direction is obtained.

  Finally, the pattern irradiation apparatus 100 records the calibration data generated in step S42 and step S43 (step S44).

  According to the calibration data generated by the above processing, the pattern irradiation apparatus 100 can be accurately calibrated. For this reason, the pattern irradiation apparatus 100 can accurately form a two-dimensional light pattern at an arbitrary depth position. In addition, the pattern irradiation apparatus 100 can form a three-dimensional light pattern with high accuracy.

  This point will be further described. Since the path of the light beam passing through the apparatus is different for each depth position where the light pattern is formed, the calibration data (particularly, the two-dimensional coordinate conversion information) is different for each depth position. For this reason, when the shift of the pattern in the depth direction is not sufficiently taken into account, even if a plurality of calibration data is generated by changing the depth position, the depth at which the light pattern is actually formed The calibration data corresponding to the position cannot be specified. Therefore, it is difficult to calibrate the apparatus with high accuracy. On the other hand, in the pattern irradiation apparatus 100, the calibration data of each depth position is produced | generated after specifying the irradiation depth position in which the irradiation pattern was formed correctly. For this reason, it is possible to correctly specify calibration data to be used, and as a result, it is possible to calibrate the apparatus with high accuracy.

  In the pattern irradiation apparatus 100, calibration data at an arbitrary depth position can be generated by interpolating calibration data at other depth positions. That is, in order to generate calibration data at an arbitrary depth position, it is not necessary to actually irradiate the pattern each time, and the number of times the sample is irradiated with the light pattern can be suppressed. Therefore, calibration data at an arbitrary depth position can be easily added as necessary.

  FIG. 14 is a diagram illustrating a configuration of the pattern irradiation apparatus 200 according to the present embodiment. The pattern irradiation apparatus 200 shown in FIG. 2 is different from the pattern irradiation apparatus 100 of the first embodiment in the configuration of the microscope.

  The microscope constituting the pattern irradiation apparatus 200 includes an illumination system (laser 1, beam expander 2, prism 3, LCOS-SLM 4, pupil relay lens 18) for forming an arbitrary light pattern on the sample S, and the sample S. Is different from the microscope of the pattern irradiation apparatus 100 in that an optical system (laser 21, beam expander 22, galvano mirror 23, pupil relay lens 24) is separately provided. For this reason, an image of the specimen S can be acquired while forming an arbitrary light pattern on the specimen S. In addition, switching from the calibration pattern irradiation process to the image acquisition process can be performed quickly.

  In addition to the non-confocal detection system (relay lens 13, PMT 14), the microscope constituting the pattern irradiation apparatus 200 is a detection system that acquires an image of the specimen S with a CCD camera 27 having a two-dimensional CCD image sensor. It is also different from the microscope of the pattern irradiation apparatus 100 in that it is provided with (IR cut filter 25, imaging lens 26, CCD camera 27). When an image of the sample S is acquired by the CCD camera 27, the dichroic mirror 10 is removed out of the optical path.

  Also by the pattern irradiation apparatus 200 according to the present embodiment, it is possible to generate the calibration data by executing the same process as the pattern irradiation apparatus 100 according to the first embodiment. For this reason, the effect similar to the pattern irradiation apparatus 100 which concerns on Example 1 can be acquired.

  The above-described embodiments are specific examples for facilitating understanding of the invention, and the present invention is not limited to these embodiments. The calibration data generation method and the pattern irradiation apparatus can be variously modified and changed without departing from the concept of the present invention defined by the claims.

  For example, in the above-described embodiment, the LCOS-SLM is exemplified as the wavefront modulator, but, for example, a deformable mirror may be employed. In the above-described embodiment, the configuration of the two-photon excitation microscope is exemplified. However, a light branching element such as a dichroic mirror is disposed on the light source side of the galvano mirror, and a detector such as a pinhole and a PMT is disposed. The present invention may be applied to a confocal laser microscope using excitation. Moreover, although the example which specifies the irradiation depth position based on the contrast value of image data was shown, you may specify an irradiation depth position by calculating another evaluation value instead of a contrast value. For example, the irradiation depth position may be specified based on the position at which the faded area is minimum on the two-dimensional image or the barycentric position of the faded three-dimensional area. Moreover, although the case where the calibration pattern includes a plurality of the same target patterns at different depth positions has been described as an example, the plurality of target patterns may be different patterns. However, when the calibration data is generated by interpolating the three-dimensional coordinate information, it is desirable that the plurality of target patterns are the same pattern. Further, since the intensity distribution of the pattern formed by the depth position changes, the target pattern may be set in advance in consideration of the change in the intensity distribution that occurs depending on the depth position. For this reason, the calibration pattern may be set to include a plurality of target patterns having the same two-dimensional coordinate information and different intensity distributions.

1, 21 Laser 2, 22 Beam expander 3 Prism 4 LCOS-SLM
5, 13 Relay lens 6, 23 Galvano mirror 7, 18, 24 Pupil relay lens 8, 26 Imaging lens 9 Mirror 10, 19, 20 Dichroic mirror 11 Objective lens 12 Stage 14 PMT
15 Computer 15a Arithmetic Control Unit 15b Storage Unit 16 Monitor 17 Input Device 25 IR Cut Filter 27 CCD Camera 100, 200 Pattern Irradiation Device S Sample

Claims (11)

A method for generating calibration data of a pattern irradiation apparatus including a wavefront modulator,
Irradiating the specimen with light whose wavefront is modulated by the wavefront modulator based on a calibration pattern including a two-dimensional target pattern orthogonal to the depth direction of the specimen;
Generate image data of the specimen irradiated with the light for each depth position,
Based on the generated plurality of image data, specify the irradiation depth position, which is the depth position where the two-dimensional irradiation pattern is formed,
A calibration data generation method for generating calibration data for forming a pattern at the irradiation depth position based on the image data of the irradiation depth position and the target pattern. The calibration data generation method according to claim 1,
Generating the calibration data for forming a pattern at the illumination depth position;
Acquiring three-dimensional coordinate information of the irradiation pattern formed at the irradiation depth position based on the image data of the irradiation depth position;
Generating calibration data for forming a pattern at the irradiation depth position based on the three-dimensional coordinate information of the irradiation pattern and the three-dimensional coordinate information of the target pattern. Generation method. The calibration data generation method according to claim 2,
Irradiating the sample with light whose wavefront is modulated by the wavefront modulator based on the calibration pattern including the target pattern at a plurality of different depth positions;
Based on the plurality of image data, identify a plurality of irradiation depth positions where a two-dimensional irradiation pattern is formed,
Based on a plurality of image data at the plurality of irradiation depth positions, three-dimensional coordinate information of the plurality of irradiation patterns formed at the plurality of irradiation depth positions is acquired, and three-dimensional coordinate information of the plurality of irradiation patterns is obtained. And generating a plurality of calibration data for forming a pattern at the plurality of irradiation depth positions based on the three-dimensional coordinate information of a plurality of target patterns having different depth positions,
A method for generating calibration data, comprising: interpolating the plurality of calibration data to generate calibration data for forming a pattern at a first depth position other than the plurality of irradiation depth positions. The calibration data generation method according to claim 2,
Irradiating the sample with light whose wavefront is modulated by the wavefront modulator based on the calibration pattern including the target pattern at a plurality of different depth positions;
Based on the plurality of image data, identify a plurality of irradiation depth positions where a two-dimensional irradiation pattern is formed,
Based on a plurality of image data at the plurality of irradiation depth positions, three-dimensional coordinate information of the plurality of irradiation patterns formed at the plurality of irradiation depth positions is acquired, and three-dimensional coordinate information of the plurality of irradiation patterns is obtained. And generating a plurality of calibration data for forming a pattern at the plurality of irradiation depth positions based on the three-dimensional coordinate information of a plurality of target patterns having different depth positions,
Based on the three-dimensional coordinate information of the irradiation pattern formed at the first depth position other than the plurality of irradiation depth positions generated by interpolating the three-dimensional coordinate information of the plurality of irradiation patterns, the first A calibration data generation method for generating calibration data for forming a pattern at a depth position. In the calibration data generation method according to claim 3 or 4,
2. The calibration data generation method according to claim 1, wherein the calibration pattern includes the plurality of target patterns having the same two-dimensional coordinate information and different intensity distributions. In the calibration data generation method according to any one of claims 1 to 5,
The calibration data for forming a pattern at the irradiation depth position includes two-dimensional coordinate conversion information of the irradiation depth position and a depth position correction amount for forming a pattern at the irradiation depth position. The calibration data generation method characterized by the above-mentioned. The calibration data generation method according to any one of claims 1 to 6,
The calibration data generation method, wherein the target pattern is a two-dimensional pattern including two or more condensing points. In the calibration data generation method according to any one of claims 1 to 7,
A calibration data generation method, wherein the contrast value of each of the plurality of image data is calculated, and the irradiation depth position is specified based on the calculated plurality of contrast values. The calibration data generation method according to any one of claims 1 to 8,
The calibration data generation method, wherein the wavefront modulator is a spatial light modulator equipped with LCOS or a deformable mirror. In the calibration data generation method according to any one of claims 1 to 9,
A method for generating calibration data, wherein the specimen is a three-dimensional fluorescent specimen. An irradiating means for irradiating the sample with light whose wavefront is modulated by the wavefront modulator based on a calibration pattern including a wavefront modulator and including a two-dimensional target pattern orthogonal to the depth direction of the sample;
Image data generation means for generating image data of the specimen irradiated with the light by the irradiation means for each depth position;
Based on a plurality of image data generated by the image data generation means, an irradiation depth position that is a depth position where a two-dimensional irradiation pattern is formed is specified, and the image data of the specified irradiation depth position and the Calibration data generation means for generating calibration data for forming a pattern at the irradiation depth position based on a target pattern.