WO2021161432A1

WO2021161432A1 - Microsope system, setting search method, and program

Info

Publication number: WO2021161432A1
Application number: PCT/JP2020/005412
Authority: WO
Inventors: 伸吾鈴木
Original assignee: オリンパス株式会社
Priority date: 2020-02-12
Filing date: 2020-02-12
Publication date: 2021-08-19
Also published as: JPWO2021161432A1; JP7369801B2

Abstract

A microscope system 1 is provided with a microscope device and a computing device 20. The microscope device has a correction collar 111, and acquires a plurality of pieces of image data by acquiring image data in each of a plurality of states having different settings of the correction collar 111. The computing device 20 specifies the setting of the correction collar 111 that is to be corrected for spherical aberration on the basis of a plurality of contrast values including the respective contrast values of the plurality of pieces of image data. The computing device 20 specifies, as a first position, the position of pixel data having saturated pixel values included in the plurality of pieces of image data, and except for the pixel data at the first position included in the plurality of image data, computes each of the plurality of contrast values.

Description

Microscope system, setting search method, and program

The present disclosure relates to a microscope system, a setting search method for searching a setting for correcting spherical aberration, and a program.

Conventionally, the correction ring of the microscope system has been used as a means for correcting spherical aberration caused by the thickness of the cover glass. In recent years, when a method for observing a deep part of a sample (for example, a biological sample) has been developed, a correction ring is also used for the purpose of correcting spherical aberration that changes according to the depth of an observation target surface. Such techniques are described, for example, in Patent Documents 1 to 3.

Also, the amount of spherical aberration generated inside the sample depends on the refractive index distribution of the sample. Therefore, if the position of the correction ring in which the spherical aberration is corrected can be known, the refractive index of the sample can be calculated back. Such a technique is described in, for example, Patent Document 2 and Patent Document 3.

Whether or not the spherical aberration is corrected can be determined based on an evaluation value (hereinafter referred to as a contrast value) that evaluates the contrast of an image, as described in Patent Documents 1 to 3. .. This is because in the state where the spherical aberration is corrected, an image having higher contrast can be obtained as compared with the state where the spherical aberration is not corrected.

Japanese Unexamined Patent Publication No. 2014-160213 JP-A-2017-0266664 JP-A-2017-0266665

By the way, in order to accurately determine whether or not spherical aberration is corrected, it is desirable to calculate the contrast value using an image having a high S / N ratio. This is because in an image having a low S / N ratio, the influence of noise irrelevant to the amount of spherical aberration becomes relatively large, and it is difficult to accurately grasp the change in contrast due to spherical aberration. An image having a high S / N ratio can be obtained, for example, by increasing the intensity of the illumination light and increasing the signal component of the image with respect to the noise component.

However, if the intensity of the illumination light becomes too strong, the proportion of saturated pixels contained in the image will increase. As the proportion of saturated pixels increases, the contrast value may be calculated lower than it actually is, and as a result, the inaccurate position may be recognized as the position of the correction ring in which spherical aberration is corrected. .. Further, even if the intensity of the illumination light is adjusted in a state where the optimum position of the correction ring for which spherical aberration is corrected is not determined, the brightness can be further increased by changing the setting of the correction ring in the process of adjusting the correction ring. It may change, resulting in an increase in the proportion of saturated pixels. That is, the same problem can occur even if the intensity of the illumination light is properly adjusted before the correction ring adjustment is started.

In the above, the correction ring has been described as an example, but the same technical problem may occur if the correction device is not limited to the correction ring and corrects spherical aberration.

Based on the above circumstances, one object of the present disclosure is to provide a technique for accurately specifying a setting in which spherical aberration is corrected even when saturated pixels occur.

The microscope system according to one aspect of the present invention has a spherical aberration correction device, and acquires a plurality of image data by acquiring image data in each of a plurality of states in which the settings of the spherical aberration correction device are different. And an arithmetic device that specifies the setting of the spherical aberration correction device that corrects spherical aberration based on a plurality of contrast values including the contrast values of the plurality of image data. The position of the pixel data in which the pixel values included in the plurality of image data is saturated is specified as the first position, and the plurality of image data are excluded from the pixel data of the first position included in each of the plurality of image data. Each of the contrast values is calculated.

The setting search method according to the embodiment of the present invention is a setting search method for searching for a setting in which spherical aberration is corrected, and acquires image data in each of a plurality of states in which the settings of the spherical aberration correction device are different. Acquires a plurality of image data in The spherical aberration correction in which the contrast value of each of the plurality of image data is calculated excluding the pixel data of the above, and the spherical aberration is corrected based on the plurality of contrast values including the contrast value of each of the plurality of image data. Identify the device settings.

The program according to the embodiment of the present invention specifies to the computer the position of the pixel data saturated with the pixel values included in the plurality of image data acquired with different settings of the spherical aberration correction device as the first position. , The contrast value of each of the plurality of image data is calculated excluding the pixel data of the first position included in each of the plurality of image data, and the plurality of including the contrast value of each of the plurality of image data. The process of specifying the setting of the spherical aberration correction device for which the spherical aberration is corrected based on the contrast value of the above is executed.

According to the present disclosure, it is possible to accurately specify the setting in which spherical aberration is corrected even when saturated pixels occur.

It is a figure which illustrated the structure of the microscope system which concerns on 1st Embodiment. It is a figure which illustrated the structure of the arithmetic unit shown in FIG. It is a figure which illustrated the structure of the microscope apparatus shown in FIG. It is a flowchart of the correction ring setting process which concerns on 1st Embodiment. It is a flowchart of the saturated pixel position specifying process. It is a flowchart of contrast calculation processing. It is a figure for demonstrating the conventional method of calculating a contrast value. It is a figure for demonstrating an example of the calculation method of the contrast value in 1st Embodiment. It is a figure for demonstrating another example of the method of calculating a contrast value in 1st Embodiment. It is a flowchart of the correction ring setting process which concerns on 2nd Embodiment. It is a flowchart of the modification of the correction ring setting process shown in FIG. It is a flowchart of the relation calculation process which concerns on 3rd Embodiment. It is a figure for demonstrating an example of the calculation method of the relationship between the setting of a correction ring and the observation depth. It is a flowchart of the Z-Series shooting process which concerns on 3rd Embodiment. It is a flowchart of the refractive index display processing which concerns on 4th Embodiment. It is a figure for demonstrating an example of the display method of a refractive index. It is another figure for demonstrating an example of the display method of a refractive index. It is a figure for demonstrating still another example of the calculation method of a contrast value.

[First Embodiment]
FIG. 1 is a diagram illustrating the configuration of the microscope system 1 according to the present embodiment. FIG. 2 is a diagram illustrating the configuration of the arithmetic unit 20 shown in FIG. FIG. 3 is a diagram illustrating the configuration of the microscope 100 shown in FIG.

The microscope system 1 shown in FIG. 1 includes a microscope 100, a microscope control device 10, an arithmetic unit 20, a display device 30, and a plurality of input devices (keyboard 40, correction ring) for inputting instructions to the arithmetic unit 20. The operation device 50 and the focusing operation device 60) are provided. Hereinafter, the microscope 100 and the microscope control device 10 will be collectively referred to as a microscope device.

The microscope control device 10 is a device that controls the operation of the microscope 100 according to an instruction from the arithmetic unit 20, and generates control signals for controlling the operation of various electric parts of the microscope 100. In addition, image data is generated based on the signal from the microscope 100. The microscope control device 10 includes a light source control device 11 that controls the output of the light source, a zoom control device 12 that controls the zoom magnification, and a position of the observation target surface in the optical axis direction (hereinafter, simply referred to as the position of the observation target surface). ), And a correction ring control device 14 for controlling the setting of the correction ring 111. The correction ring 111 is an example of a spherical aberration correction device that corrects spherical aberration, and the correction ring control device 14 is an example of a correction control device. The setting of the correction ring 111 is, for example, the rotation angle of the correction ring 111 with respect to the reference position (hereinafter, simply referred to as the angle of the correction ring 111).

The arithmetic unit 20 is a computer that performs various arithmetic processes. For example, as shown in FIG. 2, the processor 21, the memory 22, the input I / F device 23, the output I / F device 24, and the portable recording medium 26 are included. A portable recording medium drive device 25 to be inserted is provided, and these are connected to each other by a bus 27. Note that FIG. 2 is an example of the configuration of the arithmetic unit 20, and the arithmetic unit 20 is not limited to this configuration.

Processor 21 includes one or more processors. The one or more processors may include, for example, a central processing unit (CPU: Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), and the like. Further, ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) and the like may be included. The processor 21 may, for example, execute a predetermined software program to perform arithmetic processing.

The memory 22 includes a non-temporary computer-readable medium that stores a software program executed by the processor 21. The memory 22 may include, for example, one or more arbitrary semiconductor memories, and may further include one or more other storage devices. The semiconductor memory includes, for example, a volatile memory such as a RAM (Random Access Memory), a non-volatile memory such as a ROM (Read Only Memory), a programmable ROM, and a flash memory. The RAM may include, for example, a DRAM (Dynamic Random Access Memory), a SRAM (Static Random Access Memory), or the like. Other storage devices may include, for example, a magnetic storage device including a magnetic disk, for example, an optical storage device including an optical disk.

The input I / F device 23 receives signals from the keyboard 40, the correction ring operation device 50, the focusing operation device 60, and the display device 30. The input I / F device 23 also receives a signal from the microscope 100. The output I / F device 24 outputs a signal to the display device 30 and the microscope control device 10. The portable recording medium driving device 25 accommodates the portable recording medium 26.

The arithmetic unit 20 operates as various means by the processor 21 reading the program stored in the memory 22 or the portable recording medium 26 into the memory 22 and executing the program. The calculation device 20 uses, for example, a means for calculating the contrast value of the image data (contrast calculation means), a means for calculating the setting of the correction ring 111 for which spherical aberration is corrected (target value calculation means), and a refractive index of the sample S. It operates as a means for calculating (refractive index calculation means) and a means for controlling the display device 30 (display control means).

The display device 30 is, for example, a liquid crystal display device, an organic EL display device, a CRT display device, or the like. The display device 30 may include a touch panel sensor, and in that case, also functions as an input device.

The correction ring operation device 50 is an input means for instructing the setting of the correction ring 111. When the user instructs the correction ring operation device 50 to set the correction ring 111, the correction ring control device 14 changes the setting of the correction ring 111 to the instructed setting.

The focusing operation device 60 is an input means for instructing a change in the position (that is, the observation depth) of the observation target surface. When the user instructs the change of the position of the observation target surface using the focusing operation device 60, the focusing control device 13 moves the focusing device 109 in the optical axis direction to change the position of the observation target surface.

The microscope 100 is, for example, a two-photon excitation microscope. Sample S is, for example, a biological sample such as mouse brain, but is not limited to the biological sample. As shown in FIG. 3, the microscope 100 includes a laser 101, a scanning unit 102, a pupil projection optical system 103, a mirror 104, a dichroic mirror 105, and an objective lens 110 on the illumination optical path.

The laser 101 is, for example, an ultrashort pulse laser, which oscillates laser light in the near infrared region. The output of the laser 101 is controlled by the light source control device 11. That is, the light source control device 11 is a laser control device that controls the output of the laser light irradiating the sample.

The scanning unit 102 is a scanning means for scanning the sample S two-dimensionally with a laser beam, and includes, for example, a galvano scanner and a resonant scanner. The zoom magnification changes as the scanning range of the scanning unit 102 changes. The scanning range of the scanning unit 102 is controlled by the zoom control device 12.

The pupil projection optical system 103 is an optical system that projects the image of the scanning unit 102 onto the pupil position of the objective lens 110. The dichroic mirror 105 is an optical separation means for separating the excitation light (laser light) and the detection light (fluorescence) from the sample S, and separates the laser light and the fluorescence according to the wavelength.

The objective lens 110 is a dry or immersion objective lens provided with a correction ring 111, and is attached to the focusing device 109. The focusing device 109 is a means for moving the objective lens 110 in the optical axis direction of the objective lens 110, and the movement of the focusing device 109 (that is, the movement of the objective lens 110) is controlled by the focusing control device 13. ..

The correction ring 111 is a correction device that corrects spherical aberration by moving a part of the lenses constituting the objective lens 110 in the optical axis direction according to the setting. The setting of the correction ring 111 is changed by the correction ring control device 14 (correction device control device). The setting of the correction ring 111 can also be changed manually by directly operating the correction ring 111.

The microscope 100 further includes a pupil projection optical system 106 and a photodetector 107 on the detection optical path (reflected optical path of the dichroic mirror 105). The signal output from the photodetector 107 is output to the A / D converter 108.

The pupil projection optical system 106 is an optical system that projects the pupil image of the objective lens 110 onto the photodetector 107. The photodetector 107 is, for example, a photomultiplier tube (PMT), which outputs an analog signal according to the amount of incident fluorescence. The A / D converter 108 converts the analog signal from the photodetector 107 into a digital signal (luminance signal) and outputs it to the microscope control device 10. The sensitivity of the photodetector 107 is controlled by a detection sensitivity control device (not shown). The detection sensitivity control is, for example, any one of adjustment of the applied voltage applied to the photodetector 107, adjustment of the amplification factor of the analog signal output from the photodetector 107, adjustment of the amplification factor at the stage of the digital signal, or , A combination of them.

In the microscope system 1 configured as described above, the microscope 100 uses the scanning unit 102 to scan the sample S with laser light in a direction orthogonal to the optical axis of the objective lens 110, and from each position of the sample S. The fluorescence is detected by the photodetector 107. The microscope control device 10 generates image data based on the analog signal sampled by the A / D converter 108 in accordance with the scanning timing of the scanning unit 102, and outputs the image data to the arithmetic unit 20. That is, the image data is acquired by a microscope device including the microscope 100 and the microscope control device 10.

FIG. 4 is a flowchart of the correction ring setting process according to the first embodiment. FIG. 5 is a flowchart of the saturated pixel position specifying process. FIG. 6 is a flowchart of the contrast calculation process. Hereinafter, the correction ring setting process performed by the microscope system 1 in order to correct the spherical aberration generated on the observation target surface will be described with reference to FIGS. 4 to 6. The correction ring setting process shown in FIG. 4 is an example of a setting search method for searching for a setting in which spherical aberration is corrected. For example, a user operates a focusing operation device 60 to determine the position of an observation target surface. Started by specifying.

First of all, the microscope system 1 accepts the designation of the range (hereinafter referred to as the region of interest) to be the target of the contrast evaluation in the sample S (step S1). The designation of the region of interest by the observer has, for example, a portion to be observed better (for example, a characteristic shape in the sample S) while the observer is viewing the live image of the sample S displayed on the display device 30. Part) is included by using an input device such as a keyboard 40. When evaluating the contrast of the entire image, step S1 may be omitted.

Next, the microscope system 1 changes the setting of the correction ring 111 to the initial setting (step S2). Here, the arithmetic unit 20 controls the correction ring control device 14 to set the angle of the correction ring 111 to, for example, the angle of one end of the movable range.

After that, the microscope system 1 acquires the image data (step S3) and determines whether or not the image data has been acquired with all the predetermined correction ring 111 settings (step S4). When it is determined that the image data has not been acquired by setting all the correction rings 111 (step S4NO), the microscope system 1 changes the settings of the correction ring 111 (step S5), returns to step S3, and re-uses the image data. get. As a result, the microscope system 1 acquires the image data (step S3) and changes the settings of the correction ring 111 (step S5) until the image data is acquired with all the predetermined correction ring 111 settings (step S4YES). )repeat. Here, the arithmetic unit 20 controls the correction ring control device 14 to move the angle of the correction ring 111 from the initial position, which is one end of the movable range, by a predetermined angle to the other end of the movable range. During such a series of control operations of the correction ring control device 14, the microscope control device 10 acquires image data at each angle of the correction ring 111 and outputs the image data to the calculation device 20, so that the calculation device 20 is the same. Acquire a plurality of image data for the observation target surface. That is, in steps S3 to S5, the microscope device acquires a plurality of image data by acquiring image data in each of a plurality of states in which the setting of the correction ring 111 is different, and outputs the plurality of image data to the arithmetic unit 20. The arithmetic unit 20 stores a plurality of image data acquired from the microscope device in the memory 22.

When a plurality of image data are acquired, the microscope system 1 identifies the positions of saturated pixels included in the image data (step S6). Here, the arithmetic unit 20 starts, for example, the saturated pixel position specifying process shown in FIG. 5, and specifies the position of the pixel data in which the pixel values included in the plurality of image data are saturated as the first position.

Specifically, the arithmetic unit 20 reads one image data from the plurality of image data stored in the memory 22 (step S11), and does the read image data include pixel data with saturated pixel values? Whether or not it is determined (step S12). Then, when pixel data in which the pixel value is saturated is included, the position of the pixel of the pixel data (hereinafter referred to as saturated pixel) is stored as the first position (step S13). The arithmetic unit 20 performs the processes of steps S11 to S13 for all of the plurality of image data stored in the memory 22 (step S14YES), and ends the saturated pixel position specifying process shown in FIG.

When the saturated pixel position identification process is completed, the microscope system 1 calculates the contrast value of each of the plurality of image data (step S7). Here, the arithmetic unit 20 starts, for example, the contrast calculation process shown in FIG. 6 and calculates a plurality of contrast values from the plurality of image data.

Specifically, the arithmetic unit 20 reads the first position, which is the position of the saturated pixel, from the memory 22 (step S21). Further, the arithmetic unit 20 reads one image data from the plurality of image data stored in the memory 22 (step S22), and uses the image data and the first position to set the contrast value of the read image data. Calculate (step S23). The details of the method of calculating the contrast value will be described later. The arithmetic unit 20 performs the processes of steps S22 and S23 on all of the plurality of image data stored in the memory 22 (step S24YES), and ends the contrast calculation process shown in FIG.

When the contrast calculation process is completed and a plurality of contrast values including the contrast values of the plurality of image data are calculated, the microscope system 1 specifies a setting for correcting spherical aberration as a target setting (step S8). .. Here, the arithmetic unit 20 specifies the setting of the correction ring 111 for which the spherical aberration is corrected, that is, the angle of the correction ring 111, based on the plurality of contrast values calculated in step S7. More specifically, the arithmetic unit 20 specifies, for example, the maximum contrast value among a plurality of contrast values, and sets a target for the angle of the correction ring 111 when the image data having the maximum contrast value is acquired. May be specified as. Further, the angle of the correction ring 111 that maximizes the contrast value may be estimated from the combination of the plurality of contrast values and the angle of the correction ring 111 corresponding thereto, and the estimated angle may be specified as the target setting.

Finally, the microscope system 1 changes the setting of the correction ring 111 to the target setting (step S9), and ends the correction ring setting process. Here, the correction ring control device 14 changes the setting of the correction ring 111 to the setting specified in step S8 according to the instruction of the arithmetic unit 20. As a result, spherical aberration is corrected, so that the sample S can be observed with good image quality.

FIG. 7 is a diagram for explaining a conventional method of calculating a contrast value. FIG. 8 is a diagram for explaining an example of the method of calculating the contrast value in the present embodiment. Hereinafter, the contrast calculation method performed by the microscope system 1 will be described with reference to FIGS. 7 and 8, focusing on points different from the conventional contrast calculation method.

As one of the evaluation formulas for evaluating the contrast of an image, an evaluation formula for evaluating the contrast of an image based on the difference in pixel values between pixels is known. As a specific evaluation formula, for example, the following formula for calculating the contrast value by integrating the square of the difference between the pixel values of two pixels located at positions shifted by n pixels in the x direction over the entire image data is known. Has been done. The following formula is J. F. It is an evaluation formula proposed by Brenner et al. And is called a Brenner gradient.

Here, _FBrenner is a contrast value, x is a variable that specifies a column of pixels that make up an image, and y is a variable that specifies a row of pixels that make up an image. W is the number of pixels in the x direction (that is, the number of columns) of the pixels that make up the image, and H is the number of pixels in the y direction (that is, the number of rows) of the pixels that make up the image. f is a pixel value of pixel data corresponding to the specified pixel. n is a shift amount, and is an integer (for example, 2 or the like) indicating an interval between pixels for which the difference between pixel values is calculated.

An evaluation formula that evaluates the contrast of an image based on the difference in pixel values between pixels, such as Brenner gradient, is more than an evaluation formula that evaluates the contrast of an image from the maximum pixel value and the minimum pixel value included in the image data. , The contrast change that occurs in the image can be captured in detail. Therefore, it is suitable for evaluating the corrected state of spherical aberration.

On the other hand, when an image is acquired in a state where the illumination intensity is strong in order to increase the S / N ratio, as shown in FIG. 7, the number of saturated pixels increases as the spherical aberration is corrected and the image becomes brighter. In some cases, the conventional contrast calculation method using the Brener gradient may make a mistake in setting the target. This is because when the number of saturated pixels increases, the difference in pixel values between saturated pixels becomes 0, for example, and the difference in pixel values between pixels including saturated pixels is estimated to be smaller than the actual value. As a result, the calculated contrast value falls below the actual contrast value.

In FIG. 7, pixel data included in the image data acquired when the angle θ of the correction ring 111 is 0 °, 10 °, and 20 ° is schematically drawn. In FIG. 7, an image composed of pixels arranged in 1 row and 6 columns is drawn for simplification of the description, but the number of rows and columns of the pixels constituting the image is not limited to this example. In this example, when the pixel data is compared between the image data, it can be confirmed that the pixel value tends to increase as the angle θ is increased. Therefore, when the angle θ is 20 °, the spherical aberration is most corrected. It is thought that it is in. However, since the calculated contrast value is larger when the angle θ = 10 ° than when the angle θ = 20 °, the arithmetic unit 20 misidentifies the angle θ = 10 ° as the target setting. Resulting in.

In this way, if the brightness is adjusted to increase the S / N ratio in order to accurately detect the change in contrast due to the fluctuation of spherical aberration, the conventional calculation method may cause the target setting to be incorrect. obtain.

Therefore, in the present application, the above problem is solved by paying attention to the fact that even if the images are compared by excluding a part of the pixel data included in the image data, the relationship of brightness between the images is not significantly affected. Propose a technology to do.

Specifically, in step S23 of FIG. 6, the arithmetic unit 20 excludes the pixel data at the first position included in the image data read in step S22 from the calculation for calculating the contrast value of the read image data. do. That is, the contrast value of the image data is calculated by excluding the pixel data at the first position included in the image data. More specifically, as shown in FIG. 8, when calculating the contrast value of each of the plurality of image data, the position of the pixel data in which the pixel value is saturated in at least one of the plurality of image data (the first position). 1 position) is specified, and the pixel data corresponding to the first position is excluded from the calculation of the contrast value of each image data regardless of whether the pixel value of the corresponding pixel data is saturated or not. do. The contrast calculation method performed by the microscope system 1 is the same as the conventional contrast calculation method in that an evaluation formula for evaluating the contrast of an image based on the difference in pixel values between pixels is used.

A more detailed explanation will be given with reference to FIG. Similar to FIG. 7, FIG. 8 schematically depicts pixel data included in the image data acquired when the angles θ of the correction ring 111 are 0 °, 10 °, and 20 °. In this example, the image data is 8-bit image data, and the pixel values are saturated in the third and fourth pixels from the left of the image acquired at an angle θ = 20 °. Therefore, in step S23, the arithmetic unit 20 calculates the contrast value of the image data by excluding the third and fourth pixel data from the left of all the image data. More specifically, for all images (images acquired at angles θ = 0 °, 10 °, 20 °), between pixels including at least one of the third and fourth pixels from the left (specifically, , 2nd and 3rd from the left, 3rd and 4th, 4th and 5th) are excluded from the integration target for calculating the contrast value, and the contrast value is calculated.

This makes it possible to eliminate the inaccuracy of the difference between pixels caused by the inclusion of saturated pixels. Furthermore, even if the contrast value is calculated by ignoring the presence of some pixels, as long as the contrast values are equally ignored in each of the plurality of images, as shown by the contrast value relationship in FIG. The relative relationship of contrast is generally maintained. Therefore, even when saturated pixels are included, the contrast between a plurality of images can be correctly compared and evaluated. Therefore, according to the microscope system 1, it is possible to accurately specify the setting in which the spherical aberration is corrected even when saturated pixels are generated.

Further, according to the microscope system 1, spherical aberration can be appropriately corrected even when saturated pixels are generated, so that the output of the light source is adjusted before starting the process of searching for the target setting shown in FIG. Brightness adjustment such as sensitivity adjustment of the detector and detector becomes easy. That is, the user can adjust the brightness to obtain a good S / N ratio without being excessively worried about the generation of saturated pixels. Further, even when a new saturated pixel is generated by changing the setting of the correction ring in the process of adjusting the spherical aberration, the setting for correcting the spherical aberration can be accurately specified.

In the above, an example is shown in which the position of the saturated pixel is specified as the first position and the contrast value of each image data is calculated by excluding the pixel data of the pixel at the first position. Pixel data may be excluded from the calculation. Specifically, the arithmetic unit 20 specifies the position of the pixel data whose pixel value is less than the threshold value included in the plurality of image data acquired for the same observation target surface as the second position. Each of the plurality of contrast values may be calculated excluding the pixel data of the first position and the second position included in each of the plurality of image data. This point is the same in the subsequent embodiments. It is desirable that this threshold value is determined based on, for example, a pixel value (hereinafter referred to as background luminance) corresponding to a signal output from a photodetector in a state where no light is incident.

It is assumed that the pixel data having a pixel value less than the threshold value determined based on the background brightness corresponds to the part of the image where nothing is displayed. Therefore, by excluding pixel data having a pixel value less than the threshold value from the calculation of the contrast value, the contrast value can be calculated while suppressing the influence of background noise. Therefore, by specifying both the first position and the second position and excluding the pixel data at those positions from the calculation of the contrast value, it is possible to more accurately specify the setting in which the spherical aberration is corrected. .. Explaining in detail with reference to FIG. 9, at the correction ring position of θ = 0 °, the brightness value of the sixth pixel from the left (the rightmost pixel) is the background brightness (here, the background brightness value is 10). ) Is less than. Therefore, in the image obtained by acquiring the difference between the fifth and sixth pixel values from the left at all the correction ring positions (θ = 0 °, θ = 10 °, θ = 20 °), the contrast value is calculated. It may be excluded from the calculation target.

[Second Embodiment]
FIG. 10 is a flowchart of the correction ring setting process according to the present embodiment. The microscope system according to the present embodiment (hereinafter, simply referred to as a microscope system) is different from the microscope system 1 in that the correction ring setting process shown in FIG. 10 is performed instead of the correction ring setting process shown in FIG. It's different. Other points are the same as those of the microscope system 1. The correction ring setting process shown in FIG. 10 is different from the correction ring setting process shown in FIG. 4 in that the amount of light is automatically adjusted during the correction ring setting process.

Specifically, the microscope system first acquires a plurality of image data by acquiring image data in each of a plurality of states in which the setting of the correction ring 111 is different in steps S31 to S36, and the plurality of image data. The saturated pixel position included in is specified as the first position. These processes are the same as the processes of steps S1 to S6 of FIG.

Next, the microscope system determines whether or not the ratio of saturated pixels is the threshold value TH1 or more (step S37). Here, the arithmetic unit 20 first reads the first position stored in the memory 22. After that, the arithmetic unit 20 calculates the ratio of saturated pixels, that is, the ratio of the number of first positions to the number of pixels of the image (= the number of first positions / the number of pixels of the image). Further, the arithmetic unit 20 makes the above determination by comparing the calculated ratio with the threshold value TH1. The threshold value TH1 is, for example, 0.5%.

As a result of the determination in step S37, when it is determined that the ratio of saturated pixels is equal to or greater than the threshold value TH1, the microscope system suppresses the amount of light emitted from the laser 101 (step S38), and further, the saturation specified in step S36. The pixel position is reset (step S39). After that, the microscope system again performs the processes from step S32 to step S37. This is because the saturated pixels are generated above the threshold value TH1, and it is considered that the amount of illumination light is too strong. In step S38, the arithmetic unit 20 controls the light source control device 11, and the light source control device 11 follows an instruction from the arithmetic unit 20 to determine the ratio of pixel data in which the pixel values included in the plurality of image data are saturated (saturated pixels). The output of the laser 101 is controlled according to the ratio of the laser 101. More specifically, the light source control device 11 reduces the output of the laser 101 when the ratio of saturated pixels is the threshold value TH1 or more. The amount of decrease in the output of the laser 101 is not particularly limited, but is, for example, a predetermined constant amount. Further, the output of the laser 101 may be estimated so that the ratio of saturated pixels does not exceed the threshold value TH1, and the output of the laser 101 may be reduced so as to obtain the estimated output. The reset process in step S39 is performed for the purpose of preventing the saturated pixel position specified before the light amount suppression from being confused with the saturated pixel position in the current light amount setting.

As a result of the determination in step S37, when it is determined that the ratio of saturated pixels is less than the threshold value TH1, the microscope system calculates the contrast value of each of the plurality of image data (step S40), and further corrects the spherical aberration. The setting to be performed is specified as a target setting (step S41). Finally, the microscope system changes the setting of the correction ring 111 to the target setting (step S42), and ends the correction ring setting process. The processing of steps S40 to S42 is the same as the processing of steps S7 to S9 of FIG.

The same effect as that of the microscope system 1 can be obtained by the microscope system according to the present embodiment. If the proportion of pixels whose brightness is saturated in the microscope system 1 is too high, the accuracy of adjustment may decrease because there are few pixels that can be used for contrast calculation. Further, in the present embodiment, when the output of the laser 101 is too high, the amount of light is automatically adjusted to an appropriate level. According to this method, the proportion of pixels whose brightness is saturated can be reduced to increase the number of pixels that can be used for contrast calculation, so that the adjustment accuracy can be further improved as compared with the microscope system 1.

FIG. 11 is a flowchart of a modified example of the correction ring setting process shown in FIG. FIG. 10 shows an example in which the necessity of light intensity adjustment is determined based on the ratio of saturated pixels after the images are acquired with all the settings of the correction ring 111, but the determination of the necessity of light intensity adjustment is arbitrary timing. For example, as shown in FIG. 11, the necessity of adjusting the amount of light may be determined each time the image data is acquired.

In the correction ring setting process shown in FIG. 11, each time image data is acquired, the saturated pixel position is specified and whether or not the ratio of saturated pixels is equal to or greater than the threshold value is determined (steps S54 and S55). It is different from the correction ring setting process shown in.

The processing content in each step is the same as the processing content in the corresponding step of the correction ring setting processing shown in FIG. Therefore, detailed description of each step will be omitted.

According to the correction ring setting process according to the modification, the state in which the amount of light needs to be adjusted is determined by specifying the saturated pixel position and determining whether or not the ratio of saturated pixels is equal to or higher than the threshold value each time image data is acquired. It can be detected early. Therefore, it is possible to suppress the number of image acquisitions with an inappropriate light amount setting, and it is possible to suppress the adverse effect on the sample caused by the image acquisition. Further, since the light amount setting is optimized in a shorter time, the total processing time required for the setting processing can be shortened.

In the microscope system of this embodiment and the modified example of this embodiment, the output of the laser light source is controlled (suppressed) in order to reduce the proportion of saturated pixels. Alternatively or in addition to this, the sensitivity of the photodetector may be controlled (decreased) by a detection sensitivity control device (not shown). In this case, in the step S38 of the flowchart of FIG. 10 and the step of step S56 of the flowchart of FIG. 11, "photodetector sensitivity suppression" may be performed in place of or in addition to "light amount suppression". That is, in a microscope system, when the percentage of saturated pixels exceeds the threshold, both the output of the laser-light source and the sensitivity of the photodetector may be controlled, or only one of them may be controlled. ..

[Third Embodiment]
In the first embodiment and the second embodiment, when the user observes the observation target surface designated by operating the focusing operation device 60, a plurality of image data are acquired and the observation target surface is observed. An example is shown to identify the goal setting in. That is, an example is shown in which a plurality of image data are acquired each time the observation target surface is changed to specify the target setting on the observation target surface. On the other hand, in the present embodiment, the relationship between the position (observation depth) of the observation target surface and the target setting is specified in advance, and then the relationship specified in advance when observing the observation target surface designated by the user. Specify the target setting on the observation target surface specified from. That is, the target setting on an arbitrary observation target surface is specified without acquiring a plurality of image data each time the observation target surface is changed.

FIG. 12 is a flowchart of the relationship calculation process according to the present embodiment. FIG. 13 is a diagram for explaining an example of a method of calculating the relationship between the setting of the correction ring and the observation depth. FIG. 14 is a flowchart of the Z-Series shooting process according to the present embodiment. Hereinafter, with reference to FIGS. 12 to 14, a process of calculating the relationship between the setting of the correction ring 111 and the observation depth in the sample S and a process of performing Z-Series imaging using the calculated relationship will be described.

The Z-Series imaging is an imaging method in which acquisition of a two-dimensional image is repeated while moving the observation target surface by a predetermined distance in the depth direction of the sample S (that is, the optical axis direction of the objective lens 110). , Used to obtain 3D information of the sample. Z-Series photography is also referred to as Z-stack photography.

When the relationship calculation process shown in FIG. 12 is started according to the instruction of the user, the microscope system 1 first focuses on the interface between the sample S and the stage (step S71). Here, the arithmetic unit 20 controls the focusing control device 13, and the focusing control device 13 focuses on the interface between the sample S and the stage. This step can be performed by any known method. The observation depth (position of the observation target surface) at this time is set to D0 and is used as the reference for the observation depth.

After that, the microscope system 1 accepts the designation of the observation depth range (step S72). Here, the arithmetic unit 20 accepts the designation of the observation depth range by inputting the depth range that the user may observe as the observation depth range using an input device such as a keyboard 40. .. The observation depth range may be specified at least at both ends of the observation depth range, and any depth within the observation depth may be specified in addition to both ends of the observation depth range. That is, in step S72, at least two observation depths are specified.

Next, the microscope system 1 moves the observation target surface to the first observation depth (step S73). Here, the arithmetic unit 20 controls the focusing control device 13, and the focusing control device 13 transfers the observation target surface to, for example, the observation depth D1 which is one end of the observation depth range specified in step S72. Moving.

Further, the microscope system 1 specifies the setting of the correction ring 111 in which the spherical aberration on the observation target surface at the observation depth D1 is corrected in steps S74 to S81. These processes are the same as the processes of steps S1 to S8 of FIG.

After that, the microscope system 1 determines whether or not the setting is specified at all the observation depths specified in step S72 (step S82). Then, when there is an observation depth for which the setting is not specified (step S82NO), the microscope system 1 moves the observation target surface to the next observation depth (step S83), and moves to the observation target surface after the movement. On the other hand, the processes of steps S74 to S82 are repeated. Here, the arithmetic unit 20 controls the focusing control device 13, and the focusing control device 13 determines, for example, the observation depth D2 at which the observation target surface is the other end of the observation depth range specified in step S72. Move to.

When the setting is specified for all the observation depths specified in step S72 (step S83YES), the microscope system 1 calculates the relationship between the setting for correcting spherical aberration and the observation depth (step S84). Here, the arithmetic unit 20 uses the angle θ1 which is the target setting at the observation depth D1 specified in step 82 and the angle θ2 which is the target setting at the observation depth D2 specified in step 82 to form a spherical surface. The relationship between the setting at which the aberration is corrected and the observation depth is calculated. Specifically, as shown in FIG. 13, a broken line is formed by linearly interpolating the point P1 specified by the observation depth D1 and the angle θ1 and the point P2 specified by the observation depth D2 and the angle θ2. Calculate the relationship shown by. Here, an example of calculating the relationship by linear interpolation of two points is shown, but the relationship may be calculated from information of three or more points. In that case, for example, a function approximation such as the least squares method is used. The relationship may be calculated.

Finally, the microscope system 1 stores the calculated relationship (step S85) and ends the relationship calculation process shown in FIG. Here, the arithmetic unit 20 stores the calculated relationship in the memory 22.

When the Z-Series imaging process shown in FIG. 14 is started for the sample S after the relationship calculation process is completed, the microscope system 1 first accepts the designation of the observation depth range to be captured in the Z-Series imaging ( Step S91). Here, for example, the user inputs the range of the observation depth using an input device such as the keyboard 40, and the arithmetic unit 20 accepts the designation as the observation depth range. The arithmetic unit 20 further determines a plurality of observation depths for acquiring image data within the observation depth range. The plurality of observation depths may be distributed, for example, at predetermined intervals, and the predetermined intervals may be specified by the user together with the observation depth range in step S91.

When the observation depth range is specified and a plurality of observation depths to be photographed by Z-Series are determined, the microscope system 1 moves the observation target surface to the first observation depth (step S92). Here, the arithmetic unit 20 controls the focusing control device 13, and the focusing control device 13 moves, for example, the observation target surface to one observation depth determined in step S91.

Further, the microscope system 1 changes the setting of the correction ring 111 (step S93). Here, the arithmetic unit 20 first reads the relationship calculated in advance by the relationship calculation process shown in FIG. 12 from the memory 22, and calculates the target setting (angle of the correction ring 111) corresponding to the observation depth of the observation target surface. do. After that, the correction ring control device 14 changes the setting of the correction ring 111 to the calculated target setting according to the instruction of the arithmetic unit 20. As a result, spherical aberration is corrected.

After that, the microscope system 1 acquires image data (step S94). As a result, it is possible to obtain an image in which spherical aberration is corrected. Further, the microscope system 1 determines whether or not the image data has been acquired at all the observation depths determined in step S91 (step S95), and if not, moves the observation target surface to the next observation depth. Move (step S96). Then, when the acquisition of the image data is completed at all the observation depths (step S95YES), the microscope system 1 ends the Z-Series imaging process shown in FIG.

The same effect as that of the microscope system 1 can be obtained by the microscope system according to the present embodiment. Further, in the present embodiment, since the relationship between the observation depth and the target setting is calculated in advance, it is not necessary to acquire a plurality of image data in order to search for the target setting each time the observation depth is changed. In addition, the relationship between the depth of observation and the target setting is also calculated using interpolation and function approximation from information at a relatively small number of observation depths. Therefore, the number of preliminary shots for exploring the target setting can be significantly reduced. Therefore, damage to the sample S can be suppressed. Further, it is possible to finish the Z-Series imaging process in a relatively short time while satisfactorily correcting the spherical aberration.

[Fourth Embodiment]
In the third embodiment, an example of calculating the relationship between the target setting and the observation depth in order to obtain a high-quality image in which the spherical aberration is corrected on the observation target surface at an arbitrary depth is shown. The relationship of observation depth may be calculated for purposes other than image data acquisition. The present embodiment is different from the third embodiment in that the relationship between the target setting and the observation depth is used to display the refractive index of the sample S.

FIG. 15 is a flowchart of the refractive index display process according to the present embodiment. 16 and 17 are diagrams for explaining an example of a method of displaying the refractive index. Hereinafter, a process of displaying the refractive index using the relationship between the target setting and the observation depth will be described with reference to FIGS. 15 to 17.

When the refractive index display process shown in FIG. 15 is started according to the instruction of the user after the relationship calculation process shown in FIG. 12 is performed, the microscope system 1 is first calculated by the relationship calculation process shown in FIG. The relationship is displayed (step S101). Here, the arithmetic unit 20 causes the display device 30 to display the graph shown in FIG. 16, for example. The graph shown in FIG. 16 is an example in the case where a plurality of image data are acquired at each of the observation depths D1, D2, and D3 and the target setting at each depth is calculated in the relationship calculation process shown in FIG. Is.

After that, the microscope system 1 accepts the designation of the observation depth at which the refractive index should be displayed (step S102). Here, for example, as shown in FIG. 17, the user selects the observation depth corresponding to the portion of the sample S for which the refractive index is desired by using the cursor C or the like on the graph displayed on the display device 30. By doing so, the arithmetic unit 20 detects the specified observation depth.

When the observation depth is specified, the microscope system 1 prepares the sample based on the information displayed on the display device 30, that is, the setting of the correction ring 111 for correcting the spherical aberration specified for each observation depth. Calculate the refractive index of S at the specified observation depth. Specifically, the microscope system 1 first calculates the set change rate in the depth direction (step S103), and further calculates the refractive index of the sample S at the observation depth specified in step S102 (step S104). ).

As described in Patent Documents 2 and 3, the rate of change in the amount of spherical aberration on the surface to be observed (= the amount of change in the amount of spherical aberration per amount of change in the depth of the surface to be observed) is the same as that of the objective lens 110. It depends on the refractive index of the medium (eg, air or immersion liquid) between the samples S and the refractive index of the sample S on the surface to be observed. Further, once the objective lens is determined, the relationship between the setting of the correction ring and the amount of spherical aberration is known. In step S104, the arithmetic unit 20 calculates the rate of change in the amount of spherical aberration from the set rate of change calculated in step S103, and further, based on the rate of change in the amount of spherical aberration and the refractive index of the medium, in the observation depth. The refractive index of sample S is calculated.

When the refractive index is calculated, the microscope system 1 displays the calculated refractive index (step S105). Here, the display device 30 displays the refractive index calculated by the arithmetic unit 20, for example, as shown in FIG. Note that FIG. 17 shows how the refractive index of the sample S at the observation depth specified by the user is superimposed and displayed on the graph shown in FIG.

According to the microscope system according to the present embodiment, the user can easily know the refractive index of the sample S on any surface regardless of the structure of the sample S.

The above-described embodiment shows a specific example for facilitating the understanding of the invention, and the embodiment of the present invention is not limited thereto. The microscope system, the setting search method, and the program can be variously modified and changed without departing from the description of the claims.

For example, in the above-described embodiment, the formula (1) is used as the evaluation formula in the calculation of the contrast value, but the evaluation formula may be any one that calculates the contrast value using the difference between the pixel values between the pixels. For example, the following equation may be used.

By using the equation (2) that uses a plurality of different shift amounts (for example, n = 1, 2, 3, 5, 10, etc.) instead of the equation (1), the contrast is stabilized in a wide spatial frequency domain. Can be evaluated. Therefore, the contrast of the image can be stably evaluated regardless of the frequency component included in the image, that is, regardless of the magnification of the sample or the optical system.

Further, in the above-described embodiment, an example of excluding the pixel data at the first position from the calculation for calculating the contrast value has been shown. For example, as shown in FIG. 18, the pixel data of saturated pixels is processed. It may be used to calculate the contrast value. That is, the presence or absence of saturated pixels is determined for each line, and if saturated pixels are present, an approximate curve for the pixel value is calculated from the pixel data of the pixels around the saturated pixels in the line. Then, the pixel value of the pixel data of the saturated pixels when there is no gradation limitation due to the number of bits of the image is estimated based on the calculated approximate curve. In this way, by processing and using the pixel values included in the image data in the contrast calculation without using them as they are, it is possible to suppress a decrease in the accuracy of the contrast evaluation due to saturation.

1 Microscope system 10 Microscope control device 11 Light source control device 12 Zoom control device 13 Focus control device 14 Correction ring control device 20 Arithmetic device 21 Processor 22 Memory 26 Portable recording medium 30 Display device 40 Keyboard 50 Correction ring operation device 60 Focusing Operation device 100 Microscope 101 Laser 102 Scanning unit 107 Light detector 108 A / D converter 109 Focusing device 110 Objective lens 111 Correction ring

Claims

A microscope device having a spherical aberration correction device and acquiring a plurality of image data by acquiring image data in each of a plurality of states in which the settings of the spherical aberration correction device are different.
A computing device for specifying the setting of the spherical aberration correction device, which corrects spherical aberration based on a plurality of contrast values including each contrast value of the plurality of image data, is provided.
The arithmetic unit
The position of the pixel data in which the pixel values included in the plurality of image data are saturated is specified as the first position.
A microscope system characterized in that each of the plurality of contrast values is calculated excluding the pixel data of the first position included in each of the plurality of image data.
In the microscope system according to claim 1,
The arithmetic unit
The position of the pixel data whose pixel value included in the plurality of image data is less than the threshold value is specified as the second position.
A microscope system characterized in that each of the plurality of contrast values is calculated excluding the pixel data at the first position and the pixel data at the second position included in each of the plurality of image data.
In the microscope system according to claim 1 or 2.
The microscope device further includes a correction control device that controls the setting of the spherical aberration correction device.
The correction control device is a microscope system characterized in that the setting of the spherical aberration correction device is changed to the setting specified by the arithmetic unit.
In the microscope system according to claim 1 or 2.
The calculation device is a microscope system characterized in that the refractive index of a sample is calculated based on the setting of the spherical aberration correction device in which the spherical aberration specified for each observation depth is corrected.
In the microscope system according to claim 4, further
Including display device
The display device is a microscope system characterized by displaying the refractive index of the sample calculated by the arithmetic unit.
In the microscope system according to claim 1 or 2.
The microscope device further
It comprises at least one of a first combination of a light source and a light source control device and a second combination of a photodetector and a detection sensitivity control device.
When the first combination is provided, the light source control device controls the output of the light source according to the ratio of the pixel data in which the pixel values included in the plurality of image data are saturated.
When the second combination is provided, the detection sensitivity control device is characterized in that the sensitivity of the photodetector is controlled according to the ratio of the pixel data in which the pixel values included in the plurality of image data are saturated. Microscope system.
In the microscope system according to claim 6,
The light source control device includes the first combination, and when the ratio is equal to or greater than a threshold value, the output of the light source is reduced.
The detection sensitivity control device is a microscope system including the second combination and lowering the sensitivity of the photodetector when the ratio is equal to or higher than a threshold value.
In the microscope system according to claim 1 or 2.
The arithmetic apparatus is a microscope system characterized in that a plurality of contrast values are calculated using an evaluation formula that evaluates the contrast of an image based on the difference in pixel values between pixels.
In the microscope system according to claim 1 or 2.
The microscope device further includes an objective lens.
The spherical aberration correction device is a microscope system characterized by being a correction ring that moves a part of a lens constituting the objective lens in the optical axis direction.
It is a setting search method for searching for a setting in which spherical aberration is corrected.
A plurality of image data are acquired by acquiring image data in each of a plurality of states having different settings of the spherical aberration corrector.
The position of the pixel data in which the pixel values included in the plurality of image data are saturated is specified as the first position.
The contrast value of each of the plurality of image data is calculated by excluding the pixel data of the first position included in each of the plurality of image data.
A setting search method for specifying a setting of the spherical aberration correction device in which spherical aberration is corrected based on a plurality of contrast values including each contrast value of the plurality of image data.
On the computer
The position of the pixel data in which the pixel values included in the plurality of image data acquired with different settings of the spherical aberration correction device are saturated is specified as the first position.
The contrast value of each of the plurality of image data is calculated by excluding the pixel data of the first position included in each of the plurality of image data.
The setting of the spherical aberration correction device for which spherical aberration is corrected based on a plurality of contrast values including the contrast values of each of the plurality of image data is specified.
A program characterized by executing processing.