JP2016017880A - Specimen observation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To acquire high S/N images that allow for generating a super high resolution image.SOLUTION: A specimen observation device 1 includes: a stage 3; an objective lens 15; a scanner 7; a confocal pinhole 21; a detector 25 and image data acquisition unit 33 for detecting fluorescence passing through the confocal pinhole 21 and acquiring original image data for generating a super high resolution image; an image data integration unit 41 for generating an integrated image by integrating a plurality of original image data sets of a same area of a specimen S; a difference computation unit 37 for computing inter-image difference between the original image data and reference image data in S, Y directions; and a system control unit 31 that controls the scanner 7 such that an inter-image difference of the original image data acquired by the image data acquisition unit 33 is kept no greater than a predetermined second difference threshold value when inter-image difference that exceeds the predetermined second difference threshold value is computed by the difference computation unit 37.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a specimen observation apparatus.

  Conventionally, a specimen observation apparatus that generates a super-resolution image by enhancing a high-frequency region exceeding the diffraction limit in the spatial intensity distribution of an image acquired with a spatial resolution exceeding the diffraction limit is known (for example, Patent Documents). 1). The specimen observation apparatus described in Patent Document 1 acquires an image with a spatial resolution exceeding the diffraction limit by reducing the diameter of the confocal pinhole, and performs image processing that emphasizes a high-frequency region on the image. A super-resolution image is generated.

JP 2013-20083 A

  However, when the pinhole diameter is reduced as in the specimen observation apparatus described in Patent Document 1, the amount of light incident on the photodetector is reduced, and S sufficient in an image for generating a super-resolution image is obtained. / N cannot be obtained.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide a specimen observation apparatus that can acquire a high S / N image capable of generating a super-resolution image.

In order to achieve the above object, the present invention provides the following means.
The present invention includes a stage on which a specimen is placed, an objective lens that irradiates the specimen with excitation light emitted from a light source, and a modulation unit that modulates the spatial intensity distribution of the excitation light emitted to the specimen by the objective lens And a return limiting member capable of passing the return light returning from the focal position of the objective lens in the specimen and blocking light from other than the focal position, and detecting the return light passing through the passage limiting member. An image data acquisition unit for acquiring original image data for generating a resolution image and a plurality of the original image data obtained by irradiating the same range in the specimen with the excitation light a plurality of times to obtain an integrated image A position where an amount of positional deviation in a direction in which the spatial intensity distribution of the original image data acquired by the image data acquisition unit to be generated and the reference image data serving as a reference spreads is calculated. The positional deviation amount of the original image data acquired by the image data acquisition unit when the positional deviation amount calculation unit and the positional deviation amount exceeding a predetermined positional deviation detection threshold are calculated by the positional deviation amount calculation unit And a control unit that controls the stage, the modulation unit, and / or the image data acquisition unit so as to be equal to or less than the predetermined displacement detection threshold.

  According to the present invention, the excitation light emitted from the light source is modulated in the spatial intensity distribution by the modulation unit and irradiated to the sample by the objective lens, and only the return light that has passed through the passage restriction member is returned from the sample. It is detected by the image data acquisition unit. Thereby, the resolution by the image data acquisition unit can be improved, and the original image data can be acquired with a spatial resolution exceeding the diffraction limit. Further, by integrating the plurality of original image data in the same range in the sample by the image data integrating unit, the original image can be obtained even if the amount of return light acquired by the image data acquiring unit is reduced by the light shielding by the passage range limiting member. It is possible to generate an integrated image whose luminance is increased by adding the data.

  In this case, if the observation position is shifted during the integration of the original image data by the image data integration unit, an accurate integration result of the same range of the specimen cannot be obtained, and an integrated image with a high S / N may be generated. Can not. On the other hand, when the positional deviation amount calculation unit calculates the positional deviation amount of the original image data with respect to the reference image data, and the positional deviation amount exceeding a predetermined positional deviation detection threshold is calculated, the stage by the control unit, Under the control of the modulation unit and / or the image data acquisition unit, the image data acquisition unit acquires original image data whose positional deviation amount is equal to or less than a predetermined positional deviation detection threshold value, and the image data integration unit determines these positional deviation amounts. It is possible to integrate original image data having a predetermined positional deviation detection threshold value or less.

  Therefore, it is possible to accurately accumulate the same range in the specimen and obtain a high S / N accumulated image capable of generating a super-resolution image. As a result, an excellent super-resolution image in which the high-frequency region is visualized can be generated by performing image processing for emphasizing the high-frequency region using the integrated image as an original image.

  In the above invention, the control unit may adjust the irradiation position of the excitation light on the specimen by the control of the stage and / or the modulation unit.

  With this configuration, when a deviation occurs in the observation position in the sample, the observation position before the deviation, that is, the same position as the observation position of the reference image can be returned. Accordingly, it is possible to easily acquire a high S / N integrated image capable of generating a super-resolution image by easily matching observation positions between a plurality of original image data constituting the integrated image.

  In the above invention, the image data acquisition unit is an image sensor arranged at a position where the image of the sample is formed, and the control unit adjusts the position of the image sensor with respect to the image of the sample Also good.

  With this configuration, it is possible to prevent the observation position shift caused by the stage or the modulation unit from appearing in the original image data, as in the principle of camera shake correction. For example, in the case of a disc confocal microscope, an image is acquired by an imaging device such as a CCD arranged at the imaging position of the specimen while scanning a large number of beams on the specimen at high speed by rotating the disk. It is particularly effective in such cases.

  In the above invention, the positional deviation amount calculation unit calculates the positional deviation amount every time the original image data is acquired by the image data acquisition unit, and the image data acquisition unit includes the positional deviation amount calculation unit. The acquisition of the original image data is interrupted when a positional deviation amount exceeding the predetermined positional deviation detection threshold is calculated by, and the acquisition of the image data is performed after the positional deviation of the original image data is eliminated by the control unit. May be resumed.

  With this configuration, it is possible to minimize the number of times of unnecessary excitation light irradiation to the specimen. Therefore, after the acquisition of a series of original image data, the positional deviation is detected, and the sample is irradiated with the excitation light for the re-acquisition as in the case of acquiring all the original image data in which the positional deviation has occurred. Therefore, there is no need to worry about fluorescent discoloration or accumulation of light damage to the specimen.

  In the above invention, the image data acquisition unit acquires image data of a position shift detection position that is a position different from the observation position in the specimen, and the position shift amount calculation unit is acquired by the image acquisition unit. The positional deviation amount between the original image data and the reference image data may be calculated using image data at the positional deviation detection position.

  By configuring in this way, the original image data at the observation position is not wasted for detecting the position shift as in the case of detecting the position shift using the image data at the observation position. The number of acquisitions of the original image data can be minimized. Thereby, the fluorescence fading and the light damage to a sample can be suppressed. Even when the contrast of the original image data at the observation position is weak, the detection accuracy of the position shift can be improved by using image data with a strong contrast at the position shift detection position.

  In the above-described invention, the control unit includes a positional deviation amount of the original image data with respect to the reference image data, and the stage, the modulation unit, and the stage necessary for setting the positional deviation amount to be equal to or less than the predetermined positional deviation detection threshold. It is also possible to have a table in which control amounts of the image data acquisition unit are associated with each other.

  With this configuration, it is possible to easily grasp how much the position of the observation position in the specimen is detected as the amount of positional deviation of the original image data based on the table. Thereby, the positional deviation amount of the observation position can be accurately grasped from the positional deviation amount of the original image data reflecting the sample state and the observation condition.

  In the above-mentioned invention, it is good also as providing a notifying part which notifies that it is in an abnormal state, when the amount of position gap computed by the amount of position gap calculation part exceeds a predetermined abnormality detection threshold.

  By configuring in this way, when the amount of positional deviation between the original image data and the reference image data becomes a size that cannot occur due to the deviation of the observation position or the like, an abnormality on the observation side or a positional deviation amount calculation unit If there is some abnormality such as an abnormality on the side, it can be dealt with so that excessive control by the control unit is not performed.

In the above invention, the control unit may correct the displacement of the observation position by the control of the modulation unit.
With this configuration, it is possible to eliminate the positional deviation without using an electric stage as the stage.

  In the above invention, a second positional deviation amount calculation unit that calculates a second positional deviation amount in a direction intersecting a direction in which the spatial intensity distribution of the original image data and the reference image data spreads, and the second positional deviation When the second positional deviation amount exceeding the predetermined second threshold is calculated by the quantity calculating unit, the positional deviation amount of the original image data acquired by the image data acquiring unit is equal to or less than the predetermined positional deviation detection threshold. It is good also as providing the 2nd control part which controls the said stage and / or the said objective lens so that it may become.

  With this configuration, the second positional deviation amount calculation unit calculates the second positional deviation amount of the original image data with respect to the reference image data, and calculates the second positional deviation amount exceeding the predetermined second threshold. In this case, the image data acquisition unit acquires original image data having a second positional deviation amount equal to or less than a predetermined second threshold value by controlling the stage and / or the objective lens by the second control unit. Therefore, the image data integration unit can integrate the original image data in which the second positional deviation amount is equal to or less than the predetermined second threshold value. Thereby, even when the observation position is shifted in the optical axis direction of the objective lens, it is possible to eliminate the positional shift in the direction intersecting the direction in which the spatial intensity distribution of the excitation light in the original image data spreads.

In the above invention, an active autofocus unit that adjusts a focusing state of the objective lens in the optical axis direction with respect to the specimen may be provided.
With this configuration, even when a position shift occurs in the optical axis direction of the objective lens, the active autofocus unit adjusts the focusing state with respect to the sample, and the shift of the observation position in the optical axis direction of the objective lens. Is resolved. As a result, all three-dimensional deviations in the observation position can be eliminated.

  The present invention provides an objective lens that irradiates the specimen with excitation light emitted from a light source, a modulator that modulates a spatial intensity distribution of the excitation light emitted to the specimen by the objective lens, and the objective lens in the specimen A return limiting member that allows the return light returning from the focal position to pass and blocks light from other than the focal position, and the return light that has passed through the passage limiting member to generate a super-resolution image. An image data acquisition unit that acquires original image data; an image data integration unit that generates an integrated image by integrating a plurality of the image data obtained by irradiating the same range in the specimen with excitation light multiple times; A positional deviation amount calculating unit for calculating a positional deviation amount in a direction in which the spatial intensity distribution of the original image data acquired by the image data acquisition unit and the reference image data serving as a reference spreads; When the calculation unit calculates the amount of misregistration exceeding a predetermined misregistration detection threshold, the misregistration amount of the original image data integrated by the image data integration unit with respect to the reference image data is the predetermined misregistration detection. Provided is a sample observation device including a misregistration correction unit that corrects the misregistration by shifting pixels of the original image data so as to be equal to or less than a threshold value.

  According to the present invention, even when an amount of misregistration that exceeds a predetermined misregistration detection threshold occurs in the original image data, the pixel is shifted by the misregistration correction unit, and the misregistration amount is detected as a predetermined misregistration. An integrated image can be generated by the image data integration unit using the original image data corrected to a threshold value or less.

  Therefore, it is possible to accurately accumulate the same range in the specimen and obtain a high S / N accumulated image capable of generating a super-resolution image. As a result, an excellent super-resolution image in which the high-frequency region is visualized can be generated by performing image processing for emphasizing the high-frequency region using the integrated image as an original image.

In the above invention, the image data acquisition unit may acquire the original image data having a wider field of view than the observation target region.
With this configuration, when correcting the displacement by shifting the pixels of the original image data, the observation target region is included in the original image data even if the surrounding pixels of the original image data are discarded. be able to.

  In the above invention, the displacement amount calculation unit may also calculate a displacement in a direction intersecting a direction in which the spatial intensity distribution of the original image data and the reference image data spreads.

  With this configuration, if the positional deviation amount calculation unit calculates original image data in which the positional deviation amount in the direction intersecting the direction in which the spatial intensity distribution of the excitation light with respect to the reference image data spreads exceeds a predetermined threshold value, Based on other reference image data whose position in the direction intersecting the direction in which the spatial intensity distribution of the excitation light spreads is different, the original image data is corrected so that the positional deviation amount is equal to or less than a predetermined threshold value. The observation positions can be matched.

  According to the present invention, there is an effect that a high S / N image capable of generating a super-resolution image can be acquired.

It is a schematic block diagram which shows the sample observation apparatus which concerns on 1st Embodiment of this invention. (A) is a figure which shows the reference image data of an observation position, (b) is a figure which shows the original image data of an observation position, (c) is a figure which shows the difference image data of reference | standard image data and original image data (D) shows the luminance difference of each pixel in (c). It is a figure which shows an example of the reference | standard image data and each original image data which are stored in the image data storage part, and each difference image data. It is a figure which shows an example which linked | related the difference between images, and the direction and quantity of offset. It is a figure which shows an example which matched the positional offset amount of + X, -X, + Y, and -Y, and the difference between images. It is a figure which shows an example of the standard image data of an observation position. It is a figure which shows an example of the original image data of an observation position. It is a flowchart explaining the process of producing | generating a super-resolution image. It is the figure which arranged reference image data, each original image data, and an example of each of these difference image data in time series. It is a figure which shows a mode that the original image data is integrated and the integrated image is filtered. It is a flowchart explaining the process of producing | generating a super-resolution image by the sample observation apparatus which concerns on the 1st modification of 1st Embodiment of this invention. It is a figure which shows an example of an observation position and a marker position. FIG. 6 is a diagram in which an example of image data for detecting misalignment and an example of observation image data are arranged in time series. It is a figure which shows an example of the original image data of a reference position, the original image data of the position which shifted | deviated +/- 1um in the Z direction with respect to a reference position, and difference image data with respect to each reference image data. It is the figure which arranged the example of the original image data in Z = 1 in a sample, and its difference image data in time series. It is the figure which arranged the example of the original image data in Z = 2 in a sample, and its difference image data in time series. It is the figure which arranged the example of the original image data in Z = 3 in a sample, and its difference image data in time series. It is a schematic block diagram which shows the sample observation apparatus which concerns on 3rd Embodiment of this invention. It is a figure which shows the positional relationship of the observation position of FIG. 18, an objective lens, a confocal disc, and an image pick-up element. It is a figure which shows a mode that the position of the image pick-up element was minutely adjusted with respect to image formation. It is a schematic block diagram which shows the sample observation apparatus which concerns on 4th Embodiment of this invention. It is a figure which shows a mode that the pixel of original image data is shifted and position shift is correct | amended. It is a figure which shows an example of each reference | standard image data and each original image data of a different position in a Z direction.

[First Embodiment]
A sample observation apparatus according to a first embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the sample observation device 1 according to the present embodiment two-dimensionally transmits a stage 3 on which a sample S is placed, a light source 5 that generates excitation light, and excitation light emitted from the light source 5. A scanner (modulation unit) 7 for scanning, a pupil projection lens 9 for condensing excitation light scanned by the scanner 7, and an imaging lens 13 for converting the excitation light collected by the pupil projection lens 9 into parallel light. And an objective lens 15 that irradiates the sample S with excitation light transmitted through the imaging lens 13 and condenses fluorescence (return light) generated in the sample S. Reference numeral 11 denotes a reflection mirror that reflects the excitation light from the imaging lens 13 toward the pupil projection lens 9.

The stage 3 can move in the optical axis direction (Z direction) of the objective lens 15 and the direction (X direction, Y direction) orthogonal to the optical axis direction.
The scanner 7 is, for example, a proximity galvanometer mirror, and includes two galvanometer mirrors (not shown) that can be oscillated around the oscillating axes orthogonal to each other. The scanner 7 controls the swing angle of each of the two galvanometer mirrors around each peristaltic axis so that the excitation light can be scanned in the X and Y directions to modulate the spatial intensity distribution. It has become.

  In addition, the specimen observation apparatus 1 includes a dichroic mirror 17 that branches the fluorescence collected by the objective lens 15 and returning the optical path of the excitation light from the optical path, and a confocal lens 19 that collects the fluorescence branched by the dichroic mirror 17. A confocal pinhole (passage limiting member) 21 that restricts the passage of fluorescence that is collected by the confocal lens 19, a collimator lens 23 that converts the fluorescence that has passed through the confocal pinhole 21 into parallel light, and a collimator lens 23 And a detector (image data acquisition unit) 25 such as a photomultiplier tube (PMT) that detects fluorescence transmitted through the light source.

The confocal pinhole 21 is disposed at a position optically conjugate with the focal position of the objective lens 15, and allows only fluorescence generated at the focal position of the objective lens 15 in the sample S to pass therethrough. Further, the confocal pinhole 21 can change the size of the opening through which the fluorescence passes.
The detector 25 outputs a light intensity signal corresponding to the detected fluorescence intensity.

  In addition, the specimen observation apparatus 1 includes a PC (Personal Computer) 27 that performs image data acquisition and image data processing, a monitor 29 that displays image data acquired by the PC 27, a stage 3, a light source 5, A scanner 7, a confocal pinhole 21, a detector 25, and a system control unit (control unit) 31 for controlling the PC 27 are provided.

  The PC 27 includes an image data acquisition unit 33 that acquires image data, an image data storage unit 35 that stores image data acquired by the image data acquisition unit 33, and image data stored in the image data storage unit 35. A difference calculation unit (position shift amount calculation unit) 37 that calculates a position difference, a position shift determination unit 39 that determines a position shift of image data based on the position difference calculated by the difference calculation unit 37, and a sample A data integration unit (image data integration unit) 41 that generates an integrated image by integrating a plurality of image data in the same range in S, and an image processing unit 43 that performs image processing on the integrated image are provided.

  The image data acquisition unit 33 converts the light intensity signal sent from the detector 25 into luminance information for each pixel corresponding to the scanning position of the excitation light by the scanner 7 so as to acquire the image data of the sample S. It has become. Hereinafter, among the image data at the same observation position acquired by the image data acquisition unit 33, the reference image data is referred to as “reference image data”, and the image data for integration for generating the super-resolution image is referred to as “reference image data”. This is called “original image data”.

  The image data storage unit 35 stores the reference image data at address 0, and sequentially stores the original image data in the same range acquired sequentially in order from address 1.

  The difference calculation unit 37 is configured to calculate the difference between the positions of the original image data and the reference image data in the direction in which the spatial intensity distribution of the excitation light spreads, that is, the X direction and the Y direction. For example, the difference calculation unit 37 calculates the absolute value of the luminance difference between the pixels of the original image data and the reference image data, and performs this over all the pixels. Hereinafter, an absolute value is used for the luminance difference.

  For example, image data (hereinafter referred to as difference image data) obtained by calculating a luminance difference in each pixel between the reference image data shown in FIG. 2A and the original image data shown in FIG. 2B is shown in FIG. ). In the difference image data, as shown in FIG. 2D, when there is a change in a strong contrast portion on the original image data, the luminance difference from the reference image data is large in the pixel of that portion. . It is ideal that the difference in luminance from the reference image data is zero in the pixels where no change has occurred, but in reality, a difference in luminance having random fluctuations due to noise from the detector 25, an amplifier circuit (not shown), or the like. Will be obtained. In FIG. 2D, the numerical value of each pixel shows an example of the difference in luminance.

  Therefore, the difference calculation unit 37 obtains a pixel having a luminance difference exceeding a predetermined first difference threshold value (positional deviation detection threshold value) for distinguishing between a noise level and a difference that clearly exceeds the noise level (hereinafter referred to as a position). It is designed to count only shifted pixels). By determining that there is a significant change only in the misregistration pixels, the influence of noise can be suppressed. By counting the number of misaligned pixels on the difference image data, there is a correlation in which the number of counts increases as the number of parts that change in the original image data increases. FIG. 3 shows an example of the reference image data and the original image data stored in the image data storage unit 35 and the difference image data thereof.

  In addition, every time one piece of original image data is acquired by the image data acquisition unit 33, the difference calculation unit 37 determines the difference in position between the original image data and the reference image data, that is, the count number of these misaligned pixels. (Hereinafter, the count number of misaligned pixels is referred to as “difference between images”).

  The positional deviation determination unit 39 generates a positional deviation at the observation position when the amount of positional deviation of the original image data, that is, the inter-image difference calculated by the difference calculation unit 37 exceeds a predetermined second difference threshold. It comes to judge that there is. In this way, it is possible to determine whether or not the observation position is misaligned without malfunction due to the influence of noise or the like.

  The second difference threshold value varies depending on the shape of the sample S and the contrast state. Further, the relationship between the direction and amount of displacement of the observation position and the difference between images depends on the state of the sample S. Therefore, as shown in FIG. 4, the system control unit 31 acquires reference image data and image data at an observation position offset in the X and Y directions with respect to the observation position of the reference image data. It has a table in which the inter-image difference is associated with the offset direction and amount.

  For example, the position difference determination unit 39 sets a second difference threshold value that determines that the observation position is shifted when the original image data is shifted by 1 μm or more in dimension on the sample S. In this case, image data offset by 1 μm at + X, −X, + Y, and −Y is acquired in advance, and the difference image data between each image data and the reference image data, as shown in FIG. Measure the difference between the two.

  In this method, the direction of the observation position shift cannot be determined from the inter-image difference. For example, in the example shown in FIGS. 4 and 5, the inter-image difference is the smallest, and the image is shifted by 1 μm in the X direction. When the second difference threshold is set to the difference between the intervals = 1588, it is determined that the displacement has occurred when the difference between the images is exceeded, so that the displacement is detected at least 1 μm or more in any direction. Can do. Note that this is not the case when the original image data is shifted in an oblique direction, but there are many cases where the actual position of the microscope or stage is shifted in one direction of + X, −X, + Y, and −Y. Therefore, the above method is sufficiently effective for detecting the deviation of the observation position.

  The image processing unit 43 uses the integrated image generated by the data integrating unit 41 as an original image to emphasize the high frequency region, that is, the high frequency region is visualized by applying a high frequency enhancement filter. A super-resolution image is generated.

  When the positional deviation determination unit 39 determines that a positional deviation exceeding a predetermined second difference threshold has occurred, the system control unit 31 has a positional deviation amount of the acquired original image data that is equal to or smaller than the predetermined second differential threshold. Thus, the scanner 7 is controlled. For example, the scanner 7 can eliminate the positional deviation of the original image data by finely adjusting the scanning range according to the deviation of the observation position.

  The positional deviation of the original image data may occur due to the drift of the position of the stage 3, or may occur due to the drift of the irradiation position of the excitation light even if the position of the stage 3 does not move. is there. However, since the position on the original image data is determined only by the relative positional relationship between the sample S and the irradiation range of the excitation light that irradiates the sample S, the control of the scanner 7 regardless of the portion that causes the positional deviation, The purpose of eliminating the displacement can be achieved.

  For example, with respect to the standard image data at the observation position shown in FIG. 6, as shown in FIG. 7, the observation position drift in the specimen S generated by shifting the excitation light irradiation range to the right by the offset of the scanner 7. Can be adjusted to restore the original image data.

The operation of the specimen observation apparatus 1 according to the present embodiment configured as described above will be described below.
In the case where a super-resolution image of the sample S is generated by the sample observation device 1 according to the present embodiment, first, a process of acquiring the original image data of the sample S will be described.
The excitation light generated from the light source 5 is reflected by the dichroic mirror 17, scanned by the scanner 7, and irradiated on the sample S by the objective lens 15 through the pupil projection lens 9, the reflection mirror 11, and the imaging lens 13. Thereby, the excitation light is scanned two-dimensionally on the specimen S in accordance with the peristaltic operation of each galvanometer mirror of the scanner 7.

  The fluorescence generated in the specimen S by being irradiated with the laser light is collected by the objective lens 15 and then the optical path of the excitation light through the imaging lens 13, the reflection mirror 11, the pupil projection lens 9 and the scanner 7. Then, the light passes through the dichroic mirror 17, is branched from the optical path of the excitation light, and is collected by the confocal lens 19.

  Of the fluorescence collected by the confocal lens 19, only the light generated at the focal position of the objective lens 15 in the specimen S passes through the confocal pinhole 21 and is detected by the detector 25 via the collimator lens 23. . In the detector 25, a light intensity signal corresponding to the detected fluorescence intensity is sent to the PC 27 via the system control unit 31.

  In the PC 27, the image data acquisition unit 33 converts the light intensity signal transmitted from the detector 25 into luminance information for each pixel corresponding to the scanning position of the excitation light by the scanner 7, and the image data of the sample S is converted into the image data. To be acquired. The image data acquired by the image data acquisition unit 33 is stored in the image data storage unit 35.

Next, the process of generating a super-resolution image will be described with reference to the flowchart of FIG.
In the present embodiment, for example, 64 pieces of original image data are acquired.
First, the reference image data of the observation position in the specimen S is acquired by the procedure described above (step SA1) and stored at address 0 of the image data storage unit 35 (step SA2).

  Next, the address of the image data storage unit 35 advances by 1 (step SA3), the original image data at the observation position is acquired (step SA4), and is stored at address 1 (current address) of the image data storage unit 35 (step SA4). SA5). Then, the difference calculation unit 37 calculates the difference between the reference image data at the address 0 and the originality image data at the address 1 (step SA6), and the pixel of the misregistration pixel from which the luminance difference exceeding the first difference threshold is obtained. The number is counted.

  Next, based on the count number calculated by the difference calculation unit 37, that is, the inter-image difference, the positional deviation determination unit 39 determines the presence or absence of the positional deviation of the observation position (step SA7). When it is determined that the inter-image difference is equal to or smaller than the second difference threshold value (step SA7 “Yes”), it is confirmed whether or not the final address, that is, the desired number of shots (in this embodiment, 64) has been reached. (Step SA8). Until the final address is reached (step SA8 “No”), the address is advanced by 1 (step SA3), and steps SA3 to SA8 are repeated.

  On the other hand, when it is determined that the inter-image difference exceeds the second difference threshold (step SA7 “No”), the scanner 7 is controlled by the system control unit 31 and the irradiation position of the excitation light is adjusted. As a result, the position of the original image data is offset adjusted in the + X, −X, + Y, and −Y directions, and the positional deviation is corrected so that the inter-image difference is equal to or smaller than the second difference threshold (step SA9).

  When the misregistration correction is completed, the original image data is reacquired without being advanced in the address of the image data storage unit 35 and stored in the image data storage unit 35. The difference calculation unit 37 calculates the inter-image difference from the reference image data. Calculated (step SA10). If the inter-image difference in the re-acquired original image data has decreased (step SA11 “Yes”), the positional deviation determination unit 39 determines the positional deviation again (step SA12).

  If it is determined that the inter-image difference is equal to or less than the second difference threshold value (step SA12 “Yes”), it is confirmed whether it is the final address (step SA8). On the other hand, when it is determined that the inter-image difference exceeds the second difference threshold value (step SA12 “No”), the process returns to step SA9, and steps SA9 to SA12 are performed without advancing the address of the image data storage unit 35. Repeated.

  In this way, when a positional deviation occurs while sequentially acquiring the original image data, the acquisition of the original image data is temporarily interrupted to correct the positional deviation of the observation position, and the acquisition of the original image data is resumed after the correction. . For example, as shown in FIG. 9, when the difference between images exceeds the second difference threshold at the time when the fourth original image data (T = 4 in FIG. 9) is acquired, the system control unit 31 The irradiation position of the excitation light is adjusted, the positional deviation of the original image data is corrected, and the fourth original image data is acquired again. Then, when the inter-image difference between the re-acquired fourth original image data and the reference image data is equal to or smaller than the second difference threshold value, the original image data after the fifth image (T = 5 in FIG. 9). Acquisition continues. In FIG. 9, T = 1, T = 2, T = 3... Indicate the number of original image data.

  Then, in step SA8 in the flowchart of FIG. 8, when the final address is reached (step SA8 “Yes”), acquisition of the original image data ends. Thereby, in the image data storage part 35, the difference between images is below a 2nd difference threshold value, ie, the 64 original image data with almost no position shift can be arrange | equalized.

  Subsequently, the data integration unit 41 integrates the 64 original image data stored in the image data storage unit 35 to generate an integrated image. Then, the image processing unit 43 creates a super-resolution image by applying a high frequency enhancement filter using the integrated image as an original image.

  In addition, in step SA11 of the flowchart of FIG. 8, when the difference between the images of the original image data that has been re-acquired has increased, the observation position greatly deviates far beyond the degree of positional deviation of the original image data. It is suspected that an abnormal state such as a change in observation conditions has occurred. In such a case, no matter how much misalignment is made by adjusting the offset of the observation position, the original image data can no longer be returned to the observation position and the control does not converge. Therefore, the position deviation determination unit 39 determines that an abnormality has occurred and ends abnormally.

  Here, when a deviation occurs in the observation position during acquisition of the original image data by the image data acquisition unit 33, if the acquired plurality of original image data are integrated as they are, an accurate integration result of the same range of the sample S is obtained. It cannot be obtained, and a high S / N integrated image cannot be generated. On the other hand, according to the specimen observation device 1 according to the present embodiment, when the positional deviation amount of the original image data with respect to the reference image data exceeds the predetermined second difference threshold, the observation position deviation is controlled by the control of the system control unit 31. Is corrected, and the original image data is reacquired. Then, the original image data having almost no displacement is integrated.

  Therefore, as shown in FIG. 10, it is possible to accurately accumulate the same range in the sample S and obtain a high S / N accumulated image capable of generating a super-resolution image. A good super-resolution image in which the high frequency region is visualized can be generated by performing image processing that emphasizes the high frequency region on the integrated image as an original image.

  In addition, when a misalignment occurs, the acquisition of the original image data is temporarily interrupted, and after acquiring a series of original image data by restarting the acquisition of the original image data after correcting the misalignment appropriately. Compared to a case where all the original image data in which the positional deviation has occurred is reacquired, the number of times of excessive excitation light irradiation to the specimen S can be minimized. Therefore, it is not necessary to continuously irradiate the specimen S with excitation light, and there is no need to worry about fluorescent discoloration or accumulation of light damage to the specimen.

  In this embodiment, the position of the excitation light is adjusted by controlling the scanner 7 to eliminate the positional deviation of the original image data. Instead, the system controller 31 moves the stage 3 to X, Y. By moving in the direction, the irradiation position of the excitation light may be adjusted to eliminate the positional deviation of the original image data. Further, the positional deviation may be eliminated by both the scanner 7 and the stage 3.

  In the present embodiment, the specimen observation device 1 may include a known microscope active focus drift correction device (active autofocus unit) that adjusts the focusing state of the objective lens in the optical axis direction. . Since the time required for acquiring the original image data is within a few minutes and is relatively short, the influence of the positional deviation in the Z direction is limited and rarely causes a problem. However, in the case where the positional deviation in the Z direction also affects the image accuracy, the positional deviation in the Z direction can be performed simultaneously with the correction of the positional deviation in the X and Y directions by using the active focus drift correction device for microscope. Can always be corrected.

Further, in the present embodiment, when the difference between images calculated by the difference calculation unit 37 exceeds a predetermined abnormality detection threshold, a notification unit (not shown) that notifies the user of an abnormal state by voice or the like. It is good also as providing.
By doing in this way, when the difference between the images becomes a size that cannot occur due to the deviation of the observation position or the like, there is some abnormality such as an abnormality on the observation side or an abnormality on the difference calculation unit 37 side, It is possible not to perform excessive control by the system control unit 31.

This embodiment can be modified as follows.
In the present embodiment, as the difference between the positions of the reference image data and the original image data, the number of misaligned pixels whose luminance difference exceeds the first difference threshold is counted. On the other hand, as a first modified example, the difference calculating unit 37 calculates an integral value of the luminance difference over a certain area of the original image data, and the positional deviation determining unit 39 has a predetermined second value of the integrated value of the luminance difference. It may be determined that a positional deviation has occurred when the difference threshold is exceeded. In this case as well, a threshold value is set so that it is detected when the integrated value of the noise is clearly exceeded so that it is not erroneously detected by the integrated value of the detected noise when there is no positional deviation of the original image data. And it is sufficient.

  In the present embodiment, the difference between the luminance data of each pixel is calculated, but as a second modification, the difference calculation unit 37 performs an image processing algorithm between the reference image data and the original image data. The direction and amount of the positional deviation of the observation position may be calculated by pattern matching.

  By doing in this way, the direction and amount of positional deviation can be calculated more accurately than the method based on the difference in luminance data of each pixel described above. Therefore, in the control for offsetting the positional deviation of the original image data, it is possible to match the original image data with the observation position by applying the direction and amount of the positional deviation of the observation position calculated by the image processing algorithm. .

  In this modification, as shown in the flowchart of FIG. 11, pattern matching may be performed as routine B instead of the difference calculation in step SA6 in routine A of FIG. 8 (step SB6). When the pattern matching of the second routine B (step SB10) is completed, it is sufficient to determine the positional deviation by the positional deviation determination unit 39 without determining whether or not the difference is decreasing (step SB10). SA11).

  In the present embodiment, the displacement of the observation position is determined using the original image data itself of the observation position in the sample S. As a third modification, the image data acquisition unit 33 uses the sample S. It is also possible to acquire image data at a position detection position that is different from the observation position in FIG.

  Further, the difference calculation unit 37 may calculate an inter-image difference between the image data acquired at the same position detection position by the image acquisition unit 33 and the reference image data. Note that, as shown in FIG. 12, a marker position in which a marker is labeled at a position different from the actual observation position is set as a position shift detection position, and the movement between the marker position and the actual observation position is reproducible. The accuracy is guaranteed.

  In this case, after the reference image data is first acquired at the marker position, as shown in FIG. 13, when the original image data at the observation position is sequentially acquired, the image data is temporarily moved to the marker position for each acquisition. And the inter-image difference between the original image data and the reference image data may be calculated using this image data.

  When it is determined by the position shift determination unit 39 that a position shift has occurred from the difference between the reference image data at the marker position and the image data, the system control unit 31 determines the shift of the observation position. After correction, the original image data for integration may be acquired by moving to the actual observation position.

  By doing so, the original image data at the observation position is not wasted for detecting the position deviation as in the case of detecting the position deviation using the image data at the observation position, and the actual observation is performed. The number of times of image acquisition at the position, that is, the number of times of irradiation of the excitation light to the observation position can be minimized. Thereby, the light damage to the sample S and the influence of fluorescent fading can be minimized. Even if the contrast of the original image data at the observation position is weak, if the original image data having a strong contrast at the position detection position is used, the detection accuracy of the position shift can be improved.

[Second Embodiment]
Next, a specimen observation apparatus according to the second embodiment of the present invention will be described.
In the sample observation device 1 according to the present embodiment, the difference calculation unit (second positional deviation amount calculation unit) 37 calculates the positional deviation amount (second positional deviation amount) in the Z direction between the original image data and the reference image data. When the system control unit (second control unit) 31 calculates a positional deviation amount exceeding a predetermined second difference threshold (second threshold) by the difference calculation unit 37, the system control unit (second control unit) 31 acquires the image data acquisition unit 33. This is different from the first embodiment in that the stage 3 and / or the objective lens 15 are controlled so that the amount of misalignment of the original image data is equal to or less than a predetermined second difference threshold.
In the following, portions having the same configuration as those of the sample observation apparatus 1 according to the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  When the shape of the cross section of the specimen S in the Z direction changes, the Z position shifts between the reference image data and the original image data (hereinafter simply referred to as “between image data”), as shown in FIG. In addition, the luminance difference between the image data naturally increases. For example, if the inter-image difference does not fall below the second difference threshold no matter how much the X and Y direction misalignment correction is performed, it is suspected that a misalignment in the Z direction has occurred.

  The system control unit 31 adjusts the focal point according to the position of the objective lens 15 and / or the height of the stage 3. If the inter-image difference between the reference image data and the original image data is equal to or smaller than the second difference threshold value, the Z position deviation is appropriately corrected. Alternatively, before performing the positional deviation correction in the X and Y directions, first, the inter-image difference may be minimized by focus adjustment in the Z direction, and then the positional deviation in the X and Y directions may be corrected.

The operation of the specimen observation apparatus 1 configured as described above will be described.
In the present embodiment, an example in which three images (three slices) having different cross-sectional positions in the Z direction are generated as an integrated image for generating a super-resolution image will be described.
In this case, first, as shown in FIGS. 15 to 17, reference image data is respectively acquired at three reference positions (Z = 1, Z = 2, Z = 3) having different positions in the Z direction.

  Next, the original image data of Z = 1 is acquired by the image data acquisition unit 33 as shown in FIG. Then, every time the original image data is acquired by the difference calculation unit 37, an inter-image difference is calculated with respect to the reference image data with Z = 1, and if the second difference threshold is exceeded, the system control unit 31 , Correction of displacement of the observation position in the Y direction is performed.

If the inter-image difference still exceeds the second difference threshold value, the system controller 31 corrects the misalignment in the Z direction. If the inter-image difference is less than or equal to the second difference threshold value, the next original image data is acquired. Transition. In this way, in the slice with Z = 1, a series of original image data that matches the reference image data in the X, Y, and Z directions is acquired.
For the slices after Z = 2, the original image data corrected in the X, Y, and Z directions is acquired when the observation position is shifted by the same procedure as in the case of Z = 1.

  According to the sample observation device 1 according to the present embodiment, the difference calculation unit 37 calculates the inter-image difference of the original image data with respect to the reference image data, and calculates the inter-image difference exceeding the predetermined second difference threshold. In this case, the original image data in which the inter-image difference is equal to or less than a predetermined second difference threshold is acquired by controlling the stage 3 and / or the objective lens 15 by the system control unit 31.

  Accordingly, the image data integration unit 41 can integrate the original image data having these positional deviation amounts equal to or less than the predetermined second difference threshold. Thereby, even when the observation position is shifted in the optical axis direction of the objective lens 15, the positional shift in the direction intersecting the direction in which the spatial intensity distribution of the excitation light in the original image data spreads can be eliminated.

  In the present embodiment, if the drift in the Z direction occurs very slowly compared to the drift in the X and Y directions, the position in the Z direction is offset-adjusted with the first original image data of the slice at each Z position. Then, when the acquisition of the reference image data at each Z position is started, the inter-image difference between the original image data and the reference image data at the Z position may be minimized.

  By this operation, the state corrected to the same focal position as the observation position (Z = 1) of the reference image data acquired first can be reproduced. When acquiring the original image data for integration, it is possible to correct the deviation of the observation position only in the X and Y directions from the inter-image difference. Even in this case, since the time is short, the observation position in the Z direction is hardly displaced during that time. The same applies to slices after Z = 2.

  In the present embodiment, the difference calculation unit 37 is employed as the second positional deviation amount calculation unit, and the system control unit 31 is employed as the second control unit. However, the sample observation apparatus 1 includes the difference calculation unit 37 and In addition to the system control unit 31, a second misregistration amount calculation unit and a second control unit may be provided.

  In the present embodiment, by using a known microscope active focus drift correction device (active autofocus unit), the positional deviation in the Z direction is corrected simultaneously with the correction of the positional deviation in the X and Y directions. It is good as well.

[Third Embodiment]
Next, a specimen observation apparatus according to the third embodiment of the present invention will be described.
As shown in FIG. 18, the sample observation apparatus 101 according to the present embodiment includes a condensing lens array 151 and a confocal disk (modulation unit) 153 instead of the scanner 7 and the confocal pinhole 21, and Instead of the collimating lens 23, the detector 25, and the image data acquisition unit 33, it differs from the first embodiment in that an image sensor (image data acquisition unit) 157 such as a relay lens 155 and a CCD is provided.
In the following, portions having the same configuration as those of the sample observation apparatus 1 according to the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  The condenser lens array 151 condenses the excitation light from the light source 5 and makes it incident on the confocal disc 153. By passing through the condensing lens array 151, it is possible to improve the incident efficiency of the excitation light emitted from the light source 5 onto the confocal disk 153.

  The confocal disc 153 has a large number of minute apertures, and the condensing lens array 151 can be rotated in synchronism around a predetermined rotation axis by a rotation motor 159. The confocal disc 153 divides the incident excitation light into a plurality of parts and collects the plurality of excitation lights on the sample S.

Further, the confocal disc 153 can be rotated around a predetermined rotation axis so that a plurality of excitation lights can be scanned two-dimensionally on the specimen S. Thereby, a plurality of excitation lights can be scanned evenly over the observation target region on the specimen S.
The relay lens 155 relays fluorescence returning through the confocal disk 153 to form an image on the light receiving surface of the image sensor 157.

  The image sensor 157 is disposed at a position optically conjugate with the focal position of the objective lens 15. The image sensor 157 is mounted on an attachment (not shown) such as a piezo element that can be finely adjusted, and the system control unit 31 finely shifts the position of the image formed by the relay lens 155. Can be done.

The operation of the specimen observation apparatus 101 configured as described above will be described.
When image data is acquired by the sample observation apparatus 101 according to this embodiment, the condensing lens array 151 and the confocal disk 153 are rotated around a predetermined rotation axis by the rotation motor 159 to generate excitation light from the light source 5. .

  The excitation light emitted from the light source 5 is condensed by the condenser lens array 151, passes through the dichroic mirror 17, passes through a plurality of openings of the confocal disk 153, and is divided into a plurality of parts. The divided fluorescence is imaged by the imaging lens 13 and irradiated on the sample S by the objective lens 15. Thus, a plurality of excitation lights are scanned two-dimensionally on the specimen S according to the rotation of the confocal disk 153.

  The fluorescence generated in the specimen S by being irradiated with the excitation light returns through the optical path of the excitation light via the objective lens 15 and the imaging lens 13 and passes again through each opening of the confocal disc 153. The fluorescence is reflected by the dichroic mirror 17 and then received by the image sensor 157 via the relay lens 155. Thereby, image data is acquired in the image sensor 157.

In this way, while rotating the confocal disk 153, image data is continuously acquired by the image sensor 157, or original image data is acquired by using the time during which a plurality of excitation light scans the observation area uniformly as an exposure time. You can do it.
Calculation of the inter-image difference between the original image data and the reference image data by the difference calculation unit 37, determination of the displacement of the observation position by the position shift determination unit 39, and acquisition of the original image data when a shift occurs in the observation position The same as in the first embodiment.

  In this case, as shown in FIG. 20, the system control unit 31 slightly shifts and adjusts the position of the image sensor 157 arranged at the imaging position of the image of the sample S as shown in FIG. The observation object in the acquired original image data can be finely shifted to correct the positional deviation.

  Therefore, according to the sample observation apparatus 101 according to the present embodiment, the same range in the sample S can be accurately integrated, and a high S / N integrated image capable of generating a super-resolution image can be acquired. As a result, an excellent super-resolution image in which the high-frequency region is visualized can be generated by performing image processing for emphasizing the high-frequency region using the integrated image as an original image.

[Fourth Embodiment]
Next, a specimen observation apparatus according to the fourth embodiment of the present invention will be described.
As shown in FIG. 21, the sample observation apparatus 201 according to the present embodiment is first in that it includes an image shift correction unit (position shift correction unit) 161 that corrects original image data in which position shift has occurred by post-processing. Different from the embodiment.
In the following, portions having the same configuration as those of the sample observation apparatus 1 according to the first embodiment are denoted by the same reference numerals and description thereof is omitted.

  The difference calculation unit 37 calculates the direction and amount of positional deviation with respect to the reference image data by pattern matching in a series of original image data stored in the image data storage unit 35.

  The image shift correction unit 161 is configured so that a difference between images of the original image data determined by the position shift determination unit 39 to have a shift in the observation position is less than or equal to a predetermined second difference threshold. The positional deviation is corrected by shifting the pixels of the original image data. In addition, the image shift correction unit 161 is configured to correct the positional deviation of the original image data in which the positional deviation has occurred after all the original image data for integration is acquired by the image acquisition unit 33.

  The image data acquisition unit 33 acquires original image data having a field of view wider than the observation target region by the amount of observation position deviation that is predicted to some extent. When the positional deviation of the original image data is adjusted by pixel shift, the image area that can be used for the integration process is only the overlapping area at the center of the image, and the data at the periphery of the image is discarded. Therefore, by acquiring image data of a region that is as wide as the amount of observation position deviation that is predicted to some extent with respect to the observation region to be acquired, when performing alignment of each original image data by pixel shift thereafter, It is possible to realize a state in which a desired observation region overlaps in all original image data.

  According to the sample observation device 201 configured as described above, the image data acquisition unit 33 sequentially acquires the reference image data of the observation position and all the original image data, and stores them in the image data storage unit 35. Then, the difference calculation unit 37 calculates an inter-image difference from the reference image data by pattern matching for all the original image data, and the position shift determination unit 39 determines the position shift of the observation position.

  Next, as illustrated in FIG. 22, when any one of the original image data is displaced by an amount exceeding a predetermined second difference threshold, the image shift correction unit 161 shifts the pixel of the original image data. Thus, the difference between the images is corrected so as to be equal to or less than a predetermined second difference threshold value. Thereby, in the image data storage unit 35, the difference between images is equal to or less than the second difference threshold value, that is, the original image data having almost no positional deviation can be aligned, and the image data integrating unit 41 uses these original image data. Thus, an integrated image can be generated.

  Therefore, it is possible to accurately integrate the same range in the sample S and obtain a high S / N integrated image that can generate a super-resolution image. As a result, an excellent super-resolution image in which the high-frequency region is visualized can be generated by performing image processing for emphasizing the high-frequency region using the integrated image as an original image.

  In the present embodiment, the case of correcting only the positional deviation in the X and Y directions has been described. However, the positional deviation in the Z direction may be corrected from the inter-image difference. For example, as an integrated image for generating a super-resolution image, as shown in FIG. 23, a case where four images are acquired at each cross section at five positions (5 slices) having different cross-sectional positions in the Z direction is illustrated. To explain.

  First, reference image data from Z = 1 to 5 is acquired, then original image data in a slice with Z = 1, original image data at Z = 2, and so on up to original image data at Z = 5. Go. Next, the difference calculation unit 37 calculates the positional deviation in the X and Y directions with respect to the original image data at each Z position by pattern matching or the like with the reference image data at the Z position.

  In this case, when the pattern matching is disturbed to the extent that it is not only the positional deviation in the X and Y directions, matching is also performed on the reference image data at different slice positions in the Z direction. If matching with reference image data at different slice positions is optimized, it can be determined that drift in the Z direction has occurred during acquisition of the original image data.

Next, the image shift correction unit 161 corrects the positional deviation in the X and Y directions of the original image data by pixel shift. Also, as for the Z position, it can be determined whether it is the Z position that is currently assumed to be acquired, or whether it has drifted and moved to another Z position. In such a case, the image can be corrected as an image at an appropriate Z position by correcting the order of the Z stack.
Since the Z slices at both ends may be repelled, it is preferable to set the Z slices in advance wider than expected for drift.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included. For example, the present invention is not limited to those applied to the above-described embodiments and modifications, but may be applied to embodiments in which these embodiments and modifications are appropriately combined, and is not particularly limited. .

1,101,201 Specimen observation device 3 Stage 7 Scanner (modulation unit)
15 Objective lens 21 Confocal pinhole (passage range limiting member)
25 Detector (image data acquisition unit)
31 System control unit (control unit, second control unit)
33 Image data acquisition unit 37 Difference calculation unit (position deviation amount calculation unit, second position deviation amount calculation unit)
41 Image Data Integration Unit 157 Image Sensor 161 Image Shift Correction Unit (Position Deviation Correction Unit)
S specimen

Claims (13)

A stage for placing the specimen;
An objective lens that irradiates the specimen with excitation light emitted from a light source;
A modulator for modulating the spatial intensity distribution of excitation light irradiated onto the specimen by the objective lens;
A passage limiting member capable of passing the return light returning from the focal position of the objective lens in the specimen and blocking light from other than the focal position;
An image data acquisition unit that detects return light that has passed through the passage restriction member and acquires original image data for generating a super-resolution image;
An image data integration unit that generates an integrated image by integrating a plurality of the original image data obtained by irradiating the same range in the specimen with excitation light multiple times;
A misregistration amount calculation unit that calculates a misregistration amount in a direction in which the spatial intensity distribution of the original image data acquired by the image data acquisition unit and the reference image data serving as a reference spreads;
When the positional deviation amount exceeding the predetermined positional deviation detection threshold is calculated by the positional deviation amount calculation unit, the positional deviation amount of the original image data acquired by the image data acquisition unit is the predetermined positional deviation. A specimen observation apparatus comprising: a control unit that controls the stage, the modulation unit, and / or the image data acquisition unit so as to be equal to or lower than a detection threshold.   The specimen observation apparatus according to claim 1, wherein the control unit adjusts the irradiation position of the excitation light on the specimen under the control of the stage and / or the modulation unit. The image data acquisition unit is an image sensor arranged at a position where an image of the sample is formed,
The specimen observation apparatus according to claim 1, wherein the control unit adjusts a position of the imaging element with respect to imaging of the specimen. The positional deviation amount calculation unit calculates the positional deviation amount each time the original image data is acquired by the image data acquisition unit,
The image data acquisition unit interrupts the acquisition of the original image data when the misregistration amount calculation unit calculates a misregistration amount exceeding the predetermined misregistration detection threshold, and the control unit causes the original image data to be acquired. The specimen observation apparatus according to any one of claims 1 to 3, wherein the acquisition of the image data is resumed after the positional deviation of the image is resolved. The image data acquisition unit acquires image data of a misalignment detection position that is a position different from the observation position in the specimen;
The position shift amount calculation unit calculates a position shift amount between the original image data and the reference image data using image data at the position shift detection position acquired by the image acquisition unit. The specimen observation apparatus according to any one of 4.   The control unit controls the stage, the modulation unit, and / or the image data required for the positional deviation amount of the original image data with respect to the reference image data and the positional deviation amount to be equal to or less than the predetermined positional deviation detection threshold. The sample observation apparatus according to claim 1, further comprising a table that associates the control amount of the acquisition unit.   7. The notification unit according to claim 1, further comprising: a notification unit that notifies that an abnormal state is detected when the positional shift amount calculated by the positional shift amount calculation unit exceeds a predetermined abnormality detection threshold value. Sample observation device.   The sample observation apparatus according to claim 1, wherein the control unit corrects a displacement of an observation position by the control of the modulation unit. A second misregistration amount calculation unit that calculates a second misregistration amount in a direction intersecting a direction in which the spatial intensity distribution of the original image data and the reference image data spreads;
In a case where the second positional deviation amount exceeding the predetermined second threshold is calculated by the second positional deviation amount calculation unit, the positional deviation amount of the original image data acquired by the image data acquisition unit is the predetermined value. The specimen observation apparatus according to any one of claims 1 to 7, further comprising a second control unit that controls the stage and / or the objective lens so as to be equal to or less than a positional deviation detection threshold value.   The specimen observation apparatus according to any one of claims 1 to 8, further comprising an active autofocus unit that adjusts an in-focus state of the objective lens in the optical axis direction with respect to the specimen. An objective lens that irradiates the specimen with excitation light emitted from a light source;
A modulator for modulating the spatial intensity distribution of the excitation light irradiated on the specimen by the objective lens;
A passage limiting member capable of passing the return light returning from the focal position of the objective lens in the specimen and blocking light from other than the focal position;
An image data acquisition unit that detects return light that has passed through the passage restriction member and acquires original image data for generating a super-resolution image;
An image data integration unit that generates an integrated image by integrating a plurality of the image data obtained by irradiating the same range in the specimen with excitation light multiple times;
A misregistration amount calculation unit that calculates a misregistration amount in a direction in which the spatial intensity distribution of the original image data acquired by the image data acquisition unit and the reference image data serving as a reference spreads;
When the positional deviation amount exceeding the predetermined positional deviation detection threshold is calculated by the positional deviation amount calculation unit, the positional deviation amount of the original image data integrated by the image data integration unit with respect to the reference image data is the predetermined amount. A sample observation apparatus comprising: a positional deviation correction unit that corrects the positional deviation by shifting pixels of the original image data so as to be equal to or smaller than the positional deviation detection threshold value.   The specimen observation apparatus according to claim 11, wherein the image data acquisition unit acquires the original image data having a wider field of view than the observation target region. The specimen observation apparatus according to claim 11 or 12, wherein the positional deviation amount calculation unit also calculates a positional deviation in a direction intersecting a direction in which the spatial intensity distribution of the original image data and the reference image data spreads. .