JP2016173511A - Image acquisition device, and image acquisition method using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image acquisition device and image acquisition method capable of efficiently acquiring an image obtained by reducing an artifact.SOLUTION: An image acquisition device includes: an imaging optical part 209; an imaging part 213; a change part 204; a plane generation part 217 for generating an assumption acquisition plane by using positions in an optical axis direction of an analyte in a subject at each of a plurality of different positions of the subject 208; a position acquisition part 218 for acquiring an estimated position of a focal plane at one or more positions in a first area by using the position in the optical axis direction of the focal plane when imaging a second area at one or more positions in the second area adjacent to the first area; and a correction amount acquisition part 219 for acquiring a difference between the position in the optical axis direction of the assumption acquisition plane and the estimated position, and acquiring a correction amount on the basis of a result of comparing the difference with the depth of field of the imaging optical part.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image acquisition device and an image acquisition method and system using the same. The present invention is suitably used for WSI (Whole Slide Imaging) and WSI devices.

  Conventionally, in cancer examinations and the like, pathological diagnosis has been performed in which a tissue (specimen) collected from a patient is prepared and observed with an optical microscope to identify the type and state of a lesion. On the other hand, by using a WSI device that digitally images a slide and uses it as an image, it is possible to make a diagnosis using an image displayed on the display.

  Some WSI apparatuses use a method of acquiring an image of a subject in a large area by acquiring the image by dividing the imaging range of the subject into small areas when acquiring the image of the entire slide surface. Hereinafter, a divided area obtained by dividing the imaging range into small areas is referred to as a tile, and image data acquired by imaging in each tile is referred to as a tile image.

  In a WSI apparatus that generates an image of a subject by combining such tile images, there may be a relative inclination between the image sensor 801 that acquires the image and the sample 804 that is the imaging target. As an example, FIG. 7A shows a case where a plurality of tile images are acquired in a state where the specimen 804 and the image sensor 801 are not parallel. In such a case, as the number of tiles increases, the focal plane when acquiring the tile image may deviate from the specimen. As a result, an artifact in which regions having different positions in the optical axis direction exist in the combined image has occurred.

  In addition, when combining tile images, the position in the optical axis direction differs for each tile image at the joint portion between adjacent tiles, and artifacts may occur. This is influenced by the parallelism of the stage, which is one of the general driving accuracy of the stage. A conceptual diagram of the influence of the running parallelism of the stage is shown in FIG. Due to the influence of the running parallelism, an unintended displacement of the objective lens 802 in the optical axis direction (Z-axis direction) may occur when driven in a plane (XY plane) direction perpendicular to the optical axis. Another cause is that there is a relative tilt between the image sensor 801 and the sample 804, and the moving direction of the stage, that is, the moving direction of the sample 804 and the tilt direction of the focal plane are different. This occurs because the position in the optical axis direction in the tile is different between the right end of the tile and the left end of the tile, as shown in the conceptual diagram of FIG.

  Artifacts as described above have caused a reduction in the efficiency of diagnosis using digital images. In order to reduce artifacts, in Patent Document 1, a change in the position of the specimen in the optical axis direction is obtained from focal plane position information obtained at a plurality of positions on the specimen, and each tile image is obtained from this change. It is disclosed that it is reflected when the position of the focal plane is determined. According to this method, when the above-described image is acquired, artifacts due to the focal plane deviating from the specimen can be reduced.

  In Patent Document 2, the contrast of tile images acquired at different positions in the optical axis direction at each tile position is evaluated, and the image acquired at the position in the optical axis direction where the contrast is maximized is set as a focused image. It is disclosed. If so-called auto-focusing is performed on each tile as in this method, the above-described focal plane deviates from the specimen, and artifacts due to focal plane fluctuations due to the influence of the stage running parallelism can be reduced. Japanese Patent Laid-Open No. 2004-133867 acquires in-focus positions at a plurality of different positions in the plane direction of the subject, and corrects the position of the specimen so that the acquired in-focus position and the light receiving surface of the imaging sensor are conjugate. It is disclosed.

JP2013-83925A JP 2012-37861 A JP 2004-191959 A

  However, the method disclosed in Patent Document 1 cannot cope with the fact that the focal plane fluctuates unintentionally for each tile due to the influence of the running parallelism of the stage. In other words, the position of the focal plane set by the method of Patent Document 1 may change from the position of the focal plane where the tile image is actually acquired. Further, the method disclosed in Patent Document 2 cannot cope with artifacts caused by the tilt of the focal plane. Furthermore, since auto-focusing is executed for each tile, the in-focus position may be greatly different between adjacent tiles depending on the structure in the specimen (for example, the position of the nucleus). As a result, artifacts may occur due to the difference in the position of the focal plane in the optical axis direction at the joint when combined. In addition, the method disclosed in Patent Document 3 does not consider the difference in the position in the optical axis direction between tile images, and therefore an artifact may occur at the joint between adjacent tiles.

  The present invention has been made in view of the above problems, and an object thereof is to provide an image acquisition apparatus and an image acquisition method that can efficiently acquire an image with reduced artifacts.

  An image acquisition device according to one aspect of the present invention is an image acquisition device that acquires an image of a subject by imaging the subject for each of a plurality of regions, and an imaging optical unit that forms an image of light from the subject An imaging unit that captures an image of the subject, a focal plane that is conjugate with an imaging surface of the imaging unit, and a change unit that changes a relative position of the imaging optical unit in the optical axis direction of the subject, and the subject A surface generation unit that generates an assumed acquisition surface using the position in the optical axis direction of the specimen in the subject at each of the plurality of positions, and one or more in a second region adjacent to the first region The predicted position of the focal plane at one or more positions in the first area is acquired using the position of the focal plane in the optical axis direction when the second area is imaged at the position A position acquisition unit and the optical axis of the assumed acquisition surface; Obtaining the difference between the orientation position and the predicted position, and comparing the difference with the depth of field of the imaging optical unit, the focal plane from the assumed acquisition surface to the depth of field And a correction amount acquisition unit that acquires a correction amount for correcting the relative position so as to fall within the range.

  An image acquisition apparatus according to another aspect of the present invention is an image acquisition apparatus that acquires an image of a subject by imaging the subject for each of a plurality of regions, and forms an image of light from the subject. An imaging optical unit, an imaging unit that captures an image of the subject, and a changing unit that changes a relative position in the optical axis direction of the imaging optical unit between a focal plane conjugate with an imaging surface of the imaging unit and the subject. A surface generation unit that generates an assumed acquisition surface using positions in the optical axis direction of the specimen in the subject at each of a plurality of different positions of the subject, and differs in the optical axis direction of the first region. The range of the depth of field of the imaging optical unit from the assumed acquisition plane when the plurality of images are captured from the plurality of images acquired by imaging a plurality of positions with the imaging unit. Included in and , And having a extraction unit for extracting a portion of the first region and the image or the plurality of images included from a second region adjacent to the range of the depth of field.

  According to the image acquisition device and the image acquisition method as one aspect of the present invention, an image with reduced artifacts can be acquired efficiently.

The figure explaining WSI apparatus system configuration The figure explaining the structure of a microscope apparatus and an image processing apparatus Block diagram of hardware configuration of image processing apparatus The flowchart of the image acquisition process which concerns on 1st Embodiment. The figure explaining the WSI apparatus system configuration concerning a 2nd embodiment Flowchart of an image acquisition process according to the second embodiment Explanatory drawing of the problem of the present invention The figure explaining the image acquisition process which concerns on 1st Embodiment. Diagram explaining the relationship between tile, Z layer, and Z stack

(First embodiment)
<WSI device system configuration>
A WSI apparatus system as an image acquisition apparatus of the present embodiment will be described with reference to FIG. FIG. 1 is a diagram for explaining a device configuration of a WSI device system. This WSI apparatus system is a system for acquiring an optical microscope image of a test sample on a preparation serving as a subject 208 as a high-resolution and large-size (wide angle of view) digital image.

  The WSI apparatus system includes a microscope apparatus 101 (hereinafter referred to as “apparatus 101”), an image processing apparatus 102 (hereinafter referred to as “apparatus 102”), a computer 103, and a display apparatus 104. The apparatus 102 is incorporated in the computer 103 as a dedicated processing board. The device 101 and the computer 103 are connected by a dedicated or general-purpose I / F cable 105, and the computer 103 and the display device 104 are connected by a general-purpose I / F cable 106. In the present embodiment, the system configuration shown in the figure is adopted. For example, the device 102 may be incorporated in the device 101, a notebook PC in which the computer 103 and the display device 104 are integrated, or a device in which all are integrated. You may comprise using.

  Next, the configuration of the apparatuses 101 and 102 will be described with reference to FIG. FIG. 2 is a diagram illustrating the configuration of the apparatuses 101 and 102.

<Microscope device>
The apparatus 101 acquires tile image data while switching color filters at a plurality of different positions along two orthogonal axes in a plane. The apparatus 101 includes an illumination unit 201, a filter wheel unit 202, an illumination optical unit 203, a stage 204, a stage control system 205, an imaging optical unit 209, an imaging unit 213, and a main control system 214.

  The illumination unit 201 is a unit that illuminates the subject (preparation) 208 disposed on the stage 204 uniformly using the illumination optical unit 203, and includes a light source and a light source drive control system. Is done. The preparation 208 is a member in which a section of tissue to be observed and smeared cells (hereinafter referred to as “specimen 804”) are attached on a slide glass and fixed under the cover glass together with an encapsulant. The filter wheel unit 202 includes a plurality of color filters, a filter wheel for switching them, and a filter wheel control system for controlling switching.

  The stage 204 is a changing unit that changes the relative position of the focal plane and the subject 208 in the optical axis direction of the imaging optical unit 209 and the direction perpendicular to the optical axis direction. The stage 204 is driven and controlled by the stage control system 205, and can move in the three-axis directions of XYZ. The stage control system 205 is a part that drives and controls the stage 204, and includes a drive control system 206 and a stage drive mechanism 207. The drive control system 206 receives an instruction from the main control system 214 and performs drive control of the stage 204. The stage drive mechanism 207 drives the stage 204 in accordance with instructions from the drive control system 206. The imaging optical unit 209 is a lens group for forming an optical image of the subject 208 on the imaging sensor light receiving element surface in the imaging unit 213.

  Note that the “focal plane” in this specification is a plane optically conjugate with the imaging surface (light receiving surface) of the imaging sensor 231 of the imaging unit 213 via the imaging optical unit 209. Further, the focal position is a position conjugate with the position where the optical image of the subject 208 is formed by the imaging optical unit 209, and the focal plane includes the focal position. Furthermore, the fact that the sample in the subject 208 and the light receiving surface of the imaging sensor 231 are optically conjugate is referred to as “focusing”.

  The imaging unit 213 includes an imaging sensor 231, an analog front end (AFE) 232, and a black adjustment unit 233. The imaging unit 213 captures an image of the subject 208 and acquires tile image data by driving the stage 204 in the XY plane direction with the plane orthogonal to the optical axis direction of the imaging optical unit 214 as the XY plane. . The tile image data acquired by the imaging unit 213 is hereinafter referred to as original image data.

  The imaging sensor 213 is a two-dimensional image sensor that changes a two-dimensional optical image into an electrical physical quantity by photoelectric conversion. For example, a CCD or a CMOS device is used. An electrical signal corresponding to the intensity of light is output from the image sensor 231. The imaging unit 213 captures a sample tile image by driving the stage 204 in the XY plane direction. The AFE 232 is a circuit that converts an analog signal output from the image sensor 231 into a digital signal. In the case of an image sensor using a CMOS, a configuration in which the function of the AFE 232 is integrated with the imaging sensor may be employed. The black correction unit 233 performs a process of subtracting the black correction data obtained at the time of shading from each pixel of the tile image data acquired by the imaging sensor 231 and the AFE 232.

  The main control system 214 performs control of each configuration described so far using the control instruction acquired from the sub-control system 228 in the apparatus 102 and information for determining the instruction.

<Image processing device>
The apparatus 102 calculates a correction amount for correcting the position of the focal plane from the original image data acquired from the apparatus 101. Further, display data to be displayed on the display device 104 is generated in response to a user request based on the original image data. The apparatus 102 includes a user information acquisition unit 215 (hereinafter referred to as “acquisition unit 215”), a surface generation unit 217 (hereinafter referred to as “generation unit 217”), and a focal position correction unit 216 (hereinafter referred to as “correction unit 216”). And an image position correction unit 220 (hereinafter referred to as “correction unit 220”). Further, the apparatus 102 includes a development processing unit 221, a display data generation unit 227 (hereinafter referred to as “generation unit 224”), and a sub control system 228.

  The correction unit 216 includes a position acquisition unit 218 (hereinafter referred to as “acquisition unit 218”) and a correction amount acquisition unit 219 (hereinafter referred to as “acquisition unit 219”). The development processing unit 221 includes a gain adjustment unit 222 (hereinafter referred to as “adjustment unit 222”), an image composition processing unit 223 (hereinafter referred to as “processing unit 223”), and a color reproduction image generation unit 224 (hereinafter referred to as “ A generation unit 224 "). Further, the development processing unit 221 includes a digital filter processing unit 225 (hereinafter referred to as “processing unit 225”) and a compression processing unit 226 (hereinafter referred to as “processing unit 226”).

  The acquisition unit 215 acquires imaging condition information of the apparatus 101 and color reproduction information when generating color reproduction image data by the development processing unit 221 from a storage device in the computer 103. Imaging condition information and color reproduction information are input by a user in advance via a UI or the like and held in a storage device. Alternatively, a preset one may be used, or the sub-control system 228 may automatically determine. As an example, if it is an imaging range, a reduced image of the slide is displayed, and the user is allowed to specify the imaging range using a GUI. Similarly, as an example of obtaining color reproduction information, color matching function candidates and the like are displayed and selected by the user. Among the acquired information, the imaging condition information is sent to the main control system 214 and the development processing unit 221 through the sub-control system 228, and the color reproduction information is sent to the development processing unit 221.

  The correction unit 216 acquires the assumed acquisition plane and the expected focal plane based on the focal plane position information detected from the specimen, and calculates a correction amount for correcting the focal plane position at the time of tile image acquisition. Acquisition of the focal plane position information and the calculated correction amount are performed via the sub-control system 228. In the present embodiment, the correction amount is the amount of movement in the height direction (optical axis direction) of the stage. Detailed processing steps in each functional block of the correction unit 216 will be described later. The assumed acquisition surface is a surface of the subject 208 for which an image is to be acquired by imaging, and is generated so that the position in the subject 208 where the specimen exists can be imaged.

  The generation unit 217 determines the stage XY position for detecting the position of the specimen, and detects and records the position of the specimen at the stage XY position. An assumed acquisition surface is generated based on the recorded position of the specimen. The acquisition unit 218 acquires a relative inclination (inclination of the focal plane of the image sensor 231) parameter between the image sensor 231 and the stage 204 through the sub-control system 228. Next, the positional variation of the focal plane due to the influence of the parallelism of the stage running between the tile from which the tile image has been acquired and the adjacent tile is corrected. Thereafter, an expected position at an arbitrary XY position in the adjacent tile is calculated together with the inclination parameter described above. The acquisition unit 219 calculates a correction amount of the focal plane position at the time of capturing the tile image based on the assumed acquisition plane and the expected position.

  The correction unit 220 corrects misalignment between a plurality of original image data acquired from the apparatus 101.

  The development processing unit 221 generates color reproduction image data to be displayed on the display device 104 using the original image data acquired by the device 101.

  The adjustment unit 222 corrects the difference in exposure amount by adjusting the gain of the image data for each color filter in the apparatus 101. The processing unit 223 joins the original image data and generates image data of a large area subject based on the imaging condition information. The generation unit 224 converts the image data generated by the processing unit 223 into XYZ chromaticity coordinate values. The converted tile image data is referred to as XYZ image data. The processing unit 225 has a function of a digital filter that realizes suppression of high-frequency components included in XYZ image data, noise removal, and resolution enhancement.

  The processing unit 226 is a still image compression encoding process performed for the purpose of improving the efficiency of transmission of data of a two-dimensional image of a large area subject and reducing the data capacity when storing the data. As a still image compression technique, standardized encoding methods such as JPEG (Joint Photographic Experts Group) and JPEG 2000 and JPEG XR improved and evolved from JPEG are widely known. The compressed image data is sent to a storage device in the computer 103 and stored. The generation unit 227 converts the XYZ image data generated by the development processing unit 221 into an RGB color system that can be displayed on the display device 104 using a lookup table.

  The sub control system 228 controls various units described so far in the apparatus 102 and transmits / receives information related to the control of the apparatus 101 through the main control system 214. If necessary, the imaging conditions and the like are automatically determined and determined using information input to the acquisition unit 215 and information related to imaging stored in the storage device.

  FIG. 3 is a block diagram illustrating a hardware configuration of the computer 103. In the present embodiment, a device 102 that is a dedicated processing board is incorporated in a general PC (Personal Computer). The computer 103 includes a CPU (Central Processing Unit) 301, a RAM (Random Access Memory) 302, a storage device 303, a data input / output I / F 305, and an internal bus 304 that connects these components to each other.

  The CPU 301 appropriately accesses the RAM 302 or the like as necessary, and performs overall control of the entire blocks of the PC while performing various arithmetic processes. The RAM 302 is used as a work area for the CPU 301 and temporarily holds the OS, various programs being executed, and various data. A program corresponding to the flowcharts of FIGS. 4A to 4G is stored in the RAM 302, and each process is performed by the CPU 301 reading and executing the program.

  The storage device 303 is an auxiliary storage device that records and reads information in which firmware such as an OS, a program, and various parameters executed by the CPU 301 is fixedly stored. A semiconductor device such as an SSD (Solid State Disk) or a magnetic disk drive such as an HDD (Hard Disk Drive) or a flash memory is used.

  The data input / output I / F 305 is connected to a device 102 that is a dedicated processing board, a LAN I / F 306 for connecting to a network, a graphics board 307, an external device I / F 308, and an operation I / F 309. Further, the display device 104 is connected to the graphics board 307, the device 101 is connected to the external device I / F 308, and the keyboard 310 and the mouse 311 are connected to the operation I / F 309.

<Display device>
The display device 104 is a display device using, for example, liquid crystal, EL (Electro-Luminescence), CRT (Cathode Ray Tube), or the like. As described in the schematic configuration of the WSI device system, a notebook PC in which the display device 104 and the computer 103 are integrated may be assumed. As a connection device with the operation I / F 409, a pointing device such as a keyboard 410 and a mouse 411 is assumed. It is also possible to adopt a configuration in which the screen of the display device 104 such as a touch panel directly serves as an input device. In that case, the touch panel can be integrated with the display device 104.

<Image acquisition process>
The image data acquisition process in the apparatus 101 and the apparatus 102 of this embodiment will be described with reference to FIG. In the present embodiment, the position of the focal plane is corrected when image data is acquired.

  When acquiring image data, before starting to capture a tile image, first, the position of the sample is detected at at least three points on the sample. The point where this position detection is performed is acquired at a position where calibration data, which will be described later, exists in order to reduce the influence of the stage running parallelism.

  Here, the calibration data is data obtained by performing position detection at each of a plurality of specific positions in the XY direction of the stage 204 in advance. If this calibration data is used, a method of correcting the focal plane position fluctuation due to the influence of the stage running parallelism and reducing the influence of the stage running parallelism can be considered. However, this method can execute correction only at the position where the calibration data is acquired. Therefore, it is necessary to acquire calibration data every time the sample is replaced and the position and size of the imaging range are changed. Otherwise, the position and size of the imaging range, and the tile assignment for the imaging range described above are fixed, so that flexibility is lacking, and it may be necessary to image an extra area where no specimen exists. There is sex.

  Position detection is performed by evaluating the contrast of an image captured by the image sensor 231 when the position of the stage 204 is changed in the optical axis direction (height direction, Z-axis direction), and matching the stage position where the contrast becomes the highest. The focus position. Of course, the method is not limited to this method, and a method of performing autofocus, such as a method of using another optical system or a method of detecting a phase difference, can be used. If position detection is performed, the assumption acquisition surface 901 including the point which performed position detection is produced | generated (FIG. 8 (a)). The generated assumed acquisition plane 901 is used as an index for calculating the position of the focal plane when the tile image is captured. By using this assumed acquisition plane 901, the focal plane when acquiring an image due to the influence of the relative tilt between the imaging sensor 231 and the stage 204 loading surface and the specimen 910 that is the imaging target described above is used. The problem of moving away from the specimen 910 is addressed.

  Subsequently, the tile image is captured while repeating a step operation including imaging and stage movement. At this time, as shown in FIG. 8B, a tile captured in a certain step is set as a reference tile (second region) 920, and a tile image acquired by imaging the reference tile 920 is set as a reference tile image. Furthermore, a tile adjacent to the reference tile 920 that is imaged in another step is a corrected tile (first region) 921, and a tile image acquired by imaging the corrected tile 921 is a corrected tile image.

  From each of the reference tile image and the corrected tile image, the position of the focal plane in the overlapping area (in the overlapping area 922) of the reference tile 920 and the corrected tile 921 is detected, and the position of the focal plane in the overlapping area 922 is determined. The correction amount is obtained so as to be the same. By correcting the focal plane position between the reference tile 920 and the tile to be corrected 921, the focal plane position variation due to the influence of the stage running parallelism and the focal position shift due to the focal plane tilt described above can be handled. To do.

However, if a large number of tile images are joined together, the difference in focal plane position at the end of the tile image due to the tilt of the focal plane is accumulated. For this reason, there is a possibility that the focal plane of the acquired image deviates from the specimen 910. Therefore, in the corrected tile 921, the focal plane position after the correction of the focal plane position between the reference tile 920 and the corrected tile 921 is added to the focal plane inclination of the image sensor 231. The predicted focal plane 931 is calculated (FIG. 8C). The expected focal plane 931 is brought close to the assumed acquisition plane 901 within a range within the depth of field of the imaging optical unit 209 from the reference tile 920 (FIG. 8D). In the present specification, “depth of field (DOF)” is ± D. O. It is defined as represented by the formula F = ± λ / {2 (NA) ² }. Here, λ is the wavelength of light, and NA is the numerical aperture of the imaging optical system 102.

  In this way, the position of the focal plane is aligned in the overlapping area 923 between adjacent tiles, and the predicted focal plane 931 is acquired as the focal plane of the image sensor 231. By bringing the predicted focal plane 931 closer to the assumed acquisition plane 901 in the range of the depth of field, the focal plane is prevented from being separated from the specimen, and an image with reduced artifacts is acquired.

  The details of the image acquisition process will be described with reference to the flowcharts of FIGS. In step S401, the main control system 214 initializes the apparatus 101. A process related to the initialization of the apparatus 101 is shown in the flowchart of FIG.

  In step S415, the user uses the computer 103 to acquire imaging condition information such as the imaging range on the slide 208, the exposure time of the imaging sensor 231, the filters used for imaging and their transmission wavelength bands, and the magnification of the imaging optical unit 209. To the device 102. At this time, color reproduction information such as a color matching function is also input as necessary. The input information is once stored in the storage device 303 and then acquired by the acquisition unit 215. In addition to being input by the user, the imaging condition information may be selected by a user from preset values, or may be automatically determined by the sub-control system 228 or the like. In step S 416, the main control system 214 acquires the information input in step S 415 from the acquisition unit 215 through the sub control system 228.

  In step S417, the main control system 214 calculates the depth of field of the objective lens, divides the imaging range into tiles, calculates the moving distance of the stage 204 based on the tile division results, according to the conditions acquired in step S416. Determine the order of movement. Further, according to the moving order, an ID number is assigned to each tile and associated with the XY position on the stage 204. At the same time, initialization of the image sensor 231, the stage 204, the light source of the illumination unit 201, and the like is performed. Note that when the imaging range is divided into tiles, the tiles are divided so as to have overlapping regions between the tiles (see FIGS. 7 and 8B). In addition, the depth of field, movement order, ID number, and stage XY position information obtained in this step are stored in the RAM 302 or the storage device 303, and are appropriately read and used in a process described later.

  In step S <b> 402, the main control system 214 acquires a parameter (inclination parameter) representing the inclination of the focal plane of the image sensor 231 with respect to the stage 204 caused by the relative inclination between the image sensor 231 and the stage 204. The tilt parameter is calculated based on the position of the focal plane in the image captured using a calibration slide having a chart or target pattern that is uniformly flat and capable of focus detection by the image. The position detection is automatically performed, for example, by evaluating the contrast of the captured image. In addition, it is possible to calculate using a distance measuring device or the like.

  The calculation of the slope parameter may be started when the process of step S402 is reached, or may be calculated in advance such as when the apparatus is activated. In this case, the calculated result is stored in the storage device 303, and is read and used as appropriate. Further, the focal plane tilt parameter may be expressed by an expression representing a plane in a general three-dimensional space, or by using a table in which the position in the optical axis direction at an arbitrary position in the focal plane is recorded. Also good.

  In step S403, the generation unit 217 generates the assumed acquisition surface 901. Each step relating to the generation of the assumed acquisition plane 901 is shown in the flowchart of FIG.

  In step S418, the generation unit 217 determines a position used when generating the assumed acquisition surface and a position on the stage XY plane (focus detection XY position) from which the position is acquired. A process related to the determination of the focus detection XY position is shown in the flowchart of FIG.

  In step S421, the generation unit 217 generates or acquires the calibration data described above. The calibration data is a table in which the variation in the stage height direction for each XY coordinate of the stage 204, that is, the focal plane position variation with respect to the specimen 910 is recorded. The variation in the position of the focal plane is represented by a displacement amount with reference to the focal position detected at the origin position in the XY direction of the stage 204, for example. Similar to the focal plane tilt parameter described above, the displacement is calculated based on the focal position in the image captured using the calibration slide, or is detected by detecting the displacement using a laser displacement meter or the like. be able to.

  The calibration data may be generated when the process of step S421 is reached. However, taking into account the execution time of the imaging process, the calibration data is generated when the apparatus is activated, and the data stored in the storage device 303 is acquired. Is desirable. As will be described later, the focus detection XY position is set at a position where calibration data exists. Therefore, it is not necessary for the calibration data to have flexibility with respect to the position and size of the imaging range as described above.

  In step S422, the generation unit 217 sets the number of focus detection XY positions. The number of focus detection XY positions may be determined by the sub-control system 228 according to the imaging range, or may be input by the user. However, at least two points (in this case, it is necessary to specify the inclination of a plane including a straight line passing through these two points), or three or more points need to be set. At this time, the distance between the points is set to be longer than the length of each tile. In step S423, the imaging range acquired in step S416 is compared with the calibration data acquired in S421, and the calibration data at the XY position in the imaging range is examined.

  In step S424, the generation unit 217 provisionally determines the focus detection XY position based on the result of comparison in step S423. If the focus detection XY position has never been determined, the temporarily determined position is set inside the tile corresponding to the outer peripheral portion of the imaging range in the calibration data at the XY position in the imaging range. If another focus detection XY position has already been determined, the position is set inside the tile that is farthest from the determined focus detection XY position. If it is determined in the subsequent step S425 that the sample 910 does not exist at the tentatively determined position, a tile adjacent to the tile selected at the tentative determination is newly selected.

  In step S425, the generation unit 217 determines whether the sample 910 exists at the XY position provisionally determined in step S424. If the sample 910 exists, the process proceeds to step S426, and if not, the process proceeds to step S424. In step S426, the XY position temporarily determined in step S424 is recorded in the assumed acquisition surface generation table as the focus detection XY position. In step S427, it is determined whether focus detection XY positions are determined by the number acquired in step S422. If it is determined, the process proceeds to step S419. If it is not determined, the process proceeds to step S424.

  In this embodiment, in order to automatically determine the focus detection XY position, the processes from step S424 to step S427 are executed. However, the user can manually specify the focus detection XY position. In that case, the process from step S424 to step S427 may not be executed. Further, the focus detection XY position determined by the execution of steps S424 to S427 may be presented to the user as a candidate, and the user may be determined whether to adopt it as the focus detection XY position.

  In step S419, the generation unit 217 performs position detection at the focus detection XY position determined in step S418, and records the result in the assumed acquisition surface generation table. The position detection is automatically detected by the method described above, but may be performed manually by the user if necessary. When position detection is performed by evaluating image contrast or the like, the position of the focal plane to be detected may deviate from the optimum position due to the presence of a clear structure such as a nucleus in the specimen 910. . In order to reduce this possibility, a range of a certain size (tile size or the like) including the focus detection XY position is defined, and several small positions (patches) are provided in the range. The results of position detection in each patch are statistically processed by averaging or the like, and are set as the focal plane position at the focus detection XY position. The same effect can be achieved by using one patch and increasing the patch range.

  In step S420, the generation unit 217 generates the assumed acquisition surface 901 based on the record of the assumed acquisition surface generation table. In generating the assumed acquisition plane 901, an expression that defines a plane in a general three-dimensional space can be used. Moreover, you may prescribe | regulate a curved surface using functions, such as a multidimensional function instead of a plane, as needed. In the present embodiment, the assumed acquisition plane is a plane in the three-dimensional space.

  In step S404, the stage control system 205 moves the stage 204 to the tile to be imaged first in accordance with the movement order determined in step S401. Thereafter, the apparatus 101 focuses on the tissue in the specimen 910 of the preparation 208 according to the assumed acquisition surface 901 generated in step S403. At this time, it is desirable to adjust the position of the focal plane using the calibration data acquired in step S421. Thereafter, the imaging unit 213 captures a tile image and stores the image data in the storage device 303. In the image data, the position information of the focal plane when the tile image is captured is recorded in the header portion. Alternatively, it may be recorded separately in the storage device 303 in association with the image data.

  In step S405, the stage control system 205 drives the stage 204 according to the movement order determined in step S401, and moves to the next tile to be imaged. Hereinafter, the tile after movement, that is, the tile from which a tile image is to be captured will be referred to as a current position tile. This corresponds to the above-described corrected tile (first region) 921 in the focal plane position correction process described later.

  In step S406, the acquisition unit 218 includes the sample 910 in the overlapping area 922 with the current position tile among all the tiles that have acquired the tile image data and are adjacent to the current position tile. Determine what you have. If there is a tile in which the sample 910 exists, the process proceeds to step S408, and if there is no tile, the process proceeds to step S407.

  In step S407, the acquisition unit 218 registers the ID of the current position tile in the skip list. The tile determined to have no specimen in the overlapping area 922 in step S406 cannot correct the focal plane position fluctuation. Therefore, the tile image is not captured in the current process step, but is registered in the skip list, and the tile image is retried in the subsequent process.

  In step S408, the acquisition unit 218 acquires the predicted position. The process for obtaining the predicted position is shown in the flowchart of FIG.

  In step S428, the acquisition unit 218 sets a tile in which the sample 910 exists in the overlapping area 922 determined in step S406 as a reference tile 920 (second area). When there are a plurality of tiles in which the specimen 910 exists in the overlapping area 922, the tile that has been imaged most recently in time series is set as the reference tile 920 in order to reduce the accumulation of correction errors.

  In step S429, the acquisition unit 218 acquires or detects and compares the positions in the optical axis direction at the same point in the overlapping area 922 of the reference tile 920 and the corrected tile 921 set in step S428. By this comparison, the fluctuation amount of the positional fluctuation of the focal plane that occurs due to the influence of the stage running parallelism is calculated. It is desirable from the viewpoint of processing speed that the position of the focal plane in the reference tile 920 is obtained by reading information on the position of the focal plane when the tile image is captured from the header portion of the image data or the storage device 303.

  In step S430, the acquisition unit 218 calculates the predicted position using the variation amount calculated in step S429 and the focal plane tilt parameter acquired in step S402. The predicted position is a position at an arbitrary position in the predicted focal plane 931 on the corrected tile 921 when the focal plane position variation is corrected and the influence of the focal plane tilt is taken into consideration.

  In step S409, the acquisition unit 219 performs a process for correcting the imaging position where the imaging sensor 231 actually captures an image. The process for adjusting the imaging position is shown in the flowchart of FIG.

  In step S431, the acquisition unit 219 compares the assumed acquisition surface 901 generated by the generation unit 217 in step S403 with the predicted position acquired by the acquisition unit 281 in step S408, and calculates the amount of deviation. The shift amount is calculated by taking the difference between the predicted position at an arbitrary XY position and the position of the focal plane formed by the assumed acquisition surface 901 at the same XY position. In order to increase the correction accuracy, it is desirable to take a difference at a plurality of XY positions. In the case of taking a plurality of differences, it is desirable to carry out the outer peripheral portion of the corrected tile 921 in order to reliably capture the influence of the focal plane tilt. For example, if the tile to be corrected 921 is rectangular, it is desirable to perform it at the four corners. The largest difference among the acquired differences is taken as the amount of deviation.

  In step S432, the acquisition unit 219 compares the amount of deviation calculated in step S431 with the depth of field of the imaging optical unit 209. If the amount of deviation is smaller than the depth of field, the process proceeds to step S433, and if larger, the process proceeds to step S434.

  In step S433, the acquisition unit 219 drives the stage 204 to a position where the expected position is brought closer to the assumed acquisition plane 901 by the amount of deviation based on the amount of deviation calculated in step S432, and the focal plane at the time of capturing the tile image. Change the position of.

  In step S434, the acquisition unit 219 brings the predicted position closer to the assumed acquisition plane 901 by the depth of field from the focal plane position in the overlapping area 922 acquired from the image data of the reference tile 920 in step 428. The position of is calculated. The predicted position is updated based on the calculation result of this position.

  In step S435, the acquisition unit 219 changes the XY position of the corrected tile 921 so that all of the predicted positions updated in step S434 are within the range of the depth of field around the assumed acquisition surface 901. The overlapping area 922 is enlarged. Note that the enlargement of the overlapping area 922 is not performed by changing the tile size, but by the parallel movement of the XY position of the corrected tile 921. The acquisition unit 219 acquires the amount of change of the focal plane position due to the parallel movement at this time as a correction amount. In other words, in steps S434 and S435, the acquisition unit 219 moves the corrected tile 921 in the optical axis direction and the direction perpendicular to the optical axis direction so that the predicted position is within the range of the depth of field. The amount of change in position 204 is set as a correction amount.

  In step S436, the acquisition unit 219 updates the tile division state of the imaging range set in step S416 based on the parallel movement amount of the tile XY position calculated in step S435. At this time, if an area where no tile is assigned within the imaging range, that is, an area where no image is captured (missing area) occurs due to a change in the XY position of the tile, a new tile is allocated to the missing area. The new tile has the same tile size as the other tiles, and covers the missing area by adjusting the size of the overlapping area with the other tiles.

  In step S437, the stage control system 205 drives the stage 204 based on the calculation results of steps S434 and S435, and the stage 204 changes the XY position or the focal position.

  In step S410, the image sensor 231 captures the tile to be corrected and acquires tile image data. Similar to step S <b> 404, the image data is stored in the storage device 303. In the image data, information on the position of the focal plane when the tile image is captured is recorded in the header portion. Alternatively, it may be recorded separately in the storage device 303 in association with the image data. In step S411, the sub-control system 208 determines whether all tiles other than the tiles registered in the skip list have been imaged. If all tiles have been imaged, the process proceeds to step S412. If there is a tile that has not yet been imaged, the process proceeds to step S405.

  In step S412, the apparatus 101 performs re-imaging of the tile (skip tile) registered in the skip list in step S407. A process related to re-imaging of the skip tile is shown in the flowchart of FIG.

  In step S438, the XY position of the skip tile is read from the skip list, and the stage control system 205 changes the stage XY position. At this time, it is desirable to capture an image from a newly registered tile in time series (so-called Last In First Out), because the time required for moving the stage 204 can be shortened. In step S439, as in step S406, the acquisition unit 218 has a sample 910 in the overlapping area with the current position tile of tiles that have been captured by the tile image and that are adjacent to the current position tile. Determine if there is. If there is a sample 910, the process proceeds to step S408. If there is no sample 910, the process proceeds to step S440.

  In steps S <b> 408 and S <b> 409, the correction unit 216 performs processing for correcting the imaging position of the tile image as in the above-described steps. However, if a missing area occurs and a new tile is assigned, the sub-control system 228 updates the division status of the tile in step S436, and adds the XY position of the new tile to the skip list.

  In step S440, the acquisition unit 218 determines whether the sample 910 exists in the current position tile. When the sample 910 does not exist, the current position tile is regarded as a blank tile. In this case, since it is not necessary to consider the connection with adjacent tiles, the process proceeds to step S410 without adjusting the imaging position. If the sample 910 exists, the process proceeds to step S441.

  The sample 910 in the current position tile has a size smaller than the tile size and can be regarded as being isolated from the surrounding samples 910 because the sample 910 does not exist in the overlapping area with all adjacent tiles. Therefore, since it is not necessary to consider the connection with adjacent tiles, in step S441, the position is individually detected using the above-described method, and the stage 204 is moved to the position of the focal plane.

  In step S410, the image sensor 231 images the current position tile. In step S442, the sub-control system 228 determines whether imaging has been performed for all tiles registered in the skip list. If all tiles have been imaged, the process proceeds to step S413. If there is a tile that has not yet been imaged, the process proceeds to step S438.

  The subsequent steps are processing in the correction unit 220, the processing unit 221, and the generation unit 227. In step S413, first, the correction unit 220 acquires data of each tile image from the storage device 303. The tile image data is corrected for positional deviation in the XY direction between tile images by a known method such as a corner detection method (such as the Harris method) or a SURF (Speeded Up Robust Features) method. The adjustment unit 222 performs gain adjustment on the tile image data in which the positional deviation is corrected. Next, the data of the tile images whose gains have been adjusted are connected by the logic unit 223. After that, the generation unit 224 generates a color reproduction image based on the image data of the subject 208 generated by joining. The generation unit 224 generates a color reproduction image based on the color reproduction information input by the user in step S401 or the color reproduction information stored in the storage device 303 in advance. Further, the processing unit 225 performs processing to generate color reproduction image data.

  In step S414, the processing unit 226 compresses the color reproduction image data generated in step S415, and sends the compressed data to the generation unit 227 and the storage device 303. The generation unit 227 generates display data using the compressed data and outputs the display data to the display device 104. The processing in the processing unit 226 and the processing in the processing unit 227 may proceed in parallel.

  As described above, in this embodiment, the focal plane is brought closer to the assumed acquisition plane 901 while the change in the focal plane position when the stage 204 is moved is corrected from the difference in focal plane position in the overlapping area between adjacent tiles. be able to. As a result, the focal plane position variation due to the influence of the parallelism of the stage running, the deviation of the focal plane from the specimen caused by the relative inclination between the stage 204 and the imaging sensor 231, and the artifacts at the joint between the tile images are eliminated. Can be reduced. As a result, it is possible to efficiently and accurately acquire a high-quality image in which artifacts generated at the joints at the time of tile image synthesis are suppressed.

(Second Embodiment)
The configuration of the image acquisition apparatus of this embodiment is the same as that of the first embodiment. The image acquisition method of this embodiment adds a step of performing pre-imaging to the image acquisition method of the first embodiment. Further, when acquiring tile images, imaging is performed at a plurality of positions different in the optical axis direction, and data of a plurality of tile images is acquired. Only differences from the image acquisition method of the first embodiment will be described.

  In the present embodiment, before starting the imaging of the tile image in the first embodiment, specifically, in step S401, a low-magnification objective lens that can capture the entire imaging range as the imaging optical unit 209 is used. In addition, pre-imaging that captures the entire imaging range of the subject 208 is incorporated. Steps for determining the presence of the specimen 910 in determining the focus detection XY position in generating the assumed acquisition plane 901 by using image data of the entire imaging range obtained by pre-imaging (S423, S424, S425), etc. Can be reduced. In addition, the setting of the reference tile 920 (S428) and the registration to the skip list (S407), which were sequentially performed in the first embodiment, can be performed collectively before the start of imaging. As described above, by incorporating pre-imaging, an image can be acquired at a higher speed.

  In the first embodiment, the difference between the expected acquisition plane 901 and the expected position is the difference between the expected position at an arbitrary XY position and the position of the focal plane formed by the assumed acquisition plane 901 at the same XY position. Went to take. In the present embodiment, the deviation is calculated using the size of the angle formed between the assumed acquisition surface and the surface constituted by the expected position. In this case, the size of the angle formed between the assumed acquisition surface and the expected position may be held as a parameter. As a result, it is not necessary to always hold the focal plane positions of the assumed acquisition plane and the expected position, and the capacity of a storage medium such as the RAM 302 for storing the focal plane position information can be saved. This method is particularly effective as the area of the assumed acquisition surface and the area of the tile increase.

  Further, in the first embodiment, when there is a shift greater than the depth of field between the assumed acquisition surface 901 and the predicted position, the area of the overlapping region 922 is adjusted by moving the XY position of the corrected tile 921. . As a result, the expected position falls within the depth of field from the assumed acquisition surface 901. In the present embodiment, as a method for dealing with the shift beyond the depth of field, a plurality of tile images (Z stack images) are acquired at a plurality of positions different in the optical axis direction. From the acquired plurality of tile images, those whose deviation is equal to or smaller than the depth of field are extracted. In this method, it is not necessary to update the division state of the captured image and acquire a newly assigned tile image in order to eliminate the missing area as in the first embodiment. Therefore, even if the specimen 910 has a strong undulation that causes many deviations beyond the depth of field, the data of the tile image to be acquired can be finally reduced, and the image can be acquired at a higher speed. It becomes possible.

  In the present embodiment, as in the first embodiment, the change in the position of the focal plane when the stage 204 is moved is corrected from the difference in the focal plane position in the overlapping area between adjacent tiles, while the assumption acquisition plane 901 is displayed. The focal plane can be brought closer. As a result, an image with reduced artifacts can be efficiently acquired.

(Third embodiment)
The WSI apparatus system of this embodiment will be described with reference to FIG. FIG. 5 is a diagram for explaining the configuration of the WSI apparatus system of the present embodiment. The slide scanner system includes a microscope apparatus 501 (hereinafter referred to as “apparatus 501”) and an image processing apparatus 502 (hereinafter referred to as “apparatus 502”), which is a dedicated processing board, via a network. And connect. This is a system configuration for acquiring a microscope image of the specimen 910 on the preparation 208 serving as a subject as a high-resolution and large-size (wide angle of view) digital image.

  The WSI device system includes a device 501, a device 502, a computer 503, and a display device 104. The device 501 and the computer 503 are connected by a dedicated or general-purpose I / F LAN cable 505 via the network 504, and the computer 503 and the display device 104 are connected by a general-purpose I / F cable 106. Is done.

  The apparatus 601 takes the same form as the apparatus 101 in the first embodiment, but differs in that it includes a LAN I / F for network connection. The device configurations of the device 502 and the display device 104 are the same as those in the first embodiment. The hardware configuration of the computer 503 is the same as that of the first embodiment, but is different in that the computer 503 is connected to the device 501 via a LAN I / F and a network.

  The focal plane position correction in the apparatus 501 and the apparatus 502 of the present embodiment will be described with reference to FIG. In the present embodiment, the imaging range 1010 determined from the range in which the sample 910 exists is divided into a plurality of tiles 1011 and imaged. Using tile images captured at a plurality of positions different in the optical axis direction (Z-axis direction) in each tile 1011, an image focused on the entire surface when the tile images are combined is acquired. From this point onward, each position in the optical axis direction of each tile 1011 is a Z layer 1001, images acquired by each Z layer are Z layer images, a plurality of Z layers are combined, a Z stack 1002, and a plurality of Z layer images are combined. Things are called Z-stack images. The correction unit 216 of the apparatus 501 extracts a Z layer image or a part of the Z layer image used for image synthesis from the Z stack image so that when the tile images are synthesized, an image focused on the entire surface can be acquired. This is an extraction unit.

  In the present embodiment, processing in the correction unit 216 is different from that in the first embodiment. Specifically, the correction unit 216 uses the above-described Z stack image to perform correction at the time of tile image synthesis, not at the time of tile image capture. The Z stack image in each tile 1011 is captured in advance by the device 501 and stored in the storage device 303 or the like. A detailed description of the process related to acquisition of the Z stack image is omitted.

  Next, the apparatus 502 acquires a captured Z stack image. The correction unit 216 generates the assumed acquisition plane 901 using the stage XY position information of the tile 1011 when the Z stack image is captured and the focal plane position information detected from the Z stack image. Subsequently, the correcting unit 216 gives a width corresponding to the depth of field of the objective lens in the optical axis direction of the generated assumed acquisition surface 901 to obtain an assumed acquisition range. For a Z layer image captured at each of a plurality of positions different in the optical axis direction constituting the Z stack image, a Z layer image or a part of the image whose focal plane is within the assumed acquisition range is extracted. This is performed for each tile 1011 and the extracted images are combined to obtain an image focused on the entire surface as described above.

  A process related to focal plane position correction using a Z stack image will be described with reference to the flowcharts of FIGS. Note that a program corresponding to the flowcharts of FIGS. 6A to 6E is stored in the RAM 302 of the present embodiment, and each process is performed by the CPU 301 reading and executing the program.

  In step S601, the main control system 214 acquires imaging conditions when a Z stack image is captured. The imaging conditions include, for example, the imaging range on the slide 208, the tile division status, the tile ID number, the stage XY position information associated with the ID number, and the focal plane position information at the time of imaging. Further, color reproduction information such as an exposure time of the image sensor, filters used for imaging, transmission wavelength bands thereof, and color matching functions may be included. These imaging conditions are acquired by the main control system 214 from the storage device 303 or the like through the sub control system 228. In addition, it may be acquired from a server or the like at a remote location via the network 504.

  In step S602, the correction unit 216 acquires Z stack image data from the storage device 303 or the like. Similar to the acquisition of the imaging condition described above, it may be acquired from a remote server or the like via the network 504. In step S402, the main control system 214 acquires a focal plane tilt parameter. Since it is the same as that of 1st Embodiment, description is abbreviate | omitted. In step S603, the generation unit 217 generates an assumed acquisition surface. The process related to the generation of the assumed acquisition surface is shown in the flowchart of FIG.

  In step S608, the generation unit 217 determines the position of the focal plane used when generating the assumed acquisition plane and the XY position (focus detection XY position) for acquiring the position. A process related to the determination of the focus detection XY position is shown in the flowchart of FIG.

  In step S611, the generation unit 217 sets the number of focus detection XY positions as in the first embodiment. In step S612, the generation unit 217 provisionally determines a tile for focus detection with respect to the Z stack image of the tile acquired in step S602. The tile to be provisionally determined is determined by the same method as in the first embodiment. In step S613, the generation unit 217 determines whether the sample 910 exists in the tile provisionally determined in step S612. If the sample 910 exists, the process proceeds to step S614, and if not, the process proceeds to step S612.

  In step S614, the generation unit 217 sets an in-tile focus detection XY position for performing position detection in the region where the sample 910 exists, with respect to the tile provisionally determined in step S612. Using the in-tile focus detection XY position and the stage XY position information when the tile is imaged, the focus detection XY position is obtained and recorded in the assumed acquisition plane generation table.

  In step S615, the generation unit 217 determines whether the focus detection XY position has been determined by the number of points acquired in step S611. If determined, the process proceeds to step S604, and if not determined, the process proceeds to step S609.

  In step S609, the generation unit 217 performs position detection at the focus detection XY position determined in step S608, and records the result in the assumed acquisition surface generation table. The position detection is performed using a patch as in the first embodiment. In step S610, the generation unit 217 calculates the assumed acquisition surface 901 based on the record of the assumed acquisition surface generation table by the same method as in the first embodiment.

  In step S604, the correction unit 216 has a three-dimensional range in a three-dimensional space in which the assumption acquisition plane 901 generated in step S610 has a width corresponding to the depth of field of the imaging optical unit 209 in the optical axis direction. Is generated as an assumed acquisition range. In step S605, the correction unit 216 extracts a Z layer image that falls within the assumed acquisition range from the Z stack image acquired for one tile. The process related to the extraction of the Z layer image is shown in the flowchart of FIG.

  In step S616, based on the focal plane position information at the time of imaging acquired in step S601 and the focal plane tilt parameter acquired in step S402, the correction unit 216 has a focal plane at an arbitrary position in each Z layer image. Position (Z layer focal position) is calculated. In step S617, the correction unit 216 compares the calculation result of step S616 with the assumed acquisition range, and determines whether there is a Z layer image in which the entire focal plane when each Z layer image is captured is within the assumed acquisition range. to decide. If it exists, the process proceeds to step S618, and if it does not exist, the process proceeds to step S619.

  In step S618, the corresponding Z layer image is extracted based on the comparison result in step S617. In step S619, the correction unit 216 captures an image of a portion that falls within the assumed acquisition range from the Z layer image in which the focal plane when each Z layer image is captured is within the assumed acquisition range (part Extract data of image 1).

  In step S620, the correction unit 216 is within the assumed acquisition range from the Z layer image (one above, one below, or both) of the Z layer image used for extracting the data of the partial image 1 in step S619. Data of the partial image (partial image 2) is extracted. If necessary, data of Z layer image partial images acquired at different positions in the optical axis direction is further extracted. At this time, in order to efficiently perform the synthesizing process in the subsequent step S621, it is preferable to extract so that a part of the partial image 1 and a part of the partial image 2 overlap.

  In step S621, the processing unit 223 combines the images extracted in steps S619 and S620. The joining process at the time of combining is performed by using the overlapping area between the partial images described above, and performing the same process as the joining process of tile images described in the first embodiment. Subsequent processing is performed assuming that this composite image corresponds to the Z layer image extracted in step S618.

  In step S606, the sub-control system 228 determines whether or not the extraction of the Z layer image in step S605 has been performed for all tiles based on the imaging conditions acquired in step S601. If extraction has been performed, the process proceeds to step S413, and if not, the process proceeds to step S607.

  In step S607, the sub-control system 228 refers to a tile for which a Z layer has not yet been extracted based on the imaging conditions acquired in step S601. Thereafter, the process of step S605 is performed on the referenced tile. In this embodiment, the Z layer extraction is performed for each tile in steps S605 to S607. However, step S605 may be performed in parallel (simultaneously for a plurality of tiles 1011). Since subsequent steps S413 and S414 are the same as those in the first embodiment, description thereof will be omitted.

  As described above, in this embodiment, a Z layer image that falls within the assumed acquisition range is extracted from a Z stack image captured in advance, and a large image is synthesized. Thereby, it is possible to efficiently acquire a high-quality image in which artifacts are suppressed.

(Other embodiments)
In the above-described embodiment, the apparatus 102 is incorporated in the computer 103 as a dedicated board. However, the same function may be realized by software executed on the computer 103.

  In the third embodiment, the Z stack image data acquired by the apparatus 501 may be stored in a server arranged on the network 504. In that case, a system configuration in which the device 502 and the computer 503 generate a color reproduction image using the Z stack image data in the server may be adopted.

  In the above-described embodiment, the relative position between the position of the focal plane and the subject is changed by changing the position of the stage 204 in the optical axis direction and the position in the direction perpendicular to the optical axis direction. However, the present invention is not limited to this, and the relative position may be changed by moving the imaging sensor and the imaging optical unit 209. Further, the relative position of the focal plane and the subject in the optical axis direction and the relative position in the direction perpendicular to the optical axis direction may be changed using different changing units.

  In the above-described embodiment, contrast is used as an image evaluation method for determining the position of the focal plane. The evaluation method is not limited to this, and an evaluation method used in a known autofocus algorithm such as a Tenenbaum Gradient method using a gradient vector of each pixel or an entropy method using an information amount of an image may be used.

  Although the focal plane position information at the time of capturing the tile image is recorded in the header portion of the image data, it may be stored as a separate file as link information on a storage device or a network.

  The object of the present invention may be realized by combining the above-described embodiments described above and the other embodiments described above. For example, in the first embodiment, the device 101 and the device 102 may be connected via a network as in the third embodiment. A configuration obtained by appropriately combining various techniques in each embodiment also belongs to the category of the present invention.

  Further, the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

209 Imaging optical unit 213 Imaging unit 217 Surface generation unit 216 Focus position correction unit, extraction unit 218 Position acquisition unit 219 Correction amount acquisition unit

Claims (18)

An image acquisition device that acquires an image of the subject by imaging the subject for each of a plurality of regions,
An imaging optical unit for imaging light from the subject;
An imaging unit that captures an image of the subject;
A changing unit that changes a relative position in the optical axis direction of the imaging optical unit between the imaging plane of the imaging unit and a focal plane conjugate with the subject;
A surface generation unit that generates an assumed acquisition surface using the position in the optical axis direction of the specimen in the subject at each of a plurality of positions of the subject;
In the first region, using the position of the focal plane in the optical axis direction when the second region is imaged at one or more positions in the second region adjacent to the first region. A position acquisition unit that acquires an expected position of the focal plane at one or a plurality of positions;
Based on the result of obtaining the difference between the position of the assumed acquisition surface in the optical axis direction and the expected position and comparing the difference with the depth of field of the imaging optical unit, the focal plane is assumed to be the assumed surface. An image acquisition apparatus comprising: a correction amount acquisition unit that acquires a correction amount for correcting the relative position so as to be within the range of the depth of field from an acquisition surface. The image acquisition according to claim 1, wherein the expected position is within the range of the depth of field from the position in the optical axis direction of the focal plane when the second region is imaged. apparatus. The position in the optical axis direction of the focal plane when the predicted position and the second area are imaged is acquired at the same point in the overlapping area of the first area and the second area. The image acquisition apparatus according to claim 1. The position acquisition unit acquires the predicted position using information regarding the position of the focal plane in the optical axis direction and the tilt of the focal plane when the second region is imaged. Item 4. The image acquisition device according to any one of Items 1 to 3. The difference is a difference between the position in the optical axis direction of the assumed acquisition surface at the one or more positions and the expected position at the one or more positions,
The correction amount acquisition unit compares the difference with the depth of field, and if the difference is equal to or less than the depth of field, the difference is used as the correction amount. 5. The image acquisition device according to any one of 4. If the difference is greater than the depth of field,
(I) The correction amount acquisition unit sets a change amount for changing the relative position in the optical axis direction and the direction perpendicular to the optical axis direction as the correction amount so that the difference is equal to or less than the depth of field. Or
(Ii) The imaging unit captures the first region at a plurality of different positions in the optical axis direction to acquire a plurality of images, and the correction amount acquisition unit acquires the assumed acquisition surface from the plurality of images. The image acquisition apparatus according to claim 5, wherein an image in which a difference between the two is less than the depth of field is extracted. 5. The difference according to claim 1, wherein the difference is acquired using a magnitude of an angle formed between an expected focal plane including a plurality of the expected positions and the assumed acquisition plane. Image acquisition device. An image acquisition device that acquires an image of the subject by imaging the subject for each of a plurality of regions,
An imaging optical unit for imaging light from the subject;
An imaging unit that captures an image of the subject;
A changing unit that changes a relative position in the optical axis direction of the imaging optical unit between the imaging plane of the imaging unit and a focal plane conjugate with the subject;
A surface generation unit that generates an assumed acquisition surface using positions in the optical axis direction of the specimen in the subject at each of a plurality of different positions of the subject;
From the plurality of images acquired by imaging the plurality of different positions in the optical axis direction of the first region by the imaging unit, the focal plane when each of the plurality of images is captured from the assumed acquisition plane. An image included in the range of depth of field of the imaging optical unit, and an image included in the range of depth of field from the second region adjacent to the first region, or the plurality of images An image acquisition apparatus comprising: an extraction unit that extracts a part. The extraction unit extracts an image when the entire focal plane when each of the plurality of images is captured is included in a range of the depth of field from the assumed acquisition plane, and extracts the entire focal plane If there is no image included in the range of depth of field from the assumed acquisition plane, a portion in which the focal plane is included in the range of depth of field from the assumed acquisition plane from the plurality of images. The image acquisition apparatus according to claim 8, wherein the image acquisition apparatus extracts and combines. The assumed acquisition plane is a plane including a straight line connecting two points within a range where the specimen in the subject exists.
2. Each of the two points is a position where the imaging unit and the specimen are conjugate, and the distance between the two points is longer than the length of each of the plurality of regions. The image acquisition apparatus as described in any one of thru | or 9. The image acquisition apparatus according to claim 10, wherein the two points are arranged at relatively the same position in each of the regions where the two points are arranged. The assumed acquisition plane is a plane including three or more points within a range where the specimen in the subject exists.
Each of the three or more points is a position where the imaging unit and the specimen are conjugate, and the distance between the three or more points is longer than the length of each of the plurality of regions. The image acquisition device according to any one of claims 1 to 9. The image acquisition apparatus according to claim 12, wherein the three points are arranged at relatively the same position in each of the regions where the three points are arranged. The position of the specimen in the optical axis direction is a position in which the surface generation unit corrects a variation in the position of the focal plane in the optical axis direction due to a stage parallelism using calibration data. The image acquisition device according to any one of claims 1 to 13. The positions of the specimen in the optical axis direction are obtained by acquiring the positions of the specimen in the optical axis direction within a range including a plurality of different positions of the subject, and the positions of the plurality of specimens in the optical axis direction. The image acquisition device according to claim 1, wherein the image acquisition device is a position acquired by statistically processing the image. An image of the subject is obtained by imaging the subject for each of a plurality of regions using an imaging device having an imaging optical unit that images light from the subject and an imaging unit that captures an image of the subject. An image acquisition method,
A surface generation step of generating an assumed acquisition surface using a position in the optical axis direction of the imaging optical unit of the specimen in the subject at each of a plurality of different positions of the subject;
In the first region, using the position of the focal plane in the optical axis direction when the second region is imaged at one or more positions in the second region adjacent to the first region. A position acquisition step of acquiring an expected position of a focal plane conjugate with an imaging surface of the imaging unit at one or a plurality of positions;
Based on the result of obtaining the difference between the position of the assumed acquisition surface in the optical axis direction and the expected position and comparing the difference with the depth of field of the imaging optical unit, the focal plane is assumed to be the assumed surface. A correction amount acquisition step of acquiring a correction amount for correcting the relative position so as to be within the range of the depth of field from the acquisition plane. An image of the subject is obtained by imaging the subject for each of a plurality of regions using an imaging device having an imaging optical unit that images light from the subject and an imaging unit that captures an image of the subject. An image acquisition method,
A surface generation step of generating an assumed acquisition surface using a position in the optical axis direction of the imaging optical unit of the specimen in the subject at each of a plurality of different positions of the subject;
From the plurality of images acquired by imaging the plurality of different positions in the optical axis direction of the first region by the imaging unit, the focal plane when each of the plurality of images is captured from the assumed acquisition plane. An image included in the range of depth of field of the imaging optical unit, and an image included in the range of depth of field from the second region adjacent to the first region, or the plurality of images An image acquisition method comprising: an extraction step of extracting a part.   A program for causing a computer to execute each step of the image acquisition method according to claim 16 or 17.

Images