JP2015156011A - Image acquisition device and method for controlling the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of efficiently executing processing for acquiring an image of a plurality of layers from a sample.

SOLUTION: An image acquisition device includes an optical system which condenses light from the sample, division means which divides an optical path from the optical system into a plurality of optical paths, a plurality of imaging means which have light-receiving surfaces on the plurality of optical paths respectively, control means which performs first processing and second processing by using the plurality of imaging means, and changing means which changes the interval of a plurality of reference surfaces which are surfaces on the side of the sample optically conjugate with the light-receiving surfaces of the plurality of imaging means. The changing means changes the interval of the plurality of reference surfaces between a case where the plurality of imaging means perform imaging for the first processing and a case where the plurality of imaging means perform imaging for the second processing.

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to an image acquisition apparatus, and more particularly to an image acquisition apparatus for acquiring a multilayer image of a sample at high speed.

  In the field of pathology and the like, as an alternative to an optical microscope, an image acquisition apparatus that acquires a digital image by imaging a sample has attracted attention. In this apparatus, a medical condition diagnosis by a doctor is made possible using the acquired image data. This type of apparatus is called a digital microscope system, a slide scanner, a virtual slide system, or the like. By digitizing a pathological diagnosis image, for example, it is possible to present various information useful for pathological diagnosis by grasping the shape of a cell, calculating the number of cells, and calculating the area ratio of cytoplasm and nucleus.

  On the other hand, since the cells are thick, the doctor needs to make a pathological diagnosis comprehensively as to how structures such as cell nuclei are distributed in the thickness direction (Z direction). For this reason, in an image acquisition apparatus having an optical system with a shallow depth of focus, such as a microscope, for example, a structure is detected while moving a stage on which a sample is placed in the Z direction, and a Z range in which an image is to be acquired (high Range). Then, images of a plurality of layers within the Z range are acquired while finely moving the height of the stage. From the plurality of images thus captured, an omnifocal image in which all regions are in focus and a three-dimensional image in which the distribution of structures in the thickness direction can be grasped are constructed. The image of each layer is also called a layer image, and the layer image group of a plurality of layers is also called a Z stack image.

  However, since it is unknown at which height the structure to be observed is distributed in the sample, it is necessary to search a wide area in the sample, and the search may take time. On the other hand, in order to acquire a high-resolution image in the height direction, it is necessary to control the movement of the stage height at a narrow pitch, and it takes more time to acquire an image of the target layer. In a device that acquires images of a large number of samples such as a slide scanner for pathological diagnosis, if time is required in this way, the image acquired in the first and the image acquired in the last will be affected by the change over time. Correlation cannot be ensured and may interfere with accurate comparative diagnosis. For example, in pathological diagnosis, diagnosis may be performed by acquiring images by changing a plurality of conditions such as the wavelength of light to be irradiated on the same sample and comparing them. At this time, it is difficult to determine whether the observed difference is a symptom observed due to a difference in irradiation conditions or an error that appears differently due to a change with time (for example, temperature fluctuation of the light source).

  In Patent Document 1, light from a subject is divided into a plurality of lights and received by a plurality of imaging elements whose arrangement positions are different from each other along the optical axis, whereby a plurality of focus positions are different from each other. An image signal recording apparatus capable of recording an image signal is disclosed. With this configuration, it is possible to reduce the time for acquiring a multi-layer image compared to the conventional technique.

  In Patent Document 2, the distance between the microscope and the object is sequentially changed, and the coarse adjustment means for obtaining the distance at which the contrast is maximized, and the vicinity of the distance obtained by the coarse adjustment means are finely moved to maximize the contrast. An image processing apparatus including fine adjustment means for further obtaining a distance is disclosed. If this method is used, the fine movement near the distance far from the distance where the contrast is maximum can be omitted, so the search time for the Z range in which an image is to be acquired can be reduced as compared with the conventional method.

However, none of the documents discloses a method for reducing the total processing time for a plurality of processes having different purposes such as acquisition of a multilayer image and search in a sample. In addition, it is possible to reduce the total processing time by separating the imaging system that acquires multilayer images and the imaging system that searches within the sample, and processing in parallel. This increases the cost and is not preferable.

  Another problem is a problem related to acquisition of a color image. Since most specimens to be pathologically diagnosed are stained, it is essential to obtain a color image when performing pathological diagnosis. However, it is difficult to achieve high definition in a configuration using an image sensor with a Bayer array. On the other hand, for high definition, there is a method of imaging while switching the color of the light source to R, G, B, but this configuration leads to an increase in the size of the apparatus and requires illumination and imaging three times. There is a drawback that it takes imaging time.

JP-A-10-257369 JP-A-8-021950

  This invention is made | formed in view of such a situation, and it aims at providing the technique which can perform efficiently the process which acquires the image of several layers from a sample. Another object of the present invention is to provide a technique capable of efficiently executing image acquisition according to a plurality of processes having different purposes with a single imaging system.

  According to a first aspect of the present invention, there is provided an optical system that collects light from a sample, a dividing unit that divides an optical path from the optical system into a plurality of optical paths, and a plurality of light receiving surfaces respectively on the plurality of optical paths. An imaging means; a control means for performing a first process and a second process using the plurality of imaging means; and a plurality of references that are surfaces on the sample side optically conjugate with the light receiving surfaces of the plurality of imaging means. Changing means for changing the interval between the surfaces, wherein the changing means performs imaging for the first processing and imaging for the second processing when the plurality of imaging means perform imaging for the first processing. The image acquisition apparatus is characterized in that the interval between the plurality of reference planes is changed.

  According to a second aspect of the present invention, there is provided an optical system that condenses light from a sample, a dividing unit that divides an optical path from the optical system into a plurality of optical paths, and a plurality of light receiving surfaces respectively on the plurality of optical paths. An image acquisition device control method comprising: an image acquisition device comprising: a step of setting an interval between the plurality of reference planes to a first interval; and the plurality of image pickup units set to the first interval. Performing the first process, setting the interval between the plurality of reference planes to a second interval smaller than the first interval, and the plurality of imaging means set to the second interval And a step of performing a second process using the image processing method.

  According to the present invention, it is possible to efficiently execute a process of acquiring a plurality of layers of images from a sample. Further, according to the present invention, it is possible to efficiently execute image acquisition according to a plurality of different purposes with a single imaging system.

1 is a schematic diagram of an image acquisition apparatus according to a first embodiment. The unit arrangement | positioning figure of an image acquisition apparatus. The functional block diagram of a control unit. The hardware block diagram of a control unit. The figure for demonstrating the relationship between a Z range search process and a multilayer image acquisition process. An example of an evaluation index profile. The flowchart showing the whole process of an image acquisition apparatus. The flowchart showing Z range search processing. The flowchart showing a multilayer image acquisition process. Schematic of the image acquisition apparatus which concerns on 2nd Embodiment. The unit arrangement | positioning figure of an image acquisition apparatus. The figure for demonstrating the relationship between a Z range search process and a color image acquisition process. An example of an evaluation index profile. The flowchart showing the whole process of an image acquisition apparatus. 5 is a flowchart showing color image acquisition processing. Schematic which shows the modification of the image acquisition apparatus which concerns on 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of this embodiment, a digital microscope (slide scanner) is presented as an image acquisition device, and a slide (also referred to as a preparation) is presented as a sample from which images are to be obtained. It is not limited to. Further, numerical values exemplified for embodying the description are not intended to limit the scope of the present invention unless otherwise specified.
In addition, in each figure, the same reference number is attached | subjected about the same member and the overlapping description is abbreviate | omitted.

<First Embodiment>
An image acquisition apparatus according to a first embodiment of the present invention will be described with reference to FIG.

(Configuration of image acquisition device)
FIG. 1 is a diagram illustrating a configuration of an image acquisition apparatus 100 according to the first embodiment. The configuration of the image acquisition device 100 will be described with reference to FIG. In the following description, the Z direction is defined as the optical axis direction of the objective lens 102 that is an optical system, and the XY direction is defined as a direction perpendicular to the optical axis. The Z direction also coincides with the height direction (thickness direction) of the sample.

  The image acquisition apparatus 100 includes an objective lens 102, an optical path dividing unit 103, imaging units 104A to 104D, imaging unit moving mechanisms 108A to 108D, a stage 105, a control unit 106, and a display unit 107. The objective lens 102 is an optical system that collects light (transmitted light or reflected light) from the slide 101 as a sample. The optical path splitting unit 103 is a splitting unit that splits the light beam from the objective lens 102 into four optical paths. The imaging units 104A to 104D are a plurality of imaging means arranged on the optical axes of the divided optical paths. The imaging unit moving mechanisms 108A to 108D are changing units that move the imaging units 104A to 104D in the optical axis direction, respectively. The stage 105 is a second changing unit that holds and moves the slide 101. The control unit 106 is control means for controlling the image acquisition apparatus 100 to generate display image data. The display unit 107 is display means for displaying a digital image. The movement of the XYZ position of the slide 101 is controlled by the stage 105, and four layers having different heights in the slide 101 are simultaneously imaged by the imaging units 104A to 104D via the objective lens 102 and the optical path dividing unit 103.

  The slide 101 is a preparation specimen in which a sample (a biological sample such as a tissue section) is placed on a slide glass and the sample is fixed with an encapsulant and a cover glass.

The objective lens 102 is composed of a combination of a lens and a mirror, and is held by a main body frame and a lens barrel (not shown). The objective lens 102 and the optical path dividing unit 103 constitute an imaging optical system for forming an optical image of the slide 101 on the light receiving surfaces of the imaging units 104A to 104D. In addition, the objective lens 102 enlarges the optical image of the slide 101 at a predetermined magnification, and projects it at the same magnification on any light receiving surface of the imaging units 104A to 104D. The depth of field of the objective lens 102 is very narrow, about 1 μm (micrometer) to several um.

  The optical path splitting unit 103 is an optical system that is held by a body frame (not shown) or a lens barrel of the objective lens 102 and divides the light beam from the objective lens 102 into four toward the imaging units 104A to 104D. The optical path dividing unit 103 is configured and arranged so that the entire field of view of the objective lens 102 is projected onto the light receiving surfaces of the imaging units 104A to 104D.

  The imaging units 104A to 104D are held by a body frame (not shown) or a lens barrel of the objective lens 102, and are configured by (two-dimensional) imaging elements such as a CCD and a CMOS sensor. By using a high pixel sensor, it is possible to collectively acquire a wide area and high spatial resolution image. In this embodiment, by combining a full-size CMOS of 6.4 um (micrometer) pitch with the objective lens 102 of optical 25.6 times, an image of 0.25 um / pixel from a 1.4 mm × 0.9 mm region. Realize batch acquisition. In this specification, the surface on the sample side (object side) that is optically conjugate with the light receiving surfaces of the imaging units 104A to 104D with respect to the objective lens 102 is referred to as a “reference surface”. By varying the positions of the four reference planes corresponding to the four imaging units 104A to 104D in the Z direction (optical axis direction), layers at different Z positions in the slide 101 can be imaged simultaneously. Acquisition of output data from the imaging units 104 </ b> A to 104 </ b> D is controlled by the control unit 106.

  The imaging unit moving mechanisms 108 </ b> A to 108 </ b> D move the imaging units 104 </ b> A to 104 </ b> D in the optical axis direction according to the control target value output from the control unit 106. The imaging unit moving mechanisms 108 </ b> A to 108 </ b> D only need to be able to be driven with a positioning accuracy of about the square of the optical magnification of the objective lens 102 with respect to the positioning accuracy of the Z stage described later. The moving mechanism includes a linear motor, a stage using a linear motion guide, a DC motor using a linear motion ball screw, a linear motor system driven by a pulse motor, a VCM, etc., a guide mechanism and a piezo using elastic deformation such as a leaf spring. A mechanism using an actuator can be used.

  By moving the imaging units 104A to 104D, the interval (pitch) in the Z direction between the four reference planes is changed, and a plurality of (four) layers at an arbitrary height can be simultaneously imaged. In the present embodiment, the interval between the four reference planes is changed between a Z range search process (first process) and a multilayer image acquisition process (second process) described later. In the present embodiment, the four reference planes are arranged at equal intervals. However, the intervals between the reference planes can be set at unequal intervals according to the sample or according to the purpose of image acquisition.

  In this embodiment, all the imaging units 104A to 104D are configured to move. However, a configuration in which one imaging unit is fixed (the imaging unit moving mechanism is not installed) and the relative position of the other imaging units is moved is also possible. Good. Thereby, the number of imaging unit moving mechanisms is reduced, and the configuration is simplified.

  In the present embodiment, as an example, the interval between the four reference planes is changed by moving the imaging units 104A to 104D in the optical axis direction. Not limited to this configuration, a configuration may be adopted in which a plurality of mirrors and relay lenses are arranged on each of the divided optical paths, and the optical path length of the divided optical path is changed by moving them to change the interval between the four reference planes. .

The stage 105 includes a holding unit that holds the slide 101, an XY stage that moves the holding unit in the XY direction according to a control target value output from the control unit 106, and a Z stage that moves the holding unit in the Z direction. Included (both not shown). XY stage is especially 25m
Those that can be driven over a wide range of m or more are preferable. The XY stage can be configured by a linear motor, a stage using a linear motion guide, a DC motor using a linear motion ball screw, a pulse motor, a linear motion system driven by a VCM, or the like. On the other hand, the Z stage is preferably one that can be driven with a positioning accuracy of 0.1 μm or less. The Z stage is a linear motor, a stage using a linear motion guide, a DC motor using a linear motion ball screw, a pulse motor, a linear motion system driven by a VCM, a mechanism using a leaf spring guide mechanism and a piezoelectric actuator, etc. Can be configured.

  By moving the XY stage, the XY relative position between the slide 101 and the objective lens 102 is changed, and a divided image of a desired region of the slide 101 can be acquired. Further, by repeatedly acquiring the divided images while continuously controlling the XY movement, the wide area of the slide 101 can be imaged. The XY movement order for acquiring images from a wide area is determined based on the XY acquisition target range. The XY acquisition target range is information that defines an XY range (expansion of a region) in a sample from which an image is to be acquired. The XY acquisition target range may be determined based on the XY shape information of the sample measured in advance by a preliminary measurement system (not shown), or may be based on an instruction from the user as necessary. By setting the XY acquisition target range, image data of an area necessary for pathological diagnosis can be selectively generated. For example, an area where a sample does not exist is deleted to reduce the capacity of the display image data, and the data Can be handled easily. Normally, the XY acquisition target area is determined so as to be equal to the detected area of the sample.

  On the other hand, when the Z stage moves, the Z relative positions of the slide 101 and the four reference planes are changed, and images of four layers having different heights (depths) in the slide 101 can be simultaneously acquired. Further, by repeatedly acquiring images while continuously controlling the Z movement, it is possible to acquire four or more layers of images for the same XY region. This process is called a multilayer image acquisition process. The Z movement order in the multilayer image acquisition process is determined based on the Z acquisition target range. The Z acquisition target range is information that defines a Z range (height range) in a sample from which a multilayer image is to be acquired, and is determined based on a result of a Z range search process to be described later. In the present embodiment, as an example, the Z relative position between the slide 101 and the four reference surfaces of the objective lens 102 is changed by moving the Z stage. However, the objective lens 102 or the imaging units 104A to 104D is used. It is good also as a structure which moves.

  The control unit 106 is configured by a general-purpose computer or workstation with high-speed arithmetic processing including a CPU, memory, hard disk, etc., a dedicated graphic board, or a combination thereof. The control unit 106 includes an interface for inputting / outputting control information and image data to / from the stage 105, the imaging units 104A to 104D, the imaging unit moving mechanisms 108A to 108D, and the display unit 107. Further, the control unit 106 may include an interface for changing the setting of the image acquisition apparatus 100 and an interface for inputting the position information and shape information of the sample. The function of the control unit 106 to be described later is realized by loading a program stored in a storage medium such as a hard disk into a memory and executing it by the CPU.

  The display unit 107 has a function of displaying an observation image suitable for pathological diagnosis based on the display image data generated by the image acquisition device 100. The display unit 107 can be configured by a monitor such as a CRT or a liquid crystal.

(Division of optical path)
FIG. 2 is a layout diagram of the objective lens 102, the optical path dividing unit 103, and the imaging units 104A to 104D for imaging four reference planes. The optical path splitting unit 103 includes three beam splitters 31 to 33 arranged on the image side of the objective lens 102.

  As shown in FIG. 2, the four reference surfaces of the objective lens 102 are OSa to OSd from the side close to the objective lens 102. The positions of the light receiving surfaces of the imaging units 104A to 104D are arranged so as to be optically conjugate with the positions of the reference surfaces OSa to OSd, respectively. That is, the light beam LFa from the reference surface OSa is reflected by the beam splitter 31, passes through the beam splitter 33, and forms an image on the light receiving surface of the imaging unit 104A. The light beam LFb from the reference surface OSb is reflected by the beam splitter 31 and the beam splitter 33, and forms an image on the light receiving surface of the imaging unit 104B. The light beam LFc from the reference plane OSc passes through the beam splitter 31 and the beam splitter 32 and forms an image on the light receiving surface of the imaging unit 104C. The light beam LFd from the reference surface OSd passes through the beam splitter 31, is reflected by the beam splitter 32, and forms an image on the light receiving surface of the imaging unit 104D. Each of the imaging units 104A to 104D outputs imaging data Da to Dd in response to a control command from the control unit 106, respectively.

  By making the Z position of a desired layer in the sample coincide with the uppermost reference plane OSa by the stage 105, imaging data Da focused on that layer can be obtained from the imaging unit 104A. At the same time, three layers of imaging data Db to Dd are obtained from the imaging units 104B to 104D, respectively, which are separated from the layer by a predetermined distance (distance from the reference plane OSa of the reference planes OSb to OSd).

  As will be described later, in the present embodiment, a Z range search process (first process) for searching a sample and determining a Z range in which an image is to be acquired, and a multilayer image acquisition for acquiring a multilayer image for the Z range The process (second process) is executed. In the Z range search process, it is preferable that the interval between the reference planes OSa to OSd be larger (for example, several times) than the depth of field of the objective lens 102. By doing so, it is possible to reduce the number of movements of the Z stage. On the other hand, in the multilayer image acquisition process, it is preferable that the interval between the reference planes OSa to OSd be equal to or smaller than the depth of field of the objective lens 102. By doing so, it is possible to ensure that an image focused on the structure in the sample is always obtained by any of the imaging units, and it is possible to acquire a multi-resolution image with high resolution in the height direction. In the present embodiment, the interval (first interval) between the reference planes OSa to OSd during the Z range search process is set to four times the depth of field of the objective lens 102, and the reference planes OSa to OSd during the multilayer image acquisition process are set. The interval (second interval) is made equal to the depth of field of the objective lens 102.

  In the present embodiment, as an example, four layers can be simultaneously imaged by three beam splitters and four imaging units. However, the present invention is not limited to this. For example, two layers can be simultaneously imaged by one beam splitter and two imaging units. Or it can be set as the structure which images three layers simultaneously with one dichroic mirror and three imaging units. A configuration in which five or more layers are simultaneously imaged can also be employed. Thus, it can be determined as appropriate depending on the depth of focus of the objective lens 102 and the shape and configuration of the optical path dividing unit 103.

(Function of control unit)
FIG. 3 is a functional block diagram of the control unit 106. As shown in FIG. 3, the control unit 106 includes a main control unit 60, a stage position control unit 61, an imaging unit position control unit 62, an imaging control unit 63, and a data processing unit 64.

The main control unit 60 controls the operation of each unit constituting the image acquisition apparatus 100 in an integrated manner. For example, the main control unit 60 acquires a multi-layer image of the sample, generates omnifocal image data, three-dimensional image data, or composite image data thereof, and performs synchronous control for outputting as display image data. In addition, the main control unit 60 adjusts each part of the image acquisition apparatus 100 associated with sample image acquisition such as dimming control of a light source (not shown) and switching of various optical elements, and notifies the user of the state of each part as appropriate. Make it possible.

  The stage position control unit 61 controls the XY movement and the Z movement of the stage 105 via the output interface, and sends a desired layer in a desired area of the slide 101 to any one of the reference planes OSa to OSd. Further, the stage position control unit 61 acquires the XYZ position coordinates of the stage 105 via the input interface.

  The stage position control unit 61 further determines the Z movement order (control target value for moving the stage 105 Z) during the Z range search process and the multilayer image acquisition process. In the present embodiment, after the stage 105 is first moved so that the upper end (or lower end) of the reference plane is aligned with the upper end (or lower end) of the search range or the Z acquisition target range, the same direction (downward or The stage 105 is shifted and moved upward. As a result, stage movement and alignment can be made more efficient, and the time can be reduced. Here, the movement amount (change amount) in one step may be changed between the Z range search process and the multilayer image acquisition process. Preferably, the amount of movement in one step is set to 4 times the interval between the reference planes. As a result, the search range or the Z acquisition target range can be imaged at equal intervals, and efficient image acquisition is possible. In the case of the present embodiment, assuming that the depth of field of the objective lens 102 is d, the amount of movement in one step is set to 16 × d in the Z range search process and 4 × d in the multilayer image acquisition process. .

  The imaging unit position control unit 62 controls the movement of the imaging units 104A to 104D in the optical axis direction by the imaging unit moving mechanisms 108A to 108D via the output interface, and changes the interval (relative distance) between the reference planes OSa to OSd. To do. In this embodiment, the reference plane interval is set to 4 × d during the Z range search process, and the reference plane interval is set to d during the multilayer image acquisition process. Further, the imaging unit position control unit 62 acquires the coordinates in the optical axis direction of the imaging units 104A to 104D via the input interface, and calculates the Z positions of the reference planes OSa to OSd. Here, the Z positions of the reference planes OSa to OSd may be calculated based on the optical path length from the imaging units 104A to 104D to the image side focal position of the objective lens 102 and the optical magnification of the objective lens 102.

  The imaging control unit 63 controls the imaging by the imaging units 104A to 104D via the input / output interface in synchronization with the completion of the movement of the stage 105 and the imaging units 104A to 104D, and inputs the imaging data groups Da to Dd. At the time of the Z range search process, since imaging data with high spatial resolution is not required, the imaging units 104A to 104D may be controlled to thin out the imaging data. This can speed up the Z range search process. In addition, the imaging control unit 63 acquires position information in association with imaging data. The position information is information indicating which divided area (XY position) and which layer (Z position) of the slide 101 each image pickup unit has imaged. For example, it can be acquired from the XYZ position coordinates as the control target of the stage 105, the XYZ position coordinates after the movement is completed, and the Z position of the reference plane. Moreover, as a method of associating, for example, the same header information may be added to the imaging data and the position information data. The header information is preferably unique information such as a time stamp. Alternatively, position information to be associated may be included in the header information of the imaging data. By referring to this position information, it is possible to identify which layer (Z position) of which divided area (XY position) of the slide 101 is the imaging data. In the following description, the XY position and the Z position on the slide 101 represented by the position information associated with the imaging data are simply referred to as the XY position and the Z position of the imaging data.

In the Z range search process, the data processing unit 64 receives the imaging data groups Da to Dd via the imaging control unit 63 and calculates the Z acquisition target range. For example, the data processing unit 64 estimates (detects) the Z position at which the image with the maximum contrast is obtained using the obtained imaging data group, and acquires the Z based on the Z position (including the Z position). It is good to set the target range. As a method for detecting a Z position where a high-contrast image can be obtained, a passive autofocus technique in a camera can be used.

  Specifically, the data processing unit 64 of the present embodiment calculates, for each imaging data, a luminance differential value or a difference value with a neighboring pixel for each pixel, and sums the values for all pixels to evaluate the index. Is calculated. This evaluation index represents the contrast of the image. Next, the data processing unit 64 creates an evaluation index profile (change in the evaluation index with respect to the Z position (contrast distribution)) from the evaluation index and the Z position of each imaging data, and determines the peak Z position where the evaluation index shows a peak. Ask. The Z acquisition target range is determined based on the peak Z position. As a determination method of the Z acquisition target range, for example, a predetermined Z range centered on the peak Z position may be set as the Z acquisition target range, or the Z acquisition target range is set based on the half width of the evaluation index profile. Also good. Alternatively, a unit region for calculating the evaluation index is set (for example, 1/16 of the entire region), the peak Z position is obtained for each region, and the range from the minimum value to the maximum value of the peak Z position is set as the Z acquisition target range.

  In addition, the data processing unit 64 synchronizes with the completion of image acquisition within the Z acquisition target range, and based on the imaging data group and the Z position information thereof, the omnifocal image data focused at all points in the divided region. And / or 3D image data that can grasp the shape distribution in the thickness direction is generated. For the omnifocal image data, for example, for each pixel in the divided image, a luminance differential value or a difference value with a neighboring pixel is calculated, and this value is used as an evaluation value. Then, it is compared with the evaluation value of other image data acquired from within the Z acquisition target range, and the pixel having the highest evaluation value is regarded as the focus data at the pixel position. The three-dimensional image data can be constructed based on the Z coordinate of each pixel regarded as the focal point. Alternatively, the three-dimensional image data may be generated by mapping each pixel of the imaging data group to the voxel space. Alternatively, the display data may be an image associated with a plurality of layers of image data and XYZ positions (Z stack image) so that the images can be switched to and displayed at the XYZ positions designated by the user.

  Further, the data processing unit 64 generates composite image data based on the omnifocal image data and / or 3D image data for each divided region and its XY position in synchronization with the completion of image acquisition within the XY acquisition target range. To do. The composite image data is image data representing the entire range of the XY acquisition target range of the slide 101, and is generated by aligning and connecting the image data of each divided region. As a connection method, any method may be used such as joining (tiling), superimposing (blending overlapping portions between adjacent images), and smoothly connecting by interpolation processing.

  In this way, the data processing unit 64 generates display image data (observation image data) such as omnifocal image data, three-dimensional image data, and composite image data using the imaging data group. The data processing unit 64 outputs these display image data via the output interface, and allows the display unit 107 to display them. Note that the generation order of the omnifocal image data or the three-dimensional image data and the generation order of the composite image data may be reversed. That is, it is possible to generate the omnifocal image data and the three-dimensional image data from the composite image data of a plurality of layers after connecting the imaging data of each divided region to generate the composite image data of each layer. The display image data generation method described above is merely an example, and other image processing methods may be used. In addition, the data processing unit 64 may generate display image data other than the above, for example, image data in which image features useful for observation or diagnosis are emphasized or extracted, image data in which the line-of-sight direction (observation direction) is changed, and the like. it can.

(Hardware configuration of control unit)
FIG. 4 is a block diagram showing a hardware configuration of the control unit 106.

As the control unit 106, for example, a personal computer (PC) 200 is used. The PC 200 includes a CPU (Central Processing Unit) 201 and an HDD (Hard Disk D).
rive) 202, a RAM (Random Access Memory) 203, an input / output I / F 204, and a bus 205 for connecting them together.

  The CPU 201 accesses the RAM 203 and the like as necessary, and comprehensively controls the entire PC 200 while performing various arithmetic processes. The HDD 202 is an auxiliary storage device that stores programs to be executed by the CPU 201 and various parameters. Imaging data acquired from the imaging unit, generated omnifocal image data, three-dimensional image data, composite image data, display data, and the like can be stored in the HDD 202. The RAM 203 is used as a work area for the CPU 201 and temporarily holds various programs being executed and various data to be processed in the control unit 106.

The input / output I / F 204 includes optical communication, Gigabit Ethernet (registered trademark), USB (Universal Serial Bus), and PCIe (Peripheral Component Interconnect Express) (registered trademark).
, A general-purpose interface using DVI (Digital Visual Interface) or the like. The input / output I / F 204 is connected to a stage 105, imaging units 104A to 104D, imaging unit moving mechanisms 108A to 108D, a display unit 107, an input device (not shown), and the like. An image server for storing the generated display data and various data can be connected to the input / output I / F 204. In FIG. 1, it is assumed that the display unit 107 is connected as an external device. However, the control unit 106 and the display unit 107 may be configured as a single device using a PC integrated with the display device. The input device is, for example, a pointing device such as a mouse, a keyboard, a touch panel, and other operation input devices. A device that doubles as the display unit 107 and an input device such as a touch panel display can also be used.

(Relationship between Z range search processing and multilayer image acquisition processing)
(1) Z Range Search Process FIG. 5 is a diagram showing the difference in the interval between the four reference planes in the Z range search process and the multilayer image acquisition process. The horizontal axis indicates time t.

  T0 represents a state when the divided area to be acquired is sent into the field of view of the objective lens 102 by the XY stage and the Z range search process is started. At this time, the interval between the reference planes OSa to OSd is set wide (for example, four times the depth of field of the objective lens 102). With this setting, four layers are simultaneously imaged at a rough interval. In the first search, the stage 105 is controlled so that the upper end of the sample (the position above which no layer for which an image is to be acquired exists, for example, the upper surface of the cover glass) coincides with the Z position of the reference plane OSa. In this state, the uppermost layer LT and the lowermost layer LB of the Z acquisition target range are determined based on the output imaging data groups Da to Dd.

  A method of determining the Z acquisition target range will be described with reference to FIG. FIG. 6 is an evaluation index profile calculated by the data processing unit 64 described above, with the horizontal axis representing the Z position and the vertical axis representing the evaluation index. Evaluation indexes calculated from the imaging data groups Da to Dd are DCa to DCd, respectively. From this profile, the peak Z position ZP at which the evaluation index reaches a peak is estimated. As a peak estimation method, a method of estimating a peak by mapping a predetermined profile shape model to DCa to DCd as shown in FIG. 6 may be used, or the maximum one of DCa to DCd is simply selected as a peak. The method is fine. Then, a predetermined Z range centered on the estimated peak Z position ZP is set as a Z acquisition target range, and the Z position ZT of the uppermost layer LT and the Z position ZB of the lowermost layer LB are determined. The peak Z position ZP, Z position ZT, and ZB may be obtained as relative values from the upper surface of the Z stage.

When the peak cannot be estimated from the evaluation index profile, the structure to be observed is not included in the range of the reference planes OSa to OSd, or the structure to be observed extends beyond the range of the reference planes OSa to OSd. After moving the Z position of the stage 105 by one step, the same processing as described above may be executed. At this time, it is desirable to estimate the peak using DCa to DCd before moving the Z position of the stage 105 by one step.

  Here, the peak may not be estimated even if the Z range search process is performed from the upper end to the lower end of the sample. The upper end of the sample is a position above which no layer for image acquisition is present, for example, the upper surface of the cover glass. Further, the lower end of the sample is a position where there is no layer from which an image is to be acquired, for example, the upper surface of the slide glass. Cases where the peak cannot be estimated include a case where the evaluation indices are all small values, a case where the evaluation indices are all large values, and a case where the evaluation indices change discontinuously. For example, if there is no evaluation index that exceeds a certain threshold value (the lower limit value regarded as a peak), it can be determined that all of the evaluation indices are small, and conversely if all or most of the evaluation indices exceed the threshold value , It can be determined that both evaluation indices are large. The case where the evaluation index changes discontinuously is a case where only one evaluation index protrudes and is large, or a case where two or more peaks appear. These can be determined, for example, by evaluating the magnitude of the difference (slope of the evaluation index) and the change in the sign from the adjacent evaluation index.

  If the peak cannot be estimated even if the Z range search process is performed from the upper end to the lower end of the sample as described above, the interval between the reference planes OSa to OSd is searched for the first time when the peak position cannot be estimated (first process). ) Change the time and re-search (second process). For example, when the change in the evaluation index is not continuous, the search may be too rough for the sample, and the peak position may be difficult to estimate. In this case, the interval between the reference planes OSa to OSd is narrowed, and the stage 105 is controlled so that the reference planes OSa to OSd are arranged around the Z position that shows the maximum value in the initial evaluation index profile. Then, imaging is performed again by the imaging units 104A to 104D, and peak estimation is executed from the evaluation index profile. According to this method, based on the first search result, the vicinity where the possibility of the peak position is high is re-searched, so that the peak position (Z position where the structure to be observed exists) can be accurately detected. become.

  Alternatively, the re-search process may be performed by setting the NA (numerical aperture) of the objective lens 102 to a different value. For example, when the evaluation indices are all small values, the reference planes OSa to OSd and the peak positions are separated from each other at any stage position, and a blurred image is incident on any of the imaging units 104A to 104D. The position may be difficult to estimate. In this case, by setting to a low NA (wide focal depth), it is possible to achieve a state similar to that in which the intervals between the reference surfaces OSa to OSd are relatively narrow. In this way, the blur of the image incident on the imaging unit corresponding to the reference plane relatively close to the peak position can be reduced, and the calculated evaluation index value can be increased. On the other hand, when the evaluation indices are all large values, any of the reference planes OSa to OSd is sufficiently close to the peak position, and no blurred image is incident on any of the imaging units 104A to 104D, and the peak position is estimated. May be difficult to do. In this case, by setting to a high NA (narrow focal depth), it is possible to obtain a state similar to the case where the intervals between the reference planes OSa to OSd are relatively wide. In this way, it is possible to increase the blur of the image incident on the imaging unit corresponding to the reference plane that is relatively far from the peak position, and to reduce the value of the calculated evaluation index. As a configuration for adjusting the NA of the objective lens 102, an NA diaphragm in which a plurality of field-shielding plates with different openings are arranged according to the application, an iris diaphragm including a plurality of field-shielding blades, or the like can be used.

(2) Multilayer image acquisition process T1 represents a state when the multilayer image acquisition process of the Z acquisition target range determined in the Z range search process is started. At this time, the interval between the reference planes OSa to OSd is set to be the same as (or less than) the depth of field of the objective lens 102. In addition, the uppermost layer LT in the Z acquisition target range is the reference plane O.
The stage 105 is controlled so as to coincide with the Z position of Sa. In this state, the range from the uppermost layer LT in the Z acquisition target range to the layer corresponding to the Z position of the reference plane OSd is imaged by the imaging units 104A to 104D. In this state, when the lowest layer LB within the Z acquisition target range is out of any depth of field of the reference planes OSa to OSd, the Z position of the stage 105 is moved by one step to acquire an image. repeat.

(Image acquisition method)
The image acquisition method of the image acquisition apparatus 100 in the present embodiment will be described with reference to the flowchart shown in FIG.

  First, in step S601, the main control unit 60 acquires processing conditions (such as an XY acquisition target range) of the image acquisition apparatus 100. Further, the main control unit 60 calculates the XY movement order (XY stage control target value table) of the stage based on the XY acquisition target range. In this table, control target values of the XY stage for moving each divided area within the field of view of the objective lens 102 in order to sequentially capture a plurality of divided areas on the slide 101 are described. Note that the size of each divided area corresponds to the size of the effective imaging area of the imaging unit projected onto the object side of the objective lens 102.

  In the next step S602, the main control unit 60 moves the stage position control unit 61 based on the XY movement order (for example, by reading the target value described in the top row from the XY stage control target value table). The stage 105 is controlled via Further, the XY position coordinates of the stage 105 after movement are acquired. Further, the main control unit 60 controls the imaging unit moving mechanisms 108A to 108D via the imaging unit position control unit 62 to move the imaging units 104A to 104D in the optical axis direction. Then, the interval between the reference planes OSa to OSd is set to the first interval (for example, four times the depth of field of the objective lens 102) during the Z range search process.

  In the next step S603, the main control unit 60 executes a Z range search process via the stage position control unit 61, the imaging control unit 63, and the data processing unit 64, and determines a Z acquisition target range. Details of this step will be described later with reference to FIG.

  In the next step S604, the main control unit 60 controls the imaging unit moving mechanisms 108A to 108D via the imaging unit position control unit 62 to move the imaging units 104A to 104D in the optical axis direction. Then, the interval between the reference planes OSa to OSd is set to the second interval (for example, the depth of field of the objective lens 102 or less) at the time of the multilayer image acquisition process.

  In the next step S605, the main control unit 60 executes a multilayer image acquisition process via the stage position control unit 61, the imaging control unit 63, and the data processing unit 64, and sets a plurality of observation target layers within the Z acquisition target range. Get an image. Details of this step will be described later with reference to FIG.

  In the next step S606, the main control unit 60 determines whether image acquisition within the XY acquisition target range has been completed (has been executed up to the last row of the XY stage control target value table). If it is determined that the process has been completed, the process proceeds to step S607. On the other hand, if it is determined that the process is incomplete, the process proceeds to step S602, where one line read from the XY stage control target value table is advanced, and the control of the stage 105 via the stage position control unit 61 is continued.

  In the next step S607, the main control unit 60 generates and controls display image data via the data processing unit 64 and completes image acquisition.

(Z range search process)
Details of the Z range search processing will be described with reference to the flowchart shown in FIG.

  First, in step S <b> 611, the stage position control unit 61 determines a target position for the Z movement of the stage 105 and controls the stage 105. Further, the stage position control unit 61 acquires the Z position coordinates of the stage 105 after the movement.

  When the stage movement and the imaging are repeated a plurality of times, the content of the determination process of the target position for the Z movement in step S611 differs between the first time and the second and subsequent times. In the first time, the Z stage is moved so that the reference plane is closer to the upper end or lower end of the search range. For example, the stage position control unit 61 compares the difference between the Z position of the upper end of the sample and the reference plane OSa and the difference of the Z position of the lower end of the sample and the reference plane OSd. The upper end of the sample is a position above which no layer for image acquisition is present, for example, the upper surface of the cover glass. Further, the lower end of the sample is a position where there is no layer from which an image is to be acquired, for example, the upper surface of the slide glass. The positions of the upper surface of the cover glass and the upper surface of the slide glass are prepared in advance in a predetermined value (for example, the maximum value of the upper surface of the cover glass and the lowest value of the upper surface of the slide glass, which are supposed to be observed), or prepared separately. The measured value obtained by the measuring means, the value designated by the user, etc. may be acquired at the same time as the processing conditions in step S601. If the former is small, the Z stage is moved so that the upper end of the sample and the Z position of the reference plane OSa coincide with each other, and the second and subsequent Z stage movement directions are set to “up”. When the latter is small, the Z stage is moved so that the lower end of the sample and the Z position of the reference surface OSd coincide with each other, and the movement direction of the second and subsequent Z stages is set to “down”. In the second and subsequent steps S611, the Z stage is shifted by one step (four times the interval between the reference planes OSa to OSd) in the set moving direction.

  In the present embodiment, the determination as described above is performed, and the first Z position and the second and subsequent stage movement directions are determined, whereby the amount of stage movement can be minimized and the time required for the Z range search process can be shortened. be able to. However, when the stage moving time does not matter, the determination may be performed without using such a determination, and the control may be performed in a predetermined moving direction or a moving direction instructed by the user.

  In the next step S612, the imaging control unit 63 controls imaging by the imaging units 104A to 104D, and acquires imaging data groups Da to Dd. Further, the imaging control unit 63 acquires positional information of the acquired image in association with the imaging data. Here, at the time of the Z range search process, imaging data with high spatial resolution is not required. Therefore, it is possible to increase the speed by controlling the imaging units 104A to 104D to thin out the imaging data.

  In the next step S613, the data processing unit 64 calculates evaluation indices DCa to DCd from the imaging data groups Da to Dd. Further, the data processing unit 64 creates a profile of the evaluation index from the evaluation index and the Z position information on the slide 101 associated with the imaging data, and the evaluation index shows a peak (based on the Z stage top surface). Estimate the Z position. A Z acquisition target range is determined based on the peak Z position.

  In the next step S614, the main control unit 60 determines whether or not the Z range search is completed. As a determination method, it is determined whether or not the peak Z position (based on the upper surface of the Z stage) at which the evaluation index is a peak can be estimated. If it is determined that the process has been completed, the Z range search process is terminated. On the other hand, if it is determined that it is incomplete, the process proceeds to step S611, and the stage 105 is controlled via the stage position control unit 61. Here, when the peak Z position cannot be estimated and the sample reaches the upper end or the lower end of the sample, the above-described re-search process may be performed.

(Multilayer image acquisition processing)
Details of the multilayer image acquisition processing will be described with reference to the flowchart shown in FIG.

  First, in step S <b> 621, the stage position control unit 61 determines a target position for Z movement of the stage 105 and controls the stage 105. Further, the stage position control unit 61 acquires the Z position coordinates of the stage 105 after the movement.

  When stage movement and imaging are repeated a plurality of times, the content of the Z movement target position determination process in step S621 differs between the first time and the second and subsequent times. In the first time, the Z stage is moved so that the reference plane is closer to the upper end (uppermost layer LT) or lower end (lowermost layer LB) of the Z acquisition target range. For example, the stage position control unit 61 compares the difference between the Z positions of the uppermost layer LT and the reference plane OSa in the Z acquisition target range and the difference between the Z positions of the lowermost layer LB and the reference plane OSd in the Z acquisition target range. . When the former is small, the Z stage is moved so that the Z positions of the uppermost layer LT and the reference surface OSa coincide with each other, and the second and subsequent Z stage moving directions are set to “up”. When the latter is small, the Z stage is moved so that the Z positions of the lowest layer LB and the reference plane OSd coincide with each other, and the movement direction of the second and subsequent Z stages is set to “down”. In step S621 after the second time, the Z stage is shifted by one step (four times the interval between the reference planes OSa to OSd) in the set moving direction.

  In the present embodiment, the determination as described above is performed, and the first Z position and the second and subsequent stage movement directions are determined, whereby the amount of stage movement can be minimized and the time required for multilayer image acquisition processing can be shortened. be able to. However, when the stage moving time does not matter, the determination may be performed without using such a determination, and the control may be performed in a predetermined moving direction or a moving direction instructed by the user.

  In the next step S622, the imaging control unit 63 controls imaging by the imaging units 104A to 104D, and inputs imaging data groups Da to Dd. Further, the imaging control unit 63 acquires positional information of the acquired image in association with the imaging data.

  In the next step S623, the main control unit 60 determines whether image acquisition within the Z acquisition target range is completed. As a determination method, the Z position of the lowest layer LB in the Z acquisition target range (when the movement direction of the second and subsequent Z stages is “up”) or the Z position of the uppermost layer LB in the Z acquisition target range (second time) Judgment is made based on whether or not the subsequent Z stage moving direction is “down”) is within the depth of field of any of the reference planes OSa to OSd. If it is determined that the process has been completed, the process proceeds to step S624. On the other hand, if it is determined that it is incomplete, the process proceeds to step S621, and the stage 105 is controlled via the stage position control unit 61.

  In the next step S624, the main control unit 60, through the data processing unit 64, based on the imaging data group and the Z position information thereof, the omnifocal image data and / Alternatively, three-dimensional image data that can grasp the shape distribution in the thickness direction is generated. Then, the multilayer image acquisition process within the Z acquisition target range is completed.

(Advantages of this embodiment)
According to the configuration of the present embodiment described above, the reference planes of the plurality of imaging units are at different heights (Z positions) in the sample, and the interval between the reference planes is changed according to the target processing. be able to. As a result, when it is necessary to observe (analyze) a wide Z range in the sample, for example, as in the Z range search process, a setting that emphasizes efficiency (reduction of processing time) such as increasing the interval between the reference planes. Can be. In addition, when it is necessary to observe (analyze) a limited Z range in the sample with high resolution as in the multi-layer image acquisition process, the interval between the reference planes can be narrowed to set the image quality as important. As described above, the interval between the reference planes is adaptively controlled in accordance with the target process, whereby the efficiency of the process as a whole and the reduction of the processing time can be achieved. In addition, since a single imaging system can flexibly cope with different purposes such as Z range search processing and multilayer image acquisition processing, the entire apparatus can be reduced in size and cost.

Second Embodiment
Next, an image acquisition apparatus according to a second embodiment of the present invention will be described. In the first embodiment, the reference plane interval is changed between the Z range search process and the multilayer image acquisition process. In the second embodiment, the reference plane interval is changed between the Z range search process and the color image acquisition process. change.

(Configuration of image acquisition device)
FIG. 10 is a diagram illustrating a configuration of an image acquisition apparatus 300 according to the second embodiment. The configuration of the image acquisition device 300 will be described based on FIG. In the following description, the Z direction is defined as the optical axis direction of the imaging optical system 302 and the XY direction is defined as the direction perpendicular to the optical axis. The Z direction also coincides with the height direction (thickness direction) of the sample.

  The image acquisition apparatus 300 includes an imaging optical system 302, a light source 309, an optical path color division unit 303, imaging units 304A to 304C, imaging unit moving mechanisms 308A to 308C, a stage 305, a control unit 306, and a display unit 307. The imaging optical system 302 is an optical system that collects light incident from the slide 301 as a sample. The light source 309 is illumination that applies light to the slide 301. The optical path color splitting unit 303 is an optical member having functions of optical path splitting and color splitting, and splitting means for splitting the light beam from the imaging optical system 302 into three optical paths and colors. The imaging units 304A to 304C are a plurality of imaging units disposed on the optical axes of the divided optical paths. The imaging unit moving mechanisms 308A to 308C are changing units that move the imaging units 304A to 304C in the optical axis direction, respectively. The stage 305 is a second changing unit that holds and moves the slide 301. The control unit 306 is control means for controlling the image acquisition device 300 to generate display image data. The display unit 307 is display means for displaying a digital image.

  The position of the slide 301 is moved and controlled by the stage 305, and the same or different layers of the slide 301 are simultaneously imaged by the imaging units 304A to 304C via the imaging optical system 302 and the optical path color dividing unit 303. By the optical path color division unit 303, the imaging units 304A, 304B, and 304C capture images of different colors. In this embodiment, as an example, the imaging unit 304A is set to capture R, the imaging unit 304B is set to G, and the imaging unit 304C is set to capture B. That is, the image acquisition apparatus 300 according to the present embodiment can acquire a single layer image of three layers by one shot (one exposure and imaging), or can acquire an RGB high-definition color image of one layer. You can also.

  The slide 301 is an image acquisition target of the image acquisition apparatus 300 of the present embodiment, and includes a cover glass, a sample, and a slide glass. Specifically, a sample (such as a biological sample such as a tissue section) placed on a slide glass is sealed with a cover glass and an encapsulant.

  The imaging optical system 302 is an optical system composed of a combination of optical components such as an objective lens, an imaging lens, and a mirror, and is held by a main body frame and a lens barrel (not shown). The imaging optical system 302 forms an image of the slide 301 on the light receiving surfaces of the imaging units 304A to 304C in combination with the optical path color dividing unit 303. In addition, the imaging optical system 302 enlarges the optical image of the slide 301 at a predetermined magnification, and projects the light image on any light receiving surface of the imaging units 304A to 304C at the same magnification. Further, the imaging optical system 302 is arranged such that three reference surfaces perpendicular to the optical axis and the light receiving surfaces of the imaging units 304A to 304C are optically conjugate with each other, and the object side corresponds to the reference surface. The image side corresponds to the light receiving surface (imaging surface).

  The optical path color division unit 303 is held by a body frame (not shown) or a lens barrel of the imaging optical system 302, and divides the optical path incident from the imaging optical system 302 into three toward the imaging units 304A to 304C. It is an optical system. For the optical path color dividing unit 303, for example, a dichroic prism or the like can be applied. The optical path color division unit 303 is configured and arranged so that the entire field of view of the imaging optical system 302 is projected onto the light receiving surfaces of the imaging units 304A to 304C.

  Here, it is important that a plurality of pairs of reference planes (object planes) and light receiving planes (imaging planes) having a conjugate relationship exist for each color and discretely (spaced apart). In the image acquisition apparatus 300 of this embodiment, this is realized by an optical system composed of a combination of the imaging optical system 302 and the optical path color dividing unit 303, but the present invention is not limited to this optical system. For example, as shown in FIG. 16, the imaging lenses 310 </ b> A to 310 </ b> C may be arranged on each optical path subsequent to the optical path color dividing unit 303. Here, the light split on the optical path color splitting unit 303 falls within a desired incident angle range on the color splitting surface, and each of the image pickup units 304A to 304C acquires an image for each color.

  The imaging units 304A to 304C are held by a body frame (not shown) or a lens barrel of the imaging optical system 302, and are configured by (two-dimensional) imaging elements such as a CCD and a CMOS sensor. By using a high pixel sensor, it is possible to collect high spatial resolution images from a wide area. Further, the imaging units 304A to 304C are configured and arranged so that the entire field of view of the imaging optical system 302 is projected. A surface on the sample side (object side) at a position optically conjugate with the light receiving surfaces of the imaging units 304A to 304C is referred to as a “reference surface”. By matching the positions of the three reference planes corresponding to the three imaging units 304A to 304C in the Z direction (optical axis direction), an R image, a G image, and a B image of a certain layer in the slide 301 can be simultaneously captured. On the other hand, by varying the positions of the three reference planes in the Z direction, images of three layers at different Z positions in the slide 301 can be taken simultaneously. Capture of output data from the imaging units 304 </ b> A to 304 </ b> C is controlled by the control unit 306.

  The imaging unit moving mechanisms 308A to 308C move the imaging units 304A to 304C in the optical axis direction according to the control target value output from the control unit 306, respectively. The imaging unit moving mechanisms 308 </ b> A to 308 </ b> C only need to be able to be driven with a positioning accuracy of about the square of the optical magnification of the imaging optical system 302 with respect to the positioning accuracy of the Z stage described later. The moving mechanism includes a linear motor, a stage using a linear motion guide, a DC motor using a linear motion ball screw, a pulse motor, a linear motion system driven by a VCM, a guide mechanism using elastic deformation of a leaf spring, and the like. A mechanism using a piezo actuator can be used.

  By moving the imaging units 304A to 304C, the interval (pitch) in the Z direction between the three reference planes is changed, and a plurality of (three) layers at an arbitrary height can be simultaneously imaged. In the present embodiment, the Z relative positions of the three reference planes are changed between a Z range search process (first process) described later and a color image acquisition process (second process) of the searched Z range. Let With this configuration, each imaging unit can be configured with a relatively simple mechanism because it only needs to move two positions, the position during the Z range search process and the position during the color image acquisition process.

  In the present embodiment, all the imaging units 304A to 304C move. However, one imaging unit may be fixed (the imaging unit moving mechanism is not installed) and the relative position of the other imaging units may be moved. Good. Thereby, the number of imaging unit moving mechanisms is reduced, and the configuration is simplified.

In the present embodiment, as an example, the Z relative position between the three reference planes is changed by moving the imaging units 304A to 304C in the optical axis direction. In addition to this configuration, a plurality of mirrors and relay lenses are arranged on each divided optical path, and the optical path length of the divided optical path is changed by moving them, and the Z relative position between three or more reference planes is changed. It is good also as a structure.

  The stage 305 includes a holding unit that holds the slide 301, an XY stage that moves the holding unit in the XY direction according to a control target value output from the control unit 306, and a Z stage that moves the holding unit in the Z direction. Included (both not shown). In particular, the XY stage is preferably capable of being driven over a wide range of a general slide size of 76 mm × 26 mm or more. The XY stage can be configured by a linear motor, a stage using a linear motion guide, a DC motor using a linear motion ball screw, a pulse motor, a linear motion system driven by a VCM, or the like. On the other hand, the Z stage is preferably one that can be driven with a positioning accuracy of 0.1 μm or less. The Z stage is a linear motor, a stage using a linear motion guide, a DC motor using a linear motion ball screw, a pulse motor, a linear motion system driven by a VCM, a mechanism using a leaf spring guide mechanism and a piezoelectric actuator, etc. Can be configured.

  By moving the XY stage, the XY relative position between the slide 301 and the imaging optical system 302 is changed, and a divided image of a desired region of the slide 301 can be acquired. In addition, by continuously controlling the XY movement, it is possible to acquire a divided image from a wide area of the slide 301. The XY movement order for acquiring images from a wide area is determined based on the XY acquisition target range. The XY acquisition target range may be determined based on the XY shape information of the sample measured in advance by a preliminary measurement system (not shown), or may be based on an instruction from the user as necessary. By setting the XY acquisition target range, it is possible to selectively generate image data of a region necessary for pathological diagnosis and the like. Can be handled easily. Normally, the XY acquisition target area is determined so as to be equal to the detected area of the sample.

  On the other hand, when the Z stage moves, the Z relative position between the slide 301 and the three reference planes of the imaging optical system 302 is changed, and a plurality of desired observation target layers in the slide 301 are within the depth of field. Layer images can be acquired simultaneously. Further, by repeatedly acquiring images while continuously controlling the Z movement, it is possible to acquire multilayer images for the same XY area of the slide 301. This process is called a multilayer image acquisition process. The Z movement order in the multilayer image acquisition process is determined based on the Z acquisition target range. In this embodiment, as an example, the Z relative position between the slide 301 and the three reference planes of the imaging optical system 302 is changed by moving the Z stage. The units 304A to 304C may be configured to move.

  The control unit 306 is configured by a general-purpose computer or workstation with high-speed arithmetic processing including a CPU, memory, hard disk, etc., a dedicated graphic board, or a combination thereof. The control unit 306 includes an interface for inputting and outputting control information of the stage 305, the imaging units 304A to 304C, and the imaging unit moving mechanisms 308A to 308C. In addition, the control unit 306 includes an interface for changing the setting of the image acquisition device 300 in order to output display image data to the display unit 307. Furthermore, an interface for inputting position information and shape information of the sample may be provided. The function of the control unit 306 is realized by loading a program stored in a storage medium such as a hard disk into a memory and executing it by the CPU.

The control unit 306 sequentially controls the stage 305, the imaging units 304A to 304C, and the imaging unit moving mechanisms 308A to 308C. Then, the Z acquisition target range is calculated from the imaging data output from the imaging units 304A to 304C, and based on the result, a divided image in which the multiple observation target layers of the slide 301 are within the depth of field is acquired.

  Thereafter, omnifocal image data and three-dimensional image data are generated for each divided image by image composition processing. Further, omnifocal image data, three-dimensional image data, or composite image data for displaying them in a wide area is output as display image data.

  The display unit 307 has a function of displaying an observation image suitable for pathological diagnosis based on the display image data generated by the image acquisition device 300. The display unit 307 can be configured by a monitor such as a CRT or a liquid crystal.

(Division of optical path)
FIG. 11 is a layout diagram of the imaging optical system 302, the optical path color division unit 303, and the imaging units 304A to 304C for imaging three reference planes in the image acquisition device 300.

  The light beam incident from the imaging optical system 302 is divided into three light beams by the optical path color dividing unit 303. In the present embodiment, light in the R wavelength band of the incident light beam is reflected to the imaging unit 304A side, and light in the G wavelength band is transmitted through the optical path color division unit 303 and guided to the imaging unit 304B. The band light is reflected toward the imaging unit 304C. As described above, the optical path color division unit 303 performs optical path division and color division of the light flux from the slide 301.

  As shown in FIG. 11, the three reference surfaces of the imaging optical system 302 are OSa to OSc from the side close to the imaging optical system 302. The positions of the light receiving surfaces (imaging surfaces) of the imaging units 304A to 304C are arranged so as to be optically conjugate with the positions of the reference surfaces OSa to OSc, respectively. Accordingly, the light LFa in the R wavelength band of the light flux from the reference surface OSa forms an image on the light receiving surface of the imaging unit 304A. The light LFb in the G wavelength band of the light flux from the reference plane OSb forms an image on the light receiving surface of the imaging unit 304B, and the light LFc in the B wavelength band of the light flux from the reference plane OSc is the imaging unit 304C. An image is formed on the light receiving surface.

  In response to a control command from the control unit 306, the imaging unit 304A takes in a light image formed on the light receiving surface and outputs imaging data Da. Similarly, the imaging units 304B and 304C output imaging data Db and Dc, respectively, in response to a control command from the control unit 306. In this embodiment, the imaging data Da is R monochrome image data, the imaging data Db is G monochrome image data, and the imaging data Dc is B monochrome image data.

  The arrangement in the optical axis direction of the imaging units 304A to 304C during the Z range search process described later is such that the interval between the reference planes OSa to OSc is larger (for example, several times) than the depth of field of the imaging optical system 302. It is preferable to make it. By doing so, it is possible to reduce the number of movements of the Z stage when searching for the Z range.

  In the present embodiment, as an example, three layers can be simultaneously imaged with different colors using one dichroic prism and three imaging units. However, the present invention is not limited to this. For example, two flat dichroic mirrors may be arranged on the optical axis, and the incident light beam may be divided into an R light beam, a G light beam, and a B light beam, and each may be guided to three imaging units. Alternatively, two half mirrors may be arranged on the optical axis, an incident light beam may be divided in three directions, and R, G, and B color filters may be arranged between the half mirror and the imaging unit. Further, as shown in FIG. 16, the imaging lens may be arranged on each of the divided R, G, and B optical paths. Thus, it can be determined as appropriate depending on the depth of focus of the imaging optical system 302 and the shape and configuration of the optical path color dividing unit 303.

(Function of control unit)
Next, the function of the control unit 306 will be described. Since the control unit 306 has basically the same configuration as the control unit 106 of the first embodiment shown in FIG. 3, the following description will be made with reference to the same reference numerals as those in FIG. The control unit 306 includes a main control unit 60, a stage stage position control unit 61, an imaging unit position control unit 62, an imaging control unit 63, and a data processing unit 64.

  The main control unit 60 controls the operation of each unit constituting the image acquisition apparatus 300 in an integrated manner. Specifically, the main control unit 60 acquires a monochrome image of each layer in a multilayer of samples, and combines them to acquire a color image. Then, synchronous control is performed for outputting omnifocal image data, three-dimensional image data, or composite image data thereof as display image data from a color image or a single color image acquired from multiple layers. In addition, the main control unit 60 can adjust each part of the image acquisition apparatus 300 accompanying the sample image acquisition, such as dimming control of the light source 309 and switching of various optical elements, and can appropriately notify the state of each part to the user. To do.

  The stage position control unit 61 controls XY movement and Z movement of the stage 305 via an output interface. Then, the layer of the desired divided area of the slide 301 is sent to the reference plane position. Also, the XYZ position coordinates of the stage 305 are acquired via the input interface.

  The stage position control unit 61 further controls the Z movement order during Z range search processing and color image acquisition processing (control to move the stage 305 in Z based on the Z acquisition target range of the sample and the current Z position of the Z stage. Target value). In this embodiment, the stage 305 is first moved so that the upper end (or lower end) of the reference plane is aligned with the upper end (or lower end) of the Z acquisition target range, and then the same direction (downward or upward) step by step. Next, the stage 305 is shifted. As a result, stage movement and alignment can be made more efficient, and the time can be reduced.

  The imaging unit position control unit 62 controls movement of the imaging units 304A to 304C in the optical axis direction by the imaging unit moving mechanisms 308A to 308C via the output interface, and the interval between the reference planes OSa to OSc according to processing. Change (relative distance) as appropriate. Thereby, a “Z range search process” for simultaneously capturing images of coarsely spaced layers in the sample and determining a Z acquisition target range, and a “color image acquisition process” for acquiring a plurality of layers of color images within the Z acquisition target range. Is possible. The setting of the reference plane in each process will be described later with reference to FIG. In addition, the imaging unit position control unit 62 acquires the coordinates in the optical axis direction of the imaging units 304A to 304C via the input interface, and calculates the Z positions of the reference planes OSa to OSc. Here, the Z positions of the reference planes OSa to OSc may be calculated based on the optical path length from the imaging units 304A to 304C to the image side focal position of the imaging optical system 302 and the optical magnification of the imaging optical system 302. Alternatively, the Z positions of the reference planes OSa to OSc that are optically conjugate with the imaging units 304A to 304C may be acquired in advance by measurement.

  Since the functions of the imaging control unit 63 and the data processing unit 64 are the same as those of the first embodiment, description thereof is omitted here.

  Since the hardware configuration of the control unit 306 is the same as that of the control unit 106 (see FIG. 4) of the first embodiment, the description thereof is omitted here.

(Relationship between Z range search processing and color image acquisition processing)
(1) Z Range Search Process FIG. 12A shows a slide 301 in the Z range search process and the color image acquisition process.
It is the figure shown about the relationship between the inside sample and three reference planes.

  T0 represents a state when the divided area to be acquired is sent into the visual field of the imaging optical system 302 by the XY stage and the Z range search process is started. At this time, the interval (relative distance) d0 between the reference surfaces OSa to OSc is set to a wide interval (for example, about several times the depth of field of the imaging optical system 302). With this setting, three layers in the sample are simultaneously imaged at rough intervals, and imaging data Da to Dc are obtained. In the first search, the stage 305 is controlled so that the upper end of the sample (the position above which no layer for which image acquisition is desired exists, for example, the upper surface of the cover glass) coincides with the Z position of the reference plane OSa. When performing a plurality of searches as necessary, in the second and subsequent searches, the Z stage is moved by one step while maintaining the interval between the reference planes OSa to OSc. The amount of movement in one step is set to 3 × d0, for example.

  The uppermost layer LT and the lowermost layer LB of the Z acquisition target range are determined based on the imaging data groups Da to Dc obtained by one or more searches in this way.

  FIG. 13 shows an evaluation index profile calculated by the data processing unit 64 described above, where the horizontal axis represents the Z position and the vertical axis represents the evaluation index. Evaluation indexes calculated from the imaging data groups Da to Dc are DCa to DCc, respectively. FIG. 13 shows an example in which six evaluation indices are obtained from six pieces of imaging data obtained by two searches.

ここで、Ｉ（ｉ，ｊ）は撮像データであり、（ｉ，ｊ）に位置する画素の画素値を表す。ｍは対象とする近傍画素との間隔であり、例えば２が使われる。Ｎ，Ｍは撮像データのＸ方向、Ｙ方向の画素数である。 The evaluation index is obtained by calculating a differential value or a difference value of luminance with neighboring pixels for each pixel for each imaging data. As an evaluation index, for example, there is a Brenner differentiation in which a difference from neighboring pixels is squared and added over the entire image. The Brenner derivative B can be expressed by the following equation.

Here, I (i, j) is imaging data and represents the pixel value of the pixel located at (i, j). m is an interval with a target neighboring pixel, and for example, 2 is used. N and M are the numbers of pixels in the X direction and Y direction of the imaging data.

  Since the imaging data Da to Dc are data obtained by imaging the light color-divided by the optical path color dividing unit 303, the imaging data Da to Dc are single-color image data having different colors. The data processing unit 64 calculates an evaluation index profile from these single color data having different colors. It is desirable to perform correction between colors when calculating the evaluation index profile. For example, when axial chromatic aberration occurs (for example, the conjugate position of B is deviated with respect to R / G), the change in the image (optical magnification) for each color is obtained by measurement or the like in advance. The horizontal axis Z shown in FIG. 13 is corrected accordingly.

また、最上位ＬＴおよび最下位ＬＢは、近似式のピーク値に対して特定割合となる閾値で決定してもよい（例えば、ピーク値の３０％を閾値と定め、上記近似式の値が閾値以上となるＺ範囲の上端と下端をそれぞれＬＴ，ＬＢとする）。 The data processing unit 64 estimates the peak Z position ZP where the evaluation index is a peak from the calculated profile, and determines the Z positions ZT and ZB of the uppermost LT and the lowermost LB, respectively. The Z positions of ZP, ZT, and ZB are obtained using the upper surface of the Z stage as a reference (for example, Z = 0 on the upper surface). In the estimation of the peak Z position ZP, for example, the evaluation index at each Z position may be approximated by the Lorentz function B (Z) of the following equation, and the peak value of the approximate equation may be set as the position ZP.

Further, the most significant LT and the least significant LB may be determined by a threshold value that is a specific ratio with respect to the peak value of the approximate expression (for example, 30% of the peak value is set as the threshold value, and the value of the approximate expression is the threshold value). The upper end and the lower end of the Z range as described above are LT and LB, respectively).

  When the peak position cannot be estimated from the three image data groups Da to Dc obtained by one search, the Z position of the stage 305 is moved by one step as shown in FIG. Repeat position estimation. At this time, it is desirable to estimate the peak using the imaging data groups Da to Dc obtained in the previous search. If the peak cannot be estimated even if the Z range search process is performed from the upper end to the lower end of the sample, the NA of the imaging optical system is changed or the distance between the reference planes (relative to the first embodiment) as described in the first embodiment. The Z range search process may be performed by changing the (distance).

(2) Color Image Acquisition Processing T1 represents a state at the time of color image acquisition processing of the Z acquisition target range determined by the Z range search processing at T0. In this embodiment, during the color image acquisition process, the interval (relative distance) between the reference planes OSa to OSc is set to zero, and the same Z position in the sample is imaged by the three imaging units 304A to 304C. Imaging data Da, Db, and Dc that are imaged with the Z position of the reference plane matched are monochrome image data of R, G, and B of the same Z position (depth) layer in the sample. By combining these monochrome image data, the color image data at the Z position can be acquired.

  For example, in the first color image acquisition process, the stage 305 is controlled so that the uppermost LT of the Z acquisition target range matches the Z position of the reference planes OSa to OSc. Thereafter, the color image acquisition process is repeated while sequentially moving the Z position of the stage 305 by the step amount d1. When the image is captured up to the Z position of the lowest LB of the Z acquisition target range or a layer below it, the color image acquisition process is terminated. As a result, multiple layers of color images in the Z acquisition target range are acquired. At this time, the step amount d1 is preferably equal to or smaller than the depth of field of the imaging optical system 102, and is preferably smaller than the interval d0 between the reference planes OSa to OSc at the time T0 described above.

  FIG. 12B shows another embodiment of the color image acquisition process. T11 to T18 represent states at the time of color image acquisition processing of the Z acquisition target range determined by the Z range search processing at T0 in FIG. In the method shown in FIG. 12B, the interval (relative distance) between the reference planes OSa to OSc is set to d2, and different Z positions in the sample are imaged by the three imaging units 304A to 304C. The distance (relative distance) d2 at this time is preferably equal to or smaller than the depth of field of the imaging optical system 102, and is preferably smaller than the distance (relative distance) d0 at the time T0 described above.

As an imaging procedure, first, at time T11, the Z position of the stage 305 is adjusted so that the reference plane OSc is positioned at the uppermost LT of the Z acquisition target range, and imaging is performed. Thereby, a B color image of the LT layer is captured. Next, at time T12, the stage 305 is moved by the same amount as the relative distance d2 of the reference plane, and imaging is performed with the Z position of the reference plane OSc aligned with the L1 layer. At this time, the reference plane OSb coincides with the LT layer, and thereby, a B color image of the L1 layer and a G color image of the LT layer are captured. Next, at time T13, the stage 305 is moved by the same amount as the relative distance d2 of the reference plane, and imaging is performed with the Z position of the reference plane OSc aligned with the L2 layer.
At this time, the reference plane OSb matches the L1 layer, and the reference plane OSa matches the LT layer. Thereby, the B color image of the L2 layer, the G color image of the L1 layer, and the R color image of the LT layer are captured. At this point, the LT layer has RGB three-color images, and by synthesizing these three-color images, a color image of the LT layer can be acquired. In this way, the stage 305 is sequentially moved to acquire color images up to the L5 layer. At this time, the L5 layer is the same or lower than the lowest LB of the Z acquisition target range.

(Image acquisition method)
An image acquisition method of the image acquisition apparatus 300 in the present embodiment will be described with reference to a flowchart shown in FIG.

  First, in step S1601, the main control unit 60 acquires processing conditions (such as an XY acquisition target range) of the image acquisition device 300. Further, the main control unit 60 calculates the XY movement order (XY stage control target value table) of the stage based on the XY acquisition target range. In this table, control target values of the XY stage for moving each divided region into the field of view of the imaging optical system 302 in order to sequentially capture a plurality of divided regions on the slide 301 are described. Note that the size of each divided area corresponds to the size of the effective imaging area of the imaging unit projected onto the object side of the imaging optical system 302.

  In the next step S1602, the main control unit 60 moves the stage position control unit 61 based on the XY movement order (for example, by reading the target value described in the top row from the XY stage control target value table). The stage 305 is controlled via Further, the XY position coordinates of the stage 305 after movement are acquired. Further, the main control unit 60 controls the imaging unit moving mechanisms 308A to 308C via the imaging unit position control unit 62, and moves the imaging units 304A to 304C in the optical axis direction. Then, the interval between the reference planes OSa to OSc is set to the first interval (for example, four times the depth of field of the imaging optical system 302) during the Z range search process.

In the next step S1603, the main control unit 60 performs a Z range search process via the stage position control unit 61, the imaging control unit 63, and the data processing unit 64, and determines a Z acquisition target range. The details of the Z range search process are basically the same as those shown in FIG. 8 of the first embodiment. (However, the first implementation is that the number of reference planes and the number of imaging data that can be acquired in one search are three, and the step amount in the second and subsequent searches is d0 × 3). Different from form.)
In the next step S1604, the main control unit 60 controls the imaging unit moving mechanisms 308A to 308C via the imaging unit position control unit 62 to move the imaging units 304A to 304C in the optical axis direction. Then, as shown in FIGS. 12A and 12B, the interval between the reference planes OSa to OSc is set to the second interval (interval = zero or interval ≦ imaging optics) in the color image acquisition process. To the depth of field of the system 302).

  In the next step S1605, the main control unit 60 executes a color image acquisition process via the stage position control unit 61, the imaging control unit 63, and the data processing unit 64, and sets a plurality of observation target layers within the Z acquisition target range. Get a color image. Details of this step will be described later with reference to FIG.

  In the next step S1606, the main control unit 60 determines whether image acquisition within the XY acquisition target range has been completed (has been executed up to the last row of the XY stage control target value table). If it is determined that the process has been completed, the process proceeds to step S1607. On the other hand, if it is determined that the process is incomplete, the process proceeds to step S1602, and the line read from the XY stage control target value table is advanced by one, and the control of the stage 305 via the stage position control unit 61 is continued.

In the next step S 1607, the main control unit 60 generates and controls display image data via the data processing unit 64 and completes image acquisition.

(Color image acquisition processing)
Details of the color image acquisition process will be described with reference to the flowchart shown in FIG.

  First, in step S1621, the stage position control unit 61 determines a target position for the Z movement of the stage 305 and controls the stage 305. Further, the stage position control unit 61 acquires the Z position coordinates of the stage 305 after the movement.

  When the stage movement and the imaging are repeated a plurality of times, the content of the determination process of the target position for the Z movement in step S1621 is different between the first time and the second and subsequent times. In the case of the embodiment shown in FIG. 12A, at the first time, the reference planes OSa to OSc (all of them are closer to the upper end (uppermost layer LT) or lower end (lowermost layer LB) of the Z acquisition target range. The Z stage is moved so that the same Z position matches. For example, the stage position controller 61 determines the difference between the Z position of the uppermost layer LT in the Z acquisition target range and the reference planes OSa to OSc, and the Z position of the lowermost layer LB of the Z acquisition target range and the reference planes OSa to OSc. Compare the differences. When the former is small, the Z stage is moved so that the Z positions of the uppermost layer LT and the reference planes OSa to OSc (all the same Z positions) coincide with each other, and the movement direction of the second and subsequent Z stages is set to “Up (+ Z Direction) ”. When the latter is small, the Z stage is moved so that the Z position of the lowest layer LB and the reference planes OSa to OSc (all the same Z positions) coincide with each other. Z direction) ”. In step S1621 after the second time, the Z stage is shifted by one step (d1 in FIG. 12A) in the set moving direction.

  In the embodiment shown in FIG. 12B, in the first time, the stage position control unit 61 determines the difference between the Z position of the top layer LT in the Z acquisition target range and the reference plane OSa, and the Z acquisition target range. The difference in the Z position between the lowest layer LB and the reference plane OSc is compared. If the former is small, the Z stage is moved so that the Z positions of the uppermost layer LT and the reference plane OSc coincide with each other, and the movement direction of the second and subsequent Z stages is set to “up (+ Z direction)”. If the latter is smaller, the Z stage is moved so that the Z position of the lowest layer LB and the reference plane OSa coincide, and the second and subsequent Z stage movement directions are set to “down (−Z direction)”. In the second and subsequent steps S1621, the Z stage is shifted by one step (d2 in FIG. 12B) in the set moving direction.

  In the present embodiment, the determination as described above is performed, and the first Z position and the second and subsequent stage movement directions are determined, whereby the amount of movement of the stage can be minimized and the time for color image acquisition processing can be shortened. be able to. However, when the stage moving time does not matter, the determination may be performed without using such a determination, and the control may be performed in a predetermined moving direction or a moving direction instructed by the user.

  In the next step S1622, the imaging control unit 63 controls imaging by the imaging units 304A to 304C and inputs imaging data groups Da to Dc. Further, the imaging control unit 63 acquires positional information of the acquired image in association with the imaging data.

  In the next step S <b> 1623, the main control unit 60 determines whether image acquisition within the Z acquisition target range is completed. If it is determined that the process has been completed, the process proceeds to step S1624. On the other hand, if it is determined that the process is not completed, the process proceeds to step S1621, and the stage 305 is controlled via the stage position control unit 61.

In the next step S <b> 1624, the main control unit 60 generates an RGB color image corresponding to each Z position in the sample based on the imaging data group and its Z position information via the data processing unit 64. At this time, an RGB color image is generated by synthesizing imaging data Da that is R color image data captured at the same Z position, imaging data Db that is G color image data, and imaging data Dc that is B color image data. Is done.

  In the next step S1625, the main control unit 60 can obtain the omnifocal image data and / or the shape distribution in the thickness direction based on the RGB color image data group and its Z position information via the data processing unit 64. Generate original image data. Then, the color image acquisition of the observation target layer within the Z acquisition target range is completed.

(Advantages of this embodiment)
According to the configuration of the present embodiment described above, there are a plurality of imaging units capable of imaging image data of different colors (wavelength bands), and the interval between the reference planes of each imaging unit according to the target processing (Relative distance) can be changed. As a result, when it is necessary to observe (analyze) a wide Z range in the sample, for example, as in the Z range search process, a setting that emphasizes efficiency (reduction of processing time) such as increasing the interval between the reference planes. Can be. On the other hand, when it is necessary to acquire a high-resolution color image as in the color image acquisition process, the color image is acquired in one shot with the reference plane interval set to zero as shown in FIG. A color image is acquired by shifting the reference plane step by step as in 12 (b). As described above, the interval between the reference planes is adaptively controlled in accordance with the target process, whereby the efficiency of the process as a whole and the reduction of the processing time can be achieved. In addition, since a high-resolution color image can be acquired at a high speed from multiple layers of samples, a comparative diagnosis can be performed accurately in pathological diagnosis or the like. In addition, since the single imaging system can flexibly cope with different purposes such as the Z range search process and the color image acquisition process, the entire apparatus can be reduced in size and cost.

<Third Embodiment>
The image acquisition method of the present invention can be supplied to a system or apparatus in the form of a recording medium (or storage medium) that records software program codes that implement all or part of the functions of the above-described embodiments. This can be achieved by reading and executing the program code stored in the recording medium by the computer (or CPU or MPU) of the system or apparatus. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

  Further, when the computer executes the read program code, an operating system (OS) or the like running on the computer performs part or all of the actual processing. The case where the functions of the above-described embodiment are realized by the processing is also included in the present invention.

  Furthermore, it is assumed that the program code read from the recording medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion card or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. Included in the invention.

  When the present invention is applied to the recording medium, program code corresponding to the flowchart described above is stored in the recording medium.

<Other embodiments>
As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary. Also, from the first to the third
The configurations described in the embodiments can be used in combination with each other. Therefore, a person skilled in the art can easily conceive that a new system is configured by appropriately combining various technologies in the above-described embodiments, and a system based on such various combinations is also within the scope of the present invention. Belongs.

  For example, when the Z acquisition target range is almost the entire region in the height direction of the slide, such as when it is desired to acquire a multilayer image of a relatively thick sample as in cytodiagnosis in pathological diagnosis, the acquired image data is enormous. End up. In such a case, the imaging data acquired by the Z range search process may be recorded as it is as the image data of the divided area, and may be used to generate omnifocal image data, three-dimensional image data, or composite image data. Since the resolution in the height direction (Z direction) is low only with the image data obtained by the Z range search process, in the multilayer image acquisition process, image acquisition for complementing between the image data obtained by the Z range search process is performed. Do. In other words, after performing the first process of acquiring images of a plurality of layers in the sample at a rough interval, in order to complement between the images acquired in the first process, at a narrower interval than the first process A second process for acquiring images of a plurality of layers in the sample is performed. At this time, the entire area of the Z acquisition target range may be supplemented, but the multilayer image acquisition process may be performed only on the periphery of the layer that particularly requires the resolution in the height direction. The layer that requires the resolution in the height direction may be specified by the user or may be determined based on the evaluation index calculated by the Z range search process (for example, the periphery of the layer from which a high-contrast image is obtained) Etc.).

  100: Image acquisition device, 101: Slide, 102: Objective lens, 103: Optical path division unit, 104A to 104D: Imaging unit, 105: Stage, 106: Control unit, 108A to 108D: Imaging unit moving mechanism

Claims (23)

An optical system for collecting light from the sample;
Splitting means for splitting the optical path from the optical system into a plurality of optical paths;
A plurality of imaging means each having a light receiving surface on the plurality of optical paths;
Control means for performing a first process and a second process using the plurality of imaging means;
Changing means for changing the interval between a plurality of reference surfaces that are optically conjugate with the light receiving surfaces of the plurality of imaging means,
The changing unit may change the interval between the plurality of reference planes when the plurality of imaging units perform imaging for the first processing and when imaging for the second processing. A characteristic image acquisition device. In the first process, the plurality of reference planes are arranged at different heights in the sample, and based on the image data obtained by the plurality of imaging means, the height of the layer from which an image is to be acquired in the sample Is a process for determining
In the second process, the plurality of imaging units are selected from a range based on the height determined in the first process by making the interval between the plurality of reference surfaces smaller than in the first process. The image acquisition apparatus according to claim 1, wherein the image acquisition process is a process of imaging a plurality of layers of the sample. The image acquisition apparatus according to claim 2, wherein in the second process, the interval between the plurality of reference surfaces is set to zero. The plurality of imaging means are for imaging monochrome image data of different colors,
The color image data is generated by synthesizing single-color image data of a plurality of colors obtained by the plurality of imaging units for each of the plurality of layers in the second processing. 4. The image acquisition device according to 3. In the first process, the plurality of reference planes are arranged at different heights in the sample, and a height at which a maximum contrast image is obtained is estimated based on image data obtained by the plurality of imaging units. The image acquisition apparatus according to claim 2, wherein a height of a layer from which the image is to be acquired is selected so as to include the estimated height. The first process is a process of imaging the plurality of layers of the sample by the plurality of imaging units by arranging the plurality of reference planes at different heights in the sample,
In the second process, the layers that complement the gap between the plurality of layers imaged in the first process are imaged by the plurality of imaging units in which the intervals of the plurality of reference planes are changed from the first process. The image acquisition apparatus according to claim 1, wherein the image acquisition apparatus performs the processing. In the first process, the plurality of reference planes are arranged at different heights in the sample, and based on the image data obtained by the plurality of imaging means, the height of the layer from which an image is to be acquired in the sample Is a process for determining
In the second process, when the height of the layer from which the image is to be acquired cannot be determined in the first process, the interval between the plurality of reference planes is made smaller than that in the first process. The image acquisition apparatus according to claim 1, wherein the image acquisition apparatus is a process of determining a height of a layer from which an image is to be acquired in the sample based on image data obtained by the plurality of imaging units. A second changing means for changing a relative position in the height direction between the sample and the plurality of reference planes;
By changing the relative position in the height direction of the sample and the plurality of reference planes by the second changing unit and imaging by the plurality of imaging units a plurality of times, more than the number of the reference planes The image acquisition apparatus according to claim 1, wherein the image data of the layer can be acquired. The second changing unit includes the sample and the plurality of reference planes when the plurality of imaging units perform imaging for the first processing and when imaging for the second processing is performed. The image acquisition apparatus according to claim 8, wherein the amount of change in the relative position in the height direction of the image is changed. The second changing means sets the change amount of the relative position in the height direction of the sample and the plurality of reference surfaces to a value obtained by multiplying the interval between the plurality of reference surfaces by the number of the reference surfaces. The image acquisition apparatus according to claim 9, wherein the apparatus is an image acquisition apparatus. The second changing means is configured such that, at a first time, the sample and the plurality of reference planes are aligned with an upper end or a lower end of a range in which image data is to be acquired. The relative position in the height direction is changed, and after the second time, the relative positions of the sample and the plurality of reference surfaces are changed in the same direction. Image acquisition device. The image acquisition apparatus according to claim 8, wherein the second changing unit is a stage that holds the sample and moves in the optical axis direction. The said change means sets the space | interval of these reference surfaces in the case of imaging for said 2nd process to be below the depth of field of the said optical system, It is characterized by the above-mentioned. The image acquisition apparatus of any one of them. The change unit changes the interval between the plurality of reference surfaces by changing the position of the light receiving surfaces of the plurality of image pickup units in the optical axis direction. The image acquisition device according to item. The light receiving surface of one of the plurality of imaging means is fixed,
The image acquisition apparatus according to claim 14, wherein the changing unit changes a position of a light receiving surface of another imaging unit. The image acquisition apparatus according to claim 1, wherein the control unit associates image data of the imaging unit with information on a height of a corresponding reference surface. The control unit generates image data to be used for observation of the sample by using the image data of the plurality of layers obtained in the second process. The image acquisition device according to claim 1. An optical system for collecting light from the sample;
Splitting means for splitting the optical path from the optical system into a plurality of optical paths;
A plurality of image pickup means each having a light receiving surface on the plurality of optical paths,
Setting the interval between the plurality of reference planes to a first interval;
Performing a first process using the plurality of imaging means set to the first interval;
Setting the interval between the plurality of reference surfaces to a second interval smaller than the first interval;
And a step of performing a second process using the plurality of imaging means set at the second interval. In the first process, the plurality of reference planes are arranged at different heights in the sample, and based on the image data obtained by the plurality of imaging means, the height of the layer from which an image is to be acquired in the sample Is a process for determining
The second process is a process of imaging a plurality of layers of the sample by the plurality of imaging units from a range based on the height determined in the first process. 18. A method for controlling the image acquisition device according to 18. The method of controlling an image acquisition apparatus according to claim 19, wherein in the second process, the interval between the plurality of reference planes is set to zero.   In the first process, the plurality of reference planes are arranged at different heights in the sample, and a height at which a maximum contrast image is obtained is estimated based on image data obtained by the plurality of imaging units. 21. The method of controlling an image acquisition apparatus according to claim 19, wherein a height of a layer from which the image is to be acquired is selected so as to include the estimated height. The image acquisition according to any one of claims 18 to 21, wherein an interval between the reference surfaces is changed by changing a position of a light receiving surface of the plurality of imaging units in an optical axis direction. Control method of the device. The method of controlling an image acquisition apparatus according to any one of claims 18 to 22, wherein the second interval is set to be equal to or less than a depth of field of the optical system.

Images