JP2016166941A - Focusing position detection device, focusing position detection method and imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for stable and quick detection of a focusing position of a pathological sample.SOLUTION: A focusing position detection device detects a focused position of a sample photographed while changing the focus position in an optical axis with respect to the sample using image data of a plurality of layers in the sample, in which the image data is color image data composed of data of a plurality of color planes, and the focusing position detection device includes: selection means that selects the color plane corresponding to a color largest in absorption by the sample of the plurality of color planes as the color plane for detection of the focused position; and focused position determination means that evaluates a degree of focusing of the data of the color plane for the detection of the focused position of each of the plurality of layers to thereby determine the focused position of the sample.SELECTED DRAWING: Figure 15

Description

  The present invention relates to a focus position detection device, a focus position detection method, and an imaging system.

Attention has been focused on a virtual slide system capable of acquiring virtual slide image data by imaging a stained biological sample in a slide with a digital microscope and displaying the image on a monitor for observation (Patent Literature). 1).
In addition, when imaging a thick biological sample, there is a technology for acquiring multiple layer image data focused on different depths in the sample while changing the focal position in the optical axis direction using a digital microscope. (Patent Document 2). Three-dimensional image data composed of a plurality of layer image data is called Z stack image data.
In addition, a technique for stably detecting a focus position of a microscope is known (Patent Document 3).
In addition, a technique for determining a focus position of a phase object such as an unstained cell that is substantially transparent is known (Patent Document 4).

JP 2011-118107 A JP 2010-191000 A JP-A-10-232343 JP 2008-20498 A

  Conventionally, in focus position detection of a microscope or a virtual slide system, a contrast value is calculated from each of a plurality of Z stack image data of a biological sample or a pathological specimen, and the Z position of the maximum contrast value is set as the focus position.

  However, even if it is a stained biological sample or pathological specimen, not a non-stained cell, the refractive index differs for each component such as the cell nucleus, cytoplasm, cell membrane, etc. The present inventor has intensively studied and found out that there is a case in which the above occurs. Furthermore, the present inventor has found a problem that the detection of the in-focus position becomes unstable in a biological sample or a pathological specimen stained for the phenomenon.

With the focus detection technique disclosed in Patent Document 3, it is possible to stably detect the focus position of an object whose light transmittance changes. However, a technique for stably detecting a focus position on a biological sample or a pathological specimen having a refractive index distribution is not disclosed.
The focus detection technique disclosed in Patent Document 4 can detect the focus position of a phase object such as an unstained cell that is substantially transparent. This is a method of using difference image data of two Z stack image data. Since it is premised on the accumulation of Z stack image data in the data memory, it is not suitable for speeding up.

  Therefore, an object of the present invention is to provide a technique for detecting a focus position stably and at high speed with respect to a pathological specimen (stained cell or stained biological sample) having a refractive index distribution.

A first aspect of the present invention is a focus position detection for detecting a focus position of the specimen using image data of a plurality of layers in the specimen, which is imaged while changing a focal position in the optical axis direction with respect to the specimen. The image data is color image data composed of data of a plurality of color planes, and the in-focus position detection device is a color having the largest absorption by the sample among the plurality of color planes. Selecting a color plane corresponding to the color plane for detecting the in-focus position, and evaluating the in-focus degree of the data of the in-focus position color plane for each of the plurality of layers, An in-focus position detecting device comprising: an in-focus position determining means for determining an in-focus position of a sample is provided.

  A second aspect of the present invention is an imaging apparatus including an imaging unit that images the specimen, a control unit that controls the imaging unit, and a focus position detection device according to the present invention, wherein the control unit includes: An imaging apparatus is provided, wherein control is performed to acquire information on a focus position determined by the focus position detection apparatus and to acquire image data of the sample at the focus position by the imaging unit. To do.

  The third aspect of the present invention is a focus position detection for detecting a focus position of the specimen using image data of a plurality of layers in the specimen, which is imaged while changing a focal position in the optical axis direction with respect to the specimen. The image data is color image data composed of data of a plurality of color planes, and the in-focus position detection method is a color having the largest absorption by the sample among the plurality of color planes. Selecting the color plane corresponding to the in-focus position detection color plane, and evaluating the in-focus degree of the data of the in-focus position color plane data of each of the plurality of layers. And a step of determining an in-focus position.

  According to a fourth aspect of the present invention, there is provided an imaging system comprising: an imaging apparatus according to the present invention; and an image processing apparatus that processes an image acquired by the imaging apparatus.

  According to the present invention, it is possible to provide a technique for detecting a focused position of a pathological specimen stably and at high speed.

1 is an overall view of a device configuration of an imaging system. A display example of an image display application. The functional block diagram of the imaging device in an imaging system. The hardware block diagram of an image processing apparatus. The schematic diagram which shows the slide in which the sample (biological sample) was mounted. The schematic diagram explaining a dyeing | staining specimen (biological sample). The schematic diagram explaining a Z-axis code | symbol. FIG. 6 is a schematic diagram for explaining layer and color plane data. Schematic diagram illustrating the basic structure of the microscope. The functional block diagram of an imaging unit. The functional block diagram of a lighting unit. The schematic diagram explaining the contrast characteristic of the object which has refractive index distribution. Contrast characteristics of an object having a refractive index distribution. The functional block diagram regarding the 1st focus position detection method. The flowchart regarding the 1st focus position detection method. The functional block diagram regarding the 2nd focus position detection method. The flowchart regarding the 2nd focus position detection method. The schematic diagram explaining the table regarding the 2nd focus position detection method. 6 is another configuration example of an imaging device in an imaging system. 6 is another configuration example of an imaging device in an imaging system. The schematic diagram explaining the focus index value of the object which has refractive index distribution.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Example 1>
(Device configuration of the imaging system)
The focus position detection device according to the embodiment of the present invention can be used in an imaging system including an imaging device and a display device. The configuration of this imaging system will be described with reference to FIG. The imaging system includes an imaging apparatus (digital microscope apparatus or virtual slide scanner) 101, an image processing apparatus 102, a display apparatus 103, and a data server 104, and acquires and displays a two-dimensional image of a specimen to be imaged. It is a system with functions. The imaging apparatus 101 and the image processing apparatus 102 are connected by a dedicated or general-purpose I / F cable 105, and the image processing apparatus 102 and the display apparatus 103 are connected by a general-purpose I / F cable 106. The data server 104 and the image processing apparatus 102 are connected by a general-purpose I / F LAN cable 108 via a network 107.

The imaging device 101 is a virtual slide scanner having a function of acquiring a plurality of two-dimensional image data (layer image data) having different focal positions in the optical axis direction (Z-axis direction) and outputting a digital image. CCD (Charge Coupled) is used to acquire 2D images.
A solid-state imaging device such as a device or a CMOS (Complementary Metal Oxide Semiconductor) is used. Note that, instead of the virtual slide scanner, the imaging apparatus 101 can be configured by a digital microscope apparatus in which a digital camera is attached to an eyepiece of a normal optical microscope.

  The image processing apparatus 102 is an apparatus having a function of generating data to be displayed on the display apparatus 103 from a plurality of original image data (layer image data) acquired from the imaging apparatus 101 in response to a request from the user. . The image processing apparatus 102 is configured by a general-purpose computer or workstation including hardware resources such as a CPU (Central Processing Unit), a RAM, a storage device, an operation unit, and various I / Fs. The storage device is a large-capacity information storage device such as a hard disk drive, and stores programs, data, OS (operating system) and the like for realizing each processing described later. Each function described later is realized by the CPU loading a necessary program and data from the storage device to the RAM and executing the program. The operation unit includes a keyboard, a mouse, and the like, and is used by the user to input various instructions.

The display device 103 is a display that displays observation image data that is a result of the arithmetic processing performed by the image processing device 102, and includes a liquid crystal display or the like.
The data server 104 is a server that stores image data for observation, which is a result of arithmetic processing performed by the image processing apparatus 102.

  In the example of FIG. 1, the imaging system is configured by four devices, that is, the imaging device 101, the image processing device 102, the display device 103, and the data server 104, but the configuration of the present invention is not limited to this configuration. . For example, an image processing device integrated with a display device may be used, or the function of the image processing device may be incorporated in the imaging device. Further, the functions of the imaging device, the image processing device, the display device, and the data server can be realized by a single device. Conversely, the functions of the image processing apparatus or the like may be divided and realized by a plurality of apparatuses.

(Display screen)
FIG. 2 shows an example in which image data of a stained specimen (biological sample) photographed in advance is displayed on the display device 103 through an image display application.

  FIG. 2 shows the basic configuration of the screen layout of the image display application. In the entire window 201 of the display screen, there are an information area 202 indicating the status of display and operation and information on various images, an entire image 203 of a specimen to be observed, a display area 205 for detailed observation of specimen image data, and a display area 205 A magnification display area 206 is arranged. A frame line 204 drawn on the entire image 203 indicates the position and size of the area that is enlarged and displayed in the display area 205 for detailed observation. With the whole image 203 and the frame line 204, the user can easily grasp which part of the whole specimen image data is being observed.

  The image data displayed in the detailed observation display area 205 is updated in response to a movement instruction or an enlargement / reduction instruction from the input operation device. For example, the movement can be realized by dragging the mouse on the screen, and the enlargement / reduction can be realized by rotating the mouse wheel or the like (for example, the forward rotation of the wheel is enlarged and the backward rotation is assigned to reduction). In addition, switching to image data with a different focal position can be realized by rotating a mouse wheel or the like while pressing a predetermined key (for example, the Ctrl key) (e.g., moving the wheel forward to moving to a deep image, Assign backward rotation to move to a shallower image). As the user changes the display image data as described above, the display area 205, the display magnification in the magnification display area 206, and the frame line 204 in the entire image 203 are updated. In this way, the user can observe the desired in-plane position, depth position, and magnification image data.

(Functional configuration of imaging device)
FIG. 3 is a block diagram illustrating a functional configuration of the imaging apparatus 101. The imaging apparatus 101 generally includes an illumination unit 301, a stage 302, a stage control unit 305, an imaging optical system 307, and an imaging unit 310. Further, the imaging apparatus 101 includes a development processing unit 319, a pre-measurement unit 320, a main control unit 321, an I / F unit 323, and a focus position detection unit 324.

  The illumination unit 301 is a unit that uniformly irradiates light onto the slide 306 disposed on the stage 302. Components and functional blocks of the lighting unit 301 will be described with reference to FIGS. 9 and 11.

The stage 302 is driven and controlled by the stage control unit 305, and can move in the three-axis directions of XYZ.
The slide 306 is a member in which a section of tissue to be observed is attached on a slide glass and fixed under the cover glass together with an encapsulant.
The stage control unit 305 has a drive control system 303 and a stage drive mechanism 304. The drive control system 303 receives an instruction from the main control unit 321 and performs drive control of the stage 302. The moving direction, moving amount, and the like of the stage 302 are determined based on the position information and thickness information (distance information) of the sample measured by the pre-measurement unit 320 and an instruction from the user as necessary. The stage drive mechanism 304 drives the stage 302 in accordance with instructions from the drive control system 303.

  The imaging optical system 307 is a lens group for forming an optical image of the specimen on the slide 306 on the imaging sensor 308. The optical configuration of the imaging optical system 307 will be described with reference to FIG.

The imaging unit 310 includes an imaging sensor 308 and an analog front end (AFE) 309. The imaging sensor 308 is a one-dimensional or two-dimensional image sensor that changes a two-dimensional optical image into an electrical physical quantity by photoelectric conversion, and for example, a CCD or a CMOS device is used. In the case of a one-dimensional sensor, a two-dimensional image is obtained by electrically scanning in the scanning direction and moving the stage 302 in the sub-scanning direction. An electric signal having a voltage value corresponding to the intensity of light is output from the image sensor 308. When color image data is desired as the captured image data, for example, a single-plate image sensor to which a Bayer array color filter is attached may be used, or an RGB three-plate image sensor may be used. The imaging unit 310 acquires the divided image data of the specimen by driving the stage 302 in the XY axis direction. The AFE 309 is a circuit that controls the operation of the image sensor 308 and a circuit that converts an analog signal output from the image sensor 308 into a digital signal. The AFE 309 includes an H / V driver, a CDS (Correlated Double Sampling), an amplifier, an AD converter, and a timing generator. The H / V driver converts a vertical synchronizing signal and a horizontal synchronizing signal for driving the image sensor 308 into potentials necessary for driving the sensor. CDS is a correlated double sampling circuit that removes fixed pattern noise. The amplifier is an analog amplifier that adjusts the gain of an analog signal from which noise has been removed by CDS. The AD converter converts an analog signal into a digital signal. When the output at the final stage of the imaging apparatus is 8 bits, the AD converter converts the analog signal into digital data quantized from about 10 bits to about 16 bits and outputs it in consideration of subsequent processing. The converted sensor output data is called RAW data. The RAW data is developed by a subsequent development processing unit 319. The timing generator generates a signal for adjusting the timing of the image sensor 308 and the timing of the development processing unit 319 at the subsequent stage. When a CCD is used as the image sensor 308, the AFE 309 is indispensable. However, in the case of a CMOS image sensor capable of digital output, the function of the AFE 309 is included in the sensor. In the imaging unit 310 of FIG. 3, only the minimum functional blocks necessary for explaining the flow of captured image data are shown. Other components and functional blocks of the imaging unit 310 will be described with reference to FIGS. 9 and 10.

The development processing unit 319 includes a black correction unit 311, a demosaicing processing unit 312, a white balance adjustment unit 313, an image composition processing unit 314, a filter processing unit 316, and a gamma correction unit 317. The black correction unit 311 performs a process of subtracting the background (black correction data obtained at the time of light shielding) from the value of each pixel of the RAW data. The demosaicing processing unit 312 performs processing for generating image data of each color of RGB from RAW data in the Bayer array. The demosaicing processing unit 312 calculates the value of each color of RGB of the target pixel by interpolating the values of peripheral pixels (including pixels of the same color and other colors) in the RAW data. In addition, the demosaicing processing unit 312 also performs defective pixel correction processing (interpolation processing). Note that when the image sensor 308 does not have a color filter and single-color image data is obtained, the demosaicing process is not necessary, and the defective pixel correction process is executed. The demosaicing process is unnecessary even when the three-plate image sensor 308 is used. The white balance adjustment unit 313 performs a process of reproducing a desired white color by adjusting the gain of each RGB color according to the color temperature of the light of the illumination unit 301. The image composition processing unit 314 performs a process of generating a large size image data in a desired imaging range by connecting a plurality of divided image data captured by the imaging sensor 308. In general, since the existence range of the specimen is wider than the imaging range that can be acquired by one imaging with an existing image sensor, one piece of two-dimensional image data is generated by joining the divided image data. A region where an image is acquired by one imaging is also referred to as a “tile”, and tile divided image data is also referred to as a “tile image”. For example, assuming that a 10 mm square range on the slide 306 is imaged with a resolution of 0.25 μm, the number of pixels on one side is 40,000 pixels of 10 mm / 0.25 μm, and the total number of pixels is 1.6 billion, which is the square. It becomes a pixel. In order to acquire image data of 1.6 billion pixels by using the image sensor 308 having 10M (10 million) pixels, it is necessary to divide the image into 160 (= 1,600,000,000) tiles and perform imaging. . Note that as a method of connecting a plurality of tile images, there are a method of connecting by aligning based on position information of the stage 302, a method of connecting corresponding points or lines of a plurality of tile images, and the like. At the time of joining, it can be smoothly connected by interpolation processing such as zero-order interpolation, linear interpolation, and high-order interpolation. The filter processing unit 316 is a digital filter that realizes suppression of high-frequency components included in an image, noise removal, and resolution enhancement. The gamma correction unit 317 executes processing for adding an inverse characteristic of the gradation expression characteristic of a general display device to an image, or gradation that matches human visual characteristics by gradation compression or dark part processing of a high luminance part. Or perform a conversion. In this embodiment, in order to acquire an image for the purpose of morphological observation, gradation conversion suitable for the subsequent synthesis processing and display processing is applied to the image data.

  The pre-measurement unit 320 is a unit that performs pre-measurement to calculate the position information of the specimen on the slide 306, the distance information to the desired focal position, and the parameter for adjusting the amount of light caused by the specimen thickness. By acquiring information by the pre-measurement unit 320 before the main measurement (acquisition of captured image data), it is possible to perform image capturing without waste. A two-dimensional image sensor having a resolution smaller than that of the image sensor 308 is used for acquiring position information on the two-dimensional plane. The pre-measurement unit 320 grasps the position of the sample on the XY plane from the acquired image data. A measuring instrument such as a laser displacement meter is used to acquire distance information and thickness information. Also included is a label reader that reads information on the specimen recorded on (or linked to) the label (see FIG. 5) on the slide 306. Specimen information is information such as patient ID, specimen ID, specimen attributes (parts, staining information, etc.).

  The main control unit 321 is a function for controlling the various units described so far. The control functions of the main control unit 321 and the development processing unit 319 are realized by a control circuit having a CPU, a ROM, and a RAM. That is, the program and data are stored in the ROM, and the functions of the main control unit 321 and the development processing unit 319 are realized by the CPU executing the program using the RAM as a work memory. For example, a device such as an EEPROM or a flash memory is used as the ROM, and a DRAM device such as DDR3 is used as the RAM. Note that the function of the development processing unit 319 may be replaced with an ASIC implemented as a dedicated hardware device.

  The I / F unit 323 performs compression processing on the image data generated by the development processing unit 319 and transmits the compressed image data to the image processing apparatus 102. The compression processing unit 318 executes an encoding process of compression performed for the purpose of improving the efficiency of transmission of large-capacity two-dimensional image data and reducing the capacity when storing the data. As a still image compression technique, standardized encoding methods such as JPEG (Joint Photographic Experts Group) and JPEG 2000 and JPEG XR improved and evolved from JPEG are widely known. The external device I / F 322 is an interface for sending the compressed image data to the image processing device 102. The imaging apparatus 101 and the image processing apparatus 102 are connected by an optical communication cable. Alternatively, a general-purpose interface such as USB or Gigabit Ethernet (registered trademark) is used.

  The focus position detection unit 324 is a focus position detection device having a function of detecting the focus position. The processing contents of the focus position detection unit will be described later (the first focus position detection method will be described with reference to FIGS. 14 and 15, and the second focus position detection method will be described with reference to FIGS. 16, 17 and 18). .)

(Hardware configuration of image processing device)
FIG. 4 is a block diagram showing a hardware configuration of the image processing apparatus 102 according to the present invention. For example, a PC (Personal Computer) is used as an apparatus for performing image processing. The PC includes a CPU 401, a main memory 402, an arithmetic processing board 403, a graphics board 404, and an internal bus 405 for connecting them together. The PC includes a LAN I / F 406, a storage device I / F 407, an external device I / F 409, an operation I / F 410,
An input / output I / F 413 is provided.

  The CPU 401 comprehensively controls the entire blocks of the PC using programs and data stored in the main memory 402 as necessary. The main memory 402 is configured by a RAM (Randam Access Memory). The main memory 402 is used as a work area for the CPU 401, and temporarily stores various data to be processed such as an OS, various programs being executed, and display data generation. The main memory 402 is also used as a storage area for image data. A high-speed transfer of image data between the main memory 402 and the graphics board 404 can be realized by a DMA (Direct Memory Access) function of the CPU 401. The graphics board 404 outputs the image processing result to the display device 103. The display device 103 is a display device using, for example, liquid crystal, EL (Electro-Luminescence), or the like. Although the display device 103 is assumed to be connected as an external device, a PC integrated with the display device may be assumed. For example, a notebook PC corresponds to this. The arithmetic processing board 403 includes a processor and a buffer memory (not shown) in which specific arithmetic functions such as image processing are enhanced. In the following description, the CPU 401 is used for various arithmetic processes and data processes, and the main memory 402 is used as a memory area. However, a processor or a buffer memory in the arithmetic processing board can also be used. Category.

The input / output I / F 413 includes a data server 104 via a LAN I / F 406, a storage device 408 via a storage device I / F 407, an imaging device 101 via an external device I / F 409, and an operation I / F 410. A keyboard 411 and a mouse 412 are connected via the terminal. The storage device 408 is an auxiliary storage device that permanently stores information such as an OS, a program, firmware, and various parameters that are executed by the CPU 401. The storage device 408 is also used as a storage area for image data sent from the imaging device 101. Magnetic disk drives such as HDD (Hard Disk Drive), SSD (Solid)
A semiconductor device such as a state drive) or a flash memory is used. Although a pointing device such as a keyboard 411 and a mouse 412 is assumed as a connection device with the operation I / F 410, a configuration in which the screen of the display device 103 directly serves as an input device such as a touch panel may be employed. In that case, the touch panel can be integrated with the display device 103.

(slide)
FIG. 5 is a schematic diagram showing a slide 306 on which a specimen (biological sample) 501 is placed. In a pathological specimen slide, a specimen 501 placed on a slide glass 502 is enclosed by an encapsulating agent (not shown) and a cover glass 503 placed thereon. The size and thickness of the specimen 501 vary depending on the specimen. Further, a label area 504 in which information related to the specimen is recorded exists on the slide glass 502. Information can be recorded in the label area 504 by pen entry or printing of a barcode or a two-dimensional code. In addition, a storage medium that can store information by an electrical, magnetic, or optical method may be provided in the label area 504. In the following embodiments, the pathological specimen slide shown in FIG. 5 will be described as an example of the subject.

(Stained specimen)
FIG. 6 is a schematic diagram for explaining a stained specimen (biological sample). In the pathological specimen, the specimen (biological sample) 501 has a thickness of about 4 to 5 μm and is large enough to fit on a 2 to 3 inch slide. When the specimen (biological sample) 501 is enlarged, the arrangement of the cells 601 corresponding to the tissue can be observed. A cell 601 is composed of a nucleus 602 and a cytoplasm 603. Depending on the tissue, the arrangement and size of cells and the shape and size of the nuclei that make up the cells vary. For example, in the case of hepatocytes, the cell diameter is about 20 μm, and the nucleus size is about 5 to 15 μm. The biological sample is colorless and transparent, and the tissue structure is difficult to understand if observed directly. Therefore, in order to make it easy to observe the tissue structure of the specimen, the pathological specimen is stained. In histopathological examination, HE (hematoxylin and eosin) staining is basically performed, but depending on the purpose of diagnosis, immunostaining and FISH (fluorescence in situ) are performed.
A hybridization method is also performed. In the present embodiment, HE staining generally performed in histopathological diagnosis will be described as an example. In HE staining, the chromatin in the nucleus is dyed dark blue and the cytoplasm is dyed light red.

(Focus position and Z-axis code)
Fig.7 (a)-FIG.7 (c) are the schematic diagrams explaining a focus position and a Z-axis code | symbol. In the present embodiment, the Z-axis has a light source side (illumination unit 301 side) as a negative (−) direction and an objective lens side (imaging optical system 307 side) as a positive (+) direction. The focal position 701 is the position of a surface that is optically conjugate with the imaging surface of the imaging sensor 308.

  FIG. 7A shows a state where the Z position of the thickness center of the specimen (biological sample) 501 is on the + side with respect to the focal position 701. In the case of a translucent specimen (biological sample), a negative (bright) image having a negative (bright) contrast is obtained. A negative (bright) image is an image in which a region with a high refractive index is bright and a region with a low refractive index is dark. FIG. 7B shows that the Z position at the center of the thickness of the specimen (biological sample) 501 is at the focal position 701. If the sample has a small change in refractive index (biological sample), an image is hardly obtained. FIG. 7C shows a state where the Z position of the thickness center of the specimen (biological sample) 501 is on the negative side with respect to the focal position 701. If it is a translucent specimen (biological sample), a positive (dark) image having a positive (dark) contrast is obtained. A positive (dark) image is an image in which a region having a high refractive index is dark and a region having a low refractive index is brightly imaged.

  Here, the terms indicating the position in the Z-axis direction are summarized. The focal position 701 is a position of a surface that is optically conjugate with the imaging surface of the imaging sensor 308. The in-focus position is a position where an object to be observed (such as a nucleus) is present at the Z position of the thickness center of the sample and the thickness center of the sample matches the focal position. The position of the defocus amount d = 0 is a position where the thickness center of the sample coincides with the focal position, and is the in-focus position. A defocus amount d> 0 means that the relationship between the thickness center of the sample and the focal position is in the state shown in FIG. The defocus amount d <0 means that the relationship between the thickness center of the specimen and the focal position is in the state shown in FIG.

(Layer and color plane data)
FIG. 8 is a schematic diagram illustrating layers and color plane data. It is a figure which shows the positional relationship of the image side layer and the optical axis direction (Z-axis direction) of color plane data. A specific layer in the specimen in the optical axis direction (Z-axis direction) is called a layer. Two-dimensional image data included in a layer and acquired at a specific position in the optical axis direction (Z-axis direction) is layer image data. When acquiring color image data, the position in the optical axis direction (Z-axis direction) on the image side when acquiring layer image data for each color need not be limited to the same position. For example, when acquiring RGB layer image data, the position of the imaging surface in the optical axis direction (Z-axis direction) may be different for each color in order to correct axial chromatic aberration. In this specification, when color layer image data is particularly indicated, it is referred to as color plane data, and when not limited to color, it is simply referred to as layer image data.

(Basic structure of microscope optical system)
FIG. 9 is a schematic diagram for explaining a basic structure of an optical system of a microscope. The basic structure of the optical system of the microscope includes an illumination unit 301, an imaging optical system 307, and an imaging unit 310. The illumination unit 301 is a Kohler illumination system, and includes a surface light source on the light source surface 901, a filter wheel A909, illumination lenses 903A to 903C, a field stop 907, and an aperture stop 908. A surface light source centered on the intersection of the light source surface 901 and the optical axis 902 is provided on the light source surface 901. The imaging optical system 307 includes an objective lens 904, a pupil plane 905, an imaging lens 906, and a filter wheel B910. An image of the specimen (biological sample) 501 is formed on the image plane 911 by the imaging optical system 307. The filter wheel A909 and the filter wheel B910 are components for inserting a color filter when acquiring color image data. Therefore, it is only necessary that a color filter is inserted into either the filter wheel A909 or the filter wheel B910.

(Imaging unit)
FIG. 10 is a functional block diagram of the imaging unit. The imaging unit 310 includes an imaging sensor 308, an analog front end (AFE) 309, a filter wheel B910, an imaging color filter changing unit 1001, and an imaging sensor control unit 1002. The processing in the image sensor 308 and the analog front end (AFE) 309 and the flow of captured image data are as described in FIG. When RGB color plane data is acquired by the monochrome imaging sensor 308, RGB color filters are inserted into the filter wheel B910. The imaging color filter changing unit 1001 controls the filter wheel to switch the color filters, and acquires RGB three types of color plane data. This is a so-called time-division color imaging system. The imaging color filter changing unit 1001 controls the filter wheel based on an instruction from the main control unit 321. The imaging sensor control unit 1002 controls the imaging timing and imaging time of the imaging sensor 308.

(Lighting unit)
FIG. 11 is a functional block diagram of the lighting unit. The illumination unit 301 includes a surface light source 1101, a filter wheel A909, an illumination lens A903A, a field stop 907, an illumination lens B903B, an aperture stop 908, and an illumination lens C903C. The illumination unit 301 further includes a light source control unit 1102, an illumination color filter change unit 1103, a field stop change unit 1104, and an aperture stop change unit 1105. When RGB color plane data is acquired by the monochrome imaging sensor 308, RGB color filters are inserted into the filter wheel A909. The illumination color filter changing unit 1103 controls the filter wheel to switch color filters, and acquires RGB three types of color plane data. This is a so-called time-division color imaging system. The illumination color filter changing unit 1103 controls the filter wheel based on an instruction from the main control unit 321. The light source control unit 1102 controls the light emission timing and the light emission time of the surface light source 1101. The field stop changing unit 1104 controls the field stop 907 so that the illumination light falls on the observation range of the specimen. The aperture stop changing unit 1105 controls the aperture stop 908 so that the contrast and resolution of the observation image are balanced. The aperture stop 908 is generally set to about 70% to 80% of the diameter of the pupil of the objective lens 904.

(Relationship between imaging unit and lighting unit)
The imaging unit 310 and the illumination unit 301 need to execute an imaging operation in synchronization. Therefore, the main control unit 321 issues an operation instruction to the imaging sensor control unit 1002 and the light source control unit 1102 so that the imaging timing and the illumination timing are synchronized. A color filter for acquiring RGB color plane data may be inserted into either the imaging unit 310 or the illumination unit 301. Therefore, any one of the filter wheel B910 and the filter wheel A909 may be incorporated in the imaging device 101.

(Contrast characteristics)
FIG. 12A to FIG. 12C are schematic diagrams for explaining the contrast characteristics of an object having a refractive index distribution. A region A1201 schematically shows a semi-transparent object and a region having a relatively high refractive index. On the other hand, a region B1202 schematically shows a semi-transparent object having a relatively low refractive index. For example, the refractive index of the region A1201 is 1.39, and the refractive index of the region B is 1.37.
Therefore, the refractive index of other regions may be considered as 1.38. FIG. 12A is an observation image when the Z position of the thickness center of the specimen (semi-transparent object) is on the + side with respect to the focal position 701. The positional relationship between the focal position 701 and the specimen (semi-transparent object) in FIG. 12A corresponds to FIG. A region A1201 having a relatively high refractive index becomes bright, and a region B1202 having a relatively low refractive index becomes dark. FIG. 12B is an observation image at the in-focus position. The positional relationship between the focal position 701 and the specimen (semi-transparent object) in FIG. 12B corresponds to FIG. At the in-focus position, an image is hardly obtained if the object is a translucent object with a small change in refractive index. FIG. 12C is an observation image when the Z position of the thickness center of the specimen (semi-transparent object) is on the − side with respect to the focal position 701. The positional relationship between the focal position 701 and the specimen (semi-transparent object) in FIG. 12C corresponds to FIG. A region A1201 having a relatively high refractive index becomes dark, and a region B1202 having a relatively low refractive index becomes bright.

  In FIGS. 12A to 12C, a translucent object is assumed in order to easily explain the contrast characteristics of an object having a refractive index distribution. Although it is stained in the pathological specimen, the contrast characteristics described with reference to FIGS.

  FIG. 13 shows the contrast characteristics of an object having a refractive index distribution. The vertical axis is the contrast, and the horizontal axis is the defocus amount d. Reference numeral 1301 denotes a contrast characteristic due to the influence of the transmittance distribution. The contrast becomes maximum when the defocus amount d = 0, and the contrast decreases as the absolute value of the defocus amount increases. A region that is particularly deeply stained in a pathological specimen, for example, chromatin in the nucleus stained with hematoxylin in HE (hematoxylin and eosin), shows characteristics close to 1301. Reference numeral 1302 denotes contrast characteristics due to the influence of the refractive index distribution. The absolute value of the contrast is maximized at two locations where the defocus amount d = | d ′ |. A region that is stained lightly in a pathological specimen, for example, cytoplasm stained with eosin of HE (hematoxylin and eosin), shows characteristics close to 1302.

  The contrast characteristics shown in FIG. 13 are also affected by a coherent factor (ratio of NA of the illumination lens 903C and NA of the objective lens 904). As the coherent factor approaches 0, the influence of the contrast characteristic of 1302 increases, and as the coherent factor approaches 1, the influence of the contrast characteristic of 1301 decreases. In a microscope, the coherent factor is generally set to about 0.7 to 0.8 so that the contrast and resolution of the observation image are balanced.

(Problems due to the influence of refractive index distribution)
The contrast characteristics of a pathological specimen stained with HE (hematoxylin and eosin) are in a state where the two contrast characteristics shown in FIG. 13 are mixed. A region that is darkly dyed and has low transmittance has a contrast characteristic close to 1301. A typical example is nuclear chromatin stained with hematoxylin. A region that is lightly dyed and has a high transmittance has a contrast characteristic close to 1302. A typical example is the cytoplasm stained with eosin.

  In this embodiment, it is assumed that the observer pays attention to the nucleus, and the focusing operation focuses on the chromatin in the nucleus stained with hematoxylin. That is, a position at which the contrast (hereinafter referred to as “amplitude contrast”) due to the influence of the transmittance distribution indicated by 1301 in FIG. Assuming that the object (nucleus) to be observed exists at the Z position of the thickness center of the specimen, the position where the thickness center of the specimen coincides with the focal position is the focus position. However, the concept of the in-focus position is not limited to this. In some cases, the object (nucleus) to be observed exists at a Z position shifted from the thickness center of the specimen. In that case, a position where the Z position of the object (nucleus) to be observed coincides with the focal position may be set as the in-focus position.

  In the present embodiment, the contrast (hereinafter referred to as “phase contrast”) due to the influence of the refractive index distribution indicated by 1302 in FIG. 13 is treated as an artifact. That is, the absolute value peak position of the phase contrast 1302 (two places where the defocus amount d = | d ′ |) is not a target to be detected as the focus position. The present embodiment provides a solution to the problem that the peak position detection of the amplitude contrast 1301 becomes unstable because the peak position due to the phase contrast 1302 exists.

  Although the object of interest is a nucleus here, it is not limited to the nucleus. When examining stomach inflammation, Giemsa-stained specimens are sometimes used, but this stain stains H. pylori blue. In this case, H. pylori is the object of interest. The object of interest varies depending on the purpose of pathology observation.

(Imaging flow in the first focus position detection method)
The in-focus position detection described with reference to FIGS. 14, 15A, and 15B mainly includes two stages. One is selection of a color plane used for in-focus position detection in pre-measurement. This is referred to as “pre-measurement processing related to focusing”. The other is focus position detection in actual imaging. This is referred to as “main imaging processing related to focusing”. Here, the main imaging is processing for acquiring a plurality of divided image data by the imaging sensor 308 in order to generate large-size image data in a desired imaging range.

  As the imaging flow, “pre-measurement processing related to focusing” is executed, and “main imaging processing related to focusing” and “main imaging processing” are sequentially executed for each divided image data. This is referred to as a “sequential imaging flow”. As another imaging flow, “pre-measurement processing related to focusing” is executed, then “main imaging processing related to focusing” is executed, and finally “main imaging processing” is executed collectively. This is referred to as a “collective imaging flow”.

  With reference to FIGS. 14, 15A, and 15B, the “pre-measurement process related to focusing” and the “main imaging process related to focusing” will be described.

(Focus position detection unit in the first focus position detection method)
FIG. 14 is a functional block diagram of the focus position detection unit in the first focus position detection method. The focus position detection unit 324 includes an absorption determination unit 1402, a focus color plane selection unit 1403, and a focus position determination unit 1404. The absorption determination unit 1402 and the in-focus color plane selection unit 1403 are functional blocks related to “pre-measurement processing related to in-focus”. The in-focus position determination unit 1404 is a functional block related to “main imaging processing related to in-focus”.

  The absorption determination unit 1402 acquires all color plane data from the pre-measurement unit 320 and compares the light transmittance of the color planes. A color plane with a large integrated luminance value of color plane data is determined to have a high light transmittance (absorption at a target color sample is small), and a color plane with a low integrated luminance value has a low light transmittance (target It is determined that there is a large absorption in the color sample. The integrated luminance value is a total value of pixel values of the color plane data (the pixel value itself or a luminance value converted from the pixel value) or a value corresponding thereto (for example, a value obtained by normalizing the total value with a predetermined value). It is. For calculation of the integrated luminance value, data of all layers may be used, or only data of a part of layers (for example, a layer at the center of the thickness of the sample) may be used. In addition, for the calculation of the integrated luminance value, the values of all the pixels of the color plane data may be used, or only the values of some pixels (for example, pixels other than the high luminance pixels corresponding to the background) are used. Also good. The determination result (the integrated luminance value of each color plane data or the magnitude relationship of absorption) is transmitted to the in-focus color plane selection unit 1403.

The in-focus color plane selection unit 1403 selects a color plane to be used for in-focus position detection from the determination result of the absorption determination unit 1402. If the determination result is the integrated luminance value of each color plane data, the color plane with the smallest integrated luminance value is selected as the focus position detection color plane. If the determination result is related to the magnitude of absorption, the color plane having the greatest absorption is selected as the focus position detection color plane.

  The focus position determination unit 1404 extracts color plane data for focus position detection from the image data of a plurality of layers in the sample acquired by the imaging sensor 308, and evaluates the degree of focus of each color plane data. Specifically, the focus position determination unit 1404 calculates the focus index value of each color plane data, and obtains the distribution (change) of the focus index value in the optical axis direction (depth direction). Then, the Z position at which the focus index value takes an extreme value is interpreted as the focus position, and the Z position that is the focus position is transmitted to the imaging apparatus 101 (main control unit 321) (sequential imaging flow). Alternatively, the focus position determination unit 1404 stores the Z position, which is the focus position, in the memory (collective imaging flow). Note that the focus position determination unit 1404 performs focus position detection processing for each of the divided image data of a plurality of tiles.

FIG. 15A is a flowchart showing a focus position detection flow in the first focus position detection method.
In step S1501, color plane data is acquired from the pre-measurement unit 320. When a two-dimensional imaging sensor with a Bayer array is used in the pre-measurement, a demosaicing process is performed to obtain RGB color plane data.

  In step S1502, the light transmittance of the color plane is compared. A color plane having a large integrated luminance value of the color plane data is determined to have a high light transmittance, and a color plane having a small integrated luminance value is determined to have a low light transmittance. Step S1502 is a process executed by the absorption determination unit 1402 of FIG.

  In step S1503, a color plane used for focus position detection is selected (focus color plane selection). The color plane determined to have the smallest light transmittance in step S1502 is set as the color plane used for in-focus position detection. A color plane having a small light transmittance can easily obtain the contrast 1301 (amplitude contrast) due to the influence of the transmittance distribution shown in FIG. 13, and is hardly affected by the contrast 1302 (phase contrast) due to the influence of the refractive index distribution. It is. Step S1503 is a process executed by the in-focus color plane selection unit 1403 in FIG.

  In steps S1504 and S1505, the color plane data at a specific Z position is acquired (S1504), and the process of calculating the focus index value (S1505) is repeated. The color plane of the color plane data used here is the in-focus position detection color plane selected in step S1503. The color plane data is color plane data of one divided image data imaged by the imaging sensor 308.

  The focus index value is an index that quantifies the degree of focus of an image. Any algorithm may be used to calculate the focus index value. For example, algorithms such as contrast and normalized square intensity sum, statistical algorithms such as autocorrelation and normalized variance, differential algorithms such as Brenner differentiation and Volath-5, and algorithms using histograms such as entropy can be used. The distribution shape of the focus index value in the optical axis direction (depth direction) will be described with reference to FIG.

  It is desirable that the number of loops be the minimum number of times that the distribution shape (approximate curve) of the focus index value can be drawn. The number of loops is set so that the processing speed and the detection accuracy of the focus position (amplitude contrast peak) are balanced. The number of loops is several to several tens.

In step S1506, the degree of focus is evaluated. The details of the focus degree evaluation will be described with reference to FIG.
Steps S1504, S1505, and S1506 are processes executed by the focus position determination unit 1404 in FIG.

FIG. 15B is a flowchart for explaining the evaluation of the degree of focus in the first focus position detection method.
In step S1507, extreme values are detected from the distribution of the focus index values. For the extreme value detection, a general method such as setting the extreme value at the position where the sign of the first-order derivative is inverted can be applied. If the focus index value is unstable and not a smooth curve, a plurality of extreme values are detected. However, the extreme values can be uniquely detected by each decision such as selecting the maximum extreme value. The focus position determination unit 1404 determines the extreme Z position detected in step S1507 as the focus position.

  In step S1508, threshold determination processing is performed to determine whether the extreme value detected in step S1507 is within the threshold. A plurality of pieces of divided image data are acquired by the image sensor 308. The extreme value of the focus index value obtained from each piece of divided image data is assumed to exist within a certain range, and the range is set as a threshold value. For example, the extreme value of the focus index value obtained from adjacent divided image data is set to be within 2 micrometers. If it is determined by the threshold determination process that the extreme value is not within the threshold (step S1508: NO), the extreme value is detected again in step S1507, excluding the extreme value. If it is determined by the threshold determination process that the extreme value falls within the threshold (step S1508: YES), the process proceeds to step S1509.

  In step S1509, the focus position detection unit 324 sequentially transmits the extreme value (Z position of the focus position) detected in step S1507 to the main control unit 321 as focus position information. Alternatively, when one sample is divided into a plurality of tiles for imaging, the focusing position detection unit 324 temporarily stores in-focus position information obtained from the divided imaging data of each tile in the memory, and the plurality of tiles The in-focus position information may be collectively transmitted to the imaging apparatus 101. When the in-focus position information for each tile is sequentially transmitted, the in-focus position determination of the divided image data and the image acquisition (main image capturing) at the in-focus position are sequentially executed (sequential image capturing flow). When the in-focus position information of a plurality of tiles is stored in the memory, the image acquisition at the in-focus position is performed collectively after determining the in-focus positions of all the tiles (collective imaging flow). Any processing method may be used. In the case of the latter processing method, after the in-focus position information (extreme value of the in-focus index value) of all tiles is stored in the memory, outlier removal, smoothing processing, etc. are performed to perform a plurality of in-focus operations. You may perform the post-process which makes the in-focus surface comprised by a focus position a smooth surface.

  The focus position detection flow described with reference to FIGS. 15A and 15B is processing for one piece of divided image data captured by the image sensor 308. The processing from step S1504 to S1509 is executed for the divided image data of all tiles. In the focus position detection flow described with reference to FIGS. 15A and 15B, one focus index value is calculated from the layer image data (color plane data) of one divided image data. However, the present invention is not limited to this, and the divided image data may be further divided into a plurality of subtiles, and the focus index value may be calculated for each subtile in the layer image data (color plane data).

(Focus position detection by coherent factor control)
Furthermore, the accuracy of in-focus position detection can be increased by making the coherent factors different between the “main imaging process related to focusing” and the “main imaging process”. As described with reference to FIG. 13, as the coherent factor approaches 1, the contrast 1302 (phase contrast) due to the influence of the refractive index distribution becomes difficult to express. Since the coherent factor in the “main imaging process” is set to about 0.7 to 0.8, the coherent factor in the “main imaging process related to focusing” is set larger than that in the “main imaging process”. Thereby, the influence of the contrast 1302 (phase contrast) due to the influence of the refractive index distribution which is an artifact can be suppressed. That is, more stable in-focus position detection is possible. In order to enable control of the coherent factor, an illumination unit (illumination optical system) capable of changing the coherent factor is used in the present embodiment. Specifically, the control of the coherent factor can be realized by controlling the aperture stop 908 by the aperture stop changing unit 1105.

(Imaging flow in the second focus position detection method)
The in-focus position detection described with reference to FIGS. 16 and 17 is roughly composed of two stages as described above. One is selection of a color plane used for focus position detection in pre-measurement (pre-measurement processing related to focus). The other is focus position detection in actual imaging (main imaging processing related to focusing).

  There are two imaging flows, a sequential imaging flow and a batch imaging flow, as described above. 16 and 17, “pre-measurement process related to focusing” and “main imaging process related to focusing” will be described.

  In FIG. 14 and FIG. 15A, the light transmittance comparison of the color plane is performed using the image data acquired by the pre-measurement. In FIGS. 16 and 17, the light transmittance comparison of the color plane is performed by the pre-measurement. This is performed using the acquired label information (staining information). In the imaging flow in the second focus position detection method, it is not necessary to acquire image data, and the imaging flow is simpler and simpler.

(Focus position detection unit in the second focus position detection method)
FIG. 16 is a functional block diagram of the focus position detection unit in the second focus position detection method. The focus position detection unit 324 includes a staining acquisition unit 1401, an absorption determination unit 1402, a focus color plane selection unit 1403, and a focus position determination unit 1404. The staining acquisition unit 1401, the absorption determination unit 1402, and the in-focus color plane selection unit 1403 are functional blocks related to “pre-measurement processing related to in-focus”. The in-focus position determination unit 1404 is a functional block related to “main imaging processing related to in-focus”.

  The staining acquisition unit 1401 acquires staining information from the main control unit 321. The staining information is information for specifying staining (for example, HE (hematoxylin / eosin) staining) applied to the specimen.

  The absorption determination unit 1402 acquires staining information from the staining acquisition unit 1401 and compares the light transmittances of the color planes. The absorption determination unit 1402 holds in advance a table (see FIG. 18) in which the staining method and the transmittance of the color filter are associated with each other, and determines the magnitude relationship of the light transmittance on the color plane according to the table. The determination result (relationship between light transmittances in the color plane) is transmitted to the in-focus color plane selection unit 1403. However, in the second focus position detection method, the absorption determination unit 1402 is not an essential configuration. If a table in which the staining method is associated with the color plane to be selected (not the table shown in FIG. 18) is held, the in-focus color plane selection unit is based on the staining information acquired by the staining acquisition unit 1401. At 1403, the in-focus color plane can be selected.

The in-focus color plane selection unit 1403 selects a color plane to be used for in-focus position detection from the determination result of the absorption determination unit 1402. The color plane that is determined to have the lowest light transmittance (the absorption of the target color sample is large) is taken as the color plane for detecting the in-focus position. The function of the focus position determination unit 1404 is the same as that of the first focus position detection method.

FIG. 17 is a flowchart showing a focus position detection flow in the second focus position detection method.
In step S1701, staining information is acquired from the main control unit 321. In the pre-measurement, the pre-measurement unit 320 reads the staining information from the label area 504 and transmits it to the main control unit 321. The main control unit 321 transmits the staining information to the focus position detection unit 324. When the actual data of the staining information is stored in a database such as an external server instead of the label area 504, for example, the ID information is read from the label area 504, and the staining information is acquired from the database based on the ID information. There is also a possibility. Alternatively, the user may input staining information (for example, HE staining) via the image processing apparatus 102 and acquire the staining information. Alternatively, the user may input ID information via the image processing apparatus 102 and acquire staining information from a database based on the ID information.

  In step S1702, the light transmittance of the color plane is compared. The magnitude relationship between the light transmittances in the color plane is determined according to a table (see FIG. 18) in which the staining method and the color filter transmittance are associated with each other. Step S1702 is a process executed by the absorption determination unit 1402 of FIG. However, this step S1702 is not an essential configuration. What is necessary is just to hold | maintain the table which matched the dyeing | staining method and the information of the color plane which should be selected (not the table shown in FIG. 18). If such a table is used, a focused color plane can be selected in step S1703 based on the staining information acquired in step S1701.

In step S1703, selection of a color plane used for focus position detection (focus color plane selection) is performed. The color plane determined to have the smallest light transmittance in step S1702 is set as a color plane used for focus position detection. A color plane having a small light transmittance can easily obtain the contrast 1301 (amplitude contrast) due to the influence of the transmittance distribution shown in FIG. 13, and is hardly affected by the contrast 1302 (phase contrast) due to the influence of the refractive index distribution. It is. This step S1703 is processing executed by the in-focus color plane selection unit 1403 in FIG.
Steps S1504, S1505, and S1506 are the same as those described with reference to FIGS. 15A and 15B.

  Note that the processing flow in FIG. 17 is an example of a method for selecting a color plane for detecting a focus position based on staining information. For example, a table associating a staining method with a color plane to be selected, a table associating a staining method with a color or hue that a staining specimen normally has, a table associating a staining method with a color filter to be selected, etc. It may be used. As the processing of the in-focus color plane selection unit 1403, what is the processing that specifies the staining of the specimen based on the staining information and selects the color plane that has the closest hue to the normal color of the stained specimen It may be anything.

(Absorption determination table)
FIG. 18 is a schematic diagram illustrating a table used in the absorption determination unit in the second focus position detection method. This table is a table in which the staining method (staining type) and the color filter transmittance are associated with each other. The transmittance of the color filter can be determined, for example, by measuring the transmittance of a large number of stained specimens for each RGB and obtaining an average value. In the case of a HE-stained specimen, the transmittances in arbitrary regions with respect to R (center wavelength 0.7000 μm), G (center wavelength 0.5461 μm), and B (center wavelength 0.4358 μm) are about 96% and about 73%, respectively. It is about 97%. Examples of the staining method include HE staining, Masson trichrome staining, Giemsa staining, and the like. In the table, the transmittance of each color filter of R, G, B is described for each staining method.

(Treatment of slides with different staining)
The first and second focus position detection methods can be performed for each slide. Therefore, for a plurality of slides subjected to different staining, it is possible to perform in-focus position detection by selecting a color plane of a color suitable for detecting the in-focus position of the staining.

(Focus position detection by coherent factor control)
Up to this point, it was assumed that a color plane of a color suitable for focus position detection was used, but this was not premised, and the coherent factor was determined by `` main imaging processing related to focusing '' and `` main imaging processing ''. By making it different, the accuracy of in-focus position detection can be increased. For example, the control of the coherent factor can be realized by controlling the aperture stop 908 by the aperture stop changing unit 1105.

(Another configuration example of the imaging device)
19 and 20 show another configuration example of the imaging apparatus in the imaging system. In the first and second in-focus position detection methods, image data used for in-focus position detection is acquired by the image sensor 308 in the same manner as the “main imaging process”. Here, as another configuration example, a configuration will be described in which image data used for focus position detection is acquired by a focusing imaging unit that is different from the imaging sensor 308 of “main imaging processing”.

  FIG. 19 shows a configuration in which a focusing imaging unit 1901 is added to the imaging apparatus of FIG. The in-focus imaging unit 1901 (second imaging unit) is a unit that combines the functions of the imaging unit 310 (first imaging unit) and the development processing unit 319 used for actual imaging. The image data used for focus position detection is acquired by the focus imaging unit 1901, and the sensor function can be specialized for focus position detection. For example, it is possible to increase the speed by making the resolution (number of pixels) of the sensor smaller than that of the imaging sensor 308 of the “main imaging process”. Also, the optical systems used in the “main imaging process” and the “imaging process for detecting the focus position” are different (the optical system from the illumination unit 301 to the imaging unit 310 is focused from the first optical system and the illumination unit 301). The optical system up to the imaging unit 1901 is referred to as a second optical system). Therefore, a configuration in which the coherent factor of the second optical system is increased as compared with the first optical system can be adopted. In this way, by optimizing the coherent factors for “imaging processing for in-focus position detection” and “main imaging processing”, stable in-focus position detection and observation image that balances contrast and resolving power are acquired. Can do.

  FIG. 20 shows a configuration in which a focusing imaging unit 1901 and an illumination unit 301A are added to the imaging apparatus of FIG. FIG. 19 shows a configuration in which the optical path is divided by a half mirror. FIG. 20 shows a configuration in which the in-focus position detection and the “main imaging processing” optical configurations are independent of each other. In this case, the optical system from the illumination unit 301 to the imaging unit 310 is the first optical system, and the optical system from the illumination unit 301A to the in-focus imaging unit 1901 is the second optical system. This configuration has the same advantages as the configuration of FIG. Also, by optimizing the coherent factors for “imaging processing for in-focus position detection” and “main imaging processing”, stable in-focus position detection and observation images that balance contrast and resolution can be acquired. . Furthermore, there is an advantage that it is not necessary to change the aperture stop 908 mechanically.

(effect)
FIG. 21A and FIG. 21B are schematic diagrams for explaining the focus index value of an object having a refractive index distribution. FIG. 13 shows an ideal contrast characteristic of an object having a refractive index distribution, but it is not possible to acquire this ideal contrast characteristic from image data due to mechanical errors and electrical noise at the time of image data acquisition. Have difficulty. For this reason, a focus index value serving as a substitute for an ideal contrast characteristic is acquired by a specific algorithm. The focus index value in the present embodiment is an index indicating the degree of focus of an image, and is synonymous with contrast (in a broad sense) in a general sense. Examples of algorithms are algorithms such as contrast and normalized square strength sum, statistical algorithms such as autocorrelation and normalized variance, differential algorithms such as Brenner differentiation and Volath-5, and algorithms using histograms such as entropy. What can be obtained from each of the listed algorithms is a narrow sense of contrast. FIG. 21A shows a focus index value when the contrast 1302 (phase contrast) due to the influence of the refractive index distribution is acquired by a certain algorithm. FIG. 21B shows the focus index value when the contrast 1301 (amplitude contrast) due to the influence of the transmittance distribution is acquired by a certain algorithm. As described above, the contrast 1302 (phase contrast) due to the influence of the refractive index distribution in FIG. 13 can be suppressed by appropriately selecting the color filter of the image data used for in-focus position detection. That is, the shape of the focus index value is a shape close to FIG. 21B instead of FIG.

  It should be noted here that in the present invention, the color plane data corresponding to the color having the greatest absorption by the sample is used for focus position detection, but the focus index value of the color plane data is the focus of other color plane data. The point is that it is not always larger than the index value. That is, the focus index value obtained from the color plane data corresponding to a color having a large absorption in the sample may be smaller than the focus index value obtained from the other color plane data. The reason why a color having a large absorption in the sample is selected is that it is not easily affected by the phase difference of the sample (it is difficult to achieve a bimodal contrast such as 1302 in FIG. 13), and the focus index value (contrast). It is not the reason that is big. The present invention is different from the conventional idea that the focus position is stably detected by detecting the focus position with color plane data having a large focus index value (contrast). The present invention is based on the idea that the in-focus position is stably detected by suppressing the influence of the phase difference of the specimen.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101: Imaging device 324: Focus position detection unit 501: Sample 1403: Focus color plane selection unit 1404: Focus position determination unit

Claims (16)

An in-focus position detecting device that detects an in-focus position of the sample using image data of a plurality of layers in the sample, which is imaged while changing a focal position in the optical axis direction with respect to the sample,
The image data is color image data composed of data of a plurality of color planes,
The in-focus position detecting device is
A selection unit that selects a color plane corresponding to a color that is most absorbed by the sample among the plurality of color planes as a color plane for in-focus position detection;
An in-focus position determining means for determining the in-focus position of the sample by evaluating the in-focus degree of the data of the color plane for detecting the in-focus position of each of the plurality of layers;
An in-focus position detecting device comprising: The focus position determination unit calculates a focus index value from data of the focus plane detection color plane of each of the plurality of layers, and the focus index value is an extreme value in the distribution of the focus index value. The focus position detection apparatus according to claim 1, wherein a position in the optical axis direction that takes the position is determined as a focus position. The in-focus position detecting device according to claim 2, wherein the in-focus index value is a contrast of data of the color plane for detecting the in-focus position. Absorption determination means for determining a color having the greatest absorption by the sample from a plurality of colors corresponding to each of the plurality of color planes based on luminance values of the color planes of the image data of the plurality of layers. The in-focus position detecting device according to claim 1, wherein the in-focus position detecting device is provided. The absorption determination unit calculates an integrated luminance value of each color plane of the image data of the plurality of layers, and determines a color corresponding to a color plane having the smallest integrated luminance value as a color having the largest absorption by the sample. The in-focus position detecting device according to claim 4. Further comprising staining acquisition means for acquiring staining information for specifying staining performed on the specimen;
The said selection means selects the color plane for the said focus position detection from these color planes based on the said dyeing | staining information, The any one of Claims 1-3 characterized by the above-mentioned. In-focus position detection device. 7. The selection unit according to claim 6, wherein the selection unit selects a color plane having a hue closest to a sample subjected to staining specified by the staining information as the color plane for detecting the in-focus position. In-focus position detection device. The specimen is placed on a slide glass,
The dyeing information is recorded on a label provided on the slide glass, or is acquired from a database based on information recorded on the label. In-focus position detection device. An imaging unit for imaging the specimen;
A control unit for controlling the imaging unit;
An in-focus position detecting device according to any one of claims 1 to 8, comprising:
The control unit acquires information on a focus position determined by the focus position detection device, and performs control for acquiring image data of the sample at the focus position by the imaging unit. Imaging device. Before the control to acquire the image data of the specimen at the in-focus position is performed,
The control unit performs control to acquire image data of a plurality of layers in the sample by the imaging unit,
The imaging apparatus according to claim 9, wherein the in-focus position detecting apparatus determines an in-focus position of the specimen using image data of the plurality of layers acquired by the imaging unit. It has an optical system that can change the coherent factor,
The control unit is coherent when the image data of the plurality of layers in the sample used by the in-focus position detection device acquires image data of the sample at the in-focus position. The image pickup apparatus according to claim 9 or 10, wherein the optical system is controlled to be larger than a factor. A first optical system and a second optical system having a coherent factor larger than that of the first optical system,
When acquiring image data of the plurality of layers in the specimen used by the in-focus position detection device, the second optical system is used,
The imaging apparatus according to claim 9 or 10, wherein the first optical system is used when acquiring image data of the specimen at the in-focus position. The imaging unit includes a first imaging unit and a second imaging unit having a resolution smaller than that of the first imaging unit,
When acquiring image data of the plurality of layers in the specimen used by the in-focus position detection device, the second imaging unit is used,
The imaging apparatus according to any one of claims 9 to 12, wherein the first imaging unit is used when acquiring image data of the specimen at the in-focus position. The imaging apparatus according to claim 9, wherein the imaging apparatus is a microscope. An in-focus position detection method for detecting the in-focus position of the sample using image data of a plurality of layers in the sample, which is imaged while changing the focal position in the optical axis direction with respect to the sample,
The image data is color image data composed of data of a plurality of color planes,
The in-focus position detection method is:
Selecting a color plane corresponding to a color having the largest absorption by the sample among the plurality of color planes as a color plane for in-focus position detection;
Determining the in-focus position of the sample by evaluating the in-focus degree of the color plane data for detecting the in-focus position of each of the plurality of layers;
A focus position detection method characterized by comprising: The imaging device according to any one of claims 9 to 14,
An image processing device for processing an image acquired by the imaging device;
An imaging system comprising:

Images