JP2013238483A - Virtual slide device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable color correction to be performed with higher accuracy.SOLUTION: An imaging element 107 picks up an image formed on a first plane and acquires imaged data of an image of a first region. A color sensor 109 acquires spectral information of light guided to a second plane. A composition distribution detection unit 113 acquires composition distribution information of a sample 101 included in the first region and included in a third region including a second region on the basis of the image data acquired by the imaging element 107. A storage unit 114 stores a plurality of types of standard data items used in color correction of the image data. A standard data selection unit 115 selects a standard data item used in color correction from among the standard data items stored in the storage unit 114 on the basis of the composition distribution information of the sample 101 acquired by the composition distribution detection unit 113. An image file generation unit 111 performs color correction of the image data on the basis of the standard data item selected by the standard data selection unit 115 and the spectral information acquired by the color sensor 109.

Description

  The present invention relates to a virtual slide device.

  Patent Document 1 describes a technique for improving color reproducibility in a virtual slide device that digitally captures a slide image of a pathological specimen. Hereinafter, the virtual slide device described in Patent Document 1 will be described.

  FIG. 3 is a schematic diagram illustrating a configuration of a conventionally known virtual slide apparatus 1000. In the illustrated example, the light from the light source 1001 passes through the sample 1002 disposed on the stage 1005, passes through the objective lens 1003, and is branched by the half mirror 1009. Incident on sensor 1010. The other light branched by the half mirror 1009 is incident on the digital photographing apparatus 1004.

  The photographing control unit 1006 controls the stage 1005, the digital photographing device 1004, and the color sensor 1010 for photographing a partial image of the sample 1002. Specifically, the image capturing control unit 1006 controls the digital image capturing apparatus 1004 to capture an image while moving the stage 1005, and at the same time, the color sensor 1010 acquires spectrum information. Note that the imaging control unit 1006 controls the digital imaging apparatus 1004 to capture an image while moving the stage 1005 (sample 1002) so that a margin between tilings can be made.

  The image combining information generating unit 1007 performs image processing on the partial image input from the digital photographing apparatus 1004 to recognize the margin, and generates image combining information. The image file generation unit 1008 corrects the color of the image captured by the digital image capturing device 1004 based on the output value of the color sensor 1010 and the standard data stored in advance. For example, the color of the pathological sample may vary from sample to sample depending on the degree of staining liquid staining. Therefore, by performing color correction using the output value of the color sensor 1010 and standard data, the color of the image captured by the digital imaging device 1004 can be corrected appropriately.

JP 2011-99823 A

  However, the spectral information acquisition visual field of the color sensor included in the conventionally known virtual slide device is, for example, an imaging region of the digital imaging means in order to separately acquire spectral information such as the cell nucleus, cytoplasm, and cavity of the pathological sample. As small as a few percent. Furthermore, the spectral information acquisition visual field of the color sensor is fixed, for example, in the vicinity of the center of the image of the digital imaging apparatus, and it is not possible to arbitrarily select and acquire the spectral information of a specific tissue included in the pathological sample. Therefore, when performing color correction from the acquired spectrum information, there is a problem that the accuracy of color correction is not sufficient depending on the sample.

  The present invention has been made to solve the above problems, and an object thereof is to provide a virtual slide device capable of performing color correction with higher accuracy.

  According to the present invention, a stage on which a sample is set and the sample is sequentially presented in an imaging region, a light source that illuminates the sample, and an image of a first region included in the entire region of the sample are connected to a first plane. An optical system configured to image and introduce light from a second region included in the first region into a second plane; and an image formed on the first plane is captured; and the first region An image sensor that acquires image data of the image of the image, a color sensor that acquires spectral information of light introduced into the second plane, and the first region based on the image data acquired by the image sensor. A composition distribution acquisition unit for acquiring composition distribution information of the sample included in the third region including the second region, and a standard used for color correction of the image data corresponding to the distribution state of the composition Memorize multiple types of data A standard data selection unit that selects the standard data used for the color correction among the standard data stored in the storage unit, based on the composition distribution information of the sample acquired by the component distribution acquisition unit, A virtual slide device comprising: a correction processing unit that performs color correction of the image data based on the standard data selected by a standard data selection unit and the spectral information acquired by the color sensor.

  In the virtual slide device of the present invention, the composition distribution acquisition unit is included in the first region based on thinned image data obtained by thinning predetermined pixels from the image data acquired by the imaging element. The composition distribution information of the sample included in the third region including the region is acquired.

  In the virtual slide device of the present invention, the imaging element includes an R pixel that detects red light, a G pixel that detects green light, and a B pixel that detects blue light, and obtains the composition distribution. The unit is included in the third region including the second region and included in the first region based on the output of any one of the R pixel, the G pixel, and the B pixel. The composition distribution information of the sample is acquired.

  In the virtual slide device of the present invention, the third area is the same area as the first area.

  In the virtual slide device of the present invention, the storage unit includes the standard data corresponding to a state where cell nuclei are distributed in the sample, and the standard data corresponding to a state where cytoplasm is distributed in the sample. In addition, at least one standard data among the standard data corresponding to a state in which no tissue is distributed in the sample is stored.

  According to the present invention, the stage installs samples and sequentially presents the samples in the imaging area. The light source illuminates the sample. The optical system forms an image of the first region included in the entire region of the sample on the first plane, and introduces light from the second region included in the first region into the second plane. Let The imaging device captures an image formed on the first plane and acquires image data of the image in the first region. Further, the color sensor acquires spectral information of light introduced into the second plane. The composition distribution acquisition unit acquires the composition distribution information of the sample included in the third region that is included in the first region and includes the second region, based on the image data acquired by the imaging element. The storage unit stores a plurality of types of standard data used for color correction of image data corresponding to the distribution state of the composition. The standard data selection unit selects the standard data used for color correction from the standard data stored in the storage unit based on the sample composition distribution information acquired by the composition distribution acquisition unit. The correction processing unit performs color correction of the image data based on the standard data selected by the standard data selection unit and the spectrum information acquired by the color sensor.

  Thereby, since color correction of image data can be performed based on the standard data corresponding to the composition distribution of the sample and the spectrum information acquired by the color sensor, color correction can be performed with higher accuracy.

It is the schematic which showed the structure of the virtual slide apparatus in one Embodiment of this invention. In this embodiment, it is the schematic which showed the example of the thinning image data which a thinning image production | generation part produces | generates. It is the schematic which showed the structure of the conventionally known virtual slide apparatus.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram illustrating a configuration of a virtual slide device 1 according to the present embodiment. In the illustrated example, the virtual slide device 1 includes a stage 102, a stage driving unit 103, a light source 104, an objective lens 105, a half mirror 106, an imaging element 107 (imaging unit), and a visual field region changing unit 108. , Color sensor 109, imaging control unit 110, image file generation unit 111 (correction processing unit), thinned image generation unit 112, composition distribution detection unit 113 (composition distribution acquisition unit), storage unit 114, standard A data selection unit 115 and a bonding information generation unit 116 are provided. For example, the objective lens 105 and the half mirror 106 correspond to the optical system according to the claims.

  The stage 102 is a stage for placing a sample on which a sample 101 is placed on a slide glass. The stage driving unit 103 drives the stage 102 in a horizontal direction and a vertical direction (three-dimensional direction) with respect to the imaging surface of the imaging element 107 based on the control of the imaging control unit 110. The light source 104 generates light that irradiates the sample 101.

  The objective lens 105 is an objective lens used for enlarging and capturing an image (partial image) of a partial region (first region) of the sample 101. The objective lens 105 is disposed so as to face the sample 101, collects a light beam from a predetermined region of the sample 101, and irradiates the collected light to the half mirror 106. The half mirror 106 reflects a part of the light from the objective lens 105 in the direction of the visual field region changing unit 108 and transmits a part thereof.

  The image sensor 107 is disposed at a position for receiving the light transmitted through the half mirror 106, captures an image by photoelectric conversion into an electrical signal corresponding to the intensity of the received light, and acquires image data. Since the light incident on the image sensor 107 is a partial region of the sample 101, the image data acquired by the image sensor 107 is image data of the partial region (first region) of the sample 101. The image sensor 107 includes a Bayer array color filter. Thereby, among the pixels of the image sensor 107, the pixel (R pixel) in which the color filter that transmits red light is arranged detects red light. Of the pixels included in the image sensor 107, a pixel (G pixel) in which a color filter that transmits green light is disposed detects green light. Of the pixels included in the image sensor 107, a pixel (B pixel) in which a color filter that transmits blue light is disposed detects blue light.

  The field-of-view area changing unit 108 is disposed between the half mirror 106 and the color sensor 109, and blocks a part of the light from the half mirror 106, and a part of the area (second area). ). The light that has passed through the visual field region changing unit 108 enters the color sensor 109. With this configuration, the visual field range of the color sensor 109 becomes the second region.

  The color sensor 109 detects spectral information of light transmitted through the visual field region changing unit 108 out of light from the sample 101. That is, the color sensor 109 detects the spectrum information of the second region. The second area is an area included in the first area. The imaging control unit 110 controls the stage driving unit 103, the objective lens 105, the imaging element 107, and the color sensor 109, and obtains image data of the first region of the sample 101 and spectral information of the second region. get. Specifically, the imaging control unit 110 controls the stage driving unit 103 to move the stage 102 so that the imaging element 107 captures an image, and at the same time, the color sensor 109 acquires spectral information. The imaging control unit 110 controls the imaging element 107 to capture an image (partial image) while moving the stage 102 (sample 101) so that a margin between tilings is possible.

  The thinned image generation unit 112 thins out information on some pixels from the image data captured by the image sensor 107 and generates thinned image data. For example, the thinned-out image generation unit 112 thins out information on pixels included in the peripheral area of the image data captured by the image sensor 107, and the pixels included in the central area (near the visual field range of the color sensor 109). Thinned image data including only information may be generated. In addition, for example, the thinned image generation unit 112 thins out pixel information from the image data captured by the image sensor 107 at predetermined intervals, and includes the entire information of the image data, while reducing the information amount of the image data. May be generated.

  In addition, the thinned-out image generation unit 112 thins out signals output from some pixels among signals output from R pixels, signals output from G pixels, and signals output from B pixels, for example. Thinned image data including only a signal output from the image may be generated. Specifically, when the image sensor 107 has a Bayer array color filter, for example, the signal output from the R pixel and the signal output from the B pixel are thinned out, and only the signal output from the G pixel is used. Thinned image data may be generated.

  In addition, the thinned image generation unit 112 may average the signals output from the R pixel, the G pixel, and the B pixel, and generate thinned image data using the signal obtained by the averaging. Specifically, when the image sensor 107 has a Bayer array color filter, for example, the pixel values of four pixel arrays (R pixel, G pixel, G pixel, B pixel) of the Bayer array are added and averaged. Then, the thinned image data may be generated by a method of setting the value of one pixel. Further, the thinned image generation unit 112 may generate thinned image data by combining the above-described plurality of methods.

  FIG. 2 is a schematic diagram illustrating an example of the thinned image data generated by the thinned image generation unit 112 in the present embodiment. FIG. 2A is a schematic diagram illustrating an example of image data captured by the image sensor 107. In the example illustrated, the image data captured by the image sensor 107 includes a signal output from the R pixel, a signal output from the G pixel, and a signal output from the B pixel. FIG. 2B is a schematic diagram illustrating the thinned image data generated by the thinned image generation unit 112 using only the signal output from the G pixel by thinning out the signal output from the R pixel and the signal output from the B pixel. FIG. In the illustrated example, the thinned image data generated by the thinned image generation unit 112 includes only a signal output from the G pixel. Further, the area of the thinned image data illustrated in FIG. 2B is the same area as the area of the image data captured by the image sensor 107.

  In FIG. 2 (3), the thinned image generation unit 112 displays information on pixels included in the peripheral area of the image data captured by the image sensor 107, a signal output from the R pixel, and a signal output from the B pixel. It is the schematic which showed the thinning image data produced | generated using only the information of G pixel contained in the area | region of the thinning | decimation and the center part (near the visual field range of the color sensor 109). In the illustrated example, the thinned image data generated by the thinned image generation unit 112 includes only a signal output from the G pixel. In addition, the area of the thinned image data illustrated in FIG. 2B is a central area of the image data area captured by the image sensor 107.

  Returning to the description of FIG. The composition distribution detection unit 113 is included in the partial region (first region) of the sample 101 based on the thinned image data generated by the thinned image generation unit 112, and the visual field range (second region) of the color sensor 109. A tissue distribution (composition distribution) included in the third region, which is the included region, is detected. Hereinafter, a method for detecting the composition distribution will be described.

(1) Method of detecting composition distribution based on the shape of tissue included in sample 101 Based on the thinned image data generated by the thinned image generating unit 112, the contrast value of the thinned image is calculated. If the contrast value is medium, it is determined that the cytoplasm is distributed. If the contrast value is low, it is determined that the cavity or the background (slide) is distributed.

(2) Method of detecting composition distribution based on tissue color included in sample 101 Based on the thinned image data generated by the thinned image generation unit 112, blood vessels or blood are distributed if the color of the thinned image is close to red. If it is close to dark purple, it is determined that cell nuclei are distributed, if it is close to pink, it is determined that cytoplasm is distributed, and if it is close to no color and transparent, the background or cavity is distributed. Is determined. When this method is used, the thinned image data generated by the thinned image generation unit 112 is thinned image data in which color information is not thinned.

(3) Method of detecting composition distribution based on tissue texture (both color and shape) contained in sample 101 A) Detecting changes in pathology of tissues such as cancer a) Cancer cells are circular in nucleus The degree is lost and becomes distorted. Furthermore, it becomes darker and darker than the normal cell nucleus color. Therefore, the color information distribution (histogram) of the pixel values of the thinned image generated by the thinned image generation unit 112 and the circularity of the cell nucleus are calculated by pattern recognition, and whether normal cells or cancer cells are distributed. judge.
b) Based on the abnormality of the composition distribution, it is determined whether normal cells or cancer cells are distributed. Specifically, pattern recognition processing is performed based on the thinned image generated by the thinned image generation unit 112, and the number of cell nuclei included in the thinned image is calculated. When the number of cell nuclei within a certain area is normal, it is determined that normal cells are distributed, and when the number of cell nuclei is abnormal, it is determined that cancer cells are distributed.
B) Detecting based on characteristics of pathological samples such as liver, spleen, lung, etc. The liver has a cell nucleus and cytoplasm that are not sparse and dense, has almost no cavity, and has a color close to pink. The spleen tends to concentrate cell nuclei and is close to a deep purple color. The lungs tend to be whitish with more cavities and less cytoplasm. Therefore, based on the thinned image generated by the thinned image generating unit 112, the state of distribution of cell nuclei and cytoplasm is extracted by pattern recognition, color information is acquired, and distributed tissue (composition distribution) is determined.

  Returning to the description of FIG. The storage unit 114 stores a plurality of types of standard data based on the distributed tissue (composition distribution). For example, an HE staining solution is generally used as the staining, but the color changes depending on the ratio and amount of the H staining solution and the E staining solution, the thickness of the pathological sample, and the like. Therefore, in order to correct the standard staining state even when the color at the time of staining varies depending on the conditions, there is a pathological sample in advance corresponding to those classified by each condition (site, composition, disease state, etc.) The spectrum information acquired in step 1 is stored as standard data. When the virtual slide device 1 captures an image, based on the composition distribution information of the sample 101, the standard data used for color correction is selected from the standard data stored in the storage unit 114, and the spectrum of the closest standard data is selected. Image color correction is performed to match the information. In the storage unit 114, for example, standard data when cell nuclei are distributed, standard data when cytoplasm is distributed, and cavity or background (slide) are distributed (tissue is not distributed). Standard data of the case is stored. The standard data stored in the storage unit 114 is not limited to this, and standard data based on any composition distribution may be stored. Further, the storage unit 114 may store a plurality of types of standard data in advance, and may arbitrarily add standard data to the storage unit 114.

  The standard data selection unit 115 selects standard data used for image data correction from among a plurality of types of standard data stored in the storage unit 114 based on the composition distribution detected by the composition distribution detection unit 113. Specifically, for example, when the composition distribution detected by the composition distribution detection unit 113 is cytoplasm, the standard data selection unit 115 selects standard data based on the cytoplasm.

  The pasting information generation unit 116 recognizes adjacent partial images for each of the plurality of partial images captured by the image sensor 107, further recognizes a margin between adjacent partial images, and based on the recognized result. Then, all the partial images are combined to generate the combined information used for generating one whole image.

  The image file generation unit 111 performs color correction of the partial image captured by the image sensor 107 based on the spectrum information acquired by the color sensor 109 and the standard data selected by the standard data selection unit 115. Hereinafter, a color correction method for partial images will be described.

(1) A method for calculating a correction value based on a color difference between the standard data selected by the standard data selection unit 115 and the spectrum information acquired by the color sensor 109. Then, the correction value is calculated by calculating the difference of the spectrum information of the standard data. Then, the image file generation unit 111 corrects the image data captured by the image sensor 107 using the calculated correction value. Thereby, the image file generation unit 111 can correct the color of the image data captured by the image sensor 107 more accurately.
In addition, the method similar to the method described in Unexamined-Japanese-Patent No. 2011-99823 public information is used for the method of calculating a correction value using the difference of spectrum information and standard data, for example.

(2) A method for calculating a correction value based on the standard data selected by the standard data selection unit 115 and the type of staining liquid determined based on the spectrum information acquired by the color sensor 109. For example, pathology such as HE staining. The color distribution of a staining solution typically used in diagnosis is determined as H (hematoxylin) and E (iodine) as standard. The color distribution of the staining liquid is stored as standard data according to each state, and the correction value is calculated by comparing the spectrum information acquired by the color sensor 109 with the standard data. Then, the image file generation unit 111 corrects the image data captured by the image sensor 107 using the calculated correction value. Thereby, the image file generation unit 111 can correct the color of the image data captured by the image sensor 107 more accurately.

  In addition, the image file generation unit 111 generates a single sample image by combining all the partial images subjected to color correction based on the combination information generated by the combination information generation unit 116. Note that the image file generation unit 111 may perform color correction after generating a single sample image by combining partial images based on the combination information generated by the combination information generation unit 116.

  As described above, according to the present embodiment, the image sensor 107 generates image data of the sample 101. In addition, the thinned image generation unit 112 generates thinned image data based on the image data captured by the image sensor 107. Further, the composition distribution detection unit 113 detects the composition distribution included in the image data based on the thinned image data. In addition, the standard data selection unit 115 selects standard data that is optimal for image data correction based on the composition distribution detected by the composition distribution detection unit 113. In addition, the image file generation unit 111 performs color correction of the image data captured by the image sensor 107 based on the standard data selected by the standard data selection unit 115 and the spectrum information acquired by the color sensor 109.

  As a result, the color correction of the image data can be performed based on the standard data corresponding to the composition distribution of the sample 101 and the spectrum information acquired by the color sensor 109, so that the color correction can be performed with higher accuracy. .

  Although one embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes design and the like within a scope not departing from the gist of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Virtual slide apparatus, 101 ... Sample, 102 ... Stage, 103 ... Stage drive part, 104 ... Light source, 105 ... Objective lens, 106 ... Half mirror, 107 ... Image sensor 108... Field of view region changing unit 109... Color sensor 110... Imaging control unit 111 ... Image file generating unit 112. Composition distribution detection unit, 114 ... storage unit, 115 ... standard data selection unit, 116 ... bonding information generation unit

Claims (5)

A stage for installing the sample and sequentially presenting the sample in the imaging area;
A light source for illuminating the sample;
An optical system for forming an image of a first region included in the entire region of the sample on a first plane, and introducing light from the second region included in the first region into the second plane; ,
An image sensor that captures an image formed on the first plane and acquires image data of the image of the first region;
A color sensor for acquiring spectral information of light introduced into the second plane;
Based on the image data acquired by the imaging device, a composition distribution acquisition unit that acquires the composition distribution information of the sample included in the third region included in the first region and including the second region;
A storage unit that stores a plurality of types of standard data used for color correction of the image data corresponding to the distribution state of the composition;
Based on the composition distribution information of the sample acquired by the composition distribution acquisition unit, a standard data selection unit that selects the standard data used for the color correction among the standard data stored by the storage unit;
A correction processing unit that performs color correction of the image data based on the standard data selected by the standard data selection unit and the spectrum information acquired by the color sensor;
A virtual slide device comprising: The composition distribution acquisition unit is included in a third region that is included in the first region and includes the second region based on thinned image data obtained by thinning predetermined pixels from the image data acquired by the image sensor. The virtual slide device according to claim 1, wherein composition distribution information of the sample is acquired. The imaging device includes an R pixel that detects red light, a G pixel that detects green light, and a B pixel that detects blue light,
The composition distribution acquisition unit includes a third region that is included in the first region and includes the second region based on an output of any one of the R pixel, the G pixel, and the B pixel. The virtual slide device according to claim 1, wherein composition distribution information of the sample included in a region is acquired. The virtual slide device according to claim 1, wherein the third region is the same region as the first region. The storage unit includes the standard data corresponding to a state in which cell nuclei are distributed in the sample, the standard data corresponding to a state in which a cytoplasm is distributed in the sample, and a tissue not distributed in the sample. The virtual slide apparatus according to claim 1, wherein at least one of the standard data corresponding to a state is stored.

Images