WO2019039595A1 - Device and method for estimating cytodiagnostic category of cervix - Google Patents

Abstract

Provided is a novel system with which it is possible to easily estimate a cytodiagnosis classification for a cervical specimen prior to diagnosis by a pathologist. This device for estimating a cytodiagnosis category includes: a specimen image input unit 11 that inputs a specimen image of a stained specimen, the stained specimen being a cervical specimen that is stained by an MUC conjugate; a cell image extraction unit 12 that extracts a stained cell image from the specimen image; a cell shape classification unit 13 that classifies the stained cell in the cell image into a corresponding shape category on the basis of cell shape classification information, and determines, for the specimen image, an organizational pattern of the shape category of the stained cell in the cell image extracted from the specimen image; and a cytodiagnosis category estimation unit 14 that estimates a cytodiagnosis category for the stained specimen in the specimen image from the organizational pattern of the specimen image on the basis of cytodiagnosis category estimation information.

Description

Device and method for estimating cervical cytology category

The present invention relates to an estimation apparatus and estimation method for cervical cytology categories.

In cervical cancer screening, it is common to first perform cytology prior to diagnosis by a pathologist. This cytology is known as one of the few screening methods that has been shown to reduce mortality. In cytology, papain crow (Pap.) Stained specimens are generally prepared for specimens obtained by abrasion from the cervix. Then, a cytologist searches for epithelial cells of human papilloma virus (HPV) infection taking a characteristic form by microscopic observation of the stained specimen, and the degree of progression of precancerous lesions etc. Classified according to the system. Based on the classification result by the Bethesda system, the pathologist determines the risk of cervical cancer of the patient, and further performs a detailed examination such as a tissue diagnosis to make a final diagnosis. It is considered useful to treat patients by early detection of precancerous lesions by cytology before diagnosis by such a doctor.

In recent years, due to the age reduction of cervical cancer, the age and examination period for examinations will be every 20 years from the age of 20, and the examination rate is lower at around 20%, but the number of examinations is compared with 10 years ago And it has increased 2.5 times. In particular, the majority of cervical cancer is due to HPV infection caused by sexual intercourse, the tendency of sexual activity to become younger, and so on, it is expected that the screening for cervical cancer will further increase in the future.

However, the examination of cervical cancer by cytology is a manual judgment by a cytologist who is an expert, so labor and cost are required, and the cost of examination for the municipality is a heavy burden. Furthermore, the epithelial cells to be targeted for cytology may exhibit various forms depending on age, presence or absence of inflammation, hormonal environment, etc., and therefore, the determination may be difficult. In addition, since it is human power, it requires skill and judgment may differ depending on the cytologist.

Specifically, since 100,000 to 300,000 cells are usually present in the stained sample, it is impossible for the cytologist to confirm all the cells in the slide. It is possible. For this reason, the cytologist usually observes only the region of interest in the stained sample through a microscope to determine the form, and the time taken for this determination is also about 120 seconds per slide. is there. For this reason, there is a limit to the judgment and accuracy of the cytology category manually.

Therefore, the present invention aims to provide a new system and method capable of easily estimating cytological classification for cervical specimens, for example, prior to diagnosis of the possibility of cervical cancer by a pathologist. .

In order to achieve the above object, the estimation apparatus of the present invention is an estimation apparatus for cervical cytology category,
Sample image input unit,
Cell image extraction unit,
A cell shape classification unit, and a cytology category estimation unit;
The sample image input unit
Input the sample image of the stained sample,
The stained sample is a cervical sample stained by a MUC conjugate,
The cell image extraction unit
Extracting a cell image of stained cells from the sample image;
The cell shape classification unit
Based on cell morphology classification information in which cell morphology information is associated with each of two or more morphology categories, the stained cells in the cell image are classified into corresponding morphology categories,
The composition pattern of the morphological category of stained cells in the cell image extracted from the sample image is determined for the sample image;
The cytology category estimation unit
The sample image from the composition pattern of the sample image based on cytology category estimation information in which each of two or more cytology categories is associated with a composition pattern of a cervical morphological category belonging to each of the cytology categories It is characterized in that a cytology category is estimated for a stained sample of

The estimation method of the present invention is an estimation method of cytology category of cervix,
Sample image input process,
Cell image extraction process,
Including a cell shape classification step and a cytology category estimation step;
In the sample image input process,
Input the sample image of the stained sample,
The stained sample is a cervical sample stained by a MUC conjugate,
The cell image extraction step
Extracting a cell image of stained cells from the sample image;
The cell shape classification step
Based on cell morphology classification information in which cell morphology information is associated with each of two or more morphology categories, the stained cells in the cell image are classified into corresponding morphology categories,
The composition pattern of the morphological category of stained cells in the cell image extracted from the sample image is determined for the sample image;
The cytology category estimation step is
The sample image from the composition pattern of the sample image based on cytology category estimation information in which each of two or more cytology categories is associated with a composition pattern of a cervical morphological category belonging to each of the cytology categories It is characterized in that a cytology category is estimated for a stained sample of

The program of the present invention is characterized by causing a computer to execute the method of the present invention for estimating the cytology category of the cervix.

The recording medium of the present invention is a computer readable recording medium in which the program of the present invention is recorded.

According to the present invention, for example, cytopathic classification estimation can be easily performed on cervical specimens prior to a pathologist's diagnosis of the morbidity of cervical cancer.

FIG. 1 is a block diagram showing an example of an estimation apparatus of the present invention. FIG. 2 is a block diagram showing an example of the hardware configuration of the estimation apparatus of the present invention. FIG. 3 is a flow chart showing an example of a part of the estimation method of the present invention. FIG. 4 is a flowchart showing an example of a part of the estimation method of the present invention. FIG. 5 is a photograph showing an example of a sample image. FIG. 6 is a flowchart showing an example of extracting stained cells from a sample image. FIG. 7 is a conceptual view showing an example of labeling from a sample image. FIG. 8 is a flowchart showing an example of selecting a region with many stained pixels from the label. FIG. 9 is a photograph showing an example of classification of stained cells. FIG. 10 is a conceptual diagram showing an example of the relationship between the cytological classification of the Bethesda system and the classification ratio of stained cells. FIG. 11 is a flowchart showing an example of a method of generating a cell shape estimation model.

In the estimation device of the present invention, for example, in the cytology category estimation information, the cytology category is a category of Bethesda system.

The estimation apparatus of the present invention is, for example, a cell morphology estimation model in which the cell morphology information estimates a corresponding morphology category for cells in a cell image,
The cell shape estimation model is a model generated by learning from a learning stained cell image of the cervix corresponding to each of the shape categories,
The cell shape classification unit classifies the stained cells in the cell image into corresponding morphology categories according to the cell shape estimation model.

The estimation apparatus of the present invention is, for example, a cytology category estimation model in which the cytology category estimation information estimates a corresponding cytology category for a composition pattern of cells of a sample image,
The cytology category estimation model is a model generated by learning from composition patterns of morphological cells of stained cells in the cervix corresponding to the respective cytology categories,
The cytology category estimation unit estimates a cytology category of the stained sample of the sample image by the cytology category estimation model.

In the estimation device of the present invention, for example, the cell image extraction unit extracts a cell image of stained cells from the sample image using a cell image extraction model,
The cell image extraction model is a model generated by learning from a stained cell image in a cervical stained sample image for learning.

In the estimation device of the present invention, for example, the number of the form categories is 5 to 100.

The estimation apparatus of the present invention further includes, for example, an appropriate determination unit of the sample image,
The suitability determination unit detects a determination item for the sample image, and determines that the determination result is appropriate as the sample image in the cell image extraction unit when the detection result satisfies the appropriate result.

In the estimation device of the present invention, for example, the determination item is at least one selected from the group consisting of presence or absence of image blurring, number of cells, degree of blood contamination, and degree of inflammation in the sample image.

In the estimation method of the present invention, for example, in the cytology category estimation information, the cytology category is a category of Bethesda system.

The estimation method of the present invention is, for example, a morphology category estimation model in which the cell morphology classification information estimates a corresponding morphology category for cells in a cell image,
The cell shape estimation model is a model generated by learning from a learning stained cell image of the cervix corresponding to each of the shape categories,
The cell shape classification unit classifies the stained cells in the cell image into corresponding morphology categories according to the cell shape estimation model.

The estimation method of the present invention is, for example, a cytology category estimation model in which the cytology category estimation information estimates a corresponding cytology category for a composition pattern of cells of a sample image,
The cytology category estimation model is a model generated by learning from composition patterns of morphological cells of stained cells in the cervix corresponding to the respective cytology categories,
In the cytology category estimation step, a cytology category is estimated for a stained sample of the sample image by the cytology category estimation model.

In the estimation method of the present invention, for example, the cell image extraction step extracts a cell image of stained cells from the sample image using a cell image extraction model;
The cell image extraction model is a model generated by learning from a stained cell image in a cervical stained sample image for learning.

In the estimation method of the present invention, the number of morphology categories is 5 to 100.

The estimation method of the present invention further includes, for example, an appropriate determination unit of the sample image,
The aptitude determining step detects a determination item for the sample image, and when the detection result satisfies the appropriate result, determines that the aptitude image is suitable as the sample image in the cell image extracting step.

In the estimation method of the present invention, for example, the determination item is at least one selected from the group consisting of presence or absence of image blurring, cell number, degree of blood contamination, and degree of inflammation in the sample image.

MUC is a core protein of mucin and is known as a component of mucus. On the other hand, since cervical epithelium does not have mucus, it is technical common knowledge that there is no MUC in cervical epithelium. However, as a result of earnest studies, in the cervical intraepithelial neoplasia, the present inventors specifically isolated MUCs whose expression is not confirmed in normal epithelia because they are specifically expressed. By detecting the presence of MUC, it was found that the possibility of cervical cancer morbidity can be tested.

Further, the present inventor further uses the specimen image stained by the MUC conjugate of the cervix specimen from the relationship between MUC and cervical cancer as described above, and uses the stained specimen image of the stained specimen image. It has been found that composition patterns of morphological categories allow estimation of cytology categories such as Bethesda system. Specifically, when the cervical specimen is stained with the MUC conjugate, when the morphology of the stained cells is classified, it becomes clear that the composition pattern of the morphology is different depending on the condition of the cervix. The Therefore, by classifying the morphology of stained cells contained in the stained sample image stained with the MUC conjugate and determining the composition pattern, it is estimated what cytology category the stained sample belongs to. It became possible to

According to the present invention, for example, prior to diagnosis by a doctor such as a pathologist, even a person who does not have a medical license does not have a cytology category indirectly from the composition pattern of the form of the stained cells. It is possible to easily estimate whether it is likely to be applicable or not.

In the present invention, the type of cytology for estimating the category is not particularly limited, and examples thereof include Bethesda system (also referred to as Bethesda classification) and the like. Further, in the present invention, the cytology category for estimating the category is not limited to, for example, cytology known at the time of filing of the present application such as the Bethesda system and at the time of basic application of the present application. Also available. In the present invention, the cytology category means, for example, categories classified in a certain cytology (for example, including the meaning of class, level, stage, etc.). That is, as a specific example, if the cytology is a Bethesda system, the category is, for example, a category (NILM, ASC-US, ASC-H, LSIL, HSIL, SCC) as shown in Table 1 described later.

The Bethesda system is shown as an example of the cytology category. According to the Bethesda system, cells are classified into six categories as shown in Table 1 below. According to the present invention, for example, as described later, the cytology category by the Bethesda system can be estimated from the composition pattern of the morphological category of stained cells in the sample image.

As described above, MUC is a core protein family of mucins, and examples thereof include MUC1, MUC2, MUC3, MUC4, MUC5AC, MUC5B, MUC6, MUC7 and the like. In the present invention, MUC may be, for example, any one type or two or more types, and among them, MUC1 is preferable.

The type of MUC conjugate is not particularly limited, and it is a binding substance having binding activity to MUC, preferably a binding substance exhibiting specific binding activity, and examples thereof include MUC antibody and the like.

The specimen image used in the present invention is, as described above, an image of a cervical specimen stained with a MUC conjugate. The preparation method of the cervical sample is not particularly limited, and, for example, general slide preparation in cytology can be used. In addition, the method for staining the cervical specimen with MUC conjugate is not particularly limited, and, for example, general staining using binding of a target (for example, an antigen) to a conjugate (for example, an antibody) thereto Methods are available.

Next, an embodiment of the present invention will be described. The present invention is not limited to the following embodiments. In the following drawings, the same parts are denoted by the same reference numerals. Further, the descriptions of the respective embodiments can be mutually incorporated unless otherwise stated. Furthermore, the configurations of the respective embodiments can be combined unless otherwise stated.

In the following embodiment, as an example of the sample image, an image of a cervical sample stained with the MUC antibody is exemplified, but the present invention is not limited thereto, and has a binding property to MUC. It may be an image of a sample stained by the binding substance. Further, hereinafter, a sample to be a target for which a cytology category is to be estimated according to the present invention is referred to as a "subject".

Embodiment 1
An example of the estimation apparatus and estimation method of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing an example of an estimation apparatus of the present embodiment. The estimation apparatus 1 includes a sample image input unit 11, a cell image extraction unit 12, a cell shape classification unit 13, and a cytology category estimation unit 14. The estimation device 1 may further include, for example, a storage unit 15 and an output unit 16. The storage unit 15 includes, for example, a processing information storage unit 151, a cell image extraction information storage unit 152, a cell type classification information storage unit 153, a cytology category estimation information storage unit 154, and the like. The information storage unit 151 stores, for example, processing information (for example, input information input to the estimation device 1, output information output from the estimation device 1, and information obtained by the estimation device 1), and a cell image The extraction information storage unit 152 stores cell image extraction information, the cell type classification information storage unit 153 stores cell type classification information, and the cytology category estimation information storage unit 154 stores cytology category estimation information .

The estimation device 1 is also referred to, for example, as an estimation system. The estimation apparatus 1 may be, for example, one estimation apparatus including the respective units, or the respective units may be an estimation apparatus connectable via a communication network. The communication network is not particularly limited, and may be a known communication network, and may be wired or wireless. Specifically, for example, the Internet, telephone, LAN (Local Area Network), WiFi (Wireless Fidelity) Etc.). In the estimation device 1, the processing of each unit may be performed on the cloud.

Next, FIG. 2 illustrates a block diagram of a hardware configuration of the estimation device 1. The estimation device 1 includes, for example, a CPU (central processing unit) 101, a memory 102, a bus 103, an input device 104, a display 105, a communication device 106, a storage device 107, an imaging device 113, and the like. The respective units of the estimation device 1 are mutually connected via a bus 103 by respective interfaces (I / F).

The CPU 101 is responsible for overall control of the estimation device 1. In the estimation device 1, for example, the program of the present invention and other programs are executed by the CPU 101, and reading and writing of various information are performed. Specifically, in the estimation device 1, for example, the CPU 101 functions as the sample image input unit 11, the cell image extraction unit 12, the cell shape classification unit 13, and the cytology category estimation unit 14.

The estimation apparatus 1 can be connected to a communication network by, for example, the communication device 106 connected to the bus 103, and can also be connected to an external device via the communication network. The external device is not particularly limited, and examples thereof include an imaging device such as a camera, a terminal such as a personal computer (PC), a tablet, and a smartphone. The connection method between the estimation device 1 and the external device is not particularly limited, and may be, for example, wired connection or wireless connection. The wired connection may be, for example, a cord connection or a cable connection for using a communication network. The wireless connection may be, for example, a connection using a communication network or a connection using wireless communication. The communication network is not particularly limited, and for example, a known communication network can be used and is the same as described above. The connection type between the estimation device 1 and the external device may be, for example, USB or the like.

The memory 102 includes, for example, a main memory, and the main memory is also referred to as a main storage device. When the CPU 101 performs a process, for example, the memory 102 reads various operation programs 108 such as the program of the present invention stored in an auxiliary storage device described later, and the CPU 101 receives data from the memory 102. , Program 108 is executed. The main memory is, for example, a RAM (random access memory). The memory 102 further includes, for example, a ROM (read only memory).

The storage device 107 is also called, for example, a so-called auxiliary storage device with respect to the main memory (main storage device). The storage device 107 includes, for example, a storage medium and a drive that reads and writes to the storage medium. The storage medium is not particularly limited, and may be, for example, a built-in type or an external type, HD (hard disk), FD (floppy (registered trademark) disk), CD-ROM, CD-R, CD-RW, MO, Examples of the drive include a DVD, a flash memory, and a memory card, and the drive is not particularly limited. The storage device 107 can also be exemplified by, for example, a hard disk drive (HDD) in which a storage medium and a drive are integrated. For example, as described above, the operation program 108 is stored in the storage device 107, and as described above, the memory 102 reads the operation program 108 from the storage device 107 when the CPU 101 is executed. In addition, the storage device 107 may be, for example, the above-mentioned processing information 109 (for example, the input information, the output information, and information obtained by the estimation device 1), the cell image extraction information (for example, the cell image extraction model 110). And the cell shape classification information (for example, the cell shape estimation model 111), the cytology category estimation information (for example, the cytology category estimation model 112), and the like.

The estimation device 1 may further include an imaging device 113. The imaging device 113 is, for example, a camera. When the estimation device 1 includes the imaging device 113, for example, the imaging device 113 can image the stained sample and input the image.

The estimation device 1 may further include, for example, an input device 104 and a display 105. The input device 104 is, for example, a scanner that reads an image, a touch panel, or a keyboard. The display 105 is, for example, an LED display, a liquid crystal display or the like, and also serves as the output unit 16.

The cell image extraction information is information for extracting stained cells from the sample image. Examples of the cell image extraction information include a cell image extraction model 110. The cell image extraction model 110 is, for example, a model generated by learning from a stained cell image (for example, a cut out stained cell image) in a learning staining sample image for learning of the cervix accumulated in a database. When a cervical specimen is stained with the MUC conjugate, the resulting stained specimen includes cells that are stained by binding to the MUC conjugate and cells that are not stained because the MUC conjugate is not bound. And mixed. Therefore, by using the image of the stained sample as a learning stained sample image and learning the stained cell image contained therein as learning data, the stained cells of each of the plurality of cells included in the stained sample image of the cervix can be obtained. It is possible to generate the above-mentioned model which discriminates whether or not to extract a region including stained cells as a cell image of the stained cells. The learning data may further include, for example, a non-stained cell image in the stained sample image for learning, in which case the stained cell image and the non-stained cell image are learned as learning data, A model may be generated that can discriminate between non-stained cells. The learning may be, for example, any of AI, machine learning, deep learning, and the like. The machine learning can use, for example, SVM (Support Vector Machine) or the like, and the deep learning can use, for example, CNN (Convolutional Neural Network) or the like. The learning in the present invention is, for example, the same as the following.

The cell morphology classification information is, as described above, information in which each of two or more morphology categories is associated with cell morphology information. In the present invention, the number of the morphology categories and the cell morphology information of each of the morphology categories are not particularly limited, and can be set arbitrarily. The present invention is a technique based on the fact that the compositional pattern of the morphology of the stained cells is different depending on the condition of the cervix as described above. For this reason, it can be said that, for example, as the number of morphological categories is relatively large, it is possible to estimate cytology categories with higher accuracy. The lower limit is 2 or more, for example, 5 or more and 20 or more are preferable, and the upper limit of the number of the form categories is not particularly limited, and for example, 100 or less and 30 or less is preferable.

The cell morphology classification information may be stored in a database, for example, or may be a cell morphology estimation model 111. The cell shape estimation model 111 is, for example, a model for estimating a corresponding morphology category for cells in a cell image, and for example, from a learning stained cell image of the cervix corresponding to each of the morphology categories accumulated in the database, It can be generated by learning. As the staining cell image for learning, for example, a staining cell image of a characteristic form in each cytology category, a staining cell image of a form common to cytology categories, and the like can be used.

The type of the cell shape information is not particularly limited, and examples thereof include items as shown in Table 2 below, and as shown in Table 3 below, can be linked to each of the shape categories.

The cytology category estimation information is information in which each of two or more cytology categories is associated with a composition pattern of a cervical morphological category belonging to each of the cytology categories. As described above, when the morphology of the cells stained with the MUC conjugate was classified for the cervical sample, it became clear that the composition pattern of the morphology is different depending on the condition of the cervix. Therefore, for example, for each cytology category, the composition pattern of the morphological category of each stained cell is set in advance, and the corresponding cytology category is estimated from the composition pattern of the sample image of the subject. it can.

The cytology category estimation information may be stored in, for example, a database, or may be a cytology category estimation model 112. The cytology category estimation model 112 is, for example, a model for estimating a corresponding cytology category with respect to the composition pattern of cells of the sample image, and the stained cells in the cervix corresponding to the respective cytology categories accumulated in the database. It can be generated by learning from composition patterns of the morphological category.

Below, each part of the estimation apparatus 1 of this embodiment is demonstrated concretely.

The sample image input unit 11 inputs a sample image of the stained sample. The input of the sample image may be performed, for example, by imaging a slide of the stained sample, or reading of an image obtained by photographing the slide. The magnification of the sample image to be input is not particularly limited, and can be appropriately set or changed according to, for example, the magnification of the microscope at the time of photographing the slide.

The cell image extraction unit 12 extracts a cell image of stained cells from the sample image. The method for extracting stained cells from the sample image is not particularly limited, and can be extracted using, for example, the cell image extraction model 110 as described above. In the present embodiment, for example, cell images may be extracted for a large number of detectable stained cells contained in the sample image, preferably all detectable stained cells. As described above, since approximately 100,000 cells are usually present in a stained sample, it is impossible to confirm all stained cells by human judgment of the cytologist. . However, according to the estimation apparatus of the present invention, for example, since automatic analysis using an image is possible, it is also possible to extract an image of stained cells for all detectable stained cells.

The cell image extraction unit 12 may double as, for example, a counting unit that detects the number of cell images of stained cells extracted in the sample image, and the estimation device 1 further includes, for example, the counting unit. It is also good.

The cell morphology classification unit 13 classifies the stained cells in the cell image into a corresponding morphology category based on the cell morphology classification information, and the stained cells in the cell image extracted from the specimen image for the specimen image. Determine the composition patterns of the morphological categories. The classification into the morphology category can be performed, for example, using the cell morphology estimation model 111 as described above.

The way of expressing the composition pattern is not particularly limited, and may be, for example, the cell ratio or frequency distribution of cells belonging to each of the morphology categories. The cell shape classification unit 13 may also serve as, for example, a counting unit that detects the number of cell images classified into each of the shape categories, or the estimation device 1 may further include, for example, the counting unit. .

The cytology category estimation unit 14 estimates a cytology category for the stained sample of the sample image from the composition pattern of the sample image based on the cytology category estimation information. The estimation of the cytology category can be performed using, for example, the cytology category estimation model 112 as described above.

Below, the estimation method of this embodiment is demonstrated concretely. The estimation method of the present embodiment can be performed, for example, by the estimation device 1 of the present embodiment.

The sample image input step is a step of inputting a sample image of the stained sample, and can be executed by the sample image input unit 11 of the estimation device 1.

The cell image extraction step is a step of extracting a cell image of a stained cell from the sample image, and can be executed by the cell image extraction unit 12 of the estimation device 1.

The cell morphology classification step classifies the stained cells in the cell image into a corresponding morphology category based on the cell morphology classification information, and the stained cells in the cell image extracted from the specimen image for the specimen image. Determining the composition pattern of the morphological category of This process can be performed by the cell shape classification unit 13 of the estimation device 1.

The cytology category estimation step is a step of estimating a cytology category for the stained specimen of the specimen image from the composition pattern of the specimen image based on the cytology category estimation information. This process can be performed, for example, by the cytology category estimation unit 14 of the estimation device 1.

Second Embodiment
In the estimation apparatus and estimation method of the present invention, an example of a mode of determining the appropriateness of the sample image will be described. In addition, the description of the said Embodiment 1 can be used for this Embodiment 2, unless it shows in particular.

The estimation apparatus 1 may further include, for example, an appropriateness determination unit of the sample image. In addition, the estimation method of the present embodiment may further include, for example, a step of determining the appropriateness of the sample image, and this step can be performed by, for example, the aptitude determining unit. The aptitude determining unit and the aptitude determining step, for example, detect a determination item for the sample image, and when the detection result satisfies the aptitude result, determine that the sample image in the cell image extracting unit is suitable.

In the present embodiment, the sample image of the stained sample is used. For example, if there is a problem in the stained sample itself or there is a problem in the sample image, it may lead to an erroneous estimation. Therefore, the erroneous estimation can be suppressed by judging the appropriateness of the sample image before extracting the stained cells.

The determination item is, for example, the presence or absence of blurring of the detected image as an item for determining the appropriateness of the detected image itself. Then, for example, when blurring is detected from the detection image, the detection image is determined to be inappropriate, the cell image extraction process by the cell image extraction unit is stopped, and blurring is not detected from the detection image. In this case, it is possible to determine that the detected image is appropriate, and to execute the cell image extraction process by the cell image extraction unit.

Further, the determination items include, for example, the number of cells, the degree of blood contamination, the degree of inflammation, and the like in the sample image as items for determining the appropriateness of the stained sample itself. Then, for example, when it is detected that the number of cells is less than the threshold, the degree of blood contamination exceeds the threshold, the degree of inflammation exceeds the threshold, or the like in the detection image, the specimen image of the stained specimen and the stained specimen Is judged to be inappropriate, and the cell image extraction step by the cell image extraction unit is discontinued, while the number of cells is above the threshold and the degree of blood contamination is below the threshold in the detection image. When it is detected that the threshold value is equal to or less than a threshold value, it is possible to determine that the stained sample and the detection image are appropriate, and to execute the cell image extraction process by the cell image extraction unit. The determination item determines that the sample image is inappropriate when, for example, any one item becomes inappropriate.

The threshold value of the determination item is not particularly limited, and the setting can be changed depending on, for example, whether or not the determination is severe. The number of cells can be determined from, for example, the number of cells in a predetermined area, the degree of blood contamination can be determined, for example, the ratio of blood in a predetermined area, and the degree of inflammation can be determined from the appearance rate of neutrophils, for example.

Moreover, the determination of the aptitude can also use an aptitude determination model. The aptitude determination model can be generated, for example, by using, as learning data, an inappropriate sample image accumulated in a database and a sample image of aptitude for each determination item.

Third Embodiment
An example of an estimation method using the estimation device 1 of the first and second embodiments will be described with reference to the drawings. 3 and 4 are flowcharts of the present embodiment. In addition, the estimation method of this embodiment is not limited to use of the estimation apparatus 1 of FIG.

(A1) Sample Image Input Step The sample image input step is a step of inputting a sample image of the stained sample. The magnification of the sample image is not particularly limited. For example, when the actual size of the slide is 1 ×, an image of about 100 × can be exemplified.

(A2) Suitability Determination Step The suitability determination step includes, for example, the steps of A2-1 to A2-6.

First, the determination item is detected for the input sample image (A2-1). The determination items are, for example, the presence or absence of blurring of an image, the number of cells, the degree of blood contamination, the degree of inflammation, etc., and these are detected from the sample image. The order of detection of each of these determination items is not particularly limited.

Then, an aptitude determination is performed for each determination item of the sample image. In the determination of the aptitude, for example, as described above, the determination of the aptitude of the sample image itself and the determination of the aptitude of the stained sample itself are included. The order of the determination is not particularly limited, and any may be preceded. For example, when the sample image itself is inappropriate, for example, it is difficult to carry out the subsequent process itself, so it is preferable to determine the suitability of the stained sample itself after determining the suitability of the sample image.

As a specific example, first, the presence or absence of blurring of the image is detected with respect to the sample image (A2-2), and when blurring occurs in the image (YES), the sample image itself is determined to be inappropriate and the process ends ( END). On the other hand, when blurring does not occur (NO), the number of cells contained in the sample image is further detected (A2-3), and when the number of cells is small (YES), the stained sample is regarded as inappropriate. Judge and finish (END). On the other hand, if the number of cells is not small, that is, sufficient (NO), blood contamination is detected in the sample image (A2-4), and if blood contamination is significant (YES), the stained specimen itself is Judge as inappropriate and finish (END). On the other hand, when blood contamination is not remarkable (NO), further, inflammation in the sample image is detected (A2-5), and when the inflammation is severe (YES), the stained sample itself is judged to be inappropriate and the process is ended. (END). On the other hand, if the inflammation is not severe (NO), it is determined that the sample image is appropriate (A2-6), and the process proceeds to the next step (X).

(A3) Cell Image Extraction Step In the cell image extraction step, an image (image area) of stained cells stained with the MUC conjugate is extracted from the sample image determined to be appropriate in the (A2-6) step. (A3). The extraction of the image includes, for example, the specification of the image area of the stained cell in the sample image, and the clipping of the specified image area.

FIG. 5 is a schematic view of the extraction of the stained cells from the sample image, and FIG. 5 (A) shows a part of the sample image, and FIG. 5 (B) is each FIG. 5 (A). Shows an image of stained cells contained in As shown in FIG. 5 (A), the image areas of the stained cells are identified ( image areas 1, 2, 3 in FIG. 5 (A)) for the sample image in which a plurality of stained areas exist (FIG. 5 (B)) As shown in), the image areas 1, 2, 3 are cut out as a stained image of the stained cells.

The extraction of the image of the stained cells from the sample image may be performed, for example, by conventional image processing, or the cell image extraction model may be used.

Here, a flowchart of FIG. 6 is shown as an example of extraction of a cell image of the stained cell from the sample image.

The sample image determined to be appropriate in the step (A2) is input (A3-1), and a binarization process is performed on the sample image so that the stained area is white and the unstained area is black ( A3-2). The binarization process can use, for example, a cell image extraction model 110. Next, in the sample image, regions including overlapping and / or adjacent white are collectively labeled as one cell (A3-3). From the label (region labeled as one cell) in the sample image, a region having the largest number of stained pixels (stained pixels) is further selected (A3-4), and cut out as an image (A3-5). Specifically, for example, as shown in the schematic diagram of FIG. 7, when the image area labeled as one cell is surrounded by the boundary box X, the area Y with the largest number of stained pixels is selected, and the area Y is selected. Cut out as an image. The magnification of the clipped image is not particularly limited. For example, when the actual size of the slide is 1 ×, an image of about 400 × can be exemplified. In the case where a plurality of labels are present in one sample image, the selection and clipping of the region having the largest number of stained pixels are repeated for each label. Specifically, when the extraction of a certain label is completed, it is confirmed whether there is another labeling target in the same sample image (A3-6), and in the case of “YES”, from labeling to extraction (A3) Repeat steps -3 to A3-5), process all labels, and if (A3-6) becomes "NO", end (END).

The selection and cutting out (A3-3- to A3-5) of the region having the largest number of stained pixels from the label can be performed, for example, by a conventional method. An example of the selection and segmentation is shown in FIG. The right figure of FIG. 8 is the same as FIG. 7, and the image area labeled as one cell is a boundary box X formed with height h in the Y axis direction and width w in the X axis direction. It is the figure enclosed. Y 'in the boundary box X is a search rectangle Y' indicating a region for counting the stained pixels in a flowchart to be described later. The size of the search rectangle Y ′ is arbitrary, and can be represented by a height b (b <h) in the Y-axis direction and a width a (a <w) in the X-axis direction. h is the height h of the boundary box X in the Y-axis direction, and w is the width w of the boundary box X in the X-axis direction. In the search rectangle Y ′ in the right view of FIG. 8, for example, the variable j indicating the position in the Y-axis direction is 0, and the variable i indicating the position in the X-axis direction is 0. The left view of FIG. 8 is a flowchart of the selection and segmentation. For the image area of the boundary box X in FIG. 8, for example, as shown in the flowchart, the following steps are performed.

(A1) A variable max indicating the count number of stained pixels is set to an initial value 0 (max = 0).
(A2) A variable j indicating the position in the Y-axis direction is changed by one from 0 to [height h-height b] (j = 0, hb, 1).
(A3) A variable i indicating the position in the X-axis direction is changed one by one from 0 to [width w-width a] (i = 0, wa, 1).
(A4) Count stained pixels in the search rectangle Y 'at the positions of the variables j and i (cnt).
(A5) It is determined whether the count cnt is larger than the variable max (max <cnt).
If max <cnt (YES), the count number cnt is set to the variable max, and i and j are set to the variables x and y indicating the coordinate position indicating the count number cnt (max = cnt, x = i, y = j).
If max> cnt (NO), the variable max and the variables x and y are not changed.
(A6) (a7) As shown by the arrow in the right drawing of FIG. 8, the count position of the stained image is moved to the next position (i + 1, j) or (i, j + 1), and the above (a4) And (a5) are similarly repeated (a8) When this repeated processing is completed, the maximum value of the stained pixel count is set in cnt, and the maximum value in the search rectangle Y ′ is indicated in the coordinate position x, y. The position is set. Therefore, an image having a width a and a height b is cut out from the x and y points, and the process ends (END). When selection and clipping are performed on the right diagram of FIG. 8, for example, the region Y in the diagram of FIG. 7 is selected and clipped.

(A4) Cell Counting Step In the above estimation method, for example, a cell image of stained cells extracted in the step (A3) may be counted. As described above, the expression of MUC is not confirmed in normal epithelium, but is specifically expressed in cervical intraepithelial neoplasia. Therefore, when the cervical specimen is normal, stained cells may not be detected even by the staining with the MUC conjugate. Therefore, for example, the cell image of the stained cells extracted for the sample image is counted (A4), and when the number of stained cells is small (YES), the sample of the sample image is estimated to be normal. May end (END). In the estimation, for example, a threshold value of the number of stained cells per slide is set, and when it is less than the threshold value, it can be estimated to be normal. Generally, the slide in the cytology has 100,000 cells per sheet, so that the threshold of normality or not is, for example, a ratio of about 1% of the stained cells or the stained cells The number can be set to about 1,000. In the case of the above-mentioned Bethesda system, for example, it can be estimated to be normal, ie, the cytology category of "NILM". On the other hand, when the number of the stained cells is not small (NO), for example, when the number is larger than the threshold value, it is not possible to deduce that it is normal, and the process proceeds to the next cell shape classification step.

(A5) Cell Form Classification Step The cell form classification step classifies the stained cells in the extracted cell image into a corresponding form category based on the cell form classification information (A5), and the sample image in the form It is a process of determining the composition pattern of the morphological category of stained cells in the cell image extracted from the specimen image. The cell shape classification step can be performed using, for example, the cell shape estimation model 111 as described above.

FIG. 9 shows a schematic view of the classification of the stained cells. When four types of morphology categories are set based on information of morphology items as shown in Table 3, as shown in FIG. 9, each stained cell is classified into A category based on the cell morphology classification information, It can be classified into 4 types of B category, C category and D category. This classification can use, for example, the cell shape estimation model 111 described above.

(A6) Cytology category estimation step The cytology category estimation step estimates the cytology category for the stained sample of the sample image from the composition pattern of the sample image based on the cytology category estimation information ( A6). This estimation can use, for example, the cytology category estimation model 112 described above.

When a cervical specimen is stained with the MUC conjugate, the stained specimen contains various forms of stained cells stained with the MUC conjugate. And, the composition patterns of the various forms of stained cells are different depending on the condition of the cervix. That is, they are significantly different for each sample group belonging to each cytology category of cytology. Therefore, for example, a composition pattern of the subject and a composition pattern in each category of cytology are compared, and a cytology category similar to the composition pattern of the subject is a cytology category of the subject. It is possible to estimate. As a specific example, when the classification into the cell morphology categories A to D illustrated in FIG. 9 is performed, the stained cells in the sample image of the subject are category A 20%, category B 40%, category C 10%, category D 30% It is assumed that the composition pattern is classified into the ratio of. On the other hand, in the Bethesda system for cervical cancer, the categories NILM, LSIL, HSIL and SCC respectively have characteristic composition patterns as exemplified in FIG. 10, and the ratios of categories A to D, for example. The cytology category can be estimated from the composition pattern of the sample image by using the composition pattern of each cytology category as an estimation criterion. That is, for example, a composition pattern having a ratio of Category A 20%, Category B 40%, Category C 10%, and Category D 30% can be estimated as NILM.

Fourth Embodiment
The present embodiment describes generation of each illustrated model. The present invention is not limited to this example.

(1) Cell image extraction model 110
The cell image extraction model 110 can be generated by learning, for example, as described above. In the sample image, discrimination between a stained region and a non-stained region can be performed, for example, by setting the target staining color as positive and the other colors as negative. In the learning, for example, image data of a cell stained in a target color, data that the image data linked thereto is a positive example, image data of a cell stained in any other color, and It may be learned using data or the like that the image data linked thereto is a negative example.

A specific example of generation of the cell image extraction model 110 is shown below. First, a slide of a cervical specimen stained with a MUC conjugate is imaged, and an image of stained cells is cut out from the imaged slide image and accumulated in a stained cell database. Then, the accumulated stained cell image is input as learning data to the model generation apparatus, and learning is performed, whereby a model can be generated which selects and extracts an image area including the stained cells.

Then, using the cell image extraction model 110 constructed by learning, the sample image accumulated in the stained cell database is processed to confirm whether or not the extraction of the stained cell image is correct. Can correct the extraction error and further learn to generate the cell image extraction model 110 in which the extraction accuracy is further improved.

(2) Cell morphology classification model 111
The cell shape estimation model 111 can be generated by learning, for example, as described above. In the learning, for example, image data of cells corresponding to each morphology category may be used for learning.

A specific example of the generation of the cell shape estimation model 111 is shown below by taking the flowchart of FIG. 11 as an example. First, with regard to images of stained cells stained with MUC conjugate, any plural form categories having different forms are determined (B1-1), and each of the form categories is associated with a stained cell image belonging thereto to classify the forms. Accumulate in database 1 Then, the form category stored and the stained cell image belonging thereto are input as learning data to the model generation apparatus as learning data (B1-2), and the stained cell images to be classified are learned by learning (B1-3), A model 111 can be generated that determines (estimates) which form category it falls under.

Then, using the cell shape estimation model 111 constructed by learning, the cell image of the stained cells stored in the shape classification database 2 is classified (B1-4), and whether the classification is correct or not is confirmed. If incorrect, perform processing to correct classification errors (B1-5), and further perform learning (B1-6) to generate a cell shape estimation model 111 with further improved classification accuracy. You can also. In FIG. 11, for convenience, the database 1 used in the first learning and the database 2 used in the confirmation of the classification error are separately shown, but both may be the same database.

(3) Cytology category estimation model 112
The cytology category estimation model 112 can be generated by learning, for example, as described above. In the learning, for example, a composition pattern of a sample corresponding to each cytology category may be used to learn.

A specific example of generation of the cytology category estimation model 112 is shown below. First, a slide of the cervix belonging to each cytology category stained with a MUC conjugate is imaged, and from the imaged slide image, for example, using the cell shape estimation model 111, the composition pattern of the cells belonging to each of the above morphology categories Are linked to the cytology category and accumulated in the composition pattern database. Then, the composition pattern of the sample image to be classified corresponds to any cytology category by inputting the accumulated composition pattern and the cytology category as learning data to the model generation apparatus as learning data. A model 112 can be generated to estimate the

Then, using the cytology category estimation model 112 constructed by learning, the cytology category is classified for the composition pattern stored in the composition pattern database, and it is determined whether the classification is correct or not. In this case, a cytology category estimation model 112 with a further improved classification accuracy can be generated by performing processing for correcting classification errors and further performing learning.

The cytology category estimation model 112 may be, for example, a cluster model by cluster analysis of the composition pattern and the cytology category.

Fifth Embodiment
A program according to Embodiment 5 of the present invention is a program that can execute the estimation method of the present invention on a computer. Alternatively, the program of the present embodiment may be recorded on, for example, a computer readable recording medium. The recording medium is not particularly limited, and examples thereof include the above-described storage medium and the like.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

This application claims priority based on Japanese Patent Application No. 2017-161944 filed on Aug. 25, 2017, the entire disclosure of which is incorporated herein.

As described above, according to the present invention, cytopathic classification estimation can be easily performed on cervical specimens, for example, before a pathologist diagnoses the morbidity of cervical cancer.

Claims (18)

1. Sample image input unit,
   Cell image extraction unit,
   A cell shape classification unit, and a cytology category estimation unit;
   The sample image input unit
   Input the sample image of the stained sample,
   The stained sample is a cervical sample stained by a MUC conjugate,
   The cell image extraction unit
   Extracting a cell image of stained cells from the sample image;
   The cell shape classification unit
   Based on cell morphology classification information in which cell morphology information is associated with each of two or more morphology categories, the stained cells in the cell image are classified into corresponding morphology categories,
   The composition pattern of the morphological category of stained cells in the cell image extracted from the sample image is determined for the sample image;
   The cytology category estimation unit
   The sample image from the composition pattern of the sample image based on cytology category estimation information in which each of two or more cytology categories is associated with a composition pattern of a cervical morphological category belonging to each of the cytology categories Apparatus for estimating cervical cytology category characterized in that cytology category is estimated for a stained sample of
2. The estimation apparatus according to claim 1, wherein in the cytology category estimation information, the cytology category is a Bethesda system category.
3. The cell shape estimation model is a cell shape estimation model in which a corresponding shape category is estimated for cells in a cell image,
   The cell shape estimation model is a model generated by learning from a learning stained cell image of the cervix corresponding to each of the shape categories,
   The estimation device according to claim 1, wherein the cell shape classification unit classifies stained cells in the cell image into a corresponding shape category according to the cell shape estimation model.
4. The cytology category estimation information is a cytology category estimation model for estimating a corresponding cytology category for a composition pattern of cells of a sample image,
   The cytology category estimation model is a model generated by learning from composition patterns of morphological cells of stained cells in the cervix corresponding to the respective cytology categories,
   The estimation apparatus according to any one of claims 1 to 3, wherein the cytology category estimation unit estimates a cytology category of the stained sample of the specimen image by the cytology category estimation model.
5. The cell image extraction unit extracts a cell image of stained cells from the sample image using a cell image extraction model,
   The estimation device according to any one of claims 1 to 4, wherein the cell image extraction model is a model generated by learning from a stained cell image in a stained sample image for learning of a cervix.
6. The estimation device according to any one of claims 1 to 5, wherein the number of form categories is 5 to 100.
7. Furthermore, it includes an appropriateness determination section of the sample image,
   The said aptitude determination part detects a determination item about the said sample image, and when a detection result satisfy | fills an appropriate result, it determines that it is suitable as the said sample image in the said cell image extraction part. The estimation device according to one of the claims.
8. The estimation device according to claim 7, wherein the determination item is at least one selected from the group consisting of presence or absence of image blurring, cell number, degree of blood contamination, and degree of inflammation in the sample image.
9. Sample image input process,
   Cell image extraction process,
   Including a cell shape classification step and a cytology category estimation step;
   In the sample image input process,
   Input the sample image of the stained sample,
   The stained sample is a cervical sample stained by a MUC conjugate,
   The cell image extraction step
   Extracting a cell image of stained cells from the sample image;
   The cell shape classification step
   Based on cell morphology classification information in which cell morphology information is associated with each of two or more morphology categories, the stained cells in the cell image are classified into corresponding morphology categories,
   The composition pattern of the morphological category of stained cells in the cell image extracted from the sample image is determined for the sample image;
   The cytology category estimation step is
   The sample image from the composition pattern of the sample image based on cytology category estimation information in which each of two or more cytology categories is associated with a composition pattern of a cervical morphological category belonging to each of the cytology categories A method of estimating cervical cytology category, which comprises estimating cytology category for a stained sample of
10. The estimation method according to claim 9, wherein in the cytology category estimation information, the cytology category is a Bethesda system category.
11. The cell shape classification information is a shape category estimation model in which a corresponding shape category is estimated for cells in a cell image,
    The cell shape estimation model is a model generated by learning from a learning stained cell image of the cervix corresponding to each of the shape categories,
    The estimation method according to claim 9, wherein the cell morphology classification unit classifies stained cells in the cell image into a corresponding morphology category according to the cell morphology estimation model.
12. The cytology category estimation information is a cytology category estimation model for estimating a corresponding cytology category for a composition pattern of cells of a sample image,
    The cytology category estimation model is a model generated by learning from composition patterns of morphological cells of stained cells in the cervix corresponding to the respective cytology categories,
    The estimation method according to any one of claims 9 to 11, wherein the cytology category estimation step estimates a cytology category for a stained specimen of the specimen image by the cytology category estimation model.
13. The cell image extraction step extracts a cell image of stained cells from the sample image using a cell image extraction model.
    The estimation method according to any one of claims 9 to 12, wherein the cell image extraction model is a model generated by learning from a stained cell image in a stained sample image for learning of a cervix.
14. The estimation method according to any one of claims 9 to 13, wherein the number of form categories is 5 to 100.
15. Furthermore, it includes an appropriateness determination section of the sample image,
    The said aptitude determination process detects a determination item about the said specimen image, and when a detection result satisfy | fills an appropriate result, it determines with it being suitable as the said specimen image in the said cell image extraction process. Estimation method described in the paragraph.
16. The estimation method according to claim 15, wherein the determination item is at least one selected from the group consisting of presence or absence of image blurring, cell number, degree of blood contamination, and degree of inflammation in the sample image.
17. A program causing a computer to execute the method of estimating cervical cytology category according to any one of claims 9 to 16.
18. The computer-readable recording medium which recorded the program of Claim 17.

Images