JP6740573B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and a program for image processing, and particularly to an image showing a distribution of regions in which it is estimated that a specific part of a cell is captured in an image capturing the morphology of the cell. Image processing technology for acquiring

In the field of medical care, so-called pathological diagnosis is actively performed by observing a tissue section collected from a living body such as a person or an animal with a microscope to diagnose the presence or absence of a lesion, the type of the lesion, and the like. In this pathological diagnosis, a tissue section to be diagnosed is generally subjected to processing such as fixing, embedding, thin sectioning and staining in this order, and then subjected to observation with a microscope.

Regarding the observation by the microscope, for example, the tissue section is stained with a fluorescent labeling material and HE (hematoxylin-eosin), and the morphology of the tissue in the tissue section is obtained by imaging the fluorescence generated by irradiation of the tissue section with excitation light. Also, a technique for simultaneously obtaining information on the distribution of a specific antigen has been proposed (for example, Patent Document 1). Here, the fluorescent labeling material contains, for example, an antibody that recognizes a specific antigen and nanoparticles containing a plurality of fluorescent substances bound to the antibody as constituent elements. In addition, in this technique, for example, as the feature amount that quantitatively expresses the expression state of a specific antigen in a tissue section, for example, the number of bright spots belonging to a cell and/or the brightness of bright spots belonging to a cell. The average brightness of the values or the like can be calculated.

JP2012-208106A

However, when a specific portion such as a cell nucleus is detected by image processing for an image in which the morphology of the tissue including cells in the tissue section is captured, it is estimated that the specific portion is captured. False positives can occur where regions are not accurately detected. This erroneous detection includes, for example, [1] omission of detection, [2] unnecessary detection, [3] excessive detection, [4] underdetection, and [5] multiple estimated areas for the estimated area. It is possible to include various aspects such as detection of a block of estimated regions (estimated regions that have not been divided).

It is not easy to correct such various false detections only by image processing. Therefore, a mode in which the user corrects the erroneous detection manually by using a touch panel or the like can be considered. However, if there are too many areas to be manually corrected, the burden on the user becomes excessive and the time required for the processing becomes too long, which may cause various problems.

The present invention has been made in view of the above problems, and provides a technique capable of efficiently obtaining an image showing a distribution of regions in which a specific part of a cell is captured in an image in which the morphology of the cell is captured. The purpose is to

In order to solve the above-mentioned problems, the image processing apparatus according to the first aspect performs image processing on a cell morphology image that captures the morphology of cells to capture a specific portion of the cell in the cell morphology image. a generation unit distribution of the estimated area is estimated that things to produce a cell distribution image shown in one or more cells display element 2 above the cell form image according to the two or more conditions that are different from each other by the generating unit A display control unit for displaying on the display unit two or more cell distribution images generated by performing the image processing of 1. and the two or more cell distributions displayed on the display unit according to a user's operation. A specifying unit that specifies one cell distribution image of the images, and a correcting unit that corrects at least a part of the one cell distribution image specified by the specifying unit according to a user's operation , before Symbol generation section, the cell morphology image to image processing several times sequentially stepwise facilities, the different stages in a stepwise manner two or more image processing to be performed among the plurality of image processing The two or more cell distribution images are generated by performing each of the image processing up to the cell morphology image, and the display control unit causes the generation unit to perform the image processing of the plurality of different stages. It is characterized in that the two or more cell distribution images generated by performing the above are simultaneously displayed on the display unit.

The image processing apparatus according to a second aspect, there is provided an image processing apparatus according to the first aspect, the image processing before Symbol different stages sequentially stepwise time a plurality of image processing in the cell morphology image when applied to, including image processing of a plurality of different stages in image processing on 2 or more that is applied stepwise in the image processing of the multiple.

An image processing apparatus according to a third aspect is the image processing apparatus according to the first or second aspect, wherein the designating unit displays the first image area and the display unit according to a user's operation. A first combination with a first cell distribution image of the two or more cell distribution images being displayed, a second image region different from the first image region, and the two or more being displayed on the display unit. A second combination with a second cell distribution image different from the first cell distribution image of the first cell distribution image, and the correction unit selects one of the first cell distribution images according to the user's action. The portion corresponding to the first image area and the portion of the second cell distribution image corresponding to the second image area are corrected, and the image processing apparatus is configured to specify the first portion designated by the designation unit. One cell distribution image by combining the first cell distribution image and the second cell distribution image in at least one of the state before and after the correction by the correction unit according to the combination and the second combination. And a synthesizing unit for generating.

An image processing apparatus according to a fourth aspect is the image processing apparatus according to any one of the first to third aspects, wherein the generation unit is at least one of the two or more image processing types. In the image processing, at least two cell distribution images of the two or more cell distribution images are detected by detecting the estimated region from the cell morphology image according to at least two reference values for each of one or more parameters. To generate.

An image processing apparatus according to a fifth aspect is the image processing apparatus according to the fourth aspect, wherein the one or more kinds of parameters are one or more appearance characteristics of a region in the cell morphology image. At least one kind of parameter is included among the parameter, the parameter relating to the degree of division, and the parameter relating to the degree of expansion/contraction processing.

An image processing apparatus according to a sixth aspect is the image processing apparatus according to the fifth aspect, wherein the one or more types of features in appearance are features of at least one type of size, density and shape. including.

An image processing apparatus according to a seventh aspect is the image processing apparatus according to the sixth aspect, wherein the features related to the size are area, perimeter length, major axis length and minor axis length. The characteristics relating to the density include one or more characteristics among hue, brightness, saturation, luminance, color difference and luminance of each color, and the characteristics relating to the shape are , Circularity, oblateness, and aspect ratio.

The image processing apparatus according to the eighth aspect, the fourth An image processing apparatus according to a seventh any one aspect of the previous SL generation unit, at least two of the at least one image processing In the image processing described above, each of the at least two of the two or more cell distribution images is detected by detecting the estimated region from the cell morphology image according to a combination of two or more sets of reference values between two or more types of parameters. Generate two cell distribution images.

An image processing apparatus according to a ninth aspect is the image processing apparatus according to any one of the first to third aspects, in which the generation unit includes at least one type of appearance of an area in the cell morphology image. At least two cell distribution images out of the two or more cell distribution images are generated by detecting the estimated regions from the cell morphology image by image processing of a plurality of patterns corresponding to the features.

The image processing apparatus according to a tenth aspect, in the image processing apparatus according to the ninth aspect, the image processing before Symbol plurality of patterns, a plurality of sets of features between two or more features on the Appearance The image processing corresponding to each combination is included.
An image processing apparatus according to an eleventh aspect performs image processing on a cell morphology image that captures the morphology of cells, thereby estimating a specific region of cells in the cell morphology image. One or more types of image processing are performed on the cell morphology image according to a generation unit that generates a cell distribution image whose distribution is indicated by one or more cell display elements and two or more conditions that are different from each other by the generation unit. A display control unit that causes the display unit to display the two or more cell distribution images generated by the above, and one cell of the two or more cell distribution images that is displayed on the display unit according to the user's operation. A generating unit that corrects at least a part of the one cell distribution image specified by the specifying unit according to a user's operation; Of the two or more cell distribution images, each of the two or more cell distribution images is detected by detecting the estimated region from the cell morphology image by image processing of a plurality of patterns to correspond to one or more types of features on the appearance of the region in the morphology image. at least generates two cell distribution image, the image processing before Symbol plurality of patterns, comprising the image processing corresponding respectively to a combination of a plurality of sets of features between two or more features on the appearance of And

An image processing apparatus according to a twelfth aspect is the image processing apparatus according to the tenth or eleventh aspect, wherein the two or more types of features on the appearance include features related to density and density, Each of the image processes of the plurality of patterns corresponds to at least one of a threshold value process and a contour extraction process using a threshold value for the brightness of each color according to the feature related to the darkness, and the feature related to the degree of congestion. Area division processing using the degree of area division is included.

The image processing apparatus according to a thirteenth aspect, in the image processing apparatus according to the second aspect, the generating unit, the image processing on the two or more that is applied stepwise among the plurality of image processing In at least one type of image processing included in the remaining image processing except the image processing performed last, the cell morphological image is processed according to different conditions for each region.
An image processing apparatus according to a fourteenth aspect performs image processing on a cell morphology image that captures the morphology of cells, thereby estimating an estimated region in which a specific part of a cell is captured in the cell morphology image. One or more types of image processing are performed on the cell morphology image according to a generation unit that generates a cell distribution image whose distribution is indicated by one or more cell display elements and two or more conditions that are different from each other by the generation unit. A display control unit that causes the display unit to display the two or more cell distribution images generated by the above, and one cell of the two or more cell distribution images that is displayed on the display unit according to the user's operation. a designation unit which designates a distribution image, according to the user's operation, and a correction unit for correcting at least a portion of the designated said one cell distribution image by the designation unit, the generation unit, the 2 In at least one type of image processing of the above image processing, by detecting the estimated region from the cell morphological image according to at least two reference values for each of one or more types of parameters, the two or more cells are detected. generating at least two cell distribution image of the distribution image, before Symbol generation section, at least two kinds of image processing of the at least one image processing, two or more sets between two or more parameters The at least two cell distribution images of the two or more cell distribution images are generated by detecting the estimated regions from the cell morphology image in accordance with the combination of the reference values.

In the image processing method according to the fifteenth aspect, (a) the cell morphology is obtained by performing two or more image processing on a cell morphology image capturing a cell morphology according to two or more mutually different conditions by the generation unit. Generating two or more cell distribution images in which the distribution of the estimated region in which it is estimated that a specific part of a cell is captured in the image is indicated by at least one cell display element, and (b) display A step of displaying the two or more cell distribution images generated in the step (a) on the display section by the control section, and (c) a display section displaying the two or more cell distribution images on the display section according to the user's operation. The step of designating one cell distribution image of the two or more cell distribution images that are present, and (d) the one cell distribution designated in step (c) according to the operation of the user by the correction unit. comprising a step of modifying at least a portion of the image, and in the previous SL step (a), said cells form the image to image processing several times sequentially stepwise facilities to, among the plurality of image processing The two or more cell distribution images are generated by subjecting the cell morphology image to each of a plurality of different stages of image processing in two or more image processing performed stepwise, and in the step (b), The display controller simultaneously displays the two or more cell distribution images generated by subjecting the cell morphology image to each of the plurality of different stages of image processing in step (a). It is characterized in that

The image processing program according to the sixteenth aspect is executed by a control unit included in the information processing apparatus, so that the information processing apparatus serves as the image processing apparatus according to any one of the first to fourteenth aspects. It is characterized by making it function.

According to the present invention, a distribution of regions that are each generated by image processing under different conditions for an image in which a cell morphology is captured and in which it is estimated that a specific part of the cell is captured is shown. In the image, the user can specify the image to be modified. As a result, it is possible to efficiently obtain an image in which the distribution of the region in which the specific part of the cell is captured is shown in the image in which the morphology of the cell is captured.

9 is a flowchart illustrating an example of an image processing flow according to a reference example. It is a figure which shows an example in which the estimation area|region is detected in the proper state. It is a figure which shows an example in which the estimation area|region is detected in the improper state. It is a figure which shows an example in which the estimation area|region is detected in the improper state. It is a figure which shows an example in which the estimation area|region is detected in the improper state. It is a figure which shows an example in which the estimation area|region is detected in the improper state. It is a figure which shows an example in which the estimation area|region is detected in the improper state. 9 is a flowchart showing an example of an operation flow related to detection of a specific portion according to a reference example. It is a figure which shows an example of a cell morphology image. It is a figure which shows an example of a cell distribution image. It is a figure which illustrates the cell display element which shows the estimated area|region detected in the appropriate state. It is a figure which illustrates the cell display element which shows the estimated area|region detected in the excessive division state. It is a figure which illustrates the cell display element which shows the estimated area|region detected in the undivided state. It is a figure which shows an example of the integration process which integrates a cell display element group. It is a figure which shows an example of the division process which divides an undivided cell display element. It is a figure which illustrates the relationship between the progress of a detection process and the number of the estimation areas detected in the improper state. It is a figure which illustrates the relationship between the progress of a detection process and the number of the estimation areas detected in the improper state. It is a figure which illustrates a 1st cell morphology image. It is a figure which illustrates the cell distribution image in the 1st step. It is a figure which illustrates the cell distribution image in the 2nd step. It is a figure which illustrates the cell distribution image in the 3rd step. It is a figure which illustrates a 2nd cell morphology image. It is a figure which illustrates the cell distribution image in the 1st step. It is a figure which illustrates the cell distribution image in the 2nd step. It is a figure which illustrates the cell distribution image in the 3rd step. It is a figure which illustrates the schematic structure of the pathological diagnosis support system which concerns on embodiment. It is a figure which shows an example of a cell morphology image. It is a figure which shows an example of a fluorescence image. It is a block diagram which shows roughly the functional structure of an information processing apparatus. It is a block diagram which illustrates the functional composition realized by the control part. 6 is a flowchart illustrating an operation flow of the image processing apparatus. It is a flow chart which illustrates the operation flow which detects the presumed field which caught the specific part. It is a figure which shows an example of a cell morphology image. It is a figure which shows a typical example of a bright spot distribution image. It is a figure which illustrates the cell distribution image of the 1st step. It is a figure which illustrates the cell distribution image of the 2nd step. It is a figure which illustrates the cell distribution image of the 3rd step. It is a figure which shows an example of an image designation screen. It is a block diagram which illustrates another functional structure implement|achieved by a control part. 9 is a flowchart showing another example of the operation flow of the image processing apparatus. It is a flowchart which illustrates the operation|movement flow which designates a cell distribution image for every area|region. It is a figure which shows an example of a cell morphology image. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which illustrates a mode that a cell morphology image is divided by a mark. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which illustrates the image which the cell distribution image designated for every area|region was synthesize|combined. It is a flowchart which shows an example of the operation|movement flow which designates a cell distribution image for every area|region. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which illustrates a mode that the 1st and 2nd image area is specified. It is a figure which shows the example of a display of an area|region image designation screen. It is a figure which illustrates the some reference value for every kind of parameter. It is a figure which illustrates the combination of the conditions about several types of external features. It is a figure which illustrates the relationship between the external feature and the content of the detection process of an estimation area|region. It is a figure which illustrates the relationship between the external feature and the content of the detection process of an estimation area|region. It is a flowchart which illustrates the flow of the detection process of the estimation area|region which concerns on a 4th modification. It is a figure which illustrates the binary image for dividing a cell morphology image. It is a figure which illustrates a mode that a cell morphological image is divided. It is a figure which illustrates the result which the estimation area|region was detected on the conditions which differ for every area|region.

<(1) Outline of the present invention>
<(1-1) Image processing according to reference example and its problems>
FIG. 1 is a flowchart showing a flow of image processing according to the reference example. As illustrated in FIG. 1, in the image processing according to the reference example, for example, the processing of steps S101 to S105 is sequentially performed.

In step S101, a microscope image is acquired. The microscope image includes, for example, a cell morphology image and a fluorescence image for the same visual field. The cell morphology image is an image in which the morphology of cells is captured. That is, the cell morphology image includes information related to cell morphology (also referred to as cell morphology information). The fluorescence image is an image in which the emission of a fluorescent substance included in particles bound to a specific biological substance (specific antigen) is captured. That is, the fluorescence image captures the distribution state of a specific biological substance, and the fluorescence image includes information (also referred to as substance distribution information) related to the distribution state of the specific biological substance. Note that, as the microscope image, for example, a mode in which one image having both cell morphology information and substance distribution information is adopted is also conceivable.

In step S102, fluorescent bright spots in the fluorescent image are detected. At this time, for example, an image (also referred to as a bright spot distribution image) showing the distribution of fluorescent bright spots associated with a specific wavelength in the fluorescent image is acquired.

In step S103, a region (also referred to as an estimated region) estimated to capture a specific portion is detected by performing image processing on the cell morphology image. At this time, an image (also called a cell distribution image) in which the distribution of the estimated region is shown by a plurality of elements (also called a cell display element) is obtained. As the specific portion, for example, a specific portion or the like can be adopted. In the present embodiment, an example in which the specific portion is the cell nucleus will be described.

2 to 7 are diagrams each showing an example in which an estimated region related to a specific part is detected. FIG. 2 is a diagram illustrating an example in which the estimated region related to the specific portion is detected in an appropriate state (also referred to as an appropriate state). Each of FIGS. 3 to 7 is a diagram showing an example in which the estimated region related to the specific portion is detected in an improper state (also referred to as an improper state). In each of FIGS. 2 to 7, a partial image area of the cell morphological image is shown on the left side, and a cell display element showing an estimated area detected in the partial image area on the right side. Are shown superimposed on the part of the image area. Here, the cell display element is drawn in dark gray. Further, in FIGS. 2 to 7, a thin line is drawn on the image region shown on the right side, which shows the outline of the region in which the specific portion is actually captured.

Specifically, for example, as shown in FIG. 2, the estimated area A101 can be detected and the estimated area A101 can be shown by the cell display element E101. Here, the cell display element E101 can be shown in a specific mode that can be distinguished from various parts captured in the cell morphology image. As the specific mode, for example, a mode in which a specific color, a specific pattern, or the like is attached can be considered.

By the way, when an estimated region corresponding to a specific part is detected by performing image processing on the cell morphology image, erroneous detection of the estimated region may occur. The erroneous detection includes, for example, [1] omission of detection (FIG. 3), [2] unnecessary detection (FIG. 4), [3] excessive detection (FIG. 5), and [4] underestimation of the estimated region. Detection (FIG. 6) and [5] detection of a block of undivided estimated regions corresponding to a plurality of specific parts (FIG. 7), and the like. In the image region on the right side shown in FIGS. 3 to 7, the estimated region detected by performing the image processing on the cell morphology image is shown by the dark gray cell display element. The outline of the region where the specific portion is captured is indicated by a thin line. In other words, the dark gray cell display element is shown to be corrected so as to indicate the area surrounded by the thin line where the specific portion is actually captured.

In step S104, the cell distribution image acquired in step S103 is corrected. Here, the erroneous detection is corrected according to the operation of the user. For example, a mode in which a cell distribution image is displayed on the touch panel and the user corrects the cell distribution image with a stylus pen or the like can be considered. Specifically, for example, in order to compensate for the omission of detection, a closed curve is drawn along the contour of an undetected specific part, a mode in which cell display elements are added, and all cell display elements related to unnecessary detection. Is deleted, a part of the cell display element relating to the excessive detection is deleted, a mode in which the cell display element showing the specific region under-detected is enlarged, and a part in which a plurality of specific regions are one block A mode or the like in which the cell display element shown as is divided is considered. As a result, the generation of the cell distribution image in which the distribution of the estimated region is shown by the plurality of cell display elements is completed.

In step S105, diagnostic assistance information is generated based on the cell distribution image generated in step S104 and the bright spot distribution image acquired in step S102. Examples of the diagnosis support information include an analysis value related to the existence state of a specific biological substance in a specific region captured in a cell morphological image. The analyzed value may include various statistical values such as the number of fluorescent bright spots per unit area of a specific portion of a cell and the number of fluorescent bright spots per one cell region.

In the above-described image processing flow, in order to generate more accurate diagnosis support information, it is necessary to correct the erroneous detection that occurs when the cell distribution image is acquired. However, correction of erroneous detection requires complicated work and a great deal of time. Therefore, a technique capable of easily correcting erroneous detection, that is, a technique capable of efficiently obtaining a cell distribution image is desired.

Here, the specific processing content in the detection processing (step S103 in FIG. 1) of the estimation area in which the above-described erroneous detection occurs will be described.

FIG. 8 is a flow chart showing an operation flow of detection processing of an estimated region according to the reference example. As shown in FIG. 8, in the estimation region detection process according to the reference example, the processes of steps S201 to S211 are performed.

In step S201, preprocessing is performed on the cell morphology image. The pre-processing may include, for example, processing for removing noise using various filters or the like.

In step S202, a process of detecting an estimated region (also referred to as a region detection process) is performed on the cell morphology image preprocessed in step S201. In the area detection processing, for example, the first area detection processing (step S2021) and the second area detection processing (step S2022) are sequentially performed.

In the first area detection process, an estimated area (also referred to as a first estimated area) related to a specific portion captured in a relatively clear manner in the cell morphological image is detected. As a result, a cell distribution image (also referred to as a first-stage cell distribution image) in which the distribution of the first estimated region is shown by one or more cell display elements can be acquired. Here, the first estimation region in which the specific portion is captured in the cell morphology image can be detected by, for example, binarization and clustering. Further, in the first region detection processing, for example, the first estimated region may be or may not be detected from the cell morphological image according to the learning content obtained by machine learning using the feature amounts of many types of images. Of the regions, a region that satisfies the appearance standard may be detected as the first estimated region.

In the second area detection processing, an estimated area (also referred to as a second estimated area) related to a specific portion captured in a relatively unclear form in the cell morphology image is detected. As a result, a cell distribution image (also referred to as a second-stage cell distribution image) in which the distributions of the first and second estimated regions are shown by two or more cell display elements can be acquired. Here, for example, the second estimation region can be detected by binarization under different conditions from those of the first region detection processing, recognition of a closed region by edge extraction, and the like. Further, also in the second region detection processing, for example, of the second estimated regions that have been once detected, a region that satisfies the appearance criterion may be detected as the second estimated region.

FIG. 9 is a diagram showing an example of the cell morphology image G102a. FIG. 10 is a diagram showing an example of a second-stage cell distribution image G102b acquired by the first and second region detection processing from the cell morphology image G102a shown in FIG. FIG. 10 illustrates, for example, a second-stage cell distribution image in which the first and second estimated regions are shown by white cell display elements and the remaining regions are shown in black. 11 to 13 are diagrams illustrating specific cell display elements Ex1 to Ex3 in the cell distribution image in the second stage. FIG. 11: is a figure which shows an example of the cell display element Ex1 which shows the estimated area|region detected in the appropriate state (it is also called an appropriate state). FIG. 12 is a diagram showing an example of a cell display element Ex2 showing an estimated region detected in an excessively divided state (also referred to as an excessively divided state). FIG. 13 is a diagram illustrating an example of the cell display element Ex3 indicating the estimated region detected in the undivided state (also referred to as the undivided state).

In step S203, the detection results acquired in the area detection process of step S202 are classified. Here, it is classified whether the estimated region detected in step S202 is the estimated region detected in the proper state, the excessively divided state, or the undivided state.

For example, it can be classified based on the size, the shape, the density, etc. of the cell display element indicating the estimated region, which state is the estimated region detected in the proper state, the excessively divided state, or the undivided state. At this time, for example, a plurality of cell display elements in the excessively divided state can be recognized as a group of cell display elements (also referred to as a cell display element group) indicating one estimated region. Here, the number of cell display elements indicating the estimated region detected in the proper state, the number of cell display element groups indicating the estimated region detected in the excessively divided state, and the estimated region detected in the undivided state are shown. It is assumed that the total number (total number) with the number of cell regions is I (I is a natural number). At this time, for example, numbers 1 to I are given as identification information to the cell display elements in the proper state and the undivided state and the cell display element group in the excessively divided state.

In step S204, a natural number i indicating which number of the estimated region the cell display element or cell display element group in which the determination target (also referred to as the determination target) is is set to 1.

In step S205, the i-th cell display element or cell display element group of the cell display elements related to the estimated region detected in step S202 is designated as the determination target.

In step S206, the i-th cell display element or cell display element group designated as the determination target in step S205 is a cell display element indicating an estimated area classified into the estimated area detected in the proper state in step S203. Is determined. Here, if the i-th cell display element indicates the estimated region detected in the proper state, the process proceeds to step S210, and the estimated region in which the i-th cell display element or the cell display element group is detected in the proper state is detected. If not, the process proceeds to step S207.

In step S207, the i-th cell display element or cell display element group designated as the determination target in step S205 is a cell display element indicating the estimated area classified into the estimated area detected in the excessively divided state in step S203. It is determined whether there is any. If the determination target specified in step S205 is the cell display element group, the process proceeds to step S208. If the determination target specified in step S205 is not the cell display element group, the process proceeds to step S209.

In step S208, a process of integrating the i-th cell display element group indicating the estimated region classified into the estimated region detected in the excessive division state (also referred to as an integration process) is performed. In the integration process, for example, edge information obtained by a process of extracting an edge of a region including a portion corresponding to a cell display element group in a cell morphology image (also referred to as edge extraction process) is used to estimate a block. The area can be recognized. FIG. 14 is a diagram illustrating a state in which the integration process is performed on the cell display element group. In FIG. 14, the cell display element group on the left side may be integrated by the integration process to form the cell display element on the right side.

In step S209, a process of dividing the cell display element as the i-th determination target indicating the estimated region classified into the estimated region detected in the undivided state into a plurality of cell display elements (also referred to as division process) is performed. To be executed. In the division processing, for example, the edge information obtained by the edge extraction processing for the area including the portion corresponding to the cell display element in the cell morphology image can be used to recognize the plurality of estimated areas. Further, the cell display element may be divided by a known Watershed method or the like. FIG. 15 is a diagram exemplifying how division processing is applied to an undivided cell display element. In FIG. 15, the cell display element on the left side may be divided by the dividing process to form a plurality of cell display elements on the right side.

In step S210, it is determined whether or not the natural number i has reached the natural number I. Here, if the natural number i has not reached the natural number I, the process proceeds to step S211, and if the natural number i has reached the natural number I, the process proceeds to step S212.

In step S211, the natural number i is incremented by 1, and the process proceeds to step S205. Therefore, the processes of steps S205 to S211 are appropriately repeated until the natural number i reaches the natural number I.

In step S212, the cell display element relating to the proper state, the cell display element after integration, and the plurality of cell display elements after division are combined, so that the distribution of the estimated region is shown by the plurality of cell display elements. One cell distribution image (also referred to as a third stage cell distribution image) is acquired. That is, one cell distribution image is acquired from the cell morphology image by a series of processes.

FIG. 16 is a diagram schematically showing the relationship between the progress of the operation flow of the detection process and the number of estimated regions detected in the improper state. In FIG. 16, the horizontal axis represents the timing in the operation flow of the detection processing, and the vertical axis represents the number of estimated regions detected in the improper state.

As described above, in the first region detection process performed in the period from the start timing of the detection process to the timing T1, the first estimated region related to the specific portion captured in the cell morphology image in a relatively clear manner is detected. In the second region detection processing that is detected and performed in the period of timings T1 and T2, the second estimated region related to the specific portion captured in a relatively unclear manner in the cell morphology image is detected.

Therefore, as shown in FIG. 16, the number of estimated regions detected in an improper state gradually increases during the period from the start timing of the detection process to the timing T2. In the process of classifying the detection results performed in the period of timings T2 to T3, the distribution of the cell display elements does not change, so the number of estimated regions detected in the improper state hardly changes. Then, the number of estimated regions detected in the improper state increases due to the integration processing for the cell display element group related to the excessively divided state and the division processing for the cell display element related to the undivided state performed in the period of timings T3 to T4. You can In addition, the number of estimated regions detected in such an improper state depends on the state of a specific portion captured in the cell morphology image.

<(1-2) Main points of the present invention>
FIG. 17 is a diagram schematically showing the relationship between the progress of the operation flow of the detection process and the number of estimated regions detected in the improper state. In FIG. 17, as in FIG. 16, the horizontal axis represents the timing in the operation flow of the detection processing, and the vertical axis represents the number of estimated regions detected in the improper state.

FIG. 18 is a diagram illustrating a certain cell morphology image (also referred to as a first cell morphology image) G1. 19 to 21 show first to third regions generated by subjecting the first cell morphological image G1 to a first region detection process, a second region detection process, and an integration process and a division process, respectively. It is a figure which illustrates the cell distribution images G1a-G1c of a step. In addition, FIG. 22 is a diagram illustrating another cell morphology image (also referred to as a second cell morphology image) G2. 23 to 25, the first to third regions generated by subjecting the second cell morphology image G2 to the first region detection process, the second region detection process, and the integration process and the division process, respectively. It is a figure which illustrates the cell distribution images G2a-G2c of a step. In each of FIGS. 19 to 21 and FIGS. 23 to 25, the cell display element is drawn in dark gray as in FIG. 2 and the like.

In the first cell morphology image G1 shown in FIG. 18, each specific site is captured in a simple shape with relatively little distortion and in a relatively clear manner. Therefore, in the cell distribution images G1a to G1c at the first to third stages (FIGS. 19 to 21), the estimated region can be detected relatively appropriately. Then, as depicted by the thick solid polygonal line in FIG. 17, even if the number of estimated regions detected with the progress of the detection process increases, the number of estimated regions detected in an improper state is relatively difficult to increase. .. In such a case, among the cell distribution images G1a to G1c of the first to third stages, the third stage cell distribution image G1c showing the final detection result obtained by performing the detection process is erroneous. There is only a relatively small amount of cell display elements that indicate the estimated area of detection. Therefore, if the correction is applied to the cell distribution image G1c in the third stage, the user can relatively easily correct the cell display element indicating the estimated region related to the erroneous detection. That is, the cell distribution image can be efficiently acquired.

On the other hand, in the second cell morphological image G2 shown in FIG. 22, a plurality of specific parts are captured in a relatively complex shape and in a relatively unclear form. Such a state in which the plurality of specific sites have a relatively complicated shape may occur due to the presence of cancer cells, strain that may occur when a tissue section is collected in a needle biopsy, or the like. Therefore, in the cell distribution images G2a to G2c at the first to third stages (FIGS. 23 to 25), it is difficult to detect the estimated region relatively appropriately. Then, as depicted by the broken line in FIG. 17, as the number of estimated regions detected increases with the progress of the detection process, the number of estimated regions detected in the improper state tends to increase. In such a case, the cell distribution image G2c of the third stage has a relatively large number of cell display elements indicating the estimated region related to the erroneous detection, and the cell distribution image G2b of the second stage in the middle is erroneously detected. There are relatively few cell display elements that represent the putative region. Therefore, if the correction is applied to the cell distribution image G2b of the second stage rather than the cell distribution image G2c of the third stage, the user relatively displays the cell display element indicating the estimated region related to the false detection. It can be easily modified. That is, the cell distribution image can be efficiently acquired.

As described above, even in the cell distribution image finally obtained by the detection process, there are some cases where the number of estimated regions detected in the improper state is small, but conversely, in the estimated region detected in the improper state, Some have a large number.

Therefore, in the present invention, in addition to one cell distribution image finally obtained by the detection processing, processing of conditions different from the one cell distribution image such as a cell distribution image obtained in the middle of the detection processing is performed. The other cell distribution image obtained in step 1 is also displayed in a manner selectable as a correction target. In other words, the user selects from two or more cell distribution images that show the distribution of the estimated regions that are generated by image processing of the cell morphology image under different conditions and are estimated to capture a specific part of the cell. The cell distribution image to be corrected can be specified. Therefore, the user can select a cell distribution image that is easy to correct. As a result, the cell distribution image showing the distribution of the region where the specific portion of the cell is captured in the cell morphology image can be efficiently obtained by shortening the correction time.

Hereinafter, an embodiment and various modifications of the present invention will be described with reference to the drawings. In the drawings, parts having similar configurations and functions are designated by the same reference numerals, and redundant description will be omitted in the following description. Further, the drawings are shown schematically, and the sizes and positional relationships of various components in each drawing may be appropriately changed.

<(2) One embodiment>
<(2-1) Outline of pathological diagnosis support system>
FIG. 26 is a diagram showing a schematic configuration example of the pathological diagnosis support system 100 according to the embodiment. The pathological diagnosis support system 100 acquires, for example, a microscopic image of a tissue section of a living body stained with a predetermined staining reagent, performs various image processing on the microscopic image, and then identifies a specific section of the tissue section. An analysis process of calculating an analysis value related to the existence state of the biological substance is performed.

Here, the microscopic image includes, for example, a cell morphology image in which the morphology of cells in a tissue section of a living body is captured and an image corresponding to the presence state of a specific biological substance existing in the tissue section of the living body. The living body includes, for example, a human body or animals other than humans, and may also include animals in a broad sense including both human bodies and animals other than humans. The various image processing includes, for example, processing performed on the microscope image so that an analysis value related to the existence state of a specific biological substance in the analysis processing can be obtained from the microscope image with high accuracy.

As shown in FIG. 26, the pathological diagnosis support system 100 includes a microscope image acquisition device 1, an image processing device 2, and a communication line 3. The communication line 3 connects the microscope image acquisition device 1 and the image processing device 2 in a manner capable of transmitting and receiving data. Here, the communication line 3 may be a wired line such as a cable or a wireless line. Specifically, as the communication line 3, for example, a LAN (Local Area Network) in which at least one of a wired system and a wireless system is adopted can be adopted. Data may be exchanged between the microscope image acquisition apparatus 1 and the image processing apparatus 2 by delivery using various media such as a storage medium.

<(2-2) Microscope image acquisition device>
The microscope image acquisition device 1 is, for example, a known optical microscope with a camera. The microscope image acquisition apparatus 1 captures an optical image of a tissue section on a slide glass placed on the stage. As a result, data (also referred to as microscope image data) of an image (microscope image) related to the enlarged image of the tissue section is acquired, and the microscope image data is transmitted from the microscope image acquisition device 1 to the image processing device 2. Note that, hereinafter, the microscope image data and the microscope image are collectively referred to as “microscope image”.

Specifically, the microscope image acquisition device 1 includes, for example, an irradiation unit, an imaging unit, an imaging unit, a communication I/F, and the like. The irradiation unit has, for example, a light source, a filter, and the like, and irradiates the tissue section on the slide glass arranged on the stage with light. The image forming unit has, for example, an eyepiece lens, an objective lens, and the like, and forms an image of transmitted light, reflected light, or fluorescence emitted from the tissue section according to the irradiation of the tissue section on the slide with light. The imaging unit is, for example, a camera equipped with a CCD (Charge Coupled Device) sensor or the like, and captures a microscopic image by capturing an optical image of a tissue section imaged on the imaging surface by the imaging unit. The communication I/F transmits the microscope image to the image processing device 2.

Further, the microscope image acquisition apparatus 1 includes, for example, a bright field unit in which an irradiation unit and an image forming unit suitable for bright field observation are combined, and a fluorescence unit in which an irradiation unit and an image forming unit suitable for fluorescence observation are combined. Equipped with. In this case, by switching the unit used between the bright field unit and the fluorescence unit, the observation mode of the microscope image acquisition apparatus 1 is switched between the bright field observation mode and the fluorescence observation mode. .. Thereby, the microscope image acquired by the microscope image acquisition device 1 includes, for example, a bright field image obtained by imaging in bright field observation and a fluorescence image obtained by imaging in fluorescence observation.

The “bright field image” is a microscope image obtained by magnifying and imaging a tissue section stained with a predetermined staining reagent in the bright field in the microscope image acquisition apparatus 1. Here, as the predetermined staining reagent, for example, a hematoxylin-staining reagent (H staining reagent) and a hematoxylin-eosin staining reagent (HE staining reagent) can be adopted. Hematoxylin (H) is a blue-violet pigment and stains a part of cell nucleus, bone tissue, cartilage tissue, serous component and the like (basophil tissue and the like). Eosin (E) is a red to pink pigment that stains cytoplasm, connective tissue of soft tissues, erythrocytes, fibrin, endocrine granules and the like (eosinophilic tissue and the like). That is, in the present embodiment, for example, the bright field image is a cell morphology image that captures the morphology of cells in a tissue section of a living body stained with a predetermined staining reagent. FIG. 27 is a diagram showing an example of the cell morphology image Gb1.

The “fluorescence image” is obtained by irradiating a tissue section stained with a predetermined fluorescence staining reagent with excitation light having a predetermined wavelength in the microscope image acquisition apparatus 1 to generate fluorescence emission, and enlarging and forming this fluorescence. It is a microscope image obtained by imaging. Here, as the fluorescent staining reagent, for example, a staining reagent having a fluorescent labeling material containing fluorescent substance-encapsulating nanoparticles having a biological substance recognition site that specifically binds and/or reacts with a specific biological substance is adopted. It The fluorescent substance-containing nanoparticles are nanoparticles containing a fluorescent substance. The fluorescence appearing in the fluorescence observation is generated by exciting the fluorescent substance-encapsulating nanoparticles (specifically, the fluorescent substance) of the fluorescent staining reagent, and the specific living body corresponding to the biological substance recognition site in the tissue section. It shows the expression of a substance. In particular, when fluorescent substance-encapsulating nanoparticles (also referred to as fluorescent particles) encapsulating a fluorescent substance are adopted, the expression level of a specific biological substance can be calculated not only as the brightness of fluorescent particles but also as the number of particles. Can be quantified. That is, in the present embodiment, for example, the fluorescence image is an image in which a tissue section of a living body in which a specific biological material is stained with a fluorescent staining reagent is captured, and a distribution of the specific biological material existing in the tissue section of the living body. It is an image having substance distribution information relating to a state. FIG. 28 is a diagram showing an example of the fluorescence image Gd1.

<(2-2-1) Method for acquiring fluorescence image>
Here, a method for acquiring a fluorescence image will be described including a fluorescent staining reagent used for acquiring the fluorescence image, a method for staining a tissue section with the fluorescent staining reagent, and the like.

<(2-2-1-1) Fluorescent staining reagent>
1) Fluorescent substance.
As the fluorescent substance used for the fluorescent dyeing reagent, for example, a fluorescent organic dye and quantum dots (semiconductor particles) can be adopted. As the fluorescent substance, for example, when excited by near-infrared light from ultraviolet light having a wavelength in the range of 200 to 700 nm, it emits visible light to near-infrared light having a wavelength in the range of 400 to 1100 nm. What to present can be adopted.

Note that, for example, when tissue and eosin are irradiated with light having an excitation wavelength of 350 nm, the tissue and eosin emit fluorescence of wavelengths ranging from about 400 to 600 nm (having a peak at about 440 nm) by autofluorescence. Further, when the tissue and eosin are irradiated with light having an excitation wavelength of 490 nm, for example, the tissue and eosin emit fluorescence of wavelengths ranging from about 500 to 650 nm (having a peak at about 540 nm) by autofluorescence. Therefore, if a fluorescent substance that emits fluorescence that overlaps with the autofluorescence of tissue and eosin when irradiated with excitation light of the same wavelength is selected as the fluorescent substance used for the fluorescent staining reagent, a common cut filter is used. Thus, their fluorescence can be observed in the same field of view. In this case, since the fluorescence emitted by the fluorescent particles has high brightness, it can be identified without being buried in the autofluorescence of the tissue or the like. The intensity of autofluorescence emitted by the tissue and the intensity of fluorescence emitted by the fluorescent substance-encapsulating nanoparticles are appropriately selected by selecting the wavelength of the excitation light or the fluorescent substance such that the degree of overlap of the fluorescent wavelengths is preferable. Can be adjusted to have a proper balance.

Examples of fluorescent organic dyes include fluorescein dye molecules, rhodamine dye molecules, Alexa Fluor (registered trademark, manufactured by Invitrogen) dye molecules, BODIPY (registered trademark, manufactured by Invitrogen) dye molecules, cascade dye molecules, coumarin. -Based dye molecules, eosin-based dye molecules, NBD-based dye molecules, pyrene-based dye molecules, Texas Red (registered trademark)-based dye molecules, cyanine-based dye molecules, perylene-based dye molecules, and/or oxazine-based dye molecules obtain.

Specifically, for example, 5-carboxy-fluorescein, 6-carboxy-fluorescein, 5,6-dicarboxy-fluorescein, 6-carboxy-2',4,4',5',7,7'-hexachlorofluorescein. , 6-carboxy-2',4,7,7'-tetrachlorofluorescein, 6-carboxy-4',5'-dichloro-2',7'-dimethoxyfluorescein, naphthofluorescein, 5-carboxy-rhodamine, 6 -Carboxy-rhodamine, 5,6-dicarboxy-rhodamine, rhodamine 6G, tetramethylrhodamine, X-rhodamine, and Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, 514. Alexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 700, Alexa Fluor 633, Alexa Fluor 633, Alexa Fluor 6 750, BODIPY FL, BODIPY TMR, BODIPY 493/503, BODIPY 530/550, BODIPY 558/568, BODIPY 564/570, BODIPY 576/589, BODIPY 581/591, BODIPY 650/630, 650/650, BODIPY 630/650, 650/650. (Manufactured by Mfg. Co., Ltd.), methoxycoumarin, eosin, NBD, pyrene, Cy5, Cy5.5, Cy7 and the like, or a mixture of two or more kinds of fluorescent organic dyes may be employed.

As the quantum dots, for example, quantum dots containing II-VI group compounds as components (also referred to as II-VI group quantum dots), quantum dots containing III-V group compounds as components (III-V group quantum dots). (Also referred to as “) and a quantum dot containing a group IV element as a component (also referred to as a group IV quantum dot), one quantum dot or a mixture of two or more quantum dots may be employed.

Specifically, for example, one type of quantum dot among CdSe, CdS, CdTe, ZnSe, ZnS, ZnTe, InP, InN, InAs, InGaP, GaP, GaAs, Si and Ge, or two or more types of quantum dots. Can be used as a mixture. A quantum dot in which the above-mentioned quantum dot serves as a core and a shell is provided on the core may be adopted. In the present specification, a quantum dot having a shell is described as CdSe/ZnS when the core is CdSe and the shell is ZnS. Then, for example, CdSe/ZnS, CdS/ZnS, InP/ZnS, InGaP/ZnS, Si/SiO ₂ , Si/ZnS, Ge/GeO ₂ and Ge/ZnS can be adopted.

As the quantum dots, those subjected to a surface treatment with an organic polymer or the like may be used as necessary. For example, CdSe/ZnS having a surface carboxyl group (manufactured by Invitrogen), CdSe/ZnS having a surface amino group (manufactured by Invitrogen) and the like can be mentioned.

2) Fluorescent substance-encapsulated nanoparticles.
In the fluorescent substance-containing nanoparticles (fluorescent particles), the fluorescent substance is dispersed inside the nanoparticles, and the fluorescent substance and the nanoparticles themselves may or may not be chemically bonded. As the material forming the nanoparticles, for example, polystyrene, polylactic acid, silica, melamine and the like can be adopted. Such fluorescent substance-encapsulated nanoparticles can be produced, for example, according to a known method.

Specifically, the silica nanoparticles encapsulating the fluorescent organic dye can be synthesized with reference to, for example, the synthesis method of FITC-encapsulating silica particles described in Langmuir Vol. 8, page 2921 (1992). By using a desired fluorescent organic dye instead of FITC, silica nanoparticles containing various fluorescent organic dyes can be synthesized. Further, the polystyrene nanoparticles encapsulating a fluorescent organic dye are described in US Pat. No. 4,326,008 (1982), which is a copolymerization method using an organic dye having a polymerizable functional group, or US Pat. No. 5,326,692 (1992). It can be prepared by a method of impregnating existing polystyrene nanoparticles with a fluorescent organic dye.

Further, the silica nanoparticles encapsulating the quantum dots can be synthesized with reference to, for example, the method for synthesizing CdTe-encapsulating silica nanoparticles described in New Journal of Chemistry, Vol. 33, page 561 (2009). Moreover, the polymer nanoparticle encapsulating the quantum dot can be produced, for example, by the method of impregnating the polystyrene nanoparticle with the quantum dot, which is described in Nature Biotechnology, Vol. 19, p. 631 (2001).

As the average particle size of the fluorescent substance-encapsulated nanoparticles, for example, the accessibility to the antigen and the signal of the fluorescent particle are usually set so as not to be buried in background noise (camera noise, cell autofluorescence, etc.). Is about 30 to 800 nm. The coefficient of variation showing the variation in the particle diameter of the fluorescent substance-containing nanoparticles is usually 20% or less, for example. Regarding the average particle size, for example, the cross-sectional area of 1000 particles is measured from an image capturing the cross-sections of a large number of particles obtained by photographing using a scanning electron microscope (SEM), and each measured value is The diameter of the circle when the area of the circle is defined as the particle diameter, and the arithmetic mean thereof can be calculated as the average particle diameter. The coefficient of variation is a value (100×(standard deviation of particle size)/(average particle size)) that can be calculated from the measured particle size distribution.

3) Biomaterial recognition site.
The biological substance recognition site is a site that specifically binds and/or reacts with a target biological substance. As the target biological substance, for example, proteins (peptides) and nucleic acids (oligonucleotides, polynucleotides) and antibodies can be typically used. That is, such a substance that binds to a target biological substance has an antibody that recognizes the protein as an antigen, another protein that specifically binds to the protein, and a base sequence that hybridizes to the nucleic acid. Nucleic acid and the like can be adopted.

Specifically, as a substance that binds to a target biological substance, an anti-HER2 antibody that specifically binds to HER2, which is a protein present on the cell surface, and an estrogen receptor (ER) that exists in the cell nucleus, are specifically bound. Anti-ER antibody, an anti-actin antibody that specifically binds to actin forming a cytoskeleton, and the like can be used. Among them, those in which the anti-HER2 antibody and the anti-ER antibody are bound to the fluorescent substance-encapsulating nanoparticles can be used for selecting a dosage for breast cancer.

As a mode of binding the biological substance recognition site and the fluorescent substance-encapsulated nanoparticles, for example, covalent bond, ionic bond, hydrogen bond, coordinate bond, physical adsorption and/or chemical adsorption can be adopted. If the bond has a strong binding force such as a covalent bond, the bond can be stable. Further, the biological substance recognition site and the fluorescent substance-encapsulating nanoparticles may be connected by an organic molecule. As this organic molecule, for example, a polyethylene glycol chain or SM (PEG) 12 manufactured by Thermo Scientific may be adopted in order to suppress non-specific adsorption with a biological substance.

When the biological substance recognition site is bound to the fluorescent substance-encapsulated silica nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. For example, a silane coupling agent, which is a compound widely used for binding an inorganic substance and an organic substance, can be used for the binding. This silane coupling agent has an alkoxysilyl group that gives a silanol group by hydrolysis at one end of the molecule and a functional group (eg, carboxyl group, amino group, epoxy group, aldehyde group, etc.) at the other end. It is a compound and binds to an inorganic substance through the oxygen atom of the silanol group. Specifically, as such a silane coupling agent, mercaptopropyltriethoxysilane, glycidoxypropyltriethoxysilane, aminopropyltriethoxysilane, a silane coupling agent having a polyethylene glycol chain (for example, PEG manufactured by Gelest Co., Ltd. -Silane no.SIM6492.7) and the like are adopted. When the silane coupling agent is used, for example, two or more kinds of silane coupling agents may be used together.

Here, as a reaction procedure between the silica nanoparticles containing the fluorescent organic dye and the silane coupling agent, a known procedure can be adopted. For example, first, the obtained silica nanoparticles containing a fluorescent organic dye are dispersed in pure water, aminopropyltriethoxysilane is added, and the mixture is reacted at room temperature for 12 hours. After the completion of the reaction, the fluorescent organic dye-encapsulated silica nanoparticles having the surface modified with an aminopropyl group can be obtained by centrifugation or filtration. Then, by reacting the amino group with the carboxyl group in the antibody, the antibody can be bonded to the fluorescent organic dye-encapsulated silica nanoparticles through an amide bond. If necessary, a condensing agent such as EDC (1-Ethyl-3-[3-Dimethylaminopropyl]carbodiimide Hydrochloride: manufactured by Pierce (registered trademark)) may be used.

Further, if necessary, a linker compound having a site capable of directly binding to the fluorescent organic dye-encapsulated silica nanoparticles modified with an organic molecule and a site capable of binding to a molecular target substance may be employed. As a specific example, sulfo-SMCC (Sulfosuccinimidyl 4 [N-maleimidomethyl]-cyclohexane-1-carboxylate: Pierce (registered trademark)) having both a site that selectively reacts with an amino group and a site that selectively reacts with a mercapto group Manufactured by the company) is used, the amino group of the fluorescent organic dye-encapsulated silica nanoparticles modified with aminopropyltriethoxysilane and the mercapto group in the antibody are bound to each other, thereby binding the antibody-conjugated fluorescent organic dye-encapsulated silica nanoparticles. Can be obtained.

When the biological substance recognition site is bound to the fluorescent substance-encapsulated polystyrene nanoparticles, the same procedure can be applied regardless of whether the fluorescent substance is a fluorescent organic dye or a quantum dot. That is, by impregnating the polystyrene nanoparticles having a functional group such as an amino group with the fluorescent organic dye or the quantum dots, the fluorescent substance-encapsulating polystyrene nanoparticles having a functional group can be obtained. Thereafter, antibody-bound fluorescent substance-encapsulated polystyrene nanoparticles can be obtained by using EDC or sulfo-SMCC.

In the case of binding the biological substance recognition site to the fluorescent substance-encapsulated melamine nanoparticles, the same procedure as in the case of binding the biological substance recognition site to the fluorescent substance-encapsulated silica nanoparticles can be applied. Further, in order to further improve the reactivity, the number of surface amino groups may be increased by previously reacting the melamine nanoparticles with the polyfunctional amine compound.

Examples of antibodies that recognize specific antigens include M. actin, MS actin, SM actin, ACTH, Alk-1, α1-antichymotrypsin, α1-antitrypsin, AFP, bcl-2, bcl-6, β-catenin. , BCA 225, CA19-9, CA125, calcitonin, calretinin, CD1a, CD3, CD4, CD5, CD8, CD10, CD15, CD20, CD21, CD23, CD30, CD31, CD34, CD43, CD45, CD45R, CD56, CD57, CD61, CD68, CD79a, CD99, MIC2, CD138, chromogranin, c-KIT, c-MET, collagen type IV, Cox-2, cyclin D1, keratin, cytokeratin (high molecular weight), pankeratin, pankeratin, cytokeratin 5/6, cytokeratin 7, cytokeratin 8, cytokeratin 8/18, cytokeratin 14, cytokeratin 19, cytokeratin 20, CMV, E-cadherin, EGFR, ER, EMA, EBV, factor VIII-related antigen, Fascin, FSH, galectin-3, gastrin, GFAP, glucagon, glycophorin A, granzyme B, hCG, hGH, Helicobacter pylori, HBc antigen, HBs antigen, hepatocyte-specific antigen, HER2, HSV-I, HSV-II, HHV- 8, IgA, IgG, IgM, IGF-1R, inhibin, insulin, kappa L chain, Ki67, lambda L chain, LH, lysozyme, macrophage, melanin A, MLH-1, MSH-2, myeloperoxidase, myogenin, myoglobin , Myosin, neurofilament, NSE, p27 (Kip1), p53, p53, P63, PAX 5, PLAP, Pneumocystis carinii, podoplanin (D2-40), PGR, prolactin, PSA, prostatic acid phosphatase, Renal Cell Carcinoma, S100, somatostatin, spectrin, synaptophysin, TAG-72, TdT, thyroglobulin, TSH, TTF-1, TRAcP, tryptase, villin, vimentin, WT1, Zap-70 and the like can be mentioned.

<(2-2-1-2) Method of staining tissue section>
Here, a method of staining a tissue section will be described. Here, the case where the object to be stained is a tissue specimen related to a tissue section is described, but the object to be stained is not limited to this, and may be various specimens such as cells fixed on a substrate. A tissue specimen to which the staining method described below can be applied can be produced by, for example, a known method. For example, when a paraffin-embedded section is used as a tissue sample, staining can be performed according to the following processing procedure in which deparaffinization processing, activation processing, and staining processing are sequentially performed.

1) Deparaffinization treatment.
The tissue specimen is immersed in a container containing xylene to remove paraffin. At this time, the treatment temperature may be an appropriate temperature such as room temperature, and the immersion time may be 3 minutes or more and 30 minutes or less. Further, xylene may be exchanged during the immersion, if necessary.

Next, the tissue specimen is immersed in a container containing ethanol to remove xylene. At this time, the treatment temperature may be an appropriate temperature such as room temperature, and the immersion time may be 3 minutes or more and 30 minutes or less. Further, ethanol may be exchanged during the immersion, if necessary.

Then, the tissue specimen is immersed in a container containing water to remove ethanol. At this time, the treatment temperature may be an appropriate temperature such as room temperature, and the immersion time may be 3 minutes or more and 30 minutes or less. Further, water may be exchanged during the immersion, if necessary.

2) Activation treatment.
According to a known method, the target biological substance is activated. In the activation treatment, 0.01 M citrate buffer solution (pH 6.0), 1 mM EDTA solution (pH 8.0), 5% urea, 0.1 M Tris-hydrochloric acid buffer solution, etc. can be used as the activation solution. At this time, an autoclave, a microwave, a pressure cooker, a water bath, or the like can be adopted as the heating device. The treatment temperature may be an appropriate temperature such as room temperature. Specifically, a condition that the treatment temperature is 50 to 130° C. and the treatment time is 5 to 30 minutes can be adopted.

Next, the tissue sample after the activation treatment is immersed in a container containing PBS (Phosphate Buffered Saline) and washed. At this time, the treatment temperature may be an appropriate temperature such as room temperature, and the immersion time may be 3 minutes or more and 30 minutes or less. Further, the PBS may be replaced during the immersion if necessary.

3) Staining treatment.
A PBS dispersion of fluorescent substance-encapsulated nanoparticles having a biological substance recognition site bound thereto is placed on a tissue sample and reacted with a target biological substance. By changing the biomaterial recognition site bound to the fluorescent substance-encapsulated nanoparticles, it becomes possible to stain various biomaterials. When fluorescent substance-encapsulating nanoparticles to which several types of biological substance-recognizing sites are bound are used, the respective fluorescent substance-encapsulating nanoparticles in PBS may be mixed in advance, or they may be separately and sequentially tissue-treated. May be placed on the specimen. At this time, the treatment temperature may be an appropriate temperature such as room temperature, and the immersion time may be 30 minutes or more and 24 hours or less. Here, a known blocking agent such as PBS containing BSA may be dropped onto the tissue sample before staining with the nanoparticles encapsulating the fluorescent substance.

Next, the stained tissue sample is immersed in a container containing PBS to remove unreacted fluorescent substance-encapsulated nanoparticles. At this time, the treatment temperature may be an appropriate temperature such as room temperature, and the immersion time may be 3 minutes or more and 30 minutes or less. Further, the PBS may be replaced during the immersion if necessary.

Note that, for example, when morphological observation is performed by utilizing autofluorescence emitted by cells or tissues themselves, it is not necessary to use other staining solution, etc., but it is generally used in tissue observation by an optical microscope. HE (hematoxylin-eosin) staining may be performed together. Here, eosin can emit autofluorescence. Therefore, when HE staining is performed together with the staining with the fluorescent labeling material, when irradiating with excitation light, eosin that stains the cytoplasm and the like can emit the autofluorescence along with the autofluorescence emitted from the tissue and the like. When such a configuration is adopted, it becomes easier to obtain information on the morphology of cells or tissues by imaging one visual field.

After the staining process, a cover glass is placed on the stained tissue sample to encapsulate the tissue sample. At this time, a commercially available mounting medium may be used if necessary.

<(2-2-1-3) Acquisition of fluorescence image>
The microscope image acquisition apparatus 1 is used for a tissue sample as a stained tissue section to acquire a fluorescence image as a wide-field microscope image. At this time, in the microscope image acquisition apparatus 1, the excitation light source and the fluorescence detection optical filter corresponding to the absorption maximum wavelength and the fluorescence wavelength of the fluorescent substance used for the staining reagent are selected.

Here, for example, by irradiating a stained tissue specimen with excitation light, cell morphology information is obtained based on the autofluorescence of the tissue and eosin, and the substance distribution information is obtained based on the fluorescence of the fluorescent labeling material. May be acquired. That is, the fluorescence image as a microscope image acquired at this time is an image (also referred to as a fluorescence morphology image) including both cell morphology information and substance distribution information. That is, the fluorescence morphology image has both a role as a cell morphology image and a role as a normal fluorescence image.

In this case, when the fluorescence morphology image is acquired, the excitation light generates autofluorescence of a desired wavelength by the tissue and eosin used as necessary, and the desired wavelength by the fluorescent substance in the fluorescent labeling material. Those having an appropriate wavelength to generate the fluorescence of the above can be adopted. Eosin is a fluorescent substance, and the fluorescence of eosin contributes to improving the visibility of cell morphology information. However, as the absorption of excitation light by eosin and the intensity of fluorescence (also called fluorescence intensity or emission intensity) increase, the visibility of fluorescence from the fluorescent labeling material decreases. Therefore, the wavelength of the excitation light, for example, to suppress the excessive emission of eosin itself and improve the visibility of the fluorescent labeling material, a difference in the amount of light emitted from eosin and the fluorescent labeling material is secured, Can be set to a wavelength that can be discernible.

For example, a mode in which the wavelength of the excitation light is set to a wavelength at which eosin itself does not absorb the excitation light or hardly absorbs the excitation light is conceivable. Under such conditions, the cell morphology information and the substance distribution information can be acquired in the same visual field by the microscope image acquisition device 1. In other words, both autofluorescence of the tissue and fluorescence emitted by the fluorescent labeling material obtained from one stained section are included in the same visual field, but they are recognized separately and based on each. Cell morphology information and substance distribution information can be obtained.

Note that, if necessary, for example, by using an appropriate filter that can sufficiently reduce one of the tissue autofluorescence and the fluorescence emitted by the fluorescent labeling material, among the cell morphology information and the substance distribution information, the cell morphology information A mode in which only substance distribution information can be acquired and a mode in which only substance distribution information can be acquired may be adopted.

<(2-3) Image processing device>
The image processing device 2 receives the microscope image transmitted from the microscope image acquisition device 1 and performs image processing on the microscope image. The image processing device 2 is realized by executing a predetermined program in the information processing device. In the present embodiment, an example in which the microscope image transmitted from the microscope image acquisition device 1 to the image processing device 2 is a cell morphology image having cell morphology information and a fluorescence image having substance distribution information will be described.

<(2-3-1) Functional configuration of information processing device>
FIG. 29 is a block diagram schematically showing a functional configuration of an information processing device that realizes the functions of the image processing device 2. As shown in FIG. 29, the information processing device includes, for example, a control unit 21, an input unit 22, a display unit 23, a communication I/F 24, a storage unit 25, and the like. The respective units 21 to 25 are connected to each other via a bus 26 so that data can be transmitted and received between them.

The control unit 21 is an electric circuit including a processor and a memory. Here, as the processor, for example, a central processing unit (CPU) can be adopted. A volatile memory such as a RAM (Random Access Memory) may be used as the memory. One processor and one memory may be provided, or a plurality of processors and memories may be provided, or one may be provided and one may be provided in a plurality. That is, the control unit 21 may be an electric circuit including one or more processors and one or more memories. The control unit 21 causes the information processing device to function as the image processing device 2 by executing the image processing program P1 (FIG. 30) stored in the storage unit 25.

The input unit 22 inputs a signal according to the operation of an operator as a user who uses the image processing apparatus 2. Here, the input unit 22 may be, for example, an operation unit to which a signal (also referred to as an operation signal) according to an operation by the user is input, or a signal according to a user's utterance (also referred to as a voice signal) may be input. It may be a voice input unit for inputting. The operation unit may include a keyboard including character input keys, number input keys, various function keys, and the like, and a pointing device such as a mouse and a touch pen. According to the operation unit, for example, an operation signal corresponding to the depression of a key on the keyboard and an operation signal corresponding to the operation of the pointing device can be input to the control unit 21.

The display unit 23 displays various images according to the signal input from the control unit 21. The display unit 23 includes, for example, a display device such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display). Here, if a touch panel or the like is adopted as the display unit 23, the correction of the cell distribution image described later can be easily executed. Note that the display unit 23 may be applied not only to a configuration including one display device but also to a configuration including, for example, two or more display devices.

The communication I/F 24 is an interface for transmitting/receiving data to/from an external device located outside the image processing apparatus 2. The external device includes, for example, the microscope image acquisition device 1. Therefore, the communication I/F 24 functions as a receiving unit that receives a microscope image such as a cell morphology image and a fluorescence image from the microscope image acquisition device 1, for example. Note that, for example, a configuration may be adopted in which the image processing device 2 includes a LAN adapter, a router, and the like, and is connected to an external device via a communication network such as a LAN.

The storage unit 25 stores various programs and various data. The storage unit 25 may be composed of, for example, an HDD (Hard Disk Drive) or a non-volatile semiconductor memory.

<(2-3-2) Functional configuration of image processing device>
FIG. 30 is a block diagram illustrating a functional configuration realized by the control unit 21 of the image processing device 2. As shown in FIG. 30, the image processing device 2 has an acquisition unit 211, a generation unit 212, a display control unit 213, a designation unit 214, a correction unit 215, and a calculation unit as a functional configuration realized by the control unit 21. 216 is provided.

The acquisition unit 211 acquires a cell morphology image and a fluorescence image as a microscope image. Here, the acquisition unit 211 acquires the cell morphology image and the fluorescence image transmitted from the microscope image acquisition device 1 via the communication I/F 24.

In addition, the acquisition unit 211 acquires a bright spot distribution image by predetermined image processing. The bright spot distribution image is an image showing a distribution of fluorescent bright spots at a specific wavelength in a fluorescence image in which a tissue section of a living body in which a specific biological substance is stained with a fluorescent staining reagent is captured. As the predetermined image processing, for example, a color component corresponding to a specific wavelength of the fluorescent bright spot is extracted from the fluorescence image, and a portion of the fluorescence image after the color component is extracted, which is less than the threshold value related to the depth, is deleted. It is possible to employ a process in which a binary image is generated by performing a threshold value process. Here, for example, if the wavelength of the light emitted from the fluorescent particles is 550 nm, only the fluorescent bright spot having the wavelength component can be extracted as an image. The binary image generated here corresponds to the bright spot distribution image. In the predetermined image processing, for example, processing for removing noise components such as autofluorescence of cells and other unnecessary signal components may be performed before the threshold processing is performed.

In the bright spot distribution image, for example, in a fluorescent image, it is possible to distinguish the distribution of the region in which the fluorescent bright points of the specific wavelength are captured and the distribution of the remaining region in which the fluorescent bright spots of the specific wavelength are not captured. In a different manner. Specifically, for example, a portion corresponding to a region (also referred to as a fluorescent bright spot region) in which a fluorescent bright spot is captured and a specific substance is estimated to be present is the first color or the first It is shown by a hatched display element (also called a bright spot display element), and the remaining portion other than that is drawn as a background with a second color or second hatch different from that of the bright spot display element. .. Here, for example, red or the like can be adopted as the first color, and white or black or the like can be adopted as the second color. Although the fluorescence image is obtained from the microscope image acquisition device 1 here, the present invention is not limited to this, and for example, in the image processing device 2, even if a bright spot distribution image is obtained from another external device or storage medium. good.

The generation unit 212 generates a cell distribution image by performing one or more types of image processing on the cell morphology image. In the cell distribution image, the distribution of the estimated region in which it is estimated that the specific part of the cell is captured in the cell morphology image is shown by one or more cell display elements. In the generation unit 212, for example, two or more cell distribution images are generated by performing at least one type of image processing of one or more types of image processing on the cell morphology image according to two or more mutually different conditions. To be done. In each cell distribution image in two or more cell distribution images, the distribution of the estimated region in the cell morphology image is shown by at least one cell display element.

Here, the two or more conditions may include conditions that define up to which of the two or more types of image processing are to be performed, and/or different processing conditions in one type of image processing. .. Therefore, in the cell distribution image generated by the generation unit 212, a cell distribution image (also referred to as an intermediate cell distribution image) as a processing result that has been subjected to image processing in the middle of one or more types of image processing in the generation unit 212. Further, a cell distribution image (also referred to as a final cell distribution image) as a processing result obtained by performing the final image processing may be included.

In the cell distribution image, for example, with respect to the cell morphology image, a distribution of estimated regions in which it is estimated that a specific region is captured and a distribution of residual regions in which it is estimated that a specific region is not captured Are shown in a distinguishable manner. Specifically, for example, a portion corresponding to the estimated region is indicated by a cell display element with a third color or a third hatching, and the remaining portion other than that is displayed as a background different from the cell display element. It is drawn with four colors or a fourth hatching. Here, for example, blue or gray or the like may be adopted as the third color, and white or black or the like may be adopted as the fourth color.

Specifically, for example, a plurality of image processes are sequentially performed on the cell morphology image G2 shown in FIG. 22, so that the cell distribution images G2a to G2c at the first to third stages (FIGS. 23 to 25) are obtained. The generated cell distribution images G2a to G2c of the first to third stages can be displayed on the display unit 23. In this case, the first and second stage cell distribution images G2a and G2b are intermediate cell distribution images, and the third stage cell distribution image G2c is the final cell distribution image.

The display control unit 213 causes the display unit 23 to display various images. Here, for example, various images can be displayed on the display unit 23 by outputting data relating to various images from the display control unit 213 to the display unit 23. Here, the various images may include, for example, a cell morphology image acquired by the acquisition unit 211, a cell distribution image generated by the generation unit 212, and the like. Specifically, the display control unit 213 generates each by performing at least one type of image processing of one or more types of image processing on the cell morphology image according to two or more conditions different from each other by the generation unit 212. The two or more cell distribution images to be displayed are displayed on the display unit 23. The two or more cell distribution images are displayed, for example, in the form of a plurality of options that the user can appropriately select.

The designation unit 214 designates one cell distribution image of the two or more cell distribution images displayed on the display unit 23 as a plurality of options according to the user's operation. For example, one of the two or more cell distribution images is designated by the designating unit 214 according to a signal input by the user via the input unit 22. As a result, one cell distribution image can be designated as a target for which correction of erroneous detection is performed.

The correction unit 215 corrects at least a part of one cell distribution image designated by the designation unit 214 according to the operation of the user. Thereby, the erroneous detection of the cell distribution image can be corrected, and the generation of the cell distribution image can be completed.

The calculation unit 216, based on the cell distribution image generated by the correction unit 215 and the bright spot distribution image acquired by the acquisition unit 211, the presence of a specific biological substance at a specific site captured in the cell morphology image. Calculate the analysis value related to the state. As the analysis value calculated here, for example, a statistical value (that is, a statistical value) such as the number of fluorescent bright spots per unit area (also referred to as the number of bright spots) can be adopted. As the unit area, for example, a unit area of the cell area and one cell area can be adopted. In this case, for example, the number of bright spots per unit area, the number of bright spots per cell region, and the like can be calculated as analysis values. Here, for example, the relationship between the cell region and the fluorescent bright spot can be recognized by superimposing the cell distribution image and the bright spot distribution image in the same visual field. The cell region may be, for example, a cell nucleus as a specific portion, or may be set as a region in a predetermined range with the cell nucleus as a reference.

<(2-3-3) Operation flow of image processing device>
31 and 32 are flowcharts illustrating the operation flow of the image processing apparatus 2. Here, an example is shown in which a cell morphology image and a fluorescence image as a microscope image are input to the image processing device 2. As shown in FIG. 31, in the image processing device 2, for example, the processes of steps S1 to S7 are performed in time sequence.

In step S1, the acquisition unit 211 acquires a microscope image. Microscopic images include cell morphology images and fluorescence images. For example, the microscope image acquired by the microscope image acquisition device 1 is input to the control unit 21 via the communication I/F 24 and is acquired by the acquisition unit 211. FIG. 33 is a diagram showing an example of the cell morphology image G3 obtained here.

In step S2, the acquisition unit 211 acquires a bright spot distribution image showing the distribution of fluorescent bright spots associated with a specific wavelength in the fluorescent image acquired in step S1. FIG. 34 is a diagram schematically showing an example of the bright spot distribution image G3d acquired here.

In step S3, the generation unit 212 detects the estimated region in the cell morphology image, thereby generating a cell distribution image. Here, the generation unit 212 performs at least one type of image processing on the cell morphology image according to two or more mutually different conditions, thereby generating two or more cell distribution images. In two or more cell distribution images, at least one cell display element shows the distribution of the estimated region in which it is estimated that the specific portion of the cell is captured in the cell morphology image.

In the present embodiment, the one or more types of image processing performed on the cell morphology image to generate the cell distribution image in the generation unit 212 includes a plurality of types of image processing. Then, a plurality of types of image processing are sequentially performed. It should be noted that as for the two or more conditions, an example is shown which is a condition that defines up to which of the two or more types of image processing is to be performed. For this reason, the two or more cell distribution images generated by the generation unit 212 are, for example, cells generated by performing image processing up to two or more stages of two or more types of image processing. A distribution image is included. That is, when the generation unit 212 performs a plurality of types of image processing on the cell morphology image in time sequence, each of the two or more types of image processing of the plurality of types of image processing is performed on the cell morphology image. By doing so, two or more cell distribution images are generated.

Then, in step S3, specifically, the process according to the operation flow shown in FIG. 32 is executed.

In step S31 of FIG. 32, preprocessing is performed on the cell morphology image acquired in step S1. The preprocessing includes, for example, processing for removing noise using various filters such as a low-pass filter, a Gaussian filter, and a median filter.

In step S32, a region detection process for detecting an estimated region is performed on the cell morphology image subjected to the preprocessing in step S31. In the area detection processing, for example, the first area detection processing (step S321) and the second area detection processing (step S322) are sequentially performed.

In the first region detection process, an estimated region (also referred to as a first estimated region) in which the specific portion is estimated to be captured in a relatively clear manner in the cell morphological image is detected. Thereby, the first-stage cell distribution image in which the distribution of the first estimated region is shown by one or more cell display elements can be acquired. Here, the estimated region can be detected in the cell morphology image by, for example, binarization and clustering. The binarization may include, for example, binarization using the discriminant analysis method and the P-tile method. Clustering may include, for example, clustering using the k-means method, the EM algorithm, or the like. Further, in the first region detection processing, for example, the estimated region may be detected from the cell morphology image based on the learning content obtained by machine learning using the feature amounts of many types of images. Further, in the first region detection processing, for example, of the estimated regions once detected, an estimated region that satisfies the appearance criterion may be detected as the first estimated region, and the remaining estimated regions may be excluded. Appearance criteria may include, for example, size and circularity.

FIG. 35: is a figure which illustrates the cell distribution image G3a of the 1st step acquired by detecting a 1st estimation area|region from the cell morphology image G3 as shown in FIG. The first-stage cell distribution image G3a shown in FIG. 35 has a mode in which the cell display element D1a indicating the first estimated region is superimposed on the cell morphology image G3. Here, the cell display element D1a is shown by, for example, a specific element that can be distinguished from various parts captured in the cell morphology image G3. In FIG. 35, the specific element is an element drawn in dark gray.

In the second area detection process, an estimated area (also referred to as a second estimated area) in which it is estimated that the specific portion is captured in a relatively unclear manner in the cell morphology image is detected. As a result, a second-stage cell distribution image in which the distributions of the first and second estimated regions are shown by two or more cell display elements can be acquired. Here, for example, the estimation region can be extracted by binarization under different conditions from the first region detection process, recognition of a closed region by edge extraction, or the like. Further, also in the second region detection processing, for example, of the estimated regions once detected, the estimated region that satisfies the appearance criterion may be detected as the second estimated region, and the remaining estimated regions may be excluded. Appearance criteria may include, for example, size, shade, color and circularity. As the color, for example, the color in the estimated region once detected in the second region detection processing in the cell morphology image can be adopted.

FIG. 36: is a figure which illustrates the cell distribution image G3b of the 2nd step acquired by detecting a 2nd estimation area|region from the cell distribution image G3a of the 1st step as shown in FIG. The second-stage cell distribution image G3b shown in FIG. 36 has a mode in which the cell display element D1b indicating the first estimation region and the second estimation region is superimposed on the cell morphology image G3. Here, like the cell display element D1a, the cell display element D1b is indicated by, for example, a specific element that can be distinguished from various regions captured in the cell morphological image G3. In FIG. 36, as in FIG. 35, the specific element is an element drawn in dark gray.

In step S33, the detection results acquired by the area detection process of step S32 are classified. Here, it is classified whether the estimated region detected in step S32 is the estimated region detected in the proper state, the excessively divided state, or the undivided state.

Specifically, for example, each estimated region is detected in any state of an appropriate state, an excessively divided state, and an undivided state based on the size, shape, and density of the cell display element indicating the estimated region. It can be classified as an estimated region.

More specifically, for example, if all the numerical values indicating the size and shape of the estimated region detected in step S32 are included in the preset allowable range, the estimated region is detected in an appropriate state. Can be classified as being the estimated region. Here, the numerical value indicating the size of the estimated region is, for example, the area, and the numerical value indicating the shape of the estimated region is, for example, the circularity and the area convex hull ratio. The area can be calculated by, for example, the number of pixels forming the estimated region. The circularity Dc can be calculated by, for example, an equation of Dc=4π×S/L ^{2 in which} the area S and the circumference length L are used. The area convex hull ratio can be calculated, for example, by dividing the area of the one block of the estimation region by the sum of the area of the one block of the estimation region and the area of all the recesses of the one block of the estimation region. Here, the allowable range of the area can be appropriately set in advance, for example, according to the general size of the cell nucleus as the specific site. The allowable range of the circularity can be set in advance, for example, in a range of a predetermined value or more. The permissible range of the area convex hull ratio can be set in advance to a value range equal to or larger than a predetermined threshold value, for example.

Further, for example, if all of the numerical values indicating the size and the density of the estimated region detected in step S32 are included in the preset range (also referred to as the reference range), the estimated region is over-divided. It can be classified as being the estimated region detected in the state. Here, the numerical value indicating the size of the estimated region includes, for example, the area, and the numerical value indicating the density of the estimated region includes, for example, the number of the estimated regions per unit area. Further, here, the reference value range of the area may be set in advance to, for example, a value range less than the permissible range of the estimated area detected in the proper state. The unit area can be set in advance, for example, to an area corresponding to the general size of the cell nucleus as the specific portion. In this case, the reference value range related to the number of estimated regions per unit area can be set in advance to a value range indicating two or more, for example.

Further, for example, if all the numerical values indicating the size and the shape of the estimated region detected in step S32 are included in the preset value range (reference value range), the estimated region is detected in the undivided state. Can be determined to be the estimated region. Here, the numerical value indicating the size of the estimated region is, for example, the area, and the numerical value indicating the shape of the estimated region is, for example, the circularity and the area convex hull ratio. Further, here, the reference value range of the area may be set in advance, for example, to a value range that exceeds the allowable range related to the estimated region detected in the proper state. The reference range of the circularity can be set in advance to a range below a predetermined value, for example. The reference range of the area convex hull ratio may be set in advance to a range below a predetermined threshold, for example. Note that, instead of whether the area convex hull ratio is included in the reference value range, for example, the presence or absence of a recess may be used as an index.

Then, in step S33, for example, the plurality of cell display elements detected in the excessive division state can be recognized as a group of cell display elements (cell display element group) indicating one estimated region. In addition, here, the number of cell display elements indicating the estimated region detected in the proper state, the number of cell display element groups indicating the estimated region detected in the excessively divided state, and the estimated region detected in the undivided state It is assumed that the total (total number) with the number of cell regions indicating M is M (M is a natural number). At this time, for example, numbers 1 to M are assigned as identification information to the cell display elements in the proper state and the undivided state and the cell display element group in the excessively divided state.

In step S34, a natural number m indicating which cell display element or cell display element group the determination target belongs to is set to 1.

In step S35, the m-th cell display element or cell display element group of the cell display elements related to the estimated region detected in step S32 is designated as the determination target.

In step S36, the m-th cell display element or cell display element group designated as the determination target in step S35 is a cell display element indicating an estimated area classified into the estimated area detected in the proper state in step S33. Is determined. Here, if the m-th cell display element indicates the estimated region detected in the proper state, the process proceeds to step S40, and the estimated region in which the m-th cell display element or the cell display element group is detected in the proper state is detected. If not, the process proceeds to step S37.

In step S37, the m-th cell display element or cell display element group designated as the determination target in step S35 is a cell display element group indicating an estimated area classified into the estimated area detected in the excessively divided state in step S33. Is determined. Here, if the m-th cell display element group indicates the estimated region detected in the excessive division state, the process proceeds to step S38, and the m-th cell display element indicates the estimated region detected in the excessive division state. If not, the process proceeds to step S39. Here, for example, if the m-th determination target is the cell display element group, the process proceeds to step S38, and if the m-th determination target is the cell display element, the process may proceed to step S39.

In step S38, an integration process is performed in which the m-th cell display element group determined to be the cell display element group indicating the estimated region classified into the estimated region detected in the excessively divided state in step S37 is integrated. To be done. In the integration process, for example, as shown in FIG. 14, the cell display element group on the left side of FIG. 14 can be integrated to form the cell display element on the right side of FIG. 14. As the integration process, for example, a series of processes in which the following processes A1 to A5 are sequentially performed can be adopted.

[Processing A1] In the cell morphology image, an edge extraction process of extracting an edge is performed on a region including the estimated region detected in the excessively divided state corresponding to the cell display element group as the determination target. The edge extraction process can be realized by, for example, a process using a Sobel filter, a Canny filter, a differential image, or the like.

[Process A2] An estimated point (also referred to as an estimated center point) of the center point of the arc formed by the edge is obtained from the curvature of the curved edge extracted by the edge extraction process of the process A1.

[Processing A3] Among the edges extracted by the edge extracting process of the process A1, the edge satisfying the following three conditions (conditions A3a to A3c) is recognized as the one that captures the contour of the cell nucleus as the specific portion.

Condition A3a) The arc formed by the edge has a convex shape in the opposite direction to the estimated center point obtained in the process A2.

Condition A3b) a line segment connecting a point moving in one direction on the edge and the estimated center point obtained in the process A2, and a line connecting the reference point on the edge and the estimated center point (also referred to as a reference line). The angle formed by is continuously changing.

-Condition A3c) The curvatures of the arcs formed by the edges are substantially the same.

[Processing A4] The closed region formed by the edge capturing the contour of the cell nucleus recognized in the process A3 is determined as indicating a single estimated region.

[Processing A5] The cell display element group as the determination target is replaced with a cell display element of a block indicating the estimated region of the cell determined in the process A4. As a result, the cell display element group as the determination target is substantially integrated.

In the process A1, one continuous curved edge forming the closed region may not be extracted, and an edge may be extracted from the cell morphology image in the form of two or more curved edges. In this case, for example, in the process A2, a generally common estimated center point is obtained for two or more curved edges. Next, for example, in the process A3, if the two or more curved edges constitute a part of one substantially circular virtual curve that satisfies the above conditions A3a to A3c, Recognized as capturing the outline. At this time, for example, in the process A4, two or more curved edges form one substantially circular curve by two or more substantially arc-shaped curves (also referred to as connection curves) centered on the estimated center point. Connected to form. Thereby, the closed region formed by the one substantially circular curve can be determined as indicating a block of estimated regions.

In step S39, division processing is performed in which the m-th cell display element determined to be the cell display element indicating the estimated region classified into the estimated region detected in the undivided state in step S37 is divided. .. In the division processing, for example, as shown in FIG. 15, the cell display element on the left side of FIG. 15 may be divided to form a plurality of cell display elements on the right side of FIG. 15. As the division process, for example, a series of processes in which the following processes B1 to B4 are sequentially performed can be adopted.

[Process B1] The edge extraction process is performed on the region including the estimated region detected in the undivided state corresponding to the cell display element as the determination target in the cell morphology image. The edge extraction process can be realized by a process using, for example, a Sobel filter, a Canny filter, a differential image, or the like, similar to the process A1.

[Process B2] An estimated point (also referred to as an estimated center point) of the center point of the arc formed by the edge is obtained from the curvature of the curved edge extracted by the edge extraction process of the process B1. Here, the estimated center point is obtained for each of the two or more edges extracted in the process A1.

[Process B3] In process B2, two or more estimated center points related to two or more edges are obtained, and two arc-shaped edges are opposed to each other in opposite directions to the respective estimated center points. , Are recognized as edges corresponding to the contours of different estimation regions.

[Process B4] The m-th cell display element to be determined is divided between the two arc-shaped edges corresponding to the contours of the different estimation regions recognized in process B3. As the division process, for example, a process of dividing the cell display element by drawing a boundary line on the m-th cell display element to be determined by a known Watershed method or the like can be adopted.

In step S40, it is determined whether or not the natural number m has reached the natural number M. Here, if the natural number m has not reached the natural number M, the process proceeds to step S41, and if the natural number m has reached the natural number M, the process proceeds to step S42.

In step S41, the natural number m is incremented by 1, and the process proceeds to step S35. Therefore, the processes of steps S35 to S41 are appropriately repeated until the natural number m reaches the natural number M.

In step S42, the cell display element related to the proper state, the cell display element after integration, and the plurality of cell display elements after division are combined to show the distribution of the estimated region by the plurality of cell display elements. One third stage cell distribution image is acquired.

FIG. 37 shows that the cell distribution image G3b at the second stage as shown in FIG. 36 is subjected to integration processing and/or division processing, and then a plurality of cell display elements indicating a plurality of estimated regions are combined. It is a figure which illustrates the acquired cell distribution image G3c of the 3rd step. The cell distribution image G3c at the third stage shown in FIG. 37 shows the first estimation region and the second estimation region on the cell morphology image G3, and the cell display element D1c after the division process and the integration process is performed. It has a superimposed aspect. Here, the cell display element D1c is shown as a specific element that can be distinguished from various parts captured in the cell morphological image G3, like the cell display elements D1a and D1b. In FIG. 37, as in FIGS. 35 and 36, the specific element is an element drawn in dark gray.

That is, a series of processes including the first region detection process (step S321), the second region detection process (step S322), the integration process (step S38), the division process (step S39), and the combination process (step S42) are executed. To be done. At this time, for example, a third-stage cell distribution image G3c as one cell distribution image is acquired from the cell morphology image G3. This completes the generation of the cell distribution image in step S3.

In step S4 of FIG. 31, the display control unit 213 causes the display unit 23 to display the two or more cell distribution images generated in step S3. At this time, the display control unit 213 causes the display control unit 213 to display two or more cell distribution images generated by subjecting the cell morphology image to each of two or more types of image processing of the plurality of types of image processing in step S3. 23 is displayed. For example, a mode in which the cell distribution images G3a to G3c (FIGS. 35 to 37) of the first to third stages are presented in a state where they are arranged as options can be considered. Furthermore, for example, by displaying the cell morphology image G3 (FIG. 33) that is the source of the cell distribution images G3a to G3c in the first to third stages, the user can select the cell morphology image G3 and the first to third stages. Can be compared with the cell distribution images G3a to G3c.

FIG. 38 is a diagram illustrating a screen (also referred to as an image designation screen) Gs1 in which the cell morphology image G3 and the cell distribution images G3a to G3c in the first to third stages are displayed side by side. In the image designation screen Gs1, for example, one desired cell distribution image among the cell distribution images G3a to G3c in the first to third stages can be designated in accordance with the user's operation. Note that FIG. 38 shows a state in which the second-stage cell distribution image G3b is designated as one option and the second-stage cell distribution image G3b is surrounded by a bold frame. When the confirm button B1 is pressed in this state, the designation of the cell distribution image is confirmed. On the other hand, when the clear button B2 is pressed, the cell distribution image is designated again from the beginning.

In step S5, the designation unit 214 designates one cell distribution image of the two or more cell distribution images displayed on the display unit 23 according to the user's operation. At this time, for example, a mode in which one of the cell distribution images G3a to G3c of the first to third stages displayed on the display unit 23 is designated can be considered. At this time, for example, if the cell morphology image G3 is displayed together with the cell distribution images G3a to G3c of the first to third stages as options, the user can select the cell morphology image G3 and the cell distribution images G3a of the first to third stages. One cell distribution image can be designated by comparing with G3c. This allows the user to selectively designate a cell distribution image that is easy for the user to correct.

In step S6, the correction unit 215 corrects at least a part of the one cell distribution image designated in step S5 according to the user's operation. Thereby, the generation of the cell distribution image can be completed.

In step S7, the calculation unit 216 generates diagnostic assistance information based on the cell distribution image generated in step S6 and the bright spot distribution image acquired in step S2. Examples of the diagnosis support information include an analysis value related to the existence state of a specific biological substance in a specific region captured in a cell morphological image. The analyzed value may include various statistical values such as the number of fluorescent bright spots per unit area of a specific portion of a cell and the number of fluorescent bright spots per one cell region.

<(2-4) Summary>
As described above, according to the image processing device 2 according to the embodiment, a region in which a specific portion of a cell is estimated to be captured by image processing under different conditions for a cell morphological image capturing a cell morphology Two or more cell distribution images each showing the distribution of are generated and displayed. Then, of the two or more cell distribution images, the cell distribution image is designated by the user as a correction target. This allows the user to selectively designate a cell distribution image that can be easily modified. As a result, it is possible to efficiently obtain a cell distribution image in which the distribution of the region in which the specific portion of the cell is captured is shown in the cell morphology image in which the cell morphology is captured.

Then, for example, if different conditions related to image processing define up to what number of image processing of two or more types of image processing is performed, an intermediate step in the two or more types of image processing performed stepwise To a final stage, a plurality of cell distribution images obtained in a plurality of stages can be displayed. This makes it possible to easily obtain a plurality of cell distribution images in which the distributions of regions presumed to capture a specific portion of a cell in a cell morphology image capturing the morphology of cells are different from each other. .. As a result, the cell distribution image can be easily and efficiently acquired.

<(3) Modification>
The present invention is not limited to the above-described embodiment, and various modifications, improvements, etc. can be added without departing from the scope of the present invention.

<(3-1) First Modification>
In the above-described one embodiment, the user specifies one cell distribution image of the two or more cell distribution images, and at least a part of the one cell distribution image is corrected so that one cell distribution image The generation has been completed, but is not limited to this.

For example, the degree of strain of cells and the density of cell nuclei may vary depending on the type of organ, the type and malignancy of cancer, and the method of collecting tissue sections. Therefore, a case may occur in which one cell morphological image has different characteristics for each region.

Assuming such a case, for example, a cell distribution image that is different for each region is used as a base, and at least a part of these cell distribution images is corrected to complete the generation of one cell distribution image. A configuration that obtains may be adopted. If such a configuration is adopted, even if the cell morphology image has different characteristics for each region, the user can specify a cell distribution image suitable for making corrections for each region. As a result, the cell distribution image can be efficiently generated. Hereinafter, a specific example of the configuration and operation according to this modification will be described.

FIG. 39 is a block diagram illustrating a functional configuration according to the first modification realized by reading and executing the program P1A stored in the storage unit 25 in the control unit 21 of the image processing apparatus 2. is there. As shown in FIG. 39, the image processing device 2 has an acquisition unit 211, a generation unit 212, a display control unit 213, a designation unit 214A, a correction unit 215A, and a calculation unit as a functional configuration realized by the control unit 21. In addition to 216, a synthesis unit 217A is further provided.

The designation unit 214A determines the first image area and the first cell distribution for the first image area and the second image area of the entire area of the cell morphology image or the image area corresponding to the entire area according to the user's operation. A combination with an image (also called a first combination) and a combination of a second image area and a second cell distribution image (also called a second combination) are designated. The first cell distribution image and the second cell distribution image are images that form two or more cell distribution images displayed on the display unit 23.

Here, the first image area and the second image area may be arbitrarily designated, for example, according to a user's operation, or may be preset.

As an aspect in which the first image area and the second image area are arbitrarily designated according to the user's action, for example, an aspect in which a mark that arbitrarily divides the cell morphological image is drawn according to the user's action, etc. Is mentioned. The mark includes, for example, a line formed of at least one of a straight line and a curved line, a line defining the outer edge of the region, and a semitransparent figure. Further, for example, the cell morphology image may be divided into a large number of small regions in advance, and the first image region and the second image region, each of which is composed of a plurality of small regions, may be designated according to the user's operation.

As for the aspect in which the first combination and the second combination are designated, for example, the following aspects [C1] to [C4] can be considered.

[C1] A mode in which one cell distribution image is designated for each of the first and second image regions. In this aspect, for example, the first and second image areas may be designated according to a user's operation, or may be preset.

[C2] A mode in which, after the first combination of the first image area and the first cell distribution image is specified, the second combination of the second image area and the second cell distribution image is specified.

[C3] A mode in which the first image area and the second image area are specified after the first cell distribution image and the second cell distribution image are specified.

[C4] A first cell distribution image is designated, and a partial image region of the first cell distribution image is designated as a second image region, so that at least a part of the remaining region other than the second image region is designated. Is designated as the first image area. In this aspect, for example, all the remaining regions of the first cell distribution image other than the second image region may be the first region. Further, for example, when three or more image areas including the first and second image areas are set, two or more image areas other than the first image area out of the three or more image areas are set for the first cell distribution image. By designating the above image regions, the remaining regions other than the two or more image regions can be designated as the first image region. The designation of the second cell distribution image may be performed, for example, at any timing before or after the designation of the second image area.

It should be noted that, in the present embodiment, an aspect in which two image areas such as the first image area and the second image area are designated will be described, but the present invention is not limited to this, and for example, the first image area and the second image area. A configuration in which three or more image areas including the area are designated may be adopted. In this case, for example, a combination of the image area and the cell distribution image may be designated for each image area. Such a designation mode includes, for example, a mode in which a combination of an image region group including two or more image regions and a cell distribution image is designated.

The correction unit 215A corrects a portion of the first cell distribution image corresponding to the first image area and a portion of the second cell distribution image corresponding to the second image area according to the operation of the user. Thereby, the image can be efficiently corrected.

The synthesizing unit 217A, in accordance with the first combination and the second combination designated by the designating unit 214A, displays the first cell distribution image and the second cell distribution image in at least one state before and after the correction by the correcting unit 215A. To generate one cell distribution image.

Here, as the timing of the correction of the first and second cell distribution images by the correction unit 215A, for example, the following modes [D1] to [D4] can be considered.

[D1] After the first and second cell distribution images are combined into one cell distribution image (also referred to as a synthetic cell distribution image) according to the first and second combinations, the combination is performed according to the user's operation. A mode in which a correction is applied to the cell distribution image.

[D2] After the portion corresponding to the first image area of the first cell distribution image and the portion corresponding to the second image area of the second cell distribution image are corrected according to the user's action, the first and second portions are corrected. A mode in which the modified first and second cell distribution images are combined according to the combination.

[D3] After the portion corresponding to the first image region of the first cell distribution image is corrected according to the user's action, the corrected first cell distribution image and the uncorrected first cell distribution image are changed according to the first and second combinations. A mode in which the portion corresponding to the second image region of the second cell distribution image is corrected after the two-cell distribution image is combined.

[D4] After the portion corresponding to the second image area of the second cell distribution image is corrected according to the user's action, the first cell distribution image before correction and the first cell distribution image after correction are corrected according to the first and second combinations. A mode in which the portion corresponding to the first image area of the first cell distribution image is corrected after the two-cell distribution image is combined.

40 and 41 are flowcharts illustrating an operation flow according to the first modification of the image processing apparatus 2. Here, an example in which a cell morphology image and a fluorescence image as a microscope image are input to the image processing device 2 and two image regions (first and second image regions) are designated for the entire region of the cell morphology image It is shown. As shown in FIG. 40, in the image processing apparatus 2, for example, the processes of steps S1 to S4 and S5A to S7A are performed in time sequence. Regarding steps S1 to S4, the same processing as in the above-described embodiment is performed.

In step S5A, the designation unit 214A designates a first combination of the first image region and the first cell distribution image and a second combination of the second image region and the second cell distribution image according to the user's action. To be done. The first and second cell distribution images are the images displayed on the display unit 23 in step S4.

FIG. 41 is a flow chart exemplifying one operation flow of the processing in step S5A. FIG. 41 shows an operation flow according to the above-mentioned aspect [C1]. As shown in FIG. 41, in step S5A, for example, the processes of steps S51A and S52A are performed in time sequence. Here, the cell morphological image G4 (FIG. 42) is acquired in step S1, the cell morphological image G4 (FIG. 42) acquired in step S1 is displayed on the display unit 23 in step S4, and the step It is assumed that the first to third stage cell distribution images G4a to G4c (FIG. 43) generated in S3 are displayed as options. FIG. 43 is a diagram illustrating a screen (also referred to as a region image designation screen) Gs2 in which the cell morphology image G4 and the cell distribution images G4a to G4c in the first to third stages are displayed side by side.

In step S51A of FIG. 41, the first and second image areas are designated according to the operation of the user. For example, on the area image designation screen Gs2 (FIG. 43), as shown in FIG. 44, a curved broken line L1 as a mark for dividing the cell morphology image G4 is drawn according to the user's operation. Thereby, for example, the left side of the broken line L1 is designated as the first image area A1, and the right side of the broken line L1 is designated as the second image area A2. At this time, on the area image designation screen Gs2, for example, as shown in FIG. 45, the display element D1 is attached to the first image area A1 so that the first and second image areas can be identified by the user. , And the display element D2 is attached to the second image area A2. When the confirm button B1 is pressed after the broken line L1 is drawn, the designation of the first and second image areas A1 and A2 is confirmed, and the process proceeds to step S52A.

In step S52A, a cell distribution image to be combined with each of the first and second image regions is designated according to the user's operation. 45 and 46 show how a cell distribution image is designated on the area image designation screen Gs2. For example, in the area image designation screen Gs2, a desired one cell distribution of the cell distribution images G4a to G4c in the first to third stages in a state where the display element D1 related to the first image area A1 is highlighted by designation. An image may be designated for the first image area A1. At this time, for example, as shown in FIG. 46, in the area image specifying screen Gs2, the cell distribution image specified for the first image area A1 is surrounded by a thick frame and is referred to as “area 1”. A display element is added, and a cell distribution image designated for the second image area A2 is surrounded by a thick frame and a display element called "area 2" is added. .. Then, when the confirm button B1 is pressed, the designation of the cell distribution image combined with each of the first and second image regions is confirmed. When the confirm button B1 is pressed in the state of the area image designation screen Gs2 shown in FIG. 46, the first combination of the first image area A1 and the cell distribution image of the third stage, and the second image area A2. The designation of the second combination of the first stage cell distribution image and the second stage cell distribution image is confirmed. When the clear button B2 is pressed on the area image designation screen Gs2, the cell distribution image is designated again from the beginning.

Next, in step S6A of FIG. 40, the synthesizing unit 217A synthesizes the first cell distribution image and the second cell distribution image according to the first and second combinations designated by the designating unit 214A, thereby Two synthetic cell distribution images are generated. FIG. 47 is a diagram illustrating a synthetic cell distribution image G4d. Here, in the synthetic cell distribution image G4d, the third-stage cell distribution image G4c is applied to the first image area A1, and the first-stage cell distribution image G4a is applied to the second image area A2. It is generated by synthesizing the cell distribution image G4a of the first stage and the cell distribution image G4c of the third stage. Further, in step S6A, at least a part of one synthetic cell distribution image G4d generated here is corrected by the correction unit 215A according to the operation of the user. Thereby, the generation of the cell distribution image can be completed.

In step S7A, similar to step S7 according to the above-described embodiment, the calculation unit 216 determines a cell region based on the cell distribution image generated in step S6A and the bright spot distribution image acquired in step S2. An analysis value related to the existence state of the specific biological substance is calculated.

Further, FIG. 48 is a flow chart exemplifying one operation flow of the process in step SP5A which is performed instead of the process in step S5A of FIG. FIG. 48 shows one operation flow along the above-mentioned aspect [C4]. As shown in FIG. 48, in step SP5A, for example, the processing of steps SP51A to SP53A is performed in time sequence. Here, the cell morphological image G4 (FIG. 42) is acquired in step S1, the cell morphological image G4 (FIG. 42) acquired in step S1 is displayed on the display unit 23 in step S4, and the step It is assumed that the first to third stage cell distribution images G4a to G4c (FIG. 43) generated in S3 are displayed as options. FIG. 49 is a diagram illustrating a screen (also referred to as a region image designation screen) Gs3 in which the cell morphology image G4 and the cell distribution images G4a to G4c in the first to third stages are displayed side by side.

In step SP51A of FIG. 48, the first cell distribution image is designated as the first cell distribution image according to the user's operation. For example, FIGS. 49 and 50 show a state in which the first cell distribution image is designated on the area image designation screen Gs3. For example, in the area image designation screen Gs3 shown in FIG. 49, a desired one cell distribution image among the cell distribution images G4a to G4c in the first to third stages is designated as the first cell distribution image. At this time, for example, as shown in FIG. 50, the cell distribution image designated as the first cell distribution image is surrounded by a thick frame and a display element called "Area 1" is added to the cell distribution image. It Then, when the confirm button B1 is pressed, the designation of the cell distribution image to be combined with the first image region is confirmed. For example, when the confirm button B1 is pressed in the state of the region image designation screen Gs3 shown in FIG. 50, designation of the first combination of the first image region A1 and the third-stage cell distribution image G4c is confirmed. To be done. In this case, as shown in FIG. 51, the state transits to a state in which the image area can be designated. In the area image designation screen Gs3, for example, when the clear button B2 is pressed, the cell distribution image is designated again from the beginning.

Next, in step SP52A, a part of the image area of the first cell distribution image specified in step S51A is specified as the second image area A2 according to the user's operation. At this time, for example, the area excluding the second image area A2 in the first cell distribution image is determined as the first image area A1. Here, on the region image designation screen Gs3 (FIG. 51), as shown in FIG. 52, the second image region A2 is designated with respect to the third-stage cell distribution image G4c as the first cell distribution image. A broken line L2 as a mark is drawn according to the user's action. Thereby, for example, the area surrounded by the broken line L2 is designated as the second image area A2, and the remaining portion not surrounded by the broken line L2 is designated as the first image area A1. At this time, for example, a thick line as a mark designating the first image area A1 is drawn according to the user's operation on the cell distribution image G4c at the third stage as the first cell distribution image. Then, for example, when the confirm button B1 is pressed after drawing the thick line and the broken line L2, the designation of the first and second image areas A1 and A2 is confirmed, and the process proceeds to step SP53A.

Next, in step SP53A, a cell distribution image to be combined with the second image area A2 as a partial image area of the first cell distribution image is designated according to the user's operation. FIG. 53 shows how the cell distribution image for the second image region A2 is designated on the region image designation screen Gs3. For example, in the area image designation screen Gs3, in the state where the second image area A2 is designated on the first cell distribution image, a desired one cell distribution image among the cell distribution images G4a to G4c in the first to third stages Can be specified for the second image area A2. At this time, for example, as shown in FIG. 53, the area image designation screen Gs3 defines a contour of a broken line that defines the cell distribution image designated for the second image area A2 so as to surround the second image area A2. A line and a display element called "area 2" are attached. Then, when the confirm button B1 is pressed, the designation of the cell distribution image to be combined with each of the first and second image areas A1 and A2 is confirmed. When the confirm button B1 is pressed in the state of the area image designation screen Gs3 shown in FIG. 53, the first combination of the first image area A1 and the cell distribution image G4c of the third stage, and the second image area. The designation of the second combination of A2 and the first-stage cell distribution image G4a is confirmed.

<(3-2) Second Modification>
In the above-mentioned one embodiment and the above-mentioned 1st modification, two or more cell distribution is performed by performing up to each of two or more types of image processing of a plurality of types of image processing performed time-sequentially on a cell morphology image. An image was generated and the two or more cell distribution images were displayed as options. However, it is not limited to this.

For example, the appearance characteristics of the cell nucleus as a specific site may differ depending on the type and malignancy of cancer, the type of organ, and the like. Appearance features may include, for example, features related to size, density, color, shape, and the like. That is, the appearance feature of the cell nucleus as the specific portion in the cell morphological image may be different. Therefore, the condition of the image processing for accurately detecting the estimated region related to the specific portion from the cell morphology image may change.

Assuming such a case, for example, in the same image processing, at least two cell distribution images are detected by detecting an estimated region from a cell morphology image according to two or more reference values with different parameters that define processing conditions. May be generated. Then, two or more cell distribution images including the at least two cell distribution images may be displayed as options. With such a configuration, even when the cell distribution image is generated from the cell morphology images having various characteristics, the user can specify the cell distribution image suitable for correction. As a result, the cell distribution image can be efficiently generated. Hereinafter, a specific example of the configuration and operation according to the second modification will be described.

In this modification, as shown in FIG. 30, the control unit 21 of the image processing apparatus 2 reads and executes the program P1B stored in the storage unit 25 so that the functional configuration according to this modification is realized. Will be realized. In this modification, the image processing apparatus 2 includes an acquisition unit 211, a generation unit 212B, a display control unit 213B, a designation unit 214, a correction unit 215, and a calculation unit 216 as a functional configuration realized by the control unit 21. ..

The generation unit 212B detects an estimated region from the cell morphology image according to at least two reference values for each of the one or more types of parameters for at least one type of the one or more types of image processing. As a result, at least two cell distribution images of the two or more cell distribution images to be displayed as options are generated. With such a configuration, regardless of the characteristics of the cell morphology image, cell distribution images of various aspects can be generated from the cell morphology image. In the generation unit 212B, for example, for each of two or more reference values of each parameter, only the final cell distribution image as a processing result performed up to the final image processing of one or more types of image processing is displayed as an option. Can be generated for. Further, in the generation unit 212B, for example, for each of two or more reference values of each parameter, an intermediate cell distribution image as a processing result obtained by performing image processing in the middle of one or more types of image processing is displayed as an option. May be generated for.

Here, the one or more types of parameters include, for example, a parameter relating to one or more types of appearance features, a parameter relating to the degree of division, and a parameter relating to the degree of expansion/contraction processing of the estimated region in the cell morphology image. At least one of the parameters may be included. Thereby, when a cell distribution image is generated from a cell morphology image, it becomes possible to deal with various characteristic changes.

Furthermore, if the one or more appearance features include, for example, at least one of size, density, and shape, the appearance features are easily represented by a quantitative value. obtain. The size-related feature may include, for example, one or more types of features of area, perimeter length, major axis length, and minor axis length. The characteristics related to the depth may include, for example, one or more kinds of characteristics among hue, brightness, saturation, luminance, color difference, and luminance of each color. The shape-related features may include, for example, one or more features of circularity, flatness, and aspect ratio. This allows the appearance feature to be relatively easily and accurately expressed with a quantitative value.

Here, in the generation unit 212B, for example, if the values of the parameters related to the size, density, and shape of the estimated region are less than the reference values, it is determined that the estimated region is not detected in the proper state. Can be adopted.

FIG. 54 is a diagram illustrating a plurality of reference values for each type of parameter.

As shown in FIG. 54, as a parameter indicating the size of the estimated region, for example, a reference value for the area is selectively set between three values of “large value, medium value, and small value”. It is possible that Further, as a parameter indicating the degree of division, for example, a case where the reference value for the area is selectively set among three values of “large value, medium value, and small value” can be considered. Here, when the estimated region as the determination target is determined not to be in the proper state but to be detected in the undivided state, the area or the like is used as the parameter indicating the size of the estimated region. Is possible. If the area of the estimated region is expressed by the number of pixels, for example, a small area may be set to 1000 pixels, a medium area to 2000 pixels, and a large area to 3000 pixels.

Further, for example, in the case where the estimated region once detected by the image processing is uniformly expanded or contracted, the parameter indicating the degree of expansion or contraction is set as shown in FIG. It is conceivable that a plurality of stages of expansion rates or contraction rates are set. Here, as a parameter indicating the degree of expansion/contraction, for example, the expansion rate may be selectively set between three reference values. The three reference values related to the expansion rate can be set to values such as -20%, +20%, and +50%.

Further, as shown in FIG. 54, as the parameter indicating the shape of the estimated region, for example, the reference value for the circularity is selectively set between three values of “high value, medium value and low value”. It may be set to. Here, when the circularity Dc is calculated by the equation of Dc=4π×S/L ² using the area S and the peripheral length L so that the circularity of the perfect circle becomes 1, for example, , A low circularity value of 0.4, a medium circularity value of 0.6, a high circularity value of 0.8, and so on.

Further, as shown in FIG. 54, as the parameter indicating the darkness, for example, the reference value for the luminance of each color is selectively set between three values of “large value, medium value and small value”. It may be set. Here, when the brightness is represented by 256 gradations, for example, a large brightness value is 150, a medium brightness value is 100, and a small brightness value is 50. Can be set to a value.

The display control unit 213B causes the display unit 23 to display at least two cell distribution images obtained by one or more image processings including at least one type of image processing in which the reference values of the parameters are mutually different by the generation unit 212B. At least two final cell distributions obtained by subjecting at least two cell distribution images to, for example, one or more types of image processing including at least one type of image processing having mutually different parameters Images may be included. Further, at least two cell distribution images are subjected to, for example, image processing in the middle of one or more types of image processing including at least one type of image processing having mutually different parameters with respect to the cell morphology image. At least two intermediate cell distribution images obtained by the above may be included.

Furthermore, in the present modification, for example, in one or more image processes, the estimated region is detected from the cell morphology image according to a combination of two or more sets of reference values between two or more types of parameters that define the processing conditions. Thus, at least two cell distribution images may be generated. Then, two or more cell distribution images including the at least two cell distribution images may be displayed as options. According to such a configuration, even when the cell distribution image is generated from the cell morphological images having various characteristics, the cell distribution image suitable for the user to correct can be presented as an option. As a result, the cell distribution image can be efficiently generated.

When such a configuration is adopted, the generation unit 212B determines that at least two types of image processing of at least one type of image processing included in one or more types of image processing are performed between two or more types of parameters. Each estimated region is detected from the cell morphology image according to the combination of two or more reference values in. As a result, at least two cell distribution images of the two or more cell distribution images to be displayed as options are generated. With such a configuration, cell distribution images of various aspects can be generated from the cell morphology image regardless of the characteristics of the cell morphology image.

In the case where the above configuration is adopted, the generation unit 212B, for example, for each combination of two or more sets of reference values between two or more types of parameters, up to the final image processing of the two or more types of image processing. Only the final cell distribution image as a result of the applied processing may be generated for display as an option. Further, in the generation unit 212B, for example, for each combination of two or more sets of reference values between two or more types of parameters, an intermediate image processing result obtained by performing image processing in the middle of two or more types of image processing. A cell distribution image may be generated for display as the selection.

<(3-3) Third Modification>
In the second modified example, in the same image processing, at least two cell distribution images are generated by detecting the estimated region from the cell morphology image according to two or more reference values having different parameters that define the processing conditions. However, it is not limited to this.

For example, regarding the cell morphology image, even if the same organ or the same organ is captured, the appearance characteristics of the cell morphology image greatly differ depending on the presence or absence of a carcinoma, the type of the carcinoma, the method of collecting the tissue section, and the like. Therefore, for example, the cancer state is patterned, image processing suitable for detection of an estimated region corresponding to each pattern is prepared in advance, and image processing of a plurality of patterns is performed on the cell morphology image, respectively. A plurality of cell distribution images obtained by doing so may be displayed as options.

Specifically, for example, by performing image processing of a plurality of patterns to correspond to one or more types of appearance features in the cell morphology image, each of the estimated regions is detected from the cell morphology image, so that at least two cell distributions are obtained. An image may be generated. Then, two or more cell distribution images including the at least two cell distribution images may be displayed as options. According to such a configuration, even when the cell distribution image is generated from the cell morphological images having various characteristics, the cell distribution image suitable for the user to correct can be presented as an option. As a result, the cell distribution image can be efficiently generated. Hereinafter, a specific example of the configuration and operation according to the third modification will be described.

In the present modification, as shown in FIG. 30, the control unit 21 of the image processing apparatus 2 reads and executes the program P1C stored in the storage unit 25, whereby the functional configuration according to the present modification is achieved. Will be realized. In this modification, the image processing apparatus 2 includes an acquisition unit 211, a generation unit 212C, a display control unit 213C, a designation unit 214, a correction unit 215, and a calculation unit 216 as a functional configuration realized by the control unit 21. ..

The generation unit 212C detects each estimated region from the cell morphological image by performing image processing of a plurality of patterns to correspond to one or more types of features on the appearance of the estimated region in the cell morphological image, thereby generating two or more cells. At least two cell distribution images among the distribution images are generated. Then, at this time, the display control unit 213C causes the display unit 23 to display at least two cell distribution images obtained by the image generation processing of the patterns different from each other by the generation unit 212C.

With such a configuration, cell distribution images of various aspects can be generated from the cell morphology image regardless of the characteristics of the cell morphology image. In the generation unit 212C, for example, for the image processing of each pattern, only the final cell distribution image as a processing result performed up to the final image processing of one or more types of image processing can be generated to be displayed as options. .. Further, in the generation unit 212C, for example, for the image processing of each pattern, an intermediate cell distribution image as a processing result performed up to image processing in the middle of one or more types of image processing is generated to be displayed as an option. May be.

Here, the one or more appearance features may include, for example, at least one feature of size, density, and shape. Thereby, when a cell distribution image is generated from a cell morphology image, it becomes possible to deal with various variations in characteristics.

Further, in the present modification, for example, if one or more types of external features include two or more types of external features, image processing of a plurality of patterns may result in, for example, two external features. Image processing may be included depending on a combination of a plurality of types of features among types of features or more. As a result, a cell distribution image can be generated from the cell morphology image according to changes in two or more types of features.

Further, for example, two or more types of features on the appearance include features relating to the density and the degree of congestion, and each of the plurality of patterns of image processing corresponds to the feature relating to the concentration, and A mode that includes a division process for dealing with the feature related to the degree may be adopted. Here, the processing for dealing with the characteristic relating to the darkness includes, for example, at least one of a threshold value processing using a threshold value for the luminance of each color and a contour extracting processing according to the characteristic relating to the darkness. Processing may be included. Further, the division processing according to the feature related to the degree of congestion may include area division processing using the degree of division of the region according to the feature related to the degree of congestion. As a result, a cell distribution image can be generated from the cell morphology image according to the characteristics relating to the density and the degree of congestion as the two or more types of external appearance characteristics.

FIG. 55 is a diagram exemplifying a combination of conditions for a plurality of types of appearance features in a cell morphology image. In FIG. 55, as an example of a plurality of types of appearance characteristics, the state of staining of cell nuclei as a specific portion and the density are listed. As shown in FIG. 55, the state of dyeing a specific portion includes, for example, a dark state, a state in which the density is uneven (also referred to as an uneven state), a thin state, and a dark outline with a very thin interior. A state (also referred to as a hollow state) may be included. That is, the dyeing state of the specific region may be included in the characteristics related to the density of the estimated region in which the specific region is estimated to be captured. Further, the density of the specific portion may include, for example, low density, medium density and high density. In FIG. 55, a total of 12 appearance features are listed by combining three types of denseness states and four types of dyeing states. At this time, image processing of a pattern corresponding to a combination of twelve types of appearance features is performed on the cell morphology image.

56 and 57 are diagrams illustrating the relationship between the appearance feature and the content of the estimation region detection process.

As shown in FIG. 56, for example, division processing is set according to the degree of division according to the density of estimated regions. Specifically, for example, a low degree of division corresponding to low density, a degree of medium division corresponding to medium degree of congestion, and a high degree of division corresponding to high degree of congestion are set. Here, when the division process is, for example, a series of processes in which the processes B1 to B4 are sequentially performed, for example, in the edge extraction process, a threshold value that determines how thick an edge is extracted is By being changed, the degree of division in the division processing can be changed. At this time, for example, by performing the detection of the thin edge corresponding to the high density, the division processing can be performed assuming that the thin edge is the contour of the specific portion.

Further, as shown in FIG. 57, for example, the content of the estimation region detection processing according to the staining state is set. Specifically, for example, a threshold process (binarization process) using a normal threshold as a reference is set as the detection process of the estimated region corresponding to the deeply stained state. In addition, for example, as a detection process of an estimated region corresponding to a stain state of an uneven state, a smoothing process using a low-pass filter or the like is performed as a preprocess, and then a threshold process using a normal threshold as a reference (2 (Value processing) is set. In addition, for example, as the detection processing of the estimated region corresponding to the stained state of the light state, the inside of the edge detected by the threshold processing (binarization processing) and the edge extraction processing in which the threshold lower than the normal threshold is used as a reference. A process for filling in the voids as estimated regions is set. Further, for example, as the detection processing of the estimated area corresponding to the dyeing state of the hollow state, the processing of filling the area other than the background recognized by the downsampling processing and the detection of the background as the estimated area is set. The edge extraction processing can be realized by, for example, filter processing using a Sobel filter and/or Canny filter, processing using a differential image, and the like.

<(3-4) Fourth Modification>
In the above-mentioned one embodiment and the above-mentioned 1st modification, for example, regarding a cell morphology image, an intermediate cell distribution image obtained by performing image processing in the middle of two or more types of image processing and a final image processing may be performed. The final cell distribution image displayed was displayed as an option. However, it is not limited to this. For example, further, in at least one type of image processing included in the remaining image processing other than the image processing performed last among the two or more types of image processing in the detection processing of the estimated area, the area is added to the cell morphology image. Processing may be performed according to different conditions. At this time, the intermediate cell distribution image and the final cell distribution image can be generated by synthesizing the images obtained by the image processing according to the conditions different for each region. As a result, even when regions having mutually different characteristics coexist in the cell morphology image, a plurality of cell distribution images including a cell distribution image that can be easily corrected are easily generated.

In this modification, as shown in FIG. 30, the control unit 21 of the image processing apparatus 2 reads the program P1D stored in the storage unit 25 and executes the program P1D, whereby the functional configuration according to this modification is achieved. Will be realized. In this modification, the image processing apparatus 2 includes an acquisition unit 211, a generation unit 212D, a display control unit 213D, a designation unit 214, a correction unit 215, and a calculation unit 216 as a functional configuration realized by the control unit 21. ..

The generation unit 212D processes the cell morphology image in at least one type of image processing included in the remaining image processing other than the last image processing performed in the two or more types of image processing in the estimation region detection processing. Processing is performed according to different conditions for each area. As a result, at least two cell distribution images of the two or more cell distribution images are generated. Then, at this time, the display control unit 213D causes the display unit 23 to display at least two cell distribution images generated by the generation unit 212D.

Even with such a configuration, even if the aspect of the specific portion captured in the cell morphology image is not uniform, a plurality of cell distribution images including a cell distribution image that can be easily corrected are easily generated.

FIG. 58 is a flowchart showing an example of the operation flow of the process in step S3D according to the fourth modification, which is executed instead of step S3 in FIG. In this step S3D, for example, the processes of steps ST31 to ST48 are performed. Among these processes, in steps ST31, ST35 to ST45, the same processes as those in steps S31 and S32 to S42 described above are executed. Therefore, the processes of steps ST32 to ST34 and steps S46 to S48 will be described here.

In step ST32, the generation unit 212D performs segmentation processing on the cell morphology image subjected to the preprocessing in step ST31. The segmentation process is, for example, a process in which the cell morphology image is segmented into a plurality of image regions according to one or more types of appearance features. The one or more types of appearance characteristics may include, for example, characteristics relating to the size, the density, the color, the shape, etc. of the specific part. Specifically, as the classification processing, for example, the cell morphology image is converted into a binary image by a simple thresholding process, and the cell morphology image is converted according to the appearance feature of one or more estimated regions in the binary image. Can be adopted. Here, the cell morphology image is divided into N (N is a natural number of 2 or more) regions, and specific examples in which the numbers 1 to N are assigned to the N regions as identification information will be described.

FIG. 59 is a diagram illustrating a binary image G5a for a segmentation process, and FIG. 60 is a diagram illustrating a manner in which a cell morphology image is segmented. The binary image G5a shown in FIG. 59 can be obtained by performing threshold processing on the cell morphological image G4 shown in FIG. Then, for example, in the binary image G5a, the density of the estimated region is relatively high in the region on the right side of the region extending from the center to the left. At this time, for example, as shown in FIG. 60, the cell morphology image G4 is divided into a region Ar1 located to the left of the thick broken line and a region Ar2 located to the right of the thick broken line.

It should be noted that the level of density as a feature of appearance may be automatically determined by, for example, the size of a block of estimated regions that changes depending on the degree of connection of estimated regions, or the binary image G5a may be determined. It may be determined by the user who visually recognizes it. Then, for example, when the appearance feature is automatically determined, the cell morphology image is automatically segmented, and when the appearance feature is determined by the user, the cell morphology image is segmented. It can be done manually.

Next, in step ST33, a natural number n indicating which number of divided regions the image processing target is is set to 1.

Next, in step ST34, the n-th area is designated as a target for image processing. At this time, for example, a mode is conceivable in which the content of the image processing is set according to the external feature of the n-th area. For example, it is conceivable to set the content of the image processing such as increasing the degree of division in the division processing according to the density. Specifically, for example, in the case as shown in FIG. 60, the degree of division is set to be larger in the area Ar2 than in the area Ar1 in the cell morphology image G4.

Then, in steps ST35 to ST45, the same processes as in steps S32 to S42 described above are performed. Here, for example, in steps ST351 and ST352, image processing of content according to the external feature of the n-th region is executed.

In step ST46, it is determined whether or not the natural number n has reached the natural number N. Here, if the natural number n has not reached the natural number N, the process proceeds to step ST47, and if the natural number n has reached the natural number N, the process proceeds to step ST48.

In step ST47, the natural number n is incremented by 1, and the process proceeds to step ST34. Therefore, the processes of steps ST34 to ST47 are appropriately repeated until the natural number n reaches the natural number N.

In step ST48, a cell distribution image obtained by performing different image processing for each divided area is combined. Here, it is obtained by performing different image processing for each region in the three stages up to the first region detection process of step ST351, up to the second region detection process of step ST352, and up to the combining process of step ST45. The cell distribution image is synthesized. As a result, a three-step cell distribution image can be acquired. FIG. 61 is a diagram showing an example of the cell distribution image G5c that can be obtained by the synthesis in step ST48.

<(3-5) Other modifications>
For example, in the above-mentioned one embodiment and each modification, the microscope image transmitted from the microscope image acquisition device 1 to the image processing device 2 is a cell morphology image having cell morphology information and a fluorescence image having substance distribution information. However, it is not limited to this. For example, the microscope image transmitted from the microscope image acquisition apparatus 1 to the image processing apparatus 2 may be a fluorescence morphology image including both cell morphology information and substance distribution information. At this time, for example, from the fluorescence morphology image, a bright spot distribution image showing the distribution of fluorescent bright spots of a specific wavelength is obtained, and by removing the components of the bright spot distribution image from the fluorescence morphology image, a cell morphology image is obtained. obtain.

Further, in the above-described one embodiment and each of the modifications, the cell distribution image is manually corrected by the user, but the present invention is not limited to this. For example, the cell distribution image is automatically deleted according to the appearance standard, such as cell display elements that do not meet the appearance criteria such as size, color, darkness, and shape are deleted. May be modified.

Further, in the above-described one embodiment and each modification, the display unit 23 is a display device such as a CRT or LCD, but the present invention is not limited to this. For example, a tangible area such as a screen onto which an image is projected, or It may be a fluid such as a liquid onto which an image is projected or an intangible region such as a space. When such a configuration is adopted, for example, an image may be displayed on a tangible object or an intangible fluid by a technique such as a projector, or an intangible image may be displayed by a technique such as a three-dimensional (3D) hologram. The image may be displayed in a spatial area. Then, for example, a configuration may be adopted in which a projection device that projects an image on a tangible or intangible region is controlled by the display control unit 213 (213B to 213D).

It is needless to say that all or part of each of the above-described one embodiment and each of the modified examples can be appropriately combined in a consistent range.

DESCRIPTION OF SYMBOLS 1 Microscopic image acquisition device 2 Image processing device 21 Control part 22 Input part 23 Display part 25 Storage part 100 Pathological diagnosis support system 211 Acquisition part 212,212B,212C,212D Generation part 213,213B,213C,213D Display control part 214, 214A Designating part 215, 215A Modifying part 216 Calculating part 217A Synthesizing part A1 First image area A2 Second image area D1a, D1b, D1c Cell display element E101, Ex1, Ex2, Ex3 Cell display element G1 to G4, G102, Gb11 cell Morphological image G1a to G1c First to third stage cell distribution image G2a to G2c First to third stage cell distribution image G3a to G3c First to third stage cell distribution image G4a to G4c First to third stage cell distribution image G4d synthetic cell distribution image G5c cell distribution image Gd1 bright spot distribution image P1, P1A to P1D program

Claims (16)

By performing image processing on a cell morphology image that captures the morphology of cells, one or more cell display elements show the distribution of estimated regions in which it is estimated that specific parts of cells are captured in the cell morphology image. A generation unit that generates a cell distribution image
A display control unit that causes a display unit to display two or more cell distribution images generated by performing two or more image processes on the cell morphology image according to two or more mutually different conditions by the generation unit;
A designation unit for designating one cell distribution image of the two or more cell distribution images displayed on the display unit according to a user's action;
A correction unit that corrects at least a part of the one cell distribution image designated by the designation unit according to a user operation;
Equipped with
Before Symbol generation unit,
The cell morphology image to image processing several times sequentially stepwise facilities in each until a plurality of stages of image processing which is different in stepwise two or more image processing to be performed among the plurality of image processing Is generated on the cell morphology image to generate the two or more cell distribution images,
The display control unit causes the display unit to simultaneously display the two or more cell distribution images generated by subjecting the cell morphology image to each of the plurality of different stages of image processing by the generation unit. An image processing device characterized by the above. The image processing apparatus according to claim 1 , wherein
Image processing prior Symbol different stages,
In making the cell morphology image into a plurality of image processing time sequential stepwise, the image processing of a plurality of different stages in image processing on 2 or more that is applied stepwise in the image processing of the multiple An image processing device comprising: The image processing apparatus according to claim 1 or 2, wherein
The designated section is
The first combination of the first image area and the first cell distribution image of the two or more cell distribution images displayed on the display unit according to the user's action, and the first image area A second combination of a different second image area and a second cell distribution image different from the first cell distribution image of the two or more cell distribution images displayed on the display unit is designated,
The correction unit,
Depending on the user's action, a portion of the first cell distribution image corresponding to the first image area and a portion of the second cell distribution image corresponding to the second image area are corrected,
The image processing device,
According to the first combination and the second combination designated by the designation unit, the first cell distribution image and the second cell distribution image in at least one state before and after the correction by the correction unit are displayed. A synthesizing unit that generates one cell distribution image by synthesizing,
An image processing apparatus further comprising: An image processing apparatus according to any one of claims 1 to 3, wherein:
The generation unit,
In at least one type of image processing of the two or more image processing, by detecting the estimated region from the cell morphology image according to at least two reference values for each of the one or more types of parameters, the two or more An image processing apparatus, which generates at least two cell distribution images among the cell distribution images. The image processing apparatus according to claim 4, wherein
The one or more parameters are
It is characterized by including at least one kind of parameter related to one or more kinds of appearance features, a parameter related to degree of division, and a parameter related to degree of expansion/contraction processing, regarding the region in the cell morphology image. Image processing device. The image processing apparatus according to claim 5, wherein
One or more characteristics of the appearance,
An image processing apparatus including at least one type of characteristics of size, density, and shape. The image processing apparatus according to claim 6,
The characteristics related to the size are
Including one or more features of area, perimeter length, major axis length and minor axis length,
The characteristics related to the depth are
Including one or more features of hue, brightness, saturation, luminance, color difference and luminance of each color,
Features related to the shape,
An image processing apparatus comprising one or more features of circularity, flatness, and aspect ratio. An image processing apparatus according to any one of claims 4 to 7 , wherein:
Before Symbol generation unit,
In at least two types of image processing of the at least one type of image processing, by respectively detecting the estimated region from the cell morphology image according to a combination of two or more sets of reference values between two or more types of parameters, An image processing apparatus, which generates the at least two cell distribution images of the two or more cell distribution images. An image processing apparatus according to any one of claims 1 to 3, wherein:
The generation unit,
Each of the two or more cell distribution images is detected by detecting each of the estimated regions from the cell morphology image by image processing of a plurality of patterns to correspond to one or more types of features on the appearance of the region in the cell morphology image. An image processing apparatus, which generates a cell distribution image of at least two of the above. The image processing apparatus according to claim 9 ,
Image processing prior Symbol plurality of patterns,
The image processing apparatus characterized by comprising an image processing corresponding respectively to a combination of a plurality of sets of features between two or more features on the appearance. By performing image processing on a cell morphology image that captures the morphology of cells, one or more cell display elements show the distribution of estimated regions in which it is estimated that specific parts of cells are captured in the cell morphology image. A generation unit that generates a cell distribution image
A display control unit for displaying on the display unit two or more cell distribution images generated by subjecting the cell morphology image to one or more types of image processing according to two or more mutually different conditions by the generation unit;
A designation unit for designating one cell distribution image of the two or more cell distribution images displayed on the display unit according to a user's action;
A correction unit that corrects at least a part of the one cell distribution image designated by the designation unit according to a user operation;
Equipped with
The generation unit,
Each of the two or more cell distribution images is detected by detecting each of the estimated regions from the cell morphology image by image processing of a plurality of patterns to correspond to one or more types of features on the appearance of the region in the cell morphology image. Generate at least two of the cell distribution images ,
Image processing prior Symbol plurality of patterns,
The image processing apparatus characterized by comprising an image processing corresponding respectively to a combination of a plurality of sets of features between two or more features on the appearance. The image processing device according to claim 10 or 11, wherein
The above two or more features in appearance are
Including features related to density and density,
Each of the image processing of the plurality of patterns,
At least one of a threshold value process and a contour extraction process that uses a threshold value for the brightness of each color according to the feature related to the darkness, and a region using the degree of division of the region according to the feature related to the degree of congestion. An image processing apparatus comprising division processing. The image processing apparatus according to claim 2, wherein
The generation unit,
In at least one image processing included in the image processing of the remaining except the last image processing performed among the stepwise image processing on the 2 or more that is applied among the plurality of image processing, the cell morphology An image processing apparatus, which processes an image according to different conditions for each area. By performing image processing on a cell morphology image that captures the morphology of cells, one or more cell display elements show the distribution of estimated regions in which it is estimated that specific parts of cells are captured in the cell morphology image. A generation unit that generates a cell distribution image
A display control unit for displaying on the display unit two or more cell distribution images generated by subjecting the cell morphology image to one or more types of image processing according to two or more mutually different conditions by the generation unit;
A designation unit for designating one cell distribution image of the two or more cell distribution images displayed on the display unit according to a user's action;
A correction unit that corrects at least a part of the one cell distribution image designated by the designation unit according to a user operation;
Equipped with
The generation unit,
In at least one type of image processing of the two or more image processing, by detecting the estimated region from the cell morphology image according to at least two reference values for each of the one or more types of parameters, the two or more At least two cell distribution image of the cellular distribution image to generate a,
Before Symbol generation unit,
In at least two types of image processing of the at least one type of image processing, by respectively detecting the estimated region from the cell morphology image according to a combination of two or more sets of reference values between two or more types of parameters, An image processing apparatus, which generates the at least two cell distribution images of the two or more cell distribution images. (a) The generation unit performs two or more image processes on a cell morphology image that captures the morphology of a cell according to two or more mutually different conditions, thereby capturing a specific portion of the cell in the cell morphology image. Generating two or more cell distribution images in which the distribution of the estimated region estimated to be one is indicated by at least one cell display element,
(b) causing the display control unit to display the two or more cell distribution images generated in step (a) on the display unit;
(c) a step of designating one cell distribution image of the two or more cell distribution images displayed on the display unit by the designating unit according to a user's operation;
(d) a step of correcting at least a part of the one cell distribution image designated in step (c) according to a user's operation by a correction unit,
Equipped with
In the previous Symbol step (a),
The cell morphology image to image processing several times sequentially stepwise facilities in each until a plurality of stages of image processing which is different in stepwise two or more image processing to be performed among the plurality of image processing Is generated on the cell morphology image to generate the two or more cell distribution images,
In the step (b),
At the same time, the display control unit simultaneously displays the two or more cell distribution images generated by subjecting the cell morphology image to each of the different stages of image processing in step (a). An image processing method characterized by displaying. Image processing characterized by being executed by a control unit included in the information processing device, thereby causing the information processing device to function as the image processing device according to any one of claims 1 to 14. Program for.