JP5745290B2 - Image processing apparatus, cell classification apparatus, incubator, image processing method, cell classification method, image processing program, and cell classification program - Google Patents

Description

  The present invention relates to an image processing device, a cell classification device, an incubator, an image processing method, a cell classification method, an image processing program, and a cell classification program.

  Currently, techniques for staining cells are used in detection of spontaneous cancer in cancer research, detection of heterogeneous cells other than therapeutic cells in regenerative medicine, clinical research of mesenchymal stem cells (MSC), and the like. This is because cell types, states, and the like can be classified visually only by sensory means without staining the cells. A technique for staining cells expressing a transcription factor has been disclosed (see Patent Document 1). Further, a technique for extracting a specific cell group from all cells in a captured image using both images obtained by binarizing a bright field image of a cell and a fluorescence image of the cell is disclosed (Patent Document 2). reference).

JP 2007-195533 A JP-A-2-12060

  However, since the image after the binarization process includes a lot of noise, there is a problem that an image area of a desired cell cannot be extracted with high accuracy. An object of the present invention is to provide a technique capable of accurately extracting an image area of a desired cell from an image.

In order to solve the above problem, an image processing apparatus according to an aspect of the present invention includes a cell candidate object extraction unit that extracts a cell candidate object that is a candidate for a cell object from image data obtained by imaging a cell, and the cell candidate A feature quantity extraction unit that extracts a morphological feature quantity of an object, and an appearance frequency of the value based on the morphological feature quantity for each division when the value based on the morphological feature quantity is divided at predetermined intervals, or A cumulative distribution calculation unit that calculates, for each cell type, a cumulative distribution obtained by sequentially adding one of the values based on the morphological feature amount for each category, and correcting a threshold specific to the cell type. Correction information and correction information storage unit storing correction information specific to the cell type in association with the cell type, and the specific information corresponding to the cell type input from the outside Positive information is read from the inherent correction information storage unit, the morphological feature amount that minimizes the differential value of the cumulative distribution is calculated, and based on the calculated morphological feature amount and the read inherent correction information For each cell type, a unique threshold value calculation unit that calculates a threshold value specific to the cell type from the slope of the cumulative distribution according to the cell type, and based on the cumulative distribution, An extractor for extracting an object related to noise which is an object other than the cell object or the cell object, and extracting the cell object from the cell candidate objects based on the specific threshold ; It is characterized by providing.

In addition, the cell classification device according to one aspect of the present invention includes a cell captured in the image data based on the image processing device and the cell object extracted from the image processing device or the noise-related object. A cell classification unit for classification.

  An incubator according to one embodiment of the present invention contains a culture vessel for culturing cells or a group of cells, and is contained in the culture vessel in a constant temperature room capable of maintaining the inside in a predetermined environmental condition, and in the constant temperature room. An image pickup device for picking up an image of the cell or cell group, and the image processing device.

An image processing method according to an aspect of the present invention is an image processing method executed by an image processing apparatus, which extracts a cell candidate object that is a candidate for a cell object from image data obtained by imaging a cell. An extraction procedure, a feature quantity extraction procedure for extracting a morphological feature quantity of the cell candidate object, and a morphological feature quantity for each division when a value based on the morphological feature quantity is divided at predetermined intervals. A cumulative distribution calculation procedure for calculating, for each cell type, a cumulative distribution obtained by sequentially adding either the appearance frequency of the value based on the value or the value based on the morphological feature value for each category, and the cell type Correction information for correcting the threshold value specific to the cell type, and correction information specific to the cell type is input from the outside from a unique correction information storage unit stored in association with the cell type. Reading out the specific correction information corresponding to the type of the cell, calculating the morphological feature value that minimizes the differential value of the cumulative distribution, and calculating the calculated morphological feature value and the read out specific correction information For each cell type, a specific threshold calculation procedure for calculating a threshold specific to the cell type from the slope of the cumulative distribution according to the cell type, and the cell based on the cumulative distribution A procedure for extracting an object related to noise that is the cell object or an object other than the cell object from candidate objects, and extracts the cell object from the cell candidate objects based on the unique threshold value And an extraction procedure.

  The cell classification method according to one aspect of the present invention classifies cells photographed in image data based on the image processing method and the cell object extracted by the image processing method or the object related to noise. A cell sorting procedure.

Further, the present invention provides an image processing program that is an aspect of the present invention, wherein a computer as an image processing apparatus extracts a cell candidate object extraction that extracts a cell candidate object that is a candidate for a cell object from image data obtained by imaging a cell. A feature amount extracting step for extracting a morphological feature amount of the cell candidate object, and a value based on the morphological feature amount based on the morphological feature amount for each division when the value is divided at predetermined intervals. A cumulative distribution calculation step of calculating for each cell type a cumulative distribution obtained by sequentially adding one of the values based on the appearance frequency of the value or the value based on the morphological feature amount for each category; A unique correction information storage unit that corrects a unique threshold value and stores correction information unique to a cell type in association with the cell type Then, reading out the specific correction information according to the cell type input from the outside, calculating the morphological feature amount that minimizes the differential value of the cumulative distribution, and reading out the calculated morphological feature amount and the reading out A unique threshold value calculating step for calculating a threshold value specific to the cell type from the slope of the cumulative distribution corresponding to the cell type for each cell type based on the specific correction information; And extracting a cell object or an object related to noise that is an object other than the cell object from the cell candidate objects, based on the unique threshold value, from among the cell candidate objects. An image processing program for executing the extraction step of extracting the cell object .

  Further, the present invention provides a cell classification program according to one aspect of the present invention, the computer as a cell classification device, the image processing program, and the cell object extracted from the image processing program or the object related to noise. And a step of classifying the imaged cells based on the image data.

  According to the present invention, since an object related to noise can be removed from image data obtained by imaging a cell, an image area of a desired cell can be accurately extracted from an image.

It is an example of the functional block diagram of the classification model production | generation apparatus 1 by the 1st Embodiment of this invention. 6 is an explanatory diagram for explaining an operation of a noise estimation unit 24. FIG. It is a figure which shows the outline | summary of each feature value. 4 is an example of information stored in a cell object storage unit 72. 4 is an example of information stored in an extraction condition storage unit 74. It is an example of the information memorize | stored in the candidate classification | category model memory | storage part. 4 is an example of information stored in a classification model storage unit 80. It is a block diagram which shows the outline | summary of the incubator 311 connected with the classification model production | generation apparatus 1. FIG. It is a front view of the incubator 311. It is a top view of the incubator 311. 3 is a flowchart showing an example of the operation of the classification model generation device 1. 5 is a flowchart illustrating an example of the operation of a noise estimation unit 24. It is an example of the functional block diagram of the classification model production | generation apparatus 2 by the 2nd Embodiment of this invention. It is an example of the functional block diagram of the cell classification device 3 by the 3rd Embodiment of this invention. 4 is an example of information stored in a search object temporary storage unit 170. It is an example of the information memorize | stored in the cell catalog memory | storage part 190. FIG. 5 is a flowchart showing an example of the operation of the cell classification device 3. It is an example of the functional block diagram of the cell classification device 4. 7 is an example of information stored in a search object temporary storage unit 270. 5 is a flowchart showing an example of the operation of the cell classification device 4. It is a flowchart which shows an example of operation | movement of the incubator 311 which has a function of the cell classification apparatus 4. FIG. It is a figure for demonstrating the outline | summary of the process of each embodiment after 5th Embodiment in this invention. FIG. 23A is a diagram showing an example of the cumulative frequency distribution with respect to the area of the cell candidate object. FIG. 23B is a diagram showing an example of the distribution of differential values with respect to the area of the cumulative frequency of cell candidate objects. It is a figure for demonstrating the calculation method of a specific threshold value (custom e value) using a cumulative frequency differential function. It is the conceptual diagram which showed the relationship of the ratio ER of the error with respect to the intrinsic | native threshold value (e + (beta)) obtained when culture | cultivating for 8 hours in a certain three cell. It is the figure which displayed the ratio ER of the error in the image data by which A cell was imaged, using a threshold value and culture time as parameters. It is a figure for demonstrating the method of calculating a specific threshold value. It is a figure for demonstrating the method of calculating a common threshold value (temporary threshold value). It is a block block diagram of the image processing apparatus in the 5th Embodiment of this invention. It is the figure which showed an example of the table in which intrinsic | native correction | amendment information was linked | related and memorize | stored with the cell type and culture | cultivation time. It is a functional block block diagram of the cell object extraction part in the 5th Embodiment of this invention. 10 is a flowchart illustrating a flow of processing for extracting a cell object from image data captured in a certain culture time in a certain cell type in the image processing apparatus according to the fifth embodiment. The error ratio ER when the cell object is extracted with the area where the minimum value is simply taken as the threshold (e in FIG. 24) and when the cell object is extracted with the unique threshold (custom e value) as the threshold (e + α in FIG. 24) It is the table which compared. It is the figure which showed one example of the table T2 in which the ratio ER of the error of the predetermined cell type was stored in association with the threshold shift amount α ′ and the culture time t. 6 is a flowchart showing a flow of processing for calculating specific correction information α of a certain cell type in a predetermined culture time in the image processing apparatus. It is a block block diagram of the image processing apparatus in the 6th Embodiment of this invention. It is an example of the table T3 in which the culture time stored in the common correction information storage unit is associated with the common correction information γ. It is a block block diagram of the cell object extraction part in the 6th Embodiment of this invention. It is a block block diagram of the cell type estimation part in the 6th Embodiment of this invention. 4 is an example of information stored in a classification model storage unit 462. In the image processing apparatus in 6th Embodiment, it is the flowchart which showed the flow of the process which extracts a cell object from the image data imaged in a certain culture | cultivation time with a certain cell type. It is the figure which showed one example of the table of the ratio ER of the error memorize | stored in the recognition number memory | storage part. 14 is a flowchart showing a flow of processing for calculating specific correction information β and common correction information γ of a certain cell type in a predetermined culture time in the image processing apparatus according to the sixth embodiment. It is a block block diagram of the cell classification device in the 7th Embodiment of this invention. It is a block block diagram which shows the outline | summary of the incubator which combines the function of the cell classification device shown by 7th Embodiment. It is a flowchart which shows an example of operation | movement of the incubator which combines the function of the cell classification apparatus shown by 7th Embodiment.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an example of a functional block diagram of a classification model generation device 1 according to the first embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining the operation of the noise estimation unit 24. FIG. 3 is a diagram showing an outline of each feature value. FIG. 4 is an example of information stored in the cell object storage unit 72. FIG. 5 is an example of information stored in the extraction condition storage unit 74. FIG. 6 is an example of information stored in the candidate classification model storage unit 76. FIG. 7 is an example of information stored in the classification model storage unit 80. The classification model generation device 1 converts an unknown attribute captured image, which is a captured image obtained by capturing an unknown cell (hereinafter referred to as “unknown attribute captured image”), or a cell in the unknown attribute captured image into a morphological form of the cell. A classification model is generated for classification for each attribute based on the feature.

  As shown in FIG. 1, the classification model generation device 1 includes an input / output unit 10, a cell object recognition unit 20, a classification model generation unit 30, a cell object storage unit 72, an extraction condition storage unit 74, a candidate classification model storage unit 76, and A classification model storage unit 80 is provided. The cell object recognition unit 20 includes a binarization processing unit 22, a noise estimation unit (binarized data acquisition unit) 24, and a cell object feature amount calculation unit (noise removal unit) 26. The classification model generation unit 30 includes a characteristic object extraction unit 32, a candidate classification model formation unit 34, and a classification model determination unit 36.

  The input / output unit 10 obtains, from the outside, a captured image obtained by capturing a plurality of cells having a known identical attribute (hereinafter referred to as “known identical-attribute captured image”) together with attribute information related to the cell, and a cell object recognizing unit 20 to the binarization processing unit 22. The attribute information is information including the name of the cell, culture conditions, and the like. Accordingly, the cells imaged in the known identical attribute captured image are all cells having the same name, culture conditions, and the like. The outside from which the known identical attribute captured image is acquired is, for example, an incubator 311. The incubator 311 will be described later.

  The input / output unit 10 may acquire a known identical attribute captured image from the outside together with the attribute information and the imaging condition information, and output the acquired image to the binarization processing unit 22. The imaging condition information is information including the observation magnification, the aberration of the optical system of the microscope, the observation point in the container, and the like.

  The cell object recognizing unit 20 recognizes an imaging region of each cell in the known identical attribute captured image as a cell object. That is, the binarization processing unit 22 of the cell object recognition unit 20 binarizes the known identical attribute captured image, and the noise estimation unit 24 of the cell object recognition unit 20 performs data after binarization processing (hereinafter referred to as “binary processing”). An object related to noise (hereinafter referred to as a “noise object”) is estimated from all objects obtained from “binary data”), and the cell object feature amount calculation unit 26 determines an object other than the noise object (ie, an object other than the noise object). Cell object). Hereinafter, the binarization processing unit 22, the noise estimation unit 24, and the cell object feature amount calculation unit 26 will be described in order.

  The binarization processing unit 22 binarizes the known identical attribute captured image acquired from the input / output unit 10 with a predetermined threshold. The predetermined threshold may be in accordance with the type of cell. For example, the binarization processing unit 22 captures an image of an adhesive cell that is a cell that adheres to a culture container in a known identical-attribute captured image in which a cell region is captured in black compared to other regions. A binarization process is performed on a known identical attribute captured image using two threshold values. More specifically, the binarization processing unit 22 uses the pixel value 84 as one threshold value, a region less than the pixel value 84 is a cell region (for example, inverted white), and a region greater than the pixel value 84 is a cell. Regions other than (for example, black by inversion). Further, for example, the binarization processing unit 22 is a cell that floats in the culture solution without being fixed to the culture vessel in a known identical-attribute captured image in which the cell region is imaged blacker than other regions. When a floating cell is imaged, a binarized process is performed on a known identical attribute captured image using two threshold values. More specifically, the binarization processing unit 22 uses the pixel values 80 and 149 as the two threshold values, converts the region of the pixel value 80 or more and less than 149 to the cell region (for example, inverted white), and the pixel value of less than 80. Alternatively, the region of 149 or more is set as a region other than the cell (for example, inverted to black). Note that the binarization processing unit 22 uses the input / output unit 10 to send information indicating whether the cells imaged in the known identical attribute captured image are adhesive cells or floating cells to the incubator 311. And the threshold value may be determined.

  The binarization processing unit 22 that binarized the known identical attribute captured image outputs the binarized data to the noise estimation unit 24 together with the attribute information acquired from the input / output unit 10. The binarization processing unit 22 outputs the binarized data to the noise estimation unit 24 together with the attribute information and the imaging condition information when acquiring the imaging condition information from the input / output unit 10.

  The noise estimation unit 24 estimates a noise object from all objects obtained from the binarized data acquired from the binarization processing unit 22. Hereinafter, a noise object estimation method by the noise estimation unit 24 will be described. The noise estimation unit 24 includes functions of a cell candidate object extraction unit, a feature amount extraction unit, and a cumulative distribution calculation unit. The noise estimation unit 24 as a cell candidate object extraction unit extracts all objects (cell candidate objects) that are cell object candidates from the binarized data. The noise estimation unit 24 as a feature amount extraction unit first measures the object size of each object obtained from the binarized data acquired from the binarization processing unit 22.

  The noise estimation unit 24 that measures the object size of each object is obtained as a cumulative distribution calculation unit by sequentially adding the number of objects (cell candidate objects) having each area in the order of the object size (area). Cumulative degree distribution (cumulative frequency distribution) is calculated. For example, the noise estimation unit 24 generates a cumulative distribution (cumulative frequency distribution) as shown in FIG. In the cumulative frequency distribution (cumulative frequency distribution) shown in FIG. 2A, the X-axis direction is the object size in units of pixels, and the Y-axis direction is the cumulative level (cumulative frequencies) in units.

  The noise estimation unit 24 that calculates the cumulative degree distribution (cumulative frequency distribution) estimates a noise object based on the differential value of the calculated cumulative degree distribution (cumulative frequency distribution). For example, as shown in FIG. 2B, the noise estimation unit 24 calculates a differential value (slope) of the cumulative degree distribution (cumulative frequency distribution) shown in FIG. 2A, and based on the differential value, Estimate the noise object. Specifically, the noise estimation unit 24 searches for the minimum value of the differential value, and determines the minimum value or the vicinity of the minimum value as a threshold value. For example, the noise estimation unit 24 may determine the minimum value (value A in FIG. 2B) or a value near the minimum value and smaller than the minimum value (value (A−α in FIG. 2B)). )) In the vicinity of the minimum value and larger than the minimum value (value (A + α) in FIG. 2B) is determined as the threshold value. From experience, it is preferable that the threshold value is a value near the minimum value and smaller than the minimum value.

  The noise estimation unit 24 estimates an object whose object size is less than the threshold as a noise object. Alternatively, the noise estimation unit 24 assigns a label indicating that the object size is less than the threshold value to the object that is a noise object, and assigns a label indicating that the object size is a cell object to an object that the object size is equal to or greater than the threshold value.

  Note that the words “noise” and “cell” at the top of FIGS. 2A and 2B are more likely to be noise objects when the object size is smaller than the value (A−α). The object size equal to or larger than the value (A + α) indicates that the larger the object size, the higher the possibility of being a cell object.

  Note that the noise estimation unit 24 may calculate a cumulative distribution obtained by sequentially adding values of objects (cell candidate objects) each having an area in order of the object size (area). In this case, the noise estimation unit 24 determines an area when the slope of the cumulative distribution exceeds a predetermined value or the vicinity of the area as a threshold value.

  In the present embodiment, the noise estimation unit 24 calculates the cumulative distribution by sequentially adding the number of cell candidate objects having each area in order of the size of the area of the cell candidate object. Not limited to other morphological features of the cell candidate object (for example, the total cell length (major axis), the lateral width (minor axis) of the cell, the length of the outer circumference, etc. The cumulative distribution is calculated by sequentially adding the number of cell candidate objects having the morphological feature value or the morphological feature value itself in the order of the morphological feature value. May be. In that case, the noise estimation unit 24 as the feature amount extraction unit extracts the morphological feature amount of the cell candidate object.

  In addition, the noise estimation unit 24 calculates values (for example, major axis × ellipticity, minor axis × ellipticity, calculated from two or more morphological features extracted from the morphological features of the cell candidate object). (Major axis x minor axis), the number of cell candidate objects having the calculated value or the calculated value itself may be sequentially added in the order of the calculated value.

  Further, for example, two morphological features (for example, area and ellipticity) are extracted from the morphological features of the cell candidate object, and the two morphological features are extracted on a two-dimensional plane. Assume that each cell candidate object is plotted. In that case, the noise estimation unit 24 extracts the cluster that is the most densely packed first and the cluster that is the second most densely packed, and calculates the coordinates of the center of gravity of each cluster. Then, the noise estimation unit 24 calculates an angle θ formed by a straight line passing through the two centroids and the origin with one axis (for example, an axis whose area is a parameter is one axis). Then, the noise estimation unit 24 transforms the coordinates by an angle θ, and with respect to the value of one axis after rotation (for example, cos θ × area + sin θ × ellipticity), cell candidates having the values in the order of the values The cumulative distribution may be calculated by sequentially adding the number of objects or the value itself.

  That is, the noise estimation unit 24 extracts two or more morphological features from the morphological features of the cell candidate object, and performs rotational transformation of the angle θ on the coordinates with the extracted morphological feature amounts as axes. Regarding the calculated value (for example, cos θ × area + sin θ × ellipticity, where θ is an angle), the number of cell candidate objects having the calculated value or the calculated value in the order of the calculated value They may be added sequentially.

  Further, in the present embodiment, when calculating the cumulative distribution, the noise estimation unit 24 sequentially adds the number of cell candidate objects having each area in order of the size of the area of the cell candidate object. However, the present invention is not limited to this, and the noise estimation unit 24 may sequentially add the number of areas for each division when the areas are divided at predetermined intervals. Specifically, for example, the noise estimation unit 24 sets a predetermined value to 5 [pixels], the number of cell candidate objects having an area of 1 to 5 pixels is 5, and cell candidates having an area of 6 to 10 pixels. When there are 10 objects, the noise estimation unit 24 calculates the cumulative degree of the area of 6 to 10 pixels as 15. Then, by repeating this, the cumulative distribution is calculated.

  In summary, the noise estimation unit 24 calculates values (for example, area, major axis, major axis × ellipticity, cos θ × area + sin θ × ellipticity) based on the morphological feature amount of the cell candidate object at predetermined intervals. What is necessary is just to calculate the cumulative distribution obtained by sequentially adding one of either the appearance frequency of the value based on the morphological feature quantity or the value based on the morphological feature quantity for each division when the classification is performed.

  In addition, the noise estimation unit 24 in the present embodiment extracts a cell object or an object related to noise based on the differential value of the cumulative distribution, but the present invention is not limited thereto, and the slope of the cumulative distribution, the cumulative value of the cumulative distribution itself, A cell object or an object related to noise may be extracted based on a shape of a distribution curve, a change pattern of a cumulative value of the cumulative distribution, or an approximate expression of the cumulative distribution.

  Specifically, for example, the noise estimation unit 24 extracts a cell object or an object related to noise using, as a threshold, the area when the slope of the cumulative distribution curve approaches 0 in the cumulative distribution of area. May be. In addition, for example, the noise estimation unit 24 relates to the cell object or the noise using the area when the cumulative degree value of the cumulative degree distribution exceeds a predetermined value (for example, 100) as a threshold in the cumulative degree distribution of the area. Objects may be extracted.

  In addition, for example, the noise estimation unit 24 extracts a cell object or an object related to noise from the shape of the cumulative degree distribution curve, using the area where the curve is most parallel to the x-axis as a threshold value. May be. In addition, for example, the noise estimation unit 24 uses the area when the increase in the cumulative value for each predetermined area of the cumulative degree distribution is equal to or lower than a predetermined threshold in the cumulative distribution of the area as a threshold, An object related to noise may be extracted. Further, for example, the noise estimation unit 24 calculates a third-order approximate expression of the cumulative distribution in the cumulative distribution of area, calculates the x coordinate of the inflection point from the coefficient of the third-order approximate expression, and A cell object or an object related to noise may be extracted using the value of the x coordinate of the inflection point as a threshold value.

    The noise estimation unit 24 that estimated the noise object from all the objects obtained from the binarized data includes the binarized data and information indicating the estimation result by the noise estimation unit 24 (hereinafter referred to as “estimation result”) along with the attribute information. Information ”) is output to the cell object feature amount calculation unit 26. An example of the estimation result information is information indicating the area of the noise object, or information indicating the area of each object to which a label indicating that the object is a noise object or a cell object is given. Note that when the imaging condition information is acquired from the binarization processing unit 22, the noise estimation unit 24 sends the binarized data and the estimation result information to the cell object feature amount calculation unit 26 together with the attribute information and the imaging condition information. Output.

    The noise estimation unit 24 may search for the minimum value of the differential value using an extreme value search method such as the steepest descent method, but searches for the minimum value of the differential value in a range where the object size is equal to or larger than a predetermined value. May be. Specifically, it is preferable that the range of a predetermined value or more is 100 pixels or more based on the result of comparison and verification of several types of images.

    Moreover, the noise estimation part 24 may calculate a differential value using two points | pieces of a predetermined space | interval among each point of the object size and accumulation value which comprise accumulation degree. For example, the noise estimator 24 may set the differentiation of the cumulative degree distribution as points, that is, the point at which the slope is calculated as 10-point intervals.

    Based on the binarized data and the estimation result information acquired from the noise estimating unit 24, the cell object feature amount calculating unit 26 removes the noise object from all the objects obtained from the binarized data. In other words, the cell object feature amount calculation unit 26 recognizes individual cell objects other than the noise object from the binarized data acquired from the noise estimation unit 24.

  To summarize the above processing, the noise estimation unit 24 extracts all objects (cell candidate objects) that are candidates for cell objects from the image data obtained by imaging the cells, calculates the area of the cell candidate object, and calculates the area of the area. The cumulative frequency distribution obtained by sequentially adding the number of cell candidate objects having each area in order of size is calculated, and based on the slope of the cumulative frequency distribution, an object other than the cell object is selected from among the cell candidate objects. An object related to a certain noise is extracted. Then, the cell object feature amount calculation unit 26 extracts a cell object by removing an object related to noise from the cell candidate objects.

  In other words, the noise estimation unit 24 and the cell object feature amount calculation unit 26 function as an extraction unit, and the extraction unit is an object other than a cell object or a cell object from among cell candidate objects based on a cumulative distribution. An object related to noise is extracted.

  The cell object recognizing unit 20 first removes the object related to noise based on the cumulative distribution of the area of the cell candidate object, and then selects the object related to noise based on the cumulative distribution of the morphological feature amount other than the area. It may be removed. Further, the cell object recognition unit 20 may remove the object related to noise in the reverse order to the above, that is, the object related to noise is first removed based on the cumulative distribution of morphological features other than the area. Later, the object related to noise may be removed based on the cumulative distribution of the area of the cell candidate object.

  Thereby, the cell object recognition unit 20 removes the object related to noise twice using the cumulative distribution of the area and the cumulative distribution of the morphological feature other than the area, so that the noise removal accuracy can be improved.

In addition, the cell object recognition unit 20 first extracts a cell candidate object having a predetermined area or more, and removes an object related to noise based on the cumulative distribution of one morphological feature amount of the extracted cell candidate object. May be.
Thereby, the cell object recognition unit 20 first extracts the cell object roughly, and then removes the object related to the noise based on the cumulative distribution of one morphological feature amount, so that the noise removal accuracy can be improved. it can. In addition, since the cell object recognition unit 20 can calculate the cumulative distribution once, the calculation time can be shortened compared to calculating the cumulative distribution twice.

  The cell object recognizing unit 20 selects one different morphological feature amount from the morphological feature amount for each trial of noise removal, and uses the cumulative distribution of the selected morphological feature amount to select an object related to noise. It may be removed multiple times. Thereby, since the cell object recognition part 20 removes noise several times using the cumulative distribution of a different morphological feature-value, it can improve noise removal precision.

    Subsequently, the cell object feature amount calculation unit 26 calculates the morphological feature amount of each recognized cell object. As an example, the cell object feature amount calculation unit 26 calculates the following 15 types of morphological feature amounts for each cell object.

  Total area (see FIG. 3A) “Total area” is a value indicating the area of the cell of interest. For example, the cell object feature amount calculation unit 26 can calculate the value of “Total area” based on the number of pixels in the cell area of interest.

  -Hole area (refer to (b) of Drawing 3) "Hole area" is a value which shows the area of Hole in the cell of interest. Here, Hole refers to a portion where the brightness of the image in the cell is equal to or greater than a threshold value due to contrast (a portion that is close to white in phase difference observation). For example, lysosomes of intracellular organelles are detected as Hole. Further, depending on the image, a cell nucleus and other organelles can be detected as Hole. Note that the cell object feature amount calculation unit 26 may detect a group of pixels in which the luminance value in the cell is equal to or greater than the threshold as Hole, and calculate the value of “Hole area” based on the number of pixels of the Hole.

  • relative hole area (see (c) of FIG. 3) “relative hole area” is a value obtained by dividing the value of “Hole area” by the value of “Total area” (relative hole area = Hole area / Total) . The “relative hole area” is a parameter indicating the ratio of the organelle in the cell size, and the value varies depending on, for example, enlargement of the organelle or deterioration of the shape of the nucleus.

  Perimeter (see FIG. 3D) “Perimeter” is a value indicating the length of the outer periphery of the cell of interest. For example, the cell object feature amount calculation unit 26 can acquire the value of “Perimeter” by the contour tracking process when extracting cells.

  Width (see (e) in FIG. 3) “Width” is a value indicating the length of the cell of interest in the horizontal direction (X direction) of the image.

  Height (see (f) in FIG. 3) “Height” is a value indicating the length of the cell of interest in the image vertical direction (Y direction).

  Length (see (g) in FIG. 3) “Length” is a value indicating the maximum value (total length of the cell) of lines crossing the cell of interest.

  -Breadth (see (h) of FIG. 3) "Breadth" is a value indicating the maximum value (lateral width of the cell) among lines orthogonal to "Length".

  Fiber Length (see (i) in FIG. 3) “Fiber Length” is a value indicating the length when the target cell is assumed to be pseudo linear. The cell object feature amount calculation unit 26 calculates the value of “Fiber Length” by the following equation (1).

    However, in the formulas in this specification, “P” indicates the value of Perimeter. Similarly, “A” indicates the value of Total Area.

  Fiber Blade (see (j) in FIG. 3) “Fiber Breath” is a value indicating a width (a length in a direction perpendicular to Fiber Length) when a target cell is assumed to be pseudo linear. The cell object feature amount calculation unit 26 calculates the value of “Fiber Breath” by the following equation (2).

  Shape Factor (see (k) in FIG. 3) “Shape Factor” is a value indicating the roundness (roundness of the cell) of the cell of interest. The cell object feature amount calculation unit 26 calculates the value of “Shape Factor” by the following equation (3).

  Elliptical form factor (see (l) in FIG. 3) “Elliptical form factor” is a value obtained by dividing the value of “Length” by the value of “Breadth” (Elliptical form Factor = Length / Breadth) This is a parameter indicating the degree of cell slenderness (ellipticity).

Inner radius (see (m) in FIG. 3) “Inner radius” is a value indicating the radius of the inscribed circle of the cell of interest.

Outer radius (see (n) in FIG. 3) “Outer radius” is a value indicating the radius of the circumscribed circle of the cell of interest.

Mean radius (see (o) of FIG. 3) “Mean radius” is a value indicating the average distance between all points constituting the outline of the cell of interest and its centroid point.

Equivalent radius (see (p) of FIG. 3) “Equivalent radius” is a value indicating the radius of a circle of the same area as the cell of interest. The parameter “Equivalent radius” indicates the size when the cell of interest is virtually approximated to a circle.

  Here, the cell object feature value calculation unit 26 may calculate each of the morphological feature values by adding an error to the number of pixels corresponding to the cell. At this time, the cell object feature amount calculation unit 26 may calculate the morphological feature amount in consideration of imaging conditions of the microscope image (observation magnification, aberration of the optical system of the microscope, and the like). When calculating “Inner radius”, “Outer radius”, “Mean radius”, and “Equivalent radius”, the cell object feature amount calculation unit 26 calculates the center of gravity of each cell based on a known center of gravity calculation method. What is necessary is just to obtain | require and calculate each parameter on the basis of this barycentric point.

  The cell object feature amount calculation unit 26 that has calculated the morphological feature amount of each cell object stores the morphological feature amount of each cell object in the cell object storage unit 72 in association with the attribute information. When the cell object feature value calculation unit 26 acquires the imaging condition information from the noise estimation unit 24, the cell object feature value calculation unit 26 associates the morphological feature value of each cell object with the cell object storage in association with the attribute information and the imaging condition information. Store in the unit 72.

The cell object storage unit 72 stores information related to cell objects recognized from a plurality of known identical attribute captured images obtained by imaging cells having various attributes. Specifically, as shown in FIG. 4, the cell object storage unit 72 associates attribute information, imaging condition information, and morphological characteristics of each cell object with an image ID that identifies known identical attribute imaging. Remember the amount. For example, in the example illustrated in FIG. 4, the cell object storage unit 72 associates the image ID “P001” with the attribute information including the name “A”, the culture condition “OO”, and the activity “XX”, and the observation magnification. The imaging condition information including “X times” and the morphological feature amount of N ₁ cell objects are stored.

  The extraction condition storage unit 74 includes a characteristic object (hereinafter referred to as “candidate classification model”) that is used by the characteristic object extraction unit 32 to generate a model that is a classification model candidate (hereinafter referred to as “candidate classification model”) from the cell object storage unit 72. The extraction condition when extracting “characteristic object”) is stored. For example, as illustrated in FIG. 5, the extraction condition storage unit 74 stores extraction condition information related to a plurality of morphological feature amounts of individual cell objects. In the example shown in FIG. 5, the extraction condition storage unit 74 has the extraction condition 1 “extract the upper 25% from the average value”, the extraction condition 2 “extract the lower 25% from the average value”,. As extraction condition 5, “extract 25% higher than average value” is stored, and as extraction condition 6, “extract 25% lower than average value” is stored.

  In the example shown in FIG. 5, the extraction condition storage unit 74 stores extraction conditions relating to three types of morphological feature amounts of “area”, “length”, and “width”, but the cell object storage unit It is preferable to store extraction conditions relating to all the morphological feature amounts corresponding to the morphological feature amounts stored in 72.

  Further, the extraction condition shown in FIG. 5 uses the average value as a reference value, but may use a value other than the average value such as a maximum value or a median value as a reference. Further, in the extraction condition shown in FIG. 5, the value defining the predetermined range is 25%, but may be a value other than 25%. Further, for example, as shown in the extraction condition 1 and the extraction condition 2, for example, the extraction condition shown in FIG. 5 has the same value for defining the predetermined range for the same morphological feature amount. The same morphological feature amount within a predetermined range, such as “extract the upper 20% of the area from the average value” as the extraction condition 1 and “extract the lower 30% of the area from the average value” as the extraction condition 2 Different values may be specified. Further, for example, in the extraction condition shown in FIG. 5, for example, the value that defines the predetermined range is the same value for any morphological feature amount. For example, as the extraction condition 1, “the area is higher than the average value” For example, “extracting 20%” and “extracting 30% higher than the average value” as the extraction condition 5 may be different values for different morphological feature values that define the predetermined range.

  The classification model generation unit 30 generates a classification model based on the morphological feature amount of the cell object and the attribute of the cell recognized as the cell object. That is, the characteristic object extraction unit 32 of the classification model generation unit 30 extracts a cell object having a predetermined morphological feature amount from the cell objects stored in the cell object storage unit 72 as a characteristic object, and classifies it. The candidate classification model formation unit 34 of the model generation unit 30 forms a plurality of candidate classification models from the morphological feature amounts of the characteristic objects and the attributes of the cells recognized as the characteristic objects, and the classification model determination unit 36 includes the plurality of classification model determination units 36. A classification model is determined from the candidate classification models. Hereinafter, the characteristic object extraction unit 32, the candidate classification model formation unit 34, and the classification model determination unit 36 will be described in order.

  The characteristic object extraction unit 32 extracts a cell object having a morphological characteristic value within a predetermined range from the reference value as a characteristic object. Specifically, the characteristic object extraction unit 32 extracts a characteristic object from a plurality of cell objects stored in the cell object storage unit 72 according to the extraction conditions stored in the extraction condition storage unit 74. . More specifically, the characteristic object extraction unit 32 sequentially extracts the extraction conditions stored in the extraction condition storage unit 74 and extracts from the plurality of cell objects stored in the cell object storage unit 72. A characteristic object is extracted for each condition.

There are various modes in which the characteristic object extracting unit 32 extracts the characteristic objects. For example, only N ₁ cell objects, N ₂ cell objects, and N ₃ cell objects shown in FIG. 4 are stored in the cell object storage unit 72, and the extraction condition information shown in FIG. Several extraction modes will be described by taking as an example the case where is stored.

(First extraction mode)
The characteristic object extraction unit 32 first calculates an average area value, an average length value, and an average width value of (N ₁ + N ₂ + N ₃ ) cell objects. Next, the characteristic object extraction unit 32 satisfies either one of the extraction conditions 1 and 2 which are extraction conditions related to the area among (N ₁ + N ₂ + N ₃ ) cell objects, and is long. A cell object that satisfies any one of the extraction conditions 3 and 4 that are extraction conditions related to the depth and that satisfies the extraction condition 5 or 6 that is the extraction conditions related to the width is defined as a characteristic object. Extract. For example, the characteristic object extraction unit 32 has an area within a range of an average value to a value 25% larger than the average value (extraction condition 1) among (N ₁ + N ₂ + N ₃ ) cell objects, In addition, the length is within the range of the average value to 25% smaller than the average value (extraction condition 4), and the width is within the range of the average value to 25% larger than the average value (extraction condition 5). A certain cell object is extracted as a characteristic object. The AND condition (that is, “and”) may be OR (that is, “or”). In the first extraction mode, the number of patterns for extracting a characteristic object from a cell object is a {2 morphological feature quantity} power pattern. For example, in the case of the above example, since the morphological feature amount is 3, there are 8 patterns.

(Second extraction mode)
The characteristic object extraction unit 32 calculates an average value of the area of (N ₁ + N ₂ + N ₃ ) cell objects according to the extraction condition 1, and from among the (N ₁ + N ₂ + N ₃ ) cell objects, A cell object whose area is in the range of an average value to a value 25% larger than the average value is extracted as a characteristic object. The same applies to extraction conditions 2 to 6. In the second extraction mode, the number of patterns for extracting characteristic objects from cell objects is 2 × {number of morphological feature values} patterns. For example, in the case of the above example, since the morphological feature amount is 3, there are 6 patterns.

(Third extraction mode)
The characteristic object extraction unit 32 calculates an average value of the areas of the N ₁ , N ₂ , and N ₃ cell objects according to the extraction condition 1 set in advance among the extraction conditions 1 to 6, and N ₁ Among the cell objects, a cell object whose area is in the range of an average value to a value 25% larger than the average value is extracted as a characteristic object, and the area is an average value to an average value among the N ₂ cell objects. A cell object within a range of 25% larger value is extracted as a characteristic object, and a cell object whose area is within a range of an average value to a value 25% larger than the average value is selected from N ₃ cell objects. You may extract as a characteristic object. The same applies when the extraction conditions 2 to 6 are used. In the third extraction mode, the number of patterns for extracting characteristic objects from cell objects is a 2 × {number of images} power pattern. For example, in the case of the above example, since the number of images is 3, there are 8 patterns.

In the description of the extraction modes 1, 2, and 3, the characteristic object extraction unit 32 uses, as the characteristic object, a cell object whose area is in the range of the average value to a value 25% larger than the average value according to the extraction condition 1. Although extracted, the characteristic object extraction unit 32 calculates an average value of the areas of (N ₁ + N ₂ + N ₃ ) cell objects in accordance with the extraction condition 1 and calculates (N ₁ + N ₂ + N ₃ ) Cell objects having an area of the average value or more of the cell objects may be set as a population, and cell objects within 25% from the smaller area of the population may be extracted as characteristic objects. In addition, the characteristic object extraction unit 32 may extract cell objects included in the population from the smallest area ((N ₁ + N ₂ + N ₃ ) / 4) as characteristic objects. The same applies to the extraction conditions 2 to 6.

  When the characteristic object extraction unit 32 extracts the characteristic object from the cell objects stored in the cell object storage unit 72, the characteristic object extraction unit 32 uses the attribute information and the morphological characteristic amount of each characteristic object as a candidate classification model formation unit. 34. For example, each time the characteristic object is extracted by changing the extraction condition, the characteristic object extraction unit 32 extracts the characteristic object attribute information and the morphological characteristic amount extracted according to each extraction condition as the candidate classification model formation unit 34. Output to.

In the present embodiment, the characteristic object extraction unit 32 extracts characteristic objects from a plurality of cell objects according to the extraction conditions stored in the extraction condition storage unit 74, but it is essential to follow the extraction conditions. is not. For example, the characteristic object extraction unit 32 may function a process of extracting a characteristic object from all cell objects according to the purpose.

    The candidate classification model forming unit 34 includes the morphological feature amount of the characteristic object and the attribute of the cell recognized as the characteristic object, that is, the attribute information and the morphological feature of the characteristic object acquired from the characteristic object extraction unit 32. A candidate classification model is formed (generated) using the quantity.

Hereinafter, a method for forming a candidate classification model by the candidate classification model forming unit 34 will be described.
The candidate classification model forming unit 34 sets classification criteria (type and threshold value of the morphological feature amount) related to the morphological feature amount obtained from the characteristic object acquired from the characteristic object extraction unit 32, and forms a candidate classification model. To do. In other words, the candidate classification model forming unit 34 sets (selects) the classification standard (the type and the threshold value of the morphological feature value) constituting the candidate classification model from the morphological feature values obtained from the characteristic object. Thus, a candidate classification model is formed.

    The first and Mth candidate classification models shown in FIG. 6 are examples of candidate classification models formed by the candidate classification model forming unit 34 as described above. The first candidate classification model is classified by the candidate classification model forming unit 34 into the first stage classification standard (morphological feature amount “length” and threshold “12”), and the second stage classification standard (morphological feature). The candidate classification model and the Mth candidate classification model in which the amount “roundness” and the threshold value “0.7” are set are classified by the candidate classification model forming unit 34 into the first-stage classification criterion (morphological feature quantity “ Roundness ”and threshold value“ 12 ”), and the second-stage classification criteria (morphological feature amounts“ width ”and“ 1.2 ”) are set. The candidate classification model forming unit 34 stores each formed candidate classification model in the candidate classification model storage unit 76. Note that the candidate classification model in which the classification criteria are set in multiple stages as shown in FIG. 6 classifies the unknown attribute captured image or the cells in the unknown attribute captured image in stages. A method for forming the classification model will be described later.

    The candidate classification model forming unit 34 sets a classification destination ID (also referred to as a classification index) indicating a cell classification destination in each branch of the candidate classification model. Each classification destination ID corresponds to cell attribute information. Therefore, the attribute information of the cell in the unknown attribute captured image or the unknown attribute captured image can be obtained from the unknown attribute captured image or the cell classification destination ID in the unknown attribute captured image. In FIG. 6, in the first candidate classification model and the Mth candidate classification model, the classification destination ID “N0001” has the name “A”, the culture condition “OO”, and the activity “XX” in FIG. The classification destination ID “N0002” corresponds to the attribute information including the name “B”, the culture condition “ΔΔ”, and the activity “□□”, and the classification destination ID “N0003” has the name “B”. Corresponds to attribute information including “C”, culture condition “「 ”, and activity“ □□ ”. In order to associate the classification destination ID with the attribute, the cell object feature amount calculation unit 26 associates the morphological feature amount of each cell object with the cell object in association with new attribute information (attribute information and imaging condition information). When storing in the storage unit 72, the classification destination ID may be numbered, and the classification destination ID numbered in association with the attribute information (attribute information and imaging condition information) may be stored.

    Hereinafter, a method of forming a candidate classification model that classifies the attributes of cell objects in stages will be described. The candidate classification model forming unit 34 forms a candidate classification model for variable selection based on the decision tree method. For example, ten characteristic objects having an attribute A, ten characteristic objects having an attribute B, and ten characteristic objects having an attribute C are given, and the morphological feature amount (dimension) of each characteristic object is given. Are 15 types (feature amount 1, feature amount 2,..., Feature amount 15), the candidate classification model forming unit 34 first increases the value of feature amount 1 from the minimum to the maximum, The value of the feature amount 1 when the three types of characteristic objects A, B, and C are best classified is specified.

    For example, the candidate classification model forming unit 34 has, for a certain value of the feature amount 1, a certain object (branch destination) has 9 of 10 characteristic objects A and 1 of 10 characteristic objects B. When there is one object C out of ten, the classification accuracy of the node is calculated as 90%. Similarly, the candidate classification model forming unit 34 calculates the classification accuracy of all nodes for the value of the feature quantity 1. Since the classification accuracy of each node becomes higher when each characteristic object is assigned to each node, the candidate classification model forming unit 34 calculates the average classification accuracy of all nodes in the value of the feature amount 1. . Similarly, the candidate classification model forming unit 34 increases the value of the feature quantity 1 and calculates the average classification accuracy of all nodes at each feature quantity 1 value. Then, the candidate classification model forming unit 34 uses the feature quantity 1 having the highest average classification accuracy of all nodes as the value of the feature quantity 1 when the three types of characteristic objects A, B, and C are best classified. Identify the value.

    Similarly, the candidate classification model formation unit 34 specifies the value of feature quantity 2, the value of feature quantity 3,..., The value of feature quantity 15 having the highest average classification accuracy of all nodes. Then, the candidate classification model forming unit 34 calculates the average of all nodes as a first-stage rule in the candidate classification model from among the identified feature value 1, feature value 2 value,..., Feature value 15 value. Select the one with the highest classification accuracy. Similarly, the candidate classification model forming unit 34 selects a rule after the second stage.

    The classification model determination unit 36 inputs the cell object stored in the cell object storage unit 72 into each candidate classification model, and evaluates the classification accuracy of each candidate classification model. The classification model determination unit 36 may input a known identical attribute captured image into each candidate classification model and evaluate the classification accuracy of each candidate classification model. Further, the classification model determination unit 36 determines the candidate classification model with the highest accuracy as a classification model to be used for the subsequent classification, and stores it in the classification model storage unit 80. For example, as shown in FIG. 7, the classification model determination unit 36 may store the calculated classification accuracy evaluation value in the classification model storage unit 80 together with the determined classification model.

    For example, the arrows in FIG. 1 indicate the relationship between the characteristic object extraction by the characteristic object extraction unit 32, the formation of the candidate classification model by the candidate classification model formation unit 34, and the determination of the classification model by the classification model determination unit 36. Although omitted, it may be realized as follows.

    After storing the morphological feature amount in the cell object storage unit 72, the cell object feature amount calculation unit 26 notifies the candidate classification model forming unit 34 that the morphological feature amount has been stored. When the candidate classification model forming unit 34 obtains a notification that the morphological feature value is stored in the cell object storage unit 72 from the cell object feature value calculation unit 26, the candidate classification model formation unit 34 requests the characteristic object extraction unit 32 to perform extraction.

When the candidate classification model forming unit 34 requests extraction, the characteristic object extracting unit 32 extracts the characteristic object according to the unextracted pattern when the characteristic object has not yet been extracted according to the entire extracted pattern. The result is output to the candidate classification model forming unit 34.
On the other hand, if the characteristic object extraction unit 32 has already extracted characteristic objects according to the entire extraction pattern, the characteristic object extraction unit 32 notifies the candidate classification model formation unit 34 that extraction has been completed.

    When the candidate classification model formation unit 34 acquires a characteristic object from the characteristic object extraction unit 32, the candidate classification model formation unit 34 forms a candidate classification model, stores the candidate classification model in the candidate classification model storage unit 76, and again stores the characteristic object extraction unit 32 in the characteristic object extraction unit 32. Request extraction. On the other hand, when the candidate classification model forming unit 34 obtains notification from the characteristic object extraction unit 32 that extraction has been completed, the candidate classification model formation unit 34 requests the classification model determination unit 36 to determine a classification model. When the classification model determination unit 36 is requested by the candidate classification model formation unit 34 to determine a classification model, the classification model determination unit 36 evaluates the candidate classification model and determines a classification model.

    Hereinafter, the incubator 311 will be described. FIG. 8 shows an example of an incubator 311 connected to the classification model generation device 1. 9 and 10 are front views of the incubator 311. FIG.

    The incubator 311 has an upper casing 312 and a lower casing 313. In the assembled state of the incubator 311, the upper casing 312 is placed on the lower casing 313. Note that the internal space between the upper casing 312 and the lower casing 313 is vertically divided by a base plate 314.

    First, the outline of the configuration of the upper casing 312 will be described. A constant temperature room 315 for culturing cells is formed inside the upper casing 312. The temperature-controlled room 315 includes a temperature adjusting device 315a and a humidity adjusting device 315b, and the temperature-controlled room 315 is maintained in an environment suitable for cell culture (for example, an atmosphere having a temperature of 37 ° C. and a humidity of 90%) ( Note that illustration of the temperature adjusting device 315a and the humidity adjusting device 315b in FIGS. 9 and 10 is omitted).

    A large door 316, a middle door 317, and a small door 318 are arranged on the front surface of the temperature-controlled room 315. The large door 316 covers the front surfaces of the upper casing 312 and the lower casing 313. The middle door 317 covers the front surface of the upper casing 312 and isolates the environment between the temperature-controlled room 315 and the outside when the large door 316 is opened. The small door 318 is a door for carrying in / out a culture vessel 319 for culturing cells, and is attached to the middle door 317. By carrying in / out the culture vessel 319 from this small door 318, it becomes possible to suppress the environmental change of the temperature-controlled room 315. The large door 316, the middle door 317, and the small door 318 are kept airtight by the packings P1, P2, and P3, respectively.

    In the temperature-controlled room 315, a stocker 321, an observation unit 322, a container transport device 323, and a transport base 324 are arranged. Here, the transfer table 324 is disposed in front of the small door 318, and carries the culture container 319 out of the small door 318.

    The stocker 321 is disposed on the left side of the temperature-controlled room 315 when viewed from the front surface of the upper casing 312 (the lower side in FIG. 10). The stocker 321 has a plurality of shelves, and each shelf of the stocker 321 can store a plurality of culture vessels 319. Each culture container 319 contains cells to be cultured together with a medium.

    The observation unit 322 is disposed on the right side of the temperature-controlled room 315 when viewed from the front surface of the upper casing 312. This observation unit 322 can execute time-lapse observation of cells in the culture vessel 319.

    Here, the observation unit 322 is disposed by being fitted into the opening of the base plate 314 of the upper casing 312. The observation unit 322 includes a sample stage 331, a stand arm 332 projecting above the sample stage 331, and a main body portion 333 incorporating a microscopic optical system for phase difference observation and an imaging device 334. The sample stage 331 and the stand arm 332 are disposed in the temperature-controlled room 315, while the main body portion 333 is accommodated in the lower casing 313.

    The sample stage 331 is made of a translucent material, and a culture vessel 319 can be placed thereon. The sample stage 331 is configured to be movable in the horizontal direction, and the position of the culture vessel 319 placed on the upper surface can be adjusted. The stand arm 332 has a built-in LED light source 335. The imaging device 334 can acquire a microscopic image of the cells by imaging the cells in the culture vessel 319 that are transmitted and illuminated from above the sample stage 331 by the stand arm 332 via the microscopic optical system.

    The container transport device 323 is disposed at the center of the temperature-controlled room 315 when viewed from the front surface of the upper casing 312. The container transport device 323 delivers the culture container 319 between the stocker 321, the sample table 331 of the observation unit 322, and the transport table 324.

As shown in FIG. 10, the container transport device 323 includes a vertical robot 334 having an articulated arm, a rotary stage 335, a ministage 336, and an arm portion 337.
The rotary stage 335 is attached to the tip of the vertical robot 334 via a rotary shaft 335a so that it can rotate 180 ° in the horizontal direction. Therefore, the rotation stage 335 can make the arm unit 337 face the stocker 321, the sample table 331, and the transport table 324, respectively.

    The mini stage 336 is attached to the rotary stage 335 so as to be slidable in the horizontal direction. An arm portion 337 for holding the culture vessel 319 is attached to the mini stage 336.

    Next, an outline of the configuration of the lower casing 313 will be described. In the lower casing 313, the main body portion 333 of the observation unit 322 and the control device 341 of the incubator 311 are housed.

    The control device 341 is connected to the temperature adjustment device 315a, the humidity adjustment device 315b, the observation unit 322, and the container transport device 323, respectively. The control device 341 comprehensively controls each part of the incubator 311 according to a predetermined program.

    As an example, the control device 341 includes a CPU 342 and a storage unit 343, and controls the temperature adjustment device 315a and the humidity adjustment device 315b, respectively, to maintain the inside of the temperature-controlled room 315 at predetermined environmental conditions. The control device 341 controls the observation unit 322 and the container transport device 323 based on a predetermined observation schedule, and automatically executes the observation sequence of the culture vessel 319.

    The storage unit 343 is configured with a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 343 stores management data regarding each culture vessel 319 stored in the stocker 321 and data of a microscope image captured by the imaging device. Further, the storage unit 343 stores a program executed by the CPU 342.

    The management data includes (a) index data indicating individual culture containers 319, (b) the storage position of the culture container 319 in the stocker 321 and (c) the type and shape of the culture container 319 (well plate, (Dish, flask, etc.), (d) type of cells cultured in culture vessel 319, (e) observation schedule of culture vessel 319, (f) imaging conditions during time-lapse observation (magnification of objective lens, observation in vessel) Point, etc.). In addition, regarding the culture container 319 that can simultaneously culture cells in a plurality of small containers such as a well plate, management data is generated for each small container.

    Hereinafter, the operation of the classification model generation device 1 will be described using a flowchart. FIG. 11 is a flowchart illustrating an example of the operation of the classification model generation device 1. FIG. 12 is a flowchart illustrating an example of the operation of the noise estimation unit 24. In FIG. 11, the input / output unit 10 acquires attribute information, imaging condition information, and a known identical attribute captured image from the outside (step S112). The input / output unit 10 outputs the attribute information, the imaging condition information, and the known identical attribute captured image acquired from the outside to the binarization processing unit 22.

    The binarization processing unit 22 binarizes the known identical attribute captured image acquired from the input / output unit 10 with a predetermined threshold (step S114). The binarization processing unit 22 outputs the binarized data to the noise estimation unit 24 together with the attribute information and the imaging condition information.

    The noise estimation unit 24 acquires the binarized data from the binarization processing unit 22, and estimates the noise object from all the objects obtained from the acquired binarized data (step S120). Specifically, the noise estimation unit 24 executes each operation shown in FIG.

    In FIG. 12, the noise estimation unit 24 acquires binarized data from the binarization processing unit 22 (step S121). Next, the noise estimation unit 24 measures the object size of each object obtained from the binarized data (step S122). Next, the noise estimation unit 24 generates a cumulative distribution representing the frequency of the cumulative value of the object size up to one object size with respect to the total object size of each object (step S123).

Next, the noise estimation unit 24 searches for the minimum value of the slope (differential value) of each two points at predetermined intervals within a range equal to or greater than the predetermined value in the cumulative distribution (step S124). Next, the noise estimating unit 24 determines a value near the minimum value and smaller than the minimum value as a threshold (step S125). Next, the noise estimation unit 24 estimates an object whose object size is less than the threshold as a noise object (step S126).
The noise estimation unit 24 outputs the binarized data and the estimation result information to the cell object feature amount calculation unit 26 together with the attribute information and the imaging condition information. Then, the flowchart shown in FIG. 12 ends.

    Returning to FIG. 11, the cell object feature amount calculation unit 26 that has acquired the binarized data and the estimation result information together with the attribute information and the imaging condition information from the noise estimation unit 24, the binarized data and the estimation acquired from the noise estimation unit 24. Based on the result information, the noise object is removed from all the objects obtained from the binarized data (step S130). In other words, the cell object feature quantity calculation unit 26 recognizes individual cell objects other than the noise object from the binarized data acquired from the noise estimation unit 24 (step S130).

    Next, the cell object feature amount calculation unit 26 calculates the morphological feature amount of each recognized cell object (step S132). The cell object feature amount calculation unit 26 that calculated the morphological feature amount of each cell object associates the calculated morphological feature amount of each cell object with the attribute information and the imaging condition information in the cell object storage unit 72. Store (step S134).

    The input / output unit 10 determines whether or not acquisition of a known identical-attribute captured image or the like has been completed (step S136). For example, when the input / output unit 10 obtains a notification indicating that output of a known identical attribute captured image or the like has been completed from the incubator 311 or a notification indicating that a classification model should be generated, the input / output unit 10 knows the known identical attribute captured image. And so on. If the input / output unit 10 determines that acquisition of a known identical-attribute captured image or the like has not been completed (step S136: No), the process returns to step S112.

    On the other hand, when the input / output unit 10 determines that the acquisition of the known identical attribute captured image or the like has been completed (step S136: Yes), the cell object feature amount calculation unit 26 indicates that the morphological feature amount is stored as a candidate classification model. Notify the forming unit 34. The candidate classification model forming unit 34 that has received the notification that the morphological feature quantity has been stored requests the characteristic object extraction unit 32 to perform extraction. The characteristic object extraction unit 32 requested to extract extracts a characteristic object from the cell objects stored in the cell object storage unit 72 according to one extraction pattern (step S142). The characteristic object extraction unit 32 that has extracted the characteristic object outputs the attribute information and the morphological feature amount of each characteristic object to the candidate classification model formation unit 34.

    The candidate classification model forming unit 34 that acquired the attribute information and the morphological feature amount of each characteristic object forms (generates) a candidate classification model based on the attribute information and the morphological feature amount of each characteristic object. The candidate classification model is stored in the candidate classification model storage unit 76 (step S144). Next, the candidate classification model forming unit 34 determines whether all candidate classification models have been formed (step S146). For example, when the candidate classification model forming unit 34 requests extraction from the characteristic object extraction unit 32 and obtains notification from the characteristic object extraction unit 32 that extraction has been completed, all candidate classification models are formed. Judge. If the candidate classification model forming unit 34 determines that not all candidate classification models have been formed (step S146: No), the process returns to step S142. In step S142 to be executed again, the characteristic object extraction unit 32 extracts characteristic objects according to the unextracted pattern.

    If the candidate classification model forming unit 34 determines that all candidate classification models have been formed (step S146: Yes), it requests the classification model determination unit 36 to determine a classification model. The classification model determination unit 36 requested to determine the classification model determines a classification model from among a plurality of candidate classification models, and stores the classification model in the classification model storage unit 80 (step S150). Then, the flowchart shown in FIG. 11 ends.

  Note that the estimation of the noise object by the noise estimation unit 24 and the deletion of the noise object by the cell object feature amount calculation unit 26 are not limited to the processing for determining the classification model for classifying the cell object described in this embodiment. Absent. The estimation of the noise object by the noise estimation unit 24 and the deletion of the noise object by the cell object feature amount calculation unit 26 do not depend on the purpose of image processing, and any image can be used when image processing is performed on image data obtained by capturing cells. You may use also when processing.

(Second Embodiment)
The second embodiment of the present invention will be described below with reference to the drawings. FIG. 13 is an example of a functional block diagram of the classification model generation device 2 according to the second embodiment of the present invention. As shown in FIG. 13, the classification model generation device 2 includes an input / output unit 10, a cell object recognition unit 20, a classification model generation unit 40, a cell object storage unit 72, and a classification model storage unit 80.
The classification model generation unit 40 includes a representative feature quantity calculation unit 42 and a classification model formation unit 46.
The input / output unit 10, the cell object recognition unit 20, the cell object storage unit 72, and the classification model storage unit 80 are the same as the input / output unit 10 and the cell object recognition of the classification model generation device 1 of the first embodiment shown in FIG. The same as the unit 20, the cell object storage unit 72, and the classification model storage unit 80.

    The representative feature amount calculation unit 42 calculates a representative feature amount that represents the morphological feature amounts of the plurality of cell objects recognized from the known identical attribute captured images. For example, the representative feature amount calculation unit 42 calculates the average value of each morphological feature amount of a plurality of cell objects extracted from the known identical attribute captured image, and calculates the average value of each morphological feature amount as the representative feature amount. And When the representative feature amount calculating unit 42 calculates the representative feature amount from each known identical attribute captured image, the representative feature amount calculated from each known identical attribute captured image, and the attribute information of each known identical attribute captured image Is output to the classification model forming unit 46. In other words, the representative feature quantity calculation unit 42 generates a virtual cell object (hereinafter referred to as “virtual object”) having the representative feature quantity as its morphological feature quantity for each known identical attribute captured image. Then, the attribute information and the morphological feature amount (that is, the representative feature amount) of each virtual object are output to the classification model forming unit 46.

    The classification model formation unit 46 generates a classification model using the attribute information and morphological feature amounts of a plurality of virtual objects for each known identical attribute captured image acquired from the representative feature amount calculation unit 42. The method for generating the classification model by the classification model forming unit 46 is the same as the method for generating the candidate classification model by the candidate classification model forming unit 34 of the classification model generating apparatus 1 according to the first embodiment. The classification model forming unit 46 stores the generated classification model in the classification model storage unit 80.

    Note that the representative feature quantity calculation unit 42 uses the average value of each morphological feature quantity as a representative feature quantity, but other than the average value may be used as the representative feature quantity. For example, the representative feature quantity calculation unit 42 may use the median value and standard deviation of each morphological feature quantity as the representative feature quantity.

(Third embodiment)
The third embodiment of the present invention will be described below with reference to the drawings. FIG. 14 is an example of a functional block diagram of the cell classification device 3 according to the third embodiment of the present invention. FIG. 15 is an example of information stored in the search object temporary storage unit 170. FIG. 16 is an example of information stored in the cell catalog storage unit 190.

    As shown in FIG. 14, the cell classification device 3 includes an input / output unit 110, a cell object recognition unit 120, a characteristic object extraction / representative feature amount calculation unit 152, a cell classification unit 160, a search object one-time billion unit 170, a classification A model storage unit 180 and a cell catalog storage unit 190 are provided. The cell object recognition unit 120 includes a binarization processing unit 122, a noise estimation unit 124, and a cell object feature amount calculation unit 126.

    The input / output unit 110 acquires an unknown attribute captured image from the outside. Specifically, an unknown attribute captured image obtained by imaging a plurality of cells with unknown different attributes (hereinafter referred to as “unknown different attribute captured image”), or an unknown attribute obtained by imaging a plurality of unknown identical attributes. A captured image (hereinafter referred to as an “unknown identical attribute captured image”) is acquired from the outside. The input / output unit 110 outputs the unknown attribute captured image acquired from the outside to the binarization processing unit 122 of the cell object recognition unit 120. The outside from which the unknown attribute captured image is acquired is, for example, an external medium (for example, a USB memory) that can be connected to the cell classification device 3, other devices connected via a network such as a LAN, the Internet, Alternatively, the incubator 311 is used.

    Further, the input / output unit 110 may acquire image type information from the outside together with the unknown attribute captured image and output the image type information to the binarization processing unit 122. The image type information is information for identifying whether the unknown attribute captured image is an unknown heterogeneous attribute captured image or an unknown identical attribute captured image.

    The cell object recognition unit 120 recognizes individual cell objects (hereinafter referred to as “search objects”) in the unknown attribute captured image. Specifically, the binarization processing unit 122 of the cell object recognition unit 120 is the same as the binarization processing unit 22 of the classification model generation device 1 according to the first embodiment described above. The attribute captured image is binarized by a predetermined threshold value. The binarization processing unit 122 that has performed binarization processing outputs the binarized data to the noise estimation unit 124. Note that the binarization processing unit 122 outputs the binarized data and the image type information to the noise estimation unit 124 when the image type information is acquired from the input / output unit 110.

    Similar to the noise estimation unit 24 of the classification model generation device 1 according to the first embodiment, the noise estimation unit 124 is a noise object among all objects obtained from the binarized data acquired from the binarization processing unit 122. Is estimated. The noise estimation unit 124 that has estimated the noise object outputs the estimation result information to the cell object feature amount calculation unit 126 together with the binarized data. When the noise estimation unit 124 acquires the image type information from the binarization processing unit 122, the noise estimation unit 124 outputs the estimation result information to the cell object feature amount calculation unit 126 together with the binarized data and the image type information. To do.

    Based on the binarized data and the estimation result information acquired from the noise estimating unit 124, the cell object feature amount calculating unit 126 removes the noise object from all the objects obtained from the binarized data. In other words, the cell object feature amount calculation unit 126 recognizes individual cell objects other than the noise object from the binarized data acquired from the noise estimation unit 124.

    Subsequently, the cell object feature quantity calculation unit 126 calculates the morphological feature quantity of each of the recognized search objects in the same manner as the cell object feature quantity calculation unit 26 of the classification model generation device 1 according to the first embodiment. . The cell object feature amount calculation unit 126 that has calculated the morphological feature amount of the search object stores the morphological feature amount of each search object in the search object temporary storage unit 170. Note that the cell object feature amount calculation unit 126 stores the image type information and the morphological feature amount of each search object in the search object temporary storage unit 170 when the image type information is acquired from the noise estimation unit 124. To do.

The search object temporary storage unit 170 stores information related to the search object. Specifically, as shown in FIGS. 15A and 15B, the search object temporary storage unit 170 stores morphological feature amounts of individual search objects in association with image type information. In the example shown in FIG. 15A, the search object temporary storage unit 170 associates N ₄ with image type information (for example, value 0) indicating that the search object extraction source is an unknown heterogeneous attribute captured image. The morphological feature amount of each search object is stored. In the example shown in FIG. 15B, the search object temporary storage unit 170 is associated with image type information (for example, value 1) indicating that the search object extraction source is an unknown same-attribute captured image. N stores the morphological features of the _five search objects.

    Note that the initial value of the item “image type information” in the search object temporary storage unit 170 is image type information indicating that the extraction source of the search object is an unknown different-attribute captured image. Therefore, when the cell object feature quantity calculation unit 126 does not acquire the image type information from the noise estimation unit 124, the item “image type information” indicates that the search object extraction source is an unknown heterogeneous attribute captured image. Image type information is stored.

The extraction condition storage unit 174 stores extraction conditions for extracting a search object from among a plurality of search objects stored in the search object temporary storage unit 170.
Hereinafter, the search object extracted from the search object temporary storage unit 170 is referred to as a characteristic search object. The extraction condition storage unit 174 stores extraction conditions having the same contents as the contents stored in the extraction condition storage unit 74 of the classification model generation device 1 according to the first embodiment.

    The characteristic object extraction / representative feature quantity calculation unit 152 includes the characteristic object extraction unit 32 of the classification model generation device 1 according to the first embodiment and the classification model generation device 2 according to the second embodiment. The representative feature amount calculation unit 42 is provided. In other words, the characteristic object extraction / representative feature amount calculation unit 152, when an unknown heterogeneous attribute captured image is input to the cell classification device 3, is characterized by the classification model generation device 1 according to the first embodiment. When processing similar to that performed by the object extraction unit 32 and an unknown identical attribute captured image is input to the cell classification device 3, the representative feature amount calculation unit 42 of the classification model generation device 2 according to the second embodiment described above. The same processing is performed.

    Specifically, the characteristic object extraction / representative feature amount calculation unit 152 stores image type information indicating that the search object extraction source is an unknown heterogeneous attribute captured image in the search object temporary storage unit 170. If there is, the characteristic search object is extracted from the search objects stored in the search object temporary storage unit 170 in accordance with the extraction conditions stored in the extraction condition storage unit 174. Note that the feature object extraction / representative feature amount calculation unit 152 may acquire information indicating which feature search object is extracted according to which extraction condition from the outside via the input / output unit 110. When the characteristic object extraction / representative feature quantity calculation unit 152 extracts characteristic search objects from the search objects stored in the search object temporary storage unit 170, the attribute information and morphological features of each characteristic search object are extracted. The amount is output to the cell classification unit 160.

    On the other hand, the characteristic object extraction / representative feature amount calculation unit 152 stores, in the search object temporary storage unit 170, image type information indicating that the search object extraction source is an unknown identical attribute captured image. Calculates a representative feature amount representative of the morphological feature amount of the search object from the search object stored in the search object temporary storage unit 170. When the representative object extraction / representative feature amount calculation unit 152 calculates the representative feature amount from the search object stored in the search object temporary storage unit 170, the representative feature amount calculated from the unknown identical attribute captured image, And the attribute information of an unknown identical attribute picked-up image is output to the cell classification | category part 160. FIG. In other words, the characteristic object extraction / representative feature quantity calculation unit 152 generates a virtual object (hereinafter, referred to as “virtual search object”) having the representative feature quantity as its own morphological feature quantity. The attribute information and the morphological feature amount (that is, the representative feature amount) of the target search object are output to the cell classification unit 160.

    The classification model storage unit 180 stores an unknown attribute captured image or a classification model for classifying the cells in the unknown attribute captured image for each attribute based on the morphological characteristics of the cells. For example, as shown in FIG. 7, the classification model storage unit 180 may store a classification model that classifies the attributes of cell objects in stages according to a predetermined classification standard.

    The cell catalog storage unit 190 stores a cell catalog as shown in FIG. In the cell catalog shown in FIG. 16, for example, classification destination ID “N0001”, attribute information (name “A”, culture condition “OO”, etc.), and morphological feature (area “27”, length “10.5”). Etc.) in association with each other. Note that the cell catalog may further store an image of the cell object (for example, binarized data representing the cell object).

    The cell classification unit 160 inputs the morphological feature amount of the cell object recognized from the unknown attribute captured image by the cell object recognition unit 120 or the feature amount calculated from the morphological feature amount into the classification model, thereby making the unknown The cells in the attribute captured image or the unknown attribute captured image are classified.

    For example, when an unknown heterogeneous attribute captured image is input to the cell classification device 3, the cell classifying unit 160 extracts features extracted from cell objects (that is, search objects) recognized from the unknown heterogeneous attribute captured image. Based on the morphological feature amount of the target object (that is, the characteristic search object), the cells in the unknown heterogeneous attribute captured image are classified. Specifically, when the cell classification unit 160 acquires attribute information and morphological feature amounts of a plurality of characteristic search objects from the characteristic object extraction / representative feature amount calculation unit 152, the plurality of characteristic search objects The attribute information and the morphological feature amount are input into the classification model, and the individual characteristic search objects are classified.

    For example, when an unknown identical attribute captured image is input to the cell classification device 3, the cell classification unit 160 uses the morphological feature amount of the cell object (that is, the search object) recognized from the unknown identical attribute captured image. Based on the calculated feature quantity (ie, representative feature quantity), the unknown identical-attribute captured images are classified. Specifically, when the cell classification unit 160 acquires the attribute information and the morphological feature amount (that is, the representative feature amount) of the virtual search object from the characteristic object extraction / representative feature amount calculation unit 152, The attribute information and the morphological feature amount of the virtual search object are input to the classification model, and the unknown identical attribute captured image is classified. That is, the cell classification unit 160 sets the classification destination of the virtual search object as the classification destination of the unknown identical-attribute captured image.

    When classifying the unknown attribute captured image or the cells in the unknown attribute captured image, the cell classification unit 160 outputs the classification result to the input / output unit 110. Specifically, when the individual characteristic search objects obtained from the unknown heterogeneous attribute captured images are classified, the cell classification unit 160 is set to at least the branch destination of the individual characteristic search objects. The classification destination ID is output to the input / output unit 110. For example, in addition to the classification destination IDs of the individual characteristic search objects, the cell classification unit 160 can select either attribute information associated with each classification destination ID or the morphological feature amount of each characteristic search object. Either or both may be output to the input / output unit 110.

    Note that when the individual characteristic search objects are classified, the cell classification unit 160 associates the individual characteristic search objects with the classification destination ID indicating the classification destination of the individual characteristic search objects, and the morphological feature amounts of the individual characteristic search objects. May be added to the cell catalog. In addition, when the characteristic search object is classified, the cell classification unit 160 is a classification result in addition to information obtained from the classification model as a classification result (for example, classification destination ID, attribute information for each classification destination ID). The attribute information and morphological feature amount corresponding to each classification destination ID may be acquired from the cell catalog storage unit 190 and output to the input / output unit 110.

    When classifying the unknown identical attribute captured image based on the virtual search object obtained from the unknown identical attribute captured image, the cell classification unit 160 is set to at least the branch to which the unknown identical attribute captured image is classified. The classification destination ID is output to the input / output unit 110. For example, the cell classification unit 160 adds the attribute information associated with the classification destination ID or the morphological feature amount of the virtual search object (that is, the representative feature) in addition to the classification destination ID of the unknown identical attribute captured image. Any one or both of (quantity) may be output to the input / output unit 110.

    In addition, when the cell classification unit 160 classifies the unknown identical attribute captured image, the cell classification unit 160 associates the unknown attribute captured image with the classification destination ID indicating the classification destination of the unknown attribute captured image, and the morphological feature of the virtual search object used for the classification The quantity may be added to the cell catalog. In addition, when classifying an unknown identical-attribute captured image, the cell classification unit 160 adds attribute information and morphological features corresponding to the classification destination ID that is the classification result in addition to the information obtained from the classification model as the classification result. It may be acquired from the cell catalog storage unit 190 and output to the input / output unit 110.

    The relationship between the characteristic object extraction / representative feature amount calculation unit 152 extraction of the characteristic search object and the cell classification unit 160 classification may be realized as follows, for example.

    After storing the morphological feature amount of the search object in the search object temporary storage unit 170, the cell object feature amount calculation unit 126 notifies the cell classification unit 160 that the morphological feature amount has been stored. When the cell classification unit 160 obtains the notification that the morphological feature amount is stored from the cell object feature amount calculation unit 126, the cell classification unit 160 requests the characteristic object extraction / representative feature amount calculation unit 152 to perform extraction or calculation.

    The feature object extraction / representative feature amount calculation unit 152 refers to the image type information stored in the search object temporary storage unit 170 when extraction or calculation is requested from the cell classification unit 160, and extracts the search object. It is determined whether the original is an unknown different-attribute captured image or an unknown identical-attribute captured image. Specifically, the characteristic object extraction / representative feature amount calculation unit 152 indicates that the image type information stored in the search object temporary storage unit 170 indicates that the search object extraction source is an unknown heterogeneous attribute captured image. If the search object extraction source is an unknown different-attribute captured image, the search object extraction source is image type information indicating that the search object extraction source is an unknown identical-attribute captured image. Determines that the extraction source of the search object is an unknown identical-attribute captured image.

    When the characteristic object extraction / representative feature quantity calculation unit 152 determines that the search object extraction source is an unknown heterogeneous attribute captured image, the characteristic search object is extracted from the search object stored in the search object temporary storage unit 170. Are extracted, the classification is requested, and the attribute information and the morphological feature amount of each characteristic search object are output to the cell classification unit 160.

    When the cell classification unit 160 obtains the attribute information and the morphological feature amount of each characteristic search object from the characteristic object extraction / representative feature amount calculation unit 152 together with the classification request, the cell classification unit 160 Are put into a classification model to classify individual characteristic search objects.

    On the other hand, when the characteristic object extraction / representative feature quantity calculation unit 152 determines that the extraction source of the search object is an unknown identical-attribute captured image, the characteristic object extraction / representative feature value calculation unit 152 determines whether the search object is stored in the search object temporary storage unit 170 from A search object is generated, classification is requested, and attribute information and morphological feature of the virtual search object are output to the cell classification unit 160.

    The cell classification unit 160 acquires the attribute information and the morphological feature amount of the virtual search object from the characteristic object extraction / representative feature amount calculation unit 152 together with the classification request, and the cell classification unit 160 acquires the virtual search object. Is input to the classification model, and the unknown identical attribute captured image is classified.

    Hereinafter, the operation of the cell classification device 3 will be described using a flowchart. FIG. 17 is a flowchart showing an example of the operation of the cell classification device 3. Further, it is assumed that nothing is stored in the search object temporary storage unit 170 at the start of the flowchart shown in FIG.

    In FIG. 17, the input / output unit 110 acquires image type information from the outside together with the unknown attribute captured image (step S212). The input / output unit 110 outputs the unknown attribute captured image and image type information acquired from the outside to the binarization processing unit 122 of the cell object recognition unit 120.

    The binarization processing unit 122 binarizes the unknown attribute captured image acquired from the input / output unit 110 using a predetermined threshold (step S214). The binarization processing unit 122 outputs the binarized data and the image type information to the noise estimation unit 124. The noise estimation unit 124 acquires the binarized data from the binarization processing unit 122, and estimates the noise object from all the objects obtained from the acquired binarized data (step S220). The noise estimation unit 124 that has estimated the noise object outputs estimation result information to the cell object feature amount calculation unit 126 together with the binarized data and the image type information. The details of the process of step S220 by the noise estimation unit 124 are the same as the details of the process of step S120 by the noise estimation unit 24 of the classification model generation device 1 according to the first embodiment (FIG. 12).

    The cell object feature amount calculation unit 126 that has acquired the estimation result information together with the binarized data and the image type information from the noise estimation unit 124, based on the binarization data and the estimation result information acquired from the noise estimation unit 124, the 2 A noise object is removed from all objects obtained from the valuation data (step S230). In other words, the cell object feature quantity calculation unit 126 recognizes individual cell objects other than the noise object from the binarized data acquired from the noise estimation unit 124 (step S230).

    Next, the cell object feature value calculation unit 126 calculates the morphological feature value of each recognized search object (step S232). The cell object feature amount calculation unit 126 that has calculated the morphological feature amount of each search object stores the image type information and the morphological feature amount of each search object in the search object temporary storage unit 170 (step S234). . In addition, the cell object feature amount calculation unit 126 notifies the cell classification unit 160 that the morphological feature amount is stored. The cell classification unit 160 that has acquired the notification that the morphological feature amount is stored requests the feature object extraction / representative feature amount calculation unit 152 to perform extraction or calculation.

    The characteristic object extraction / representative feature amount calculation unit 152 requested to extract or calculate from the cell classification unit 160 determines whether or not the unknown attribute captured image acquired in step S212 is an unknown heterogeneous attribute captured image ( Step S280).

    When the characteristic object extraction / representative feature amount calculation unit 152 determines that the unknown attribute captured image acquired in step S212 is an unknown heterogeneous attribute captured image (step S280: Yes), the characteristic object extraction / representative feature amount calculation unit 152 stores the extracted in the extraction condition storage unit 174. The characteristic search object is extracted from the search object stored in the search object temporary storage unit 170 in accordance with the extraction condition (step 282). The characteristic object extraction / representative feature quantity calculation unit 152 that has extracted the characteristic search object requests classification, and outputs attribute information and morphological feature quantity of each characteristic search object to the cell classification unit 160. The cell classification unit 160 that obtains the attribute information and the morphological feature amount of each characteristic search object from the characteristic object extraction / representative feature amount calculation unit 152 together with the request for the classification, classifies each characteristic search object as a classification model. And classifying the individual characteristic search objects in the unknown different-attribute captured image (step S284).

    On the other hand, the characteristic object extraction / representative feature amount calculation unit 152 determines that the unknown attribute captured image acquired in step S212 is not an unknown heterogeneous attribute captured image (step S280: No), that is, acquired in step S212. If it is determined that the unknown attribute captured image is an unknown identical attribute captured image, a virtual search object is generated from the search object stored in the search object temporary storage unit 170 (step 286). The characteristic object extraction / representative feature quantity calculation unit 152 that has generated the virtual search object outputs a request for classification and the attribute information and the morphological feature quantity of the virtual search object to the cell classification unit 160. The cell classification unit 160 that has acquired the attribute information and morphological feature amount of the virtual search object from the characteristic object extraction / representative feature amount calculation unit 152 together with the request for classification, inputs the virtual search object to the classification model, The unknown identical attribute captured images are classified (step S288).

    Subsequent to step S284 or step S288, the cell classification unit 160 outputs the classification result to the input / output unit 110 (step S290). Specifically, when classifying individual characteristic search objects in step S284, the cell classification unit 160 corresponds to, for example, a classification destination ID indicating a classification destination of each characteristic search object, and each classification destination ID. The attached attribute information and the morphological feature amount of each characteristic search object are output to the input / output unit 110. When the cell classification unit 160 classifies the unknown identical attribute captured image in step S288, for example, the classification destination ID indicating the classification destination of the unknown identical attribute captured image, the attribute information associated with the classification destination ID, for example. The morphological feature amount of the virtual search object is output to the input / output unit 110.

    Next, the cell classification unit 160 deletes the search object stored in the search object temporary storage unit 170 (step S292), and the flowchart shown in FIG. 17 ends. For example, the cell classification unit 160 stores the search object temporary storage unit 170 by requesting the cell object feature amount calculation unit 126 to delete the search object stored in the search object temporary storage unit. Delete the search object that has been deleted.

(Fourth embodiment)
The fourth embodiment of the present invention will be described below with reference to the drawings. FIG. 18 is an example of a functional block diagram of the cell classification device 4 according to the fourth embodiment of the present invention. FIG. 19 is an example of information stored in the search object temporary storage unit 270.

    As shown in FIG. 17, the cell classification device 4 includes an input / output unit 210, a cell object recognition unit 220, a characteristic object extraction / representative feature amount calculation unit 252, a candidate classification model formation unit 254, a classification model determination / formation unit. 256, a cell classification unit 260, a search object temporary storage unit 270, a cell object storage unit 272, an extraction condition storage unit 274, a candidate classification model storage unit 276, a classification model storage unit 280, and a cell catalog storage unit 290. The cell object recognition unit 220 includes a binarization processing unit 222, a noise estimation unit 224 (binarized data acquisition unit), and a cell object feature amount calculation unit 226 (noise removal unit).

    The input / output unit 210 acquires a captured image and image type information from the outside. Specifically, the input / output unit 210, together with attribute information (or attribute information and imaging condition information), as well as the known identical attributes, is the same as the input / output unit 10 of the classification model generation devices 1 and 2 according to the first embodiment. Image type information indicating that the image is a captured image and a known image having the same attribute is acquired from the outside. Also, the input / output unit 210 is similar to the input / output unit 110 of the cell classification device 3 according to the third embodiment described above, image type information indicating that it is an unknown heterogeneous attribute captured image and an unknown heterogeneous attribute captured image, or Image type information indicating that it is an unknown identical attribute captured image and an unknown identical attribute captured image is acquired from the outside. The input / output unit 210 outputs information acquired from the outside to the binarization processing unit 222.

    Further, the input / output unit 210 may acquire the classification destination range designation information from the outside together with the unknown attribute captured image (unknown different attribute captured image or unknown identical attribute captured image), and output the classification destination range designation information to the binarization processing unit 222. . The classification destination range designation information is information that designates a classification destination range of an unknown attribute captured image or a classification destination range of cells in an unknown attribute captured image.

    Similar to the cell object recognition unit 20 of the classification model generation device 1 according to the first and second embodiments, the cell object recognition unit 220 recognizes an imaging region of each cell in a known identical attribute captured image as a cell object. . In addition, the cell object recognition unit 220 recognizes each cell object in the unknown attribute captured image as a search object, like the cell object recognition unit 120 of the cell classification device 3 according to the third embodiment.

    That is, the binarization processing unit 222 binarizes a known identical attribute captured image with a predetermined threshold as in the binarization processing unit 22 of the classification model generation apparatuses 1 and 2 according to the first and second embodiments. Process. Similarly to the binarization processing unit 122 of the cell classification device 3 according to the third embodiment, the binarization processing unit 222 binarizes the unknown attribute captured image with a predetermined threshold. The binarization processing unit 222 outputs the binarized data to the noise estimation unit 124 together with the information acquired from the input / output unit 210.

    The noise estimation unit 224 estimates a noise object from all the objects in the known identical attribute captured image, like the noise estimation unit 24 of the classification model generation apparatuses 1 and 2 according to the first and second embodiments. Moreover, the noise estimation part 224 estimates a noise object from all the objects in an unknown attribute picked-up image similarly to the noise estimation part 124 of the cell classification device 3 by said 3rd Embodiment. The noise estimation unit 224 outputs the estimation result information to the noise estimation unit 124 together with the information acquired from the binarization processing unit 222.

    The cell object feature amount calculation unit 226 is similar to the cell object feature amount calculation unit 26 of the classification model generation apparatuses 1 and 2 according to the first and second embodiments described above, and each cell object form in the known identical attribute captured image. The characteristic feature amount is calculated. The cell object feature amount calculation unit 226 that has calculated the morphological feature amount of each cell object stores the morphological feature amount of each cell object in the cell object in association with the attribute information (or attribute information and imaging condition information). Store in the unit 272.

    In addition, the cell object feature quantity calculation unit 226 is similar to the cell object feature quantity calculation unit 126 of the cell classification device 3 according to the third embodiment described above, and each cell object (that is, search object) in the unknown attribute captured image. The morphological feature amount of is calculated. The cell object feature value calculation unit 226 that calculates the morphological feature value of each search object stores the morphological feature value of each search object in the search object temporary storage unit 270 in association with the image type information. In addition, the cell object feature amount calculation unit 226 stores the classification destination range designation information in the search object temporary storage unit 270 when the classification destination range designation information is acquired from the noise estimation unit 224.

    The search object temporary storage unit 270 stores information related to the search object. Specifically, as shown in FIGS. 18A and 18B, the search object temporary storage unit 270 associates the morphological feature amount of each search object with the image type information and the classification destination range designation information. Remember. The initial value of the item “image type information” in the search object temporary storage unit 270 is the same as that of the search object temporary storage unit 170 of the cell classification device 3 according to the third embodiment. The initial value of the item “classification destination range designation information” in the search object temporary storage unit 270 is classification destination range designation information indicating that no classification range is designated. Therefore, when the cell object feature quantity calculation unit 226 has not acquired the classification destination range designation information from the noise estimation unit 224, the classification destination indicating that no classification range is designated in the item “classification destination range designation information”. Range designation information is stored.

The cell object storage unit 272 is the same as the cell object storage unit 72 of the classification model generation apparatuses 1 and 2 according to the first and second embodiments. The extraction condition storage unit 274 is the same as the extraction condition storage unit 74 of the classification model generation device 1 according to the first embodiment.
The extraction condition storage unit 274 extracts an extraction condition for extracting a characteristic object from the cell object storage unit 272 (that is, an extraction condition for extracting a characteristic object used when generating a classification model), and a search object temporary storage. The extraction condition for extracting the characteristic search object from the unit 270 (that is, the extraction condition for extracting the characteristic search object to be classified) may be stored separately.

The candidate classification model storage unit 276 is the same as the candidate classification model storage unit 76 of the classification model generation device 1 according to the first embodiment. The classification model storage unit 280 is the same as the classification model storage unit 80 of the classification model generation apparatuses 1 and 2 according to the first and second embodiments.
The cell catalog storage unit 290 is the same as the cell catalog storage unit 190 of the cell classification device 3 according to the third embodiment.

    The characteristic object extraction / representative feature quantity calculation unit 252 includes the characteristic object extraction unit 32 of the classification model generation device 1 according to the first embodiment and the classification model generation device 2 according to the second embodiment. The representative feature amount calculation unit 42 is provided.

    Specifically, the characteristic object extraction / representative feature amount calculation unit 252 stores the characteristic object in the cell object storage unit 272 according to the extraction condition stored in the extraction condition storage unit 274 when the characteristic object is to be extracted. A characteristic object is extracted from the stored cell objects. However, when the classification object range designation information indicating that the classification range is designated is stored in the search object temporary storage unit 270, the characteristic object extraction / representative feature amount calculation unit 252 includes the cell object storage unit 272. A characteristic object is extracted from the cell objects in the specified classification range among the cell objects stored in. When the characteristic object extraction / representative feature quantity calculation unit 252 extracts a characteristic object from the cell objects stored in the cell object storage unit 272, the attribute information and the morphological feature quantity of each characteristic object are extracted. Is output to the candidate classification model forming unit 254.

    Further, the characteristic object extraction / representative feature amount calculation unit 252 calculates a representative feature amount from the cell object stored in the cell object storage unit 272 when the representative feature amount is to be calculated. In other words, the characteristic object extraction / representative characteristic amount calculation unit 252 generates a virtual object of each known identical attribute captured image. However, when the classification object range designation information indicating that the classification range is designated is stored in the search object temporary storage unit 270, the characteristic object extraction / representative feature amount calculation unit 252 includes the cell object storage unit 272. The representative feature amount is calculated by using the cell objects in the specified classification range among the cell objects stored in. When the characteristic object extraction / representative feature amount calculation unit 252 generates a virtual object of each known identical attribute captured image, the classification model determination / formation unit 256 determines the attribute information and the morphological feature amount of each virtual object. Output to.

    Further, the characteristic object extraction / representative feature quantity calculation unit 252 stores the image type information indicating that the search object extraction source is an unknown heterogeneous attribute captured image in the search object temporary storage unit 270. Extracts a characteristic search object from the search objects stored in the search object temporary storage unit 270 in accordance with the extraction conditions stored in the extraction condition storage unit 274. When the characteristic object extraction / representative characteristic amount calculation unit 252 extracts characteristic search objects from the search objects stored in the search object temporary storage unit 170, the attribute information and the morphological features of each characteristic search object are extracted. The amount is output to the cell classification unit 260.

    Further, the characteristic object extraction / representative feature amount calculation unit 252 stores, when the search object temporary storage unit 270 stores image type information indicating that the search object extraction source is an unknown same-attribute captured image. Calculates a representative feature amount from the search object stored in the search object temporary storage unit 270. In other words, the characteristic object extraction / representative characteristic amount calculation unit 252 generates a virtual search object for an unknown identical attribute captured image. The characteristic object extraction / representative feature amount calculation unit 252 outputs the attribute information and the morphological feature amount of the virtual search object to the cell classification unit 260 when the virtual search object of the unknown identical attribute captured image is generated.

    The candidate classification model formation unit 254 is the same as the candidate classification model formation unit 34 of the classification model generation device 1 according to the first embodiment described above. A candidate classification model is formed (generated) using the attribute information and the morphological feature amount.

    The classification model determination / formation unit 256 includes the classification model determination unit 36 of the classification model generation device 1 according to the first embodiment and the classification model formation unit 46 of the classification model generation device 2 according to the second embodiment. It has the function of.

    Specifically, the classification model determination / formation unit 256, when a plurality of candidate classification models are formed by the candidate classification model formation unit 254, a plurality of candidate classification models stored in the candidate classification model storage unit 276. A classification model is determined from the above and stored in the classification model storage unit 280.

    In addition, when the classification model determination / formation unit 256 acquires the attribute information and the morphological feature amount of a plurality of virtual objects for each known identical attribute captured image from the characteristic object extraction / representative feature amount calculation unit 252. The classification model is generated (formed) using the attribute information and the morphological feature amount of the plurality of virtual objects.

    Similar to the cell classification unit 160 of the cell classification device 3 according to the third embodiment, the cell classification unit 260 is an unknown attribute captured image or a cell (specifically, individual characteristic captured image). Search object), and the classification result is output to the input / output unit 210.

    Specifically, when the cell classification unit 260 acquires attribute information and morphological feature amounts of a plurality of characteristic search objects from the characteristic object extraction / representative feature amount calculation unit 252, the plurality of characteristic features are extracted. The attribute information and the morphological feature amount of the search object are input to the classification model, and the individual characteristic search object is classified.

    In addition, when the cell classification unit 260 acquires the attribute information and the morphological feature amount of the virtual search object from the characteristic object extraction / representative feature amount calculation unit 252, the attribute information and the form of the virtual search object The characteristic feature amount is input to the classification model, and the unknown identical attribute captured image is classified.

    Hereinafter, the operation of the cell classification device 4 will be described using a flowchart. FIG. 20 is a flowchart showing an example of the operation of the cell classification device 4. Note that, at the start of the flowchart shown in FIG. 20, it is assumed that a plurality of cell objects respectively recognized from a plurality of known identical attribute captured images are stored in the cell object storage unit 272. Further, it is assumed that nothing is stored in the search object temporary storage unit 270 at the start of the flowchart shown in FIG.

    In FIG. 20, the input / output unit 210 obtains an unknown attribute captured image together with image type information and classification destination range designation information from the outside (step S312), and outputs it to the binarization processing unit 222. The binarization processing unit 222 binarizes the unknown attribute captured image with a predetermined threshold (step S314). The binarization processing unit 222 outputs the binarized data, the image type information, and the classification destination range designation information to the noise estimation unit 224.

The noise estimation unit 224 estimates a noise object among all objects obtained from the binarized data acquired from the binarization processing unit 222 (step S320). The noise estimation unit 224 that estimated the noise object outputs the estimation result information to the cell object feature amount calculation unit 226 together with the binarized data, the image type information, and the classification destination range designation information.
The details of the process of step S320 by the noise estimation unit 224 are the same as the details of the process of step S120 by the noise estimation unit 24 of the classification model generation device 1 according to the first embodiment (FIG. 12).

    The cell object feature amount calculation unit 226 that has acquired the estimation result information together with the binarized data, the image type information, and the classification destination range designation information from the noise estimation unit 224 has the binarized data and the estimation acquired from the noise estimation unit 224. Based on the result information, the noise object is removed from all objects obtained from the binarized data (step S330). In other words, the cell object feature quantity calculation unit 226 recognizes individual cell objects other than the noise object from the binarized data acquired from the noise estimation unit 224 (step S330).

    Next, the cell object feature value calculation unit 226 calculates the morphological feature value of each recognized search object (step S332). The cell object feature amount calculation unit 226 that has calculated the morphological feature amount of each search object stores the image type information, the classification destination range designation information, and the morphological feature amount of each search object in the search object temporary storage unit 270. Store (step S334). In addition, the cell object feature amount calculation unit 226 notifies the cell classification unit 260 that the morphological feature amount of the search object has been stored. The cell classification unit 260 that has acquired the notification that the morphological feature amount of the search object has been stored requests the feature object extraction / representative feature amount calculation unit 252 to perform extraction or calculation.

    The characteristic object extraction / representative feature quantity calculation unit 252 requested to extract or calculate from the cell classification unit 260 determines whether or not the unknown attribute captured image acquired in step S312 is an unknown heterogeneous attribute captured image ( Step S340). The characteristic object extraction / representative feature quantity calculation unit 252 is unknown if the search object temporary storage unit 270 stores image type information indicating that the search object extraction source is an unknown heterogeneous attribute captured image. It is determined that the image is a heterogeneous attribute captured image.

    When the characteristic object extraction / representative feature amount calculation unit 252 determines that the unknown attribute captured image acquired in step S312 is an unknown heterogeneous attribute captured image (step S340: Yes), the characteristic object extraction / representative feature amount calculation unit 252 stores the extracted attribute attribute image in the extraction condition storage unit 274. In accordance with the extraction conditions, a characteristic object is extracted from the cell objects stored in the cell object storage unit 272 (step S342). The feature object extraction / representative feature amount calculation unit 252 that has extracted the feature object outputs the attribute information and the morphological feature amount of each feature object to the candidate classification model formation unit 254.

    The candidate classification model forming unit 254 that has acquired the attribute information and the morphological feature amount of each characteristic object forms a candidate classification model based on the attribute information and the morphological feature amount of each characteristic object. The model is stored in the candidate classification model storage unit 276 (step S344). Next, the candidate classification model forming unit 254 determines whether all candidate classification models have been formed (step S346). For example, the candidate classification model forming unit 254 requests extraction from the characteristic object extraction / representative feature amount calculation unit 252 and obtains notification from the characteristic object extraction / representative feature amount calculation unit 252 that extraction has been completed. If so, it is determined that all candidate classification models have been formed. If the candidate classification model forming unit 254 determines that not all candidate classification models have been formed (step S346: No), the process returns to step S342. If it is determined that all candidate classification models have been formed (step S346: Yes), the candidate classification model forming unit 254 requests the classification model determination / formation unit 256 to determine a classification model.

    The classification model determination / formation unit 256 requested to determine the classification model determines a classification model from among a plurality of candidate classification models, and stores the classification model in the classification model storage unit 280 (step S350). If the previous classification model is stored in the classification model storage unit 280, the classification model determination / formation unit 256 overwrites it with the classification model determined this time. The classification model determination / formation unit 256 that stores the classification model in the classification model storage unit 280 requests the characteristic object extraction / representative feature amount calculation unit 252 to perform extraction.

    The characteristic object extraction / representative feature quantity calculation unit 252 requested to be extracted by the classification model determination / formation unit 256 is characterized by the search object temporary storage unit 270 according to the extraction conditions stored in the extraction condition storage unit 274. A search object is extracted (step 382). The characteristic object extraction / representative feature quantity calculation unit 252 that has extracted the characteristic search object requests the cell classification unit 260 to classify and outputs the attribute information and the morphological feature quantity of the characteristic search object. The cell classification unit 260 that has acquired the attribute information and the morphological feature amount of the characteristic search object together with the request for classification from the characteristic object extraction / representative feature amount calculation unit 252, uses each characteristic search object as a classification model. And classifying the individual characteristic search objects in the unknown different-attribute captured image (step S384).

    On the other hand, when the characteristic object extraction / representative feature amount calculation unit 252 determines that the unknown attribute captured image acquired in step S312 is not an unknown heterogeneous attribute captured image (step S340: NO), the characteristic object extraction / representative feature amount calculation unit 252 stores it in the cell object storage unit 272. The representative feature amount is calculated from the cell object that has been set (step S352). In other words, the characteristic object extraction / representative characteristic amount calculation unit 252 generates a virtual object from the cell objects stored in the cell object storage unit 272 (step S352). The characteristic object extraction / representative feature amount calculation unit 252 that calculates the representative feature amount from each of the known identical attribute captured images sends the attribute information and the morphological feature amount of each virtual object to the classification model determination / formation unit 256. Output.

    The classification model determination / formation unit 256 that has acquired the attribute information and the morphological feature amount of each virtual object forms a classification model based on the attribute information and the morphological feature amount of each virtual object. Is stored in the classification model storage unit 280 (step S360). If the previous classification model is stored in the classification model storage unit 280, the classification model determination / formation unit 256 overwrites it with the classification model determined this time. The classification model determination / formation unit 256 that stores the classification model in the classification model storage unit 280 requests the characteristic object extraction / representative feature amount calculation unit 252 to calculate.

    The characteristic object extraction / representative feature amount calculation unit 252 requested to calculate by the classification model determination / formation unit 256 calculates a representative feature amount from the search object stored in the search object temporary storage unit 270 (step S386). That is, the characteristic object extraction / representative characteristic amount calculation unit 252 generates a virtual search object from the search object stored in the search object temporary storage unit 270 (step S386). The characteristic object extraction / representative feature quantity calculation unit 252 that has calculated the representative feature quantity requests the cell classification unit 260 to classify and outputs the attribute information and the morphological feature quantity of the virtual search object. The cell classification unit 260 that has acquired the attribute information and the morphological feature amount of the virtual search object together with the classification request from the characteristic object extraction / representative feature amount calculation unit 252 inputs the virtual search object to the classification model. Then, the unknown identical attribute captured images are classified (step S388).

    Subsequent to step S384 or step S388, the cell classification unit 260 outputs the classification result to the input / output unit 210 (step S390). Next, the cell classification unit 260 deletes the search object stored in the search object temporary storage unit 270 (step S392), and the flowchart shown in FIG.

    Note that in step S342, the characteristic object extraction / representative feature amount calculation unit 252 determines that the classification destination range designation information indicating that the classification range is designated is stored in the search object temporary storage unit 270, the cell A characteristic object is extracted from the cell objects in the specified classification range among the cell objects stored in the object storage unit 272. Similarly, in step S352, the characteristic object extraction / representative characteristic amount calculation unit 252 stores classification destination range designation information indicating that a classification range is designated in the search object temporary storage unit 270. The representative feature amount is calculated using the cell objects in the designated classification range among the cell objects stored in the cell object storage unit 272.

Note that, in step S382, the characteristic object extraction / representative characteristic amount calculation unit 252 uses a classification object as a classification model among the extracted patterns when the characteristic object is extracted from the cell objects stored in the cell object storage unit 272. It is preferable to extract the characteristic search object from the search object temporary storage unit 270 according to the extraction pattern when the determined candidate classification model is extracted. For example, in the example shown in FIGS. 4 and 5, the characteristic object extraction / representative characteristic amount calculation unit 252 has an area from an average value to an average value among (N ₁ + N ₂ + N ₃ ) cell objects. It is within the range of 25% larger value (extraction condition 1), and the length is within the range of the average value to 25% larger than the average value (extraction condition 3), and the width is the average value. When a cell object that is within a range of 25% larger than the average value (extraction condition 5) is extracted as a characteristic object, and finally a candidate classification model formed from the characteristic object is determined as a classification model The search object temporary storage unit 270 extracts the search object that is the extraction condition 1, the extraction condition 3, and the extraction condition 5 as a characteristic search object. . That is, in short, the characteristic object extraction / representative characteristic amount calculation unit 252 extracts characteristic search objects using the same extraction pattern as the extraction pattern reflected in the classification model.

    There are various methods for extracting a characteristic search object by using the same extraction pattern as the extraction pattern reflected in the classification model. For example, the characteristic object extraction / representative feature amount calculation unit 252 performs the extraction pattern. Is output to the candidate classification model formation unit 254, the candidate classification model formation unit 254 stores the information indicating the extraction pattern in the candidate classification model storage unit 276, and the classification model determination / formation unit 256 outputs the extraction pattern. Information to be stored is stored in the classification model storage unit 280. After that, the characteristic object extraction / representative feature quantity calculation unit 252 refers to the classification model storage unit 280 when extracting the characteristic search object, and extracts when the candidate classification model determined as the classification model is extracted. Identify the pattern.

Similarly, in step S386, the characteristic object extraction / representative characteristic amount calculation unit 252 stores the search object temporarily according to the calculation method when the virtual object is generated from the cell object stored in the cell object storage unit 272. It is preferable to generate a virtual search object from the search object stored in the unit 270.
For example, the feature object extraction / representative feature amount calculation unit 252 calculates the median value of each morphological feature amount of a plurality of cell objects extracted from the known identical attribute captured image as a representative feature amount, When a virtual object having the median feature value as its morphological feature value is generated, the median value of each morphological feature value of the search object stored in the search object temporary storage unit 270 is a representative feature. As a quantity, a virtual search object is generated in which the median value of each morphological feature quantity is its own morphological feature quantity. That is, in short, the characteristic object extraction / representative feature quantity calculation unit 252 generates a virtual search object by the same calculation method as the representative feature quantity calculation method reflected in the classification model.

    There are various methods for generating a virtual search object by the same calculation method as the representative feature amount reflected in the classification model. For example, a feature object extraction / representative feature amount calculation unit 252 outputs information indicating the calculation method when the virtual object is generated to the classification model determination / formation unit 256, and the classification model determination / formation unit 256 transmits the information indicating the calculation method to the classification model storage unit 280. Remember. After that, the characteristic object extraction / representative feature quantity calculation unit 252 refers to the classification model storage unit 280 when generating the virtual search object, and specifies the calculation method when the virtual object is generated.

    As described above, the classification model generation device 1 according to the first embodiment, the classification model generation device 2 according to the second embodiment, the cell classification device 3 according to the third embodiment, and the cell classification device 4 according to the fourth embodiment. As described above, according to the cell classification device 3, by using a classification model that classifies the attributes of the cell object according to a predetermined classification standard, the cell type, The state etc. can be confirmed. Moreover, according to the classification model generation apparatuses 1 and 2, the classification model used in the cell classification apparatus 3 can be easily generated. Furthermore, according to the cell classification device 4, since the latest classification model is generated every time the cell is confirmed, the type and state of the cell can be confirmed with higher accuracy.

    Moreover, since the noise object is accurately removed by the noise estimation units included in the classification model generation devices 1 and 2 and the cell classification devices 3 and 4, the cell object can be extracted with high accuracy.

    In addition, although the example of the classification model classified in steps was demonstrated as the classification model which the classification model production | generation apparatus 1 produces | generates, you may produce | generate the classification model classified at a stretch in one step. The same applies to the classification model generated by the classification model generation device 2, the classification model used by the cell classification device 3, and the classification model generated and used by the cell classification device 4. In addition, the classification model according to each embodiment includes one that determines whether or not an attribute is at most.

    In the cell classification device 3 according to the third embodiment, the characteristic object extraction / representative characteristic amount calculation unit 152 extracts a characteristic search object from the search objects stored in the search object temporary storage unit 170, and Although the attribute information and the morphological feature amount of each characteristic search object are output to the cell classification unit 160, the characteristic object extraction / representative feature amount calculation unit 152 performs a search that is not a characteristic search object and a characteristic search object. In a manner that can distinguish an object (referred to as a non-characteristic search object) (for example, with a label), in addition to the characteristic search object, a non-characteristic search object may be output to the cell classification unit 160.

    The cell classification unit 160 that has acquired the non-characteristic search object in addition to the characteristic search object also inputs the non-characteristic search object into the classification model, as well as the characteristic search object. The classification result of the characteristic search object may be output, and the classification result of each non-characteristic search object may be output together with the information that the classification accuracy is not high. Further, the cell classification unit 160 that has acquired the non-characteristic search object in addition to the characteristic search object may output “non-classifiable” for each non-characteristic search object. The same applies to the characteristic object extraction / representative feature amount calculation unit 252 and the cell classification unit 260 of the cell classification device 4 of the fourth embodiment.

    In the fourth embodiment, the characteristic object extraction / representative feature amount calculation unit 252 of the cell classification device 4 performs the classification model from the cell objects stored in the cell object storage unit 272 when generating the classification model. A virtual object is generated as an object serving as a basis for the search, and a virtual search object to be classified is generated from a search object stored in the search object temporary storage unit 270 at the time of classification. The characteristic object extraction / representative feature quantity calculation unit 252 generates all the cell objects stored in the cell object storage unit 272 without generating a virtual object when generating the classification model. In addition, the virtual search object is Without growth, all the search objects stored in the retrieval object temporary storage unit 270 (i.e., all search objects obtained from an unknown identity attribute image) may be a classification target. In the above case, for example, the cell classification unit 260 inputs all the search objects stored in the search object temporary storage unit 270 into the classification model, and classifies the branch with the largest number of search objects as the classification of the unknown identical attribute image. It may be the destination.

    In the first embodiment, the cell object storage unit 72 of the classification model generation device 1 has been described as an example of storing morphological feature quantities having no temporal elements. However, the input / output unit 10 captures images at different times. The morphological feature amount having temporal elements may be stored in the cell object storage unit 72 by inputting the plurality of known identical attribute captured images. For example, the input / output unit 10 captures a known identical attribute captured image a captured at a first time (for example, the eighth hour of culture) for a cell having a certain attribute, and a second time (for example, the culture 16) for the cell. A known identical attribute captured image b captured at time) is input. The cell object feature amount calculation unit 26 calculates the morphological feature amount of the cell object of the known identical attribute captured image a, and stores the cell object storage unit 72 in the cell object storage unit 72 from the first time to the second time. The temporal change amount of the morphological feature amount is stored. An example of the temporal change amount of the morphological feature amount is difference information for each morphological feature amount such as an area temporal change amount, a length temporal change amount, and a width change amount.

    Of course, when the morphological feature quantity having a temporal element is stored in the cell object storage unit 72, the candidate classification model generated by the candidate classification model forming unit 34 and the classification model determining unit 36 are The classification model to be determined and stored in the classification model storage unit 80 has a temporal element. For example, the rule of each stage in the classification model has a morphological element having a temporal component such that “the amount of change in cell length from the first time to the second time is a value“ 1 ”or less”. In addition, the extraction conditions stored in the extraction condition storage unit 74, the evaluation by the classification model determination unit 36, and the like also correspond to the morphological feature amounts having temporal elements.

    Similarly, in the classification model generation device 2 of the second embodiment, the cell classification device 3 of the third embodiment, and the cell classification device 4 of the fourth embodiment, a classification model having a temporal element is used. May be. Of course, when a classification model having a temporal element is used, classification (retrieval) is also performed based on the temporal element. Therefore, for example, the input / output unit 110 of the cell classification device 3 externally captures the unknown attribute captured image captured at the first time and the unknown attribute captured image captured at the second time when the captured image and the subject are the same. Get from.

    In the first embodiment, it has been described that the classification model generation device 1 is separate from the incubator 311. However, the function of the classification model generation device 1 may be implemented in the incubator 311. For example, a program for executing each process of the classification model generating device 1 may be stored in the storage unit 343 illustrated in FIG. 8 and executed by the CPU 342. The same applies to the classification model generation device 2 and the cell classification devices 3 and 4.

    In the third embodiment, when the function of the cell classification device 3 is implemented in the incubator 311, the culture conditions of the cells or cell groups in the culture container 319 are determined based on the classification result of the cell classification device 3. You may make it change dynamically. Thereby, a desired cell can be easily proliferated. In the fourth embodiment, the same applies to the case where the function of the cell classification device 4 is implemented in the incubator 311.

    Hereinafter, the operation of the incubator 311 having the function of the cell classification device 4 will be described using a flowchart. FIG. 21 is a flowchart showing an example of the operation of the incubator 311 having the function of the cell classification device 4. At the start of the flowchart shown in FIG. 21, it is assumed that a culture vessel 319 for culturing desired cells to be proliferated is placed in the incubator 311. Further, as described above, the cell classification device 3 included in the incubator 311 outputs attribute information including the culture condition together with the classification destination ID as the classification result.

    In FIG. 21, the incubator 311 starts cell culture (step S500). That is, the CPU 342 controls the temperature adjustment device 315a and the humidity adjustment device 315b so as to culture the cells under the set culture conditions (for example, predetermined temperature, humidity, etc.). After a predetermined period of time, the incubator 311 classifies the cells in culture (step S510). That is, the CPU 342 controls the imaging device 334 to image cells in culture, and then reads out a program for executing each process of the cell classification device 3 from the storage unit 343 and executes the read program. As shown in FIG. 17 above, the cells in culture are classified.

    Next, the incubator 311 controls whether to stop the cell culture or change the cell culture conditions based on the classification result. Specifically, first, the CPU 342 determines whether or not the cell quality indicated by the classification result matches the target quality set in step S500 (step S520). If the CPU 342 determines that the cell quality indicated by the classification result matches the set target quality (step S520: Yes), the CPU 342 monitors whether the cell quality indicated by the classification result matches the set target quality, etc. (Not shown) is displayed (step S522). And it progresses to step S560 mentioned later.

    On the other hand, when the CPU 342 determines that the cell quality included in the classification result does not match the predetermined target quality (step S520: No), the CPU 342 indicates that the cell quality indicated by the classification result does not match the set target quality. It is displayed on a monitor or the like (not shown) (step S524), and it is determined whether or not the culture is stopped (step S530). Specifically, for example, the CPU 342 determines that the culture is to be stopped when the degree of mismatch between the cell quality indicated by the classification result and the set target quality is very large. For example, when the difference between the value indicating the cell quality indicated by the classification result and the set value indicating the target quality is equal to or greater than a predetermined first threshold, it is determined that the culture is to be stopped. Note that the CPU 342 determines that the culture is to be stopped even when there is an input from the operator to stop the culture.

    When the CPU 342 determines that the culture is to be stopped (step S530: Yes), the CPU 342 displays on the monitor or the like (not shown) that the culture is stopped (step S532), and ends the culture process. Then, the flowchart shown in FIG. 20 ends. On the other hand, when determining that the culture is not stopped (step S530: No), the CPU 342 determines whether or not to change the culture conditions (step S540). Specifically, for example, the CPU 342 determines to change the culture condition when the degree of mismatch between the cell quality indicated by the classification result and the set target quality is large. For example, the difference between the value indicating the quality of the cell indicated by the classification result and the value indicating the target quality that has been set is greater than or equal to a predetermined second threshold (where the first threshold is greater than the second threshold). In some cases, it is determined that the culture conditions are changed.

    If the CPU 342 determines that the culture conditions are to be changed (step S540: Yes), the CPU 342 displays on the monitor or the like (not shown) that the culture conditions are to be changed (step S542), and changes the culture conditions (step S550). That is, the CPU 342 controls the temperature adjustment device 315a and the humidity adjustment device 315b so that cells are cultured under new culture conditions. Then, the process returns to step S510. Note that both qualities (cell quality indicated by the classification result and set target quality) and new culture conditions (modified culture conditions) are associated with each other and stored in the storage unit 343, and the CPU 342 New culture conditions may be acquired from both cell qualities. Note that the CPU 342 may change the culture condition to the set value when the operator inputs a new set value of the culture condition.

    Following step S522 or when determining that the culture conditions are not to be changed (step S540: No), the CPU 342 controls the imaging device 334 to image the cells in culture after the predetermined period has elapsed, The number of cells (that is, the number of cell objects) is counted (step S560). Note that the CPU 342 reads out a program for executing the processing of the cell object feature quantity calculation unit 126 of the cell classification device 3 capable of recognizing individual cell objects from the storage unit 343, and executes the read program to execute the number of cells. Count.

    Next, the CPU 342 determines whether or not the number of cells counted in step S560 has reached a predetermined number of cells (step S570). When the CPU 342 determines that the counted number of cells has not reached the predetermined number of cells (step S570: No), the CPU 342 returns to step S510 after a predetermined time has elapsed. On the other hand, when the CPU 342 determines that the counted number of cells has reached the predetermined number of cells (step S570: Yes), the CPU 342 displays on the monitor or the like (not shown) that the culture of the cells has been completed (step S572). . Then, the flowchart shown in FIG. 21 ends.

    As described above, as shown in the flowchart of FIG. 21, the culture conditions of the cells in the culture vessel 319 can be dynamically changed based on the classification result of the cell classification device 4, so that desired cells can be easily propagated. Will be able to.

    The classification model generation device 1 according to the first embodiment of the present invention, the classification model generation device 2 according to the second embodiment of the present invention, the cell classification device 3 according to the third embodiment of the present invention, the first of the present invention. By recording a program for executing each process of the cell classification device 4 according to the third embodiment on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, The above-described various processes related to the classification model generation devices 1 and 2 and the cell classification devices 3 and 4 may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

    Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  As in the first to fourth embodiments, the image processing apparatus in each of the fifth and subsequent embodiments of the present invention uses cells from image data obtained by capturing cells input from the outside. A cell object that is an image area of the image is extracted. At the time of the extraction, the image processing device (401, 401b) determines that an image region having an area equal to or larger than a predetermined threshold is a cell object, and extracts the cell object. In each of the fifth and subsequent embodiments of the present invention, there is a technical feature in how to determine the threshold value. As the threshold value, a common threshold value (provisional threshold value) common to cell types and a unique threshold value specific to the cell type are used. And are used.

  FIG. 22 is a diagram for explaining the outline of the processing of each embodiment in the present invention. As shown in process 1 of FIG. 22, in the fifth embodiment of the present invention, when the cell type is known, the image data input by the image processing apparatus 401 is converted into the cell type shown in process 1 of FIG. A cell object that is an image region of a cell is extracted using a unique threshold that is a unique threshold.

  In the sixth embodiment of the present invention, the image processing device 401b extracts a cell object from image data regardless of whether the cell type is unknown or known. Specifically, the image processing device 401b determines whether the cell type is unknown or known. When the cell type is known, the image processing apparatus 401b extracts a cell object using the unique threshold value in the process 1.

  On the other hand, when the cell type is unknown, the image processing apparatus 401b extracts a cell object using the common threshold value (provisional threshold value) common to all the cell types by the process 2 of FIG. Next, the image processing apparatus 401b estimates a cell type based on the cell object extracted by the process 3 in FIG. Next, the image processing apparatus 401b extracts a cell object by using the unique threshold value that is a unique threshold value for the estimated cell type in the process 1 of FIG.

  In the seventh embodiment of the present invention, whether the cell type is unknown or known, the cell classification device 404 uses the cell object extracted by the image processing device 401b in the cell classification device 404 to generate an image. The cells imaged in the image are classified and the activity or quality of the cells imaged in the image is estimated.

  In the eighth embodiment of the present invention, regardless of whether the cell type is unknown or known, based on the culture condition output by the cell classification device 404 in the incubator 511, the culture condition is changed to match the culture condition.

  Hereinafter, the unique threshold value and the common threshold value (temporary threshold value) described above will be described. First, the unique threshold will be described. Here, the following description will be made on the assumption that a cell candidate object is extracted from image data by performing binarization processing based on a predetermined threshold.

  FIG. 23A is a diagram showing an example of the cumulative frequency distribution with respect to the area of the cell candidate object. In the figure, the vertical axis represents the cumulative frequency, and the horizontal axis represents the area of the cell candidate object. Here, the cumulative frequency will be described. First, the cumulative frequency distribution is obtained by sequentially adding the number of cell candidate objects having each area in the order of the area size.

  For example, when there are a cell candidate objects having an area of 1 pixel, the cumulative frequency on the vertical axis is a where the area of the horizontal axis is 1 pixel. Further, when there are b cell candidate objects having an area of 2 pixels, the cumulative frequency on the vertical axis is a + b when the area of the horizontal axis is 2 pixels.

  When the area of the cell candidate object is smaller than a predetermined threshold, the cell candidate object is considered to be a noise object that is not a cell. On the other hand, when the area of the cell candidate object is greater than or equal to a predetermined threshold, the cell candidate object is considered to be a cell object.

  FIG. 23B is a diagram illustrating an example of a distribution of differential values obtained by differentiating the cumulative frequency distribution of cell candidate objects by area. In the figure, the vertical axis represents a differential value obtained by differentiating the cumulative frequency by area (the slope of the cumulative frequency), and the horizontal axis represents the area of the cell candidate object. The differential value with respect to the area of the cumulative frequency shown in the figure is a differential value obtained by differentiating the cumulative frequency distribution of FIG. Here, an area having a minimum value in a function obtained by differentiating the cumulative frequency with respect to the area (hereinafter referred to as a cumulative frequency differential function) is assumed to be e.

Here, the percentage of error that represents a recognition number n is the number of recognized image processing apparatus (401,401b) cell object, the ratio of the difference in viewing the number n ₀ is the number of measured cells object from the image data ER [%] is represented by the following formula (4).

ER = (n ₀ −n) / n ₀ × 100 (4)

Note that the number of visible cells n ₀ may be a number obtained by staining a cell to be imaged and counting the number of cells by a person or another device from the stained image.

  Expression (4) indicates that the accuracy of recognition of the cell object by the image processing device (401, 401b) improves as the error ratio ER approaches 0 [%] from a positive value. On the other hand, the smaller the error rate ER0 [%] is in the negative direction, the more the image processing apparatus (401, 401b) means that an object that is not originally a cell is erroneously recognized as a cell object.

  FIG. 24 is a diagram for explaining a method of calculating the intrinsic threshold value (custom e value) using the cumulative frequency differential function. In the figure, the vertical axis represents the differential value (the slope of the cumulative frequency) with respect to the area of the cumulative frequency, and the horizontal axis represents the area of the cell candidate object. A value e + β obtained by adding the specific correction information β to the area e having the minimum value of the cumulative frequency differential function is a specific threshold (custom e value).

  FIG. 25 is a conceptual diagram showing the relationship of the error ratio ER to the intrinsic threshold (e + β) obtained when culturing for 8 hours in a certain three cells. In the figure, the vertical axis represents the error ratio ER [%], and the horizontal axis represents the inherent threshold value (e + β). In the A cell, when the intrinsic threshold value (e + β) is an area e where the cumulative frequency differential function takes a minimum value, the error ratio ER is 40 [%]. Further, it is shown that the error ratio ER is 0 [%] when the intrinsic threshold value (e + β) is a value obtained by subtracting 50 [pixel] from the area e where the cumulative frequency differential function takes the minimum value.

As shown in FIG. 25, the image processing apparatus (401, 401b) uses, from the binarized image data, an area e that takes the minimum value of the cumulative frequency differential function as a threshold value, and cell candidate objects having an area equal to or larger than the threshold value as cells. When the object is extracted, the error ratio ER takes a value away from 0 [%]. That is, the extracted recognition count n is the number of cells objects, viewing number is the number of measured cells object from the image data n ₀ and causes very different, the image processing apparatus (401,401b) of the image data There is a problem that the cell object cannot be recognized accurately.

  Therefore, in order for the image processing apparatus (401, 401b) to extract a large number of cell objects accurately from the image data, the threshold value of the area when extracting the cell object is shifted from the area e that takes the minimum value in the cumulative frequency differential function. There is a need.

  Further, FIG. 25 shows that the relationship of the error ratio ER to the area threshold when extracting the cell object for each cell type is different. Therefore, the image processing apparatus (401, 401b) needs to calculate the specific correction information β that is an amount shifted from the area e that takes the minimum value in the cumulative frequency differential function for each cell type.

  FIG. 26 is a diagram in which the error ratio ER in the image data obtained by imaging the A cell is displayed using the area threshold value and the culture time when the cell object is extracted as parameters. It is shown that even for the same A cells, the error rate ER varies with the culture time. Therefore, the image processing apparatus (401, 401b) needs to calculate the unique correction information β every culture time even for the same cell type.

  FIG. 27 is a diagram for explaining a method of calculating a unique threshold value. In the figure, the error rate ER in the image data obtained by imaging the A cells is displayed using the threshold value and the culture time as parameters, and the range of the error rate ER is 0 to 20%. Yes. The image processing apparatus (401, 401b) is a threshold shift amount that is an amount shifted from the area e that takes a minimum value for each culture time, and a threshold at which the error ratio ER is in the range of 0 to 20%. All the shift amounts are extracted, and the median value of the extracted threshold shift amounts is set as the inherent correction information β.

  Next, the common threshold (temporary threshold) will be described. FIG. 28 is a diagram for explaining a method of calculating a common threshold (temporary threshold). In FIG. 28A, the error ratio ER in the image data obtained by imaging the A cell, B cell, or C cell is displayed using the threshold value and the culture time as parameters. In FIG. 28 (b), the error rate ER in the A cell is in the range of 10-30 [%], the error rate ER in the B cell is in the range of 10-30 [%], and the error rate ER in the C cell is A common range is shown in all of the ranges of 10 to 30%.

As shown in FIG. 28B, the image processing apparatus 401b extracts a threshold shift amount in which the error ratio ER is in the range of 10 to 30% as an example for all cells, as shown in FIG. 28 (b). The median value of the extracted threshold shift amounts is set as common correction information γ. A value e + γ obtained by adding correction information γ to the area e that takes the minimum value of the cumulative frequency differential function is a common threshold (temporary threshold).
Above, description of a common threshold value (provisional threshold value) is complete | finished.

<Fifth Embodiment: Image Processing Device>
First, an image processing apparatus 401 according to the fifth embodiment of the present invention will be described. The image processing apparatus 401 according to the fifth embodiment of the present invention extracts a cell object that is an image area of a cell from image data when the cell type is known. FIG. 29 is a block diagram of an image processing apparatus 401 according to the fifth embodiment of the present invention. The image processing apparatus 401 includes a cell object extraction unit 420, a unique correction information generation unit 440, and a unique correction information storage unit 450.

<Cell object extraction method>
First, the cell object extraction method of the image processing apparatus 401 will be described. First, the outline of the process of the cell object extraction unit 420 will be described. The cell object extraction unit 420 is unique in accordance with image data obtained by imaging an externally input cell, information indicating a cell type, and information indicating a culture time that is an elapsed time since the cell was seeded. The correction information β (i, t) is read from the table T1 stored in the unique correction information storage unit 450. Here, i is identification information indicating a cell type, and t is a culture time.

  The unique correction information storage unit 450 stores a table T1 in which the unique correction information β is associated with information indicating the culture time t and identification information i indicating the cell type. FIG. 30 is a diagram showing an example of the table T1 in which the unique correction information β is stored in association with the identification information i indicating the cell type and the culture time t. The table T1 indicates that the unique correction information β is determined as one according to the culture time t and the identification information i indicating the cell type. The culture time t is in increments of 4 hours from 8 hours on the basis of 0 hours. The identification information i indicating the cell type is normal human fibroblast (NHDF), human cervical cancer-derived cancer cell (Hela), normal human umbilical vein endothelial cell (HUVEC), neural progenitor cell (PC12), or mesenchymal system Stem cells (MSC) and the like.

  Returning to FIG. 29, the cell object extraction unit 420 generates binarized data described later from the image data based on the read unique correction information β (i, t), selects a cell object, and selects the selected cell. Information indicating the object o and binarized image data are output to the outside.

Next, details of the processing of the cell object extraction unit 420 will be described. FIG. 31 is a functional block configuration diagram of the cell object extraction unit 420 according to the fifth embodiment of the present invention. The cell object extraction unit 420 includes a cell candidate object extraction unit 421, an area calculation unit 422, a cumulative frequency distribution calculation unit (cumulative distribution calculation unit) 423, a specific threshold value calculation unit 425, and a cell object selection unit 426. .

  The cell candidate object extraction unit 421 includes a binarized image acquisition unit 410 and an extraction unit 411. The binarized image acquisition unit 410 acquires image data that is captured image data obtained by capturing a state where a plurality of cells of known cell types are cultured from the outside. The binarized image acquisition unit 410 binarizes the acquired image data with a predetermined threshold. The predetermined threshold may be in accordance with the type of cell.

For example, the binarized image acquisition unit 410 has one threshold value when an adhesive cell that is a cell that adheres to a culture container is imaged in image data in which a cell region is captured in black compared to other regions. Is used to binarize the image data to generate binarized image data. Here, it is assumed that each pixel of the image data takes a pixel value of any value from 0 to 255. For example, in the image data, 0 represents black and 255 represents white.
For example, the binarization processing unit 422 uses the pixel value 84 as one threshold, a region less than the pixel value 84 is a cell candidate region (eg, black when viewed in the binarized image data), the pixel value 84 The above regions are regions other than the cell candidates (for example, white when viewed in the binarized image data).

Further, for example, the binarized image acquisition unit 410 is a floating cell that is floating in the culture solution without sticking to the culture vessel in the image data in which the region of the cell candidate is captured in black compared to other regions. When a system cell is imaged, the image data is binarized using two threshold values, and binarized image data is generated.
For example, the binarized image acquisition unit 410 uses the pixel values 80 and 149 as the two threshold values, and sets the region having the pixel value of 80 or more and less than 149 as the cell candidate region, and the region having the pixel value of less than 80 or 149 or more as the cell candidate. Other areas. The binarized image acquisition unit 410 acquires information indicating whether the cell imaged in the image data is an adhesive cell or a floating cell from the incubator 511, and determines the threshold value. May be.
The binarized image acquisition unit 410 that binarizes the image data outputs the generated binarized image data to the cell object extraction unit 420.

  The extraction unit 411 extracts, from the binarized image data input from the binarized image acquisition unit 410, a cell candidate region that has an area of 1 pixel or more as a cell candidate object that is a cell object candidate. To do. The extraction unit 411 outputs information indicating the extracted cell candidate object to the area calculation unit 422.

The area calculation unit 422 calculates the area of each cell candidate object from the input information indicating the cell candidate object, and outputs information indicating the calculated area to the cumulative frequency distribution calculation unit 423.
The cumulative frequency distribution calculation unit 423 calculates a cumulative frequency distribution obtained by sequentially integrating the areas of the cell candidate objects in order of the size of the area from the information indicating the input area, and calculates the calculated cumulative frequency Distribution information is output to the differential function calculation unit 424.

  The cumulative frequency distribution calculation unit 423 obtains the cumulative area calculated at the end as the overall area value, and sequentially divides each cumulative area by the overall area value and multiplyes by 100, starting from the object with the smallest area. The accumulated frequency distribution may be calculated. In that case, the cumulative frequency distribution calculation unit 423 outputs information of the calculated cumulative frequency distribution to the differential function calculation unit 424.

  Returning to FIG. 31, the differential function calculation unit 424 calculates a differential function of the cumulative frequency distribution by differentiating the cumulative frequency distribution with respect to the area using the input information of the cumulative frequency distribution. That is, the differential function calculation unit 424 calculates a differential function of the cumulative frequency distribution indicating the increase / decrease rate of the cumulative area with respect to a change in the size of the area using the size of the area as a variable from the cumulative frequency distribution. The differential function calculation unit 424 outputs information on the calculated differential function of the cumulative frequency distribution to the specific threshold value calculation unit 425.

  The differential function calculation unit 424 may calculate a cumulative frequency increase / decrease distribution indicating a rate of increase / decrease of the cumulative frequency with respect to a change in the size of the area of the cell candidate object. Specifically, for example, the differential function calculation unit 424 calculates a differential value with respect to the area of the cumulative frequency by differentiating the cumulative frequency shown in FIG. Thereby, the differential function calculation unit 424 can obtain a cumulative frequency differential function as shown in FIG.

The inherent threshold value calculation unit 425 calculates an area e that takes a minimum value in the cumulative frequency differential function calculated by the differential function calculation unit 424.
Specifically, for example, the unique threshold value calculation unit 425 calculates an area e that takes a minimum value within a predetermined range according to the cell from the differential function of the cumulative frequency shown in FIG.

  In order to calculate a specific threshold value according to the culture time and the cell type, the specific threshold value calculation unit 425 performs a specific correction on the specific correction information β corresponding to the information indicating the culture time input from the outside and the cell type information. Read from the table T1 of the information storage unit 450. The unique threshold value calculation unit 425 calculates a unique threshold value (e + β) that is a value obtained by adding the area e having the minimum value and the unique correction information β, and information indicating the calculated unique threshold value (e + β) is the cell object selection unit 426. Output to.

Here, the calculation method of the unique threshold value (custom e value) by the unique threshold value calculation unit 425 will be described using a specific example. The cell object selection unit 426 uses the information indicating the input unique threshold value (e + β) to select a cell candidate object having an area equal to or larger than the unique threshold value (e + β) as the cell object from the binarized image data, and selects the selected cell object. Information indicating the cell object and the binarized image data acquired from the binarized image acquisition unit 410 are output to the outside.
In addition, the cell object selection unit 426 outputs information indicating the cell object to the cell counting unit 442 (to be described later) of the specific correction information generation unit 440 together with the specific correction information β and the information indicating the culture time.

  FIG. 32 is a flowchart showing a flow of processing for extracting a cell object from image data captured at a certain culture time in a certain cell type in the image processing apparatus according to the fifth embodiment. First, the binarized image acquisition unit 410 generates binarized image data from the input image data (step S601). Next, the extraction unit 411 extracts a cell candidate object from the binarized image data (step S602). Next, the area calculation unit 422 calculates the area of each cell candidate object (step S603). Next, the cumulative frequency distribution calculation unit 423 calculates a cumulative frequency distribution related to the area of the cell candidate object (step S604).

  Next, the differential function calculation unit 424 calculates a cumulative frequency differential function (step S605). Next, the specific threshold value calculation unit 425 calculates an area e that takes a minimum value in the cumulative frequency differential function (step S606). Next, the unique threshold value calculation unit 425 reads the unique correction information β corresponding to the information indicating the cell type input from the outside (step S607).

  Next, the specific threshold value calculation unit 425 calculates a specific threshold value (e + β) obtained by adding the area e having the minimum value and the specific correction information β (step S608). Next, the cell object selection unit 426 selects a cell candidate object having an area equal to or larger than the unique threshold (e + β) as a cell object (step S609). Above, the process of this flowchart is complete | finished.

As a result, the image processing apparatus 401 can use an appropriate inherent threshold value (e + β) according to the cell type and the culture time, and thus can accurately extract the cell object. Further, the image processing apparatus 401 can use the inherent threshold value (e + β) corresponding to the cell type even if the cell area is different among the cell types, and thus can accurately extract the cell object.
Further, the image processing apparatus 401 can use the inherent threshold value (e + β) corresponding to the culture time even if the area of the cell changes according to the culture time, so that the cell object can be extracted with high accuracy.

FIG. 33 shows the error ratio ER when the cell object is extracted with the area where the minimum value is simply taken as the threshold (e in FIG. 24) and when the cell object is extracted with the unique threshold as the threshold (e + β in FIG. 24). It is the table compared. The figure shows the number of visual recognitions n ₀ when using image data in three culture times (8 hours, 12 hours, and 16 hours) of a certain cell, and the number of recognized cells n when the threshold value is e. In addition, the error rate ER in that case is compared with the number n of cells recognized when the threshold is e + β and the error rate ER in that case.

  When image data when the culture time is 8 hours is used, when the area e that takes the minimum value is used as a threshold, the recognition number n is 54 [pieces] and the error rate ER is 60.3 [%]. On the other hand, when the unique threshold value (e + β) in the fifth embodiment is set as the threshold value, the number n of recognition is 136 [pieces] and the error rate ER is 0.0 [%]. The accuracy is significantly improved. Similarly, in the case of using image data when the culture time is 12 hours and 16 hours, the extraction accuracy of the cell object is improved when the intrinsic threshold value (e + β) in the fifth embodiment is used as the threshold value. doing.

  As described above, the image processing apparatus 401 according to the fifth embodiment can extract a cell object with higher accuracy than extracting a cell object using an area having a minimum value in a differential function of a cumulative frequency distribution function as a threshold value. .

  The cumulative frequency distribution function calculated by the cumulative frequency distribution calculating unit 423 is an example, and the cumulative frequency distribution function may be calculated as in the following example. The cumulative frequency distribution calculation unit 423 calculates the corresponding number of classified cell candidate objects for each size of the area of each cell candidate object. The cumulative frequency distribution calculation unit 423 calculates a product value of the corresponding number and the area value based on the calculated corresponding number for each area size and the area value, and calculates the calculated product value as the area size. The cumulative frequency distribution function obtained by sequentially integrating from the above order may be calculated.

<Data construction method of unique correction information storage unit>
Next, returning to FIG. 29, a method for calculating the unique correction information β stored in the unique correction information storage unit 450 will be described. In the case where the unique correction information β is calculated, it is assumed that information indicating the visual recognition number n ₀ that is the number of cells actually measured from the image data is input to the image data processing unit 1. The cell object extraction unit 420 extracts the cell object with the threshold e + β ′ while shifting the threshold shift amount β ′ within a predetermined range at a certain interval instead of the inherent correction information β with the threshold e + β. Specifically, the cell object extraction unit 420 extracts the cell object with the threshold e + β ′ while shifting the threshold shift amount β ′ in the range of −100 to 100 and the threshold shift amount β ′ with a predetermined shift interval. To do.

As an example, the cell object extraction unit 420 increases the threshold shift amount β ′ in the range from −100 to 100, and increases the threshold shift amount β ′ from −100 to 2 in order by 2 every time, with the threshold e + β ′. To extract.
The cell object extraction unit 420 uses information indicating the threshold shift amount β ′, information indicating the culture time t, information indicating the cell type i, and information o indicating the cell object, which will be described later in the specific correction information generation unit 440. Output to the unit 442.

The unique correction information generation unit 440 generates data in the table T1 stored in the unique correction information storage unit 450. The unique correction information generation unit 440 includes a cell counting unit 442, an error calculation unit 444, an error storage unit 446, and a correction information setting unit 448.
The cell counting unit 442 counts the cell objects extracted by the cell object extracting unit 420. The cell counting unit 442 includes information indicating the number of recognitions n, which is the number of counted cell objects, together with information indicating the threshold shift amount β ′, information indicating the culture time t, and information indicating the cell type i, an error calculating unit. To 444.

Error calculation unit 444 receives information indicating the viewing number n ₀ is the number of cells measured from the image data from the outside. Each time the cell counting unit 442 counts the recognition number n according to β ′, the error calculation unit 444 includes information indicating the recognition number n input from the cell counting unit 442 and information indicating the visual recognition number n _0. Based on the above equation (1), an error ratio ER representing the ratio of the difference between the recognition number n and the visual recognition number n ₀ is calculated.

  Subsequently, regarding the processing of the error calculation unit 444, first, one cell type will be described. The error calculation unit 444 stores the information indicating the calculated error ratio ER in the table T2 for each cell type in the error storage unit 446 in association with the information indicating the threshold shift amount β ′ and the information indicating the culture time. .

  The error calculation unit 444 determines whether or not the error ratio ER has been calculated for all threshold shift amounts β ′. If the error calculation unit 444 determines that the error ratio ER is not calculated for all the threshold shift amounts β ′, the error calculation unit 444 causes the inherent threshold calculation unit 425 of the cell object extraction unit 420 to increase the threshold shift amount β ′. Control as follows.

The error calculation unit 444 performs the above-described series of processing on the number (recognition number) of cell objects extracted based on all threshold shift amounts β ′. Thereby, the error calculation unit 444 can calculate the error ratio ER with all the threshold shift amounts β ′ during the culture time, and can store the error ratio ER in the error storage unit 446.
Further, the error calculation unit 444 performs the same processing as described above at different culture times. As a result, the error calculation unit 444 can calculate the error rate ER for each culture time, and store the calculated error rate ER in the error storage unit 446.

By the processing described above, the error storage unit 446 stores a table in which the error ratio ER is associated with information indicating the threshold shift amount β ′ and information indicating the culture time.
FIG. 34 is a diagram showing an example of a table T2 in which the error rate ER of a predetermined cell type is stored in association with the threshold shift amount β ′ and the culture time t. In the figure, the error ratio ER [%] is determined according to the shift amount of the threshold and the culture time t. The threshold shift amount β ′ is from −100 to 100 in increments of 2. The culture time is in increments of 4 hours from 8 hours after 0 hours when the cells are seeded.

  Returning to FIG. 29, for all the cell types, after all the error ratios ER calculated by the error calculation unit 444 based on the information indicating the threshold shift amount β ′ are stored in the table T2 of the error storage unit 446, The correction information setting unit 448 extracts, from the table T2 for each cell type in the error storage unit 446, a threshold shift amount β ′ in which the error ratio ER is within a predetermined range for each culture time. Specifically, for example, the correction information setting unit 448 extracts the threshold shift amount β ′ in which the error ratio ER is in the range of 0 [%] to 20 [%].

  As an example, the correction information setting unit 448 selects the median value of the threshold shift amount β ′ extracted for each culture time as the specific correction information β of the cell type during the culture time. The correction information setting unit 448 stores the selected unique correction information β in the table T1 of the unique correction information storage unit 450 in association with the cell type information and the information indicating the culture time.

  For example, in FIG. 34, when the culture time is 8 hours, the threshold shift amount β ′ when the error rate ER is in the range of 0 [%] to 20 [%] is −6, −4, and −2. Therefore, the median value −2 is the unique correction information β.

  The correction information setting unit 448 selects the median value of the threshold shift amount as the specific correction information. However, the correction information setting unit 448 selects any value as the specific correction information β as long as the extracted threshold shift amount is not limited thereto. Also good.

  The cell object extraction unit 420, the cell counting unit 442, the error calculation unit 444, and the correction information setting unit 448 perform the same processing as described above on the image data of different cell types. Thereby, the specific correction information β can be generated for each cell type.

  FIG. 35 is a flowchart showing a flow of processing for calculating specific correction information β of a certain cell type in a predetermined culture time in the image processing apparatus. The processing from step S701 to step S706 is the same as the processing from step S601 to step S606 in FIG.

  Next, the unique threshold value calculation unit 425 initializes the threshold value shift amount β ′, and sets the threshold value shift amount β ′ to −100 as an example (step S707). Next, the unique threshold value calculation unit 425 calculates a unique threshold value (custom e value) that is the sum of the threshold shift amount β ′ and the area e that takes the minimum value (step S708). Next, the cell object selection unit 426 selects a cell candidate object having an area equal to or larger than the inherent threshold value (custom e value) as a cell object (step S709).

Next, the cell counting unit 442 calculates a recognition number n, which is the number of selected cell objects (step S710). Next, the error calculation unit 444, based on the information indicating the viewing number n ₀ which is input from the information and the external indicating a recognition number n, and calculates the ratio ER of the error, the information indicating a ratio ER of the calculated error The error is stored in the error storage unit 446 (step S711). Next, the error calculation unit 444 determines whether or not the error ratio ER has been calculated with all threshold shift amounts β ′ (step S712). When the error calculation unit 444 determines that the error ratio ER is not calculated for all threshold shift amounts β ′ (NO in step S712), the unique threshold calculation unit 425 adds the threshold shift amount β ′ by two. (Step S713), and the process returns to Step S708.

  On the other hand, when it is determined that the error calculation unit 444 has calculated the error ratio ER with all the threshold shift amounts β ′ (YES in step S712), the correction information setting unit 448 determines that the input image data has been acquired. The unique correction information β of the cell type in time is calculated (step S714). Next, the correction information setting unit 448 stores the calculated unique correction information β in the table T1 of the unique correction information storage unit 450 in association with information indicating the cell type and information indicating the culture time (step S715). Above, the process of this flowchart is complete | finished.

As a result, the unique correction information storage unit 450 of the image processing apparatus 401 stores the proper correction information β according to the cell type and the culture time, and thus the image processing apparatus 401 uses the threshold value. The cell object can be extracted with high accuracy.
Above, description about the calculation method of the specific correction information (beta) memorize | stored in the specific correction information storage part 450 is complete | finished.

<Sixth Embodiment: Image Processing Device>
Next, a sixth embodiment of the present invention will be described. The image processing apparatus 401 in the fifth embodiment extracts a cell object from image data when the cell type is known. The image processing apparatus 401b in the sixth embodiment extracts a cell object from image data regardless of whether the cell type is unknown or known. For this reason, the image processing apparatus 401b according to the sixth embodiment of the invention performs the processing separately when the information indicating the cell type is input and when the information is not input.

FIG. 36 is a block diagram of an image processing apparatus 401b according to the sixth embodiment of the present invention. In addition, the same code | symbol is attached | subjected to the element which is common in FIG. 1, and the description is abbreviate | omitted.
The configuration of the image processing device 401b in FIG. 36 is different from the configuration of the image processing device 401 in FIG. 29 in that the cell object extraction unit 420 is the cell object extraction unit 420b and the unique correction information generation unit 440 is the unique correction information generation unit 440b. The common correction information storage unit 452 is added.

In the common correction information storage unit 452, common correction information γ, which is correction information common among cell types associated with the culture time, is stored as a table T3.
FIG. 37 is an example of a table T3 in which the culture time stored in the common correction information storage unit is associated with the common correction information γ. In the figure, in the table T3, the culture time and the common correction information γ are associated one-to-one.

<Cell object extraction method>
FIG. 38 is a block configuration diagram of the cell object extraction unit 420b according to the sixth embodiment of the present invention. Details of the change of the cell object extraction unit 420b will be described below. The configuration of the cell object extraction unit 420b is different from the configuration of the cell object extraction unit 420 of FIG. 31 in that the differential function calculation unit 424 is changed to a differential function calculation unit 424b, and the unique threshold value calculation unit 425 is a unique threshold value calculation unit 425b. The determination unit 427 and the cell type extraction unit 430 are added.

The differential function calculation unit 424b calculates a cumulative frequency differential function, and outputs information on the calculated cumulative frequency differential function to the determination unit 427.
The determination unit 427 determines that the cell type is known when information indicating the cell type is input from the outside. On the other hand, when the information indicating the cell type is not input from the outside, the determination unit 427 determines that the cell type is unknown, and uses the cumulative frequency differential function information input from the determination unit 427 as described later in the cell type extraction unit 430. To the temporary threshold setting unit 431.

  The cell type extraction unit 430 extracts, as a sample object, a cell candidate object having an area equal to or larger than the temporary threshold value by using a value obtained by adding the area e having the minimum value in the cumulative frequency differential function and the common correction information γ as a temporary threshold value. The information indicating the cell type is estimated from the extracted sample object. Hereinafter, details of the cell type extraction unit 430 will be described. The cell type extraction unit 430 includes a temporary threshold setting unit 431, a sample object selection unit 432, and a cell type estimation unit 433.

  The temporary threshold setting unit 431 reads the common correction information γ corresponding to the information indicating the culture time input from the outside from the common correction information storage unit 452. The temporary threshold setting unit 431 calculates an area e where the cumulative frequency differential function takes a minimum value using the input information of the cumulative frequency differential function. The temporary threshold setting unit 431 calculates a temporary threshold obtained by adding the common correction information γ to the area e where the calculated minimum value is taken, and outputs the calculated temporary threshold to the sample object selection unit 432.

The sample object selection unit 432 selects a cell candidate object having an area equal to or larger than the temporary threshold as a sample object, and outputs information indicating the sample object to the cell type estimation unit 433.
The cell type estimation unit 433 uses the information indicating the sample object to estimate the cell type by a method described later, and outputs the information indicating the cell type to the unique threshold value calculation unit 425b.

  When the determination unit 427 determines that the cell type is known or when receiving information indicating the cell type from the cell type estimation unit 433, the unique threshold value calculation unit 425b is input with information indicating the cell type from the outside The unique correction information β corresponding to the information indicating the culture time is read from the table T2 of the unique correction information storage unit 450. The inherent threshold value calculation unit 425b calculates an area e where the cumulative frequency differential function takes a minimum value. The unique threshold value calculation unit 425b calculates a unique threshold value (e + β) obtained by adding the unique correction information β to the area e where the calculated cumulative frequency differential function takes the minimum value, and the calculated unique threshold value (e + β) is the cell object selection unit 426. Output to.

<Cell type estimation method by cell type estimation unit>
FIG. 39 is a block configuration diagram of a cell type estimation unit in the sixth embodiment of the present invention. The cell type estimation unit 433 includes a cell object feature amount calculation unit 460, a classification model storage unit 462, and a cell classification unit 464.

  The cell object feature amount calculation unit 460 recognizes each cell object using the binarized image data acquired from the binarized image acquisition unit and the information indicating the cell object acquired from the sample object selection unit 432, The morphological feature amount of each recognized cell object is calculated. As an example, the cell object feature value calculation unit 460 calculates the 15 types of morphological feature values shown in FIG. 3 for each cell object.

For example, the cell object feature amount calculation unit 460 can calculate the value of “Total area” based on the number of pixels in the cell area of interest.
Note that the cell object feature amount calculation unit 460 may detect a group of pixels whose luminance value in the cell is equal to or greater than the threshold value as a Hole, and calculate the value of “Hole area” based on the number of pixels of the Hole.

In addition, the cell object feature amount calculation unit 460 can acquire the value of “Perimeter” by contour tracking processing when extracting cells.
In addition, the cell object feature amount calculation unit 460 calculates the value of “Fiber Length” using Expression (1).
Further, the cell object feature amount calculation unit 460 calculates the value of “Fiber Breath” according to the equation (2).
In addition, the cell object feature amount calculation unit 460 calculates the value of “Shape Factor” using Expression (3).

  Here, the cell object feature value calculation unit 460 may calculate each of the morphological feature values by adding an error to the number of pixels corresponding to the cell. At this time, the cell object feature amount calculation unit 460 may calculate the morphological feature amount in consideration of the imaging condition of the microscope image (observation magnification, aberration of the optical system of the microscope, etc.). When calculating “Inner radius”, “Outer radius”, “Mean radius”, and “Equivalent radius”, the cell object feature amount calculation unit 460 calculates the center of gravity of each cell based on a known center of gravity calculation method. What is necessary is just to obtain | require and calculate each parameter on the basis of this barycentric point.

  Returning to FIG. 39, the cell object feature amount calculation unit 460 that has calculated the morphological feature amount of each cell object outputs information indicating the morphological feature amount of each cell object to the cell classification unit 464.

In the classification model storage unit 462, a classification model that classifies the attributes of the cell object in stages according to a predetermined classification criterion is stored in association with an evaluation value indicating the evaluation of the classification model. Further, information indicating the classification destination ID, which is unique identification information (ID) of the classified class, and information indicating the cell type are stored in association with each other.
FIG. 40 is an example of information stored in the classification model storage unit 462. In the figure, in the table T4, the classification model using the decision tree and the evaluation value indicating the evaluation of the classification model are associated one-to-one.

Returning to FIG. 39, the cell classification unit 464 reads out the classification model having the highest evaluation value from the classification model storage unit 462, and uses the classification model to classify information indicating individual input morphological feature amounts. Classify.
Specifically, for example, the cell classification unit 464 calculates the average of each morphological feature amount of the cell object, and compares the condition determined for each branch in the classification tree with the calculated average of each morphological feature amount. By doing so, it is decided which branch to branch at that branch. The cell classification unit 464 extracts information indicating the classification destination ID associated with the finally classified branch. The cell classification unit 464 reads information indicating the cell type corresponding to the information indicating the extracted classification destination ID from the classification model storage unit 462, and outputs the information indicating the read cell type to the unique threshold value calculation unit 425b.

  FIG. 41 is a flowchart showing a flow of processing for extracting a cell object from image data captured at a certain culture time in a certain cell type in the image processing apparatus 401b according to the sixth embodiment. Since the processing from step S801 to step S805 is the same as the processing from step S601 to step S605 in FIG. 32, the description thereof is omitted.

  Next, the determination unit 427 determines whether or not information indicating a cell type has been input (step S806). When the information indicating the cell type is input (NO in step S806), the determination unit 427 determines that the cell type is known, and proceeds to the process of step S810. On the other hand, when information indicating the cell type is input (YES in step S806), the determination unit 427 determines that the cell type is unknown, and outputs information on the cumulative frequency differential function to the temporary threshold setting unit 431.

  Next, the temporary threshold setting unit 431 calculates a temporary threshold (step S807). Next, the sample object selection unit 432 selects a cell candidate object having an area equal to or larger than the temporary threshold from the cell candidate objects as a sample object (step S808). Next, the cell type estimation unit 433 estimates the cell type of the sample object from the information indicating the sample object (step S809).

  Next, the unique threshold value calculation unit 425b calculates a unique threshold value (e + β) based on the information indicating the cell type (step S810). Next, the cell object selection unit 426 selects a cell candidate object having an area equal to or larger than the specific threshold (e + β) from the cell candidate objects as a cell object (step S811). Above, the process of this flowchart is complete | finished.

  As described above, even if image data whose cell type is unknown is input to the image processing device 401b, after estimating the cell type and estimating the cell type, the image processing apparatus 401b uses the inherent threshold (e + β) specific to the cell type to Since the object is extracted, the cell object can be accurately extracted even when the cell type is unknown.

  Regardless of whether the cell type is unknown or known, the image processing apparatus 401b can extract many and accurate cell objects in the image data from the image data. Further, the image processing apparatus 401b can prevent erroneous recognition of noise included in the image data as a cell object regardless of whether the cell type is unknown or known.

<Data Construction Method of Unique Correction Information Storage Unit and Common Correction Information Storage Unit>
Returning to FIG. 36, the details of the change of the specific correction information generation unit 440b will be described. The configuration of the specific correction information generation unit 440b is different from the configuration of the specific correction information generation unit 440 in FIG. Are changed to an error calculation unit 444b, a recognition number storage unit 46 is changed to a recognition number storage unit 446b, and a correction information setting unit 448 is changed to a correction information setting unit 448b.

  Each time the cell object extraction unit 420b shifts the threshold shift amount β ′ for each cell type from an initial value (for example, −100) to a final value (for example, 100) with a predetermined step size (for example, 2), A cell object is extracted using a value e + β ′ obtained by adding a threshold shift amount β ′ to the minimum value e as a threshold value. The cell object extraction unit 420b outputs information indicating the extracted cell object o to the cell counting unit 442 together with information indicating the cell type i, information indicating the culture time t, and information indicating the threshold shift amount β ′.

  Based on the information indicating the cell object, the cell counting unit 442 counts the recognition number n, which is the number of the cell object, and displays the information indicating the cell type i, the information indicating the culture time t, and the threshold shift amount β ′. The information indicating the number of recognition n is output to the error calculation unit 444b together with the information indicating.

Subsequently, regarding the processing of the error calculation unit 444b, first, one cell type will be described. The error calculation unit 444b applies the information indicating the recognition number n input from the cell counting unit 442 to the equation (1) to calculate the error ratio ER.
The error calculation unit 444b associates the information indicating the error ratio ER with the information indicating the threshold shift amount β ′ and the information indicating the culture time, and stores a table (for example, a table) for each cell type in the recognition number storage unit 446b. Stored in T2a). The error calculation unit 444b performs the above process on all cell types.

  FIG. 42 is a diagram illustrating an example of a table of error ratios ER stored in the recognition number storage unit. In table T2a in FIG. 42 (a), table T2b in FIG. 42 (b), and table T2c in FIG. 42 (c), the error rate ER [%] in each different cell type indicates the culture time and the threshold shift amount. Associated with. The culture time [h] is shown in increments of 4 hours from 8 hours after the time when the cells were seeded as 0 hour. The threshold shift amount β ′ is a number from −100 to 100 in increments of 2.

  Returning to FIG. 36, the number-of-recognition storage unit 446b stores, for each cell type, information indicating the error ratio ER in relation to information indicating the threshold shift amount β ′ and information indicating the culture time. Yes. Specifically, for example, as shown in FIG. 42, the recognition number storage unit 446b is associated with information indicating the threshold shift amount β ′ and information indicating the culture time for each of the three cell types. A table of information indicating the error ratio ER is stored.

  When all the data of a table of a certain cell type stored in the recognition number storage unit 446b is stored in the correction information setting unit 448b, the error ratio ER for each culture time in the table is, for example, from 0 [%]. The threshold shift amount β ′ corresponding to 20 [%] is extracted, and the median value of the extracted threshold shift amounts β ′ is calculated. The correction information setting unit 448b stores the calculated median value in the specific correction information storage unit 450 as the specific correction information β in association with the cell type and the culture time.

  The correction information setting unit 448b sets the median value of the extracted threshold shift amount β ′ as the inherent correction information β, but is not limited thereto, and any one of the extracted threshold shift amounts β ′ is used. The inherent correction information β may be set. Further, the correction information setting unit 448b may use the average value of the extracted threshold shift amounts β ′ as the specific correction information β.

  The correction information setting unit 448b determines whether or not the error rate ER of a certain culture time has been stored in the table of all n cell types (n is a positive integer) stored in the recognition number storage unit 446b. .

  In the table of all M cell types (M is a positive integer) stored in the recognition number storage unit 446b, when the error rate ER of a certain culture time is stored, the correction information setting unit 448b stores the recognition number storage. A plurality of threshold shift amounts (hereinafter referred to as a threshold shift amount group) are extracted from the unit 446b. Specifically, the correction information setting unit 448b uses the culture time in the jth table corresponding to the jth cell type (j is an integer from 1 to M) stored in the recognition number storage unit 446b. A threshold shift amount group in which the error ratio ER is, for example, 10 [%] to 30 [%] is extracted every time.

  The correction information setting unit 448b extracts a threshold shift amount group common to all the shift amount groups among the n threshold shift amount groups extracted from the first to nth tables for each culture time. To do. The correction information setting unit 448b calculates the median value of the extracted common threshold shift amount group, and stores the calculated median value in the common correction information storage unit 452 as the common correction information γ in association with the culture time.

  The correction information setting unit 448b can calculate the common correction information γ for all the culture times by performing the above processing on the error ratio ER of all the culture times.

  The correction information setting unit 448b sets the common value of the common threshold shift amount group β ′ as the common correction information γ, but is not limited thereto, and any value of the common threshold shift amount β ′ is used. May be set in the common correction information γ. The correction information setting unit 448b may use the average value of the common threshold shift amount group β ′ as the common correction information γ.

  FIG. 43 is a flowchart showing a flow of processing for calculating specific correction information β and common correction information γ of a certain cell type in a predetermined culture time in the image processing apparatus 401b according to the sixth embodiment. First, the image processing apparatus 401b calculates unique correction information β, and stores the calculated unique correction information β in the unique correction information storage unit 450 in association with the cell type and the culture time (step S901). Details of the processing in step S901 are the same as those in steps S701 to S715 in FIG. 35, and thus detailed description of the processing is omitted.

  Next, the correction information setting unit 448b stores whether or not the error rate ER of a predetermined culture time is stored in the table of all M cell types (M is a positive integer) stored in the recognition number storage unit 446b. (Step S902). When all the error rates ER of the predetermined culture time are not stored in the recognized number storage unit 446b (NO in step S902), the image processing device 401b performs the process of step S901 on the cell types that are not stored. .

  On the other hand, when all the culture time error ratios ER stored in the recognized number storage unit 446b are stored (YES in step S902), the correction information setting unit 448b calculates common correction information γ (step S903). Next, the correction information setting unit 448b stores the calculated common correction information γ in the common correction information storage unit 452 in association with the culture time. Above, the process of this flowchart is complete | finished.

As described above, the image processing apparatus 401b can store the unique correction information β in the unique correction information storage unit 450 and can store the common correction information γ in the common correction information storage unit 452. Thereby, the image processing apparatus 401b can estimate the cell type from the image data and the information indicating the culture time using the common correction information γ common to the cell type.
The image processing apparatus 401b can accurately extract a cell object from the image data, the information indicating the culture time, and the information indicating the cell type by using the inherent correction information β specific to the estimated cell type.

<Seventh Embodiment: Cell Sorting Apparatus>
Then, the cell classification device in the 7th Embodiment of this invention is demonstrated. FIG. 44 is a block diagram of a cell classification device according to the seventh embodiment of the present invention. Cell sorting apparatus 404 estimates the image data input from the outside, and information indicating the type of cell, and information indicating the incubation time, based on the information indicating the viewing number n _0, the information of the cell, the estimated Output the information of the completed cells to the outside.

The cell classification device 404 includes an image processing device 401b and a cell information estimation unit 433b. The image processing apparatus 401b extracts an image data inputted from outside, and information indicating the type of cell, and information indicating the incubation time, based on the information indicating the viewing number n _0, the cell object, cell object Along with the binarized image data calculated at the time of extraction, information indicating the extracted cell object is output to the cell type estimation unit 433b. Since the image processing apparatus 401b is the same as that in FIG. 36, a detailed description of the processing contents is omitted.

  The cell information estimation unit 433b classifies the cell object based on the binarized image data input from the image processing device 401b and information indicating the image object, reads out and reads out the cell information corresponding to the classification destination ID. Cell information is output to the outside. Similarly to the cell type estimation unit 433 in FIG. 39, the cell type estimation unit 433b includes a cell object feature amount calculation unit 460b, a classification model storage unit 462b, and a cell classification unit 464b.

  Similar to the cell object feature value calculation unit 460 of FIG. 39, the cell object feature value calculation unit 460b recognizes and recognizes each cell object using the input binary image data and information indicating the image object. The morphological feature amount of each cell object is calculated. As an example, the cell object feature amount calculation unit 460b calculates the 15 types of morphological feature amounts described above for each cell object. The cell object feature amount calculation unit 460b outputs information indicating the calculated morphological feature amount of each cell object to the cell classification unit 464b.

  In the classification model storage unit 462b, as in the case of the classification model storage unit 462 of FIG. 39, information indicating a classification model that classifies the attributes of the cell object step by step according to a predetermined classification criterion is an evaluation indicating the evaluation of the classification model. It is stored in association with information indicating a value. Further, information indicating a classification destination ID that is an ID of a classified class and cell information (for example, information indicating cell activity or information indicating cell quality) are stored in association with each other.

The cell classification unit 464b reads out the classification model having the highest evaluation value from the classification model storage unit 462b, and classifies information indicating the morphological feature amount of each input cell object using the classification model.
Specifically, for example, the cell classification unit 464b calculates the average of each morphological feature amount of the cell object, and compares the condition determined for each branch in the classification tree with the calculated average of each morphological feature amount. Depending on the situation, it is decided which branch to branch at that branch. The cell classification unit 464b reads information indicating the classification destination ID set in the finally classified branch.

  The cell classification unit 464b reads out cell information associated with the information indicating the read classification destination ID from the classification model storage unit 462b, and outputs the read cell information to the outside. For example, the cell classification unit 464b reads information indicating the activity of the cell or information indicating the quality of the cell associated with the information indicating the classification destination ID from the classification model storage unit 462b, and information indicating the read cell activity or Outputs information indicating cell quality to the outside.

  As described above, since the cell classification device 404 can extract many and accurate cell objects in the image data from the image data regardless of whether the cell type is unknown or known, the cell classification device 404 exists in one image data. Many cell objects can be classified. In addition, the cell classification device 404 can prevent erroneous recognition of noise included in image data as a cell object regardless of whether the cell type is unknown or known. From the above, the cell classification device 404 can classify the cells captured in the image data with high accuracy, and therefore can estimate the cell activity or the cell quality with high accuracy.

  In addition, when classifying image data with unknown attribute information, the cell classification unit 464b may output attribute information including culture conditions to the outside as a classification result. Specifically, when classifying individual characteristic search objects obtained from image data whose attribute information is unknown, the cell classification unit 464b sets at least the branch destination of the individual characteristic search objects. The classified classification ID is output to the outside. For example, in addition to the classification destination ID of each characteristic search object, the cell classification unit 464b can select either attribute information associated with each classification destination ID or a morphological feature amount of each characteristic search object. Either one or both may be output to the outside.

  In addition, the image processing device 401b in the cell classification device 404 in the seventh embodiment of the present invention may be replaced with the image processing device 401 in the fifth embodiment. In that case, the cell classification device 404 can classify the imaged cells from image data with known cell types into image data.

<Eighth Embodiment: Incubator>
Next, an incubator according to the eighth embodiment of the present invention will be described. FIG. 45 is a block configuration diagram showing an outline of an incubator having the function of the cell classification device 404 shown in the seventh embodiment.

  The incubator 511 includes an upper casing 512, a lower casing 513, and a cell sorting device 404. In the assembled state of the incubator 511, the upper casing 512 is placed on the lower casing 513. Note that the internal space between the upper casing 512 and the lower casing 513 is vertically divided by a base plate 514.

  First, an outline of the configuration of the upper casing 512 will be described. In FIG. 45, a temperature-controlled room 515 for culturing cells is formed inside the upper casing 512. The temperature-controlled room 515 includes a temperature adjusting device 515a and a humidity adjusting device 515b. The temperature-controlled room 515 is maintained in an environment suitable for cell culture (for example, an atmosphere having a temperature of 37 ° C. and a humidity of 90%).

  The culture medium exchange device 523 is disposed in the temperature-controlled room 515, and the culture medium exchange device 523 is a culture container in which cells are cultured based on a control signal input from a control device (culture condition control unit) 541 described later. Change the medium.

  The observation unit 522 is disposed in the temperature-controlled room 515, and the observation unit 522 can execute time-lapse observation of cells in the culture vessel 519. Here, the observation unit 522 is disposed by being fitted into the opening of the base plate 514 of the upper casing 512.

  The observation unit 522 has a built-in LED light source 335. And the imaging device 534 can acquire the microscope image of a cell by imaging the cell of the culture container permeate | transmitted and illuminated from the upper side of the sample stand through the optical system of a microscope. The imaging device 534 converts the acquired microscope image into image data, and outputs the converted image data to the cell classification device 404.

  The cell classification device 404 classifies the cells imaged in the image data based on the image data input from the imaging device 534, acquires the classification destination ID set as the classification destination, and sets the acquired classification destination ID. Read the corresponding cell information. The cell classification device 404 outputs the read cell information to the control device (culture condition control unit) 541.

  The control device (culture condition control unit) 541 is connected to the temperature adjustment device 515a, the humidity adjustment device 515b, the observation unit 522, and the culture medium exchange device 523, respectively. The control device 541 comprehensively controls each unit of the incubator 511 according to a predetermined program.

  The control device (culture condition control unit) 541 includes a CPU 542 and a storage unit 543, and changes the environmental conditions of the temperature-controlled room based on the cell information input from the cell classification device 404. Specifically, for example, the control device (culture condition control unit) 541 adjusts the temperature of the temperature-controlled room by controlling the temperature adjustment device 515a based on the cell information input from the cell classification device 404, or Alternatively, the humidity adjusting device 515b is controlled to adjust the humidity of the temperature-controlled room. Thereby, the control apparatus (culture condition control part) 541 can maintain the inside of the temperature-controlled room 515 to a predetermined environmental condition.

  In addition, the control device (culture condition control unit) 541 changes the medium in which the cells are cultured based on the cell information estimated by the cell classification device 404. Specifically, for example, the control device (culture condition control unit) 541 controls the medium replacement device 523 to replace the medium in the culture vessel based on the cell information input from the cell classification device 404. Thereby, the control apparatus (culture condition control part) 541 can maintain the culture medium in a culture container in a predetermined state.

  The storage unit 543 is configured by a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 543 stores management data related to each culture vessel 519 stored in the stocker 521 and data of a microscope image captured by the imaging device. Further, the storage unit 543 stores a program executed by the CPU 542.

  FIG. 46 is a flowchart showing an example of the operation of the incubator 511 having the function of the cell classification device 404 shown in the seventh embodiment. At the start of the flowchart shown in FIG. 46, it is assumed that the incubator 511 is provided with a culture container for culturing cells to be proliferated. Further, as described above, the cell classification device 404 included in the incubator 511 outputs attribute information including the culture condition together with the classification destination ID as the classification result.

    In FIG. 46, the incubator 511 starts cell culture (step S1000). That is, the CPU 542 controls the temperature adjustment device 515a and the humidity adjustment device 515b so that the cells are cultured under the set culture conditions (for example, predetermined temperature, humidity, etc.). After a predetermined period of time, the incubator 511 classifies the cells in culture (step S1010). That is, the CPU 542 controls the imaging device 534 to image cells in culture, and then reads out a program for executing each process of the cell classification device 404 from the storage unit 543 and executes the read program. Sort the cells in culture.

    Next, the incubator 511 controls whether or not to stop the cell culture or change the cell culture conditions based on the classification result. Specifically, first, the CPU 542 determines whether or not the cell quality indicated by the classification result matches the target quality set in step S1000 (step S1020). When the CPU 542 determines that the cell quality indicated by the classification result matches the target quality set (YES in step S1020), the cell quality indicated by the classification result matches the target quality set. A message to the effect is displayed on a monitor or the like (not shown) (step S1022). And it progresses to step S1060 mentioned later.

    On the other hand, if the CPU 542 determines that the quality of the cells included in the classification result does not match the predetermined target quality (NO in step S1020), the CPU 542 indicates the cell quality indicated by the classification result and the set target quality. Is displayed on a monitor or the like (not shown) (step S1024), and it is determined whether or not to stop the culture (step S1030). Specifically, for example, the CPU 542 determines that the culture is stopped when the quality of the cell indicated by the classification result and the set target quality do not coincide with each other is very large. For example, when the difference between the value indicating the cell quality indicated by the classification result and the set value indicating the target quality is equal to or greater than a predetermined first threshold, it is determined that the culture is to be stopped. Note that the CPU 542 determines that the culture is to be stopped also when there is an input from the operator to stop the culture.

    If the CPU 542 determines that the culture is to be stopped (YES in step S1030), the CPU 542 displays on the monitor or the like (not shown) that the culture is to be stopped (step S1032) and ends the culture process. Then, the CPU 542 ends the process of this flowchart. On the other hand, when determining that the culture is not stopped (step S1030 NO), the CPU 542 determines whether or not to change the culture conditions (step S1040). Specifically, for example, the CPU 542 determines to change the culture condition when there is a large degree of mismatch between the value indicating the quality of the cell indicated by the classification result and the value indicating the set target quality. For example, the difference between the value indicating the quality of the cell indicated by the classification result and the value indicating the target quality that has been set is greater than or equal to a predetermined second threshold (where the first threshold is greater than the second threshold). In some cases, it is determined that the culture conditions are changed.

    If the CPU 542 determines that the culture conditions are to be changed (YES in step S1040), the CPU 542 displays on the monitor or the like (not shown) that the culture conditions are to be changed (step S1042), and changes the culture conditions (step S1050). That is, the CPU 542 controls the temperature adjustment device 515a and the humidity adjustment device 515b so that the cells are cultured under new culture conditions. Then, the process returns to step S1010. Note that both qualities (cell quality indicated by the classification result and set target quality) and new culture conditions (modified culture conditions) are associated with each other and stored in the storage unit 543 in advance, and the CPU 542 New culture conditions may be acquired from both cell qualities. Note that the CPU 542 may change the culture condition to the set value when the operator inputs a new set value of the culture condition.

    Following step S1022 or when it is determined that the culture conditions are not to be changed (NO in step S1040), the CPU 542 controls the imaging device 534 after a predetermined period of time to image the cells in culture, The number of cells (that is, the number of cell objects) is counted (step S1060). The CPU 542 reads from the storage unit 543 a program for executing the process of the cell object feature amount calculation unit 460b of the cell classification device 404 capable of recognizing individual cell objects, and executes the read program to execute the number of cells. Count.

    Next, the CPU 542 determines whether or not the number of cells counted in step S1060 has reached a predetermined number of cells (step S1070). When the CPU 542 determines that the counted number of cells has not reached the predetermined number of cells (NO in step S1070), the process returns to step S1010 after a predetermined time has elapsed. On the other hand, if the CPU 542 determines that the counted number of cells has reached the predetermined number of cells (YES in step S1070), the CPU 542 displays on the monitor or the like (not shown) that the cell culture has been completed (step S1072). Then, the CPU 542 ends the process of this flowchart.

  As described above, even if the cell type is unknown or known, the image processing device 401b in the cell classification device 404 can accurately extract the cell object from the image data, so that the classification accuracy by the cell classification device 404 is improved. Thereby, the incubator 511 can dynamically change the culture conditions of the cells in the culture container based on the quality of the cells output by the high-precision classification by the cell classification device 404, so that the desired cells have high quality. Can be maintained in a state. The incubator 511 can cultivate cells under appropriate culture conditions according to the state of the cells.

  In this flowchart, the CPU 542 determines whether to stop the culture or change the culture conditions based on whether the cell quality indicated by the classification result matches the target quality. It is not limited to. The CPU 542 may determine whether to continue the culture or stop the culture based on whether or not the cell type indicated by the classification result matches the set cell type.

  Further, the CPU 542 may determine whether to continue the culture by changing the culture condition or to stop the culture based on whether the cell operation indicated by the classification result matches the set cell operation. At this time, if the cell operation indicated by the classification result does not match the set cell operation, the CPU 542 determines whether or not there is any abnormality in the culture process from the log indicating the culture process, and based on the determination result, the culture condition It may be determined whether to continue with the change or to stop the culture. In addition, the CPU 542 causes the storage unit 543 to store information indicating non-identity.

  Further, the image processing device 401b in the cell classification device 404 may be replaced with the image processing device 401. In that case, only when the cell type is known, the incubator 511 can change the culture conditions based on the classification result of the cell classification device 404.

  Further, the image processing device 401 according to the fifth embodiment of the present invention, the image processing device 401b according to the sixth embodiment of the present invention, the cell classification device 404 according to the seventh embodiment of the present invention, and the eighth of the present invention. By recording a program for executing each process of the incubator 511 according to the embodiment on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium, the image processing apparatus 401 The above-described various processes related to the image processing apparatus 401b and the cell classification apparatus 404 may be performed.

  Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

    Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  In all the embodiments of the present invention, the classification model generation device (1, 2), the image processing device (401, 401b), or the cell classification device (3,4, 404) has an area of a morphological feature amount. Although the object related to noise is extracted as a representative example, the present invention is not limited to this, and the object related to noise may be extracted using another morphological feature amount.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

1, 2 ... Classification model generation device
3, 4 ... Cell sorter
10 ... Input / output section
20: Cell object recognition unit
22... Binarization processing unit
24 ... Noise estimation unit (binary data acquisition unit)
26: Cell object feature amount calculation unit (noise removal unit)
30 ... Classification model generation unit
32 ... Characteristic object extraction unit
34 ... Candidate classification model forming unit
36 ... Classification model determination unit
40 ... Classification model generation unit
42 ... Representative feature amount calculation unit
46 ... Classification model forming section
72: Cell object storage unit
74 ... Extraction condition storage unit
76 ... Candidate classification model storage unit
80 ... Classification model storage unit
110 ... Input / output unit
120: Cell object recognition unit
122... Binarization processing unit
124: Noise estimation unit
126 ... Cell object feature amount calculation unit
152 ... Characteristic object extraction / representative feature quantity calculation unit
160 ... cell classification part
170 ... Search object temporary storage unit
174 ... Extraction condition storage unit
180 ... Classification model storage unit
190 ... Cell catalog storage
210 ... Input / output unit
220 ... Cell object recognition unit
222... Binarization processing unit
224 ... Noise estimation unit (binary data acquisition unit)
226 ... Cell object feature amount calculation unit (noise removal unit)
252 ... Characteristic object extraction / representative feature quantity calculation unit
254 ... Candidate classification model formation unit
256 ... Classification model determination / formation part
260 ... cell classification part
270 ... Search object temporary storage unit
272 ... Cell object storage unit
274 ... Extraction condition storage unit
276 ... Candidate classification model storage unit
280 ... Classification model storage unit
290 ... Cell catalog storage unit
311 ... Incubator
319 ... Culture vessel
401, 401b ... Image processing apparatus
404 ... Cell sorting apparatus
410... Binary image acquisition unit
411 ... Extraction unit
420, 420b ... Cell object extraction unit
421 ... Cell candidate object extraction unit
422 ... Area calculation unit
423 ... Cumulative frequency distribution calculation unit (cumulative distribution calculation unit)
424 ... Differential function calculation unit
425 ... Unique threshold value calculation unit
426 ... Cell object selection section
431 ... Temporary threshold setting unit
432 ... Sample object selection section
433 ... Cell type estimation unit
442 ... Cell counting unit
446 ... Correction information setting section
460 ... Cell object feature amount calculation unit
464b ... cell classification part
511 ... Incubator
515 ... constant temperature room
541 ... Control device (culture condition control unit)
534 ... Imaging device

Claims (21)

A cell candidate object extraction unit that extracts a cell candidate object that is a candidate for a cell object from image data obtained by imaging a cell;
A feature amount extraction unit for extracting a morphological feature amount of the cell candidate object;
Either the frequency of appearance of the value based on the morphological feature amount for each section or the value based on the morphological feature amount for each section when the value based on the morphological feature amount is partitioned at predetermined intervals Cumulative distribution obtained by sequentially adding the cumulative distribution calculation unit for calculating for each cell type ,
Correction information for correcting a threshold value specific to the cell type, and correction information specific to the cell type is stored in association with the cell type;
The unique correction information corresponding to the type of cell input from the outside is read out from the unique correction information storage unit, the morphological feature amount that minimizes the differential value of the cumulative distribution is calculated, and the calculated morphological A unique threshold value calculation unit that calculates a threshold value specific to the cell type from the slope of the cumulative distribution corresponding to the cell type for each cell type based on the feature amount and the read specific correction information When,
Based on the cumulative distribution, the cell object or object other than the cell object is extracted from the cell candidate objects, and the cell candidate object is extracted based on the specific threshold value. An extraction unit for extracting the cell object from :
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the extraction unit calculates a differential value of the cumulative distribution, and extracts the cell object from the cell candidate object based on the differential value.   The extraction unit searches for a minimum value of the differential value, determines a threshold value based on the minimum value, and extracts the cell candidate object having the morphological feature amount equal to or greater than the threshold value as the cell object. The image processing apparatus according to claim 2.   The extraction unit searches for a minimum value of the differential value in a range where the morphological feature value is equal to or greater than a predetermined value, determines the minimum value or a neighborhood of the minimum value as a threshold value, and the morphological feature value is the threshold value The image processing apparatus according to claim 2, wherein the cell candidate object is extracted as the cell object. Each time the extraction unit changes the provisional unique correction information within a predetermined range, the provision unit calculates a provisional unique threshold based on the provisional unique correction information and the minimum morphological feature amount, The image processing apparatus according to claim 4 , wherein the cell object is extracted based on the calculated temporary unique threshold.
A cell counting unit that counts the number of the cell objects extracted by the extraction unit as the number of recognitions for each changed temporary unique correction information;
For each of the changed provisional unique correction information, an error calculation unit that calculates a difference between the number of recognition and the number of visible cells that is the number of the cell objects included in the image data;
The unique correction information is extracted for each cell type from provisional unique correction information within which the difference falls within a predetermined range, and the cell type and the extracted unique correction information are extracted as the unique correction. A unique correction information setting unit to be stored in association with the information storage unit;
An image processing apparatus further comprising: The image according to any one of claims 1 to 5 , wherein the unique threshold value calculation unit further sets the unique threshold value according to a culture time of the cell input from the outside. Processing equipment. Based on the cumulative distribution calculated by the cumulative distribution calculator, a temporary threshold calculator that calculates a threshold common to the plurality of cell types,
A sample object extraction unit that extracts a sample object that is a temporary cell object from the cell candidate objects based on the common threshold;
A cell type estimation unit for estimating the type of the cell based on the extracted sample object;
With
Said unique threshold value calculation unit, according to any one of claims 6 to the unique correction information corresponding to the type of the estimated cells from claim 1, characterized in that the reading from the inherent correction information storage unit Image processing apparatus. A common correction information storage unit storing correction information common to a plurality of types of cells,
The temporary threshold value calculation unit reads the common correction information from the common correction information storage unit, calculates the morphological feature value that minimizes the differential value of the cumulative distribution, and calculates the morphological feature value and the calculated morphological feature value The image processing apparatus according to claim 7 , wherein the common threshold value is calculated based on the read common correction information. Each time the extraction unit changes temporary common correction information within a predetermined range, a temporary common threshold is calculated based on the temporary common correction information and the minimal morphological feature amount, The image processing apparatus according to claim 8 , wherein the cell object is extracted based on the calculated temporary common threshold.
For each cell type, a common cell counting unit that counts the number of cell objects extracted by the extraction unit as a recognition number for each changed temporary common correction information;
For each cell type, a common error calculation that calculates the difference between the number of recognitions and the number of visible cells, which is the number of the cell objects included in the image data, for each changed temporary common correction information. And
Common correction information is extracted from temporary common correction information in which the difference falls within a predetermined range in common with a plurality of cell types, and the extracted common correction information is stored in the common correction information storage unit. A common correction information setting unit to be stored;
An image processing apparatus further comprising: The temporary threshold value calculation unit in accordance with the incubation time of the cells, the image processing apparatus according to any one of claims 7 to 9, characterized in that to calculate the common threshold. The extraction unit includes:
Among the morphological features and the points of the cumulative distribution which constitutes the cumulative distribution, any one of claims 1 to 1 0, characterized in that to calculate the slope of the cumulative distribution by using a plurality of points The image processing apparatus according to item 1. The extraction unit includes:
A noise estimation unit for estimating an object related to the noise based on the cumulative distribution;
A noise removing unit that removes the object related to the estimated noise from the cell candidate objects;
The image processing apparatus according to any one of claims 1 to 11, further comprising: The cell candidate object extraction unit includes:
A binarized image acquisition unit that acquires binarized image data obtained by binarizing image data obtained by imaging cells;
A cell candidate object extraction unit that extracts a cell candidate object that is a candidate for a cell object from the binarized image data;
The image processing apparatus according to any one of claims 1 to 12 , further comprising: An image processing apparatus according to claim 1;
A cell classification unit that classifies cells photographed in the image data based on the cell object extracted from the image processing apparatus or the object related to the noise;
A cell sorting apparatus comprising: Containing a culture vessel for culturing cells or cell groups, and a temperature-controlled room capable of maintaining the interior in a predetermined environmental condition;
An imaging device for capturing an image of the cells or cell groups contained in the culture vessel in the temperature-controlled room;
An image processing apparatus according to claim 1;
An incubator comprising: 16. The cell classification unit according to claim 15 , further comprising a cell classification unit that classifies cells photographed in the image data based on the cell object extracted by the image processing apparatus or the object related to the noise. incubator. 17. The incubator according to claim 16 , further comprising a culture condition control unit that changes an environmental condition of the temperature-controlled room or a medium in which the cells are cultured based on information on the cells estimated by the cell classification unit. . An image processing method executed by an image processing apparatus,
A cell candidate object extraction procedure for extracting a cell candidate object that is a candidate for a cell object from image data obtained by imaging a cell;
A feature amount extraction procedure for extracting a morphological feature amount of the cell candidate object;
Either the frequency of appearance of the value based on the morphological feature amount for each section or the value based on the morphological feature amount for each section when the value based on the morphological feature amount is partitioned at predetermined intervals Cumulative distribution calculation procedure to calculate the cumulative distribution obtained by adding together sequentially for each cell type ,
Correction information for correcting a threshold value specific to a cell type, and correction information specific to a cell type is stored in association with the cell type, and stored from an inherent correction information storage unit. Read the specific correction information according to the type, calculate the morphological feature value that minimizes the differential value of the cumulative distribution, and based on the calculated morphological feature value and the read specific correction information A unique threshold calculation procedure for calculating a threshold specific to the cell type from the slope of the cumulative distribution according to the cell type for each cell type;
A procedure for extracting an object related to noise that is an object other than the cell object or the cell object from the cell candidate objects based on the cumulative distribution, the cell candidate object based on the specific threshold value An extraction procedure for extracting the cell object from :
An image processing method comprising: An image processing method according to claim 18 ,
A cell classification procedure for classifying cells photographed in image data based on the cell object extracted by the image processing method or the object related to the noise;
A cell classification method characterized by comprising: In a computer as an image processing device,
A cell candidate object extraction step for extracting a cell candidate object that is a candidate for a cell object from image data obtained by imaging a cell;
A feature amount extracting step of extracting a morphological feature amount of the cell candidate object;
Either the frequency of appearance of the value based on the morphological feature amount for each section or the value based on the morphological feature amount for each section when the value based on the morphological feature amount is partitioned at predetermined intervals The cumulative distribution obtained by sequentially adding the cumulative distribution calculating step for calculating for each cell type ,
Correction information for correcting a threshold value specific to a cell type, and correction information specific to a cell type is stored in association with the cell type, and stored from an inherent correction information storage unit. Read the specific correction information according to the type, calculate the morphological feature value that minimizes the differential value of the cumulative distribution, and based on the calculated morphological feature value and the read specific correction information A unique threshold calculation step for calculating a threshold specific to the cell type from the slope of the cumulative distribution according to the cell type for each cell type;
Extracting a cell object or an object related to noise that is an object other than the cell object from the cell candidate objects based on the cumulative distribution, the cell candidate object based on the specific threshold value An extraction step of extracting the cell object from :
An image processing program for executing To a computer as a cell sorter,
And an image processing program according to claim 2 0,
Classifying cells photographed in image data based on the cell object extracted from the image processing program or the object related to the noise;
Cell classification program for running