JP2006349533A - Image processor and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively utilize determination results, in a cell recognition process. <P>SOLUTION: This image processor 1 for processing and displaying an observed image, on which viable cells are recorded, comprises a cell recognition means for performing the cell recognition process for recognizing a cell region as an image range corresponding to the viable cells from the observation image; a feature quantity calculation means for calculating the cell feature quantity indicating the feature of each cell region recognized by the cell recognition means; a recognition appropriateness determination means for determining the appropriateness of recognition of the cell region in the cell recognition process, based on the predetermined determination standard; a recognition appropriateness storage means for correlating and storing the recognition appropriateness information, indicating the determination result to the cell region to be determined; a recognition reliability calculation means for calculating the recognition reliability indicating the degree of appropriateness for recognition of the cell region, in the cell recognition process; and a display control means for controlling displaying of the recognition result, recognition appropriateness information, recognition reliability, etc. by the observed image, cell feature quantity and the cell recognition means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program, and more particularly to an image processing apparatus and an image processing program that process and display each observation image in which live cells are recorded.

  Conventionally, observation using various cells as specimens using a microscope or the like has been performed. When observing live cells over time using a microscope, it is normal to recognize live cells recorded by visual and manual operations from observation images obtained by imaging live cells at multiple time points. The images of each living cell are compared between the series of observation images in a series, and the temporal change in the feature amount of each living cell image is observed.

  Such a conventional observation method has a problem that the load on the observer increases as the number of observation cells, observation frequency, observation range, observation time, etc. increase. On the other hand, an image processing method for automatically recognizing a cell nucleus as an observation target without being affected by noise in the image or an edge other than the observation target is proposed (for example, Patent Document 1). In this image processing method, after recognizing the cell nucleus, the recognition result is evaluated based on the luminance information of the recognized cell nucleus. When the evaluated recognition result is determined to be inappropriate, the recognition of the cell nucleus corresponding to the recognition result is performed. By deleting the information, the cell nucleus is accurately recognized.

JP 2000-331143 A Vincent & Soille, "Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations", IEEE TRANS ACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.13, NO.6, JUNE 1991

  However, in the conventional image processing method described above, when the recognition result is determined to be inappropriate, the information related to recognition is deleted, and thus there is a problem that the cell nucleus determined to be inappropriate may not be observed. there were. In addition, there is a problem in that extra time and labor may be required by repeating the recognition process again in order to acquire information related to the deleted recognition.

  The present invention has been made in view of the above, and is an image processing that can effectively use information relating to recognition of living cells even when the result of recognition of living cells included in the observed image is inappropriate or undesirable. An object is to provide an apparatus and an image processing program.

  In order to achieve the above object, an image processing apparatus according to claim 1 is an image processing apparatus that processes and displays an observation image in which a living cell is recorded, and corresponds to the living cell from the observation image. A cell recognition unit that performs a cell recognition process for recognizing a cell region that is an image region, a feature amount calculation unit that calculates a cell feature amount indicating a feature of each cell region recognized by the cell recognition unit, and a predetermined determination criterion Based on the above, a recognition success / failure determination means for determining whether or not the recognition of the cell region in the cell recognition processing is successful, and a recognition success / failure memory for storing recognition success / failure information indicating the result of the determination in association with the cell region to be determined Means.

  In the image processing apparatus according to claim 2, in the above invention, the cell recognition unit includes a cell extraction unit that performs a cell extraction process of extracting the cell region from the observation image, The determination criterion includes a predetermined region determination criterion for the cell image region, and the recognition success / failure determination unit determines whether the extraction of the cell region in the cell extraction process is successful or not based on the predetermined region determination criterion. The recognition success / failure storage means stores the extraction success / failure information as the recognition success / failure information indicating the result of the determination in association with the cell region to be determined.

  In the image processing apparatus according to claim 3, in the above invention, the predetermined region determination criterion is compatibility of the cell region with a cell model region that is a predetermined image region, and the extraction success / failure determination The means is characterized by determining the success or failure of the extraction of the cell region based on the compatibility.

  In the image processing apparatus according to claim 4, in the above invention, the observation image is a plurality of observation images captured at different observation time points, and the cell recognition means includes each of the observation images in the plurality of observation images. A cell extraction means for extracting the cell region from the observation image and a change amount of the cell feature amount between the observation images having different observation time points are calculated, and the observation time point is calculated based on the calculated change amount. Cell tracking means for determining whether or not the living cells indicated by the cell regions included in the different observation images have identity, and performing identity determination processing for associating the cell regions determined to be identical; And the predetermined determination criterion includes a predetermined identity determination criterion for the identity, and the recognition success / failure determination means is the same in the identity determination processing based on the predetermined identity determination criterion. Whether there is sex The determination of success or failure of the judgment or the degree of success or failure is performed, and the recognition success / failure storage means stores tracking success / failure information as the recognition success / failure information indicating the result of the determination in association with the cell region to be determined. To do.

  In the image processing apparatus according to claim 5, in the above invention, the predetermined identity determination criterion is a feature amount calculated using a variation amount of the cell feature amount between observation images having different observation time points. The tracking value is at least one of an evaluation value and a state change state indicating a state change of the cell region, and the tracking success / failure determination unit is configured to perform the determination based on at least one of the feature amount evaluation value and the state change state. It is characterized by determining success or failure.

  In the image processing apparatus according to claim 6, in the above invention, the predetermined identity determination criterion is a feature amount calculated using a variation amount of the cell feature amount between observation images having different observation time points. It is an evaluation value, and the tracking success / failure judging means judges whether or not the judgment is successful or the degree of success or failure according to the magnitude relationship between the feature quantity evaluation value and a predetermined threshold value.

  According to a seventh aspect of the present invention, there is provided the image processing device according to the above invention, wherein at least one of the observation image, the cell feature amount, and the recognition result by the cell recognition unit is displayed together with the recognition success / failure information. The display control means for performing is provided.

  In the image processing apparatus according to claim 8, in the above invention, the feature amount calculation unit calculates a region position of each cell region as the cell feature amount, and the observation image and the recognition by the cell recognition unit. It is characterized by comprising display control means for performing control to display at least one of the results together with the region position of the cell region in which the recognition success / failure information indicates a predetermined result.

  The image processing apparatus according to claim 9 is the image processing apparatus according to the above invention, wherein the display control unit displays the region position of the cell region in which the recognition success / failure information indicates a predetermined result by surrounding the region position. It is characterized in that the display is controlled by.

  The image processing apparatus according to claim 10 is characterized in that, in the above invention, the recognition of a cell region in the cell recognition process based on at least one of the determination contents by the recognition success / failure determination means and the cell feature amount. There is provided a reliability calculation means for calculating a recognition reliability indicating the degree of success or failure.

  In the image processing apparatus according to claim 11, in the above invention, control for displaying at least one of the observed image, the cell feature amount, and a recognition result by the cell recognition unit together with the recognition reliability is performed. The display control means for performing is provided.

  According to a twelfth aspect of the present invention, in the above invention, the image processing apparatus further comprises display control means for performing control to display the recognition success / failure information together with the recognition reliability.

  In the image processing apparatus according to claim 13, in the above invention, the reliability calculation unit classifies each cell region into one or more cell groups according to the calculated recognition reliability, And a display control means for performing control to display at least one of the cell feature amount and the recognition result by the cell recognition means together with the classification result by the reliability calculation means.

  In the image processing apparatus according to claim 14, in the above invention, the reliability calculation unit classifies the cell regions into one or more cell groups according to the calculated recognition reliability, and the recognition success / failure information. Display control means for performing control to display the image together with the classification result by the reliability calculation means.

  The image processing apparatus according to claim 15 is the above invention, wherein the reliability calculation means classifies each cell region into one or more cell groups according to the calculated recognition reliability, and calculates the feature amount. The means calculates a region position of each cell region as the cell feature amount, and at least one of the observation image and the recognition result by the cell recognition unit is combined with the region position of the cell region belonging to the predetermined cell group. A display control means for performing display control is provided.

  In the image processing apparatus according to claim 16, in the above invention, the display control unit performs control to display the region position of a cell region belonging to the predetermined cell group with an image mark surrounding the region position. It is characterized by performing.

  The image processing apparatus according to claim 17 further comprises selection information input means for receiving cell selection information for selecting the cell region in the above invention, wherein the display control means is a cell selected by the cell selection information. Control is performed to selectively display an area as a display target.

  The image processing apparatus according to claim 18 is the image processing apparatus according to the above invention, wherein the feature amount calculating unit includes the luminance sum, area, circularity, barycentric position, contour length, contour position of each cell region as the cell feature amount. It is characterized by calculating at least one of these.

  An image processing program according to claim 19 is an image processing program for processing and displaying the observation image on an image processing apparatus that processes and displays the observation image in which live cells are recorded. A cell recognition procedure for performing a cell recognition process for recognizing a cell region that is an image region corresponding to the living cell from the observed image in the image processing device, and a feature of each cell region recognized by the cell recognition procedure A feature amount calculation procedure for calculating a cell feature amount to be indicated, a recognition success / failure determination procedure for determining whether or not a cell region is recognized in the cell recognition process based on a predetermined determination criterion, and a recognition indicating a result of the determination And a recognition success / failure storage procedure for storing success / failure information in association with a cell region to be determined.

  The image processing program according to claim 20 is the above invention, wherein the cell recognition procedure includes a cell extraction procedure for performing a cell extraction process for extracting the cell region from the observation image, The determination criterion includes a predetermined region determination criterion for the cell image region, and the recognition success / failure determination procedure is based on the predetermined region determination criterion and the degree of success or failure of extraction of the cell region in the cell extraction process. The recognition success / failure storage procedure stores the extraction success / failure information as the recognition success / failure information indicating the result of the determination in association with the cell region to be determined.

  Further, in the image processing program according to claim 21, in the above invention, the observation image is a plurality of observation images taken at different observation time points, and the cell recognition procedure includes each of the observation images in the plurality of observation images. A cell extraction procedure for extracting the cell region from the observation image, and a change amount of the cell feature amount between observation images having different observation time points are calculated, and the observation time point is calculated based on the calculated change amount. A cell tracking procedure for determining whether or not the living cells indicated by the cell regions included in the different observation images have identity, and performing identity determination processing for associating the cell regions determined to be identical; The predetermined determination criterion includes a predetermined identity determination criterion for the identity, and the recognition success / failure determination procedure is the same in the identity determination processing based on the predetermined identity determination criterion. Have sex The recognition success / failure storage procedure stores the tracking success / failure information as the recognition success / failure information indicating the result of the determination in association with the cell region to be determined. It is characterized by that.

  According to the image processing device and the image processing program according to the present invention, the recognition success / failure information indicating the determination result about the success or failure of the recognition of the cell region in the cell recognition processing is recorded, so that the determination made in the cell recognition processing can be performed. The results can be used later. Thereby, it becomes the structure which can utilize the result of the determination in a cell recognition process effectively.

  Exemplary embodiments of an image processing apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments. In the description of the drawings, the same parts are denoted by the same reference numerals.

(Embodiment 1)
First, the image processing apparatus according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing the configuration of the image processing apparatus 1 according to the first embodiment. As shown in FIG. 1, an image processing apparatus 1 includes an image processing unit 2 that processes an input image, a display unit 3 that displays various types of information, an input unit 4 that receives input of various types of information, and various types of information. A storage unit 5 that stores data and a control unit 6 that controls processing and operation of each unit of the image processing apparatus 1 are provided. The image processing unit 2, the display unit 3, the input unit 4, and the storage unit 5 are electrically connected to the control unit 6.

  The image processing unit 2 includes a cell extraction unit 2a, a feature amount calculation unit 2b, an extraction result determination unit 2c as an extraction success / failure determination unit, a cell tracking unit 2d, a tracking result determination unit 2e as a tracking success / failure determination unit, and a reliability calculation unit. And a series of observation images obtained by imaging live cells at a plurality of observation points. Each unit included in the image processing unit 2 is controlled by the image processing control unit 6a. The recognition success / failure determination means of the present invention includes an extraction result determination unit 2c, a tracking result determination unit 2e, and a reliability calculation unit 2f.

  The cell extraction unit 2a performs a cell extraction process for extracting a cell region that is an image region corresponding to each living cell from each observation image. The cell extraction unit 2a divides the observation image by using, for example, an image processing technique called watershed area division (for example, see Non-Patent Document 1), and extracts a cell area. In Non-Patent Document 1, an image is divided into regions where low-brightness pixels are concentrated. However, since cells generally appear as a collection of high-brightness pixels, the cell extraction unit 2a inverts the luminance in terms of technique. It is processed and applied to the division of the high brightness area. Note that the cell extraction unit 2a may perform luminance correction processing, smoothing processing, and the like as preprocessing for the acquired observation image before extracting the cell region.

  In addition, the cell extraction unit 2a performs labeling that assigns a unique region label that is a region marker to each extracted cell region, and uses the added region label as cell region data as region information of each cell region. The region label provided by the cell extraction unit 2a may be any notation using numbers, alphabets, symbols, etc., as long as it is unique. The cell region data is recorded in the cell region data storage unit 5c in association with the observation time point and the cell region by the image processing control unit 6a.

  Furthermore, the cell extraction part 2a produces | generates a cell extraction image as a cell recognition image which displays each cell area | region extracted and labeled. At this time, the cell extraction unit 2a displays a region extraction image that displays the region shape of each extracted cell region, and a region label image that displays the correspondence between each cell region and the region label by distinguishing the hatching type and display color of the cell region. Etc. are generated as a cell extraction image. The generated cell extraction image is recorded in the processed image storage unit 5b by the image processing control unit 6a in association with the observation time point.

  The feature amount calculation unit 2b calculates a cell feature amount indicating the feature of each cell region extracted by the cell extraction unit 2a. Specifically, the feature amount calculation unit 2b includes, as the cell feature amount, a barycentric position, a contour position, a contour length, an area, a circularity, a luminance sum that is a sum of luminance values, and a region indicating the position of the cell region. The position etc. are calculated. Here, the region position may be a gravity center position, a contour position, or a position of a predetermined feature point or the like in the cell region. Such a cell feature amount indirectly indicates a feature amount of a living cell in a cell region.

  The feature amount calculation unit 2b generates a cell feature image by graphing or imaging the calculated cell feature amount. At this time, the feature quantity calculation unit 2b observes various cell feature quantities for each observation image, for each observation time point, for each cell region, or for each cell feature quantity, so that the cell feature images can be displayed in various display forms. Is generated. The generated cell feature image is recorded in the processed image storage unit 5b by the image processing control unit 6a according to the display form. Note that the feature quantity calculation unit 2b may appropriately generate various cell feature images based on instructions from the image processing control unit 6a.

  The extraction result determination unit 2c determines whether each cell region is successfully extracted in the cell extraction process performed by the region extraction unit 2a based on a predetermined region determination criterion for the cell region. Then, extraction success / failure information as recognition success / failure information indicating the success or failure of extraction is recorded in association with the cell region to be processed for each cell extraction process. Specifically, when the extraction result determination unit 2c determines that the extraction is unsuccessful, the extraction failure information as the extraction success / failure information is associated with the cell region extracted by the cell extraction process, and similarly, the extraction is determined to be successful. In this case, the extraction success information is associated with the extracted cell region. The extraction failure information and the extraction success information are added to the cell region data of the corresponding cell region in the cell region data storage unit 5c and recorded by the image processing control unit 6a.

  The region determination criteria referred to by the extraction result determination unit 2c are determined by a plurality of types of success / failure determination items. For example, the extraction result determination unit 2c determines suitability for a cell model that is a characteristic shape pattern of a cell region as a success / failure determination item. That is, the compatibility with a predetermined cell model is set as a success / failure determination item. If the cell region extracted by the cell extraction unit 2a matches any one or a plurality of predetermined failure cell models, the extraction result determination unit 2c determines that the cell extraction process for extracting the cell region is a failure. judge. Similarly, the extraction result determination unit 2c may determine that the cell extraction process is successful when it matches a predetermined successful cell model. The cell model used in this way is variously set according to the cell to be observed. For example, in the successful cell model, the cell region extracted by the cell extraction unit 2a exists as an independent closed region in the image, and is defined as an image region having an average area as compared with the surrounding cell regions. it can. Examples of the failed cell model include FIGS. 6-1 to 6-3 described later.

  In addition, the extraction result determination unit 2c determines the success or failure of the cell extraction process using, for example, a predetermined statistical amount or an evaluation amount calculated using the cell feature amount of the cell region to be determined as a success / failure determination item. Alternatively, such numerical success / failure determination items and image-like success / failure determination items such as a cell model may be used in combination.

  The extraction result determination unit 2c associates the extraction failure reason, which is a determination reason (success / failure determination item) when the cell extraction process is determined to be failure, with the extracted cell region together with the extraction failure information. When there are a plurality of extraction failure reasons (when it is determined that there are a plurality of success / failure determination items), the extraction result determination unit 2c displays all applied extraction failure reasons and the number of extraction failure reason items in the cell region. Associate with. The reason for the extraction failure and the number of items are added to the cell region data and recorded together with the extraction failure information.

  The cell tracking unit 2d performs identity determination processing of the living cells extracted at different observation points. That is, in the first embodiment, the cell tracking unit 2d calculates the change amount of the cell feature amount between the observation images with different observation time points, and based on the calculated change amount, each observation image with different observation time points. It is determined whether or not the living cell indicated by the cell region extracted from the cell region is identical, and the identity determination process is performed by associating the cell region of each living cell determined to be identical. At this time, the cell tracking unit 2d assigns the same tracking label to the cell area of each living cell determined to be identical, and uses the assigned tracking label as cell tracking data as tracking information of each cell area. The cell tracking data is recorded in the cell tracking data storage unit 5d in association with the observation time point and the cell region by the image processing control unit 6a.

  The tracking label provided by the cell tracking unit 2d may be any notation using numbers, alphabets, symbols, etc., as long as it is unique, similarly to the region label. The cell tracking unit 2d may newly add a tracking label to each cell region, or, with reference to the region label, the observation time point of each cell region determined to be the same as the previous cell region The given region label may be given as a tracking label to a cell region whose observation time is later.

  In addition, the cell tracking unit 2d associates a cell region between two observation images that are continuous in time series, for example, and sequentially performs the processing from the first observation image in time series to obtain the observation image to be processed as the observation time point. It is made to change and repeat in order, and a cell area | region is matched between arbitrary observation images. As a result, the cell tracking unit 2d can track individual cell regions at each observation time point in time series.

  Furthermore, the cell tracking unit 2d generates a cell tracking image as a cell recognition image that displays each cell region associated with a tracking label. At this time, the cell tracking unit 2d generates, as a cell tracking image, a tracking label image that displays the correspondence between each cell region and the tracking label with the hatching type and display color of the cell region. The generated cell tracking image is recorded in the processed image data storage unit 5b by the image processing control unit 6a in association with the observation time point.

  The tracking result determination unit 2e determines whether or not the determination of the identity of the living cells in the identity determination process performed by the cell tracking unit 2d is successful according to a predetermined identity determination criterion (described later). The tracking success / failure information as the recognition success / failure information indicating the success or failure of the determination of the identity of the living cell is recorded in association with the cell region to be processed. Specifically, when the tracking result determination unit 2e determines that the determination of the identity of the living cell in the identity determination process is a failure, the cell region of each living cell determined to be the same in the identity determination process Is associated with tracking failure information as tracking success / failure information. Similarly, when it is determined to be successful, the tracking result determination unit 2e associates tracking success information with each cell region. The tracking failure information and the tracking success information are added and recorded by the image processing control unit 6a to the cell tracking data of the corresponding cell region in the cell tracking data storage unit 5d.

  The identity determination criteria referred to by the tracking result determination unit 2e are determined by a plurality of types of success / failure determination items. For example, the tracking result determination unit 2e performs determination based on a plurality of success / failure determination items using a feature amount evaluation value determined based on a variation amount of a cell feature amount between observation images to be processed. In other words, the tracking result determination unit 2e compares the magnitude relationship between the feature amount evaluation value determined from the variation amount of the cell feature amount between the cell regions of the living cells determined to have the identity and the predetermined threshold value, The success or failure of the determination process is determined. Such a determination is performed for a plurality of types of cell feature amounts (success / failure determination items).

  Further, the success / failure determination item of the identity determination criterion may be, for example, a state change state indicating a state change of a cell region between observation points in time, and the tracking result determination unit 2e may determine whether there is identity. If the state change state of the cell region of the cell is clearly inappropriate in the relationship with the surrounding cell regions, the identity determination process that determines this identity is determined to be a failure. Here, the change in the state of the cell region means a change in the cell region accompanying deformation, division, connection, disappearance, etc. of the living cells in the cell region. The tracking result determination unit 2e may use a quantitative success / failure determination item such as a feature amount evaluation value and a qualitative success / failure determination item such as a state change state in combination.

  Further, the tracking result determination unit 2e associates the tracking failure reason, which is a determination reason (success / failure determination item) when the determination of the identity of the living cell in the identity determination process is failure, with the cell region together with the tracking failure information. When there are a plurality of tracking failure reasons (when it is determined that there are a plurality of success / failure determination items), the tracking result determination unit 2e displays all applied tracking failure reasons and the number of items of the tracking failure reasons as a cell region. Associate with. The tracking failure reason and the number of items of the tracking failure reason are added to the cell tracking data and recorded together with the tracking failure information.

  The reliability calculation unit 2f indicates the recognition reliability indicating the degree of success or failure of the recognition of the cell region (extraction of the cell region or determination of identity) in the cell recognition processing (cell extraction processing or identity determination processing). Calculate as The reliability calculation unit 2f calculates the recognition reliability using the determination contents in the extraction result determination unit 2c and the tracking result determination unit 2e and the cell feature amount. An example of a specific calculation method of the recognition reliability will be described later. The recognition reliability calculated by the reliability calculation unit 2f is added and recorded by the image processing control unit 6a to the cell region data or cell tracking data of the corresponding cell region.

  Further, the reliability calculation unit 2f classifies each cell region into one or more cell groups according to the calculated reliability value, and associates the cell group information such as the classified cell group name with each cell region. At this time, the reliability calculation unit 2f classifies each cell region into a cell group based on the recognition reliability, and records the classification result as cell group information in association with each cell region. The cell group information is added and recorded as the recognition success / failure information to the cell region data or the cell tracking data of the corresponding cell region by the image processing control unit 6a.

  Further, the reliability calculation unit 2f generates a grouped image that displays each cell region for each classified cell group. The grouped images are recorded in the processed image data storage unit 5b in association with the observation time by the image processing control unit 6a.

  The display unit 3 includes a CRT, a liquid crystal display, and the like, and displays various information including various processed images generated by the image processing unit 2 processing the observation image. In particular, the display unit 3 recognizes at least one of an observation image, a cell extraction image, a cell feature image, a cell tracking image, and a grouped image based on an instruction from the image display control unit 6b (extraction success / failure information). Information, tracking success / failure information, recognition reliability, cell group information).

  The input unit 4 includes a communication interface such as USB and IEEE1394, various switches, input keys, a mouse, a touch panel, and the like, and receives input of an image to be processed by the image processing apparatus 1 and various processing information. In particular, the input unit 4 includes an observation image input unit 4a and a display item selection unit 4b. The observation image input unit 4a receives an input of an observation image from an external device via a communication interface. The display item selection unit 4b receives input of display item selection information for selecting at least one of an observation image and various processed images to be displayed on the display unit 3. The observer can arbitrarily select an image to be displayed, a cell feature amount, and the like by inputting display item selection information via the display item selection unit 4b.

  Note that the observation image input unit 4a may include an interface corresponding to a portable storage medium such as a flash memory, a CD, a DVD, or a hard disk, and may accept an input of an observation image from the portable storage medium. Further, the observation image input unit 4a includes an imaging device realized by using an imaging lens, an imaging device such as a CCD, and an A / D converter, and an observation generated by imaging a living cell by the imaging device. An image may be acquired.

  The storage unit 5 is realized by a ROM that stores various processing programs and the like in advance, and a RAM that stores processing parameters, processing data, and the like of each processing. In particular, the storage unit 5 stores an image processing program for processing and displaying an observation image, which is a program executed by the control unit 6. The storage unit 5 includes an observation image storage unit 5a that stores observation images, a processing image storage unit 5b that stores various processing images, and a cell region data storage unit 5c that stores cell region data of each cell region. And a cell tracking data storage unit 5d for storing cell tracking data of each cell region. The storage unit 5 may include a portable storage medium such as a flash memory, a CD, a DVD, and a hard disk as a removable storage unit.

  The control unit 6 is realized by a CPU or the like that executes various processing programs stored in the storage unit 5. In particular, the control unit 6 executes an image processing program stored in the storage unit 5 and controls each component corresponding to each process in the image processing program. The control unit 6 includes an image processing control unit 6a, an image display control unit 6b, and a cell data integration unit 6c. The image processing control unit 6 a controls processing and operations performed by each unit of the image processing unit 2 and controls input / output of various data and parameters to the image processing unit 2.

  The image display control unit 6b displays items from the observation image stored in the observation image storage unit 5a and the cell extraction image, cell feature image, cell tracking image, and grouped image stored in the processed image storage unit 5b. Each image selected by the selection information is displayed so as to be switched simultaneously, sequentially or based on predetermined instruction information, and extraction success / failure information, extraction failure reason, tracking success / failure information, tracking failure reason, recognition reliability, cell Control to display on the display unit 3 in association with the group information or the like is performed. At this time, the image display control unit 6b may display the display brightness, the display hue, and the like in each image differently according to, for example, the observation time point, extraction and tracking success / failure information, the amount of change in the cell feature amount, and the like. Further, the region position of the corresponding cell region may be marked with a predetermined image mark.

  The cell data integration unit 6c integrates the cell region data stored in the cell region data storage unit 5c and the cell tracking data stored in the cell tracking data storage unit 5d in association with each other. At this time, the cell data integration unit 6c adds the cell tracking data to the existing cell region data, and re-records it in the cell region data storage unit 5c as the integrated region data. The cell data integration unit 6c may newly record only the association information between the cell area data and the cell tracking data, or may record the integrated area data in a dedicated storage area. May be.

  Here, a processing procedure performed by the image processing apparatus 1 will be described. FIG. 2 is a schematic diagram showing a series of observation images input to the image processing apparatus 1, and FIG. 3 shows that the image processing apparatus 1 is input by the control unit 6 executing the image processing program. It is a flowchart which shows the process sequence which processes and displays an observation image.

As shown in FIG. 2, the input series of observation images includes a plurality of living cells to be observed in a dark background. Each living cell undergoes state changes such as movement, expansion, contraction, deformation, and disappearance between observation time points of observation time points t _{i to} t _{i + 1} . This series of observation images is stored in the observation image storage unit 5a.

  First, each step shown in FIG. 3 will be briefly described. The cell extraction unit 2a extracts a cell region corresponding to each living cell from each observation image, performs labeling, and then performs a cell extraction process for generating a cell extraction image (step S101). The feature amount calculation unit 2b calculates a cell feature amount corresponding to each extracted cell region, and performs a feature amount calculation process for generating a cell feature image (step S103). The extraction result determination unit 2c determines whether the extraction of the cell region in the cell extraction process is successful, and performs an extraction result determination process that associates the extraction success / failure information, the extraction failure reason, and the like (step S105).

  The cell tracking unit 2d performs the cell tracking process of determining the identity of each living cell between observation images at different observation times, associating the cell regions of each living cell having the same identity, and generating a cell tracking image ( Step S107). The tracking result determination unit 2e determines the success or failure of the identity determination in the identity determination processing that determines the identity of each living cell, and performs tracking result determination processing that associates tracking success / failure information, tracking failure reason, and the like (step S109). ). The reliability calculation unit 2f calculates a recognition reliability, classifies each cell region into a cell group, and performs a reliability calculation process for generating a grouped image (step S111).

  The cell data integration unit 6c performs a data integration process in which cell region data and cell tracking data are associated with each other to obtain integrated region data (step S113). The display processing control unit 6b performs a result display process for displaying each image selected from the observed image and various processed images according to the display item selection information in association with the integrated region data (step S115). After the above steps S101 to S115, the image processing apparatus 1 ends the series of processes. Note that the image processing apparatus 1 may repeat the result display process in step S115 according to the updated display item selection information, for example, until receiving predetermined process end instruction information.

  Next, the cell extraction process in step S101 shown in FIG. 3 will be described. FIG. 4 is a flowchart showing the procedure of the cell extraction process. First, the image processing control unit 6a reads the first image in time series among the series of observation images stored in the observation image storage unit 5a (step S121). Next, the cell extraction unit 2a performs preprocessing of the read observation image (step S123). In step S123, the cell extraction unit 2a performs, for example, a smoothing process using a Gaussian filter or the like on the observed image.

  Subsequently, the region extraction unit 2a extracts each cell region in the observation image using the watershed region division (step S125), performs labeling for assigning a region label to each extracted cell region (step S127), and region Cell extraction images such as extracted images and region label images are generated (step S129). Thereafter, the image processing control unit 6a records the region label given to each cell region as cell region data in the cell region data storage unit 5c, and records the cell extraction image in the processed image data storage unit 5b (step S131). ).

  Then, the image processing control unit 6a determines whether or not a series of all observed images has been processed (step S133). If all the observed images have not been processed (step S133: No), no processing has been performed. If the process from step S121 is repeated for the observed image and all the observed images are processed (step S133: Yes), the process returns to step S101.

FIG. 5 is a diagram exemplifying region label images generated corresponding to the observation images at the observation times t _i and t _{i + 1} shown in FIG. In step S129, as shown in FIG. 5, the area extraction unit 2a sets the edge of each cell area extracted in step S125 as the outline of the cell area, and shows the cell area by this outline. In step S127, the area extractor 2a A region label image is generated that indicates the given region labels (Ia1 to Ia13, Ib1 to Ib15) as character information.

Next, in the feature amount calculation process of step S103 shown in FIG. 3, the feature amount calculation unit 2b uses at least the center of gravity position, the contour position, the contour length, the area, the circularity, the luminance sum, and the cell feature amount as the cell feature amount. The region position is calculated. Among these, the circularity e of the cell region is calculated by e = 4πS / L ² using the contour length L and the area S of the cell region. The circularity e satisfies the relationship 0 <e <1.

  Next, the extraction result determination process in step S105 shown in FIG. 3 will be described. In step S105, the extraction result determination unit 2c evaluates the suitability of each cell region extracted in step S101 for various cell models determined in advance and associates the extraction success / failure information with each other. For example, in the case of a failed cell model, the cell model is determined as shown in FIGS. 6-1 to 6-3. Fig. 6-1 shows a failure cell model in the case where another living cell (hereinafter referred to as an internal cell) exists inside one living cell, and Fig. 6-2 shows a hole inside the living cell. FIG. 6-3 shows a failed cell model in the case where there is a microregion surrounded by a plurality of living cells.

  FIG. 7 is a flowchart showing the processing procedure of the extraction result determination process in the case of matching the extraction success / failure information when the suitability is evaluated using the failed cell model having the internal cell shown in FIG. 6A as the success / failure determination item. is there. Here, the compatibility with the failed cell model in which the internal cell exists is performed by detecting the internal cell itself. As shown in FIG. 7, the image processing control unit 6a reads a cell extraction image from the processed image storage unit 5b (step S141), detects a pixel without an area label, and uses a background label as a background pixel indicating the background area. It adds (step S143). Here, the background pixel is always adjacent to other background pixels. The extraction result determination unit 2c refers to the cell region data and extracts a cell region to be processed for determining the extraction result from the cell extraction image (step S145).

  Subsequently, the extraction result determination unit 2c extracts cell pixels constituting the cell region extracted from the cell extraction image (step S147), and detects the region label of each adjacent pixel adjacent to the extracted cell pixel (step S149). ). Then, the extraction result determination unit 2c determines whether or not there is a background label in the detected adjacent pixel (step S151), and if there is no background label (step S151: No), all the cell pixels in the cell region. Is further determined (step S153).

  When all the cell pixels are not processed (step S153: No), the extraction result determination unit 2c repeats the process from step S147 on the unprocessed cell pixels, and processes all the cell pixels. In the case (step S153: Yes), the cell region to be processed is recognized as an internal cell region, and the cell extraction in the cell extraction process in which this cell region is extracted from the observation image is determined to be a failure (step S155). The extraction failure information is associated with the cell region (step S157).

  Furthermore, the extraction result determination unit 2c recognizes the cell region around the processing target as the region of the external cell, and determines that the cell extraction in the cell extraction process in which this cell region is extracted from the observation image is a failure (step S159). That is, the cell region corresponding to the region label detected in step S149 is recognized as an external cell region, and the extraction of the external cell in the cell extraction process in which the external cell is extracted is determined as failure. Then, the extraction result determination unit 2c associates the extraction failure information with a cell region around the processing target, that is, an external cell (step S161).

  On the other hand, if there is a background label in the adjacent pixel in step S149 (step S151: Yes), the extraction result determination unit 2c determines that the cell extraction is successful (step S163), and the extraction success information is stored in the cell region to be processed. Are associated with each other (step S165).

  After step S161 or step S165, the image processing control unit 6a adds the extraction success / failure information associated with the cell region to the cell region data, and records it in the cell region data storage unit 5c (step S167). Return.

  In this way, the extraction result determination unit 2c recognizes the area AR1 whose surrounding is the area AR2 as shown in FIG. 6A as an internal cell. When the internal cell is detected in this way, the area AR2 is determined to be an external cell, and both the areas AR1 and AR2 determine that the cell extraction in the cell extraction process has failed. On the other hand, if no internal cell is detected and the entire surrounding area is the background area AR3, the extraction result determination unit 2c determines that the cell extraction in the cell extraction process is successful in the evaluation of the suitability for the cell model in which the internal cell exists. To do.

  Note that if there are a plurality of cell regions that need to be determined in one cell extraction image, the image processing control unit 6a may repeat the processing from step S145 for each cell region. Moreover, the image processing control part 6a should just repeat the process from step S141 for every observation time, when determination about several observation time is required. Furthermore, when evaluating the compatibility with cell models (for example, FIGS. 6-2 and 6-3) other than the failed cell model in which internal cells exist (FIG. 6-1), the image processing control unit 6 a The extraction result determination process may be repeated for each determination item. That is, for each success / failure determination item, a process for verifying the suitability of the cell pixels constituting the cell region to be processed is performed.

  When the suitability is evaluated using the failed cell model having the hole shown in FIG. 6B as a success / failure determination item, the extraction result determination unit 2c detects a cell pixel from the cell region and is adjacent to the detected cell pixel. If each adjacent pixel does not have a region label or a background label, the region AR4 corresponding to this cell pixel is recognized as a hole region, and extraction failure information is associated with this cell region.

When the micro area AR5 as shown in FIG. 6-3 is used as a success / failure determination item, the extraction result determination unit 2c calculates a determination evaluation amount for the area of the micro area AR5 as follows, and performs cell extraction processing. Determine success or failure. That is, the extraction result determination unit 2c first creates an area histogram of the cell region extracted from the observation image at the observation time t _i . The class number C _ti of this area histogram is calculated by the following formula (1) using the extracted cell region number M _{ti according} to the Struges law.
C _ti = 1 + (logM _ti / log2) (1)
Further, the extraction result determination unit 2c calculates the class width W _ti from the maximum area S _timax , the minimum area S _timin and the class number C _{ti according} to the following equation (2).
W _ti = (S _timax -S _timin ) / C _ti (2)
Further, the extraction result determining unit 2c, using the cell area number X _ti belongs to most frequent class of the area histogram, and calculates the average area S _Timean by the following equation (3).
S _timean = (S _{ti, 1} + S _{ti, 2} +... + S _{ti, Xti} ) / X _ti (3)
Accordingly, the extraction result determination unit 2c uses the product _Stimean × Th of the average area _Stimean and the predetermined threshold Th as the _microregion determination evaluation amount, and when the area of the _microregion AR5 is equal to or smaller _{than the microregion} determination evaluation amount, The extraction failure information is associated with the cell region corresponding to the minute region AR5.

  Furthermore, in the extraction result determination unit 2c, in addition to using the failed cell models in FIGS. 6-1, 6-2, and 6-3 described above as success / failure determination items, cell feature amounts can be used as success / failure determination items. . For example, when the area of a cell region is smaller than a predetermined threshold value, extraction failure information is associated with the cell region, and when it is equal to or greater than the threshold value, extraction success information is associated with the cell region. Process. When performing extraction result determination processing for a plurality of types of success / failure determination items, extraction failure information and extraction success information are associated with each success / failure determination item and recorded as cell region data.

Next, in the cell tracking process of step S107 shown in FIG. 3, the cell tracking unit 2d calculates an evaluation value J for each cell region included in two observation images having different observation time points, and the calculated evaluation value J Depending on the value, the identity of living cells in each cell region is determined. The evaluation value J is the distance δ _d between the centroids for each cell region to which the region labels R _{ti, m} and R _{ti + 1, n} are _{assigned in the} observation images at the observation times t _i and t _{i + 1.} Using the difference of area δ _a , difference of circularity δ _c , and predetermined weighting coefficients k _d , k _a , k _c , the following equation (4) is used.
_{J = J (R ti, m} , R ti + 1, n) = k d δ d + k a δ a + k c δ c ··· (4)
Here, the variables m and n have a relationship of 1 ≦ m ≦ M and 1 ≦ n ≦ N with respect to the number of cell regions M and N extracted in each observation image at the observation time points t _i and t _{i + 1.} is there.

The cell tracking unit 2d calculates the evaluation value J for all possible combinations of the variables m and n. If the calculated evaluation value J is equal to or less than the predetermined threshold θ, the region labels R _{ti, m} and R _{ti + 1 , n} is determined to be identical to each cell region, and the same tracking label is assigned to the cell region. Similarly, the cell tracking unit 2d calculates an evaluation value J for each observation image other than the observation time points t _i and t _{i + 1} , and assigns each living cell whose calculated evaluation value J is equal to or less than the threshold θ to the same living cell. And a unique tracking label is assigned to the cell region.

Further, the cell tracking unit 2d generates a tracking label image that displays each cell region associated with a tracking label, as shown in FIG. 8, for example. FIG. 8 is a diagram showing tracking label images generated corresponding to the observation images at the observation times t _i and t _{i + 1} shown in FIG. As shown in FIG. 8, the cell tracking unit 2d generates a tracking label image that indicates each cell region by its outline and also indicates the tracking label (T1 to T16) given to each cell region by character information.

  Next, in the tracking result determination process in step S109 shown in FIG. 3, the tracking result determination unit 2e determines the area between the associated observation time points for the cell region of each living cell in which identity is recognized in step S107. If at least one variation amount of the circularity and the luminance sum is calculated, and the calculated variation amount is used as the above-described feature amount evaluation value, and the feature amount evaluation value is equal to or greater than a predetermined threshold value, the identity determination is performed. The failure is determined, and tracking failure information is associated with each cell region. In addition, the tracking result determination unit 2e determines whether or not the state change state between the associated observation points is inappropriate, and if it is clearly inappropriate, the determination of identity fails And tracking failure information is associated.

FIGS. 9A and 9B show cell regions of living cells in which identity is observed between the observation time points t _{n to} t _{n + 2} and the observation time points t _m and t _{m + 1} , respectively. It is a schematic diagram. As shown in FIG. 9A, each cell region to which the tracking labels Ta and Tb are assigned has a variation amount of the area or the circularity larger than the threshold at the observation time t _{n to} t _{n + 2} , and corresponds to the tracking failure information. Attached. Also, as shown in FIG. 9-2, the cell region to which the tracking label Td is assigned is determined to have disappeared between the observation times t _m and t _{m + 1} , and the state change state is clearly considered inappropriate. Therefore, tracking failure information is associated. Further, the cell region to which the tracking label Tc is assigned is determined to have absorbed the region of the tracking label Td, and the tracking failure information is associated with it. Incidentally, the state shown in Figure 9-2 tracking label Tc that existed separately in the observation time t _m, is the region of Td, the observation time t _m, resulting in a movement of such overlap or contact between t m _{+ 1} It can happen in some cases.

  As described above, in the tracking result determination process in step S109 of the first embodiment, the amount of change in the area of the cell region between the observation points is equal to or greater than a predetermined threshold, or the amount of change in the circularity of the cell region between the observation points. Pass / fail judgment based on cell feature quantities such as whether or not the amount of change in luminance of the cell area between observation points is greater than or equal to a predetermined threshold Judgment is performed as an item. When performing determination using a plurality of success / failure determination items in the tracking result determination processing in step S109 as in the first embodiment, cell failure tracking is performed by associating tracking failure information and tracking success information with each success / failure determination item. Record as data.

Next, in the reliability calculation process in step S111 illustrated in FIG. 3, the reliability calculation unit 2f determines the identity of each living cell in step S101 and the cell extraction process in which each cell region is extracted in step S101. The recognition reliability is calculated for the identity determination process. The recognition reliability R is the number of all items SN _all of the success / failure determination items when the extraction of the cell region and the success / failure of the determination of the identity are determined by the extraction result determination unit 2c and the tracking result determination unit 2d, respectively. and a has been success item number SN _j of judgment item, the difference between the total luminance and a predetermined luminance total variation threshold [delta] _b, the difference [delta] _s between the area and the predetermined area variation threshold, circularity and a predetermined degree of circularity threshold Using the difference δ _e and predetermined weighting coefficients k _b , k _s , and k _e , the following equation (5) is used.
_{R = (SN all -SN j)} / SN all + k b δ b + k s δ s + k e δ e ··· (5)
The recognition reliability R may be calculated using either the first term or the second to fourth terms on the right side of the equation (5).

Further, the reliability calculation unit 2f classifies each cell region into a cell group according to the value of the recognition reliability calculated in this way. Specifically, the reliability calculation unit 2f uses the number G of cell groups to be classified and the total number AC of recognized cell regions to determine cell regions whose recognition reliability R _gi satisfies the relationship of the following equation (6). Sort into cell group g _i (i = 1, 2,... AC).
1 / G + R _{g (i-1)} ≦ R _gi ≦ 1 / G + R _gi (6)

  Next, after performing the data integration process in step S113 shown in FIG. 3, in the result display process in step S115 shown in FIG. 3, the image display control unit 6b, for example, as shown in FIG. The cell feature image obtained by graphing the luminance sum of the cell area having “Ida” with respect to the observation time is displayed, and the integrated area data of the cell area is superimposed and displayed as text data. By simultaneously displaying the cell feature image and the integrated region data that are graphed with respect to the observation time in this way, the image display control unit 6b can change the time-dependent change of the cell feature amount and the success or failure of the extraction or tracking of the cell region Can be easily compared and observed by an observer.

  Note that the image display control unit 6b may simultaneously display the cell feature image as shown in FIG. 10 for a plurality of cell regions. In this case, for example, selection information on the cell region that the observer is interested in is input to the input unit. 4 to display the integrated area data corresponding to this selection information with priority. Further, the image display control unit 6b may display not only the cell feature quantity such as the luminance sum but the recognition reliability and the like in the same graph, and display the graphed recognition reliability together with the integrated region data. You may make it do.

  Further, for example, as shown in FIG. 11, the image display control unit 6b displays a grouped image indicating a cell region with low recognition reliability, and an observation image and a region extraction image indicating the cell region side by side. By displaying the observation image, the cell extraction image, the grouped image, and the like side by side in this way, the image display control unit 6b causes the observer to observe various characteristics of the cell region in a state that is easy to visually recognize. be able to.

  Further, for example, as shown in FIG. 12A, the image display control unit 6b clearly indicates the position of the cell region with low extraction reliability on the observation image with a frame-type image mark and is surrounded by the image mark. The integrated region data of the cell region is superimposed and displayed as text data. In this way, the image display controller 6b can easily visually recognize the cell region having the predetermined feature to the observer by clearly indicating the region position of the cell region having the predetermined feature, such as the low extraction reliability. In addition, by simultaneously displaying the integrated region data of the cell region specified by the image mark, it is possible to easily grasp information regarding extraction and tracking of the cell region of interest.

  Further, as shown in FIG. 12-2, the image display control unit 6b can also display a region extraction image and a cell tracking image instead of the observation image in FIG. The image display control unit 6b may change the hue of the display color of the image mark, the display brightness, and the like according to the recognition reliability, the cell group, the success / failure determination item of the cell recognition process, and the like. The observer can more easily grasp various characteristics of the cell region of interest.

  For example, FIGS. 12-1 and 12-2 illustrate cases where the display color is changed according to the type of success / failure determination item that is the reason for the failure. In FIG. 12A, in order to display that the factor determined that the recognition of the cell region has failed is a success / failure determination item indicating whether or not the area of the cell region is smaller than the threshold value, The position of the cell region is displayed with a red frame corresponding to the success / failure determination item. In FIG. 12-2, in order to display that the factor determined that the recognition of the cell region has failed is a success / failure determination item (see FIG. 6-1) indicating whether or not an internal cell exists. The position of the cell region is indicated by a blue frame corresponding to the success / failure determination item.

  Further, for example, as illustrated in FIG. 13, the image display control unit 6 b removes a cell region having a low recognition reliability and displays only a cell region having a high recognition reliability. As a result, the image display control unit 6b can cause the observer to observe only the cell region where the observation can be performed with high reliability.

  Further, the image display control unit 6b is not limited to these, and various types of images such as an observation image, a cell extraction image, a cell feature image, a cell tracking image, and a grouped image are combined with the integrated region data in various combinations and display forms. It is possible to display the observed cell region based on the recognition success / failure information, the recognition reliability, and the like so that the observer can easily recognize and observe the cell region.

  As described above, in the image processing apparatus 1 according to the first embodiment, the extraction success / failure information as the determination result on the success or failure of the recognition of the cell region in the cell recognition process (cell extraction process or identity determination process) Record the recognition failure information such as extraction failure reason, tracking success / failure information, tracking failure reason, extraction reliability, tracking reliability, cell group information, etc. It can be used later. Thereby, it is possible to effectively use the result of the determination in the cell recognition process.

  In the reliability calculation process in step S111 described above, the reliability calculation unit 2f calculates a comprehensive recognition reliability for the cell extraction process in step S101 and the identity determination process in step S107. However, the extraction reliability indicating the reliability of the cell region extraction in the cell extraction process of step S101 and the tracking reliability indicating the reliability of the identity determination in the identity determination process of step S107 are individually calculated. You can also

That is, reliability calculation 2f, in Formula (5), the total number of items SN _all the total number of items of judging success or failure of the item when it is judged by the extraction result determining unit 2c of the success or failure of extraction of cell areas, the number of items SN _j The extraction reliability can be calculated as the number of success / failure determination items for which extraction has failed. Similarly, the reliability calculation 2f uses the total number of items SN _all in Formula (5) as the total number of items of success / failure determination items when the success / failure determination of the cell region is determined by the tracking result determination unit 2d, The tracking reliability can be calculated by using the number of items SN _j as the number of success / failure determination items determined to be unsuccessful.

Further, the reliability calculation unit 2f uses the expression reliability as an expression reliability instead of the recognition reliability R _{gi using the} number G of cell groups as the number of extracted cell groups and the total number AC of the extracted cell regions in the equation (6). Cell regions satisfying the relationship of (6) can be classified as extracted cell groups. Similarly, the reliability calculation unit 2f uses the equation (6) to track the number of cell groups G as the number of cell groups to be tracked and the total number AC as the total number of cell regions determined to be identical, instead of the recognition reliability R _gi. A cell region whose reliability satisfies the relationship of Expression (6) can be classified as a tracking cell group. And the reliability calculation part 2f can produce | generate an extraction group image corresponding to an extraction cell group, and can produce | generate a tracking grouping image corresponding to a tracking cell group.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. In the first embodiment described above, each image selected by the display item selection information from the observed image and various processed images is displayed. In the second embodiment, the selected cell region is further displayed. Are extracted and displayed.

  FIG. 14 is a block diagram showing a configuration of the image processing apparatus 11 according to the second embodiment of the present invention. As illustrated in FIG. 14, the image processing apparatus 11 includes an input unit 14 and a control unit 16 instead of the input unit 4 and the control unit 6 included in the image processing apparatus 1. The input unit 14 includes a display cell selection unit 14 c and a correction data input unit 14 d in addition to the configuration of the input unit 4. The control unit 16 includes an image display control unit 16b and a cell data integration unit 16c instead of the image display control unit 6b and the cell data integration unit 6c provided in the control unit 6. Other configurations are the same as those of the first embodiment, and the same reference numerals are given to the same components.

  The display cell selection unit 14c as selection information input means receives input of cell selection information for selecting a cell region corresponding to at least one living cell among the living cells included in the observation image. The image display control unit 16b performs control for extracting and displaying the cell region selected by the cell selection information from each image selected by the display item selection information, for example, as shown in FIG. FIG. 15 shows a case where a cell region selected from the grouped image shown in FIG. 13 is extracted and displayed.

  By selectively displaying the cell region in this way, the image display control unit 16b displays only the cell region corresponding to the live cell that the observer is interested in, and makes the observer more easily and efficiently observed. Can be made. The observer can arbitrarily select and display a cell region of a live cell of interest from the observed image and various processed images by inputting cell selection information via the display cell selection unit 14c.

  The correction data input unit 14d receives input of correction data for correcting the integrated region data corresponding to the cell region selected by the cell selection information. The cell data integration unit 16c corrects and updates the integrated region data based on the input correction data. The observer can appropriately correct a part or all of the integrated region data of the live cell of interest by inputting the correction data via the correction data input unit 14d.

  In Embodiments 1 and 2 described above, a series of observation images input from the observation image input unit 4a and stored in the observation image storage unit 5a are processed. However, the observation image input unit 4a receives the observation image. The processing may be performed as appropriate every time.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. It is a figure which shows the observation image input into the image processing apparatus shown in FIG. 3 is a flowchart illustrating a processing procedure performed by the image processing apparatus illustrated in FIG. 1. It is a flowchart which shows the process sequence of the cell extraction process shown in FIG. It is a figure which shows an area | region label image. It is a figure which shows the failure cell model in which an internal cell exists. It is a figure which shows the failure cell model in which a hole exists. It is a figure which shows the failure cell model in which a micro area | region exists. It is a flowchart which shows the process sequence of the extraction result determination process shown in FIG. It is a figure which shows a tracking label image. It is a figure which shows the case where the variation | change_quantity of a feature-value is large and it determines with the identity determination process failing. It is a figure which shows the case where an identity determination process is determined to be unsuccessful due to an inappropriate state change situation. It is a figure which shows the cell feature image which graphed the time-dependent change of the cell feature-value. It is a figure which shows the grouped image, observation image, and area | region extraction image of a cell area | region with low reliability. It is a figure which shows the observation image which attached | subjected the image mark which shows an area | region position. It is a figure which shows the area | region extraction image which attached | subjected the image mark which shows an area | region position. It is a figure which shows the grouped image which displayed only the cell area | region with high reliability. It is a block diagram which shows the structure of the image processing apparatus concerning Embodiment 2 of this invention. It is a figure which shows the grouped image which extracted the cell area | region selected by cell selection information.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,11 Image processing apparatus 2 Image processing part 2a Cell extraction part 2b Feature-value calculation part 2c Extraction result determination part 2d Cell tracking part 2e Tracking result determination part 2f Reliability calculation part 3 Display part 4,14 Input part 4a Observation image input Unit 4b display item selection unit 5 storage unit 5a observation image storage unit 5b processing image storage unit 5c cell region data storage unit 5d cell tracking data storage unit 6 control unit 6a image processing control unit 6b, 16b image display control unit 6c, 16c cell Data integration unit 14c Display cell selection unit 14d Correction data input unit

Claims (21)

In an image processing apparatus that processes and displays an observation image in which live cells are recorded,
A cell recognition means for performing a cell recognition process for recognizing a cell region corresponding to the living cell from the observed image;
Feature quantity calculating means for calculating a cell feature quantity indicating the characteristics of each cell region recognized by the cell recognition means;
Recognition success / failure determination means for determining whether or not the recognition of the cell region in the cell recognition process is successful based on a predetermined determination criterion;
Recognition success / failure storage means for storing recognition success / failure information indicating a result of the determination in association with a cell region to be determined;
An image processing apparatus comprising: The cell recognition means has a cell extraction means for performing a cell extraction process for extracting the cell region from the observation image,
The predetermined criterion includes a predetermined region criterion for the cell image region,
The recognition success / failure determination means determines the success or failure of the extraction of the cell region in the cell extraction process based on the predetermined region determination criterion,
The image processing apparatus according to claim 1, wherein the recognition success / failure storage unit stores extraction success / failure information as the recognition success / failure information indicating a result of the determination in association with a cell region to be determined. The predetermined region determination criterion is the suitability of the cell region with respect to a cell model region that is a predetermined image region,
The image processing apparatus according to claim 2, wherein the extraction success / failure determination unit determines the success or failure of the extraction of the cell region based on the suitability. The observation images are a plurality of observation images taken at different observation time points,
The cell recognition means includes
Cell extraction means for extracting the cell region from each observation image in the plurality of observation images;
A change amount of the cell feature amount between observation images with different observation time points is calculated, and based on the calculated change amount, live cells indicated in the cell regions included in the observation images with different observation time points are calculated. Cell tracking means for determining whether or not there is identity, and performing identity determination processing for associating cell regions determined to be identical;
Have
The predetermined criterion includes a predetermined identity criterion for the identity;
The recognition success / failure determination means determines the success or failure of the determination as to whether or not there is identity in the identity determination processing based on the predetermined identity determination criterion,
The image processing apparatus according to claim 1, wherein the recognition success / failure storage unit stores tracking success / failure information as the recognition success / failure information indicating a result of the determination in association with a cell region to be determined. The predetermined identity determination criterion includes a feature amount evaluation value calculated using a variation amount of the cell feature amount between observation images at different observation times, and a state change situation indicating a state change of the cell region. At least one,
5. The image processing according to claim 4, wherein the tracking success / failure determination unit determines whether the determination is successful or not based on at least one of the feature value evaluation value and the state change status. apparatus. The predetermined identity determination criterion is a feature amount evaluation value calculated using a variation amount of the cell feature amount between observation images at different observation points,
5. The image processing according to claim 4, wherein the tracking success / failure determination unit determines whether the determination is successful or a degree of success / failure according to a magnitude relationship between the feature value evaluation value and a predetermined threshold value. apparatus.   7. A display control unit that performs control to display at least one of the observation image, the cell feature amount, and a recognition result by the cell recognition unit together with the recognition success / failure information. The image processing apparatus according to any one of the above. The feature amount calculating means calculates a region position of each cell region as the cell feature amount,
Display control means for performing control to display at least one of the observation image and the recognition result by the cell recognition means together with the region position of the cell region in which the recognition success / failure information indicates a predetermined result; The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   The said display control means performs the control which displays the said area | region position of the cell area | region where the said recognition success / failure information has shown the predetermined result with the image mark surrounding this area | region position. Image processing device.   A reliability calculation means for calculating a recognition reliability indicating a degree of success or failure of recognition of the cell region in the cell recognition processing based on at least one of the determination contents by the recognition success / failure determination means and the cell feature amount; The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   The display control means for performing control to display at least one of the observation image, the cell feature amount, and a recognition result by the cell recognition means together with the recognition reliability. Image processing apparatus.   The image processing apparatus according to claim 10, further comprising a display control unit that performs control to display the recognition success / failure information together with the recognition reliability. The reliability calculation means classifies each cell region into one or more cell groups according to the calculated recognition reliability,
And a display control unit that performs control to display at least one of the observation image, the cell feature amount, and a recognition result by the cell recognition unit together with a classification result by the reliability calculation unit. The image processing apparatus according to claim 10. The reliability calculation means classifies each cell region into one or more cell groups according to the calculated recognition reliability,
The image processing apparatus according to claim 10, further comprising a display control unit that performs control to display the recognition success / failure information together with a classification result by the reliability calculation unit. The reliability calculation means classifies each cell region into one or more cell groups according to the calculated recognition reliability,
The feature amount calculating means calculates a region position of each cell region as the cell feature amount,
The display control means for performing control to display at least one of the observation image and the recognition result by the cell recognition means together with the region position of the cell region belonging to the predetermined cell group. The image processing apparatus according to 10.   The image processing apparatus according to claim 15, wherein the display control unit performs control to display the region position of a cell region belonging to the predetermined cell group with an image mark surrounding the region position. Comprising selection information input means for receiving cell selection information for selecting the cell region;
The said display control means performs the control which selectively displays the cell area | region selected by the said cell selection information as a display object, Any one of Claims 7-9 or Claims 11-16 characterized by the above-mentioned. The image processing apparatus described.   The feature amount calculating means calculates at least one of a luminance sum, an area, a circularity, a centroid position, a contour length, and a contour position of each cell region as the cell feature amount. The image processing apparatus according to any one of the above. An image processing program for processing and displaying the observed image on an image processing apparatus that processes and displays the observed image recorded with live cells,
In the image processing apparatus,
A cell recognition procedure for performing a cell recognition process for recognizing a cell region corresponding to the living cell from the observed image;
A feature amount calculation procedure for calculating a cell feature amount indicating a feature of each cell region recognized by the cell recognition procedure;
Based on a predetermined determination criterion, a recognition success / failure determination procedure for determining the success or failure of recognition of a cell region in the cell recognition process,
A recognition success / failure storage procedure for storing recognition success / failure information indicating a result of the determination in association with a cell region to be determined;
An image processing program for executing The cell recognition procedure includes a cell extraction procedure for performing a cell extraction process for extracting the cell region from the observation image,
The predetermined criterion includes a predetermined region criterion for the cell image region,
The recognition success / failure determination procedure is based on the predetermined region determination criterion, and determines the success or failure of the cell region extraction in the cell extraction process,
The image processing program according to claim 19, wherein the recognition success / failure storage procedure stores extraction success / failure information as the recognition success / failure information indicating a result of the determination in association with a cell region to be determined. The observation images are a plurality of observation images taken at different observation time points,
The cell recognition procedure includes:
A cell extraction procedure for extracting the cell region from each observation image in the plurality of observation images;
A change amount of the cell feature amount between observation images with different observation time points is calculated, and based on the calculated change amount, live cells indicated in the cell regions included in the observation images with different observation time points are calculated. A cell tracking procedure for determining whether or not there is identity, and performing identity determination processing for associating cell regions determined to be identical;
Have
The predetermined criterion includes a predetermined identity criterion for the identity;
The recognition success / failure determination procedure is based on the predetermined identity determination criteria, and determines the success or failure of the determination as to whether or not there is identity in the identity determination process,
The image processing program according to claim 19, wherein the recognition success / failure storage procedure stores tracking success / failure information as the recognition success / failure information indicating a result of the determination in association with a cell region to be determined.

Images