JP2016027826A - Image processing apparatus, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quantitatively evaluate cooperativity of a motion of a subject to be evaluated.SOLUTION: A motion detection unit of an evaluation index data generation unit detects motions of the subject to be evaluated for each block by using image data on the subject to be evaluated obtained by an imaging unit to take images of the subject to be evaluated. A correlation calculation unit calculates a correlation coefficient of motion amounts of the subject to be evaluated between blocks and further, calculates the average value. An evaluation unit normalizes the correlation coefficient between the blocks by a predetermined function, calculates distribution, normalizes the distribution by a predetermined function, evaluates the correlation of the motions of the subject to be evaluated between the blocks by using the average value and a threshold, and calculates an evaluation value. The disclosure can be applied to an image processing apparatus.SELECTED DRAWING: Figure 4

Description

  The present disclosure relates to an image processing apparatus and method, and a program, and more particularly, to an image processing apparatus and method, and a program capable of non-invasively and quantitatively evaluating the cooperativity of movement of an evaluation target.

  In the field of regenerative medicine, the use of cultured cells produced by culturing cells is used to regenerate body cells, tissues, organs, etc. lost due to accidents or illness, and to restore their functions. ing. There are a wide variety of cell tissues that can be produced as such cultured cells, and one of them is a cardiomyocyte, which is used for the treatment of the heart. This cultured cardiomyocyte itself moves corresponding to a pulsation. Therefore, in the production stage of cultured cardiomyocytes, it is necessary to evaluate the quality of whether the above movement is good, for example.

  In evaluating the quality of such cultured cardiomyocytes, for example, in the present situation, visual observation is performed. In addition, the potential is measured by inserting an electrode into a cultured cardiomyocyte. However, visual observation is largely dependent on the subjectivity of the observer, and it is difficult to obtain an objective and accurate evaluation result. Further, when measuring the potential, there is a problem that the electrode is in contact with the cultured cardiomyocytes and is not non-invasive. Further, information that can be quantified based on the measurement by the potential is limited to, for example, about the pulsation time. Furthermore, the measurement object is limited on the electrode.

  Therefore, as a conventional technique, a measurement point is set in an imaging screen obtained by photographing cardiomyocytes, and the luminance of the measurement point is automatically measured, and the deformation period of the cardiomyocytes is measured from the measurement value. Is known (see, for example, Patent Document 1).

JP-A-63-233392 (FIG. 1)

  However, in the case of the method described in Patent Document 1, since the periodic change in luminance is the measurement object, there is a possibility that the measurement is limited to the time interval of the pulsation cycle. In other words, although it is non-invasive, the information that can be quantified remains in the cycle of pulsation, it remains the same problem as when measuring potential, and it is difficult to obtain accurate evaluation results. There was a fear.

  For example, when evaluating cardiomyocytes cultured in regenerative medicine or the like, it is desired to evaluate whether or not the movement of each part of the cultured cells of the cardiomyocytes is cooperative, but in the method described in Patent Document 1, There is a fear that it is difficult to quantitatively evaluate the cooperativity of the movement of the object to be evaluated in a non-invasive manner.

  The present disclosure has been made in view of such a situation, and an object thereof is to allow non-invasive and quantitative evaluation of the cooperativity of a motion of an evaluation object to be exercised.

  The image processing apparatus according to an aspect of the present technology uses the motion information indicating the motion of the evaluation target calculated with respect to the plurality of partial regions constituting the evaluation target image collected from the living body, to Image processing having a control unit that calculates correlation of temporal changes in the motion information of a part or all of the partial area pairs, and controls to display the correlation information indicating the correlation of the calculated partial area pairs as an image Device.

  The image processing apparatus further includes an imaging unit that captures the evaluation target and obtains the evaluation target image, and the control unit uses motion information indicating movement of the evaluation target in the evaluation target image obtained by the imaging unit. Thus, the correlation of the temporal change of the motion information between the partial areas can be calculated.

  The said control part can repeat the calculation of the correlation of the time change of the motion information between the said partial areas in multiple times.

  The evaluation object can be a cell that moves spontaneously.

  The evaluation object may be a cultured cell generated by culturing a cell collected from a living body.

  The control unit can image the correlation information so that the correlation strength of each partial region pair is represented by a color difference.

  The evaluation object can be a cardiomyocyte.

  The control unit can control the correlation information to be imaged and displayed as a 3D plot drawn in a three-dimensional space.

  The said control part can be controlled to image and display the said correlation information as a line graph, a bar graph, a distribution diagram, or a schematic diagram.

  The image processing method according to one aspect of the present technology uses the motion information indicating the motion of the evaluation target calculated with respect to a plurality of partial regions constituting the evaluation target image collected from a living body, This is an image processing method for calculating a correlation of temporal changes in motion information for some or all partial region pairs, and controlling the calculated correlation information indicating the correlation of each partial region pair to be imaged and displayed.

  The correlation of the temporal change of the motion information between the partial regions can be calculated using the motion information indicating the motion of the evaluation target in the evaluation target image obtained by imaging the evaluation target.

  The calculation of the temporal change correlation of the motion information between the partial areas can be repeated a plurality of times.

  The evaluation object can be a cell that moves spontaneously.

  The evaluation object may be a cultured cell generated by culturing a cell collected from a living body.

  The correlation information can be imaged so that the correlation strength of each partial region pair is represented by a color difference.

  The evaluation object can be a cardiomyocyte.

  The correlation information can be controlled to be imaged and displayed as a 3D plot drawn in a three-dimensional space.

  The correlation information can be controlled to be imaged and displayed as a line graph, a bar graph, a distribution diagram, or a schematic diagram.

  A program according to an aspect of the present technology uses the motion information indicating the motion of the evaluation target calculated with respect to a plurality of partial regions constituting an evaluation target image collected from a living body. The correlation of the temporal change of the motion information is calculated for a part or all of the partial region pairs, and functions as a control unit that controls to display the correlation information indicating the correlation of each calculated partial region pair as an image. It is a program for.

  In one aspect of the present technology, the motion information indicating the motion of the evaluation target calculated for a plurality of partial regions constituting the evaluation target image collected from the living body is used, and the motion information between the partial regions is calculated. The correlation of the temporal change is calculated for some or all of the partial region pairs, and the correlation information indicating the calculated correlation of each partial region pair is imaged and displayed.

  According to the present disclosure, an image can be processed. In particular, the cooperativity of the movement of the evaluation object can be quantitatively evaluated non-invasively.

It is a figure explaining the cooperation of a motion. It is a figure explaining the cooperation of a motion. It is a block diagram which shows the main structural examples of a cultured cardiomyocyte evaluation apparatus. It is a block diagram which shows the main structural examples of an evaluation parameter | index data generation part. It is a figure explaining the structural example of evaluation object image data. It is a block diagram which shows the main structural examples of a motion detection part. It is a figure explaining the example of the block division of frame image data. It is a figure explaining the structural example of motion detection data. It is a block diagram which shows the main structural examples of a correlation calculation part. It is a figure explaining the correlation of the amount of movement. It is a figure explaining the structural example of a motion correlation data log | history. It is a block diagram which shows the main structural examples of an evaluation part. It is a figure explaining the example of the function of normalization. It is a flowchart explaining the example of the flow of an evaluation process. It is a flowchart explaining the example of the flow of an evaluation parameter | index data generation process. It is a flowchart explaining the example of the flow of a correlation evaluation process. It is a block diagram which shows the other structural example of an evaluation part. It is a figure explaining other examples of evaluation of correlation. It is a flowchart explaining the other example of the flow of a correlation evaluation process. It is a block diagram which shows the main structural examples of a chemical | medical agent evaluation apparatus. It is a figure explaining the example of a block. It is a figure explaining the example of the influence on the correlation between the blocks by chemical | medical agent administration. It is a block diagram which shows the main structural examples of an evaluation parameter | index data generation part. It is a block diagram which shows the main structural examples of an evaluation part. It is a flowchart explaining the example of the flow of an evaluation process. It is a flowchart explaining the example of the flow of an evaluation parameter | index data generation process. It is a flowchart explaining the example of the flow of an impact evaluation process. FIG. 26 is a block diagram illustrating a main configuration example of a personal computer.

Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be made in the following order.
1. First embodiment (cultured cardiomyocyte evaluation apparatus)
2. Second embodiment (cultured cardiomyocyte evaluation apparatus)
3. Third embodiment (drug evaluation apparatus)
4). Fourth embodiment (personal computer)

<1. First Embodiment>
[Cooperability of movement]
First, the cooperativity of the movements to be evaluated will be described. For example, in regenerative medicine, various human tissues and organs are treated using cultured cells. A cultured cell is a cell tissue produced by culturing cells. The cultured cell 1 shown in FIG. 1A is obtained by growing and growing a culture. For example, cultured cardiomyocytes, which are cultured cells obtained by culturing cardiomyocytes, are used for heart treatment and the like.

  At present, such a cultured cell 1 is mass-produced, and a technology development for enabling a sufficient amount to be supplied to a medical site at a low cost is in progress. When the cultured cells are mass-produced in this way, it is required to be able to evaluate the manufactured cells efficiently and accurately.

  The cultured cell 1 is generated by culturing cardiomyocytes collected from a living body. Assuming that the cultured cell 1 is a cultured cardiomyocyte (a cultured cardiomyocyte), the cardiomyocyte constantly beats while repeating contraction and relaxation. Therefore, in this case, in order to evaluate the quality of the cultured cell 1, the movement of the cardiomyocyte of the cultured cell 1 is evaluated. In the cultured cell 1 which is a cultured cardiomyocyte, the cells in each part move in a predetermined direction, for example, like a motion vector 2 shown in FIG. 1B, so that the entire cell repeatedly contracts and relaxes.

  Therefore, the observation area of the cultured cell 1 is divided into a plurality of partial areas (blocks) as shown in FIG. 1C, a motion amount (motion vector) is detected for each block, and its temporal transition is observed.

  For example, a graph 4-1 in FIG. 1C shows a temporal transition of the motion amount of the block 3-1, and a graph 4-2 shows a temporal transition of the motion amount of the block 3-2. . The correlation (cooperativity) of cell movement in each of these blocks is evaluated.

  Graphs 5-1 to 5-3 in FIG. 2A show the amount of cell movement in block 3-1 shown in graph 4-1 and the amount of cell movement in block 3-2 shown in graph 4-2. It shows the temporal transition of the relationship.

  The amount of cell movement initially present in the block 3-1 and the amount of cell movement present in the block 3-2 are low in correlation with each other as shown in the graph 5-1, when collected from the living body. It is. However, as time passes and the culture progresses, the correlation between the two gradually increases as shown in graph 5-2, and as time passes, the correlation between the two increases as shown in graph 5-3. Sex is very strong.

  That is, as in the graph shown in FIG. 2B, the correlation coefficient of the amount of movement between the plurality of positions of the cultured cell 1 is gradually stabilized at a large value. That is, the cooperativity of the movement of cells in each region is strengthened. Ideally, the movements of the cells are related to each other, and the entire cultured cell 1 is pulsated as one living tissue.

  On the other hand, when the culture does not go well and the quality of the cultured cell 1 is incomplete, the cooperativity of the cells of each part does not improve, the pulsation is weak, each part moves apart, or does not move Or

  That is, the evaluation of the cooperativeness of the movement of each part of the cultured cell 1 is established as one of the evaluation methods for the quality of the cultured cell 1. Then, by calculating the strength of cooperation as an evaluation value, it becomes possible to evaluate quantitatively and correctly.

[Cultured cardiomyocyte evaluation device]
FIG. 3 is a block diagram illustrating a main configuration example of the cultured cardiomyocyte evaluation apparatus.

  The cultured cardiomyocyte evaluation apparatus 100 shown in FIG. 3 is an apparatus that evaluates the cooperativeness of the movement of the cultured cardiomyocytes 110. As illustrated in FIG. 3, the cultured cardiomyocyte evaluation apparatus 100 includes an imaging unit 101, an evaluation target image data generation recording unit 102, an evaluation index data generation unit 103, and an evaluation unit 104.

  The imaging unit 101 images the cultured cardiomyocytes 110 to be evaluated. The imaging unit 101 may image the cultured cardiomyocytes 110 directly (without passing through other members), or may image the cultured cardiomyocytes 110 through other members such as a microscope.

  Moreover, the cultured cardiomyocytes 110 may be fixed with respect to the imaging unit 101 or may not be fixed. In general, it is desirable that the cultured cardiomyocyte 110 is fixed to the imaging unit 101 in order to detect the movement (temporal change in position) of the cultured cardiomyocyte evaluation apparatus 100.

  The imaging unit 101 supplies an image signal of an image of the cultured cardiomyocytes 110 obtained by imaging to the evaluation target image data generation / recording unit 102.

  The evaluation target image data generation / recording unit 102 generates evaluation target image data based on the image signal supplied from the imaging unit 101, and records and stores the generated evaluation target image data in, for example, an internal recording medium. The evaluation object image data generated here is, for example, moving image data generated from an image signal obtained by imaging the cultured cardiomyocytes 110.

  For example, the evaluation target image data generation / recording unit 102 may extract only a part of the frame images from a plurality of frame images supplied from the imaging unit 101 and use them as evaluation target image data. Good. Further, for example, the evaluation target image data generation / recording unit 102 extracts a partial region of each frame image supplied from the imaging unit 101 as a small frame image, and a moving image including the small frame image is evaluated You may make it. Further, for example, the evaluation target image data generation / recording unit 102 may perform arbitrary image processing on each frame image supplied from the imaging unit 101, and the image processing result may be used as evaluation target image data. . As the image processing, for example, image enlargement, reduction, rotation, deformation, brightness and chromaticity correction, sharpness, noise removal, intermediate frame image generation, and the like can be considered. Of course, any image processing other than these may be used.

  The evaluation target image data generation recording unit 102 supplies the stored evaluation target image data to the evaluation index data generation unit 103 at a predetermined timing.

  The evaluation index data generation unit 103 performs an evaluation target for each block that is a partial region obtained by dividing the entire region of the image of the evaluation target (cultured cardiomyocytes 110) into a plurality of frames between the frame images of the supplied evaluation target image data. Motion detection of (cultured cardiomyocytes 110) is performed.

  The evaluation index data generation unit 103 represents the detected motion of each block as a motion vector, and obtains the correlation of the motion of each block of the evaluation target (cultured cardiomyocytes 110) from the motion vector. Further, the evaluation index data generation unit 103 generates evaluation index data that is an index for evaluating the cooperativity of the motion of each block based on the correlation of the motion.

  The evaluation index data generation unit 103 supplies the evaluation index data generated as described above to the evaluation unit 104.

  The evaluation unit 104 evaluates the supplied evaluation index data, calculates and outputs a cooperative evaluation value 114 for the movement of the cultured cardiomyocytes 110.

  The evaluation target of the cultured cardiomyocyte evaluation apparatus 100 may be other than the cultured cardiomyocytes 110. For example, cell sheets of cells other than cardiomyocytes may be evaluated. Of course, the evaluation target may be other than cells. However, it is desirable that the evaluation object is one that can move and that can be evaluated by evaluating the cooperativeness of the movement. This movement may be autonomous (spontaneous) like a cardiomyocyte, or may be due to an electric signal supplied from the outside.

[Evaluation index data generator]
FIG. 4 is a block diagram illustrating a main configuration example of the evaluation index data generation unit 103 in FIG. As illustrated in FIG. 4, the evaluation index data generation unit 103 includes a motion detection unit 121, a motion detection data storage unit 122, a correlation calculation unit 123, and a correlation data history storage memory 124.

  The motion detection unit 121 receives the evaluation target image data 112 recorded from the evaluation target image data generation recording unit 102, performs motion detection for each block, and uses the detection result (motion vector) as motion detection data to detect motion. The data is supplied to the data storage unit 122 and stored.

  The correlation calculation unit 123 uses the motion detection data stored in the motion detection data storage unit 122 to calculate a correlation coefficient of motion between blocks, and uses it as correlation data in the correlation data history storage memory 124. Supply and memorize.

  The correlation data history storage memory 124 holds the calculated correlation coefficient as correlation data during calculation of the correlation coefficient that is repeatedly performed a predetermined number of times. The correlation data history storage memory 124 supplies the held correlation data to the evaluation unit 104 as evaluation index data 113 at a predetermined timing, for example.

[Structure of image data to be evaluated]
FIG. 5 shows a structural example of the evaluation target image data 112 supplied to the evaluation index data generation unit 103. The evaluation of motion cooperativity is performed for each evaluation section of a predetermined length (for example, T + 1 frame (T is an arbitrary natural number)). Therefore, the evaluation target image data 112 supplied to the evaluation index data generation unit 103 includes the first to (T + 1) th frame image data 132-1 to 132- (T + 1) corresponding to the evaluation section.

[Configuration example of motion detection unit]
FIG. 6 is a block diagram illustrating a main configuration example of the motion detection unit 121. As illustrated in FIG. 6, the motion detection unit 121 includes a frame memory 141 and a motion vector calculation unit 142. The frame memory 141 holds frame image data 132 that is sequentially input as the evaluation target image data 112 for each frame period.

  The motion vector calculation unit 142 obtains the frame image data input as the evaluation target image data 112 at the current time and the frame image data at the next time (previous in time) held in the frame memory 141. input. Then, a motion vector indicating the motion between these two frame image data is calculated for each block. The calculated motion vector is held in the motion detection data storage unit 122 as motion detection data 151.

  The process executed by the motion detection unit 121 in FIG. 6 will be described in more detail. The motion vector calculation unit 142 receives the frame image data 132 at the current time and the frame image data 132 at the next (temporally previous) time. The motion vector calculation unit 142 divides the input frame image data 132 into M × N blocks 161 (M and N are arbitrary natural numbers) as shown in FIG. For example, motion detection is performed by a technique such as block matching between frame images to generate a motion vector. Each of the blocks 161 is composed of, for example, pixels of (16 × 16).

  The motion vector calculation unit 142 executes this motion detection processing by sequentially using the first to (T + 1) th frame image data 132. That is, the motion vector calculation unit 142 generates (M × N × T) motion detection data (motion vectors) using (T + 1) frame images. The motion vector calculation unit 142 supplies the motion vector calculated in this way as motion detection data to the motion detection data storage unit 122 for storage.

  When the last motion detection process using the Tth and (T + 1) th frame image data 132 is completed, the motion detection data storage unit 122 stores T frame-unit motion detection data 171 as shown in FIG. Motion detection data consisting of -1 to 171-T is stored.

  Each of the frame unit motion detection data 171-1 to 171 -T moves with respect to the frame image data 132 at the current time obtained for each frame period and the frame image data 132 ahead (in time). It is obtained by performing the detection process.

  For example, the third frame unit motion detection data 171-3 includes the fourth frame image data 132-4 and the third frame image data 132-3 as frame image data at the current time and one time ahead, respectively. It can be obtained by inputting and performing motion detection.

  Each of the frame corresponding motion detection data 171-1 to 171-T is formed by (M × N) block unit motion detection data 181. Each block unit motion detection data 181 corresponds to one block 161, and is data indicating a motion vector detected for the corresponding block 161.

  As described above, the motion detection data 151 according to the present embodiment has a structure having (M × N) block unit motion detection data 181 for each frame corresponding motion detection data 171.

[Correlation calculator]
FIG. 9 is a block diagram illustrating a main configuration example of the correlation calculation unit 123 of FIG. As illustrated in FIG. 9, the correlation calculation unit 123 includes an inter-block correlation coefficient calculation unit 201, a correlation coefficient storage unit 202, and an average value calculation unit 203.

The inter-block correlation coefficient calculation unit 201 calculates the inter-block motion correlation coefficient C using the motion detection data 151 for one evaluation section read from the motion detection data storage unit 122. For example, the inter-block correlation coefficient calculation unit 201 calculates the correlation coefficient C _{a, b} between the block A and the block B as in the following equation (1).

・・・（１）

... (1)

  In equation (1), Va (k) indicates the amount of motion of block A in the frame at time k. Moreover, Va (k) with an overline shows the average value of the movement amount of the block A in the time series direction within the evaluation section. Further, Vb (k) indicates the amount of motion of block B in the frame at time k. Further, Vb (k) with an overline indicates an average value of the motion amount of the block B in the time series direction in the evaluation section.

  That is, the inter-block correlation coefficient calculation unit 201 obtains the degree of correlation of the change in the amount of motion in the evaluation section between two blocks, as shown in FIG.

The inter-block correlation coefficient calculation unit 201 calculates such a correlation coefficient C for all blocks (a combination of all blocks). For example, when the inter-block correlation coefficient calculation unit 201 calculates L correlation coefficients C _{a, b} , L is calculated as in the following equation (2).

・・・（２）

... (2)

The correlation coefficient storage unit 202 stores L correlation coefficients C _{a, b} calculated by the inter-block correlation coefficient calculation unit 201.

  The average value calculation unit 203 calculates the average correlation coefficient Cave, which is the average value of the correlation coefficients between the blocks calculated as described above, as in the following equation (3).

・・・（３）

... (3)

The average value calculation unit 203 stores each correlation coefficient C _{a, b} calculated by the inter-block correlation coefficient calculation unit 201 and the average correlation coefficient Cave calculated by itself as correlation data 211 in the correlation data history storage memory 124. Supply and memorize. That is, as shown in FIG. 11, (L + 1) pieces of correlation data 211 are stored in the correlation data history storage memory 124 for each evaluation section.

  The evaluation index data generation unit 103 repeats the generation of evaluation index data as described above a predetermined number of times (for example, S times) (S is an arbitrary natural number). That is, the imaging unit 101 continues imaging and generates frame images for at least (evaluation interval (T + 1 frame) × S times), and the evaluation target image data generation recording unit 102 at least (evaluation interval × S Times) of evaluation target image data. In the evaluation object image data, the evaluation sections do not have to be continuous in time.

  For example, assume that the period from the start of culture to the end of culture is 10 days, and T = 600 frames are imaged every 2 hours for evaluation. In this case, each evaluation section is 600 frames, and the evaluation section is repeated S = 120 times.

  The correlation calculation unit 123 generates the correlation data 211 as described above for each evaluation section. As a result, (L + 1) × S pieces of correlation data 211 are stored in the correlation data history storage memory 124 as shown in FIG.

[Evaluation Department]
As described above, the correlation data 211 for the evaluation section S stored in the correlation data history storage memory 124 is supplied to the evaluation unit 104 as the evaluation index data 113.

  FIG. 12 is a block diagram illustrating a main configuration example of the evaluation unit 104. As illustrated in FIG. 12, the evaluation unit 104 includes a correlation coefficient normalization unit 221, a variance calculation unit 222, a variance normalization unit 223, and an evaluation value calculation unit 224.

The correlation coefficient normalization unit 221 converts each correlation coefficient C _{a, b} that is the evaluation index data 113 to a function fc like the curve 231 of the graph shown in FIG. 13A, as in the following equation (4). To normalize (determine correlation coefficient C ′ normalized by function fc).

・・・（４）

... (4)

For example, if L correlation coefficients C _{a, b} are calculated, the correlation coefficient normalization unit 221 normalizes each of the L correlation coefficients C _{a, b} using the function fa. .

The function fc may be any function as long as the value of the correlation coefficient C _{a, b} is larger as the value is larger and smaller as the value is smaller. That is, the normalized correlation coefficient C ′ takes a larger value as the motion amount correlation between the blocks is larger, and takes a smaller value as the motion amount correlation between the blocks is smaller.

The correlation coefficient normalization unit 221 supplies the normalized correlation coefficient C ′ and correlation coefficient C _{a, b} to the variance calculation unit 222.

The variance calculator 222 calculates the past N variances Vc for each correlation coefficient C _{a, b} as shown in the following equation (5).

・・・（５）

... (5)

In equation (5), the overlined C is the average value in the time axis direction within the evaluation interval of the correlation coefficient C _{a, b} . Further, when the calculation of the correlation coefficient C _{a, b} is repeated S times as described above, N = S. That is, the variance calculation unit 222 calculates L variances Vc of correlation coefficients in the entire frame image. The variance calculation unit 222 supplies the variance Vc of the correlation coefficient calculated in this way and the normalized correlation coefficient C ′ to the variance normalization unit 223.

  The variance normalization unit 223 normalizes the variance Vc of each correlation coefficient using a function gc like a curve 232 of the graph shown in FIG. 13B as in the following equation (6) (by the function gc). The variance Vc ′ of the normalized correlation coefficient is obtained).

・・・（６）

... (6)

  This function gc may be any function as long as the value of the correlation coefficient variance Vc is smaller as the value is larger and larger as the value is smaller. That is, the normalized correlation coefficient variance Vc ′ takes a larger value as the variation is smaller, and takes a smaller value as the variation is larger.

  The variance normalization unit 223 supplies the normalized correlation coefficient variance Vc ′ and the normalized correlation coefficient C ′ to the evaluation value calculation unit 224.

  As shown in the following equation (7), the evaluation value calculation unit 224 calculates an average value (L pieces of values) of the entire product of the product of the normalized correlation coefficient C ′ and the normalized correlation coefficient variance Vc ′. (Average value) is calculated as the evaluation value Ec.

・・・（７）

... (7)

  In this case, the evaluation value Ec increases as the normalized correlation coefficient in the entire frame image and the variance of the normalized correlation coefficient increase. That is, the higher the correlation coefficient between the blocks is, the higher the evaluation is made, the higher the correlation is (the correlation is larger and the variation in the time direction is smaller).

  Note that the evaluation value calculation unit 224 determines that the product value of the normalized correlation coefficient C ′ and the normalized correlation coefficient variance Vc ′ is equal to or greater than a predetermined threshold Tc1 as shown in the following equation (8). The ratio of the number Nc1 of correlation coefficients that become the value to the entire frame image (L) may be calculated as the evaluation value Ec.

・・・（８）

... (8)

  The threshold value Ta1 is an arbitrary value set in advance. The larger this value is set, the higher the evaluation standard (stricter evaluation conditions) and the smaller the evaluation value Ec. In this case, the evaluation value Ec increases as the correlation coefficient in which the product of the normalized correlation coefficient and the variance in the time direction thereof is more stable than a predetermined reference in the entire frame image increases. .

  That is, in this case, it is preferable that the variation of each correlation coefficient is smaller than the case where the evaluation value Ec is calculated using the average value as described above. For example, when evaluating the average value, the evaluation may be high even if the correlation coefficient varies greatly. On the other hand, when the evaluation is performed using the threshold value, the evaluation does not increase if Nc1 is not large even if some correlation coefficient values are extremely large.

In the above description, only the correlation coefficient C _{a, b} between the two blocks is evaluated. However, the present invention is not limited to this, and the evaluation unit 104 calculates the average correlation coefficient calculated by the average value calculation unit 203. Cave may also be evaluated. In that case, the evaluation unit 104 evaluates the average correlation coefficient Cave as in the case of the correlation coefficient C _{a, b} described above.

  That is, in this case, the evaluation value calculation unit 224 uses the average of the product of (L + 1) normalized correlation coefficients C ′ and the normalized correlation coefficient variance V ′ as the evaluation value Ec. calculate. Alternatively, the evaluation value calculation unit 224 determines the number Nc1 of correlation coefficients for which the product of the normalized correlation coefficient C ′ and the variance Vc ′ of the normalized correlation coefficient is equal to or greater than a predetermined threshold value Tc1. Is calculated as an evaluation value Ec in the entire frame image ((L + 1)).

  As described above, the evaluation unit 104 calculates the evaluation value Ec such that the value increases as the pulsation (movement) of each part of the cultured cardiomyocytes 110 increases, and the value decreases as the cooperation decreases. The evaluation value Ec is calculated so as to decrease. Thereby, the cultured cardiomyocyte evaluation apparatus 100 (evaluation unit 104) can quantitatively evaluate the pulsation (motion) cooperativity of each part of the cultured cardiomyocytes 110. Moreover, the cultured cardiomyocyte evaluation apparatus 100 detects an index for calculating such an evaluation value using image data obtained by imaging the cultured cardiomyocytes 110, for each part of the cultured cardiomyocytes 110. Since motion vectors are used, evaluation can be performed easily and non-invasively.

[Flow of evaluation process]
Next, an example of the flow of evaluation processing executed by the cultured cardiomyocyte evaluation apparatus 100 will be described with reference to the flowchart of FIG.

  When the evaluation process is started, the imaging unit 101 of the cultured cardiomyocyte evaluation apparatus 100 images the evaluation target in step S101. In step S102, the evaluation target image data generation / recording unit 102 generates evaluation target image data from the image signal obtained by imaging in step S101.

  In step S103, the evaluation index data generation unit 103 generates evaluation index data, which is index data for evaluating the cooperativity of the movement of the evaluation target, from the evaluation target image data generated in step S102. In step S104, the evaluation unit 104 uses the evaluation index data generated in step S103 to evaluate the correlation between the motion blocks to be evaluated, that is, the cooperation, and calculate an evaluation value.

  In step S105, the evaluation unit 104 outputs the evaluation value calculated in step S104, and ends the evaluation process.

[Flow of evaluation index data generation processing]
Next, an example of the flow of the evaluation index data generation process executed in step S103 of FIG. 14 will be described with reference to the flowchart of FIG.

  When the evaluation index data generation process is started, the motion detection unit 121 of the evaluation index data generation unit 103 detects a motion to be evaluated for each block and generates a motion vector in step S121. In step S122, the motion detection data storage unit 122 stores the motion vector of each block generated in step S121.

  In step S123, the motion detection unit 121 determines whether motion detection has been performed for a predetermined period (evaluation interval). When it is determined that there is a frame image for which motion detection has not been performed in the predetermined evaluation section, the motion detection unit 121 returns the process to step S121 and repeats motion detection for a new processing target frame image.

  If it is determined in step S123 that motion detection has been performed on all the frame images to be processed in the predetermined evaluation section, the motion detection unit 121 advances the process to step S124.

  In step S124, the inter-block correlation coefficient calculation unit 201 of the correlation calculation unit 123 calculates a correlation coefficient between the blocks from the motion vector (motion detection data) stored in step S122. In step S125, the correlation coefficient storage unit 202 of the correlation calculation unit 123 stores the correlation coefficient between the blocks calculated in step S124.

  In step S126, the inter-block correlation coefficient calculation unit 201 determines whether or not the correlation coefficients between all the blocks have been generated, and the process is performed until it is determined that the correlation coefficients between all the blocks have been generated. Returning to S124, the processing from step S124 to step S126 is repeated. If it is determined in step S126 that correlation coefficients between all blocks have been generated, the inter-block correlation coefficient calculation unit 201 advances the process to step S127.

  In step S127, the average value calculation unit 203 calculates an average correlation coefficient that is an average value of correlation coefficients between blocks calculated in step S124. In step S128, the correlation data history storage memory 124 stores the correlation coefficient between the blocks calculated in step S124 and the average correlation coefficient calculated in step S127 as correlation data.

  In step S129, the correlation calculation unit 123 determines whether the calculation of the correlation coefficient and the average correlation coefficient has been repeated a predetermined number of times. If it is determined that the calculation has not been repeated, the process returns to step S121, and the subsequent steps Repeat the process.

  If it is determined in step S129 that the calculation of the correlation coefficient and the average correlation coefficient has been repeated a predetermined number of times, the correlation calculation unit 123 advances the process to step S130.

  In step S130, the correlation data history storage memory 124 outputs the held correlation data as evaluation index data. When the process of step S130 ends, the evaluation index data generation unit 103 ends the evaluation index data generation process, returns the process to step S103 in FIG. 14, and executes the processes after step S104.

[Flow of correlation evaluation process]
Next, an example of the flow of correlation evaluation processing executed in step S104 in FIG. 14 will be described with reference to the flowchart in FIG.

  When the correlation evaluation process is started, the correlation coefficient normalization unit 221 of the evaluation unit 104 normalizes the correlation coefficient between the blocks in step S151. In step S152, the variance calculating unit 222 calculates the variance in the time direction of the correlation coefficient between the blocks.

  In step S153, the variance normalization unit 223 normalizes the variance of each correlation coefficient calculated in step S152. In step S154, the evaluation value calculation unit 224 calculates an evaluation value using the normalized correlation coefficient calculated in step S151 and the variance of the normalized correlation coefficient calculated in step S153. .

  When the evaluation value is calculated, the evaluation value calculation unit 224 ends the correlation evaluation process, returns the process to step S104 in FIG. 14, and executes the processes after step S105.

  As described above, by performing various processes, the cultured cardiomyocyte evaluation apparatus 100 can quantitatively evaluate the cooperativity of the evaluation target (for example, cell movement). Further, by using a motion vector for generating an index, it is possible to evaluate more easily and non-invasively.

<2. Second Embodiment>
[Other examples of evaluation section]
In addition, the evaluation method of cooperation is not restricted to the example mentioned above. For example, the pulsation of the cultured cardiomyocytes may be compared with that in an ideal normal culture, and the comparison result may be evaluated. In this case, an ideal pulsation transition pattern (ideal transition pattern) during normal culture is predetermined.

  FIG. 17 is a block diagram illustrating a main configuration of the evaluation unit 104 in this case. As shown in FIG. 17, in this case, the evaluation unit 104 includes a distance calculation unit 241, a distance normalization unit 242, and an evaluation value calculation unit 243.

  As shown in the graph of FIG. 18A, the distance calculation unit 241 compares the pulsation transition pattern (measurement transition pattern) of the cultured cardiomyocytes with this ideal transition pattern, and evaluates the similarity. In FIG. 18A, a solid line 251 indicates an ideal transition pattern of correlation between cell movement blocks, and a dotted line 252 indicates a measurement transition pattern between cell movement blocks. The smaller the difference between the two, the larger the evaluation value.

  The distance calculation unit 241 calculates the sum Dc of the distances between the two transition patterns at each elapsed time for each block as in the following Expression (9).

・・・（９）

... (9)

In Expression (9), C (k) is a correlation coefficient between blocks in the measurement transition pattern of the frame image at time k, and C _I (k) is in the ideal transition pattern of the frame image at time k. Correlation coefficient between blocks. k indicates the number of measurement values (elapsed time) (when S measurements are repeated, 0 ≦ k ≦ S−1). W _c (k) is a weighting coefficient, and its value is arbitrary. For example, immediately after the start of measurement, the difference between the transition patterns of the two is not considered important. However, when the elapsed time is required to be approximated as the elapsed time becomes longer, the value of the weighting factor W _c is larger than the value of k. Indeed, it is set to be larger.

  As described above, when the sum Dc of distances between the two transition patterns in each elapsed time is obtained, the distance calculation unit 241 supplies the distance sum Dc to the distance normalization unit 242.

  The distance normalization unit 242 normalizes the sum of distances Dc using a function hc such as the solid line 253 of the graph shown in FIG. 18B as in the following equation (10) (normalized distance sum Dc). 'Calculate').

・・・（１０）

(10)

  This function hc may be any function as long as the value of the distance sum Dc is smaller as the value is larger and larger as the value is smaller. That is, the normalized distance sum Dc ′ takes a larger value as the difference between the ideal transition pattern and the measurement transition pattern is smaller, and takes a smaller value as the difference between the ideal transition pattern and the measurement transition pattern is larger.

  The distance normalization unit 242 supplies the normalized distance sum Dc ′ to the evaluation value calculation unit 243.

  The evaluation value calculation unit 243 calculates the average value (M × N average values) of the entire screen of the normalized distance sum Dc ′ as the evaluation value Ec, as in the following Expression (11).

・・・（１１）

(11)

  In this case, the evaluation value Ec increases as the difference between the measurement transition and the ideal transition in the entire frame image decreases.

  Note that the evaluation value calculation unit 243 occupies the entire frame image with the number Nc2 of correlation coefficients for which the sum Dc ′ of normalized distances is equal to or greater than a predetermined threshold Tc2 as in the following Expression (12). The ratio may be calculated as the evaluation value Ec.

・・・（１２）

(12)

  In Expression (12), the threshold value Tc2 is an arbitrary value set in advance. The larger this value is set, the higher the evaluation standard (stricter evaluation conditions) and the smaller the evaluation value Ec. In this case, the evaluation value Ec increases as the number of blocks in which the difference between the measurement transition and the ideal transition is smaller than a predetermined reference and is stable in the entire frame image increases.

  As described above, the evaluation unit 104 calculates the evaluation value Ec that evaluates the correlation (that is, cooperation) between the blocks of the movement amount based on the index data regarding the movement amount of the heartbeat pulsation. More specifically, the evaluation unit 104 calculates the evaluation value Ec so that the value increases as the cooperativity of the pulsation (movement) of each part of the cultured cardiomyocytes 110 increases, and the value decreases as the cooperativity decreases. The evaluation value Ec is calculated so that becomes smaller. That is, the evaluation unit 104 can quantitatively evaluate the cooperativeness of cardiomyocyte movement.

[Flow of correlation evaluation process]
An example of the flow of correlation evaluation processing in this case will be described with reference to the flowchart of FIG.

  When the correlation evaluation process is started, the distance calculation unit 241 of the evaluation unit 104 calculates the sum of the distances between the transition patterns in step S171. In step S172, the distance normalization unit 242 normalizes the sum of the distances calculated in step S171.

  In step S173, the evaluation value calculation unit 243 calculates an evaluation value using the sum of the distances normalized in step S172. When the evaluation value is calculated, the evaluation value calculation unit 243 ends the correlation evaluation process, returns the process to step S104 in FIG. 14, and causes the processes after step S105 to be executed.

  As described above, by performing various processes, the cultured cardiomyocyte evaluation apparatus 100 can quantitatively evaluate the cooperativity of the evaluation target (for example, cell movement). Further, by using a motion vector for generating an index, it is possible to evaluate more easily and non-invasively.

<3. Third Embodiment>
[Application to other evaluations]
By evaluating the cooperativity of the movement of the evaluation object, other objects (for example, administration of gas, liquid, solid, etc.) that affect the movement of the evaluation object and any environmental conditions (for example, temperature, humidity) , Atmospheric pressure, lightness, vibration, magnetic field, etc.) may be evaluated.

  The pulsations in various regions obtained by analyzing the phase difference observation movie of cultured cardiomyocytes show cooperative pulsations depending on the number of days of culture, but show fluctuations due to administration of various drugs. By detecting such fluctuations by some method, it becomes possible to evaluate in advance the drug toxicity, effects and the like at the time of drug discovery.

  Conventionally, for example, there has been a method of detecting an external field potential of a cell with an electrode arranged on the bottom of a culture dish and capturing a pulsation behavior of the cell by a change in the membrane potential of the cell. In addition, a fluorescent dye that emits light by binding to calcium is inserted into the cell, and the pulsation rhythm of the cell is detected by detecting the calcium concentration that fluctuates due to the excitation (action potential) of the cell. There was also a way to evaluate the pattern.

  These methods require specific culture dishes, are expensive for fluorescent dyes, are cumbersome and time consuming to insert fluorescent dyes, etc. For simple and non-invasive monitoring of cells There were many problems.

  Therefore, as described above, drug toxicity and the like are evaluated using a method of detecting cell movement and evaluating the correlation (cooperation). The pulsation of cardiomyocytes consists of contraction and relaxation. For example, if the entry / exit of ions in the potassium channel of the cell is inhibited, the relaxation time is prolonged (it is difficult to return from the contracted state). The extended relaxation of each cell reduces the correlation of movement between cells. In addition, cells exchange signals with other cells via gap junctions. If transmission of signals at the gap junctions is inhibited, the correlation of movement between cells is reduced.

  That is, by evaluating the correlation of movement between cells, it is possible to evaluate the effect of an administered drug on cardiomyocytes. By evaluating the correlation of movement between cells by image analysis as described above, cell pulsation can be performed without adding any reagent such as a fluorescent dye to the cells or using a special culture dish. Since changes in behavior can be captured, drug toxicity and the like can be easily and accurately evaluated.

[Drug evaluation device]
FIG. 20 is a block diagram illustrating a main configuration example of the medicine evaluation apparatus. A drug evaluation apparatus 300 shown in FIG. 20 is an apparatus that evaluates the influence (efficacy, side effect, etc.) of a drug based on the cooperativeness of the movement of cultured cardiomyocytes 110 to which the drug is administered.

  As illustrated in FIG. 20, the drug evaluation device 300 includes an imaging unit 101 and an evaluation target image data generation / recording unit 102 similar to those of the cultured cardiomyocyte evaluation device 100 of FIG. 3. The imaging unit 101 images the cultured cardiomyocytes 110 before and after drug administration.

  The evaluation target image data generation / recording unit 102 generates evaluation target image data based on the image signal 111 supplied from the imaging unit 101, and records and stores the generated evaluation target image data in, for example, an internal recording medium. . That is, evaluation target image data is generated for each moving image of the cultured cardiomyocytes 110 before and after drug administration.

  The drug evaluation device 300 includes an evaluation index data generation unit 303 instead of the evaluation index data generation unit 103 of the cultured cardiomyocyte evaluation device 100, and further includes an evaluation unit 304 instead of the evaluation unit 104.

  The evaluation index data generation unit 303 acquires the evaluation target image data 112 from the evaluation target image data generation recording unit 102. The evaluation index data generation unit 303 generates evaluation index data 113 using the acquired evaluation target image data 112 and supplies it to the evaluation unit 304.

  More specifically, the evaluation index data generation unit 303 is a partial region obtained by dividing the entire region of the frame image into a plurality of portions between the frame images of the evaluation target image data 112 that is a moving image of the cultured cardiomyocytes 110, for example. The motion of the evaluation target (cultured cardiomyocytes 110) is detected for each block. The evaluation index data generation unit 303 represents the detected motion of each block as a motion vector, and supplies the motion vector as evaluation index data 113 to the evaluation unit 304.

  The evaluation unit 304 evaluates the supplied evaluation index data 113 to calculate and output a cooperative evaluation value 114 for the movement of the cultured cardiomyocytes 110.

  More specifically, as illustrated in FIG. 21A, for example, the evaluation unit 304 sets a partial region obtained by dividing the entire frame image (observation region) into a plurality of regions, and evaluates the correlation of motion vectors between the regions. .

  In the example of FIG. 21A, the frame image is divided into 4 × 4 partial areas (16 divisions). The size of the partial area (that is, the number of divisions) is arbitrary, but if it is an integral multiple of the block when detecting the motion vector, the arithmetic processing becomes easy.

  However, the correlation in an extremely narrow region often does not significantly affect the correlation evaluation result of motion vectors in the entire observation region (frame image). Therefore, the size of the partial area (number of divisions) is the size of the frame image (resolution), how to move the evaluation object, the evaluation criteria (how much accuracy is required), etc. so as not to increase the amount of calculation unnecessarily. It is desirable to set an appropriate size according to the various conditions.

  In general, for example, a 4 × 4 division as shown in FIG. 21A is desirable. In general, since the motion vector is obtained in a smaller unit, the evaluation unit 304 obtains the average of the motion vectors included in each region, and sets the average as the motion vector of the region.

  As shown in FIG. 21B, the evaluation unit 304 obtains a temporal change (or a temporal change in the amount of motion) of the motion vector of each region (the average of the motion vectors in the region). Since the evaluation target image data 112 is a moving image, a motion vector is obtained for each frame (for each sampling time). That is, the evaluation unit 304 obtains a motion vector of each region for each frame (for each sampling time). FIG. 21B is a graph showing the temporal change of the motion vector (absolute value) of each region calculated by the evaluation unit 304 (that is, a pulsation pattern).

  The evaluation unit 304 evaluates the correlation between each region between all the regions, and obtains a 3D plot as shown in FIG. Each 3D plot in FIG. 22 is obtained by drawing the strength of correlation between feature amounts in each region in a three-dimensional space. In the 3D plot of FIG. 22, the x and y axes indicate each region (a combination of two regions), and the z axis indicates the strength of the correlation of the temporal change of the motion vector (or the amount of motion) between the regions. The correlation coefficient C is obtained by the following equation (13) where the complete correlation is 1.

・・・（１３）

... (13)

However, (x, y) = {(x _i , y _i )} (i = 1, 2,..., N), and x and y represent the amount of motion in the x-axis direction and the amount of motion in the y-axis direction, respectively. I represents a sample number (time). The overlined x and the overlined y indicate the arithmetic mean of the data x = {x _i } and y = {y _i }, respectively. This is a cosine of an angle formed by vectors represented by the following formulas (14) and (15), which represents a deviation from the average of each data.

・・・（１４）

・・・（１５）

(14)

... (15)

  In FIG. 22, the 3D plot on the left side shows the correlation before drug administration, and the 3D plot on the right side shows the correlation 2 hours after the drug administration. Of course, the correlation at which the correlation is evaluated (for example, 3D plotting) is arbitrary.

  In the example of FIG. 22, the top 3D plot shows the correlation before and after administration of dimethyl sulfoxide used as the organic solvent. The second 3D plot from the top shows the correlation before and after administration of aspirin (aspirin (acetylsalicylic acid)). Further, the third 3D plot from the top shows the correlation before and after administration of DL-sotalol. The bottom 3D plot shows the correlation before and after administration of 18-β-Glycyrrhetinic acid.

  As shown in the 3D plot of FIG. 22, dimethyl sulfoxide and aspirin do not significantly affect the cardiomyocyte pulsatility. In contrast, DL-sotalol is known to inhibit potassium channels. That is, when DL-sotalol is administered to cultured cardiomyocytes 500, the relaxation process changes due to a change in potassium channel function that works in the relaxation process. As shown in FIG. 22, the correlation between the motion vectors in each region is reduced after the administration as shown in FIG. In addition, 18-β-glycyrrhetinic acid is known to inhibit gap junction, and as shown in FIG. 22, the correlation of motion vectors in each region is reduced after administration.

  Thus, by evaluating the correlation between the motion vectors in each region before and after the drug administration, the evaluation unit 304 can easily detect the influence of the drug on the pulsation of the cardiomyocytes. As described above, by observing the correlation of motion vectors between partial areas, that is, the correlation of pulsations between cells within the observation area of cultured cardiomyocytes 500, a specific cell (specific part Information that cannot be obtained simply by observing the state of the pulsation in the region) can be obtained. For example, when 18-β-glycyrrhetinic acid is administered, a large extension of the relaxation waveform does not occur in the pulsation of a specific cell, but the correlation between cells (between partial regions) varies greatly as described above. . Therefore, drug evaluation can be performed with an index different from that for observing the state of pulsation of a specific cell. Note that the evaluation unit 304 may provide the user with an image as in the 3D plot shown in FIG. 22 as the evaluation value 114, or numerically evaluate the correlation and perform the evaluation quantitatively. The result may be the evaluation value 114.

  Of course, the 3D plot shown in FIG. 22 is an example of imaging. For example, the correlation of motion vectors in each region is expressed by an arbitrary image such as a line graph, a bar graph, a distribution diagram, and a schematic diagram. May be. Moreover, the chemical | medical agent to evaluate is arbitrary and may be other than four types of chemical | medical agents mentioned above.

  Details of each part will be described below.

[Evaluation index data generator]
FIG. 23 is a block diagram illustrating a main configuration example of the evaluation index data generation unit 303. As illustrated in FIG. 23, the evaluation index data generation unit 303 includes a motion detection unit 121 and a motion detection data storage unit 122. That is, the evaluation index data generation unit 303 supplies the motion detection data (motion vector) to the evaluation unit 304 as the evaluation index data 113.

[Evaluation Department]
FIG. 24 is a block diagram illustrating a main configuration example of the evaluation unit 304. As illustrated in FIG. 24, the evaluation unit 304 includes a motion vector acquisition unit 341, a region feature amount calculation unit 342, a correlation coefficient calculation unit 343, a correlation evaluation unit 344, a display unit 345, and an output unit 346.

  The motion vector acquisition unit 341 receives, from the evaluation index data generation unit 303 (motion detection data storage unit 122), a desired motion vector (for example, a motion vector corresponding to a moving image instructed to evaluate the correlation) by the evaluation index data. Obtained as 113. The motion vector acquisition unit 341 supplies the acquired motion vector to the region feature amount calculation unit 342.

  The region feature amount calculation unit 342 calculates a motion vector (average value of motion vectors in the region) for evaluating the correlation of the motion vectors described above, and calculates the average value of the motion vectors (or the motion thereof). (Time) is defined as a feature amount of the region. The region feature amount calculation unit 342 calculates such a feature amount for each region. Note that the feature amount of each region may be a parameter indicating a temporal change in the motion (or motion amount) of each region, and may be other than the average value of the motion vectors in the region described above. For example, the temporal change in the motion vector (or the amount of motion) of the representative position in the region may be used as the feature amount of the region, or the temporal motion of the motion vector (or the amount of motion) that takes the maximum value in the region. The change may be a feature amount of the area. Of course, the temporal change of the motion vector (or the amount of motion) taking an intermediate value in the region may be used as the feature amount of the region. Of course, other parameters may be used. The region feature amount calculation unit 342 supplies the calculated feature amounts of the partial regions to the correlation coefficient calculation unit 343.

  The correlation coefficient calculation unit 343 calculates the correlation coefficient C of the feature amount between the partial areas using, for example, the above-described equation (13). The correlation coefficient calculation unit 343 calculates the correlation coefficient of the feature amount for all combinations of two partial areas (for all partial area pairs). Of course, the correlation coefficient of the feature amount may be obtained only for some partial region pairs. The method for calculating the correlation coefficient of the feature amount is arbitrary, and the correlation coefficient calculation unit 343 may obtain the correlation coefficient by a calculation other than the above-described equation (13).

  The correlation coefficient calculation unit 343 supplies the calculated correlation coefficient to the display unit 345 for display, or supplies the calculated correlation coefficient to the output unit 346 for supply to another device. Further, the correlation coefficient calculation unit 343 supplies the calculated correlation coefficient to the correlation evaluation unit 344.

  The correlation evaluation unit 344 quantitatively evaluates the supplied correlation coefficient value. For example, the correlation evaluation unit 344 determines whether the correlation coefficient has been reduced using a threshold value. The correlation evaluation unit 344 supplies the evaluation result to the display unit 345 for display, or supplies it to the output unit 346 for supply to another device.

  The display unit 345 includes a display device such as a monitor, for example, converts the data supplied from the correlation coefficient calculation unit 343 or the correlation evaluation unit 344 into an image, and displays the image on the display device. For example, the display unit 345 images and displays the correlation strength between the regions calculated by the correlation coefficient calculation unit 343 as a 3D plot drawn in a three-dimensional space. Further, for example, the display unit 345 images and displays the evaluation result supplied from the correlation evaluation unit 344.

  The output unit 346 has an interface such as an external terminal, for example, and outputs data supplied from the correlation coefficient calculation unit 343 or the correlation evaluation unit 344 to an external device or network.

  As described above, the evaluation unit 304 obtains the correlation coefficient of the temporal change of the motion vector (or the amount of motion) between the partial areas, so that the drug evaluation device 300 can correct the pulsation of the cardiomyocytes by the drug administration. The effect can be evaluated easily and non-invasively.

[Flow of evaluation process]
Next, an example of the flow of evaluation processing executed by the drug evaluation device 300 will be described with reference to the flowchart of FIG.

  When the evaluation process is started, the imaging unit 101 of the medicine evaluation apparatus 300 images the evaluation target in step S301. In step S302, the evaluation target image data generation / recording unit 102 generates evaluation target image data from the image signal obtained by the imaging in step S301.

  In step S303, the evaluation index data generation unit 303 generates evaluation index data using the evaluation target image data generated in step S302. In step S304, the evaluation unit 304 uses the evaluation index data generated in step S303 to observe the correlation (cooperation) between partial regions of pulsation of the cultured cardiomyocytes 110 before and after drug administration. Evaluate drug effects.

  In step S305, the output unit 346 of the evaluation unit 304 outputs the evaluation value calculated in step S304 to the outside of the drug evaluation device 300, and ends the evaluation process. In step S305, instead of the output of the output unit 346, as described above, the display unit 345 may image the evaluation value and display the image on the display device. Further, as described above, the display unit 345 may image the correlation coefficient group calculated in the process of step S304 and display it on the display device, or the output unit 346 may display the correlation coefficient group. May be output to the outside of the medicine evaluation apparatus 300.

[Flow of evaluation index data generation processing]
Next, an example of the flow of the evaluation index data generation process executed in step S303 in FIG. 25 will be described with reference to the flowchart in FIG.

  When the evaluation index data generation process is started, the motion detection unit 121 of the evaluation index data generation unit 303 detects a motion to be evaluated for each block and generates a motion vector in step S321. In step S322, the motion detection data storage unit 122 stores the motion vector of each block generated in step S321.

  In step S323, the motion detection unit 121 determines whether motion detection has been performed for a predetermined period (evaluation interval). If it is determined that there is a frame image for which motion detection is not performed in the predetermined evaluation section, the motion detection unit 121 returns the process to step S321 and repeats motion detection for a new processing target frame image.

  If it is determined in step S323 that motion detection has been performed on all the frame images to be processed in the predetermined evaluation section, the motion detection unit 121 ends the evaluation index data generation processing, and the processing is illustrated in FIG. The process after step S304 is executed.

[Flow of impact assessment processing]
Next, an example of the flow of the impact evaluation process executed in step S304 in FIG. 25 will be described with reference to the flowchart in FIG.

  When the influence evaluation process is started, the motion vector acquisition unit 341 of the evaluation unit 304 acquires a desired motion vector from the motion detection data storage unit 122 in step S341.

  In step S342, the region feature amount calculation unit 342 calculates a feature amount for each region using the motion vector acquired in step S341. In step S343, the correlation coefficient calculation unit 343 calculates an inter-region correlation coefficient for the feature amount calculated in step S342.

  In step S344, the correlation evaluation unit 344 evaluates cooperation between regions (correlation of pulsation) by evaluating the correlation coefficient calculated in step S343. When the process of step S344 ends, the correlation evaluation unit 344 ends the impact evaluation process, returns the process to FIG. 25, and executes the process of step S305.

  As described above, the evaluation unit 304 obtains the correlation coefficient of the temporal change of the motion vector (or the amount of motion) between the partial areas, so that the drug evaluation device 300 can correct the pulsation of the cardiomyocytes by the drug administration. The impact can be easily evaluated. Since this method does not use a special culture dish or fluorescent reagent, it can be evaluated simply, non-invasively and inexpensively, and is also suitable for automation. In this method, the observation area may be a relatively narrow range, for example, about 0.6 mm square, and the test can be performed with a small number of cells and a small reagent. In addition, it can be sufficiently evaluated by a commercially available high-density culture plate (1536-well plate (1.7 mm diameter / 1 well) and 384-well plate (3.6 mm diameter / 1 well). This technique is also suitable for initial screening, and can be applied to the evaluation of anything that can be evaluated by observing cultured cardiomyocytes 110.

<4. Fourth Embodiment>
[Personal computer]
The series of processes described above can be executed by hardware or can be executed by software. In this case, for example, a personal computer as shown in FIG. 28 may be configured.

  In FIG. 28, a CPU (Central Processing Unit) 1501 of the personal computer 1500 performs various processes according to a program stored in a ROM (Read Only Memory) 1502 or a program loaded from a storage unit 1513 to a RAM (Random Access Memory) 1503. Execute the process. The RAM 1503 also appropriately stores data necessary for the CPU 1501 to execute various processes.

  The CPU 1501, ROM 1502, and RAM 1503 are connected to each other via a bus 1504. An input / output interface 1510 is also connected to the bus 1504.

  The input / output interface 1510 includes an input unit 1511 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal Display), an output unit 1512 including a speaker, and a hard disk. A communication unit 1514 including a storage unit 1513 and a modem is connected. The communication unit 1514 performs communication processing via a network including the Internet.

  A drive 1515 is connected to the input / output interface 1510 as necessary, and a removable medium 1521 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is loaded. It is installed in the storage unit 1513 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 28, this recording medium is distributed to distribute a program to a user separately from the apparatus main body, and includes a magnetic disk (including a flexible disk) on which a program is recorded, an optical disk ( It is simply composed of removable media 1521 made of CD-ROM (compact disc-read only memory), DVD (including digital versatile disc), magneto-optical disc (including MD (mini disc)), or semiconductor memory. Rather, it is composed of a ROM 1502 on which a program is recorded and a hard disk included in the storage unit 1513, which is distributed to the user in a state of being incorporated in the apparatus main body in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but may be performed in parallel or It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus composed of a plurality of devices (apparatuses).

  In addition, in the above description, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit). Of course, a configuration other than that described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). . That is, the present technology is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present technology.

In addition, this technique can also take the following structures.
(1) a motion detector that detects a motion of the evaluation object using an image of the evaluation object;
A correlation calculation unit that calculates a correlation of temporal changes in the amount of motion at a plurality of locations of the evaluation target, using a motion vector indicating the motion of the evaluation target detected by the motion detection unit;
An image processing apparatus comprising: an evaluation value calculating unit that calculates an evaluation value for evaluating cooperativity of the movement of the evaluation object using the correlation calculated by the correlation calculating unit.
(2) The motion detection unit divides the entire region of the evaluation target image into a plurality of partial regions, detects the motion for each partial region,
The correlation calculation unit calculates the correlation between the partial regions using the motion vector obtained for each partial region by the motion detection unit,
The image processing apparatus according to (1), wherein the evaluation value calculation unit calculates the evaluation value using the correlation between the partial areas calculated by the correlation calculation unit.
(3) The evaluation value calculation unit includes:
A correlation normalization unit that normalizes the correlation between the partial regions calculated by the correlation calculation unit with a predetermined function;
A variance calculation unit that calculates variance in the time direction of the correlation between the partial areas calculated by the correlation calculation unit;
A variance normalization unit that normalizes the variance calculated by the variance calculation unit with a predetermined function;
The average value of the entire evaluation target image of the product of the correlation between the partial regions normalized by the correlation normalization unit and the variance normalized by the variance normalization unit is used as the evaluation value. The image processing apparatus according to (2), further comprising: an average value evaluation value calculation unit to calculate.
(4) The evaluation value calculation unit includes:
A correlation normalization unit that normalizes the correlation between the partial regions calculated by the correlation calculation unit with a predetermined function;
A variance calculation unit that calculates variance in the time direction of the correlation between the partial areas calculated by the correlation calculation unit;
A variance normalization unit that normalizes the variance calculated by the variance calculation unit with a predetermined function;
It has a value larger than a predetermined threshold for the total number of products of the correlation between the partial areas normalized by the correlation normalization unit and the variance normalized by the variance normalization unit The image processing apparatus according to (2) or (3), further including: an average value evaluation value calculation unit that calculates a ratio of the number of products as the evaluation value.
(5) The evaluation value calculation unit
An ideal transition that is a predetermined ideal temporal transition of the correlation between the partial regions calculated by the correlation calculating section and a measurement transition that is a temporal transition detected by the motion detecting section. A distance calculation unit for obtaining a difference;
A normalization unit that normalizes the difference calculated by the distance calculation unit with a predetermined function;
The image processing apparatus according to any one of (2) to (4), further including: a difference average value calculation unit that calculates an average value of the differences normalized by the normalization unit as the evaluation value.
(6) The evaluation value calculation unit
An ideal transition that is a predetermined ideal temporal transition of the correlation between the partial regions calculated by the correlation calculating section and a measurement transition that is a temporal transition detected by the motion detecting section. A distance calculation unit for obtaining a difference;
A normalization unit that normalizes the difference calculated by the distance calculation unit with a predetermined function;
An average value evaluation value calculation unit that calculates, as the evaluation value, a ratio of the number of average values having a value larger than a predetermined threshold with respect to the total number of average values of the differences normalized by the normalization unit The image processing apparatus according to any one of (2) to (5), further including:
(7) The correlation calculation unit calculates the correlation between the partial areas for a part or all of the partial area pairs,
The evaluation value calculation unit images and displays the correlation strength of each partial region pair calculated by the correlation calculation unit as a 3D plot to be drawn in a three-dimensional space. (2) to (6) The image processing apparatus according to any one of the above.
(8) The image processing device according to any one of (1) to (7), wherein the evaluation value calculation unit calculates the evaluation value so that the value increases as the cooperation increases.
(9) The image processing apparatus according to any one of (1) to (8), wherein the evaluation value calculation unit calculates the evaluation value so that the value decreases as the cooperativity decreases.
(10) It further includes an imaging unit that images the evaluation object and obtains an image of the evaluation object,
The image processing apparatus according to any one of (1) to (9), wherein the motion detection unit detects a motion of the evaluation target using the image of the evaluation target obtained by the imaging unit.
(11) The image processing device according to any one of (1) to (10), wherein the correlation calculation unit repeats the calculation of the correlation of the temporal change in the motion amount at a plurality of locations to be evaluated a plurality of times.
(12) The image processing device according to any one of (1) to (11), wherein the evaluation target is a cell that moves spontaneously.
(13) The image processing apparatus according to any one of (1) to (12), wherein the evaluation target is a cultured cell generated by culturing a cell collected from a living body.
(14) The motion detection unit of the image processing device detects the motion of the evaluation target using the evaluation target image,
The correlation calculation unit of the image processing device calculates a correlation of temporal changes in the amount of motion at a plurality of locations of the evaluation target using a motion vector indicating the detected motion of the evaluation target,
An image processing method, wherein an evaluation value calculation unit of the image processing apparatus calculates an evaluation value for evaluating cooperativity of the movement of the evaluation object, using the calculated correlation.
(15) Connect the computer
A motion detector that detects the motion of the evaluation object using an image of the evaluation object;
A correlation calculation unit that calculates a correlation of temporal changes in motion amounts at a plurality of locations of the evaluation target, using a detected motion vector indicating the motion of the evaluation target;
The program for functioning as an evaluation value calculation part which calculates the evaluation value which evaluates the cooperativity of the movement of the said evaluation object using the calculated said correlation.

  DESCRIPTION OF SYMBOLS 100 Cultured-cardiomyocyte evaluation apparatus, 101 Imaging part, 102 Evaluation object image data production | generation recording part, 103 Evaluation index data generation part, 104 Evaluation part, 121 Motion calculation part, 122 Motion detection data storage part, 123 Correlation calculation part, 124 Correlation Data history storage memory, 141 frame memory, 142 motion vector calculation unit, 201 inter-block correlation coefficient calculation unit, 202 correlation coefficient storage unit, 203 average value calculation unit, 221 correlation coefficient normalization unit, 222 variance calculation unit, 223 dispersion normalization unit, 224 evaluation value calculation unit, 241 distance calculation unit, 242 distance normalization unit, 243 evaluation value calculation unit, 300 drug evaluation device, 303 evaluation index data generation unit, 304 evaluation unit, 341 motion vector acquisition unit , 342 region feature amount calculation unit, 343 Correlation coefficient calculation unit 343, 344 Correlation evaluation unit, 345 display unit, 346 output unit

Claims (19)

By using the motion information indicating the motion of the evaluation target calculated for a plurality of partial regions constituting the evaluation target image collected from the living body, the correlation of the temporal change of the motion information between the partial regions is unified. An image processing apparatus comprising: a control unit configured to calculate a part or all of the partial region pairs and to control and display the correlation information indicating the correlation of the calculated partial region pairs. It further includes an imaging unit that images the evaluation object and obtains an image of the evaluation object,
The control unit calculates a correlation of temporal changes in the motion information between the partial areas using motion information indicating the motion of the evaluation target in the evaluation target image obtained by the imaging unit. An image processing apparatus according to 1. The image processing apparatus according to claim 1, wherein the control unit repeats the calculation of the correlation of the temporal change in the motion information between the partial areas a plurality of times. The image processing apparatus according to claim 1, wherein the evaluation target is a cell that moves spontaneously. The image processing apparatus according to claim 1, wherein the evaluation target is a cultured cell generated by culturing a cell collected from a living body. The image processing apparatus according to claim 1, wherein the control unit images the correlation information so that the strength of the correlation of each partial region pair is represented by a color difference. The image processing apparatus according to claim 1, wherein the evaluation target is a cardiomyocyte. The image processing apparatus according to claim 1, wherein the control unit controls the correlation information to be imaged and displayed as a 3D plot drawn in a three-dimensional space. The image processing apparatus according to claim 1, wherein the control unit controls the correlation information to be imaged and displayed as a line graph, a bar graph, a distribution diagram, or a schematic diagram. By using the motion information indicating the motion of the evaluation target calculated for a plurality of partial regions constituting the evaluation target image collected from the living body, the correlation of the temporal change of the motion information between the partial regions is unified. Calculate for partial or all partial region pairs,
An image processing method for controlling to display correlation information indicating the correlation of each calculated partial region pair as an image. The correlation of temporal change of motion information between the partial areas is calculated using motion information indicating the motion of the evaluation target in the evaluation target image obtained by imaging the evaluation target. Image processing method. The image processing method according to claim 10, wherein the calculation of the correlation of the temporal change in the motion information between the partial areas is repeated a plurality of times. The image processing method according to claim 10, wherein the evaluation target is a cell that moves spontaneously. The image processing method according to claim 10, wherein the evaluation object is a cultured cell generated by culturing a cell collected from a living body. The image processing method according to any one of claims 10 to 14, wherein the correlation information is imaged so that the correlation strength of each partial region pair is represented by a difference in color. The image processing method according to claim 10, wherein the evaluation object is a cardiomyocyte. The image processing method according to any one of claims 10 to 16, wherein the correlation information is imaged and displayed as a 3D plot drawn in a three-dimensional space. The image processing method according to any one of claims 10 to 16, wherein the correlation information is controlled to be imaged and displayed as a line graph, a bar graph, a distribution diagram, or a schematic diagram. Computer
By using the motion information indicating the motion of the evaluation target calculated for a plurality of partial regions constituting the evaluation target image collected from the living body, the correlation of the temporal change of the motion information between the partial regions is unified. A program for functioning as a control unit that calculates a part or all of the partial region pairs and controls to display the correlation information indicating the correlation of the calculated partial region pairs as an image.

Images