KR20230091537A - Integrated system of electro-stimulation for optimal differentiation of stem cells - Google Patents

Abstract

Provided is an electro-stimulation control system, comprising: a driving current control unit that provides, to a pair of electrodes, a driving current for supplying electrical stimulation to a stem cell culture medium contained in an incubator; an estimation unit; and a photographing device that provides, to the estimation unit, real-time images taken of the stem cell culture medium at predetermined time intervals. The estimation unit includes a neural network unit, inputs each of the plurality of taken images to an input layer of the neural network unit, and is designed to output an estimation value with a voltage corresponding to a logical high or a voltage corresponding to a logical low based on the value output by the neural network unit. The driving current control unit is configured to receive the estimation value and to use the estimation value as an on/off control signal for the driving current. An optimal control technology for determining the termination point of the electric stimulation applied to the stem cell culture medium may be provided.

Description

Integrated system of electro-stimulation for optimal differentiation of stem cells}

The present invention relates to a technique for inducing cell differentiation or functional activity using a microscopic electrical stimulation system without genetic manipulation or biochemical factors. In particular, it relates to an automatic electrical stimulation scheduling technology for optimal differentiation of stem cells.

Recently, many studies on the induction of cell differentiation and reprogramming using electrical stimulation have been promoted. Electrical stimulation plays a special role in cell proliferation and migration, and studies using cells, tissues, and animals have reported results of promoting tissue regeneration, and it is believed that apoptosis signals can be changed and immune cell migration can be controlled. It is known. Currently, it is not known what research has been done on the establishment of differentiation technology into specific cells using electrical stimulation in companion animals, leisure animals, and industrial animals and the application of the technology to diseases.

As an alternative to the conventional treatment of specific growth factors, many studies have been conducted to induce the differentiation of stem cells into cells responsible for specific functions such as nerve cells, chondrocytes, and osteogenic cells. In this regard, it has been reported that differentiation into striated muscle cells expressing myocardial-specific markers can be induced by applying electrical stimulation to human mesenchymal stem cells. In addition, it has been reported that osteogenic differentiation was successfully achieved by applying electrical stimulation to human mesenchymal stem cells seeded on a silk-fibroin film. Furthermore, it is known that electrical stimulation can be used not only to differentiate stem cells into specific cell types, but also to cross-differentiate specific cells that have already been differentiated into other types of cells. In this regard, it has been reported that skin-derived fibroblasts can be cross-differentiated into nerve cells using triboelectric nanogenerators. In addition, Korean Patent Registration No. 1653197 discloses a method of cross-differentiation into chondrocytes by applying electrical stimulation to fibroblasts.

However, the prior art only discloses a method for differentiating into a specific cell by applying specific electrical stimulation to stem cells or fibroblasts, and does not suggest optimization of electrical stimulation scheduling to induce differentiation more efficiently. there is. In particular, an optimal method for improving chondrogenic markers of certain cells differentiated from stem cells by electrical stimulation has not been proposed.

According to the observations of the inventors of the present invention, when electrical stimulation is continuously applied to the stem cell culture medium, the amount of chondrocytes changed from the stem cells increases and then decreases again. In order to differentiate chondrocytes in large quantities in this way, the first time point at which the amount of differentiated chondrocytes is maximized according to the duration of electrical stimulation is determined, and after the first time point, electrical stimulation is stopped and differentiated Chondrocytes need to be harvested. If the amount of chondrocytes formed during electrical stimulation is maintained at the maximum value for a predetermined time, the differentiation process into chondrocytes may be optimized by stopping the electrical stimulation at the first time point when the maximum value is reached.

The present invention is to provide a technique for determining the end point of electrical stimulation applied to a stem cell culture medium in order to differentiate chondrocytes from stem cells in order to solve the above problems. However, these tasks are illustrative, and the scope of the present invention is not limited thereby.

According to one aspect of the present invention, the driving current controller 30 configured to provide electrical stimulation according to a predetermined current pattern to the stem cell culture medium contained in the incubator 10; a photographing device 40 configured to provide the estimator 50 with a plurality of photographed images obtained by photographing the stem cell culture medium at predetermined time intervals; and a neural network unit 510, the estimation unit 50 inputting each of the plurality of photographed images to an input layer of the neural network unit 510 and outputting a series of output values from the neural network unit 510. An electrical stimulation control system may be provided, including ; and configured to determine an end point of providing the electrical stimulation based on a history of the series of output values over time.

In this case, the neural network unit may be supervised to output the output value as a larger value as the area of the cluster region of chondrocytes included in the captured image input to the neural network unit decreases.

In this case, the end of the provision of the electrical stimulation may be determined when it is determined that the series of output values no longer increase with time.

At this time, the neural network unit provides electrical stimulation according to a predetermined current pattern to the stem cell culture medium contained in the incubator 10, obtaining a plurality of photographed images of the stem cell culture medium at predetermined time intervals; preparing a set of labels corresponding to the plurality of captured images to train a neural network unit using the plurality of captured images; and updating an internal parameter of the neural network unit to reduce an error between a value output by the neural network unit by inputting the selected captured image to the neural network unit and a value of the label corresponding to the selected captured image. The step of preparing a set of labels is a step of allocating a reference label having a predetermined specific value to a reference captured image selected by a user from among the plurality of captured images. ; and allocating labels to other captured images other than the reference captured image among the plurality of captured images according to a predetermined rule.

At this time, the step of allocating the labels according to the predetermined rule is the step of dividing the selected captured image among the remaining captured images into two areas, an outer area including the edge of the incubator and an inner area inside the outer area. ; and determining a label value for the selected captured image based on the width of the inner region.

In this case, the dividing into two areas may include selecting a plurality of outer points among edges of an incubator image representing the incubator among the selected captured images; For each of the outer points, selecting a point at which an image attribute value of the selected captured image changes most rapidly when observed along a line connecting the outer point to the center point of the incubator image as an inner point; determining the inner area by determining a closed curve connecting the plurality of inner points selected for each of the outer points as an outer boundary of the inner area; and determining an outer portion of the closed curve as the outer area.

According to another aspect of the present invention, providing electrical stimulation according to a predetermined current pattern to the stem cell culture medium contained in the incubator 10, obtaining a plurality of photographed images of the stem cell culture medium at predetermined time intervals ; preparing a set of labels corresponding to the plurality of captured images to train a neural network unit using the plurality of captured images; and updating an internal parameter of the neural network unit to reduce an error between a value output by the neural network unit by inputting the selected captured image to the neural network unit and a value of the label corresponding to the selected captured image, The preparing of the set of labels may include allocating a reference label having a predetermined specific value to a reference captured image selected by a user from among the plurality of captured images; and allocating labels according to a predetermined rule to remaining captured images other than the reference captured image among the plurality of captured images.

At this time, the step of allocating the labels according to the predetermined rule is the step of dividing the selected captured image among the remaining captured images into two areas, an outer area including the edge of the incubator and an inner area inside the outer area. ; and determining a label value for the selected captured image based on the width of the inner region.

In this case, the dividing into two areas may include selecting a plurality of outer points among edges of an incubator image representing the incubator among the selected captured images; For each of the outer points, selecting a point at which an image attribute value of the selected captured image changes most rapidly when observed along a line connecting the outer point to the center point of the incubator image as an inner point; determining the inner area by determining a closed curve connecting the plurality of inner points selected for each of the outer points as an outer boundary of the inner area; and determining an outer portion of the closed curve as the outer area.

According to the present invention, it is possible to provide an optimal control technique for determining the end point of electric stimulation applied to a stem cell culture medium in order to differentiate chondrocytes from stem cells.

1 is a block diagram showing an electrical stimulation system 100 provided according to an embodiment of the present invention.
2 is a block diagram showing an electrical stimulation learning system provided according to an embodiment of the present invention.
Figure 3 shows an example of a photographic image observed over time when electrical stimulation is applied to a stem cell culture medium by modeling.
4 is a diagram for explaining a method of dividing an acquired captured image into two regions according to an embodiment of the present invention.
5 is a flowchart illustrating a method for determining an electrical stimulation end point using an electrical stimulation system provided according to an embodiment of the present invention.
6 is an example of a graph showing values output by an estimation unit of an electrical stimulation system according to an embodiment of the present invention according to time.
Figure 7a is for explaining a method of assigning a label according to the state of the captured image of the stem cell culture medium contained in the incubator according to an embodiment of the present invention.
FIG. 7B illustrates the configuration of the neural network unit shown in FIG. 1 .
7C is for explaining a method of training a neural network unit according to an embodiment of the present invention.
FIG. 7D illustrates the structure of an estimation unit provided after the learning of the neural network unit described in FIG. 7C is completed, including the neural network unit for which the learning is completed.
FIG. 8A shows values output by the estimator over time when photographed images over time of the stem cell culture medium to which electrical stimulation is being provided are sequentially provided to the estimator of FIG. 7D.
FIG. 8B shows a first signal for controlling on/off of the driving current control unit using the output of the estimation unit shown in FIG. 8A.
FIG. 8C is a second signal showing a time moving average of the first signal shown in FIG. 8B.
FIG. 8D is a third signal generated based on the second signal shown in FIG. 8C and having only values corresponding to '1' or '0'.
9 shows the configuration of the driving current controller 30 provided according to an embodiment of the present invention.
10 illustrates a method of photographing a plurality of incubators according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be implemented in various other forms. Terms used in this specification are intended to aid understanding of the embodiments, and are not intended to limit the scope of the present invention. Also, the singular forms used herein include the plural forms unless the phrases clearly dictate the contrary.

<General configuration of electrical stimulation control system>

1 is a block diagram showing an electrical stimulation system 100 provided according to an embodiment of the present invention.

The electrical stimulation system 100 provided according to one aspect of the present invention includes an incubator 10 configured to contain a stem cell culture medium, and a pair installed to apply current to the stem cell culture medium contained in the incubator 10. The electrode 20, the drive current controller 30 for providing electricity to the pair of electrodes 20, the photographing device 40 for photographing the stem cell culture medium, and the photographing device 40 outputs over time It may include an estimation unit 50 for inputting a plurality of photographic images, estimating and outputting a value related to the differentiation amount of chondrocytes differentiated from the stem cells included in the stem cell culture medium. At this time, the driving current controller 30 blocks the current applied through the pair of electrodes 20 based on the values of the amount of differentiation output by the estimator 50 over time.

The estimator 50 may be a computing device. The computing device may include a CPU, a power supply unit, a memory unit, a communication unit, and a peripheral device interface unit. The memory unit may include volatile and/or non-volatile memory. The memory unit may store command codes for implementing an algorithm defining a relationship between an input and an output of a neural network unit provided according to an embodiment of the present invention. The CPU may read and execute the instruction codes. The neural network unit 510 may be implemented by the instruction codes executed in the CPU. The peripheral device interface unit may be connected to the photographing device 40 and provide image and video information output from the photographing device 40 to the memory unit or the CPU.

The drive current control unit 30 is connected to ① a power element providing current to the pair of electrodes 20, ② connected to the power element, receives values related to the amount of differentiation, and outputs the values to the power element based on this. A driving chip (hardware) generating a control signal for controlling current, ③ a non-volatile memory storing firmware for setting functions of the driving chip, and ④ a communication interface unit receiving the signal provided by the estimation unit 50. can do.

In one embodiment, the signal (estimated value) provided by the estimation unit 50 may be a TTL level baseband voltage signal. At this time, the signal (estimated value) provided by the estimator 50 may provide only two pieces of information: logical high and logical low. The logical high may be provided with a DC potential of, for example, +12V, and the logical low may be provided with a DC potential of, for example, 0V.

Alternatively, in another embodiment, the signal (estimated value) provided by the estimation unit 50 is an analog signal representing a real number of 0 to 1, and may be a modulated signal. For example, the signal (estimated value) provided by the estimator 50 may have an arbitrary value between 0V and +12V, where 0V may correspond to a real number 0 and +12V may correspond to a real number 1.

In one embodiment, the value of the differentiation amount may be normalized such that the maximum value and the minimum value thereof are, for example, 1 and 0, respectively. When the value for the amount of differentiation is 1, it can be considered that the chondrocytes are differentiated as much as possible.

The estimation unit 50 may include a neural network unit 510 learned through supervised learning.

As the drive current control unit 30 starts supplying current to the pair of electrodes 20, the photographing device 40 photographs the inside of the incubator 10 and acquires a plurality of photographed images over time. .

The estimator 50 may receive the acquired plurality of captured images in real time and output an estimated value of the amount of differentiation of chondrocytes. As time passes, the estimated value output by the estimator 50 may tend to gradually increase over time, maintain a saturation state for a certain period of time, and then decrease again.

The driving current control unit 30 may be configured to monitor the estimated value output by the estimating unit 50 and cut off the current applied to the pair of electrodes 20 when it is determined that the estimated value will no longer increase. there is.

<The learning method of the neural network unit according to the first embodiment>

The neural network unit of FIG. 1 may be learned through supervised learning.

Hereinafter, a supervised learning method of the neural network unit will be described with reference to FIG. 2 . In particular, a method for determining labels to be used for supervised learning is described.

2 is a block diagram showing an electrical stimulation learning system provided according to an embodiment of the present invention.

The electric stimulation learning system of FIG. 2 is obtained by adding a supervised learning controller 520 to the electric stimulation system shown in FIG. 1 .

For the supervised learning, an expert (person) observes the plurality of photographed images output by the photographing device 40 in chronological order, and determines that the differentiation amount of chondrocytes has reached the maximum value among the plurality of photographed images. A reference label having a specific value may be assigned to a first captured image (= standard captured image), which is the first captured image. The specific value of the reference label may be, for example, 1. That is, the reference label can be associated with an image obtained by the photographing device 40 at the first point in time when it is determined that the amount of chondrocytes differentiated by electrical stimulation is maximized.

In one embodiment, labels for the remaining captured images excluding the reference captured image among the plurality of captured images may be automatically assigned by a predetermined method described below without being directly assigned by a person.

The plurality of photographed images may have characteristics according to a phenomenon described below.

That is, the stem cells contained in the culture solution before applying the electrical stimulation may be uniformly distributed in the culture solution. However, as the electrical stimulation is started and continues, the stem cells differentiate into chondrocytes, and the differentiated chondrocytes can form a cluster. However, when electrical stimulation is continuously applied, the amount of differentiation of chondrocytes increases, and then the amount of increase stagnates at some point. This phenomenon can be explained in more detail with reference to FIG. 3 .

Figure 3 shows an example of a photographic image observed over time when electrical stimulation is applied to a stem cell culture medium by modeling.

The large circles shown in (a) to (h) of FIG. 3 represent the shape of the outer edge of the incubator 10 when viewed from top to bottom when the incubator 10 is a circular petri dish. From (a) of FIG. 3 to (h) of FIG. 3 , the result of the electrical stimulation process progressing further with time is shown. (a) of FIG. 3 shows the start time of electrical stimulation.

The small circles shown in (b) to (h) of FIG. 3 represent the shape of the outer edge of the chondrocyte population differentiated by electrical stimulation. In fact, the shape of the chondrocyte population may not be exactly circular, but may be modeled as approximately circular. In the small circles shown in (b) to (h) of FIG. 3, not only differentiated chondrocytes but also stem cells may be mixed together. can be considered

Referring to FIG. 3 , in (a) of FIG. 3 , no separate chondrocyte population is observed because it is the time point at which electrical stimulation begins. From FIG. 3(a) to FIG. 3(e), it can be observed that the area of the cluster of chondrocytes gradually decreases due to electrical stimulation. Although the area of the cluster of chondrocytes decreases, the amount of chondrocytes actually present in the cluster gradually increases with time. As shown in (e) of FIG. 3, when the amount of chondrocytes reaches the maximum value, even if electrical stimulation is further applied, chondrocytes are not further differentiated as shown in (f) of FIG. 3, and the maximum state is instantaneously or constant can be maintained for a period of time. Furthermore, if the electric stimulation is continued without stopping thereafter, as shown in (g) and (h) of FIG. 3, the area of the cluster of chondrocytes begins to increase again, resulting in a side effect of reducing the amount of actually differentiated chondrocytes. will do Therefore, in order to optimize the electrical stimulation process for differentiating chondrocytes from the stem cell culture medium, it is necessary to cut off the current provided to the stem cell culture medium at an appropriate time. In the example of FIG. 3, it is preferable to block the current in FIG. 3(e) and FIG. 3(f), and even if the current is blocked in FIG. 3(e), chondrocytes can be obtained as much as possible, and stem cells It has the advantage of minimizing the time for obtaining chondrocytes from the culture medium and reducing current consumption.

Now, with reference to FIG. 2 again, a method of setting the value of the aforementioned reference label will be described.

The value of the reference label assigned to the reference photographed image captured when the amount (number) of chondrocytes included in the chondrocyte population reaches a substantially maximum value may be set to a specific value, for example, 1.

Here, the subject who determines that the differentiation amount of the chondrocytes has reached the maximum value is the expert (human). The differentiation amount of the chondrocytes corresponding to the maximum value may be referred to as a first differentiation amount, a standard differentiation amount, or a maximum differentiation amount, and the reference differentiation amount may be normalized to a specific value as described above. For example, the reference differentiation amount may be presented as 1.

Next, the supervised learning control unit 520 provided according to an embodiment of the present invention provides a unique label value determined according to a predetermined algorithm to each of the captured images other than the reference captured image among the plurality of captured images. can be granted. A label value assigned to each of the remaining captured images may be equal to or less than the specific value of the reference label. For example, a label value assigned to an arbitrary second captured image among the remaining captured images may be a value of 1 or less.

The supervised learning control unit 520 may determine a label value assigned to each of the remaining captured images based on the reference label value, the standard captured image, and each of the captured images. For example, the differentiation amount of chondrocytes estimated from the reference photographed image is referred to as the reference differentiation quantity, the differentiation amount of chondrocytes estimated from the second photographed image is referred to as the second differentiation quantity, and If the label to be assigned is the second label, the relationship of Equation 1 can be established.

[Formula 1]

Second differentiation amount/(second label value)=(standard differentiation amount)/(reference label value)

In Equation 1 above, the value of the reference label is a value already assigned by an expert. For example, the value of the reference label may be 1.

Further, the reference differentiation amount may be a normalized value based on a preset value. In one embodiment of the present invention, regardless of the specific value of the total number of differentiated chondrocytes, the reference differentiation amount, which is the differentiation amount of the chondrocytes at the point of maximum differentiation of the chondrocytes determined by the expert, is 1. can also be defined.

In addition, the second differentiation amount is a value that can be determined by a method described later. Accordingly, the value of the second label to be assigned to the second captured image may be determined as in [Equation 2].

[Equation 2]

(value of the second label) = (value of the reference label) * amount of differentiation of the second / (amount of reference differentiation)

If Equation 2 is generalized, the value of the kth label for the kth photographed image can be determined as in Equation 3.

[Formula 3]

(value of k-th label) = (value of reference label) * k-th differentiation amount / (standard differentiation amount)

A method of estimating the amount of differentiation of chondrocytes from each of the photographed images may be presented as follows. That is, a method of estimating the kth differentiation amount from the kth photographed image may be presented as follows.

First, the supervised learning controller 520 can identify the edge of the incubator 10 from the photographed image. The incubator 10 may be a circular or rectangular dish. At this time, the edge of the incubator 10 may be, for example, a circular closed curve as shown in FIG. 3 .

Then, the supervised learning control unit 520 may classify the captured image of the stem cell culture solution present in the incubator 10 into two regions according to a predetermined criterion. These two areas are presented in FIG. 3 .

The first area (outer area) 110 of the two areas may be a donut-shaped area having an outer closed curve 111 and an inner closed curve 112 . The outer closed curve 111 may be an outer edge of the incubator 10 . A second area (inner area) 120 of the two areas may be an area within the inner closed curve 112 . In one embodiment, the inner closed curve 112 may be modeled and defined as a circle or an ellipse.

A method of determining the inner closed curve in an arbitrary captured image according to an embodiment of the present invention will be described with reference to FIG. 4 .

4 is a diagram for explaining a method of dividing an acquired captured image into two regions according to an embodiment of the present invention.

A method of determining the inner closed curve 112 in an arbitrary captured image may include the following steps. First, the supervised learning controller 520 connects the central point 1120 of the incubator 10 from the first outer point 1110, which is the edge point of the incubator 10, in the arbitrary captured image. The route P1 can be determined. The path P1 may be straight or curved. When the length of the path P1 is L1, a second point 1121 moved from the first outer point to the center point by dL along the path P1 may be defined. In this way, the k+1th point moved from the kth point to the center point by dL can be continuously defined (k is a natural number greater than or equal to 1, dL < L1 ). In this case, a difference between the image attribute value at the kth point and the image attribute value at the k+1th point may be defined as Dk. The image attribute value may be, for example, an attribute value such as brightness, color value, or saturation at each point of the arbitrary photographed image. The image attribute values are sufficient according to criteria for classifying image attributes in the field of image processing, and the present invention is not limited by the specific attribute selection.

Among the difference values Dk (k=1, 2, 3, ...) between the image attribute values defined as described above, the largest value may be selected. For example, if the difference value D3 is the largest value, it can be considered that the image attribute value has changed the most between the second point 1121 and the third point 1131 on the path P1.

Now, it can be determined that the first inner point 1115 constituting the inner closed curve 112 exists between the second point 1121 and the third point 1131 .

The process of determining the first inner point 1115 starts from the step of selecting the first outer point of the edge of the incubator 10 . Now, if the process of determining the first inner point is performed starting from various other outer points of the edge of the incubator 10, a plurality of other inner points constituting the inner closed curve 112 can be determined.

The inner closed curve 112 can be completed by interpolating and connecting the plurality of inner points determined in this way. As a result, the first region (outer region) 110 and the second region (inner region) 120 can be completely defined.

If the largest value among the difference values Dk (k=1, 2, 3, ...) is smaller than a predetermined value, the first inner point 1115 is determined to be the first outer point 1110. may be

In some cases, the second area (inner area) 120 and the first area (outer area) 110 may not exist. Such a situation may occur, for example, in a state in which electrical stimulation is not applied to the stem cell culture medium at all. For example, the inner region 120 and the outer region 110 cannot be defined in (a) of FIG. 3 .

The above-described method of determining the first inner point is the second area (inner area) 120 approximated to a circle or ellipse and the first area outside the second area (inner area) 120. It is presented as one of various algorithms for classifying the area (outside area) 110 . That is, the above information is intended to explain the feasibility of dividing the captured image of the stem cell culture solution present in the incubator 10 into the above two areas, and the present invention is not limited to these embodiments.

The division of the photographed image into two regions is performed on each of a plurality of photographed images captured according to the lapse of time in which the electrical stimulation is applied.

When the second area (inner area) 120 is defined in this way, the area SB of the second area (inner area) 120 can be calculated.

Now, when the amount of differentiation of chondrocytes estimated from the kth photographic image is referred to as the kth differentiated quantity, and the area of the second region (inner region) 120 calculated from the kth photographic image is referred to as the kth area , The kth differentiation amount of chondrocytes estimated from the kth photographic image may be defined as being in inverse proportion to the kth area.

Alternatively, in another embodiment, the kth differentiation amount of chondrocytes estimated from the kth photographic image may be defined as being in inverse proportion to the root value of the kth area. That is, when the second region (inner region) 120 is modeled as a circle, the kth differentiation amount of chondrocytes estimated from the kth photographed image may be defined as being in inverse proportion to the diameter or radius of the circle. .

However, depending on embodiments, a rule may be established that the kth differentiation amount is estimated based on the kth area, but a nonlinear function intervenes in the relationship between the kth differentiation amount and the kth area. According to an aspect of the present invention, the kth differentiation amount is estimated based on the kth area, but the present invention is not limited by the specific functional relationship.

In addition, although the kth differentiation amount is estimated based on the kth area, values of other additional parameters observed from the kth photographed image may also be further used to estimate the kth differentiation amount. However, regardless of how the value of this additional parameter is defined, the fact that the kth differentiation amount is estimated based on the value for the kth area remains unchanged.

For example, the other additional parameter may be defined as a ratio between the first average brightness of the outer region and the second average brightness of the inner region observed in the kth photographed image.

In this way, when the kth differentiation amount is estimated from the kth photographed image, the value of the kth label can be determined using Equation 3.

Labels assigned to a series of photographed images of the stem cell culture medium whose state is changed by receiving electrical stimulation through the above-described process may be defined or calculated and obtained.

Now, the supervised learning control unit 520 inputs the kth captured image to the input layer of the neural network unit 510, and the value output by the neural network unit 510 sets the kth label corresponding to the kth captured image. The internal parameters of the neural network unit 510 may be updated so that an error of arithmetic is reduced.

In one embodiment, the neural network unit 510 may be, for example, a Convolutional Neural Network (CNN). The CNN is a well-known technology, and an example thereof is not separately described in this specification. Nevertheless, it is not difficult for those skilled in the art to apply any CNN implementation technology to the spirit of the present invention presented herein.

Learning of the neural network unit 510 may be repeated using a plurality of photographed images and labels differentiated for them.

In addition, learning of the neural network can be further repeated using a plurality of images taken while applying electric stimulation to each of the skin culture medium contained in the plurality of incubators 10 and corresponding labels.

In the above-described neural network learning method, the input data for learning of the neural network is the photographic data, and one feature of the present invention is a method of making the labels corresponding to the answer sheet corresponding to each of the learning input data.

<Method for Determining End Time of Electrical Stimulation Using the Electrical Stimulation System According to the First Embodiment>

According to the electrical stimulation control method provided according to one aspect of the present invention, it is possible to determine when to cut off the current provided to the stem cell culture medium contained in the incubator 10. The electrical stimulation control method may be performed using the electrical stimulation system 100 shown in FIG. 1 .

5 is a flowchart illustrating a method for determining an electrical stimulation end point using an electrical stimulation system provided according to an embodiment of the present invention.

First, in step S10, the driving current control unit 30 may apply current to the incubator 10 containing the stem cell culture medium using a pair of electrodes 20.

Next, in step S20 , the photographing device 40 may transmit photographed images of the stem cell culture medium to the estimator 50 at predetermined time intervals.

Next, in step S30 , the estimator 50 may input each transferred captured image to the input layer of the neural network unit 510 of the estimator 50 . At this time, the estimator 50 may output a scalar value.

Next, in step S40, the drive current control unit 30 may determine whether or not to terminate the provision of the electrical stimulation based on a history over time of a series of output values output by the estimator 50. .

For example, the driving current control unit 30 may determine that the application of the current is terminated at the moment when it is determined that the output value output by the estimating unit 50 will not increase any more. To this end, for example, the driving current control unit 30 observes the moving average of the output values output by the estimating unit 50, and when the moving average value starts to decrease, or when it is saturated for a certain period of time and does not increase any more. It may be decided to terminate the application of the current at a point in time.

Next, the driving current controller 30 may cut off the current applied to the incubator 10 containing the stem cell culture medium when the end time is determined.

In a modified embodiment, the function module for determining whether to end the provision of the electrical stimulation based on the history of a series of output values output by the estimator 50 over time is provided to the drive current control unit 30. It may not exist and may exist in the estimator 50 . In this case, the estimator 50 may provide a separate signal indicating the end timing of the electric stimulation to the drive current controller 30, and the estimator 50 may send the output value to the drive current controller 30. It may not be provided separately.

As described above, a method of blocking the electrical stimulation provided by the driving current control unit 30 to the stem cell culture medium can be described with reference to FIG. 6 .

6 is an example of a graph showing values output by an estimation unit of an electrical stimulation system according to an embodiment of the present invention according to time.

The graph of FIG. 6 may show the circular value output by the estimator 50 as it is, or may be a graph obtained by averaging the circular value over time.

In FIG. 6, the output value of the estimator 50 increases sequentially in the interval t0 to t1, the output value of the estimator 50 is maintained in the interval t1 to t2, and the output value of the estimator 50 decreases in the interval t2 to can In some cases, the time period t1 to t2 does not substantially exist, and the output value of the estimator 50 may start to decrease immediately after t1.

When observing the curve presented in FIG. 6 in real time according to time, ideally, the end of the electric stimulation should be commanded at the time point t1, but whether the amount of differentiation of chondrocytes has reached the maximum value Observe the output value of the estimation unit Since a certain amount of time delay is required to determine the electrical stimulation, the end of the electrical stimulation may be actually commanded after the time point t1.

<The learning method of the neural network unit according to the second embodiment>

The learning method according to the second embodiment of the present invention is also a supervised learning method.

Figure 7a is for explaining a method of assigning a label according to the state of the captured image of the stem cell culture medium contained in the incubator according to an embodiment of the present invention.

Figure 7a conceptually displays images observed at eight times (t0 to t7) while continuously applying electrical stimulation to the stem cell culture medium.

From time t0 to time t4, the amount of chondrocytes contained in the culture medium gradually increases with time. At this time, the area of the image portion 120 representing the cluster of chondrocytes gradually decreases with time. .

The time t4 indicates the time point when the amount of chondrocytes is maximized.

When electrical stimulation is continuously maintained from time t4 to time t7, the area of the image portion 120 representing a cluster of chondrocytes gradually increases again over time, while the amount of chondrocytes may actually gradually decrease over time. there is.

At this time, the texture represented by the image of the chondrocyte cluster from time t0 to time t4 may be different from the texture represented by the image of the chondrocyte cluster from time t4 to time t7. This difference in texture can be recognized by an expert observing chondrocytes. That is, for example, from time t4 to time t7, partially dead chondrocytes are generated in the cluster of chondrocytes, and the inner closed curve 112 observed from time t4 to time t7 is displayed at time t0. It can be more vaguely defined than the inner closed curve 112 observed from to the time t4. That is, it may become difficult to clearly define the boundary between the image portion 120 representing the cluster of chondrocytes and the remaining area 110 outside the image portion 120 .

The description of FIG. 4 assumes that it is possible to define the inner closed curve 112, but in the second embodiment, the end point of providing electrical stimulation is determined without defining the inner closed curve 112. .

Experts can distinguish between the first time period (t0 to t4), in which the amount of differentiation of chondrocytes increases as electrical stimulation is maintained, and the second time period (t4 to t7), in which the amount of differentiation of chondrocytes decreases as electrical stimulation is maintained. can

It may be prepared by taking images of a plurality of stem cell culture medium at regular intervals or according to a predetermined time schedule while continuously providing electrical stimulation to the stem cell culture medium.

The expert can distinguish which of the first time period and the second time period each image belongs to. The expert may assign a label having a value of '1' to an image belonging to the first time period, and may assign a label having a value of '0' to an image belonging to the second time period. Here, '1' may indicate that electrical stimulation may be continued, and '0' may be a value indicating that electrical stimulation should be discontinued. Instead of '1' and '0', you can also label them with other characters.

A plurality of stem cell culture medium samples are prepared, electrical stimulation is continuously provided to each of the stem cell culture medium samples, a plurality of images taken over time for the differentiation process of the stem cell culture medium sample are prepared, and each of the prepared images The expert assigns a label to each other, and a plurality of labels assigned in this way may be prepared. For example, for each of 500 stem cell culture medium samples, 100 images may be taken and obtained over time while maintaining electrical stimulation. As a result, a total of 50,000 images can be prepared, and labels with '1' or '0' assigned to each image can be prepared. One label value may correspond to each image, and information composed of a specific image and a label corresponding to the specific image may be referred to as image-label pair information. A total of 50,000 such image-label pair information may be prepared.

The captured image is input to the input layer of the neural network unit 510, and a label value assigned to the input image for supervised learning may be used as an answer for supervised learning.

FIG. 7B illustrates the configuration of the neural network unit shown in FIG. 1 .

The neural network unit 510 may be composed of a plurality of layers. Each layer may include a plurality of nodes. Each of the above nodes may have variable values. The variable value may have a data value provided to the node. Among two adjacent layers, nodes included in the first layer and nodes included in the second layer may be connected by a plurality of links. Variable values of nodes included in the first layer may be transferred to variables included in nodes included in the second layer. At this time, the forwarded path is the links. Weight values ( V , W ) may be assigned to each link. A value of a variable transmitted from the first layer to the second layer through a specific link may be divided into the specific link and multiplied by a weight value. The input/output characteristics of the neural network unit 510 can be determined by the number of layers constituting the neural network unit 510, the number of nodes included in each layer, the connection relationship between links, and specific values of weights assigned to each link. there is.

7C is for explaining a method of training a neural network unit according to an embodiment of the present invention.

Hereinafter, it will be described with reference to FIG. 7C.

Whenever a training image provided for learning of the neural network unit is input to the input layer of the neural network unit, the output layer (=output layer) of the neural network unit outputs an output value. This output value can be a real number between 0 and 1. At this time, the value of the label paired with the learning image may be input to the supervised learning control unit 520 . The value of the label is a value of a label directly assigned by an expert to the training image and may have a value of 1 or 0.

The supervised learning control unit and the neural network unit may be provided as dedicated hardware installed in the computing device or provided by an algorithm executed by a set of instruction codes loaded from a memory in the computing device to the CPU of the computing device. .

Values of the weights assigned to the neural network unit may be updated in a direction that minimizes an error between the output value output by the output layer and the label value input to the supervised learning control unit. The process of minimizing the error and updating values of the weights may be executed by the supervised learning control unit. Specifically, the method may use various conventional techniques such as a back-propagation method.

For example, the learning of the neural network unit 510 may be performed 50,000 times using 50,000 images and the image-label pair information generated by an expert's judgment. Information on the 50,000 image-label pairs may be stored in a table form in a memory of a computing device. The task of generating and storing the table-type information in the computing device may be completed by a separate process.

50,000 times of learning may be regarded as having performed one set of learning. This one set of learning can be repeated multiple times. When a well-designed neural network unit is repeatedly trained, an image taken in a state in which electrical stimulation can be continued further is input to the input layer of the neural network unit on which learning has been completed. is output, and when an image taken in a state in which the electrical stimulation is to be stopped is input, an output value of 1 or less and 0 or more close to '0' may be output from the output layer of the neural network unit.

FIG. 7D illustrates the structure of an estimation unit provided after the learning of the neural network unit described in FIG. 7C is completed, including the neural network unit for which the learning is completed.

As described in FIG. 1 , the estimator 50 may be a computing device. The computing device may include a CPU, a power supply unit, a memory unit, a communication unit, a peripheral device interface unit, a neural network unit 510, and a post-processing unit 530. The memory unit may include volatile and/or non-volatile memory. Functions of each component may be the same as those shown in FIG. 1 . Hereinafter, it will be described with reference to FIG. 7D.

In a preferred embodiment, when the learning-completed neural network unit 510 receives an image of the stem cell culture medium from the photographing device 40, the output layer of the neural network unit 510 outputs a real number of 0 or more and 1 or less. There may be.

The post-processing unit 530 finally outputs a voltage value (ex: +12V) corresponding to '1' when the output value of the neural network unit 510 is greater than the threshold value (ex: 0.5), and is smaller than the threshold value. If it is, finally, a voltage value (ex: 0V) corresponding to '0' can be output. The voltage value (ex: +12V) corresponding to the final '1' is referred to as a voltage value corresponding to a logical high, and the voltage value (ex: 0V) corresponding to the final '0' corresponds to a logical low It can be referred to as a voltage value that

An output terminal of the estimator 50 may be connected to the driving current controller shown in FIGS. 1 and 9 .

FIG. 8A shows the values output by the estimator 50 over time when photographed images over time of the stem cell culture medium to which electrical stimulation is being provided are sequentially provided to the estimator 50 of FIG. 7D. it did If the photographed image is discretely provided to the estimator on the time axis, values output by the estimator may also be provided discretely on the time axis. In FIG. 8A, '1(ON)' on the vertical axis may actually be a voltage value such as +12V, and '0(OFF)' may actually be a voltage value such as 0V. In FIG. 8A, the estimator provides a valid value (0V or +12V) only at the time when there is a black dot, and at the rest of the time, the output terminal of the estimator is left in a floating state, or a value other than the above valid value, for example, -12V. can also be printed out. If a value other than the above valid value is input, the driving current control unit 30 may process it as an invalid value.

FIG. 8B shows a first signal 91 for controlling on/off of the drive current controller 30 using the output of the estimation unit 50 shown in FIG. 8A. The first signal 91 may be, for example, a TTL level voltage signal. The first signal 91 can be obtained by, for example, a sample-and-hold circuit described later. In FIG. 8B, '1 (ON)' on the vertical axis may actually be a voltage value such as 5V or 12V, and '0 (OFF)' may actually be a voltage value such as 0V.

8C is a second signal 92 representing a time moving average of the first signal 91 shown in FIG. 8B. For example, the value at time tk of the second graph 92 may be given as an average value of values from time tk-Δ to time tk of the first signal 91 . The graph of FIG. 8c is arbitrarily shown for illustrative purposes, and the result of the time moving average of FIG. 8b may have a different form from that shown in FIG. 8c. The second signal 92 may be, for example, a TTL level voltage signal. The second signal 92 can be obtained, for example, by a moving average circuit described later. In FIG. 8C, '1' on the vertical axis may actually be a voltage value such as 5V or 12V, and '0' may actually be a voltage value such as 0V.

FIG. 8D is a third signal 93 having only a value corresponding to '1' or '0', which is generated based on the second signal 92 shown in FIG. 8C. The third signal 93 may be, for example, a TTL level voltage signal. The third signal 93 can be obtained by, for example, a binary digitizer 33 to be described later. The binary digitizer 33 outputs '1' or a value corresponding to '1' if the input value is greater than a predetermined threshold value, and outputs a value corresponding to '0' or '0' if it is smaller. There may be. The threshold value may be, for example, '0.5' or a value corresponding to '0.5'. In FIG. 8D, '1(ON)' on the vertical axis may actually be a voltage value such as 5V or 12V, and '0(OFF)' may actually be a voltage value such as 0V.

9 shows the configuration of the driving current controller 30 provided according to an embodiment of the present invention.

The driving current controller 30 may include a sample-and-hold circuit 31, a moving average circuit 32, a binary digitizer circuit 33, a selector 34, and a current driving circuit 35.

The driving current controller 30 may discretely receive an estimated value (a TTL level voltage indicating '1' or '0') from the estimator 50 over time.

The estimated value may be given in the form of, for example, FIG. 8A.

When the estimated value is a voltage signal, the sample-and-hold circuit 31 may receive the estimated value and output a continuous output voltage. For example, the sample-and-hold circuit 31 may output the first signal 91 as shown in FIG. 8B. A number of specific configurations of the sample-and-hold circuit 31 have been disclosed in the prior art.

The moving average circuit 32 may receive the first signal 91 and output a second signal 92 as shown in FIG. 8C. A number of specific configurations of the moving average circuit 32 are disclosed in the prior art.

The binary digitizer circuit 33 may receive the second signal 92 and output a third signal 93 as shown in FIG. 8D, for example.

The selector 24 may select and output one of the first signal 91 and the third signal 93 . The operation of the selector 24 may be controlled by a user interface switch (not shown) provided to the driving current controller 30 .

The current driving circuit 35 may or may not provide current to the pair of electrodes 20 by being controlled by the voltage output to the selector 34 . The voltage that is the value output by the selector 34 may have only a first voltage substantially indicating switch-on, for example, +12V, and a second voltage substantially indicating switch-off, eg, 0V. Accordingly, the signal input to the current driving circuit 35 is a signal output from the selector 34 and may be a signal that determines the on/off state of the current driving circuit 35 . That is, the signal output by the selector 34 is an off/on control signal.

10 illustrates a method of photographing a plurality of incubators according to an embodiment of the present invention.

10, a total of 15 incubators forming 3 rows by 5 columns are arranged on the plate 1.

An electrode pad 11 is installed on the plate 1 . A conductor is connected from the electrode pad 11 to each incubator, and each conductor may be connected to a pair of electrodes 20 disposed in the incubator. One conductor line shown in FIG. 10 simply represents a pair of conductor lines actually connected to the pair of electrodes, respectively.

Since the photographing device 40 is expensive, only one photographing device 40 may be used for one plate 1 . The photographing device 40 is a camera and may be moved to the positions of 15 incubators by an x-axis linear motor and a y-axis linear motor (not shown). A control device for controlling the operation of the x-axis linear motor and the y-axis linear motor may be separately provided.

The above-described present invention can be equally applied to an embodiment of differentiating chondrocytes from skin cells, not to an embodiment of differentiating chondrocytes from stem cells. Therefore, in the above-described embodiments, the stem cell culture medium may be replaced with the skin cell culture medium.

Using the above-described embodiments of the present invention, those belonging to the technical field of the present invention will be able to easily implement various changes and modifications without departing from the essential characteristics of the present invention. The content of each claim of the claims may be combined with other claims without reference relationship within the scope understandable through this specification.

10: Incubator
20: a pair of electrodes
30: drive current controller
31: sample-and-hold circuit
32: moving average circuit
33: binaryr digitizer
34: selector
35: current drive circuit
50: estimator (computing device)
40: filming device
510: neural network
530: post-processing unit
100: electrical stimulation system

Claims (7)

a drive current controller 30 configured to provide a pair of electrodes 20 with a drive current for providing electrical stimulation to the stem cell culture medium contained in the incubator 10;
estimation unit 50; and
a photographing device 40 configured to provide photographed images of the stem cell culture solution at predetermined time intervals to the estimator 50 in real time;
Including,
The estimation unit 50,
It includes a neural network unit 510,
Each of the plurality of captured images is input to the input layer of the neural network unit 510, and
Based on the value output by the neural network unit 510, an estimated value having a voltage corresponding to a logical high or a voltage corresponding to a logical low is output,
The drive current controller 30,
to receive the estimated value, and
The estimated value is used as an on/off control signal of the driving current,
Electrical stimulation control system. According to claim 1,
The output layer of the neural network unit outputs a real number of 0 or more and 1 or less,
The estimation unit further includes a post-processing unit 530 connected to the output layer,
The post-processing unit outputs a voltage corresponding to the logical high when the output value of the output layer of the neural network unit is greater than a predetermined threshold value greater than 0 and less than 1, and if the output value of the output layer of the neural network unit is less than the threshold value, the logical It is designed to output a voltage corresponding to a low,
Electrical stimulation control system. The method of claim 1, wherein the drive current control unit 30 samples-and-holds the input estimated value to generate an on/off control signal having only a voltage corresponding to the logical high and a voltage corresponding to the logical low. An electrical stimulation control system comprising a sample-and-hold unit 31 outputting 1 signal 91. According to claim 3,
The drive current controller 30,
a moving average circuit (32) generating a second signal (92) representing the moving average value of the first signal (91); and
When the second signal 92 is greater than a predetermined second threshold value, it has a voltage corresponding to the logical high level, and when the second signal 92 is less than the second threshold value, it has a voltage corresponding to the logical low level. a binary digitizer 33 outputting a third signal 93 which is an on/off control signal having;
Including more,
Electrical stimulation control system. According to claim 4,
The drive current controller 30,
a current driving circuit (35) for outputting the drive current to supply the pair of electrodes (20); and
A selector 34 for selecting one of the first signal 91 and the third signal 93 and providing it to the current driving circuit as an on/off control signal of the current driving circuit
Including more,
The current driving circuit is configured to provide the driving current to the pair of electrodes 20 when the signal received from the selector 34 has a voltage corresponding to the logical high. Stop providing the driving current to the pair of electrodes 20 when the received signal has a voltage corresponding to the logical low.
Electrical stimulation control system. According to claim 1,
The estimation unit 50 is a computing device including a CPU, a power supply unit, a memory unit, a communication unit, and a peripheral device interface unit,
The memory unit stores command codes for implementing an algorithm defining a relationship between an input and an output of the neural network unit,
The function of the neural network unit is provided by the CPU reading and executing the instruction codes.
Electrical stimulation control system. According to claim 6,
The neural network unit is learned by supervised learning,
The process for the supervised learning,
Information on a plurality of image-label pairs generated by labels having a value of '1' or '0' assigned by an expert to each of the learning images captured by the photographing device, and reading from the memory unit;
inputting an image for training selected from among the plurality of image-label pair information to an input layer of the neural network unit;
obtaining an output value from an output layer of the neural network unit; and
updating weight values of the neural network unit to reduce an error generated by comparing a value of a label corresponding to the selected training image with the output value obtained from the output layer;
Characterized in that it includes,
Electrical stimulation control system.

Images