JP2022121560A - Fertilization determination system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system of confirming whether an egg which has undergone fertilization treatment has been fertilized.

SOLUTION: The fertilization determination system comprises: a culture part for culturing the egg that has undergone fertilization treatment; an image acquisition part for acquiring a plurality of images of an egg cultured in the culture part captured at preset time intervals; and a determination part for determining whether the egg has been fertilized by analysing the state of the egg captured in the images.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a fertilization determination system.

BACKGROUND ART In the field of infertility treatment, embryo selection is performed by evaluating embryos in culture from the viewpoint of improving the conception rate and patient's QOL. As a non-invasive method of evaluating embryos, there is a method of evaluating the state of development of embryos based on morphological observation under a microscope.

JP 2012-29686 A

In the embryo sorting system of Patent Literature 1, embryos are sorted using the embryo morphology, oxygen consumption, and the like as indices. However, the system for confirming fertilization, which is a prerequisite for embryogenesis, has not been sufficiently considered so far. In addition, there is a limit to the number of specimens that can be confirmed, as humans have been visually checking each egg that has undergone fertilization treatment, such as insemination, to see if it is fertilized normally. there were. In order to solve such problems, there has been a demand for a technology that enables the construction of a system for confirming whether fertilized eggs are fertilized and the confirmation of fertilization in a larger number of specimens than humans can confirm. .

The present invention has been made to solve at least part of the above problems, and can be implemented as the following modes.

(1) According to one aspect of the present invention, a fertilization determination system is provided. This fertilization determination system includes a culture unit that cultures fertilized eggs, and an image acquisition unit that acquires a plurality of images by photographing the state of the eggs cultured by the culture unit at set time intervals. and a determination unit that determines whether the egg is fertilized by analyzing the state of the egg reflected in the image. With this aspect, the determination unit can determine whether the egg is fertilized based on the analysis of the captured image of the state of the fertilized egg. For this reason, a system is provided that can confirm whether an egg that has been fertilized is fertilized. In addition, since eggs, which are specimens, are mechanically analyzed, more specimens can be confirmed than can be confirmed by humans. In addition, since the determination unit determines whether or not the fertilization has occurred, it is possible to reduce the complexity of the determination compared to a human's visual confirmation.

(2) In the fertilization determination system according to the above aspect, the determination unit determines whether the egg is fertilized by analyzing temporal changes of the egg in at least two images acquired at different times. may According to this aspect, by analyzing at least two images, changes in the egg over time can be analyzed. Therefore, it is possible to determine whether the egg is fertilized based on the change over time of the egg.

(3) In the fertilization determination system according to the above aspect, the determination unit may learn characteristics of the egg captured in the image by supervised learning, and determine whether the egg is fertilized based on the learning. . With this aspect, it is possible to determine whether the egg is fertilized based on the criteria learned by the determination unit through supervised learning.

(4) In the fertilization determination system according to the above aspect, the determination unit learns the features of the egg shown in the image by deep learning using a multilayer neural network, and determines whether the egg is fertilized based on the learning. It may be determined whether With this aspect, it is possible to determine whether the egg is fertilized based on the criteria learned by the determination unit through deep learning.

(5) In the fertilization determination system according to the above aspect, the determination unit may have a notification unit that notifies a determination image used for determination among the images. With this aspect, the user who uses the fertilization determination system can know which image the determination unit used to determine fertilization.

(6) In the fertilization determination system according to the above aspect, the notification unit may notify the determination image and information based on which the determination unit bases the determination in the determination image. With this aspect, the user who uses the fertilization determination system can know which image the determination unit used to determine fertilization, and can know the information based on which the determination unit made the determination. can.

(7) According to one aspect of the present invention, there is provided a method for selecting unfertilized eggs. This method for selecting unfertilized eggs is a method for selecting unfertilized eggs that are not normally fertilized among fertilized eggs, and comprises a culturing step of culturing the eggs, and an image acquisition step of acquiring a plurality of images by photographing the state of the egg in the fertilized egg at a set time interval; and a selection step of selecting the unfertilized egg by analyzing the state of the egg reflected in the image. And prepare. With this aspect, non-fertilized eggs can be selected based on the analysis of images of the state of fertilized eggs. Therefore, a method is provided for selecting non-fertilized eggs that have not been fertilized normally among fertilized eggs.

The present invention can also be implemented in various forms other than the fertility determination system. For example, the present invention can be implemented in the form of a fertility determination device. Moreover, the present invention is not limited to the above-described forms, and can of course be embodied in various forms without departing from the gist of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS It is explanatory drawing which shows the structure of the fertilization determination system in 1st Embodiment. An example of an image labeled as an egg in which no pronuclei were identified. An example of an image labeled as an egg with one confirmed pronucleus. Example image labeled as an egg with two confirmed pronuclei. An example of an image labeled as an egg with 3 confirmed pronuclei. 6 is a flow showing image acquisition processing executed by a control unit; It is a flow which shows the fertilization determination process which a control part performs. FIG. 4 is an explanatory diagram illustrating image learning by deep learning;

A. First embodiment:
FIG. 1 is an explanatory diagram showing the configuration of a fertilization determination system 10 according to the first embodiment. The fertilization determination system 10 is a system that determines whether a fertilized human egg is fertilized. A fertilized egg is an egg that has been subjected to a fertilization treatment such as a conventional method of co-cultivating sperm and ovum or a micro-insemination method. The fertilization determination system 10 includes a culture section 110 , an image acquisition section 130 , a control section 140 , a user interface 150 and a notification section 160 .

The culture unit 110 is a so-called incubator that cultures fertilized eggs. Culturing conditions such as temperature, humidity, oxygen concentration, carbon dioxide concentration, and culturing time in the culturing unit 110 are controlled by the control unit 140 based on the content previously input by the user via the user interface 150 . The culturing unit 110 cultures the eggs while the culturing container 200, which is a container for culturing fertilized eggs, is fixed in the culturing unit 110. FIG.

The culture container 200 is a well plate having 12 wells arranged in 3 rows and 4 columns. Numbers 1 to 12, which are well numbers, are attached to the bottom of each well of the culture container 200 . The culture container 200 is a container for culturing one egg in each well.

The culture section 110 has a container transfer section 115 . The container transfer section 115 has a shape extending in the horizontal direction and constitutes the bottom portion of the culture section 110 . The vessel transfer section 115 has a fixing section (not shown) for fixing the culture vessel 200 on the surface facing upward in the gravitational direction. The culture unit 110 cultures the eggs while the culture container 200 is fixed to the fixing unit.

The container conveying unit 115 conveys the culture container 200 fixed to the fixing unit to a position below the image acquisition unit 130 in the gravitational direction, a part of which protrudes inside the culturing unit 110, by moving the fixing unit. . After the acquisition of the image by the image acquisition unit 130 is completed, the container transport unit 115 transports the culture container 200 to the initial position. In FIG. 1, the position where the culture vessel 200 is illustrated is the initial position.

The frequency (time interval) at which the container transporter 115 transports the culture container 200 is controlled by the control unit 140 based on the content previously set by the user via the user interface 150 . In this embodiment, the container transporter 115 is controlled to transport the culture container 200 to a position below the image acquisition unit 130 in the direction of gravity every 15 minutes. After the culture container 200 is transported to a position below the image acquisition unit 130 in the direction of gravity, the container transport unit 115 two-dimensionally adjusts the position of the culture container 200 as seen from the image acquisition unit 130 in the vertical and horizontal directions. , each well can be placed at a position where the image acquisition unit 130 can acquire an image. After the acquisition of the image by the image acquisition unit 130 is completed, the container transport unit 115 transports the culture container 200 to the initial position. A process in which the image acquisition unit 130 acquires an image of each well in the culture container 200 will be described later.

The image acquiring unit 130 acquires an image by photographing the state of the egg cultured by the culturing unit 110 at set time intervals. In this embodiment, the image acquiring unit 130 captures the state of the egg each time the container transport unit 115 transports the culture container 200, thereby obtaining a plurality of images for each well in the culture container 200 in time series. to get to. In another embodiment, the image acquisition unit 130 may acquire images by capturing images at time intervals directly instructed by the control unit 140 . The position of the well from which the image acquisition unit 130 acquires an image among the 12 wells of the culture container 200 is specified in advance by the user via the user interface 150 . In the following description, the position of the well designated by the user will be referred to as "designated position". In this embodiment, the image acquisition unit 130 is a CCD camera.

The control unit 140 is configured by a microcomputer having a central processing unit and a main memory. The control section 140 controls each section of the fertilization determination system 10 . In addition, the control unit 140 controls the culture unit 110, the container transport unit 115, and the image acquisition unit 130 based on the content input by the user via the user interface 150 in advance.

Control unit 140 includes image storage unit 142 and determination unit 144 .

The image storage unit 142 stores the images acquired by the image acquisition unit 130 . The image storage unit 142 sends the acquired image of each well to the determination unit 144 .

The determination unit 144 determines whether the egg is fertilized by analyzing the state of the egg shown in the image sent from the image storage unit 142 . In the present embodiment, the determination unit 144 uses images of each well acquired by the image acquisition unit 130 at 15-minute intervals during 24 hours after the incubation unit 110 starts culturing eggs. determine if the is fertilized.

In this embodiment, the determination unit 144 determines whether the egg is fertilized by checking the number of pronuclei in the egg. In a normally fertilized egg, two pronuclei generally appear within 22 hours after fertilization.

In this embodiment, the determination unit 144 determines whether the egg is fertilized by analyzing two images acquired at different times. In the present embodiment, among the images acquired by the image acquisition unit 130, two images from which the difference in the number of pronuclei in the egg can be most remarkably recognized are determined as the two images acquired at different times. Section 144 selects. Only when the two images are one image in which the number of pronuclei in the egg was not confirmed and one image in which two pronuclei were confirmed in the egg, the determination unit 144 Determine if the egg is fertilized. In this way, since the temporal change of the egg in the two images can be analyzed, it is possible to determine whether the egg is fertilized based on the temporal change of the egg. In addition, by comparing the difference in the number of pronuclei between two images, fertilization can be determined more accurately than using one image.

The determining unit 144 learns the characteristics of the egg captured in the image by supervised learning, and determines whether the egg is fertilized based on the learning. In the present embodiment, the determination unit 144 learns how to determine the number of pronuclei in an imaged egg, and determines whether the egg is fertilized based on the learning. Therefore, the determining unit 144 can determine whether the egg is fertilized based on the criterion of the number of pronuclei learned by supervised learning.

2, 3, 4, and 5 are explanatory diagrams showing examples of images used by the determination unit 144 in supervised learning. FIG. 2 is an example of an image of a fertilized egg labeled as an egg in which no pronuclei were identified. FIG. 3 is an example image of a fertilized egg labeled as an egg with one confirmed pronucleus. FIG. 4 is an example image of a fertilized egg labeled as an egg with two confirmed pronuclei. FIG. 5 is an example image of a fertilized egg labeled as an egg with three confirmed pronuclei. Eggs that have been fertilized are generally determined to be unfertilized if no pronucleus is identified. In addition, when one or more pronuclei are confirmed in a fertilized egg, it is generally judged to be an abnormally fertilized egg.

User interface 150 exchanges information with a user of fertility determination system 10 . In this embodiment, the user interface 150 is a touch panel that displays an image and receives an instruction input from the user on the image. In other embodiments, user interface 150 may include push buttons that accept input of instructions from the user.

The notification unit 160 notifies the judgment image used for the judgment among the images. The notification unit 160 notifies the determination image while the user is browsing the determination result by the determination unit 144 on the screen of the touch panel which is the user interface 150 . Therefore, the user using the fertilization determination system 10 can know which image the determination unit 144 used to determine fertilization. In this embodiment, since the acquired time is attached to each acquired image, the user can know when the egg was fertilized by checking the time attached to the determination image. can.

In addition, the notification unit 160 notifies information on which the determination unit 144 bases the determination in the determination image. The notification unit 160 notifies information based on which the determination unit 144 makes a determination when the user is viewing the notified determination image on the screen of the touch panel that is the user interface 150 . In this embodiment, the notification unit 160 notifies the position information of the portion of the determination image that the determination unit 144 has recognized as the pronucleus.

FIG. 6 is a flow showing image acquisition processing executed by the control unit 140 . The control unit 140 executes image acquisition processing when the culture container 200 is transported to a position below the image acquisition unit 130 in the direction of gravity.

When the image acquisition process is started, the control unit 140 calculates a variable A representing the number of wells designated by the user (step S100). After calculating the variable A representing the number of designated wells (step S100), the control unit 140 displays the image of the well with the smallest number among the designated wells that have not been photographed. The acquisition unit 130 is caused to take an image (step S110). A number is a well number from 1 to 12 attached to the bottom of each well.

In step S110, the image acquiring unit 130 acquires an image of the well in the following steps (1) and (2). (1) The control unit 140 causes the container transfer unit 115 to two-dimensionally adjust the position of the culture container 200 to move the position of the well to a position where the image acquisition unit 130 can perform imaging. (2) The control unit 140 moves the well to a position where the image acquisition unit 130 can perform imaging, and then causes the image acquisition unit 130 to perform imaging.

After the image of the well with the smallest number among the designated wells that have not been photographed is captured (step S110), the control unit 140 stores the number of the well that has been photographed (step S120). After storing the number of wells for which imaging has been completed (step S120), the control unit 140 decrements the variable A. (Step S130).

After variable A is decremented (step S130), control unit 140 determines whether variable A has become 0 (step S140). When the variable A becomes 0 (step S140: YES), the control unit 140 terminates the image acquisition process of FIG.

If variable A is not 0 (step S140: NO), control unit 140 returns to step S110 and repeats the processes of steps S110 to S140 until variable A becomes 0. When the variable A becomes 0 (step S140: YES), the control unit 140 terminates the image acquisition process of FIG.

FIG. 7 is a flow showing fertilization determination processing executed by the control unit 140 . The control unit 140 executes the fertilization determination process when 7 hours have passed since egg culture started. Further, the control unit 140 repeatedly executes the fertilization determination process every hour from 7 hours to 24 hours after egg culture is started.

When the fertilization determination process is started, the control unit 140 calculates a variable A representing the number of wells designated by the user (step S200). After calculating the variable A representing the number of designated wells (step S200), the control unit 140 extracts the image of the well with the smallest number among the designated wells that have not yet been judged. (Step S210). In the present embodiment, for example, in the fertilization determination process that is executed when 7 hours have passed since egg culture started, the image of the designated well is displayed after the culture unit 110 starts egg culture. Since the images are acquired by the image acquiring unit 130 at time intervals of 15 minutes during 7 hours, there are 28 images for each designated well. That is, in the fertilization determination process that is executed when 7 hours have passed since egg culture started, in step S210, control unit 140 selects 28 wells for one of the designated wells. Extract images. In the fertilization determination process, which is performed 24 hours after egg culture is started, the control unit 140 extracts 96 images for one of the designated wells.

After extracting the image of the well with the smallest number among the designated wells for which determination has not been completed (step S210), the control unit 140 extracts the image of the pronucleus in the egg from the extracted image. Two images in which the difference in numbers is most noticeable are selected (step S220).

After selecting two extracted images from which the difference in the number of pronuclei in the egg can be most remarkably recognized (step S220), the determination unit 144 in the control unit 140 compares the two images. It is determined whether the animal is fertilized (step S230).

After comparing the two images to determine whether fertilization has occurred (step S230), the control unit 140 stores the well number for which determination has been completed (step S240). After storing the determined well (step S240), the control unit 140 decrements the variable A. (Step S250).

After variable A is decremented (step S250), control unit 140 determines whether variable A has become 0 (step S260). When the variable A becomes 0 (step S260: YES), the control unit 140 terminates the fertilization determination process of FIG.

If variable A is not 0 (step S260: NO), control unit 140 returns to step S210 and repeats the processes of steps S210 to S260 until variable A becomes 0. When the variable A becomes 0 (step S260: YES), the control unit 140 terminates the fertilization determination process of FIG.

According to the embodiment described above, the determination unit 144 can determine whether the egg is fertilized based on the analysis of the captured image of the state of the fertilized egg. Thus, a system is provided that allows confirmation of fertilization of fertilized eggs. In addition, since fertilized eggs, which are specimens, are mechanically analyzed, more specimens can be confirmed than those confirmed by humans. In addition, since the determination unit 144 determines whether or not the fertilization has occurred, it is possible to reduce the complexity of the determination compared to the human visual confirmation.

B. Second embodiment:
The configuration of the fertilization determination system in the second embodiment is the same as the configuration of the fertilization determination system 10 in the first embodiment, except that a determination unit that performs learning different from the determination unit 144 in the first embodiment is provided.

The determination unit in the second embodiment learns the features (the number of pronuclei) of the egg reflected in the image by deep learning using a deep neural network 400, which is a multi-layered neural network, and the egg is fertilized based on the learning. determine whether it is Therefore, it is possible to determine whether the egg is fertilized based on the criteria learned by the determination unit through deep learning. If the feature amount of a normally fertilized egg extracted by deep learning is a feature amount that humans cannot recognize, the fertilization confirmation of the egg can be performed with higher accuracy than that performed by humans.

FIG. 8 is an explanatory diagram illustrating image learning by deep learning. The deep neural network 400 is a network modeled on the learning mechanism in the human cranial nervous system. Deep neural network 400 comprises an input layer 410 , a plurality of hidden layers 420 and an output layer 430 . The deep neural network 400 in the second embodiment comprises four hidden layers 420 .

Input layer 410 is a layer into which information is input. The intermediate layer 420 is a layer that calculates feature amounts based on information transmitted from the input layer 410 . The output layer 430 is a layer that outputs results based on information conveyed from the intermediate layer 420 .

Image 300 in FIG. 8 is an image of a fertilized egg. Among the images 300 are an image labeled as an egg with no confirmed pronuclei, an image labeled as an egg with one confirmed pronucleus, and an image labeled as an egg with two confirmed pronuclei. This includes images labeled as present, images labeled as an egg with 3 confirmed pronuclei, and so on.

A learning method by the deep neural network 400 will be described. The input layer 410, when input with an image 300 labeled for pronucleus number, conveys that information to the intermediate layer 420. FIG. The intermediate layer 420 calculates the feature amount of the pronucleus based on the information transmitted from the input layer 410 . The output layer 430 outputs the feature amount of the pronucleus calculated based on the information transmitted from the intermediate layer. The determination unit in the second embodiment determines whether the egg is fertilized based on the output feature amount.

C. Third embodiment:
In the third embodiment, a method for selecting non-fertilized eggs that are not normally fertilized from fertilized eggs will be described. This method of selecting unfertilized eggs comprises a culturing step, an image acquisition step, and a selection step. The term "unfertilized egg" as used herein includes an unfertilized egg that has no pronucleus confirmed, and an abnormally fertilized egg that has one or three or more pronuclei confirmed.

The culturing step is a step of culturing the fertilized eggs. The image acquiring step is a step of acquiring a plurality of images by photographing the state of the eggs being cultured in the culturing step at set time intervals. The selection step is a step of selecting non-fertilized eggs by analyzing the state of the eggs shown in the image.

According to the third embodiment described above, non-fertilized eggs can be selected based on the analysis of the image of the state of the fertilized eggs. Therefore, a method is provided for selecting non-fertilized eggs that have not been fertilized normally among fertilized eggs.

D. Variant:
In the first embodiment, the fertilization determination system 10 includes the container transport section 115, but the present invention is not limited to this. For example, a fertilization determination system in another embodiment may be configured without the container transport section 115 . In this case, the image acquiring unit 130 may be provided above the culture container 200 fixed to the fixing unit in the gravitational direction, or the image acquiring unit 130 may be configured to be movable upward in the gravitational direction of the culture container 200. All you have to do is

In the first embodiment, the image acquired by the image acquisition section 130 was stored in the image storage section 142, but the present invention is not limited to this. For example, the images acquired by the image acquisition unit 130 may be stored on a so-called cloud. In this case, the determination unit 144 acquires an image from the cloud and performs determination.

In the first embodiment, the determination unit 144 determines whether the egg is fertilized by analyzing two images acquired at different times, but the present invention is not limited to this. For example, the determination unit 144 may determine whether the egg is fertilized by analyzing one image in which the largest number of pronuclei in the egg (selected by the control unit 140) can be recognized. In this case, for example, when one image selected by the control unit 140 is an image in which two pronuclei are confirmed, the determination unit 144 determines that the egg is fertilized.

The determination unit 144 may also determine whether the egg is fertilized by analyzing three images acquired at different times (selected by the control unit 140). In this case, the determining unit 144 determines, for example, one image in which the number of pronuclei in the egg was not confirmed, one image in which one pronucleus was confirmed in the egg, and an image in which two pronuclei were confirmed in the egg. Eggs are determined to be fertilized only when one and three are confirmed. The determination unit 144 may also determine whether the egg is fertilized by analyzing four or more images acquired at different times (selected by the control unit 140). In this case, for example, the determining unit 144 arranges four or more images in chronological order, and only when the number of pronuclei increases or decreases in the order of 0, 1, 2, and 0 can be confirmed that the egg is fertilized. It is determined that

In the third embodiment, the input image 300 is an image labeled with the number of pronuclei, but the present invention is not limited to this. For example, the input images may be labeled as fertilized and images labeled as not fertilized.

The present invention is not limited to the above-described embodiments, examples, and modifications, and can be implemented in various configurations without departing from the spirit of the present invention. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in each form described in the outline of the invention are used to solve some or all of the above problems, or In order to achieve some or all of the above effects, it is possible to appropriately replace or combine them. Also, if the technical features are not described as essential in this specification, they can be deleted as appropriate.

DESCRIPTION OF SYMBOLS 10... Fertilization determination system 110... Culture part 115... Container conveyance part 130... Image acquisition part 140... Control part 142... Image storage part 144... Judgment part 150... User interface 160... Notification part 200... Culture container 300... Image 400... Deep Neural network 410 Input layer 420 Intermediate layer 430 Output layer

Claims (7)

A fertilization determination system,
a culture unit for culturing fertilized eggs;
an image acquisition unit configured to acquire a plurality of images by capturing the state of the egg cultured by the culture unit at set time intervals;
a determination unit that determines whether the egg is fertilized by analyzing the state of the egg reflected in the image;
A fertilization determination system. The fertilization determination system according to claim 1,
The fertilization determination system, wherein the determination unit determines whether the egg is fertilized by analyzing temporal changes of the egg in at least two images acquired at different times. The fertilization determination system according to claim 1 or claim 2,
The fertilization determination system, wherein the determination unit learns features of the egg captured in the image by supervised learning, and determines whether the egg is fertilized based on the learning. The fertilization determination system according to claim 1,
The fertilization determination system, wherein the determination unit learns the features of the egg shown in the image by deep learning using a multilayer neural network, and determines whether the egg is fertilized based on the learning. The fertilization determination system according to any one of claims 1 to 4,
The fertilization determination system, wherein the determination unit has a notification unit that reports a determination image used for determination among the images. The fertilization determination system according to claim 5,
The fertilization determination system, wherein the notification unit notifies the determination image and information based on which the determination unit determines in the determination image. A method for selecting non-fertilized eggs that are not normally fertilized from fertilized eggs,
a culturing step of culturing the eggs;
an image acquiring step of acquiring a plurality of images by photographing the state of the egg cultured in the culturing step at set time intervals;
a selection step of selecting the unfertilized egg by analyzing the state of the egg reflected in the image;
A method for selecting unfertilized eggs, comprising:

Images