WO2020090947A1

WO2020090947A1 - Program, learning model, information processing device, information processing method, information display method, and method for producing learning model

Info

Publication number: WO2020090947A1
Application number: PCT/JP2019/042705
Authority: WO
Inventors: 山下　英俊; 佑季川▲崎▼
Original assignee: 合同会社みらか中央研究所
Priority date: 2018-10-31
Filing date: 2019-10-31
Publication date: 2020-05-07
Also published as: CN112889088A; JPWO2020090947A1

Abstract

Provided are an information processing device and the like that assist in the work of selecting sperm for which the possibility of successful micro-insemination is high. This program causes a computer to execute a process in which a photographic image, in which an image of a candidate sperm for use in micro-insemination has been captured, is obtained, the acquired photographic image is input to a learning model (53) that receives a photographic image in which an image of a sperm has been captured and outputs a prediction related to the success or failure of micro-insemination using this sperm, and the prediction output from the learning model (53) on the basis of the input photographic image is output.

Description

Program, learning model, information processing device, information processing method, information display method, and learning model manufacturing method

The present invention relates to a program, a learning model, an information processing device, an information processing method, an information display method, and a learning model manufacturing method.

Assisted reproductive technology is performed as one of infertility treatment methods. One of the clinically assisted reproductive medical treatments is microinsemination in which sperm is injected into an egg under a microscope to inseminate.

In microinsemination, an embryo cultivator assisting reproductive medicine or a clinical embryologist (hereinafter referred to as an “incubator”) injects sperm selected by visual inspection into an egg under a microscope to inseminate. At this time, ICSI (Intracytoplasmic Sperm Injection) or IMSI (Intracytoplasmic Morphologically selected Sperm Injection) may be performed.

In ICSI, the incubator observes and collects sperm using a microscope with a magnification of 200 to 400 times. In the IMSI, the incubator uses a microscope with a high magnification of 1000 times or more to perform detailed observation and collection of sperm.

In microinsemination, a fertilized egg can be obtained from one sperm and one egg. Therefore, it is an effective treatment method for male infertility such as oligospermia, azoospermia, and asthenozoospermia. Is expected. However, a method for surely obtaining a normal fertilized egg by microinsemination and a method for surely obtaining a normal embryo growth after fertilization have not been established.

Fertilized eggs can be obtained with higher efficiency than in vitro fertilization, depending on the incubator performing microinsemination, and further, compared with natural pregnancy and artificial insemination, a highly efficient implantation rate or pregnancy rate, or a low miscarriage rate. In some cases (Non-Patent Document 1). Therefore, microinsemination is used as a fertility treatment method not limited to male infertility.

Sperm with abnormal morphology or motility is not suitable for insemination due to genetic problems such as sperm maturity and DNA (Deoxyribonucleic Acid) fragmentation, and for embryo growth after fertilization. Known to have problems. A method for detecting abnormal sperm morphology has been proposed (Patent Document 1).

In addition, the laboratory manual (Non-patent document 2) published by WHO (World Health Organization) regarding sperm evaluation methods introduces an index for statistically evaluating multiple sperms contained in sperm of sperm donors. Has been done. According to this index, the success rate of assisted reproductive medicine (also referred to herein as the success probability or the predicted success probability) is reduced when the proportion of sperm having a normal morphology is less than 4%.

From such a background, as one of the simple quality evaluation methods of semen to be performed when performing infertility treatment such as microinsemination, it is possible to easily evaluate a plurality of sperms in semen at low resolution and detect abnormalities among them. There are many devices on the market that numerically evaluate the percentage of sperm that did not exist (eg, check if the percentage of normal-shaped sperm is less than 4%). In addition, a technique for objectively selecting sperm by evaluating the shape of sperm on a rule basis has been proposed (Non-Patent Document 3).

Korean Published Patent No. 2011-0049606

However, the methods disclosed in Patent Document 1 and Non-Patent Documents 1 to 3 cannot be directly applied to microinsemination in which it is necessary to evaluate each sperm separately. Even if the technology could be applied, evaluation of sperm takes a long time and labor, and invasion of sperm and eggs becomes great during that time, so that it is not practical. Furthermore, it is not possible to support the selection of spermatozoa from which sperm with no abnormal morphology or motility is likely to undergo normal embryo development after microinsemination and after fertilization.

In one aspect, an object is to provide an information processing device or the like that supports selection of sperm with a high probability of successful microinsemination, or selection of sperm with a high possibility of normal development of embryos after insemination. And

The program acquires a captured image in which a candidate sperm used for microinsemination is captured, receives a captured image in which the sperm has been captured, and outputs to a learning model that outputs a prediction regarding the success or failure of microinsemination using the sperm. A captured image is input, and the computer is caused to execute a process of outputting the prediction result output from the learning model based on the input captured image.

In one aspect, it is possible to provide an information processing device or the like that supports selection of sperm with a high probability of successful microinsemination, or selection of sperm with a high probability of normal development of embryos after insemination.

It is explanatory drawing explaining the flow of a process using an information processing system. It is explanatory drawing explaining the structure of an information processing system. It is explanatory drawing explaining the progress to delivery. It is explanatory drawing explaining the frequency distribution of a success probability. It is explanatory drawing explaining the preparation method of the image encoder which extracts an image feature-value from the still image which image | photographed sperm. It is explanatory drawing explaining the extraction of the image feature-value from a moving image file. It is explanatory drawing explaining a progress learning model. It is explanatory drawing explaining the record layout of teacher data DB. It is explanatory drawing explaining the record layout of preliminary photography DB. It is explanatory drawing which shows the screen which an information processing system displays. It is explanatory drawing which shows the screen which an information processing system displays. It is explanatory drawing which shows the screen which an information processing system displays. It is a flow chart which shows a flow of processing of a program used at the 1st preparatory stage. It is a flow chart which shows a flow of processing of a program used at the 2nd preparatory stage and a sperm selection stage. 9 is a flowchart showing the flow of processing of a sample distribution calculation subroutine. FIG. 7 is an explanatory diagram illustrating a record layout of a preliminary shooting DB according to the second embodiment. 9 is a flowchart showing a flow of processing of a subroutine of sample distribution calculation according to the second embodiment. FIG. 16 is an explanatory diagram showing a screen displayed by the information processing device according to the third embodiment. It is explanatory drawing explaining the structure of the information processing system of Embodiment 4. It is explanatory drawing explaining the record layout of a normal sperm determination DB. It is explanatory drawing which shows the structure of a normal sperm determination model. FIG. 16 is an explanatory diagram showing a screen displayed by the information processing device according to the fourth embodiment. FIG. 16 is an explanatory diagram showing a screen displayed by the information processing device according to the fourth embodiment. FIG. 16 is an explanatory diagram showing a screen displayed by the information processing device according to the fourth embodiment. 16 is a flowchart showing the flow of processing of a program according to the fourth embodiment. It is a flow chart which shows a flow of processing of a subroutine of sperm image acquisition. FIG. 16 is an explanatory diagram showing a screen displayed by the information processing device according to the fifth embodiment. It is a functional block diagram of the information processing apparatus of Embodiment 6. It is explanatory drawing which shows the structure of the information processing system of Embodiment 7. It is a table which shows the incidence rate and carrier rate of a sex-linked recessive (latent) genetic disease. It is a table which shows the incidence rate and carrier rate of a sex-linked dominant (overt) genetic disease. It is a table which shows the incidence rate and carrier rate of the sex-linked genetic disease inherited via the Y chromosome. It is explanatory drawing explaining the record layout of sex teacher data DB. It is explanatory drawing explaining a sex determination learning model. FIG. 28 is an explanatory diagram showing a screen displayed by the information processing device according to the eighth embodiment. 28 is a flowchart showing the flow of processing of a program according to the eighth embodiment. FIG. 28 is an explanatory diagram showing a screen displayed by the information processing device according to the ninth embodiment. FIG. 28 is an explanatory diagram showing a screen displayed by the information processing device according to the ninth embodiment. 28 is a flowchart showing the flow of processing of a program according to the ninth embodiment. It is a flow chart which shows a flow of processing of a subroutine of an onset rate and a carrier rate calculation.

[Embodiment 1]
FIG. 1 is an explanatory diagram illustrating a flow of processing using the information processing system 10. FIG. 1A shows a first preparatory stage for processing information relating to past microinsemination.

Explain the outline of microinsemination. As described above, in microinsemination, an incubator observes sperm in semen using a microscope 41 (see FIG. 2) having a high optical magnification of 200 to 400 times or 1000 times or more. The incubator selects one sperm that has a normal shape and performs a normal movement per egg. In addition, in the camera 48, the photographing I / F 28, or the display I / F 25, the incubator may observe the image displayed on the display device 15 by further enlarging the microscope image several times using a digital zoom.

An incubator takes a microscopic image containing sperm selected from moving images or still images. In the microscopic image, the part in which the sperm is reflected may be cut out by image processing or the like. In the following description, a photographed microscope image may be referred to as a photographed image. An incubator collects selected sperm using a micropipette.

The incubator inserts the selected sperm by inserting the tip of a micropipette into the egg. Microinsemination is performed as described above. After that, the incubator cultivates the egg under predetermined conditions. An egg that has started fertilization and has started to grow is called an embryo. An obstetrician transplants an embryo that has grown to a predetermined stage, such as a quadrant or a blastocyst, to a mother. Pregnancy is established when the embryo is implanted in the uterus. After that, the fetus grows in the womb as in the case of normal natural pregnancy, and if normal, gives birth.

Whether or not fertilization is established, success or failure of normal growth up to each stage of development such as 4-splitting embryo, morula, blastocyst, presence or absence of pregnancy, presence or absence of abnormalities in pregnant woman screening, success or failure of childbirth, etc. The time course of embryo growth after insemination is recorded in association with the captured image.

Image feature amount and motion feature amount (hereinafter, referred to as feature amount) extracted from the image taken when sperm is selected, clinical profile of sperm donor (age, medical history, current medical history, treatment history, and , Genomic information such as gene mutations) and the growth process of a fertilized egg or embryo after microinsemination are recorded in the teacher data DB 51 in association with each other. Details of the teacher data DB 51 will be described later.

Machine learning with a teacher is performed based on the teacher data DB 51, and a machine learning model (hereinafter, referred to as “progress learning model”) 53 that predicts a growth process of a fertilized egg or an embryo is created. The progress learning model 53 is a learned model that outputs a success probability of normally growing to each stage in the process from fertilization to childbirth when a feature amount of a captured image of sperm is input. Details of the feature amount and the progress learning model 53 will be described later.

Note that the success probability can be easily calculated by subtracting the success probability from 1. Therefore, the above-mentioned success probability can be restated as a failure probability. This is true for all success probabilities below.

Based on the frequency distribution of the data relating to the success or failure of each stage from fertilization to childbirth and the statistical processing such as normalization of the frequency distribution, the first distribution f (shown by the solid line in the lower right of FIG. 1 and the graph of FIG. X) is created. It should be noted that the graphs shown in the lower right part of FIG. 1 and FIG. 4 are made one at each stage. The first distribution f (X) represents a probability density distribution of success probability of normally growing up to each stage, and corresponds to a so-called prior distribution in Bayesian estimation. Details of the first distribution f (X) will be described later.

In the first preparatory stage shown in FIG. 1A, images of sperm taken in a plurality of cases and data on the growth process of fertilized eggs or embryos are used. The progress learning model 53 and the first distribution f (X) created in the first preparatory stage are used in cases to be performed thereafter.

1B and 1C show the second preparatory stage and sperm selection stage, which are performed for each case of newly performing microinsemination. After receiving the semen collected from the sperm donor, the incubator preferably performs the second preparation step and the sperm selection step in succession.

In the second preparatory stage shown in FIG. 1B, a sample distribution g (X) based on the characteristics of sperm contained in semen collected from a sperm donor is created. The incubator performs pretreatment such as washing and dilution of semen, and then takes preliminary images of sperm. The feature amount extracted from the photographed image is input to the progress learning model 53 that predicts the growth process of the fertilized egg or embryo, so that the probability of normal growth at each stage from fertilization to delivery of the sperm (hereinafter, The “prediction success probability”) is obtained.

A sample distribution g (X) indicated by broken lines in the lower right part of FIG. 1 and the graph of FIG. 4 is created based on the predicted success probabilities acquired for a plurality of spermatozoa. The sample distribution g (X) represents the distribution of the prediction success probability obtained by sampling and evaluating the sperm contained in the semen of the individual sperm donor, and corresponds to the so-called likelihood distribution in Bayesian estimation. Details of the sample distribution g (X) will be described later.

A second distribution h (X) is generated based on the first distribution f (X) and the sample distribution g (X). The second distribution h (X) corresponds to a so-called posterior distribution in Bayesian estimation. Details of the second distribution h (X) will be described later.

At the sperm selection step shown in FIG. 1C, the sperm to be used for microinsemination is selected from the sperm donor's semen. The incubator takes an image of the candidate sperm. By inputting the feature amount extracted from the captured image into the progress learning model 53, the prediction success probability that the candidate sperm will normally grow to each stage from fertilization to birth is acquired.

The positioning of the prediction success probability within the second distribution h (X) is displayed by an evaluation index such as a deviation value. The incubator can use the displayed evaluation index to judge whether to use the candidate sperm for microinsemination. When it is determined that the sperm will be used for microinsemination, the incubator collects candidate sperm under a microscope and injects them into the egg. If it is determined that it will not be used for microinsemination, the incubator will evaluate other candidate sperm.

From the above, the incubator can select sperm from the collected semen that has a high probability of successful microinsemination. In the following description, sperm in which the evaluation index at each stage satisfies a predetermined criterion, that is, a sperm expected to have a sufficiently high possibility of successful microinsemination to reach normal birth is referred to as “high-quality sperm”. ".

Normal sperm move in solution at high speed in a short time due to flagella movement. When performing microinsemination, devise the observation conditions such as the viscosity of the solution and the sperm concentration in the solution to slow the sperm movement. This allows the incubator to fully observe sperm. However, when the sperm move out of the visual field of the microscope 41, it is difficult to track or find the same sperm.

Therefore, it is difficult to secure the same sperm as the spermatozoa to be used for microinsemination by the conventional method in which the incubator judges the quality of sperm by microscopic observation, even when the incubator comparatively examines multiple sperms. Is. In the present embodiment, since the evaluation of each candidate sperm can be displayed in real time, the incubator can easily determine whether or not to use the candidate sperm, and at the same time, collect the target candidate sperm.

When a large number of sperm of high quality are contained in semen in the selection of sperm, the time point when such sperm is obtained or the time point when a better index is obtained in the second distribution h (X) Sperm is collected in and transferred to microinsemination. This improves the work efficiency of the incubator while minimizing damage to sperm and eggs.

Even if there are few sperm suitable for microinsemination contained in semen, it is desirable to select the sperm of the highest quality within the range possible within a limited time. For example, a method may be considered in which a sperm with a relatively excellent evaluation result is captured, set aside separately, and the sperm of the highest quality among the sperm set aside at the final stage is used. As a method of storing, there are a method of sucking a specific sperm with a capillary and transferring to a microwell, and a method of independently retaining a droplet containing a specific sperm in a solution-free region on a slide glass.

On the other hand, due to time and labor constraints, abandoning the selection of the sperm with the highest quality, and then adopting the method with the highest probability of selecting the sperm with the highest quality. Is also desirable.

In other fields, a mathematical optimal solution is known for evaluating candidates one by one and ending the evaluation when a good candidate is found. This is called a secretary problem, which is one of the optimal stopping problems. The process of calculating the optimal solution for the secretary problem will be described. When the total number of candidates (the total number of sperms in semen) n is a sufficiently large value, the first n / e candidates (e is the number of Napiers) are evaluated but not selected. When it is determined that the candidate evaluated thereafter (candidate sperm) is better than any of the candidates evaluated so far, the candidate is determined to be the best candidate.

More specifically, since 1 / e corresponds to about 37%, it will be rejected after confirming the evaluation value of about 37% of the total sperm contained in semen. Of the remaining sperm, the sperm is adopted as soon as a sperm having a better evaluation value than any evaluated sperm is obtained.

When using the optimal solution for this secretary problem, it is not necessary to reserve sperm and statistically the highest quality work efficiency can be expected. On the other hand, even with the optimal solution of the secretary problem, it is not always possible to select the sperm with the highest evaluation value contained in semen. For example, the following two cases can be considered. One is when the best candidate was included in the first about 37% of candidates. The other is when the quality of the first about 37% of the candidates is very low and a candidate with a high quality is found in the subsequent candidates, but a better quality candidate is included in the unevaluated candidates. ..

It is considered that the statistical occurrence frequency is low in all cases, but it is desirable to avoid these events. Therefore, it is desirable to select sperm with reference to the index such as the deviation value and the reliability obtained based on the second distribution h (X). In this case, the work efficiency is statistically lower than that of the optimum solution of the secretary problem, but it is possible to select a candidate of excellent quality.

Also, when the number of sperms in semen is very large, it is not realistic to observe about 37% of sperms. Therefore, it is a realistic method to select a candidate based on the second distribution h (X). If work efficiency is completely ignored, as soon as a sperm with better quality is obtained, it may be set aside, the whole sperm may be evaluated, and then the sperm with the best quality may be used. it can.

It is known that even the same sperm donor has different characteristics such as the number of sperm and the amount of sperm contained depending on the time when semen was collected. Therefore, it is desirable that the second preparatory step described with reference to FIG. 1B and the sperm selection step described with reference to FIG. 1C be performed successively using the same semen. However, if there is an extremely small number of spermatozoa due to oligospermia, etc., use the semen collected by the same sperm donor at different times in the second preparation stage and the sperm selection stage. May be.

If you are a sperm donor who has symptoms such as azoospermia where semen does not contain sperm, instead of collecting semen, sperm should be collected by surgery such as epididymal sperm collection method or intratesticular sperm collection method. To collect. When the collected sperm are few, it may be difficult to perform both the second preparation stage and the sperm selection stage. In such a case, the second preparatory step may be omitted, and the sperm selection step may be performed by diverting the sample distribution g (X) of sperm donors having similar symptoms.

FIG. 2 is an explanatory diagram illustrating the configuration of the information processing system 10. The information processing system 10 includes an information processing device 20, a display device 15, and a microscope 41.

The information processing device 20 includes a control unit 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display I / F (Interface) 25, a photographing I / F 28, and a bus. The control unit 21 is an arithmetic and control unit that executes the program of this embodiment. The control unit 21 includes one or more CPUs (Central Processing Units), multi-core CPUs, GPUs (Graphics Processing Units), and the like. The control unit 21 is connected to each of the hardware units configuring the information processing device 20 via a bus.

The main storage device 22 is a storage device such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), and flash memory. The main storage device 22 temporarily stores information required during the process performed by the control unit 21 and a program being executed by the control unit 21.

The auxiliary storage device 23 is a storage device such as SRAM, flash memory, or hard disk. The auxiliary storage device 23 stores a teacher data DB 51, a preliminary shooting DB 52, a progress learning model 53, an image encoder 546, a program to be executed by the control unit 21, and various data necessary for executing the program.

Note that the teacher data DB 51, the preliminary shooting DB 52, the progress learning model 53, and the image encoder 546 may be stored in an external mass storage device or the like connected to the information processing device 20.

The communication unit 24 is an interface that communicates between the information processing device 20 and the network. The display I / F 25 is an interface that connects the display device 15 such as a liquid crystal display device or an organic EL (Electro Luminescence) display device to the information processing device 20. The photographing I / F 28 is an interface that connects a camera 48 described later and the information processing device 20.

The display I / F 25 is, for example, a VGA terminal, a DVI (Digital Visual Interface) terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, a USB (Universal Serial Bus) terminal, or the like. The photographing I / F 28 is, for example, a USB terminal. The display I / F 25 and the display device 15 may be wirelessly connected to each other, and the photographing I / F 28 and the camera 48 may be wirelessly connected to each other.

The microscope 41 is, for example, a differential interference microscope, a bright field microscope, a polarization microscope, a phase contrast microscope or an inverted microscope. The microscope 41 includes a stage 42, an eyepiece lens 43, an objective lens 47, and an illumination unit 44. On the stage 42, an observation container 421 containing semen that has undergone pretreatment such as washing and dilution is placed. The observation container 421 is, for example, a petri dish, a well, a slide glass, or the like.

The observation container 421 is illuminated by the illumination light emitted from the illumination unit 44. An incubator who is a user observes the sperm in the observation container 421 through the objective lens 47 and the eyepiece lens 43.

An optical path splitting unit 45 is arranged between the objective lens 47 and the eyepiece lens 43. With the camera 48 connected to the optical path splitting unit 45, the sperm under observation by the incubator can be photographed as a moving image or a still image. The photographed image is recorded in the auxiliary storage device 23 via the photographing I / F 28 and is displayed on the display device 15 in real time via the display I / F 25.

For example, when searching for candidate sperm and confirming the evaluation, the incubator looks at the sperm displayed on the display device 15. When collecting the candidate sperm into the micropipette, the incubator visually confirms the candidate sperm and the tip of the micropipette using the eyepiece lens 43. In this way, the incubator can appropriately use the display device 15 and the eyepiece lens 43, so that it is possible to provide the information processing system 10 in which fatigue of the incubator is small and sperm can be accurately collected.

The information processing device 20 of the present embodiment is a general-purpose personal computer, a tablet, a large computer, or a virtual machine that operates on a large computer. The information processing device 20 may be configured by hardware such as a plurality of personal computers, tablets or large-scale computers. The information processing device 20 may be composed of a quantum computer. The information processing device 20 may be built in the microscope 41. The information processing device 20 may be connected to the network via a hospital network system (not shown).

FIG. 3 is an explanatory diagram for explaining the process leading to childbirth. A few to a dozen or so eggs are collected from an egg donor. Of the collected eggs, those with abnormalities such as deformation and eggs that have not matured to the fertilizable stage are discarded. Tens of thousands to tens of millions of sperm are collected from sperm donors. Microinsemination in which sperm are selected and injected into an egg is performed. The egg is cultured under predetermined conditions.

After fertilization is established, the embryo starts cell division and develops into morula, blastocyst, through 2 divided embryos, 4 divided embryos, etc. The development process of the embryo is appropriately observed, and whether or not the embryo has normally grown to each development stage is recorded. Embryos found to be abnormal by observation are discarded.

 Embryos that have grown to the prescribed stage are transferred to the mother, and the remaining embryos are cryopreserved. In some cases, frozen embryo transfer is performed, in which all embryos are once frozen and stored, and embryo transfer is performed when the maternal condition is ready.

After the embryo is implanted in the uterus and the pregnancy is established, the fetus grows like natural pregnancy. Pregnancy checkups are performed and the development of the fetus is recorded. If the course is normal, the child is born and a newborn baby is born.

If the embryo does not implant in the uterus, has a miscarriage, or wishes to give birth to a second or later child, the embryo that has been cryopreserved will be re-transferred. When the fertility treatment is completed, the cryopreserved embryo is discarded.

FIG. 4 is an explanatory diagram illustrating the distribution of success probabilities. The horizontal axis X represents the probability of success as a random variable, and the vertical axis Y represents the probability corresponding to the random variable X. A success probability of 0 indicates that no case succeeds, and a success probability of 1 indicates that all cases succeed.

The graph shown in the lower right of FIG. 1 also displays the same distribution as in FIG. 4, but the vertical axis X shows the success probability as a random variable, and the horizontal axis Y shows the probability corresponding to the random variable X.

-Success probability is calculated for each stage of development such as fertilization, blastocyst formation, implantation, and childbirth. For example, the success probability of fertilization and blastocyst formation is calculated by the number of eggs that have normally grown to fertilization and blastocyst formation with respect to the number of eggs that have undergone microinsemination. The success rate of implantation and delivery is calculated based on the number of embryos successfully implanted and delivered with respect to the number of embryos transplanted into the mother.

-The probability of success for various developmental stages such as 2-split embryo, 4-split embryo, morula, early pregnancy, midgestation, etc. may be calculated. You may calculate the success probability that the baby after birth will grow normally up to various stages of growth.

-Each stage is evaluated by two choices, whether it has grown normally or not. The first distribution f (X) indicated by the solid line is a probability density distribution in which the success / failure of all recorded cases is represented by a beta distribution. The first distribution f (X) is expressed by equation (1).

p is the number of cases that grew normally up to each growth stage.
q is the number of cases that did not grow normally up to each growth stage.
C ₁ is a normalization constant.

The sample distribution g (X) indicated by a broken line indicates the distribution of success probabilities predicted from the preliminarily photographed images in the second preparatory stage described using FIG. 1B. For example, when predicting whether or not normal growth can be achieved up to each growth stage based on a pre-captured image, the sample distribution g (X) is expressed by equation (2) based on the binomial distribution. ..

a is the number of preliminary captured images predicted to grow normally up to each growth stage.
b is the number of preliminary captured images predicted not to grow normally up to each growth stage.
C ₂ is a normalization constant.

The second distribution h (X) indicated by the alternate long and short dash line is the posterior distribution obtained by Bayesian estimation using the first distribution f (X) as the prior distribution and the sample distribution g (X) as the likelihood distribution, and sperm It means the result of predicting the probability density distribution of sperm success probability provided by the provider. The second distribution h (X) is expressed by equation (3).

C ₃ is a normalization constant.

The closer the success probability calculated for each candidate sperm using the progress learning model 53 to the right end of the second distribution h (X), the higher the success probability of the provided sperms. For example, in FIG. 4, sperm with a success probability of about 0.55 are spermatozoa with a relatively low success probability among all the data shown by the first distribution f (X), but the second distribution h (X). The spermatozoa that have a relatively high probability of success among the spermatozoa provided as shown in. Based on this information, the incubator decides whether to use the sperm under observation for microinsemination or to look for sperm with a higher probability of success.

FIG. 5 is an explanatory diagram illustrating a method of creating the image encoder 546 that extracts the image feature amount from the still image obtained by photographing the sperm.

A method for extracting image feature amounts using an auto encoder will be described with reference to FIG. The image feature amount model 54 is a CNN (Convolution Neural Network) including an input layer 541, an intermediate layer 542, and an output layer 543.

CNN is a neural network in which the convolutional layer and the pooling layer are iterated using the ReLU (Rectified Linear Unit) function or the softmax function as the activation function, and then the fully connected layer is repeated multiple times. Illustration of the convolutional layer and the pooling layer is omitted.

The input layer 541 and the output layer 543 have the same number of neurons as the number of pixels of the captured image which is the teacher data. The number of neurons in the middle layer 542 is the smallest in the middle layer 545. The still image of the captured image is input to the input layer 541. Specifically, the pixel value of each pixel of the captured image is input to each neuron of the input layer 541.

The control unit 21 performs supervised machine learning that calculates the parameters of the intermediate layer 542 by using the error backpropagation method or the like so that the same captured image as the input layer 541 is output from the output layer 543. Each neuron in the central layer 545 after the supervised machine learning is finished shows the image feature amount of the captured image.

The control unit 21 divides a captured image, which is teacher data, into training data and verification data before performing supervised machine learning. The control unit 21 verifies the accuracy of the image feature amount model 54 subjected to the machine learning with the teacher using the training data, using the verification data. From the above, it is confirmed that the image feature amount model 54 does not have a problem such as over-learning. Since the learning and verification process is a process generally performed in supervised machine learning, description thereof will be omitted in supervised machine learning described below.

The part of the learned image feature amount model 54 from the input layer 541 to the central layer 545 is used for the image encoder 546 that extracts the image feature amount from the captured image. The control unit 21 cuts out the image encoder 546 from the image feature amount model 54 and records it in the auxiliary storage device 23. As described above, the image encoder 546 that outputs the image feature amount is completed when the captured image of the sperm is input.

Note that it is desirable to use an image of a sperm taken by a specialist such as a skilled incubator as the teacher data when creating the image encoder 546. By doing so, it is possible to realize the image encoder 546 that extracts the feature amount indicating the difference between normal spermatozoa.

It is desirable that the teacher data when creating the image encoder 546 include only one sperm. If an image containing a plurality of spermatozoa or foreign substances is used, the image is subjected to clipping processing or the like, and an image processed so as not to include a portion other than the target sperm is used as the teacher data.

The image encoder 546 may be created using any computer. The created image encoder 546 is transmitted to the information processing device 20 used by the microinsemination performing institution via a network or the like and recorded in the auxiliary storage device 23. In this case, it is desirable that the information processing system 10 that creates the image encoder 546 and the information processing system 10 that uses the image encoder 546 use the microscopes 41 having the same or similar specifications.

FIG. 6 is an explanatory diagram for explaining the extraction of the image feature amount from the moving image file. Extraction of the image feature amount when the captured image at the time of microinsemination is recorded as a moving image will be described with reference to FIG.

An example will be explained in which the moving image from the time t1 to the time t5 is recorded in the moving image file. Multiple still images are created by dividing the moving image file into frames. Each still image is input to a CNN for determination prepared in advance, and it is determined whether or not the whole sperm is photographed. When it is determined by the CNN for determination that the whole sperm is imaged in a plurality of frames, one frame is selected by an arbitrary method.

In the example shown in FIG. 6, it is determined that the entire sperm has been imaged in the frame at time t3. The still image determined to have the entire sperm photographed is input to the image encoder 546 described using FIG. 5, and the feature amount of the still image is extracted.

From video files, video features such as sperm movement speed, linear movement distance, and tail flagella movement cycle are extracted. The moving image feature amount is calculated, for example, based on the amount of movement of a plurality of feature points defined on the captured image between frames. The moving image feature amount may be extracted using an RNN (Recurrent Neural Network). In addition, the moving image feature amount may be extracted using an existing arbitrary moving image analysis method. The feature amount of the still image and the feature amount of the moving image are recorded in the teacher data DB 51 in association with the progress after microinsemination.

When the captured image at the time of microinsemination is recorded as both a moving image and a still image, the control unit 21 uses the still image instead of cutting out the frame from the moving image file to determine the image feature amount. Extract.

FIG. 7 is an explanatory diagram illustrating the progress learning model 53. The progress learning model 53 is a neural network including an input layer 531, an intermediate layer 532, and an output layer 533. In FIG. 7, the case where the progress learning model 53 is CNN is illustrated. Illustration of the convolutional layer and the pooling layer is omitted.

The progress learning model 53 is created at various stages of development such as fertilization, blastocyst formation, implantation, and childbirth. The input of the progress learning model 53 is the image feature amount of the captured image, that is, the image feature amount of the still image described using FIG. 5, and the image feature amount of the moving image described using FIG. The output of the progress learning model 53 is the probability of successful growth up to each stage (success probability) and the probability of failure to grow normally (unsuccessful probability).

The progress learning model 53 outputs the success and failure probabilities to the output layer 533 when the feature amounts of the still image and the moving image are input to the input layer 531. In the learning stage, the control unit 21 uses the teacher data DB 51 in which the feature amounts of the still image and the moving image are recorded in association with the progress after microinsemination, and uses the error backpropagation method or the like to set the parameters of the intermediate layer 532. The supervised machine learning is performed by calculating.

Supervised machine learning can be performed by any method such as logistic regression, SVM (Support Vector Machine), random forest, CNN, RNN, or XGBoost (eXtreme Gradient Boosting).

Only one of a still image feature amount and a moving image feature amount may be input to the input layer 531. The input layer 531 may additionally include items other than the image feature amount, such as age, medical history, current medical history including health status, family history, past infertility treatment history of the sperm donor and the egg donor. .. The input layer 531 includes sperm concentration, total sperm count, sperm motility, motile sperm concentration, linear velocity, curve velocity, average velocity, linearity, and straightness obtained by low-power microscope observation or a commercially available sperm evaluation device or the like. Items related to sperm quality such as sex, head amplitude, and head frequency may be input.

For example, after the learning of the progress learning model 53 for determining the success or failure of fertilization is completed, the control unit 21 determines the success or failure of a later stage such as a blastocyst stage by the transfer learning using the completed progress learning model 53. The progress learning model 53 may be created. From fertilization to childbirth, the number of teacher data decreases in later stages. However, by using transfer learning, it is possible to create an appropriate progress learning model 53 even in the later stage.

The progress learning model 53 may be created using any computer. The created progress learning model 53 is transmitted to the information processing device 20 used by the microinsemination performing institution via a network or the like and recorded in the auxiliary storage device 23. In this case, the information processing system 10 that creates the image encoder 546 and the progress learning model 53 and the information processing system 10 that uses the image encoder 546 and the progress learning model 53 have the same specifications or similar specifications. It is desirable to use 41.

FIG. 8 is an explanatory diagram illustrating a record layout of the teacher data DB 51. The teacher data DB 51 is a DB that records the photographed image of the sperm used for microinsemination and the process after microinsemination in association with each other. The teacher data DB 51 has a sperm ID field, an image data field, a sperm image feature amount field, and a progress information field.

The sperm image feature amount field has a still image field and a moving image field. The still image field and the moving image field each have a first feature amount field, a second feature amount field, and the like. The progress information field includes a fertilization field, a blastocyst formation field, an implantation field, a birth field, and a health status field. The teacher data DB 51 has one record for each sperm used for microinsemination.

In the sperm ID field, the sperm ID uniquely assigned to the sperm is recorded. In the image data field, image data of a sperm photographed is recorded. FIG. 8 shows an example in which a moving image file with the extension “mpg” is recorded, but the image data format is arbitrary. A plurality of files such as a moving image file and a still image file may be recorded in the image data field.

In each subfield of the still image field, the feature amount of the still image obtained by inputting the image data to the image encoder 546 is recorded. In each subfield of the moving image field, the characteristic amount of the moving image obtained by analyzing the image data is recorded.

The fertilization field records the success or failure of fertilization, the blastocyst formation field records the success or failure of blastocyst formation, the implantation field records the success or failure of implantation, and the birth field records the success or failure of childbirth. “OK” indicates that each stage has been normally reached, and “NG” indicates that each stage has not been normally reached. "-" Indicates that the success or failure of each stage cannot be determined.

The health status field indicates the health status of the newborn. "OK" indicates that the newborn is healthy. “NG” indicates, for example, a very premature baby, which is a newborn baby having a health problem.

The progress information field may have a field that records an arbitrary developmental stage of the embryo, such as a 2-split embryo field, a 4-split embryo field, or a morula embryo field. The progress information field may have a field for recording the result of the pregnant woman examination, such as the 20th week of pregnancy field. The progress information field may have a field for recording the result of prenatal diagnosis such as amniotic fluid test or NIPT (Non-Invasive Prenatal Genetic Testing).

The information recorded in each subfield of the progress information field may be acquired from an electronic medical record or the like based on the sperm ID recorded in the sperm ID field.

FIG. 9 is an explanatory diagram illustrating a record layout of the preliminary shooting DB 52. The preliminary shooting DB 52 is a DB that records a prediction result using the progress learning model 53 in the second preparatory stage described using FIG. 1B. The preliminary imaging DB 52 has a patient ID field, an imaging date field, and a prediction result field. The prediction result field has a fertilization field, a blastocyst formation field, an implantation field, a birth field, and a health status field. The preliminary photographing DB 52 has one field for one preliminary photographing.

The patient ID uniquely assigned to the sperm provider is recorded in the patient ID field. In the shooting date field, the shooting date when the shot image was shot is recorded. In each subfield of the prediction result field, the prediction result of whether or not each stage is normally reached is recorded. “OK” indicates that each stage was predicted to reach normal, and “NG” indicates that each stage did not reach normal.

The control unit 21 extracts the sperm image feature amount from the captured image captured by the camera 48 and inputs it to the progress learning model 53 created for each stage to acquire the success probability. The control unit 21 records “OK” when the success probability is equal to or higher than the predetermined threshold value, and records “NG” when the success probability is lower than the predetermined threshold value.

10 to 12 are explanatory diagrams showing screens displayed by the information processing system 10. FIG. 10 shows the screen displayed on the display device 15 by the control unit 21 when the preliminary imaging of the sperm contained in the semen provided by the sperm donor is performed in the second preparatory step described using FIG. 1B. Here is an example:

An image column 61, a target number column 62, a photographed number column 63, a photographing button 66, and an end button 67 are displayed on the screen. In the image column 61, a microscope image taken by the camera 48 is displayed in real time. In the target number column 62, the target number of preliminary shooting is displayed. According to the optimal solution to the secretary problem described above, the target number for preliminary imaging is about 37% of the number of sperm collected. However, since this number is usually an unrealistically large value, it is set to a workable value. The number of pre-photographed spermatozoa is displayed in the photographed number column 63.

When the user selects the shooting button 66, the control unit 21 extracts the still image feature amount and the moving image feature amount of the image displayed in the image column 61. The control unit 21 inputs the extracted feature amount into the progress learning model 53 created for each stage, and acquires the success probability. The control unit 21 determines a prediction result of whether or not normal growth is achieved up to each stage based on a predetermined threshold value. The control unit 21 creates a new record in the preliminary shooting DB 52 and records the prediction result.

The user determines whether to move to the sperm selection stage described using FIG. 1C based on the target number column 62, the photographed number column 63, and the state of sperm contained in the semen under observation. When the user selects the end button 67, the control unit 21 ends the preliminary imaging and shifts to the sperm selection stage. The control unit 21 may automatically shift to the sperm selection stage when the number of pre-captured sperm reaches the target number.

FIG. 11 shows an example of a screen displayed by the control unit 21 on the display device 15 at the sperm selection stage described using FIG. 1C. An image field 61, a determination button 68, and an end button 67 are displayed on the screen. In the image column 61, a microscope image taken by the camera 48 is displayed in real time.

When the user selects the determination button 68, the control unit 21 extracts the still image feature amount and the moving image feature amount of the image displayed in the image column 61. The control unit 21 inputs the feature amount into the progress learning model 53, and acquires the prediction success probability of each stage from fertilization to delivery. When the user selects the end button 67, the control unit 21 ends the process.

FIG. 12 shows an example of a screen displayed by the control unit 21 on the display device 15 when the selection of the determination button 68 is accepted. An image field 61, an evaluation field 65 and a next button 69 are displayed on the screen. The evaluation section 65 includes a first evaluation section 651, a second evaluation section 652, a third evaluation section 653, a fourth evaluation section 654, and a comprehensive evaluation section 659.

In the image column 61, the microscope image displayed in the image column 61 when the determination button 68 is accepted on the screen described with reference to FIG. 11 is displayed in a still state. The user can confirm the candidate sperm under determination from the image column 61.

The control unit 21 inputs the image feature amount of the candidate sperm under determination into the progress learning model 53 regarding success or failure of fertilization, and displays the evaluation of the success probability acquired in the first evaluation column 651. The evaluation is represented by the deviation value in the second distribution h (X) described using FIG.

Similarly, the control unit 21 displays the evaluation of the success probability obtained by inputting the image feature amount of the candidate sperm into the progress learning model 53 regarding the success or failure of blastocyst formation in the second evaluation column 652. The control unit 21 displays the evaluation of the success probability obtained by inputting the image feature amount of the candidate sperm into the progress learning model 53 regarding the success or failure of implantation in the third evaluation column 653. The control unit 21 displays the evaluation of the success probability obtained by inputting the image feature amount of the candidate sperm into the progress learning model 53 regarding the success or failure of childbirth in the fourth evaluation column 654.

The control unit 21 displays, in the comprehensive evaluation column 659, the comprehensive evaluation of candidate sperm that is a combination of the evaluations displayed in the first evaluation field 651 to the fourth evaluation field 654. In the comprehensive evaluation, for example, the candidate spermatozoa were classified into “good”, “normal”, “poor”, etc. based on the average value or the minimum value of the evaluations displayed in the first evaluation field 651 to the fourth evaluation field 654. The result.

The control unit 21 may call the user's attention by an arbitrary method such as sounding when the comprehensive evaluation is equal to or higher than a predetermined threshold.

The user determines whether to use the candidate sperm for microinsemination. When it is determined that the sperm is to be used, the user collects it in the micropipette while checking the candidate sperm with the eyepiece lens 43. When it is determined not to be used or when the collection is completed, the user selects the next button 69.

When the user selects the next button 69, the control unit 21 returns the screen displayed on the display device 15 to the screen described using FIG. 11. The user can select a new candidate sperm.

The control unit 21 may display a real-time microscope image in the image column 61. The user can determine whether or not to use the candidate sperm by viewing the motion state of the candidate sperm and the evaluation column 65 at the same time. When the candidate sperm is likely to move out of the microscope field of view, the user appropriately operates the stage 42 to keep the candidate sperm within the microscope field of view.

Both a still image of a candidate sperm and a real-time microscope image may be displayed in the image column 61 by using screen division or child screen display.

FIG. 13 is a flowchart showing the flow of processing of the program used in the first preparation stage. The flow of processing performed in the first preparatory stage described with reference to FIG. 1A will be described with reference to FIG. In the teacher data DB 51, in the initial state, the photographed image and the progress information regarding the past microinsemination are recorded in the sperm ID field, the image data field, and the progress information field, and the sperm image feature amount field is blank. ..

The control unit 21 extracts one processing target record from the teacher data DB 51. The control unit 21 acquires an image from the image data field (step S501). The control unit 21 extracts the image feature amount (step S502). Specifically, the control unit 21 cuts out a still image from the image and extracts the moving image feature amount, as described with reference to FIG. 6, for example. The control unit 21 inputs the cut out still image to the image encoder 546 to acquire the still image feature amount.

The control unit 21 records the image feature amount in each subfield of the sperm image feature amount field of the processing target record (step S503). The control unit 21 determines whether or not the processing of the captured image recorded in the teacher data DB 51 is completed (step S504). When it is determined that the processing is not completed (NO in step S504), the control unit 21 returns to step S501.

When it is determined that the processing has been completed (YES in step S504), the control unit 21 uses the data recorded in one subfield of the sperm image feature amount field and the progress information field as teacher data to perform supervised machine learning. Then, the progress learning model 53 for one stage in the process from fertilization to delivery is created (step S511).

Continued explanation using FIG. 7. The control unit 21 uses an error backpropagation method or the like so that a predetermined value is output to the output layer 533 when input data such as a sperm image feature amount is input to the input layer 531. Is calculated. When “OK” is recorded in the progress subfield, the predetermined value is “1” in the “success” neuron, “0” in the “unsuccessful” neuron, and “NG” in the progress subfield. Is recorded, it is “0” for the “successful” neuron and “1” for the “unsuccessful” neuron.

The control unit 21 stores the created progress learning model 53 in the auxiliary storage device 23 (step S512). The control unit 21 respectively acquires the number of “OK” and the number of “NG” for the progress information subfield being processed (step S513).

By substituting the number of “OK” for p and the number of “NG” for q in the equation (1), the first distribution f (X) described using FIG. 4 is obtained. The control unit 21 records the number of “OK” and the number of “NG” in the auxiliary storage device 23 in association with the progress information subfield being processed (step S514).

The control unit 21 determines whether or not processing of all subfields of the progress information field, that is, processing of all stages from fertilization to birth is completed (step S515).

If it is determined that the processing has not ended (NO in step S515), the control unit 21 returns to step S511. As described above, in step S511 after the second time, the control unit 21 may perform transfer learning using the created progress learning model 53. When it is determined that the process is completed (YES in step S515), the control unit 21 ends the process.

FIG. 14 is a flowchart showing the processing flow of the program used in the second preparation stage and the sperm selection stage. The flow of processing performed in the second preparatory step described with reference to FIG. 1B and the sperm selection step described with reference to FIG. 1C will be described with reference to FIG.

The control unit 21 activates a subroutine for sample distribution calculation (step S521). The sample distribution calculation subroutine is a subroutine for performing preliminary imaging of sperm contained in the semen provided by the sperm donor and calculating the sample distribution g (X). The process flow of the sample distribution calculation subroutine will be described later.

The control unit 21 acquires the number of OKs and the number of NGs recorded in step S514 of FIG. 13 for each stage from fertilization to delivery (step S522). The control unit 21 substitutes the number of “OK” into p of the equation (1) and the number of “NG” into q and obtains the first distribution f (X) and the sample calculated in the sample distribution calculation subroutine. A second distribution h (X) obtained by substituting the distribution g (X) and the equation (3) is calculated (step S523).

The control unit 21 displays the screen described with reference to FIG. 11 on the display device 15. The user moves the stage 42 to search for a sperm candidate to be used for microinsemination. The user selects the determination button 68 with only one normal sperm being displayed in the image column 61 as described with reference to FIG. 11.

When the selection of the determination button 68 is accepted, the control unit 21 acquires a photographed image of sperm through the camera 48 (step S524). The control unit 21 extracts the sperm image feature amount (step S525). The control unit 21 inputs the sperm image feature amount into the progress learning model 53 at each stage from fertilization to delivery, and calculates a prediction success probability of normal growth at each stage (step S526).

The control unit 21 calculates an evaluation index for the prediction success probability calculated in step S526 within the second distribution h (X) calculated in step S523 (step S527). The evaluation index is, for example, a deviation value of the prediction success probability.

The control unit 21 determines the comprehensive evaluation of the candidate sperm based on the evaluation index for each stage from fertilization to delivery (step S528). For example, the control unit 21 determines that the candidate sperm is “good”, “normal”, or “bad” based on the average value or the minimum value of the deviation values of the prediction success probability for each stage. To do. The control unit 21 displays the screen described with reference to FIG. 12 on the display device 15 (step S529).

The control unit 21 determines whether or not the selection of the next button 69 is accepted (step S530). When it is determined that the selection of the next button 69 is accepted (YES in step S530), the control unit 21 displays the screen described with reference to FIG. 11 on the display device 15, and returns to step S524.

When it is determined that the selection of the next button 69 is not accepted even after the lapse of a predetermined time, or when it is determined that the end instruction is accepted (NO in step S530), the control unit 21 ends the process.

FIG. 15 is a flowchart showing the flow of processing of a subroutine for sample distribution calculation. The sample distribution calculation subroutine is a subroutine for performing preliminary imaging of sperm contained in the semen provided by the sperm donor and calculating the sample distribution g (X).

The control unit 21 displays the screen described with reference to FIG. 10 on the display device 15. The control unit 21 displays the screen described with reference to FIG. 11 on the display device 15. The user moves the stage 42 to search for normal sperm that can be used for microinsemination. As described with reference to FIG. 10, the user selects the shooting button 66 in the state where only one normal sperm is displayed in the image column 61.

When the selection of the shooting button 66 is accepted, the control unit 21 obtains the shot image of the sperm through the camera 48 (step S541). The control unit 21 extracts the sperm image feature amount (step S542). The control unit 21 creates a new record in the preliminary shooting DB 52 (step S543).

The control unit 21 inputs the sperm image feature amount into the progress learning model 53 in one of the stages from fertilization to birth, and acquires the prediction success probability (step S544). The control unit 21 determines whether the prediction success probability is equal to or higher than a predetermined threshold value (step S545).

When it is determined that it is equal to or more than the threshold value (YES in step S545), the control unit 21 records “OK” in the field corresponding to the stage of processing the record created in step S543 (step S546). When it is determined that the value is less than the threshold value (NO in step S545), the control unit 21 records “NG” in the field corresponding to the stage of processing the record created in step S543 (step S547).

The control unit 21 determines whether or not the processing for all stages from fertilization to birth is completed (step S548). When it is determined that the processing has not ended (NO in step S548), the control unit 21 returns to step S544.

When it is determined that the pre-shooting is finished (YES in step S548), the control unit 21 determines whether or not the preliminary shooting is finished (step S549). For example, when the selection of the end button 67 is accepted on the screen described with reference to FIG. 10, the control unit 21 determines to end the preliminary shooting.

If it is determined that the preliminary shooting is not finished (NO in step S549), the control unit 21 returns to step S541. When it is determined that the preliminary shooting is finished (YES in step S549), the control unit 21 acquires the number of “OK” and the number of “NG” for the subfield being processed in the prediction result field (step S550). ..

By substituting the number of “OK” into a of the equation (2) and the number of “NG” into b, the sample distribution g (X) described using FIG. 4 can be obtained. The control unit 21 records the number of “OK” and the number of “NG” in the auxiliary storage device 23 in association with the prediction result subfield being processed (step S551).

The control unit 21 determines whether or not the processing of all subfields of the prediction result field, that is, the processing of all stages from fertilization to birth is completed (step S552). When it is determined that the processing has not ended (NO in step S552), the control unit 21 returns to step S550. When it is determined that the process is completed (YES in step S552), the control unit 21 ends the process.

According to the present embodiment, it is possible to provide the information processing system 10 that supports the selection of sperm having a high probability of successful microinsemination.

According to the present embodiment, the success probability predicted by sampling the spermatozoa in the semen with the prior distribution of the first distribution f (X) based on the data recorded by microinsemination using the IMSI in the past and recording the progress is shown. By using Bayesian estimation for each likelihood distribution, whether fertilization is established for sperm contained in semen, success or failure of normal growth up to each stage of 4-splitting embryo, morula, blastocyst, etc. It is possible to accurately predict the posterior distribution of the probability (success probability) that the embryos will normally grow after microinsemination, such as whether or not the condition is established, whether or not there is an abnormality in the pregnant woman screening, and whether or not the baby is born.

According to the present embodiment, in order to display the evaluation of each sperm in real time within the distribution of various success probabilities regarding sperm contained in semen, whether the sperm under observation is used for microinsemination or not Can support the judgment of the incubator. Therefore, even a relatively inexperienced cultivator can select sperm as accurately as a veteran cultivator.

According to the present embodiment, in order to evaluate the success probability at each stage such as the fertilization stage and the blastocyst formation stage, the incubator can select sperm in consideration of the clinical characteristics and condition of the patient. For example, when the number of collected ova is large, the incubator can select spermatozoa with a high success probability after implantation even though the success probability of fertilization is low.

The number of input dimensions of the progress learning model 53 can be reduced by using the image encoder 546 to extract the feature amount of the captured image. This makes it possible to provide the information processing system 10 that evaluates sperm in real time. Since the captured images can be classified with high accuracy by the image encoder 546, it is possible to provide the information processing system 10 that accurately evaluates sperm.

The image encoder 546 may be integrated with the progress learning model 53. The integrated progress learning model 53 is generated by connecting the output of the image encoder 546 to the input layer 531 of the progress learning model 53.

By integrating with the image encoder 546, it is possible to realize the progress learning model 53 that outputs the success probability of each stage in the process from fertilization to birth when a captured image of sperm is input. In the case where the image encoder is integrated with the progress learning model 53, in the teacher data DB 51 described using FIG. 8, instead of the subfield for recording the feature amount of the still image, it is extracted from the moving image file. It is desirable to provide a field for recording sperm images.

The progress learning model 53 may be integrated with an image encoder 546 and a CNN for determination used when extracting a still image including the entire sperm from a moving image file. In this case, the teacher data DB 51 described with reference to FIG. 8 does not need the subfield for recording the feature amount of the still image.

It is desirable to add to the teacher data additional data that is recorded by associating the image feature amount of the image taken at the sperm selection stage with the subsequent process. The accuracy of the progress learning model 53 can be improved by re-learning using new teacher data. The progress learning model 53 updated by the re-learning may be delivered to another microinsemination performing institution via a network.

For example, items such as the health status of newborns may be evaluated by a score from 0% to 100% instead of the evaluation based on success or failure. The first distribution f (X) of the item evaluated by the score is a probability density distribution obtained by approximating the scores of all the recorded cases by the beta distribution. By calculating p and q using the average E and the variance V of the frequency distribution of points, the first distribution f (X) in this case can be expressed by equation (4).

E is the average score.
V is the variance of points.
C ₄ is a normalization constant.

The first distribution f (X) may be calculated, for example, by using an arbitrary distribution of uniform distribution as a prior distribution and performing Bayes update for each piece of teacher data. The sample distribution g (X) is calculated by performing a Bayes update for each pre-photographed sperm as a prior distribution with an arbitrary distribution such as a uniform distribution or the same distribution as the first distribution f (X). Is also good.

A beta distribution may be used for the sample distribution g (X). Specifically, instead of the above equation (2), the equation (5) showing the beta distribution is used.

a is the number of preliminarily photographed images estimated to grow normally up to each growth stage.
b is the number of preliminarily photographed images estimated not to grow normally up to each growth stage.
C ₅ is a normalization constant.

A Gaussian distribution may be used for both or one of the first distribution f (X) and the sample distribution g (X).

As described using FIG. 7, the input layer 531 of the progress learning model 53 stores age information, health status information, past infertility treatment history of sperm donors and egg donors, or genomic information such as gene mutations. , Information about sperm donors and egg donors may be added. By doing so, it is possible to realize the progress learning model 53 that predicts the success probability based on the influence of aging and the like.

When capturing a still image of sperm, the amount of illumination light emitted from the illumination unit 44 may be increased and the exposure time may be reduced. Even a sperm that is actively moving can take clear still images. By using a clear still image, a progress learning model 53 capable of clearly discriminating between spermatozoa with high success and low spermatozoa can be created, and candidate spermatozoa can be evaluated with high accuracy.

It is desirable that the camera 48 has high resolution. By capturing sperm with high resolution, you can capture images with clear moving images and still images. Therefore, it is possible to create a progress learning model 53 that can clearly discriminate between spermatozoa with a high probability of success and spermatozoa with a low probability of success, and evaluate a candidate sperm with high accuracy. In order to obtain practical accuracy, it is desirable to photograph one sperm with a resolution of at least 32 pixels × 32 pixels or more.

It is desirable that the camera 48 be able to take high-contrast images. By using a high-contrast image, individual differences in sperm can be clearly identified. Therefore, it is possible to create a progress learning model 53 that can clearly discriminate between spermatozoa with a high probability of success and spermatozoa with a low probability of success, and evaluate a candidate sperm with high accuracy.

[Embodiment 2]
The present embodiment relates to the information processing system 10 that calculates the sample distribution g (X) based on the success probability acquired from the progress learning model 53. Descriptions of portions common to the first embodiment will be omitted.

FIG. 16 is an explanatory diagram illustrating a record layout of the preliminary shooting DB 52 according to the second embodiment. The preliminary shooting DB 52 is a DB that records a prediction result using the progress learning model 53 in the second preparatory stage described using FIG. 1B. The preliminary imaging DB 52 has a patient ID field, an imaging date field, and a prediction result field. The prediction result field has a fertilization field, a blastocyst formation field, an implantation field, a birth field, and a health status field. The preliminary photographing DB 52 has one field for one preliminary photographing.

The patient ID uniquely assigned to the sperm provider is recorded in the patient ID field. In the shooting date field, the shooting date when the shot image was shot is recorded. In each subfield of the prediction result field, the predicted value of the probability of reaching each step normally is recorded in percentage.

The control unit 21 extracts the sperm image feature amount from the captured image captured by the camera 48 and inputs it to the progress learning model 53 created for each stage to acquire the success probability. The control unit 21 records the acquired probability in each field of the preliminary shooting DB 52.

In the present embodiment, these success probabilities are approximated by the beta distribution g (X) represented by the equation (6). To determine the variables p and q in the approximate expression, the average and the variance are calculated from the frequency distribution regarding the success probability, and the variables are substituted into E and V, respectively.

E is the average (expected value) of the distribution of prediction success probabilities that normally grows to each stage.
V is the variance of the distribution of predictive success probabilities for normal growth to each stage.
C ₆ is a normalization constant.

FIG. 17 is a flowchart showing the processing flow of the sample distribution calculation subroutine of the second embodiment. The sample distribution calculation subroutine shown in FIG. 17 is a subroutine used instead of the sample distribution calculation subroutine described with reference to FIG. The flow of processing up to step S544 is the same as the processing of the subroutine of sample distribution calculation described with reference to FIG.

The control unit 21 records the prediction success probability acquired in step S544 in the field corresponding to the stage in processing of the record created in step S543 (step S561). The control unit 21 determines whether or not the processing has been completed for all stages from fertilization to delivery (step S562). When it is determined that the processing has not ended (NO in step S562), the control unit 21 returns to step S544.

When it is determined that the pre-shooting is finished (YES in step S562), the control unit 21 determines whether the pre-shooting is finished (step S563). When it is determined that the preliminary shooting is not finished (NO in step S563), the control unit 21 returns to step S541.

When it is determined that the preliminary shooting is finished (YES in step S563), the control unit 21 calculates the average value E and the variance V for one subfield of the prediction result field of the preliminary shooting DB52. The control unit 21 calculates the variables p, q and C ₆ based on the equation (6) (step S564). The control unit 21 records the variables p, q, and C ₆ in the auxiliary storage device 23 in association with the prediction result subfield being processed (step S565).

The control unit 21 determines whether or not processing of all subfields of the prediction result field, that is, processing of all stages from fertilization to childbirth has been completed (step S566). When it is determined that the processing has not ended (NO in step S566), the control unit 21 returns to step S564. When it is determined that the process is completed (YES in step S566), the control unit 21 ends the process.

According to the present embodiment, it is not necessary to set a threshold for determining “OK” and “NG” when creating the preliminary shooting DB 52. Therefore, it is possible to provide the information processing system 10 that evaluates the candidate sperm without being affected by the setting of the threshold value.

According to the present embodiment, it is possible to provide the information processing system 10 in which the sample distribution g (X) is stable with a small number of preliminary shootings when there are many sperm with a prediction success probability of around 50%.

[Third Embodiment]
The present embodiment relates to an information processing system 10 that displays a reliability index of a prediction success probability as well as an evaluation index of a prediction success probability of a candidate sperm. Descriptions of portions common to the first embodiment will be omitted.

FIG. 18 is an explanatory diagram showing a screen displayed by the information processing device 20 according to the third embodiment. The screen shown in FIG. 18 is an example of a screen displayed on the display device 15 instead of the screen shown in FIG. 12 at the sperm selection stage. An image field 61, an evaluation field 65 and a next button 69 are displayed on the screen. The evaluation section 65 includes a first evaluation section 651, a second evaluation section 652, a third evaluation section 653, and a fourth evaluation section 654.

In the first evaluation column 651 to the fourth evaluation column 654, the deviation value as an evaluation index and the reliability of the evaluation index are displayed, respectively. Confidence is the reciprocal of accidental uncertainty due to data inhomogeneity, for example. For random uncertainty according to Gaussian distribution, the accuracy β = 1 / σ ² which is the reciprocal of variance σ ² = Σ (X−Xm) ² / N (where N is the number of samples and Xm is the average value of X) is reliable. It becomes degree.

The reliability is the reciprocal of the uncertainty of recognition, such as the uncertainty of the model subjected to supervised machine learning, the uncertainty of parameters, or the uncertainty of completeness. The uncertainty of recognition can be calculated by using Monte Carlo Dropout Sampling etc. where the parameters of the machine learning model are replaced with Bernoulli distribution. It should be noted that, in order for the incubator to intuitively understand the degree of reliability, by adding an appropriate constant to the calculated reliability and then multiplying it by another appropriate constant, for example, a value from 0 to 100 is converted. The calculated value may be displayed as the reliability.

For example, the incubator avoids selecting a candidate sperm if the reliability is low even if the deviation of the candidate sperm is high. I can decide.

[Embodiment 4]
The present embodiment relates to an information processing system 10 that automatically determines candidate sperm. Descriptions of portions common to the first embodiment will be omitted.

FIG. 19 is an explanatory diagram illustrating the configuration of the information processing system 10 according to the fourth embodiment. The information processing device 20 has a stage I / F 26. The stage I / F 26 is, for example, a USB terminal. The microscope 41 has a stage moving unit 46 that operates the stage 42. The stage moving unit 46 is connected to the stage I / F 26.

The control unit 21 controls the stage moving unit 46 via the stage I / F 26. The control unit 21 controls the stage moving unit 46 so that the field of view of the microscope 41 scans the entire observation container 421, for example. The control unit 21 may control the stage moving unit 46 based on an instruction from the user.

FIG. 20 is an explanatory diagram illustrating the record layout of the normal sperm determination DB. The normal sperm determination DB is teacher data used for supervised machine learning for discriminating between normal sperm and other sperm. The normal sperm determination DB has an image field and a determination field. In the image field, an image obtained by photographing normal sperm, abnormal sperm, foreign matter, or the like is recorded. In the judgment field, the result of the culture person judging whether the image is “normal sperm”, “abnormal sperm” or “foreign matter” is recorded. The normal sperm determination DB has one record for one image.

FIG. 21 is an explanatory diagram showing the configuration of the normal sperm determination model 57. The normal sperm determination model 57 is a neural network including an input layer 571, an intermediate layer 572, and an output layer 573. In FIG. 21, the case where the normal sperm determination model 57 is CNN is illustrated.

The image recorded in the image field of the normal sperm determination DB is input to the input layer 571. Specifically, the input layer 571 has the same number of neurons as the number of pixels of the image, and the pixel value of each pixel of the image is input to each neuron. The output layer 573 has a total of three neurons that output the probability that the determination is “normal sperm”, the probability that the determination is “abnormal sperm”, and the probability that the determination is “foreign matter”.

The control unit 21 performs supervised machine learning that calculates the parameters of the intermediate layer 572 using an error backpropagation method or the like so that the determination result recorded in the determination field of the normal sperm determination DB is output from the output layer 543. To do. For example, when “normal sperm” is recorded in the determination field, the control unit 21 determines that the probability of being “normal sperm” is 1, the probability of being “abnormal sperm” and the probability of being “foreign body”. The parameters of the intermediate layer 572 are calculated so as to be 0.

By inputting the image captured by the camera 48 to the learned normal sperm determination model 57, the control unit 21 can determine whether normal sperm, abnormal sperm, or foreign matter is captured.

The normal sperm determination model 57 is created by using transfer learning for additionally learning the normal sperm determination DB to an existing public model such as VGG (Visual Geometry Group) -16, ResNET (Residual Network) -50 or Xception. Good. By using the public model, an accurate normal sperm determination model 57 can be created with less teacher data.

22 to 24 are explanatory diagrams showing screens displayed by the information processing device 20 according to the fourth embodiment. The screens shown in FIGS. 22 to 24 are examples of screens displayed on the display device 15 in place of the screen shown in FIG. 12 at the sperm selection stage. An image column 61 and an evaluation column 65 are displayed on the screen.

In the image column 61, the image taken by the camera 48 is displayed in real time. The control unit 21 determines whether or not the evaluation target is included in the visual field by using a known image recognition technique. The control unit 21 may input the image to a CNN for determination prepared in advance to determine whether or not the evaluation target is included in the visual field.

In the evaluation column 65, the evaluation result obtained by evaluating the image displayed in the image column 61 in real time is displayed. In the example shown in FIG. 22, normal sperm are included in the visual field, and the same evaluation results as in FIG. 12 are displayed.

In the example shown in FIG. 23, abnormal sperm are included in the visual field, and “abnormal sperm” is displayed in the comprehensive evaluation field 659. For abnormal spermatozoa, the prediction success probability is not acquired using the progress learning model 53, and “-” is displayed in the first evaluation column 651 to the fourth evaluation column 654.

In the example shown in FIG. 24, the evaluation target is not included in the field of view, and “undecidable” is displayed in the comprehensive evaluation field 659. Since the evaluation target is not included, the prediction success probability is not acquired using the progress learning model 53, and “-” is displayed in the first evaluation column 651 to the fourth evaluation column 654.

FIG. 25 is a flowchart showing the flow of processing of the program according to the fourth embodiment. In the present embodiment, the stage 42 operates automatically or forward, backward, leftward, and rightward based on an instruction from the user to scan the microscope visual field.

The control unit 21 activates a sample distribution calculation subroutine (step S601). The sample distribution calculation subroutine is a subroutine for calculating the sample distribution g (X) by performing the same processing as the sample distribution calculation subroutine described using FIG. 15 or FIG.

However, in the subroutine started in step S601, the control unit 21 starts a sperm image acquisition subroutine instead of step S541 of the subroutine described using FIG. 15 or FIG. The sperm image acquisition subroutine is a subroutine for automatically acquiring a normal sperm image from the microscope 41. The processing flow of the sperm image acquisition subroutine will be described later.

The control unit 21 acquires the number of OKs and the number of NGs recorded in step S514 of FIG. 13 for each stage from fertilization to birth (step S602). The control unit 21 calculates the second distribution h (X) based on the equations (1) to (3) (step S603).

The control unit 21 detects that there is a subject in the field of view photographed by the camera 48 using a known image analysis technique (step S604). The control unit 21 inputs the image including the subject into the normal sperm determination model 57 described using FIG. 21, and acquires the determination result (step S605).

The control unit 21 determines whether the probability that the subject is normal sperm is equal to or higher than a predetermined threshold value (step S606). When it determines with it being more than a threshold value (YES in step S606), the control part 21 extracts a sperm image feature-value (step S525). After that, the processing up to step S528 is the same as the processing of the program described with reference to FIG. The control unit 21 displays the screen described with reference to FIG. 22 on the display device 15 (step S607).

When it is determined that it is less than the threshold value (NO in step S606), the control unit 21 determines whether the probability that the subject is abnormal sperm is equal to or higher than a predetermined threshold value (step S611). When it determines with it being more than a threshold value (YES in step S611), the control part 21 displays the screen which shows that it is abnormal sperm which was demonstrated using FIG. 23 on the display device 15 (step S612). When it determines with it being less than a threshold value (NO in step S611), the control part 21 displays the screen which shows that determination is impossible on the display device 15 as described using FIG. 24 (step S613).

After completion of step S607, step S612, or step S613, the control unit 21 determines whether or not the selection of the next button 69 is accepted (step S614). When it is determined that the selection of the next button 69 is accepted (YES in step S614), the control unit 21 returns to step S604.

If it is determined that the selection of the next button 69 is not accepted even after the lapse of a predetermined time, or if it is determined that the end instruction is accepted (NO in step S514), the control unit 21 ends the process.

FIG. 26 is a flowchart showing the flow of processing of a subroutine for sperm image acquisition. The sperm image acquisition subroutine is a subroutine for automatically acquiring a normal sperm image from the microscope 41. The sperm photographing subroutine is used instead of step S541 of the sample distribution calculation subroutine described with reference to FIGS.

The control unit 21 detects that there is a subject in the field of view using a known image analysis technique (step S571). The control unit 21 inputs an image including the subject into the normal sperm determination model 57, and acquires the probability that the subject is normal sperm (step S572).

The control unit 21 determines whether the probability that the subject is normal sperm is a predetermined threshold abnormality (step S573). When it determines with it being less than a threshold value (NO in step S573), the control part 21 returns to step S571. When it determines with it being more than a threshold value (it is YES at step S573), the control part 21 complete | finishes a process.

According to the present embodiment, it is possible to provide the information processing system 10 that automatically extracts and determines normal sperm. For example, even if the number of normal sperm is extremely small, it is possible to provide the information processing system 10 that places a little burden on the incubator.

By separating the normal sperm determination model 57 that extracts normal sperm and the progress learning model 53 that evaluates normal sperm, it is possible to provide the information processing system 10 that performs highly accurate determination.

[Fifth Embodiment]
The present embodiment relates to an information processing system 10 that displays an evaluation result of candidate sperm using a graph. Descriptions of portions common to the third embodiment will be omitted.

FIG. 27 is an explanatory diagram showing a screen displayed by the information processing device 20 according to the fifth embodiment. The screen shown in FIG. 27 is an example of a screen displayed on the display device 15 instead of the screen shown in FIG. 18 at the sperm selection stage. An image column 61, an evaluation column 65 and a patient information column 64 are displayed on the screen. The evaluation section 65 includes a first evaluation section 651, a second evaluation section 652, a third evaluation section 653, a fourth evaluation section 654, and a fifth evaluation section 655.

In the patient information column 64, the age, medical history, current medical history, and infertility treatment history of each sperm donor and egg donor are displayed. In the patient information column, “M” indicates a sperm donor and “F” indicates an egg donor.

In the image column 61, the candidate sperm under evaluation is displayed as a still image. In the image column 61, a microscope image taken by the camera 48 may be displayed in real time.

A graph of the second distribution Y = h (X) at each stage is displayed in the first evaluation column 651 to the fourth evaluation column 654. The vertical axis of the graph represents the probability of success as the random variable X, and the horizontal axis represents the probability Y corresponding to the random variable X. In each graph, the success rate of the candidate sperm is displayed by the index line 81.

Under each graph, the success rate calculated for the candidate sperm, the deviation value of the success rate in the second distribution, and the reliability are displayed. When the success rate of the candidate sperm is greater than or equal to the maximum success rate of the sperm evaluated in the past contained in the same semen, the word “Best” is displayed below the reliability.

In the fifth evaluation column 655, a graph showing the distribution of sperm abnormalities is displayed. The vertical axis of the graph represents the degree of abnormality of sperm as the random variable X, and the horizontal axis represents the probability Y corresponding to the random variable X. In the graph, the degree of abnormality of the candidate sperm is displayed by the index line 81.

The degree of abnormality calculated for the candidate sperm and the deviation value and reliability of the degree of abnormality in the distribution of the degree of abnormality are displayed below the graph. When the degree of abnormality of the candidate sperm is equal to or lower than the minimum value of the degree of abnormality of sperm evaluated in the past contained in the same semen, the word “Best” is displayed below the reliability.

The abnormal degree of sperm is the probability that a sperm is an abnormal sperm, which is acquired using the normal sperm determination model 57 described using FIG. 21, for example. The abnormal degree of sperm may be calculated based on the similarity between the shape of the sperm regarded as “normal” and the shape of the candidate sperm. The graph in the fifth evaluation column 655 is, for example, the distribution of abnormalities of sperm of men of the same age as the sperm donor.

The distribution of the abnormalities of sperm of men of the same age is used as the first distribution, and the distribution of the abnormalities of sperm taken in the preliminary imaging is used as the sample distribution, and the second distribution is calculated in the graph of the fifth evaluation column 655. May be used. In such a case, first, similar to the fourth embodiment, preliminary image capturing is automatically performed to create a sample distribution of the degree of abnormality of the whole sperm contained in semen. After that, for the candidate sperm determined to have a normal shape, a sample distribution at each stage from fertilization to delivery is created.

According to the present embodiment, the incubator can intuitively grasp the position of the candidate sperm in the distribution based on the index line 81. Therefore, the cultivator can quickly determine whether or not to use the candidate sperm for microinsemination.

A moving image file obtained by shooting candidate sperms for a few seconds may be temporarily stored in the auxiliary storage device 23 and repeatedly reproduced and displayed in the image column 61. The incubator can fully observe the movement of the candidate sperm by the moving image, and can judge whether or not to use it for microinsemination. A button for instructing fast-forward, reverse-forward, pause, slow-play, high-speed play, or the like of a moving image to be repeatedly played may be displayed around or inside the image column 61.

The control unit 21 may accept the designation of the frame from which the still image feature amount is extracted and repeatedly evaluate the candidate sperm during the repeated reproduction of the moving image file. An incubator can evaluate candidate sperm based on the desired frame.

 Video files taken in the past may be displayed in the image column 61. For example, it is possible to provide an information processing system 10 to be used for conferences in medical facilities, training of incubators, and information exchange with other facilities. It is also possible to confirm the operation after updating the progress learning model 53 or the image encoder 546 using a moving image file captured in the past.

The camera 48 may be capable of high-speed shooting. By displaying a moving image file of the candidate sperm photographed by high-speed photography in the image field 61 by slow reproduction, the incubator can observe the movement of the candidate sperm. By doing so, the incubator can accurately confirm the natural movement of the sperm and the defocus of the microscopic image without using a highly viscous solution. An image of a candidate sperm suitable for evaluation can be acquired by the incubator adjusting the position of the objective lens. Furthermore, during microinsemination, it is possible to avoid injecting the highly viscous solution with the sperm into the egg.

[Sixth Embodiment]
FIG. 28 is a functional block diagram of the information processing device 20 according to the sixth embodiment. The information processing device 20 includes a captured image acquisition unit 71, an input unit 72, and an output unit 73. The captured image acquisition unit 71 acquires a captured image of a candidate sperm used for microinsemination.

The input unit 72 receives the captured image of the sperm and inputs the captured image to the learning model 53 that outputs a prediction regarding the success or failure of microinsemination using the sperm. The output unit 73 outputs the prediction output from the learning model 53 based on the input captured image.

The learning model 53 has an input layer 571, an image encoder 546, an intermediate layer 572, and an output layer 573. A captured image of a sperm is input to the input layer 571. The output layer 573 outputs the prediction success probability of normal growth up to each stage after microinsemination using sperm.

The middle layer 572 is a teacher that records the captured images of sperm used for microinsemination in the past and the normal growth up to each stage after microinsemination using the sperm. The parameters are learned using the data.

In the learning model 53, when a captured image of a candidate sperm used for microinsemination is input to the input layer 571, each stage in the case where microinsemination is performed using the candidate sperm through the calculation by the intermediate layer 572 The computer is caused to output from the output layer 573 a prediction as to whether or not it grows normally.

The learning model 53 is further provided between the input layer 571 and the intermediate layer 572, and further includes an image encoder 546 that extracts the image feature amount from the captured image. The image feature amount extracted by the image encoder 546 is stored in the intermediate layer 572. Output.

[Embodiment 7]
The present embodiment relates to a form in which the information processing system 10 of the present embodiment is realized by operating a general-purpose computer 90 and a program 97 in combination. FIG. 29 is an explanatory diagram showing the configuration of the information processing system 10 according to the seventh embodiment. Descriptions of portions common to the first embodiment will be omitted.

The information processing system 10 of the present embodiment includes a computer 90 and a microscope 41.

The computer 90 includes a control unit 21, a main storage device 22, an auxiliary storage device 23, a communication unit 24, a display I / F 25, a photographing I / F 28, a reading unit 29, and a bus. The computer 90 is an information device such as a general-purpose personal computer, tablet or server computer.

The program 97 is recorded in the portable recording medium 96. The control unit 21 reads the program 97 via the reading unit 29 and stores it in the auxiliary storage device 23. The control unit 21 may also read the program 97 stored in the semiconductor memory 98 such as a flash memory installed in the computer 90. Furthermore, the control unit 21 may download the program 97 from another server computer (not shown) connected via the communication unit 24 and a network (not shown) and store the program 97 in the auxiliary storage device 23.

The program 97 is installed as a control program of the computer 90, loaded into the main storage device 22 and executed. As a result, the computer 90 functions as the information processing device 20 described above.

[Embodiment 8]
The present embodiment relates to an information processing system 10 that determines whether the sperm under observation is an X sperm having an X chromosome or a Y sperm having a Y chromosome. Descriptions of portions common to the first embodiment will be omitted. In the following description, the effect of mutations on the sex chromosome will not be considered.

For example, various congenital genetic diseases such as red-green deficiency, hemophilia, and Alport syndrome are known. A sex-linked genetic disease is a genetic disease derived from an abnormality of the sex chromosome. The incidence of children with a congenital disorder who develops a congenital inherited disease, and the prevalence of a genetic factor that does not affect the child itself but conveys the congenital inherited disease to the offspring, is It depends on the type of disease and the sex of the child.

FIG. 30 is a table showing the incidence and carrier rate of sex-linked recessive (latent) genetic diseases (in the following description, “recessive” and “latent” are used synonymously). A sex-linked recessive disease is a disease that is not inherited by having one normal X chromosome among diseases inherited via the X chromosome. Males have a single X chromosome, so they develop if they have an abnormality. A female does not develop if only one of the two X chromosomes is abnormal, but does develop if both are abnormal. For example, a sex-linked recessive genetic disease such as red-green deficiency and hemophilia is known.

The leftmost column in FIG. 30 shows whether the sex chromosomes of the sperm donor are normal or abnormal. If it is abnormal, the X chromosome of the sperm donor is abnormal and the disease develops. The second column from the left in FIG. 30 shows the cases when X spermatozoa are used and when Y spermatozoa are used. When X sperm is used, the child's gender is female. When Y sperm are used, the sex of the child is male. In the following description, the distinction whether the sperm is X sperm or Y sperm may be described as the sex of sperm.

The three columns on the right side of FIG. 30 indicate whether the sex chromosome of the egg donor is normal or abnormal. When it is abnormal, there are two types, one is heterozygous in which one X chromosome is abnormal and the other X chromosome is normal, and the other is homozygous in which both X chromosomes are abnormal. When heterozygous, it is a carrier who does not develop but carries the genetic factor. If it is homozygous, it develops.

▽ Incidence rate indicates the probability that a child will develop a sex-linked genetic disease. Carriage rate indicates the probability that a child's sex chromosome has a genetic factor that conveys a sex-linked genetic disease, ie, the child has a genetic abnormality.

For example, if the sex donor's sex chromosome is normal and the sperm donor's sex chromosome is abnormal, the incidence is 0% regardless of the sex of the child, the carrier rate is 100% for girls, and 100% for boys. Has a carrier rate of 0%. In FIG. 30, in the part surrounded by a rounded rectangle, the incidence or carrier rate varies depending on the sex of the child.

FIG. 31 is a table showing the incidence rate and carrier rate of a sex-linked dominant (overtly) genetic disease (in the following description, “dominant” and “overtly” are used synonymously). A sex-linked dominant genetic disease is a disease that develops if there is at least one abnormal X chromosome among diseases inherited via the X chromosome. Males have a single X chromosome, so they develop if they have an abnormality. Women develop if one or both of the two X chromosomes is abnormal. For example, sex-linked dominant genetic diseases such as Rett syndrome and Alport syndrome are known.

The configuration of the table shown in FIG. 31 is the same as the table described in FIG. 30, so description will be omitted. Also in FIG. 31, in the part surrounded by the rounded rectangle, the incidence rate or the carrier rate varies depending on the sex of the child.

FIG. 32 is a table showing the incidence and carrier rate of sex-linked genetic diseases inherited via the Y chromosome. Since women do not have the Y chromosome, they are always normal for genes associated with such diseases. It is considered that a part of male infertility includes a disease inherited via the Y chromosome, such as Y chromosome microdeletion.

The left end of FIG. 32 shows whether the sex chromosome of the sperm donor is normal or abnormal, as in FIG. The egg donor is always normal. In FIG. 32, in the part surrounded by a rounded rectangle, the incidence or carrier rate varies depending on the sex of the child.

When the sex chromosomes of sperm donors and egg donors are abnormal, by determining the sex of the sperm and selecting the sex of the child, the incidence and carrier rate of the offspring can be reduced.

It is known that, depending on the disease, embryos with abnormal sex chromosomes have a low probability of success from implantation to birth and are prone to miscarriage. The success rate of microinsemination can also be increased by determining the sex of the sperm and selecting the sex of the child.

It is known that, in human genes, the amount of genome possessed by X sperm is approximately 3% higher than the amount of genome possessed by Y sperm. Due to the difference in the amount of genome, the dimensions such as head length and perimeter of X sperm are about 5% larger than the corresponding portions of Y sperm, and the mass of X sperm is about 2.8 than the mass of Y sperm. It is also known to be percent heavy. Because X sperm are heavier, Y-sperm move faster.

Thus, there is a difference in appearance between X sperm and Y sperm. However, it is difficult for an incubator to instantly identify and collect necessary sperm. In the present embodiment, the information processing system 10 for determining whether the sperm under observation is X sperm or Y sperm by using the sex determination learning model 56 (see FIG. 34) generated by machine learning. I will provide a.

FIG. 33 is an explanatory diagram illustrating a record layout of the sex teacher data DB 58. The sex teacher data DB 58 is a DB that records the photographed image of the sperm used for microinsemination and the sex of the child born by microinsemination in association with each other. The sex teacher data DB 58 has a sperm ID field, an image data field, a sperm image feature amount field, and a sex field. The sperm ID field, the image data field, and the sperm image feature amount field are the same as the fields of the teacher data DB 51 described with reference to FIG.

In the gender field, the sex of the child born by microinsemination is recorded. In the sex field, the sex of the embryo or fetus determined by prenatal diagnosis such as NIPT (Non-Invasive Prenatal genetic Testing) may be recorded. The sex teacher data DB 58 has one record for each sperm used for microinsemination.

FIG. 34 is an explanatory diagram illustrating the sex determination learning model 56. The gender determination learning model 56 is a neural network including an input layer 561, an intermediate layer 562, and an output layer 563. In FIG. 7, the case where the sex determination learning model 56 is CNN is illustrated. Illustration of the convolutional layer and the pooling layer is omitted.

The gender determination learning model 56 outputs the probability of being an X sperm and the probability of being a Y sperm to the output layer 563 when a feature amount of a still image or a moving image is input to the input layer 561. The information input to the input layer 561 is the same as the input to the progress learning model 53 described using FIG. 34, and thus the description will be omitted.

The gender determination learning model 56 outputs the probability that the sperm under observation is X sperm and the probability that it is Y sperm to the output layer 563 when the feature amounts of a still image and a moving image are input to the input layer 561. In the learning stage, the control unit 21 performs the supervised machine learning by calculating the parameters of the intermediate layer 562 by using the error backpropagation method or the like by using the sex teacher data DB 58 described using FIG. To do.

Supervised machine learning can be performed by any method such as logistic regression, SVM, random forest, CNN, RNN, or XGBoost.

The gender determination learning model 56 may be created using any computer. The created sex determination learning model 56 is transmitted to the information processing device 20 used by the microinsemination executing institution via a network or the like and recorded in the auxiliary storage device 23.

The gender determination learning model 56 may be configured integrally with any of the progress learning models 53 described using FIG. 7. In such a case, the output layer 563 of the sex determination learning model 56 has a success node that outputs a probability of success and an unsuccess node that outputs a probability of failure.

FIG. 35 is an explanatory diagram showing a screen displayed by the information processing device 20 according to the eighth embodiment. When an obstetrician gives an instruction to use sperm of a specific sex for microinsemination, the incubator who is the user causes the display device 15 to display the screen described with reference to FIG. An image field 61, a determination button 68, and an end button 67 are displayed on the screen. In the image column 61, a microscope image taken by the camera 48 is displayed in real time.

When the user selects the determination button 68, the control unit 21 extracts the still image feature amount and the moving image feature amount of the image displayed in the image column 61. The control unit 21 inputs the feature amount into the sex determination learning model 56, and acquires the probability of being an X sperm and the probability of being a Y sperm. The control unit 21 displays the probability of being X sperm and the probability of being Y sperm in the sex column 82 at the upper right of the screen.

When the user selects the determination button 68 again, the control unit 21 updates the probability of being X sperm and the probability of being Y sperm using the image displayed in real time in the image column 61. When the user selects the end button 67, the control unit 21 ends the process.

For example, by using YOLOv3 (YouLookLookOnceversion3), which is a type of CNN, for the sex determination learning model 56, the control unit 21 can quickly acquire the probability of being an X sperm and the probability of being a Y sperm. When the probability of being an X sperm and the probability of being a Y sperm can be acquired sufficiently quickly, the control unit 21 may update the sex column 82 in real time. When the gender column 82 is updated in real time, the determination button 68 is unnecessary.

After the user has found a sperm with a high probability of successful microinsemination using the screen described with reference to FIG. 12, the user determines the sex of the sperm using the screen described with reference to FIG. After discovering the sperm of the sex designated by the obstetrician using the screen described using FIG. 35, the user determines the success rate of microinsemination using the screen described using FIG. May be.

The information processing system 10 of the present embodiment may be a system independent of the information processing system 10 that determines the success probability of microinsemination described in the first embodiment and the like.

The control unit 21 may display the evaluation field 65 described using FIG. 12 and the gender field 82 described using FIG. 35 side by side on one screen.

FIG. 36 is a flow chart showing the flow of processing of the program of the eighth embodiment. The control unit 21 selects the determination button 68 with only one candidate sperm being displayed in the image field 61 as shown in FIG.

The control unit 21 determines whether or not the selection of the determination button 68 has been accepted (step S701). When it is determined that the selection of the determination button 68 has been accepted (YES in step S701), the control unit 21 acquires the captured image of the sperm through the camera 48 (step S702).

The control unit 21 extracts the sperm image feature amount (step S703). The control unit 21 inputs the sperm image feature amount into the sex determination learning model 56, and acquires the probability that the sperm under observation is X sperm and the probability that it is Y sperm (step S704). The control unit 21 displays the acquired probability in the gender column 82 (step S705).

When it is determined that the selection of the determination button 68 has not been received (NO in step S701), or after the end of step S705, the control unit 21 determines whether the selection of the termination button 67 has been received (step S706). When it is determined that the selection of the end button 67 is not accepted (NO in step S706), the control unit 21 returns to step S701. When it is determined that the selection of the end button 67 is accepted (YES in step S706), the control unit 21 ends the process.

According to the present embodiment, it is possible to provide the information processing system 10 that determines and displays the sex of sperm. An incubator can perform microinsemination by selecting spermatozoa that are likely to have the sex instructed by the obstetrician. This can reduce the possibility that a child born by microinsemination will develop a sex-linked genetic disease and the possibility of carrying a factor of the sex-linked genetic disease. It is also possible to prevent miscarriage or the like due to a genetic abnormality associated with a sex-linked genetic disease and increase the probability of successful delivery after microinsemination.

[Ninth Embodiment]
The present embodiment, based on the result of determining whether the sperm under observation is X sperm or Y sperm, information that displays the probability that a child will develop a sex-linked genetic disease and the probability that the factor will be retained. Regarding the processing system 10. Descriptions of portions common to the eighth embodiment will be omitted.

FIG. 37 is an explanatory diagram showing a screen displayed by the information processing device 20 according to the ninth embodiment. A disease setting field 830, an egg provider field 832, and a sperm provider field 833 are displayed on the screen shown in FIG. The disease setting field 830 includes a disease name field 831, an XY field 834, and a dominant recessive field 835. The egg provider column 832 includes an egg confirmation information column 836 and an egg estimation information column 837. The sperm provider column 833 includes a sperm confirmation information column 838 and a sperm estimation information column 839. The screen shown in FIG. 37 is a setting input screen for inputting settings relating to sex-linked genetic disease.

In the disease name column 831, a button for selecting the name of the sex-linked genetic disease to be judged is displayed. In the XY column 834 and the dominant recessive column 835, the inheritance pattern corresponding to the congenital genetic disease selected in the disease name column 831 is displayed by a black circle.

In FIG. 37, the XY columns 834 indicate that “hemophilia” set in the disease name column 831 is a disease inherited via the X chromosome. Similarly, "hemophilia" is a sex-linked recessive genetic disease is displayed in the dominant recessive field 835.

The user sets the name of the sex-linked genetic disease to be determined based on the medical record in the disease name column 831. The control unit 21 displays the XY column 834 and the dominant recessive column 835 based on the database in which the disease name and the inheritance pattern are recorded in association with each other.

When the name of the sex-linked genetic disease is not clearly indicated in the medical record, etc. and only the inheritance pattern is shown, the user should make a judgment in the XY column 834 and the dominant recessive column 835 instead of using the disease name column 831. Enter the inherited form of sex-linked genetic disease. When the user inputs the genetic form, the white circles and the black circles displayed at the left ends of the XY column 834 and the dominant recessive column 835 serve as input buttons that accept the input by the user.

The egg provider column 832 accepts input of information regarding the sex donor's sex chromosome. The user confirms whether or not the presence or absence of the sex chromosome abnormality of the egg donor is confirmed based on the medical record or the like.

If it has been confirmed, the user selects the check box located in the upper left of the egg confirmation information field 836 to display a check mark. The user inputs whether each of the two X chromosomes possessed by the egg donor is normal or abnormal based on the medical record or the like using the button displayed at the bottom of the egg confirmation information field 836. In FIG. 37, it is input that both of the two X chromosomes are “abnormal”.

If not confirmed, the user selects the check box located at the upper left of the egg estimation information field 837 to display a check mark. The egg estimation information column 837 displays columns for inputting the estimated probability that the sex donor's sex chromosome is normal, the estimated probability of being a heterozygous abnormality, and the estimated probability of being a homozygous abnormality in percentage. ..

If the presence or absence of the sex chromosome abnormality in the egg donor is not confirmed, an expert such as an obstetrician estimates the probability related to the presence or absence of the sex chromosome abnormality in the egg donor based on the family history and records it in the medical record, etc. .. The user inputs the estimated probability in each column of the egg estimation information column 837 based on the medical record and the like.

The sperm donor column 833 accepts input of information regarding the sex chromosome of the sperm donor. The user confirms whether the presence or absence of the sex chromosome abnormality of the sperm donor is confirmed based on the medical record or the like.

If it has been confirmed, the user selects the check box located at the upper left of the sperm confirmation information field 838 to display a check mark. The user inputs whether each of the X chromosome and the Y chromosome possessed by the sperm donor is normal or abnormal based on the medical record and the like using the button displayed at the bottom of the sperm confirmation information field 838. In FIG. 37, the fact that both the X chromosome and the Y chromosome are “normal” is input.

If not confirmed, the user selects the check box located at the upper left of the sperm estimation information field 839 to display a check mark. The sperm estimation information column 839 displays a column for inputting, as a percentage, the estimated probability that the X chromosome and the Y chromosome of the sperm donor are normal and the estimated probability that they are abnormal.

If the presence or absence of a sex chromosome abnormality in the sperm donor has not been determined, an expert such as an obstetrician estimates the probability of the presence or absence of a sex chromosome abnormality in the sperm donor based on the family history and records it in the medical record, etc. .. The user inputs the estimated probability in each column of the sperm estimation information column 839 based on the medical record or the like.

Specifically, when the XY column 834 displays that determination regarding a disease inherited via the X chromosome is displayed, the user has an estimated probability that the X chromosome is normal based on the medical record and the like, and the abnormality. The estimated probability is entered in the sperm estimation information field 839. When the XY column 834 displays that determination regarding a disease inherited through the Y chromosome is displayed, the user estimates the estimated probability that the Y chromosome is normal and the estimated probability that the Y chromosome is abnormal based on the medical record and the like. Input in the information column 839.

Note that the control unit 21 calculates and displays the remaining one so that the total probability becomes 1 when a number is input in two of the three fields included in the egg estimation information field 837. You may. Similarly, the control unit 21 may calculate and display the other so that the total probability becomes 1 when the number is input to one of the two fields included in the sperm estimation information field 839. When the total probability of the egg estimation information column 837 and the total probability of the sperm estimation information column 839 are not 1, the control unit 21 may display an error message and prompt the user to make corrections.

FIG. 38 is an explanatory diagram showing a screen displayed by the information processing device 20 according to the ninth embodiment. The screen shown in FIG. 38 is displayed instead of the screen described using FIG. In the screen shown in FIG. 38, an incidence rate column 841 and a carrier rate column 842 are displayed between the sex column 82 and the determination button 68.

The incidence rate column 841 displays the probability that a child born by microinsemination will develop a sex-linked genetic disease. The carrier rate column 842 displays the probability that a child born by microinsemination is a carrier of a sex-linked genetic disease.

An outline of the method of calculating the incidence rate and the carrier rate will be explained, taking the case of determining "X recessive genetic disease" as an example. The presence or absence of genetic abnormality in the egg donor and the sperm donor, and the incidence and carrier rate in children are shown in FIG. 30.

In the example shown in FIG. 37, abnormal genes on the X chromosome of the egg donor are homozygous with each other, and the X chromosome of the sperm donor is normal. As surrounded by the rounded rectangle in the upper right of FIG. 30, the incidence rate is 0% and the carrier rate is 100% if the child is a girl, and the incidence rate and the carrier rate are 100% if the child is a boy. Is.

The tables shown in FIGS. 30 to 32 are recorded in the auxiliary storage device 23 in the form of a database or a program that outputs the incidence rate and the carrier rate when a predetermined condition is input. The control unit 21 respectively acquires the incidence rate and the carrier rate when the child is a boy and when the child is a girl, based on the input by the user.

When sperm with a probability of X sperm of 10% and a probability of Y sperm of 90% are used for microinsemination, the probability that a fertilized egg is male, that is, the probability that a male will be born is 90%. The probability of being a woman, ie the birth of a girl, is 10%.

The control unit 21 adds the product of the probability of birth of a boy and the incidence of boys, and the product of the probability of birth of a girl and the incidence of girls to the incidence of children born by microinsemination. To calculate. Similarly, the control unit 21 adds the product of the probability of birth of a boy and the carrier rate of a boy and the product of the probability of birth of a girl and the carrier rate of a girl to give birth by microinsemination. Calculate the child carrier rate.

In the example shown in FIG. 37, the incidence rate of children born by microinsemination is 90% and the carrier rate is 100%.

Explain an example in which the abnormal gene on the X chromosome of the egg donor is heterozygous for each other and the gene of the sperm donor is normal. As shown by the rounded rectangle in the upper center of FIG. 30, if the child is a girl, the incidence is 0% and the carrier rate is 50%. If the child is a boy, the incidence and carrier rate are both. 50 percent.

When using sperm with a 10% chance of being X sperm and a 90% chance of being Y sperm for microinsemination, there is a 90% chance of having a male and a 10% chance of having a female.

The control unit 21 calculates the product of the incidence of males, 50%, the probability of males being born, 90%, the incidence of females, 0%, and the probability of being females, 10%. The disease incidence is calculated to be 45 percent by adding Similarly, the control unit 21 calculates that the carrier rate of the disease is 50%.

In the screen described using FIG. 37, when the estimated probability is input in the egg estimation information column 837 or the sperm estimation information column 839 with a value other than 0% or 100%, the control unit 21 determines the respective cases. The disease incidence and carrier rate are calculated by weighted averaging the combinations of

39 is a flowchart showing the flow of processing of the program according to the ninth embodiment. The control unit 21 displays the setting input screen described using FIG. 37 (step S711). The control unit 21 acquires the setting input by the user (step S712).

The control unit 21 activates a subroutine for calculating an onset rate and a carrier rate (step S713). The onset rate and carrier rate calculation subroutines are subroutines for calculating the onset rate and carrier rate of a sex-linked genetic disease when X sperm is used and when Y sperm is used, respectively. The processing flow of the subroutine for calculating the incidence rate and the carrier rate will be described later.

The control unit 21 extracts the sperm image feature amount (step S703). The control unit 21 inputs the sperm image feature amount into the sex determination learning model 56, and acquires the probability that the sperm under observation is X sperm and the probability that it is Y sperm (step S704).

The control unit 21 calculates the incidence rate when the observed sperm is used based on the incidence rate of each sperm by sex and the probability of each sex of the sperm acquired in step S713 (step S721). Specifically, the control unit 21 calculates the onset rate by the expression (7).
Incidence rate = Incidence rate when X sperm is used x Probability of X sperm + Incidence rate when Y sperm is used x Probability of Y sperm ... (7)

The control unit 21 calculates a carrier rate when the observed sperm is used based on the incidence of each sperm by sex and the probability of each sperm acquired in step S713 (step S722). Specifically, the control unit 21 calculates the carrier rate by the equation (8).
Incidence rate = Carrier rate with X sperm x Probability of X sperm + Carrier rate with Y sperm x Probability of Y sperm (8)

The control unit 21 displays the incidence rate calculated in step S721 in the incidence rate column 841 and the carrier rate calculated in step S722 in the carrier rate column 842 (step S723).

When it is determined that the selection of the determination button 68 has not been accepted (NO in step S701), or after the end of step S723, the control unit 21 determines whether the selection of the termination button 67 has been accepted (step S706). When it is determined that the selection of the end button 67 is not accepted (NO in step S706), the control unit 21 returns to step S701. When it is determined that the selection of the end button 67 is accepted (YES in step S706), the control unit 21 ends the process.

FIG. 40 is a flowchart showing the flow of processing of a subroutine for calculating the incidence rate and carrier rate. The onset rate and carrier rate calculation subroutines are subroutines for calculating the onset rate and carrier rate of a sex-linked genetic disease when X sperm is used and when Y sperm is used, respectively.

In the following description, the probability that the gene of the egg donor is normal is Pxn, the probability that the gene is heterozygous is Pxt, and that of the homozygous type is the input gene received through the egg donor column 832 in FIG. The probability of being abnormal is described as Pxm. Similarly, the probability that the gene of the sperm donor who received the input via the sperm donor column 833 of FIG. 37 is normal is described as Pyn, and the probability of being abnormal is described as Pya.

If the egg confirmation information field 836 is used, one of Pxn, Pxt, and Pxm is 100%, and the rest is 0%. When the sperm confirmation information field 838 is used, one of Pyn and Pya is 100% and the other is 0%.

The control unit 21 determines whether or not the sex-linked genetic disease that has been input via the disease setting field 830 is the Y genetic disease (step S731). When it is determined that the disease is Y genetic disease (YES in step S731), the control unit 21 uses the X sperm and the Y sperm, and the incidence rate of the congenital genetic disease in the fertilized egg, that is, in the newborn baby. The carrier rate is calculated (step S732).

Specifically, the incidence and carrier rate when X sperm are used is 0%. The incidence and carrier rate when Y sperm are used are equal to the probability Pya that the sex chromosome of the sperm donor is abnormal. After that, the control unit 21 ends the process.

If it is determined that the disease is not a Y genetic disease (NO in step S731), the control unit 21 acquires the incidence rate matrix Mp related to the inherited form of the sex-linked genetic disease that has been input via the disease setting field 830 (step S733). ). The incidence rate matrix Mp is created for each inheritance type and sperm gender based on the incidence rate in the table showing the incidence rate and carrier rate of the sex-linked genetic disease described using FIGS. 30 and 31. It is recorded in the auxiliary storage device 23. The incidence rate matrix Mp is expressed by the equation (9).

Mp11 indicates the incidence when the sex chromosomes are normal in both sperm donors and egg donors.
Mp12 indicates the incidence when the sex chromosome of the sperm donor is normal and the sex chromosome abnormality of the egg donor is heterozygous.
Mp13 indicates the incidence when the sex chromosome of the sperm donor is normal and the sex chromosome abnormality of the egg donor is homozygous.
Mp21 indicates the incidence when the sex chromosome of the sperm donor is abnormal and the sex chromosome of the egg donor is normal.
Mp22 indicates the incidence when the sex chromosome of the sperm donor is abnormal and the sex chromosome abnormality of the egg donor is heterozygous.
Mp21 indicates the incidence when the sex chromosome of the sperm donor is abnormal and the sex chromosome abnormality of the egg donor is homozygous.

For example, regarding the sex-linked recessive disease described using FIG. 30, the incidence rate matrix Mp when X spermatozoa is used is expressed by (10), and the incidence rate matrix Mp when Y sperm is used is (11). Each is shown in the formula.

The control unit 21 calculates the incidence rate Pp for each of the X sperm and the Y sperm based on the equation (12) (step S734).

The control unit 21 acquires the carrier ratio matrix Mc related to the inherited form of the sex-linked genetic disease, which is input via the disease setting field 830 (step S735). The carrier ratio matrix Mc is created for each inheritance type and sperm gender based on the carrier ratio in the table showing the incidence and carrier ratio of the sex-linked genetic disease described using FIGS. 30 and 31. And is recorded in the auxiliary storage device 23. The carrier ratio matrix Mc is expressed by equation (13).

Mc11 indicates the carrier rate when the sex chromosome is normal in both sperm donors and egg donors.
Mc12 indicates the carrier rate when the sex chromosome of the sperm donor is normal and the sex chromosome abnormality of the egg donor is heterozygous.
Mc13 shows the carrier rate when the sex chromosome of the sperm donor is normal and the sex chromosome abnormality of the egg donor is homozygous.
Mc21 indicates the carrier rate when the sex chromosome of the sperm donor is abnormal and the sex chromosome of the egg donor is normal.
Mc22 indicates the carrier rate when the sex chromosome of the sperm donor is abnormal and the sex chromosome abnormality of the egg donor is heterozygous.
Mc21 indicates the carrier rate when the sex chromosome of the sperm donor is abnormal and the sex chromosome abnormality of the egg donor is homozygous.

The control unit 21 calculates the carrier rate Pc for each of the X sperm and the Y sperm based on the equation (14) (step S736). After that, the control unit 21 ends the process.

According to the present embodiment, it is possible to provide the information processing system 10 that displays the incidence rate and carrier rate of a sex-linked genetic disease in addition to the sex of sperm.

Based on the information presented by the obstetrician, the incubator can select sperm with a low incidence and carrier rate of sex-linked genetic disease for microinsemination. This can reduce the possibility that a child born by microinsemination will develop a sex-linked genetic disease and the possibility of carrying a factor of the sex-linked genetic disease. Further, it is possible to prevent a miscarriage or the like due to a genetic abnormality associated with a sex-linked genetic disease and increase the probability of successful delivery after microinsemination.

▽ For each of a plurality of sex-linked genetic diseases, the incidence and carrier rate may be calculated and displayed in a list. When the egg donor and the sperm donor have different gene abnormalities related to the sex-linked genetic disease, select a sperm that is predicted to have a low incidence and carrier rate of the sex-linked genetic disease and perform microinsemination. It

The technical features (constituent elements) described in the respective embodiments can be combined with each other, and by combining them, new technical features can be formed.
The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the meanings described above but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope.

10 Information Processing System 15 Display Device 20 Information Processing Device 21 Control Unit 22 Main Storage Device 23 Auxiliary Storage Device 24 Communication Unit 25 Display I / F
26 Stage I / F
28 Shooting I / F
29 Reading unit 41 Microscope 42 Stage 421 Observation container 43 Eyepiece 44 Illumination unit 45 Optical path splitting unit 46 Stage moving unit 47 Objective lens 48 Camera 51 Teacher data DB
52 Preliminary shooting DB
53 Progressive learning model (learning model)
531 Input layer 532 Intermediate layer 533 Output layer 54 Image feature model 541 Input layer 542 Intermediate layer 543 Output layer 545 Central layer 546 Image encoder 56 Gender determination learning model 561 Input layer 562 Intermediate layer 563 Output layer 57 Normal sperm determination model 571 input Layer 572 Intermediate layer 573 Output layer 58 Gender teacher data DB
61 Image column 62 Target number column 63 Photographed number column 64 Patient information column 65 Evaluation column 651 1st evaluation column 652 2nd evaluation column 653 3rd evaluation column 654 4th evaluation column 655 5th evaluation column 659 Overall evaluation column 66 Imaging Button 67 End button 68 Judgment button 69 Next button 71 Captured image acquisition unit 72 Input unit 73 Output unit 81 Index line 82 Gender column 830 Disease setting column 831 Disease name column 832 Egg donor column 833 Sperm provider column 834 XY column 835 Dominance Recessive column 836 Egg confirmation information column 837 Egg estimation information column 838 Sperm confirmation information column 839 Sperm estimation information column 841 Incidence rate column 842 Carrier rate column 90 Computer 96 Portable recording medium 97 Program 98 Semiconductor memory

Claims

Acquire a captured image of the candidate sperm used for microinsemination,
A learning model that receives a captured image of a sperm photographed and outputs a prediction regarding success or failure of microinsemination using the sperm, inputs the captured image acquired,
A program that causes a computer to execute a process of outputting a prediction output from the learning model based on an input captured image.
The learning model is
Machines using teacher data recorded by associating captured images of sperm used for microinsemination in the past and whether or not they normally grow to each stage after microinsemination using the sperm The program according to claim 1, which is a learned model that has been learned.
The learning model includes an image encoder that extracts an image feature amount from an input captured image,
The program according to claim 1, wherein the prediction is output based on the image feature amount extracted by the image encoder.
The program according to claim 2 or 3, wherein the prediction is a prediction success probability of normally growing up to each stage after microinsemination.
Input a plurality of captured images of the sperm contained in the semen collected from one sperm donor who desires microinsemination to the learning model,
Obtaining each prediction output from the learning model,
The program according to any one of claims 2 to 4, which generates a sample distribution regarding success or failure of each stage based on the acquired prediction.
In a plurality of cases of microinsemination carried out in the past, a first distribution concerning whether or not to normally grow to each stage after microinsemination is acquired,
The program according to claim 5, wherein a second distribution is generated based on the first distribution and the sample distribution.
Obtaining a photographed image of sperm contained in semen collected from the sperm donor,
Obtain the prediction output by inputting the captured image to the learning model,
The program according to claim 6, which outputs an evaluation corresponding to the prediction in the second distribution.
The program according to claim 7, which outputs the reliability of the evaluation.
Acquire success or failure at each stage after microinsemination is performed,
The program according to any one of claims 5 to 8, which records additional data that associates a captured image of a sperm used for microinsemination with success or failure at each acquired stage.
The program according to claim 9, which updates the learning model using the additional data.
The program according to claim 10, wherein the updated learning model is distributed to a microinsemination performing institution.
Acquire a microscope image taken through a microscope,
Determine whether the acquired microscopic image contains a candidate sperm,
The program according to any one of claims 1 to 11, wherein the microscope image is used as the captured image when it is determined that a candidate sperm is included.
The program according to claim 12, wherein a portion of the microscope image containing the candidate sperm is selected and used as the captured image.
To the sex determination learning model that accepts the captured image of the sperm and outputs the prediction regarding the sex chromosome contained in the sperm, input the captured image acquired,
The program according to any one of claims 1 to 13, which outputs the prediction output from the sex determination learning model based on the input captured image.
Obtained the first information on the sex-linked genetic disease of the egg donor and the sperm donor,
Obtaining second information on the inherited form of the sex-linked genetic disease,
A child born by microinsemination develops a congenital genetic disease based on the incidence rate of the congenital genetic disease obtained by the acquired first information and the second information and the prediction acquired from the sex determination learning model. The program according to claim 14, which outputs a probability of performing.
Obtained the first information on the sex-linked genetic disease of the egg donor and the sperm donor,
Obtaining second information on the inherited form of the sex-linked genetic disease,
Based on the carrier rate of the sex-linked genetic disease obtained by the acquired first information and the second information and the prediction acquired from the sex determination learning model, the child born by microinsemination is associated with the sex-linked genetic disease. The program according to claim 14, which outputs the probability of having a factor.
An input layer to which the captured image of sperm is input,
An output layer that outputs a prediction regarding success or failure at each stage after performing microinsemination using the sperm,
The parameters were learned using the teacher data that recorded the captured images of sperm used for microinsemination in the past and the success or failure at each stage after performing microinsemination using the sperm. With layers,
When a photographed image of a candidate sperm used for microinsemination is input to the input layer, normally grows up to each stage when microinsemination is performed using the candidate sperm through the calculation by the intermediate layer A learning model that causes a computer to output possible prediction success probabilities from the output layer.
An image encoder provided between the input layer and the intermediate layer and extracting an image feature amount from a captured image,
The learning model according to claim 17, wherein the image feature amount extracted by the image encoder is output to the intermediate layer.
A captured image acquisition unit that acquires a captured image of a candidate sperm used for microinsemination,
A learning model that receives a captured image of sperm and outputs a prediction regarding success or failure of microinsemination using the sperm, an input unit that inputs the acquired captured image,
An information processing apparatus, comprising: an output unit that outputs the prediction output from the learning model based on an input captured image.
Acquire a captured image of the candidate sperm used for microinsemination,
A learning model that receives a captured image of a sperm and outputs a prediction regarding the success or failure of microinsemination using the sperm, inputs the captured image acquired,
An information processing method for causing a computer to execute a process of outputting a prediction output from the learning model based on an input captured image.
Take a picture of the candidate sperm used for microinsemination,
The captured image is input to a learning model that receives a captured image of a sperm and outputs a prediction regarding the success or failure of microinsemination using the sperm,
An information display method for displaying information based on a prediction regarding success or failure of microinsemination output from the learning model based on an input captured image.
Regarding multiple cases of microinsemination performed in the past, a photographed image of sperm used for microinsemination and whether or not it can grow normally up to each stage after microinsemination using the sperm Acquire the teacher data recorded by associating
When the photographed image is input, the photographed image in which the candidate sperm to be used for the microinsemination is photographed is input as an output indicating whether or not normal growth can be achieved up to each stage after microinsemination is performed. A method of manufacturing a learning model, which generates a learning model that outputs a prediction regarding whether or not normal growth can be achieved at each stage when microinsemination is performed using.