CN115641335B - Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator - Google Patents
Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator Download PDFInfo
- Publication number
- CN115641335B CN115641335B CN202211654883.9A CN202211654883A CN115641335B CN 115641335 B CN115641335 B CN 115641335B CN 202211654883 A CN202211654883 A CN 202211654883A CN 115641335 B CN115641335 B CN 115641335B
- Authority
- CN
- China
- Prior art keywords
- embryo
- image
- network model
- core
- division
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention provides an embryo abnormity multi-cascade intelligent comprehensive analysis system based on a time difference incubator, which comprises: the image preprocessing module, the standard database and the change monitoring module are as follows: a module for outputting cleavage characteristics and corresponding time points during embryo development, homogeneity identification: the abnormal factor used for outputting the embryogenesis unequal fragmentation, the multi-core recognition module: an abnormality factor and abnormality score module for outputting embryogenic multinuclei: and the method is used for calculating and outputting the score of the embryo abnormality in the embryo development process according to the division characteristics, abnormal factors such as the corresponding time points, the embryogenesis unevenness and the like, and abnormal factors of the embryogenesis multinuclear. The method can accurately identify the embryo division, the embryo uniformity, the multinuclear characteristics and the occurrence time of the embryo image of the time difference incubator, construct an embryo abnormity scoring model and help an embryologist to more quickly judge the embryo abnormity.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an embryo abnormity multi-cascade intelligent comprehensive analysis system based on a time difference incubator.
Background
Abnormal conditions in the process of embryo development can reduce the potential of the embryo to form a high-quality blastocyst, and abnormal embryo transplantation can directly influence the pregnancy rate of patients. When the embryo develops to the third day, an embryologist screens out the abnormal embryo by observing the embryo characteristics, and transplants the high-quality embryo to improve the pregnancy rate of the patient. The time difference incubator can provide a stable in-vitro culture environment for the embryo, and simultaneously has the function of periodically and continuously acquiring the whole process image of the embryo in-vitro development. The embryologist can accurately capture the abnormal state of the embryo and the abnormal occurrence time through the embryo image shot by the time difference incubator. And an embryologist obtains core parameters such as an embryo division time point, cell multinuclear property, cell uniformity and the like according to the embryo image shot by the time difference incubator and the morphological characteristic change of the embryo at different moments in the development process, and judges whether the embryo development is abnormal or not. In order to well solve the problem of difference in judgment of embryo abnormal results by different embryologists and the problem of timeliness of artificial judgment of the embryologists, an image processing technology can be used for rapidly and accurately helping the embryologists to finish the judgment work, however, from the viewpoint of the image processing technology, the following problems exist in the existing embryo abnormal recognition:
(1) The judgment of embryo abnormalities by embryologists requires extensive experience that relies on their long-term embryo observation to accumulate. For inexperienced embryologists, the abnormal conditions in the process of embryo development are extremely complex, the abnormal information of the embryos is difficult to accurately judge, and the screening of the quality of the embryos is influenced, so that the lack of a standard embryo data set with an abnormal label becomes one of the problems to be solved urgently at present;
(2) The image shot by the time difference incubator comprises an embryo area and a background area, the dynamics parameters of the embryo in the embryo development process are important indexes for judging the embryo activity degree, the embryo activity degree reflects the quality of the embryo, and the problem that how to accurately and intelligently obtain the dynamics parameters of the embryo by a computer calculation method becomes another problem to be solved due to the fact that the background area in the image shot by the time difference incubator has interference factors such as impurities, light sources and the like;
(3) In the process of embryo development, embryo division time points, embryo uniformity and multinucleate are the most important factors for judging whether an embryo is abnormal or not in the medical field. The evaluation of the embryo quality only depending on a single factor is inaccurate, at present, a multi-factor comprehensive embryo intelligent scoring model does not exist, and how to construct the multi-factor comprehensive embryo intelligent scoring model by using a computer method by means of expert experience knowledge has certain challenges.
Disclosure of Invention
Aiming at the problems, the invention provides an embryo abnormity multi-cascade intelligent comprehensive analysis system based on a time difference incubator, which is used for accurately identifying embryo division, embryo uniformity, multi-core characteristics and occurrence time of embryo images of the time difference incubator, constructing an embryo abnormity scoring model and helping an embryologist to quickly finish embryo abnormity judgment.
In order to achieve the purpose, the invention designs an embryo abnormity multi-cascade intelligent comprehensive analysis system based on a time difference incubator, which is characterized by comprising:
an image preprocessing module: the system is used for preprocessing the embryo image collected by the time difference incubator;
a standard database: for storing an embryo anomaly database comprising embryo images with embryo anomaly signatures;
a change monitoring module: the optical flow analysis is carried out on the preprocessed embryo image, and the division characteristics and the corresponding time points in the embryo development process are output, wherein the division characteristics refer to the first division, the second division and the third division of the embryo;
a homogeneity identification module: the system comprises a uniformity recognition network model, a standard database and a database, wherein the uniformity recognition network model is used for inputting preprocessed embryo images to the uniformity recognition network model for uniformity recognition and outputting abnormal factors of uneven embryo generation and equal fragmentation;
a multi-core identification module: the multi-core identification network model is used for inputting the preprocessed embryo image into the multi-core identification network model for multi-core identification and outputting abnormal factors of multi-core embryogenesis, and the multi-core identification network model is trained and verified by adopting a standard database;
an anomaly scoring module: and the method is used for calculating and outputting the grade of the embryo abnormity in the embryo development process according to the division characteristics, the corresponding time points, the abnormal factors of the embryogenesis, such as the uneven division and the abnormal factors of the embryogenesis multinuclear.
Further, the change monitoring module performs optical flow analysis based on a YOLO3 network model, the YOLO3 network model detects the ROI of the embryo in the embryo image according to the image shooting time sequence, the ROI of the embryo in the previous image is taken as a template to intercept the ROI of the embryo in the next image, optical flow values of two adjacent ROI images of the embryo are calculated, the image exceeding a set threshold value is an image of the embryo in a cleavage state, and the cleavage feature and the corresponding image shooting time, that is, the cleavage corresponding time point, are output.
Further, the standard database comprises a change monitoring analysis data set, a uniformity identification data set and a multi-core identification data set, wherein each data set comprises marked embryo images with embryo abnormal labels and is divided into a training set, a verification set and a test set.
Furthermore, the homogeneity identification network model performs parameter initial training on the network model by using a COCO data set to obtain an initialization model; on the basis of the initialized model, performing fine training on the network model by using the embryo image in the homogeneity recognition data set training set in the standard database, and adjusting the hyper-parameters of the network model by using the embryo image in the verification set to obtain the fine-trained network model; and (3) predicting the embryo data of the test set by using a precisely trained network model, and outputting abnormal factors or homogeneity marker bits of the embryo which is unevenly split.
Furthermore, after the embryo data is predicted by the homogeneity recognition network model through a precisely trained network model, the statistical prediction result is obtained, if the embryo images exceeding the set proportion all recognize embryo non-uniformity, the embryo is judged to be non-uniform, the time corresponding to the first image in which the non-uniformity is detected is taken as the time when the embryo appears non-uniformity, and abnormal factors of the embryo which are unevenly split are output; otherwise, the output embryo is a marker bit of homogeneity.
Furthermore, the uniformity identification network model adopts a ResNet network as a basic network, and the network mainly comprises 100 convolutional layers and 34 pooling layers and comprises an encoder and a decoder;
the encoder performs example segmentation on cells in the image by adopting a DeepLabV3+ network framework, replaces the last two convolution blocks of the DeepLabV3 with cavity convolution with the step length of 2 and the step length of 4, and blends image features by using a spatial pyramid structure to obtain features with different scales;
and the decoder performs up-sampling on the feature map generated by the encoder by 4 times of bilinear difference, fuses the feature map with the low-level features of the image, and finally completes feature decoding by 4 times of bilinear interpolation.
Furthermore, the multi-core recognition network model utilizes ImageNet data sets to carry out parameter initial training on the network model, and obtains an initialization model; on the basis of the initialized model, carrying out fine training on the network model by using the embryo image in the multi-core recognition data set training set in the standard database, and adjusting the hyper-parameters of the network model by using the embryo image in the verification set to obtain the fine-trained network model; and predicting the embryo data of the test set by using the precisely trained network model, and outputting abnormal factors or non-multi-core marker bits of the embryogenesis multi-core.
Furthermore, after the multi-core identification network model predicts embryo data by using a precisely trained network model, counting the prediction result, if the embryo images exceeding a set proportion all identify the multi-core of the embryo images, judging that the embryo is multi-core, and outputting an abnormal factor of the multi-core occurrence of the embryo by taking the time corresponding to the first detected multi-core image as the time for the multi-core occurrence of the embryo; otherwise, the marker bit of the embryo which is not multinucleate is output.
Furthermore, the multi-core identification network model adopts a Swin transform network as a basic network, the Swin transform network is divided into 4 different layers, each layer enlarges the receptive field of the image layer by layer, and the resolution of the input characteristic diagram is reduced.
Further, the abnormal scoring module calculates the score of the embryo abnormal in the embryo development process according to the formula:
in the formula (I), the compound is shown in the specification,t 1 ,t 2 ,t 3 respectively represents the cleavage time of the first cleavage, the second cleavage and the third cleavage of the embryogenesis,α 1 ,α 2 ,α 3 respectively representing the parameters of the division time of the first division, the second division and the third division of the embryogenesis, beta represents an abnormal factor of the embryogenic multinuclear, 1 (-) is an indication function, theta represents a marker bit of the occurrence of the multinuclear,tindicates the time at which the embryo is developing,γabnormal factors indicating the uneven cleavage of embryogenesis,τthe mark positions of the embryonic development unevenness and the like are shown, and F shows the scoring result of the embryonic abnormality in the embryonic development process.
The invention has the beneficial effects that:
1. according to the invention, a deep learning technology is introduced into the abnormal detection of the embryo, the YOLO3 and the template are creatively matched and fused in the embryo change monitoring process, so that the impurity interference outside the embryo area in the image is eliminated, and the Lucas-Kanade optical flow is used for quantifying the change of the embryo for the first time according to the optical flow characteristic, so that a more convenient and more intuitive data support is provided for an embryologist to monitor the abnormal change in the embryo development process.
2. The method accurately identifies the embryo division, the embryo uniformity, the multinuclear characteristics and the occurrence time of the embryo image of the time difference incubator, constructs an embryo abnormity scoring model and helps an embryologist to quickly finish the judgment of embryo abnormity.
3. The invention applies an image processing technology, eliminates the interference of light sources, impurities and the like in a background area according to the embryo change in the time difference shot image, accurately captures the embryo area in the time difference shot image and monitors the change in the embryo development process.
4. The invention comprehensively considers a plurality of embryo abnormal factors in the embryo development process, evaluates the abnormality of the embryo from two aspects of embryo dynamics and embryo morphology, and provides an abnormal scoring model by combining the consensus of embryologists and the consensus of embryo scoring at home and abroad, which is used for quantifying the embryo abnormality and assisting the embryologists to quickly screen out abnormal embryos.
5. The invention constructs a standard embryo abnormity database, provides a calculation model of embryo abnormity, realizes automatic and intelligent grading of embryo abnormity by combining a plurality of embryo abnormity phenomena, and provides effective data support for embryologists to evaluate embryo abnormity.
Drawings
FIG. 1 is a block diagram of the embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator.
FIG. 2 is a schematic view of the optical flow analysis process of the change monitoring module.
FIG. 3 (a) is an image of an embryo output from the embryo positioning unit.
FIG. 3 (b) is the ROI area where embryos were detected according to YOLO 3.
FIG. 3 (c) is a diagram of embryo change monitoring obtained by analyzing the embryo ROI by the change monitoring module, wherein t1, t2 and t3 are marked on the diagram to respectively indicate the time points of the first, second and third divisions of the embryo.
FIG. 4 is a flowchart illustrating an identification process of the uniformity identification module.
FIG. 5 is a schematic diagram of a homogeneity identification network model structure.
FIG. 6 is a graph showing the results of embryo uniformity analysis.
Fig. 7 is a schematic flow chart of the identification process of the multi-core identification module.
FIG. 8 is a diagram illustrating a multi-core recognition network model structure.
Fig. 9 is a diagram illustrating a multi-core recognition result.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments.
As shown in figure 1, the embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator comprises an image preprocessing module, a standard database, a change monitoring module, a uniformity identification module, a multi-core identification module and an abnormity scoring module, wherein:
an image preprocessing module: the system is used for preprocessing the embryo image collected by the time difference incubator; the time difference incubator shoots embryos at certain time intervals in the embryo development process, and in the actual shooting process, the time difference incubator has external interference to influence the brightness, the definition and the like of images;
a standard database: the embryo abnormity database comprises embryo images with embryo abnormity labels;
a change monitoring module: the embryo image sending to the change monitoring module is shot in a time difference incubator according to the time sequence of the same culture dish, the whole time period is D1-D3 (the first day-the third day), and three divisions can occur in the time period, so that the change monitoring module sequentially outputs three division characteristics and corresponding time points according to the embryo image of the same culture dish;
a homogeneity identification module: the system comprises a uniformity recognition network model, a standard database and a database, wherein the uniformity recognition network model is used for inputting preprocessed embryo images to the uniformity recognition network model for uniformity recognition and outputting abnormal factors of uneven embryo generation and equal fragmentation;
a multi-core identification module: the multi-core identification network model is used for inputting the preprocessed embryo image into the multi-core identification network model for multi-core identification and outputting abnormal factors of multi-core embryogenesis, and the multi-core identification network model is trained and verified by adopting a standard database;
an anomaly scoring module: and the method is used for calculating and outputting the grade of the embryo abnormity in the embryo development process according to the division characteristics, the corresponding time points, the abnormal factors of the embryogenesis, such as the uneven division and the abnormal factors of the embryogenesis multinuclear.
In order to construct a standard database of embryo abnormalities, the invention collects D1-D3 (from the first day to the third day in a time difference incubator) from a plurality of centers, contains 15000 embryos with abnormalities, and organizes a plurality of embryologists to complete data set annotation. In order to ensure the diversity of data sets, the invention adopts a random interval sampling mode in the process of collecting the data sets of embryo uniformity and multiple nuclei, and collects 50000 embryo images with abnormal uniformity of embryo cells and 40000 embryo images containing multiple nuclei. And (3) inviting a plurality of embryo experts to label each image, and taking the image with the largest vote number as a final label of the image. In order to improve the diversity of the positions and the shapes of the embryos, the invention adopts the modes of overturning, rotating and the like to expand the image data set of the embryo uniformity and the embryo multi-core aiming at the images in the training set so as to enhance the generalization of the model to the embryo recognition.
The change monitoring module carries out optical flow analysis based on a YOLO3 network model, the analysis flow is shown in figure 2, the YOLO3 network model detects an ROI (region of interest) of an embryo in an embryo image according to the image shooting time sequence, the ROI of the embryo in the image is taken as a template to intercept the ROI of the embryo in the next image, the optical flow values of two adjacent ROI images of the embryo are calculated, the image exceeding a set threshold value is an image of the embryo in a splitting state, and the splitting characteristic and the corresponding image shooting time, namely the splitting corresponding time point, are output.
The change monitoring module comprises an embryo positioning unit, an ROI extraction unit, an optical flow value calculation unit, a division time point calculation unit and a result output unit;
inputting a first image in an embryo development sequence into a trained YOLO3 model by an embryo positioning unit, outputting whether an embryo exists in the image, if so, outputting the position of the embryo in the image to an ROI extraction unit and simultaneously outputting the position of the embryo in the image, as shown in fig. 3 (a), otherwise, outputting the position of the embryo which does not exist to a result output unit;
extracting the ROI from the embryo position obtained by the embryo positioning unit by the ROI unit, as shown in FIG. 3 (b), intercepting the ROI of the embryo in the image, calculating an edge image of the ROI and an edge image of the embryo at the next moment by using a Canny edge operator, calculating the position corresponding to the embryo at the next moment by using the edge image of the ROI as a template, further obtaining the ROI of the embryo at the next moment, outputting the ROI to an optical flow value calculating unit by using the ROI as a new template, and repeating the steps to extract the ROI of the embryo in all the images;
the optical flow value calculating unit is used for sequentially arranging the extracted ROI areas according to embryo development time, calculating the optical flow values of two adjacent frames by adopting a Lucas-Kanade method, and monitoring the change of the embryo through the change of the optical flow values;
the computing unit of the division time point draws an optical flow curve reflecting the change of the optical flow value according to the time sequence, and a reference value is set by combining the comparative analysis of a large amount of embryo division data and the change of the optical flow and the development rule of the embryo, in the embodiment, 17.8 are taken, and the peak value exceeding the reference value is the division time point, as shown in fig. 3 (c);
the result output unit outputs the cleavage time of the first cleavage, the second cleavage and the third cleavage of the embryo based on the cleavage time point of the embryot 1 ,t 2 ,t 3 And cleavage time parameters of embryogenesis for first, second, and third cleavages based on a priori knowledgeα 1 ,α 2 ,α 3 The three parameters are obtained by statistics of the division time of normal embryo development of a plurality of reproductive centers, and the values are 26,44,68 respectively.
The homogeneity identification model in the homogeneity identification module provided by the invention adopts an end-to-end segmentation network, an embryo image in a non-division period is input into the trained embryo homogeneity identification model, and a segmentation area of each cell in the image is output through calculation of a DeepLabV3+ model. And counting the pixel value of each segmented cell, if the difference between the pixel value of the largest cell and the pixel value of the smallest cell exceeds 10%, judging that the embryo is non-uniform, otherwise, judging that the embryo is uniform.
The identification process of the uniformity identification module is shown in fig. 4, the uniformity identification network model performs parameter initial training on the network model by using the COCO data set to obtain an initialization model; to initialize the moduleBased on the type, performing fine training on the network model by using the embryo image in the uniformity recognition data set training set in the standard database, and adjusting the hyper-parameters of the network model by using the embryo image in the verification set to obtain the fine-trained network model; predicting the embryo data of the test set by using a precisely trained network model, completing statistics of the prediction result of the test set, judging that the embryo is non-uniform if the embryo non-uniformity is recognized by embryo images exceeding a set proportion (such as 50 percent, 60 percent and the like), and outputting abnormal factors of the embryo such as the uneven and equal splitting of the embryo by taking the time corresponding to the first image in which the non-uniformity is detected as the time when the embryo is unevenγ(ii) a Otherwise, the output embryo is the marker bit 0 of the uniformity.
In the present embodiment, the uniformity recognition network model mainly uses the ResNet network as the basic network, and as shown in fig. 5, the network mainly includes 100 convolutional layers and 34 pooling layers, including an encoder and a decoder.
The encoder performs example segmentation on cells in the image by adopting a DeepLabV3+ network framework, replaces the last two convolution blocks of the DeepLabV3+ by the cavity convolution with the step length of 2 and the step length of 4, and blends the image features into the space pyramid structure to obtain the features with different scales. The cavity convolution calculates the resolution of the features by controlling a deep neural network, adjusts the visual field of a convolution layer, captures multi-scale information of an embryo image, and further establishes standard convolution operation, wherein the specific calculation method is shown as a formula (1):
y[i]=Σkx[i+r*k]w[k] （1）
where y represents the characteristic of the output,iindicating the position in the output feature y,xa feature representing the input is presented to the user,rdenotes the step size of the hole convolution, w denotes the convolution kernel,krepresenting the index in the convolution kernel.
And the decoder performs up-sampling on the feature map generated by the encoder through 4 times of bilinear difference values, fuses the feature map with the low-layer features of the image, and finally completes decoding of the features by adopting 4 times of bilinear interpolation.
The homogeneity identification network model provided by the embodiment can realize the prediction of the homogeneity of the embryo in the cleavage stage. The time difference incubator uploads the shot embryo images in the cleavage stage to the server, the model can give a prediction result within 0.1s, and the result of embryo uniformity analysis is shown in figure 6, so that an embryologist can quickly and accurately complete the evaluation of embryo uniformity.
The multi-core recognition module designed by the invention inputs the embryo image in the non-fission stage into the trained embryo multi-core recognition model, outputs whether the multi-core exists in the image or not through the calculation of the Swin transform model, and marks the multi-core in the image if the multi-core exists in the image.
The identification process of the multi-core identification module is shown in fig. 7, the multi-core identification network model performs parameter initial training on the network model by using the ImageNet data set to obtain an initialization model; on the basis of the initialization model, performing fine training on the network model by using the embryo image in the multi-core recognition data set training set in the standard database, and adjusting the hyper-parameters of the network model by using the embryo image in the verification set to obtain the fine-trained network model; and predicting the embryo data of the test set by using a precisely trained network model, counting the prediction result, judging that the embryo is multi-core if the multi-core of the embryo image is identified by the embryo images with the ratio (for example, 50% -99%) exceeding a set ratio, and outputting an abnormal factor beta or a non-multi-core flag bit of the embryo with uneven fragmentation as 0 by taking the time corresponding to the first detected multi-core image as the time for the embryo to have the multi-core.
In this embodiment, the Swin Transformer network adopted by the multi-core recognition network model is divided into 4 different "layers" as shown in fig. 8, and each "layer" enlarges the receptive field of the image layer by layer, and reduces the resolution of the input feature map.
Before the feature map is transmitted into the first "layer", the image needs to be transformed into different blocks through one convolution layer, and the obtained blocks are flattened to obtain embedded vectors of the image.
When each layer processes the embedded vector, a moving window self-attention mechanism is mainly adopted to calculate the feature map so as to ensure that information interaction in the embryo image can be better completed and representative multi-core feature information is extracted.
And when the characteristics of the first three layers are output, the calculated characteristic blocks are subjected to down sampling, the resolution of the characteristic diagram is reduced, the number of channels is adjusted, and then the hierarchical characteristic vector is formed.
The multi-kernel recognition module extracts the feature vectors of the embryo images through a SwinTransformer, and then completes multi-kernel coordinate regression and classification prediction in the images by adopting a linear classifier.
The method provides a multi-core identification module which can predict the multi-core of the embryo with the number of 2~4 in the cleavage stage, the identification result of the multi-core of the embryo is shown in figure 9, the multi-core of the cell is detected on the embryo image shot by time difference through the model of the invention, and the embryologist is assisted to quickly and accurately complete the evaluation of the multi-core of the cell.
The abnormal scoring module calculates the scoring formula of the embryo abnormal in the embryo development process as follows:
in the formula (2), the reaction mixture is,t 1 ,t 2 ,t 3 respectively represents the division time of the first division, the second division and the third division of the embryogenesis,α 1 ,α 2 ,α 3 the parameters of the division time of the first division, the second division and the third division of the embryogenesis are respectively represented, beta represents abnormal factors of the embryogenic multinuclear, 1 (-) is an indicator function, theta represents a marker bit 0 or 1 of the occurrence of the multinuclear,tindicating the time of embryogenic development, which is displayed on the image of the corresponding frame,γabnormal factors indicating the uneven cleavage of embryogenesis,τthe marker 0 or 1,F, which indicates the unequal cleavage of embryogenesis, indicates the scoring result of abnormal embryos during the development of embryos. In the patent, the values of beta and gamma are 0.8,0.5 according to expert experience。
And the abnormality scoring module outputs the score of the abnormal state of the embryo by using a formula (2), wherein the lower the score is, the higher the potential of the embryo developing into a high-quality blastocyst is, and otherwise, the lower the potential of the embryo developing into the high-quality blastocyst is.
According to the consensus of embryologists and clinical data, the cleavage and development speed of the embryo are key factors for the normal development of the embryo, the too fast or too slow cleavage can reduce the development potential of the embryo, cause the abnormal development of the embryo, and the earlier the abnormal cleavage time of the embryo occurs, the higher the possibility of the abnormal embryo is. Meanwhile, heterogeneous embryos and multi-nucleated embryos have different degrees of influence on implantation and pregnancy of the embryos.
The formula (2) combines three abnormal points of the division time, the embryo multi-core and the embryo unequal division of the embryo in a weighting mode to calculate the abnormal score of the embryo. In the calculation term, 1 (.) is used to determine whether the embryo is abnormal, if the value is 0, it indicates that no abnormality occurs, otherwise, it indicates that an abnormality occurs. According to the time of division in the calculation itemt i (i =1,2,3) and embryo development timetThe result of scoring the embryo abnormality (set according to the prior knowledge) shows that the earlier the abnormality occurs, the higher the score of the embryo abnormality is, and further, the lower the potential of the embryo to develop into a high-quality blastocyst. It is found by analysis that the smaller the embryo abnormality score F, the lower the possibility of abnormality occurrence of the embryo and the higher the possibility of development of a good quality embryo.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Details not described in this specification are within the skill of the art that are well known to those skilled in the art.
Claims (9)
1. The utility model provides an embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator which characterized in that: the system comprises:
an image preprocessing module: the system is used for preprocessing the embryo image collected by the time difference incubator;
a standard database: the embryo abnormity database comprises embryo images with embryo abnormity labels;
a change monitoring module: the optical flow analysis is carried out on the preprocessed embryo image, and the division characteristics and the corresponding time points in the embryo development process are output, wherein the division characteristics refer to the first division, the second division and the third division of the embryo;
a homogeneity identification module: the system comprises a uniformity recognition network model, a standard database and a database, wherein the uniformity recognition network model is used for inputting preprocessed embryo images to the uniformity recognition network model for uniformity recognition and outputting abnormal factors of uneven embryo generation and equal fragmentation;
a multi-core identification module: the multi-core identification network model is used for inputting the preprocessed embryo image into the multi-core identification network model for multi-core identification and outputting abnormal factors of multi-core embryogenesis, and the multi-core identification network model is trained and verified by adopting a standard database;
an anomaly scoring module: the system is used for calculating and outputting the grade of embryo abnormity in the embryo development process according to the division characteristics, the corresponding time points, the abnormal factors of the embryogenesis, such as the uneven division and the abnormal factors of the embryogenesis, and the abnormal factors of the embryogenesis multinuclear;
the change monitoring module carries out optical flow analysis based on a YOLO3 network model, the YOLO3 network model detects an ROI (region of interest) of an embryo in an embryo image according to the image shooting time sequence, the ROI of the embryo in the previous image is taken as a template to intercept the ROI of the embryo in the next image, optical flow values of two adjacent ROI images of the embryo are calculated, the image exceeding a set threshold value is an image of the embryo in a splitting state, and the splitting characteristic and the corresponding image shooting time, namely the splitting corresponding time point, are output.
2. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 1, wherein: the standard database comprises a change monitoring analysis data set, a uniformity identification data set and a multi-core identification data set, wherein each data set comprises an embryo image which is marked and provided with an embryo abnormal label, and is divided into a training set, a verification set and a test set.
3. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 1, wherein: the uniformity recognition network model performs parameter primary training on the network model by using a COCO data set to obtain an initialization model; on the basis of the initialized model, performing fine training on the network model by using the embryo image in the homogeneity recognition data set training set in the standard database, and adjusting the hyper-parameters of the network model by using the embryo image in the verification set to obtain the fine-trained network model; and (3) predicting the embryo data of the test set by using a precisely trained network model, and outputting abnormal factors or homogeneity marker bits of the embryo which is unevenly split.
4. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 1, wherein: after the uniformity recognition network model predicts embryo data by using a precisely trained network model, counting a prediction result, if embryo non-uniformity is recognized by embryo images exceeding a set proportion, judging that the embryo is non-uniform, and outputting abnormal factors of uneven embryo fragmentation by taking the time corresponding to the first image with the detected non-uniformity as the time when the embryo is non-uniform; otherwise, the output embryo is a marker bit of homogeneity.
5. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the culture box with time difference as claimed in claim 3, wherein: the uniformity identification network model adopts a ResNet network as a basic network, and the network mainly comprises 100 convolutional layers and 34 pooling layers and comprises an encoder and a decoder;
the encoder performs example segmentation on cells in the image by adopting a DeepLabV3+ network framework, replaces the last two convolution blocks of the DeepLabV3 with the cavity convolution with the step length of 2 and the step length of 4, and blends image features into a space pyramid structure to obtain features with different scales;
the decoder performs up-sampling on the feature map generated by the encoder through 4 times of bilinear difference values, fuses the feature map with the low-layer features of the image, and finally completes feature decoding by adopting 4 times of bilinear interpolation.
6. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 1, wherein: the multi-core recognition network model utilizes ImageNet data sets to carry out parameter initial training on the network model, and obtains an initialization model; on the basis of the initialization model, performing fine training on the network model by using the embryo image in the multi-core recognition data set training set in the standard database, and adjusting the hyper-parameters of the network model by using the embryo image in the verification set to obtain the fine-trained network model; and predicting the embryo data of the test set by using the precisely trained network model, and outputting abnormal factors or non-multi-core marker bits of the embryogenesis multi-core.
7. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 6, wherein: after the multi-core identification network model predicts embryo data by using a precisely trained network model, counting a prediction result, if the multi-core of the embryo image is identified by the embryo image exceeding a set proportion, judging that the embryo is multi-core, taking the time corresponding to the detected first multi-core image as the time when the embryo is multi-core, and outputting abnormal factors of the embryo which is unevenly split; otherwise, the marker bit of the embryo which is not multinucleate is output.
8. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 7, wherein: the multi-core identification network model adopts a Swin transform network as a basic network and is divided into 4 different layers, the receptive field of an image is enlarged layer by layer in each layer, and meanwhile, the resolution of an input characteristic diagram is reduced.
9. The embryo abnormity multi-cascade intelligent comprehensive analysis system based on the time difference incubator as claimed in claim 1, wherein: the abnormal scoring module calculates the scoring formula of the embryo abnormal in the embryo development process as follows:
in the formula, t 1 ,t 2 ,t 3 Respectively represents the cleavage time of the first, second and third cleavage of the embryo, alpha 1 ,α 2 ,α 3 The division time parameters respectively represent the first division, the second division and the third division of the embryogenesis, beta represents abnormal factors of embryogenic multinuclear, 1 (-) is an indication function, theta represents a mark position of multinuclear generation, t represents the time of embryogenic development, gamma represents abnormal factors of embryogenic unequal division, tau represents a mark position of embryogenic unequal division, and F represents a grading result of embryo abnormality in the process of embryogenic development.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211654883.9A CN115641335B (en) | 2022-12-22 | 2022-12-22 | Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211654883.9A CN115641335B (en) | 2022-12-22 | 2022-12-22 | Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115641335A CN115641335A (en) | 2023-01-24 |
| CN115641335B true CN115641335B (en) | 2023-03-17 |
Family
ID=84948253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211654883.9A Active CN115641335B (en) | 2022-12-22 | 2022-12-22 | Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115641335B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116051560B (en) * | 2023-03-31 | 2023-06-20 | 武汉互创联合科技有限公司 | Embryo dynamics intelligent prediction system based on embryo multidimensional information fusion |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013004239A1 (en) * | 2011-07-02 | 2013-01-10 | Unisense Fertilitech A/S | Adaptive embryo selection criteria optimized through iterative customization and collaboration |
| CN112990319A (en) * | 2021-03-18 | 2021-06-18 | 武汉互创联合科技有限公司 | Chromosome euploidy prediction system, method, terminal and medium based on deep learning |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201913565A (en) * | 2017-08-29 | 2019-04-01 | 宣捷生物科技股份有限公司 | Evaluation method for embryo images and system thereof |
| WO2020157761A1 (en) * | 2019-01-31 | 2020-08-06 | Amnon Buxboim | Automated evaluation of embryo implantation potential |
| CN110363218B (en) * | 2019-06-06 | 2023-07-11 | 张孝东 | Noninvasive embryo assessment method and device |
| CN111785375B (en) * | 2020-06-18 | 2023-03-24 | 武汉互创联合科技有限公司 | Embryo division process analysis and pregnancy rate intelligent prediction method and system |
-
2022
- 2022-12-22 CN CN202211654883.9A patent/CN115641335B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013004239A1 (en) * | 2011-07-02 | 2013-01-10 | Unisense Fertilitech A/S | Adaptive embryo selection criteria optimized through iterative customization and collaboration |
| CN112990319A (en) * | 2021-03-18 | 2021-06-18 | 武汉互创联合科技有限公司 | Chromosome euploidy prediction system, method, terminal and medium based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| Mika Sato等.Three-dimensional imaging of the vasculature in chicken embryo by combination of ultrasonic and photoacoustic imaging.2014,第1532-1533页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115641335A (en) | 2023-01-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112288706B (en) | Automatic chromosome karyotype analysis and abnormality detection method | |
| CN111931811B (en) | Calculation method based on super-pixel image similarity | |
| CN113392775B (en) | Sugarcane seedling automatic identification and counting method based on deep neural network | |
| CN110021425B (en) | Comparison detector, construction method thereof and cervical cancer cell detection method | |
| CN110415230B (en) | CT slice image semantic segmentation system and method based on deep learning | |
| TWI684997B (en) | Establishing method of bone age assessment and height prediction model, bone age assessment and height prediction system, and bone age assessment and height prediction method | |
| CN108711148B (en) | Tire defect intelligent detection method based on deep learning | |
| CN111540006B (en) | Plant stomata intelligent detection and identification method and system based on deep migration learning | |
| CN110705639B (en) | Medical sperm image recognition system based on deep learning | |
| CN111785375B (en) | Embryo division process analysis and pregnancy rate intelligent prediction method and system | |
| CN102855485B (en) | The automatic testing method of one grow wheat heading | |
| CN112365471B (en) | Cervical cancer cell intelligent detection method based on deep learning | |
| CN109670489B (en) | Weak supervision type early senile macular degeneration classification method based on multi-instance learning | |
| CN115641335B (en) | Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator | |
| CN108305253A (en) | A kind of pathology full slice diagnostic method based on more multiplying power deep learnings | |
| CN112949517B (en) | Plant stomata density and opening degree identification method and system based on deep migration learning | |
| CN115641364B (en) | Embryo division period intelligent prediction system and method based on embryo dynamics parameters | |
| CN115909006B (en) | Mammary tissue image classification method and system based on convolution transducer | |
| CN116051560B (en) | Embryo dynamics intelligent prediction system based on embryo multidimensional information fusion | |
| CN110728666B (en) | Typing method and system for chronic nasosinusitis based on digital pathological slide | |
| CN110766665A (en) | Tongue picture data analysis method based on strong supervision algorithm and deep learning network | |
| CN111968147B (en) | Breast cancer pathological image comprehensive analysis system based on key point detection | |
| CN113241154A (en) | Artificial intelligent blood smear cell labeling system and method | |
| CN112614570A (en) | Sample set labeling method, pathological image classification method and classification model construction method and device | |
| WO2023240820A1 (en) | Chromosome karyotype analysis module |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |