CN113706481A

CN113706481A - Sperm quality detection method, sperm quality detection device, computer equipment and storage medium

Info

Publication number: CN113706481A
Application number: CN202110929246.7A
Authority: CN
Inventors: 赵荔君; 黄迪锋; 沈艺; 尹凯; 王建峰; 梁波
Original assignee: Suzhou Beikang Intelligent Manufacturing Co ltd
Current assignee: Suzhou Beikang Intelligent Manufacturing Co ltd
Priority date: 2021-08-13
Filing date: 2021-08-13
Publication date: 2021-11-26

Abstract

The present application relates to a sperm quality detection method, apparatus, computer device and storage medium. The method comprises the following steps: acquiring a plurality of sperm images to be detected; detecting the sperm image to be detected to obtain a sperm morphology detection result of the sperm in the sperm image to be detected; detecting each sperm image to be detected to obtain a sperm number detection result and a sperm position detection result corresponding to each sperm image to be detected; determining the sperm concentration according to the sperm number detection results of a plurality of sperm images to be detected; and generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected. By adopting the method, the sperm morphology detection result and the semen conventional detection result can be quickly obtained, so that the work flow of sperm quality detection is greatly simplified, and the sperm quality detection efficiency is improved.

Description

Sperm quality detection method, sperm quality detection device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a sperm quality detection method, apparatus, computer device, and storage medium.

Background

Semen analysis is an important test for judging male fertility, and the two most commonly used analyses are semen routine analysis and sperm morphology analysis. The routine analysis of semen comprises the detection of sperm concentration and motility, and the analysis of sperm morphology is an important method for detecting sperm teratogenesis. Normal sperm motility, concentration and morphological indicators are important conditions for natural conception or success of test-tube infants.

In the conventional technology, the conventional Analysis of semen can be realized by adopting technical means such as manual counting, Computer Assisted Sperm Analysis (CASA), full-automatic Sperm analyzer and the like. The sperm morphology analysis can be realized by adopting the technical means of artificial microscopic examination, full-automatic sperm morphology analyzer and the like. However, since the conventional techniques usually require separate detection for routine semen analysis and sperm morphology analysis, the detection efficiency is low.

Disclosure of Invention

In view of the above, there is a need to provide a sperm quality detection method, apparatus, computer device and storage medium capable of simplifying sperm quality detection process and improving sperm quality detection efficiency and accuracy.

In a first aspect, embodiments of the present application provide a sperm quality detection method, including:

acquiring a plurality of sperm images to be detected;

detecting the sperm image to be detected to obtain a sperm morphology detection result of the sperm in the sperm image to be detected;

detecting each sperm image to be detected to obtain a sperm number detection result and a sperm position detection result corresponding to each sperm image to be detected;

determining the sperm concentration according to the sperm number detection results of the plurality of sperm images to be detected;

and generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected.

In one embodiment, the detecting the sperm image to be detected to obtain a sperm morphology detection result of the sperm in the sperm image to be detected includes:

detecting the sperm image to be detected through an image segmentation model, and acquiring a sperm segmentation image of the sperm when the sperm is detected to be present in the sperm image to be detected;

and classifying and identifying the sperm segmentation images through a sperm morphology identification model to obtain a sperm morphology identification result.

In one embodiment, the generation manner of the image segmentation model includes:

acquiring a sperm image sample and a sperm segmentation image label corresponding to the sperm image sample;

cutting the sperm image sample according to a preset image cutting mode to obtain a plurality of training image samples;

cutting the sperm segmentation image labels according to the image cutting mode to obtain training labels corresponding to the training image samples;

and training an initial image segmentation model by using the training image sample and the training label to obtain the image segmentation model.

In one embodiment, the sperm morphology recognition model comprises a plurality of convolution parts connected in sequence, wherein each convolution part comprises a convolution embedded block and a plurality of convolution transformation blocks;

the classifying and identifying the sperm segmentation image through the sperm morphology identification model to obtain the sperm morphology identification result comprises the following steps:

sequentially carrying out feature extraction on sperm input features through convolution embedded blocks in each convolution part, wherein for a first convolution part, the sperm input features are sperm segmentation images, and for each other convolution part except the first convolution part, the sperm input features are sperm output features output by the last convolution part;

performing global information learning and local information learning on the sperm input features after feature extraction through a convolution transformation block in each convolution part to obtain sperm output features;

and generating the sperm morphology recognition result according to the sperm output characteristics output by the last convolution part.

In one embodiment, the convolution transformation block comprises a convolution mapping layer, a multi-head attention layer, a multi-layer perceptron layer, a residual connecting layer and a normalization layer;

the global information learning and the local information learning are carried out on the sperm input features after the features are extracted through the convolution transformation blocks in each convolution part, so that the sperm output features are obtained, and the method comprises the following steps:

and performing global information learning and local information learning on the sperm input features after feature extraction through the convolution mapping layer, the multi-head attention layer, the multi-layer perceptron layer, the residual error connecting layer and the normalization layer to obtain sperm output features.

In one embodiment, the detecting each sperm image to be detected to obtain the sperm number detection result and the sperm position detection result corresponding to each sperm image to be detected includes:

performing target detection on each sperm image to be detected, and when the sperm head exists in the sperm image to be detected, detecting the target

Acquiring the number of heads of the sperm heads, and taking the number of the heads as the sperm number detection result;

and acquiring head position information of the sperm head, and taking the head position information as a sperm position detection result.

In one embodiment, each sperm image to be detected is an image cut from a corresponding original sperm image, and the acquisition fields of different original sperm images are different;

the determining the sperm concentration according to the sperm number detection results of the plurality of sperm images to be detected comprises the following steps:

acquiring the sum of the number of the heads of the sperm images to be detected as the total number of the sperms under the plurality of acquisition visual fields;

determining the sperm concentration from the total number of sperm.

In one embodiment, a plurality of the sperm images to be detected are continuous multi-frame images obtained from a sperm video to be detected;

generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected, wherein the sperm motility detection result comprises:

when a plurality of sperm heads exist in each frame of image, acquiring first head position information of each sperm head in a current frame of image and second head position information of each sperm head in a next frame of image of the current frame of image;

generating a distance between first head position information and each second head position information of each sperm head in the current frame image;

determining target head position information from the second head position information according to the distance, wherein the target head position information is used as the head position information of each sperm head in the current frame image in the next frame image;

and generating the sperm motility detection result according to the head position information of each sperm head in each frame of image.

In a second aspect, embodiments of the present application provide a sperm quality detection apparatus, the apparatus comprising:

the image acquisition module is used for acquiring a plurality of sperm images to be detected;

the morphology detection module is used for detecting the sperm image to be detected to obtain a sperm morphology detection result of the sperm in the sperm image to be detected;

the target detection module is used for detecting each sperm image to be detected to obtain a sperm number detection result and a sperm position detection result corresponding to each sperm image to be detected;

the concentration determining module is used for determining the concentration of the sperms according to the sperm number detection results of the plurality of sperm images to be detected;

and the motility detection module is used for generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected.

In a third aspect, an embodiment of the present application provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the sperm quality detection method described in any one of the embodiments of the first aspect when executing the computer program.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the sperm quality detection method described in any one of the embodiments of the first aspect.

The sperm quality detection method, the sperm quality detection device, the computer equipment and the storage medium detect the acquired multiple sperm images to be detected based on the deep learning theory to obtain the sperm morphology detection result, the sperm number detection result and the sperm position detection result of the sperm. Then, determining the sperm concentration according to the sperm number detection results of a plurality of sperm images to be detected; and generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected. Based on the deep learning theory, the sperm morphology detection result and the semen conventional detection result (including sperm concentration and sperm motility) can be quickly obtained on the basis of collecting a plurality of sperm images to be detected, so that the work flow of sperm quality detection is greatly simplified, the sperm quality detection efficiency is improved, and the full-automatic analysis of the sperm quality detection is realized. In addition, the deep learning model with enough detection capability is adopted to detect the sperm image to be detected, and the accuracy of sperm quality detection can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the specification, and other drawings can be obtained by those skilled in the art without inventive labor.

FIG. 1 is a schematic flow diagram of a sperm quality detection process in one embodiment;

FIG. 2 is a schematic flow chart illustrating the steps of obtaining sperm cell morphology identification results in one embodiment;

FIG. 3 is a flowchart illustrating a training process of an image segmentation model according to one embodiment;

FIG. 4 is a schematic diagram of the structure of a sperm cell morphology recognition model in one embodiment;

FIG. 5 is a schematic diagram of a process for feature extraction of a segmented image of sperm, in accordance with an embodiment;

FIG. 6 is a schematic diagram of the structure of the convolution insert in the sperm morphology recognition model in one embodiment;

FIG. 7 is a diagram illustrating the structure of a convolution transform block in a sperm morphology recognition model according to an embodiment;

FIG. 8 is a schematic flow chart illustrating generation of sperm motility test results in one embodiment;

FIG. 9 is a schematic flow chart of a sperm cell quality detection process according to another embodiment;

FIG. 10 is a block diagram showing the structure of a sperm cell mass measuring apparatus according to an embodiment;

FIG. 11 is a diagram illustrating an internal structure of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The sperm quality detection method provided by the application can be applied to computer equipment such as a terminal and a server or a system comprising the terminal and the server. The following description will be given taking an application to a terminal as an example. At least one deep learning model which is trained is deployed in the terminal in advance. The deep learning model can be pre-trained by using other computer equipment except the terminal. Specifically, the terminal acquires a plurality of sperm images to be detected. And detecting the sperm image to be detected based on the deep learning theory to obtain the sperm morphology detection result of the sperm in the sperm image to be detected. And detecting each sperm image to be detected based on a deep learning theory to obtain a sperm number detection result and a sperm position detection result corresponding to each sperm image to be detected. The terminal is also deployed with sperm concentration determination logic and sperm motility determination logic. And generating the sperm concentration according to the sperm number detection result of the plurality of sperm images to be detected through the sperm concentration determination logic. And generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected through the sperm motility determination logic.

The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server can be implemented by an independent server or a server cluster formed by a plurality of servers.

In one embodiment, as shown in fig. 1, a sperm quality detection method is provided, which is described by taking the method as an example of being applied to a terminal, and comprises the following steps:

and step S110, acquiring a plurality of sperm images to be detected.

The sperm image to be detected can be an image obtained by performing a series of preprocessing on an original sperm image. The original sperm image is an image obtained by collecting a fresh semen sample. The fresh semen sample may be an unstained sample. A certain amount (for example, 5 μ l) of semen is dropped into a counting plate having a certain thickness (for example, 10 μm), and the counting plate is collected using an image collecting device (for example, an optical microscope). The plurality of sperm images to be detected may be images acquired from a plurality of acquisition views, for example, the setting requires detection of sperm images in n different acquisition views, and the plurality of sperm images to be detected are n images acquired from n different acquisition views. The plurality of sperm images to be detected can also be a plurality of frames of images obtained from the sperm video to be detected.

Specifically, the plurality of images of the sperm to be detected may be images acquired in real time, and the terminal may acquire the acquired images from the image acquisition device in real time. The plurality of sperm images to be detected can also be collected in advance and stored in a local database or a server database, so that the terminal can acquire the sperm images to be detected from the local database or the server database.

And step S120, detecting the sperm image to be detected to obtain the sperm morphology detection result of the sperm in the sperm image to be detected.

Specifically, the terminal detects any sperm image to be detected through the trained first deep learning model. The first deep learning model is a model with at least the target detection and classification recognition capabilities, and can be realized by an end-to-end model or by a combination of multiple independent models. And the terminal inputs any sperm image to be detected into the first deep learning model, and when the first deep learning model determines that the sperm image to be detected contains the sperm, the single sperm is continuously classified and identified, and the sperm morphology detection result of the sperm is output.

And S130, detecting each sperm image to be detected to obtain a sperm number detection result and a sperm position detection result corresponding to each sperm image to be detected.

Wherein, the sperm number detection result is used for reflecting the total number of the sperms in each sperm image to be detected. The sperm position detection result is used for reflecting the position information of each sperm in each sperm image to be detected. The position information of each sperm can be characterized by the position coordinates of the rectangular frame where the sperm is located, and the like.

Specifically, the terminal detects each sperm image to be detected through the trained second deep learning model. The second deep learning model is a model with at least target detection capability, and can be realized by an end-to-end model or by a combination of multiple independent models. And when the second deep learning model determines that the sperms exist in a certain sperm image to be detected, outputting the sperm position detection result of each sperm. And then, counting the number of the sperms to obtain a sperm number detection result of the sperm image to be detected.

Further, each sperm image to be detected can be characterized by a unique image identifier. After the sperm number detection result and the sperm position detection result of each sperm image to be detected are obtained, the terminal can establish the mapping relation among the image identification, the sperm number detection result and the sperm position detection result so as to facilitate subsequent use.

And step S140, determining the sperm concentration according to the sperm number detection results of the plurality of sperm images to be detected.

Specifically, after the terminal detects the plurality of sperm images to be detected, the terminal determines the sperm concentration based on the sperm number detection results of the plurality of sperm images to be detected through the sperm concentration detection logic. For example, the plurality of sperm images to be detected are randomly acquired by the counting plate, so that the average value of the number of the sperms of the plurality of sperm images to be detected can be acquired, and the sperm concentration can be calculated according to the average value of the number of the sperms and the semen volume.

And S150, generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected.

Wherein, the sperm motility detection result can be represented by any one or more of curvilinear motion speed, linear motion speed, linearity, swing degree, linearity and the like.

Specifically, after the detection of the plurality of sperm images to be detected is completed, the terminal monitors the movement track of a single sperm according to the position information of the single sperm in each sperm image to be detected. And generating a sperm motility detection result through sperm motility determination logic according to parameters such as the movement track, the movement time and the like of a single sperm. Wherein, the sperm motility determination logic can be realized by a preset motility calculation function and the like. The movement time of the individual sperm can be determined by the acquisition time of the image to be detected.

In the sperm quality detection method, the obtained multiple sperm images to be detected are detected based on the deep learning theory, and a sperm morphology detection result, a sperm number detection result and a sperm position detection result of the sperm are obtained. Then, determining the sperm concentration according to the sperm number detection results of a plurality of sperm images to be detected; and generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected. Based on the deep learning theory, the sperm morphology detection result and the semen conventional detection result can be quickly obtained on the basis of collecting a plurality of sperm images to be detected, so that the work flow of sperm quality detection is greatly simplified, the sperm quality detection efficiency is improved, and the full-automatic analysis of the sperm quality detection is realized. In addition, the deep learning model with enough detection capability is adopted to detect the sperm image to be detected, and the accuracy of sperm quality detection can be improved.

In an embodiment, as shown in fig. 2, in step S120, the detecting the sperm image to be detected to obtain the sperm morphology detection result of the sperm in the sperm image to be detected can be implemented by the following steps:

and S210, detecting the sperm image to be detected through the image segmentation model, and acquiring the sperm segmentation image of the sperm when the sperm image to be detected has the sperm.

The image segmentation model may be any model capable of performing image instance segmentation, such as Mask R-CNN (Mask Region volumetric Neural Networks, an image instance segmentation model), BlendMask (an instance segmentation model), SOLO-V2 (segmentation Objects by Locations, an instance segmentation model), and the like; or a model that is improved based on an existing model, etc.

Specifically, the terminal inputs any sperm image to be detected into the trained image segmentation model. When the image segmentation model detects that the sperms exist in the sperms to be detected, the boundaries of all the sperms are obtained, and corresponding sperms images are extracted from the sperms to be detected according to the boundaries of all the sperms. The terminal places each sperm image in a first predetermined size (e.g., 224 x 224 pixel size) and background images with all 0 pixel values to generate a corresponding sperm segmentation image.

And step S220, classifying and identifying the sperm segmentation images through the sperm morphology identification model to obtain a sperm morphology identification result.

The sperm morphology recognition model can be any model capable of performing classification recognition, such as AlexNet (a convolutional neural Network), ResNet (Residual Network, deep Residual Network); or a model improved based on an existing model; or a self-designed model, etc.

Specifically, the terminal inputs each obtained sperm segmentation image into a trained sperm morphology recognition model. Classifying and identifying each sperm segmentation image through a sperm morphology identification model to obtain a probability value of each morphology category, and determining a sperm morphology detection result of the sperm according to the probability value. Illustratively, the sperm morphology recognition model includes three categories: class a, class B, and class C. The detection results of the sperm segmentation images obtained by the sperm morphology recognition model are a class A probability value of 0.8, a class B probability value of 0.1 and a class C probability value of 0.1. The probability value of the class A is the maximum, and the class A can be used as the sperm morphology detection result of the sperm in the sperm segmentation image.

In the embodiment, the sperm segmentation image of a single sperm is obtained by adopting the image segmentation model, each sperm can be accurately extracted according to the boundary obtained by segmentation, the interference of other impurities is eliminated, and the problem that the sperm is interfered by other sperms under the condition that the sperms are crossed is avoided, so that the accuracy of sperm morphology detection is improved.

In one embodiment, as shown in fig. 3, a training method of the image segmentation model is described, which can be implemented by the following steps:

step S310, acquiring a sperm image sample and a sperm segmentation image label corresponding to the sperm image sample.

Wherein, the sperm image sample can be an image sample acquired under different conditions (e.g., different acquisition fields, different illumination) by using an image acquisition device. Each sperm image sample comprises sperms needing to be segmented by the image segmentation model. The sperm segmentation image label is a data label obtained by labeling the sperm boundary in the sperm image sample on the basis of the sperm image sample.

In particular, a plurality of sperm image samples under different conditions may be acquired using the image acquisition device. The plurality of sperm image samples and the sperm segmentation image tags are stored in a local database or are solidified in a server. When the initial image segmentation model needs to be trained, a plurality of sperm image samples are obtained from a local database or a server. And labeling the sperm boundary in each sperm image sample to obtain a corresponding sperm segmentation image label.

And step S320, cutting the sperm image sample according to a preset image cutting mode to obtain a plurality of training image samples.

Wherein. The preset image cutting mode is determined according to the actual training requirement, for example, the sperm image sample can be equally divided and cut; cutting out a preset size from the sperm image, and the like. In this embodiment, images of a predetermined size can be obtained by cutting out each of the four corners of each sperm image sample.

Specifically, after acquiring a plurality of sperm image samples, the terminal cuts each sperm image sample according to a preset image cutting mode, and cuts each of four corner positions (i.e., upper left, upper right, lower left, and lower right) of each sperm image sample to obtain an image with a second preset size as a training image sample.

And step S330, cutting the sperm segmentation image labels according to an image cutting mode to obtain training labels corresponding to each training image sample.

Specifically, the sperm segmentation image labels corresponding to each sperm image sample are cut according to a preset image cutting mode, and image labels with second preset sizes are obtained through cutting from four corners of each sperm segmentation image label and serve as training labels. And establishing the corresponding relation between the training image samples and the image labels belonging to the same angular position.

Step S340, training the initial image segmentation model by using the training image sample and the training label to obtain the image segmentation model.

The initial image segmentation model refers to an image segmentation model which is not trained yet.

Specifically, the terminal inputs training image samples into an initial image segmentation model, and outputs a prediction segmentation result through the initial image segmentation model. And calculating a loss value between the prediction segmentation result and the training label by using a loss function. And adjusting the model parameters of the initial image segmentation model according to the calculated loss value until a preset stop condition is reached. The preset stop condition may refer to that a preset number of iterations is reached, or that the obtained loss value satisfies a preset threshold. And finally, generating the finally used image segmentation model according to the model parameters of the initial image segmentation model with the minimum loss value or the best robustness in the training process.

In the embodiment, a plurality of training images are obtained by cutting the sperm image sample according to a preset image cutting mode, so that on one hand, an image suitable for the input size of the model can be obtained, on the other hand, data enhancement can be realized, and the overfitting problem of the image segmentation model in the training process is favorably alleviated.

In one embodiment, the sperm morphology recognition model includes a plurality of convolution portions connected in series, and a fully connected layer connected to the last convolution portion. Each convolution portion includes a convolution embedded block and a plurality of convolution transform blocks. Fig. 4 is a schematic diagram illustrating the structure of the sperm morphology recognition model. As shown in fig. 5, in step S220, the sperm segmentation image is classified and recognized by the sperm morphology recognition model to obtain the sperm morphology recognition result, which can be realized by the following steps:

and step S510, sequentially performing feature extraction on the sperm input features through the convolution embedded blocks in each convolution part.

And step S520, performing global information learning and local information learning on the sperm input features after feature extraction through the convolution transformation blocks in each convolution part to obtain sperm output features.

Wherein, for the first convolution part, the sperm input characteristic is the sperm segmentation image, and for each other convolution part except the first convolution part, the sperm input characteristic is the sperm output characteristic output by the last convolution part.

The stride of the convolution layer in the convolution embedded block is larger than 1, so that the input characteristic size of the sperms passing through the convolution embedded block is reduced, the number of channels is increased, and the quantity of parameters input to the convolution transformation block can be greatly reduced. Fig. 6 illustrates an exemplary structure of the convolution embedded block. As shown in fig. 6, the convolution embedded block includes a convolution layer, a shape conversion layer, and a normalization layer, which are connected in this order.

The convolution transform block may be used to learn global information of the input features, such as global information of the overall sperm profile; and inputting local information of the feature, for example, local information of sperm acrosome area. Fig. 7 illustrates an exemplary structure diagram of a convolution transform block. As shown in fig. 7, the convolution transform block includes a convolution mapping layer, a multi-head attention layer, a normalization layer, a multi-layered perceptron layer, and a residual connection layer, which are connected in sequence.

Specifically, the terminal inputs the sperm segmentation image to the sperm morphology recognition model. The convolution layer in the convolution embedded block in the first convolution part carries out feature extraction on the sperm segmentation image and then inputs the sperm segmentation image to the shape conversion layer. The shape conversion layer changes the shape of the feature vector output by the convolution layer to a shape suitable for the input of the convolution conversion block. Then, the normalized layer is input to a convolution transformation block connected to the convolution embedding block. And performing global information learning and local information learning on the sperm input features after feature extraction through a convolution mapping layer, a multi-head attention layer, a normalization layer, a multi-layer perceptron layer and a residual connecting layer in a plurality of convolution transformation blocks to obtain the sperm output features. Inputting the sperm output characteristics into the second convolution part by the first convolution part, and circulating the processes until the last convolution part is processed.

And step S530, generating a sperm morphology recognition result according to the sperm output characteristics output by the last convolution part.

Specifically, the last convolution portion inputs the outputted sperm output characteristics to the fully connected layer. And processing the sperm output characteristics through the full-connection layer, and outputting a vector with a preset length. And further obtaining the sperm morphology recognition result (namely the category) of the sperm in the sperm segmentation image according to the vector with the preset length.

In this embodiment, the sperm morphology recognition model is used for the sperm morphology detection, so that the sperm morphology recognition model can learn more information, thereby contributing to the improvement of the accuracy of the sperm morphology detection.

Step S130, detecting each sperm image to be detected to obtain sperm number detection results and sperm position detection results corresponding to each sperm image to be detected, and the method comprises the following steps: performing target detection on each sperm image to be detected, and when the head of the sperm exists in the sperm image to be detected, acquiring the number of the heads of the sperm, wherein the number of the heads is used as a sperm number detection result; and acquiring head position information of the head of the sperm, and taking the head position information as a sperm position detection result.

Specifically, after acquiring a plurality of sperm images to be detected, the terminal can perform image segmentation and classification identification on any one sperm image to be detected, and can also perform target detection on each sperm image to be detected synchronously. The following processing is performed for each sperm image to be detected:

and inputting the sperm image to be detected into the trained target detection model by the terminal. When the target detection model acquires that the sperm head exists in the sperm image to be detected, counting the total number of the sperm heads existing in the sperm image to be detected to obtain the number of the sperm heads, and taking the number of the sperm heads as a sperm number detection result. And acquiring head position information of each sperm head, and taking the head position information as a sperm position detection result. Correspondingly, when the target detection model acquires that the sperm head does not exist in the sperm image to be detected, the number of the head of the sperm head is set to be 0. Any model capable of performing target detection may be used as the target detection model, for example, refindet (a Single-stage-based Detector), fast R-CNN (a target detection network), SSD (Single Shot multi Detector, a target detection model), YOLO (young Only Look on, a target detection model), and the like.

In this embodiment, the sperm heads in each sperm image to be detected are detected based on the target detection model, and the number and the position information of the sperm heads are used as the number and the position information of the sperm.

In one embodiment, the sperm concentration may be calculated with reference to the sperm concentration calculation of CASA. CASA evenly divides an image in the range of 1mm (millimeters) by 1mm into 100 square cells, each square cell having a side length of 100 μm (micrometers). Sperm concentration was calculated from the number of sperm in 10 cells. In this case, the plurality of images of the sperm to be detected required for calculating the concentration of the sperm may be images acquired from a plurality of different acquisition fields. The number of the acquisition fields is preferably 10, and there is no overlap between the sperm images acquired through the respective acquisition fields.

Specifically, the terminal acquires the original sperm image acquired by the image acquisition device from each acquisition field. The original sperm image may refer to an image that has not been modified any further after acquisition. And (3) cutting the original sperm image corresponding to each acquisition visual field by the computer equipment to obtain a sperm image to be detected with a third preset size (100 micrometers by 100 micrometers). The cutting mode can be random cutting or appointed position cutting. Then, the terminal detects each sperm image to be detected through the target detection model, and the specific implementation manner of the target detection model may refer to the above embodiments, which are not described in detail herein. And obtaining the number of the sperm heads in each sperm image to be detected according to the output result of the target detection model. And the terminal acquires the sum of the number of the heads of the sperm images to be detected as the total number of the sperms in the plurality of acquisition visual fields. And calculating the sperm concentration according to the total number of the sperm by adopting a first calculation formula. Wherein, the first calculation formula may be:

concentration of sperm was Nx 10⁶/ml

Where N represents the total number of sperm cells in the multiple collection fields.

In this embodiment, the sperm concentration is calculated based on the images of the plurality of collected visual fields, and the accuracy of the obtained sperm concentration can be ensured. The sperm head in each sperm image to be detected is detected based on the target detection model, and compared with a method for detecting the whole sperm, the method can reduce the crossing probability of the sperm, thereby being beneficial to improving the accuracy of the concentration of the sperm.

In one embodiment, a means of detecting sperm motility is described. Aiming at sperm motility detection, a plurality of sperm images to be detected are continuous multi-frame images obtained from sperm videos to be detected. The terminal detects each frame of image through the target detection model, and when detecting that a plurality of sperm heads exist in each frame of image, the head position information of each sperm head in each frame of image is obtained, and the specific implementation manner of the target detection model can refer to the above embodiments, which is not described in detail herein. The terminal performs the following processing for each frame image in sequence starting from the first frame image: as shown in fig. 8, step S150 can be implemented by the following steps, based on the sperm position detection results of the plurality of sperm images to be detected, to generate the sperm motility detection result.

Step S810, when a plurality of sperm heads exist in each frame of image, acquiring first head position information of each sperm head in the current frame of image and second head position information of each sperm head in the next frame of image of the current frame of image.

In step S820, a distance between the first head position information and each second head position information of each sperm head in the current frame image is generated.

The current frame image refers to an image currently being analyzed and processed by the terminal. The head position information may include, but is not limited to, position coordinates and center point coordinates of a target detection frame in which the head of the sperm is located.

Specifically, the terminal acquires first head position information of each sperm head in a current frame image and second head position information of each sperm head in a next frame image of the current frame image according to an output result of the target detection model. And aiming at each sperm head in the current frame image, the terminal calculates the distance between the first head position information and each second head position information in the next frame image through a second calculation formula. Wherein, the second calculation formula may be:

D＝Dist-10*IoU

wherein D represents a distance; dist represents the distance between the central point of the target detection frame of each sperm head in the current frame and the central point of each sperm head target detection frame in the next frame; IoU represents the IOU (Intersection over Unit) value of each sperm head target detection box in the current frame and each sperm head target detection box in the next frame.

In step S830, according to the distance, the target head position information is determined from the plurality of second head position information, and is used as the head position information of each sperm head in the current frame image in the next frame image.

And step 840, generating a sperm motility detection result according to the head position information of each sperm head in each frame of image.

Specifically, the terminal acquires a distance having a minimum value from among a plurality of distances. And determining the second head position information corresponding to the distance with the minimum value as the head position information of each sperm head in the current frame image in the next frame image. And the terminal determines the position of each sperm in each frame of image according to the content in the steps S810 to S830, and further generates the motion trail corresponding to each sperm. And the terminal generates a sperm motility detection result through sperm motility determination logic according to parameters such as the movement track, the movement time and the like of each sperm.

In this embodiment, the sperm motility is detected based on a plurality of continuous frame images, and the accuracy of the obtained sperm motility can be ensured. The sperm head in each sperm image to be detected is detected based on the target detection model, and compared with a method for detecting the whole sperm, the method can reduce the crossing probability of the sperm, thereby being beneficial to improving the sperm motility detection accuracy.

In one embodiment, as shown in fig. 9, a particular sperm cell quality detection method is provided, comprising the steps of:

step S902, acquiring a plurality of original sperm images. The image size (pixel size) of the original sperm image was 1024 × 1536. The images of the multiple original sperms can be obtained by imaging and photographing the counting plate under a 40-time objective lens by an optical microscope equipped with a digital camera. The plurality of original sperm images comprise a plurality of images collected from a plurality of collection fields and a plurality of continuous frames of images obtained from a sperm video to be detected. Wherein the visual field range of each acquisition visual field is 240 μm × 160 μm.

Step S904, performing a first preprocessing on the image size of any original sperm image to obtain a sperm image to be detected. Wherein the first preprocessing includes a size normalization processing and a pixel value normalization processing. That is, images with a size of 768 × 1152 were obtained from the original sperm images; each pixel value of the resulting image is divided by 255 and normalized.

And step S906, detecting the obtained sperm image to be detected through the image segmentation model, and obtaining a single sperm image when the sperm image to be detected has the sperm. The sperm image is placed in a background image with the size of 224 x 224 and all pixel values of 0, and a corresponding sperm segmentation image is obtained.

One way of training the image segmentation model is described below:

the image segmentation model may employ a BlendMask model. First, a plurality of sperm image samples and sperm segmentation image labels corresponding to the sperm image samples are obtained. The image size of the sperm image sample was 1024 x 1536. And (3) cutting four corners (upper left, upper right, lower left and lower right) of each sperm image sample to obtain an image with the size of 768 × 1152, and performing pixel value normalization processing on the cut image to obtain a training image sample. And cutting the sperm segmentation image labels according to the image cutting mode to obtain the training labels corresponding to each training image sample. Training image samples were input to the initial BlendMask model. And outputting a prediction segmentation result through an initial BlendMask model. And calculating a loss value between the prediction segmentation result and the training label by adopting a Dice (Dys) loss function. Model parameters of the initial blendmak model were adjusted using an Adam (adaptive moment estimation) optimizer. And repeating the process until the loss value reaches a preset threshold or the iteration times reaches preset times, and generating the finally used BlendMask model. By using the BlendMask model, high-level global information (such as sperm head contour) and low-level fine-grained information (such as sperm head vacuole) can be fused to extract more accurate sperm example segmentation characteristics, so that each sperm can be segmented more accurately.

And step S908, identifying the sperm segmentation images through the sperm morphology identification model to obtain the sperm morphology detection result of the sperm in each sperm segmentation image.

One way of training the sperm morphology recognition model is described below:

the structural schematic diagram of the sperm morphology recognition model can refer to the above-mentioned embodiments. First, several individual sperm image samples are acquired, along with a classification label corresponding to each individual sperm image sample. And carrying out pixel value normalization processing on each single sperm image sample. And then, inputting the preprocessed single sperm image sample into the initial sperm morphology recognition model to obtain a prediction classification result. And calculating a loss value between the prediction classification result and the classification label by adopting a cross entropy loss function. Model parameters of the initial sperm morphology recognition model were adjusted using an Adam optimizer. And repeating the process until the loss value reaches a preset threshold value or the iteration times reach preset times, and generating the finally used sperm morphology recognition model.

In step S910, an image of 100 μm by 100 μm (640 pixel size) is cropped from the top left corner of the original sperm image collected from each collection field. And carrying out pixel value normalization processing on the cut image to obtain a sperm image to be detected corresponding to each acquisition visual field.

And step S912, performing target detection on each sperm image to be detected through a target detection model, and when the head of the sperm exists in the sperm image to be detected, acquiring the number of the heads of the sperm and taking the number of the heads as a sperm number detection result.

One way of training the target detection model is described below:

the target detection model adopts the YOLO model of version 5. First, several sperm head image samples are acquired, along with a data tag corresponding to each sperm head image sample. The image size of the sperm head image sample was 1024 x 1536. Images of 1024 x 1024 size were cut from each sperm head image sample on the left and right across the width. And carrying out pixel value normalization processing on the image obtained by cutting, and scaling the processed image to 640 x 640 size to obtain a training image sample. And performing the same cropping processing and scaling processing on the data labels to obtain training labels corresponding to each training image sample. The training image samples are input to the initial YOLO model. And outputting a predicted head result through an initial YOLO model. And calculating a loss value between the predicted head result and the training label by adopting a regression loss function. Model parameters of the initial YOLO model were adjusted using Adam optimizer. And repeating the process until the loss function reaches a preset threshold or the iteration times reaches preset times, and generating the finally used YOLO model.

And step S914, acquiring the sum of the number of the heads of the plurality of sperm images to be detected as the total number of the sperms in the plurality of acquisition visual fields, and determining the concentration of the sperms according to the total number of the sperms. The specific determination of the sperm concentration can be made by reference to the above examples and will not be described in detail herein.

Step S916, performing target detection on each frame of image acquired from the video to be detected through the target detection model, and acquiring head position information of the head of the sperm when the head of the sperm exists in each frame of image, and taking the head position information as a sperm position detection result.

Step S918, generating a sperm motility detection result according to the head position information of each sperm head in each frame of image. The specific detection mode of sperm motility can refer to the above embodiments, and is not specifically described herein.

It should be understood that, although the steps in the above-described flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the flowcharts shown above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the steps or stages in other steps.

Based on the above description of the embodiments of the sperm quality detection method, the present disclosure also provides a sperm quality detection apparatus. The apparatus may include systems (including distributed systems), software (applications), modules, components, servers, clients, etc. that use the methods described in embodiments of the present specification in conjunction with any necessary apparatus to implement the hardware. Based on the same innovative concept, the embodiments of the present disclosure provide an apparatus in one or more embodiments as described in the following embodiments. Since the implementation scheme of the apparatus for solving the problem is similar to that of the method, the specific implementation of the apparatus in the embodiment of the present specification may refer to the implementation of the foregoing method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.

In one embodiment, as shown in fig. 10, there is provided a sperm quality detection apparatus 1000 comprising: an image acquisition module 1002, a morphology detection module 1004, a target detection module 1006, a concentration determination module 1008, and an activity detection module 1010, wherein:

the image acquisition module 1002 is used for acquiring a plurality of sperm images to be detected; the morphology detection module 1004 is used for detecting the sperm image to be detected to obtain a sperm morphology detection result of the sperm in the sperm image to be detected; the target detection module 1006 is configured to detect each sperm image to be detected, and obtain a sperm number detection result and a sperm position detection result corresponding to each sperm image to be detected; the concentration determination module 1008 is used for determining the concentration of the sperms according to the sperm number detection results of the plurality of sperm images to be detected; and the motility detection module 1010 is used for generating a sperm motility detection result according to the sperm position detection results of the plurality of sperm images to be detected.

In one embodiment, the morphology detection module 1004 includes:

the image segmentation unit is used for detecting the sperm image to be detected through the image segmentation model, and acquiring the sperm segmentation image of the sperm when the sperm image to be detected has the sperm; and the classification identification unit is used for classifying and identifying the sperm segmentation images through the sperm morphology identification model to obtain a sperm morphology identification result.

In one embodiment, the apparatus 1000 further comprises:

the sample acquisition module is used for acquiring a sperm image sample and a sperm segmentation image label corresponding to the sperm image sample; the first cutting module is used for cutting the sperm image samples according to a preset image cutting mode to obtain a plurality of training image samples; the second cutting module is used for cutting the sperm segmentation image labels according to a preset image cutting mode to obtain training labels corresponding to each training image sample; and the model training module is used for training the initial image segmentation model by using the training image sample and the training label to obtain the image segmentation model.

In one embodiment, the sperm morphology recognition model comprises a plurality of convolution sections connected in sequence, each convolution section comprising a convolution embedded block and a plurality of convolution transformation blocks;

a classification recognition unit comprising: the characteristic extraction subunit is used for sequentially carrying out characteristic extraction on the sperm input characteristics through the convolution embedded blocks in each convolution part, wherein for the first convolution part, the sperm input characteristics are sperm segmentation images, and for each other convolution part except the first convolution part, the sperm input characteristics are sperm output characteristics output by the last convolution part; the information learning subunit is used for performing global information learning and local information learning on the sperm input features after feature extraction through the convolution transformation blocks in each convolution part to obtain sperm output features; and the result generating subunit is used for generating a sperm morphology recognition result according to the sperm output characteristics output by the last convolution part.

In one embodiment, the convolution transformation block comprises a convolution mapping layer, a multi-head attention layer, a multi-layer perceptron layer, a residual error connection layer and a normalization layer; and the information learning subunit is used for performing global information learning and local information learning on the sperm input features after feature extraction through the convolution mapping layer, the multi-head attention layer, the multilayer perceptron layer, the residual error connecting layer and the normalization layer to obtain the sperm output features.

In one embodiment, the target detection module 1006 includes: and the target detection unit is used for carrying out target detection on each sperm image to be detected. The quantity acquisition unit is used for acquiring the quantity of the heads of the sperms when the sperms heads exist in the sperms to be detected in the sperms image, and taking the quantity of the heads as a sperm number detection result; and the position information acquisition unit is used for acquiring the head position information of the sperm head when the sperm head exists in the sperm image to be detected, and taking the head position information as the sperm position detection result.

the concentration determination module 1008 is used for a sperm total number acquisition unit and is used for acquiring the sum of the number of the heads of a plurality of sperm images to be detected as the total number of the sperm under a plurality of acquisition visual fields; a concentration generating unit for determining the sperm concentration based on the total number of sperm.

In one embodiment, the plurality of sperm images to be detected are continuous multi-frame images obtained from a sperm video to be detected;

a viability detection module 1010 comprising: the position information acquisition unit is used for acquiring first head position information of each sperm head in a current frame image and second head position information of each sperm head in a next frame image of the current frame image when a plurality of sperm heads exist in each frame image; the distance generating unit is used for generating the distance between the first head position information and the second head position information of each sperm head in the current frame image; the target position determining unit is used for determining target head position information from the second head position information according to the distance, and the target head position information is used as the head position information of each sperm head in the current frame image in the next frame image; and the motility result generating unit is used for generating a sperm motility detection result according to the head position information of each sperm head in each frame of image.

For the specific limitations of the sperm quality detecting device, reference may be made to the limitations of the sperm quality detecting method described above, and the details are not repeated herein. The various modules in the sperm quality detection apparatus described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 11. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a sperm quality detection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 11 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, there is provided a computer device comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the sperm quality detection method of any of the above embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the sperm quality detection method of any of the above embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method of sperm quality detection, said method comprising:

acquiring a plurality of sperm images to be detected;

2. The method according to claim 1, wherein the detecting the sperm image to be detected to obtain the sperm morphology detection result of the sperm in the sperm image to be detected comprises:

3. The method of claim 2, wherein the image segmentation model is generated in a manner that includes:

4. The method of claim 2, wherein the sperm morphology recognition model comprises a plurality of convolution sections connected in series, each of the convolution sections comprising a convolution embedded block and a plurality of convolution transformed blocks;

5. The method of claim 4, wherein the convolution transform block comprises a convolution mapping layer, a multi-head attention layer, a multi-layer perceptron layer, a residual connection layer, and a normalization layer;

6. The method according to claim 1, wherein the detecting each sperm image to be detected to obtain the sperm number detection result and the sperm position detection result corresponding to each sperm image to be detected comprises:

7. The method according to claim 6, wherein each sperm image to be detected is an image cropped from a corresponding original sperm image, and the acquisition fields of view of different original sperm images are different;

determining the sperm concentration from the total number of sperm.

8. The method according to claim 6, wherein the plurality of sperm images to be detected are consecutive multiframe images obtained from a sperm video to be detected;

9. A sperm quality detection apparatus, said apparatus comprising:

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 8.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.