CN112200803B - Sperm nucleoprotein maturity detection method and device - Google Patents
Sperm nucleoprotein maturity detection method and device Download PDFInfo
- Publication number
- CN112200803B CN112200803B CN202011216686.XA CN202011216686A CN112200803B CN 112200803 B CN112200803 B CN 112200803B CN 202011216686 A CN202011216686 A CN 202011216686A CN 112200803 B CN112200803 B CN 112200803B
- Authority
- CN
- China
- Prior art keywords
- sperm
- network
- nucleoprotein
- training
- sperm nucleoprotein
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
- G06T2207/10061—Microscopic image from scanning electron microscope
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Investigating Or Analysing Biological Materials (AREA)
- Image Analysis (AREA)
Abstract
The application belongs to the technical field of medical image processing, and particularly discloses a sperm nucleoprotein maturity detection method and device. According to the application, through acquiring the sperm nucleoprotein staining microscopic image and passing the sperm nucleoprotein staining microscopic image through a preset sperm nucleoprotein detection model, the sperm nucleoprotein image, the position information of the sperm nucleoprotein in the image and the sperm nucleoprotein maturity detection result are obtained, and compared with the traditional target detection algorithm, the recognition rate and accuracy of the sperm nucleoprotein maturity are greatly improved.
Description
Technical Field
The application belongs to the technical field of medical image processing, and particularly relates to a sperm nucleoprotein maturity detection method and device.
Background
The detection of sperm nucleoprotein needs to be dyed by a nucleoprotein reagent (aniline blue reagent), then the maturity of the nucleoprotein is judged by color under a 100-fold microscope, the nucleoprotein component of the mature sperm is protamine, and the mature sperm is rich in arginine and cystine residues; the nucleoprotein component of immature sperm is histone and is rich in lysine residues. Under acidic conditions, aniline blue binds to lysine residues to form a violet compound, indicating the presence of immature nucleoprotein, while nucleoprotein mature sperm stain red.
The traditional method for detecting the sperm nucleoprotein mainly comprises the steps of photographing a dyed glass slide under a microscope 100 times of the microscope, then carrying out sperm target detection on an image obtained by photographing through a traditional target detection algorithm (such as Cascade+HOG/DPM+haar/SVM), and then carrying out classification judgment on the sperm nucleoprotein through a traditional classification algorithm, wherein the characteristic is not very robust due to the fact that the traditional algorithm needs manual design, and the characteristic is not very good due to the fact that the variation of the diversity of the sperm nucleoprotein image, so that the sperm nucleoprotein identification rate and accuracy are not high, and the judgment result of a clinician is greatly different.
Disclosure of Invention
In view of the above, the present application provides a method and apparatus for detecting the maturity of sperm nucleoprotein, which can solve or at least partially solve the above-mentioned problems.
In order to solve the technical problems, the technical scheme provided by the application is a sperm nucleoprotein maturity detection method, which comprises the following steps:
s11: obtaining a sperm nucleoprotein staining microscopic image;
s12: and (3) passing the sperm nucleoprotein staining microscopic image through a sperm nucleoprotein detection model to obtain a sperm nucleoprotein image, position information of the sperm nucleoprotein in the image and a detection result of sperm nucleoprotein maturation or immature.
Preferably, the sperm nucleoprotein maturity detection method further comprises S10: generating a sperm nucleoprotein detection model; the method of S10 comprises the following steps:
s101: screening out sperm nucleoprotein staining microscopic images of a large sample;
s102: marking sperm nucleoprotein mature or immature sperm nucleoprotein on a sperm nucleoprotein staining microscopic image, and dividing the marked sperm nucleoprotein staining microscopic image into a training image set and a testing image set;
s103: under the TensorFlow framework, training an image set by adopting a Faster-RCNN+ResNet network model, verifying and updating Faster-RCNN+ResNet network model parameters by using a test image set, training, and cycling the process until a final sperm nucleoprotein detection model is obtained.
Preferably, in the step S103, under the TensorFlow framework, the method for training the training image set by using the fast-rcnn+resnet network model includes:
adjusting the network structure of Faster-rcnn, replacing the basic network with a ResNet50 network, and adjusting corresponding training parameters: learning rate, number of training images, number of iterations, selection of optimizers, and selection of loss functions;
extracting the features of the training image set by using a group of basic conv+relu+pooling layers, and sharing the extracted features to a regional suggestion network layer and a full connection layer for processing;
the regional suggestion network layer is used for generating a regional suggestion frame, judging whether the suggestion frame is correct or not through a softmax loss function, and correcting the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame;
the interesting area pooling layer is used for collecting input characteristics and suggestion frames, synthesizing the information and then sending the information to the subsequent full-connection layer to judge the target category;
the full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and simultaneously obtains the final accurate position of the detection frame by using the bounding box regression again, wherein the formula is as follows:wherein x is i Input data for a Faster-RCNN+ResNet network, w ij B for the weight value of the input data to the output data i For the corresponding deviation value, y i Is the corresponding output value.
Preferably, the method for verifying and updating the parameters of the Faster-RCNN+ResNet network model by testing the image set in the step S103 and training again, and cycling the process until the final sperm nucleoprotein detection model is obtained comprises the following steps:
on the trained sperm nucleoprotein detection model, training a region suggestion network for the first time to obtain the region suggestion network trained for the first time;
collecting a suggestion frame by utilizing the first trained regional suggestion network and the test image set;
training a fast-rcnn network for the first time;
the second training area suggests a network to obtain a second trained RPN network;
collecting a suggestion frame by utilizing the region suggestion network trained for the second time and the test image set;
the fast-rcnn network is trained a second time.
The application also provides a sperm nucleoprotein maturity detection device, comprising:
the sperm image acquisition module is used for acquiring sperm nucleoprotein staining microscopic images;
the nucleoprotein detection module is used for passing the sperm nucleoprotein staining microscopic image through a sperm nucleoprotein detection model to obtain a sperm nucleoprotein image, position information of the sperm nucleoprotein in the image and a detection result of sperm nucleoprotein maturation or immature.
Preferably, the sperm nucleoprotein maturity detection device further comprises: the detection model generation module is used for generating a sperm nucleoprotein detection model; the detection model generation module comprises:
the sample data screening unit is used for screening sperm nucleoprotein staining microscopic images of large samples;
the sample data labeling unit is used for labeling sperm nucleoprotein maturation or immature sperm nucleoprotein on the sperm nucleoprotein staining microscopic image and dividing the labeled sperm nucleoprotein staining microscopic image into a training image set and a testing image set;
the detection model training unit is used for training the training image set by adopting a Faster-RCNN+ResNet network model under the TensorFlow framework, verifying and updating Faster-RCNN+ResNet network model parameters by the test image set, training again, and cycling the process until a final sperm nucleoprotein detection model is obtained.
Preferably, the method for training the training image set by the detection model training unit by adopting a fast-RCNN+ResNet network model under the TensorFlow framework comprises the following steps:
adjusting the network structure of Faster-rcnn, replacing the basic network with a ResNet50 network, and adjusting corresponding training parameters: learning rate, number of training images, number of iterations, selection of optimizers, and selection of loss functions;
extracting the features of the training image set by using a group of basic conv+relu+pooling layers, and sharing the extracted features to a regional suggestion network layer and a full connection layer for processing;
the regional suggestion network layer is used for generating a regional suggestion frame, judging whether the suggestion frame is correct or not through a softmax loss function, and correcting the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame;
the interesting area pooling layer is used for collecting input characteristics and suggestion frames, synthesizing the information and then sending the information to the subsequent full-connection layer to judge the target category;
the full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and simultaneously obtains the final accurate position of the detection frame by using the bounding box regression again, wherein the formula is as follows:wherein x is i Input data for a Faster-RCNN+ResNet network, w ij For inputting data to outputting dataWeight value of the upper pair, b i For the corresponding deviation value, y i Is the corresponding output value.
Preferably, the test model training unit verifies and updates the parameters of the Faster-RCNN+ResNet network model through the test image set, and trains again, and the method for cycling the process until the final sperm nucleoprotein test model is obtained comprises the following steps:
on the trained sperm nucleoprotein detection model, training a region suggestion network for the first time to obtain the region suggestion network trained for the first time;
collecting a suggestion frame by utilizing the first trained regional suggestion network and the test image set;
training a fast-rcnn network for the first time;
the second training area suggests a network to obtain a second trained RPN network;
collecting a suggestion frame by utilizing the region suggestion network trained for the second time and the test image set;
the fast-rcnn network is trained a second time.
The application also provides a sperm nucleoprotein maturity detection device, comprising:
a memory for storing a computer program;
and a processor for executing the computer program to realize the steps of the sperm nucleoprotein maturity detection method.
Compared with the prior art, the application has the following beneficial effects: according to the application, through acquiring the sperm nucleoprotein staining microscopic image and passing the sperm nucleoprotein staining microscopic image through a preset sperm nucleoprotein detection model, the sperm nucleoprotein image, the position information of the sperm nucleoprotein in the image and the sperm nucleoprotein maturity detection result are obtained, and compared with the traditional target detection algorithm, the recognition rate and accuracy of the sperm nucleoprotein maturity are greatly improved.
Drawings
For a clearer description of embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a schematic flow chart of a sperm nucleoprotein maturity detection method according to an embodiment of the application;
FIG. 2 is a schematic flow chart of another method for detecting the maturity of sperm nucleoprotein according to an embodiment of the application;
FIG. 3 is a schematic flow chart of a sperm nucleoprotein detection model according to an embodiment of the application;
fig. 4 is a schematic structural diagram of a sperm nucleoprotein maturity detection device according to an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without making any inventive effort are within the scope of the present application.
In order to make the technical solution of the present application better understood by those skilled in the art, the present application will be further described in detail with reference to the accompanying drawings and specific embodiments.
As shown in fig. 1, the present application provides a sperm nucleoprotein maturity detection method, which can be applied to a sperm nucleoprotein maturity detection system, comprising:
s11: obtaining a sperm nucleoprotein staining microscopic image;
specifically, firstly, sperm is stained by a nucleoprotein reagent (aniline blue reagent), then a sperm slide after staining is photographed under a microscope with a magnification of 100 times to obtain a sperm nucleoprotein staining microscopic image, and then the sperm nucleoprotein staining microscopic image is input into a sperm nucleoprotein maturity detection system, namely, the sperm nucleoprotein maturity detection system receives or acquires the sperm nucleoprotein staining microscopic image.
S12: and (3) passing the sperm nucleoprotein staining microscopic image through a sperm nucleoprotein detection model to obtain a sperm nucleoprotein image, position information of the sperm nucleoprotein in the image and a detection result of sperm nucleoprotein maturation or immature.
Specifically, a sperm nucleoprotein staining microscopic image collected under a 100-fold mirror is tested by a neural network model-sperm nucleoprotein detection model obtained through the Faster-rcnn training. The sperm nuclear protein detection model can automatically identify sperms in the image through corresponding codes, output sperm nuclear protein images and positions (upper left corner coordinates + width + height) of sperm nuclear proteins in the images, and automatically distinguish the mature or immature sperms of the nuclear proteins according to the categories (red is mature sperms of the nuclear proteins and purple blue is immature sperms of the nuclear proteins).
It should be noted that, as shown in fig. 2, the present application further provides a method for detecting the maturity of sperm nucleoprotein, and step S10 is added on the basis of the embodiment of fig. 1: generating a sperm nucleoprotein detection model.
Specifically, as shown in fig. 3, the method of S10 includes:
s101: screening out sperm nucleoprotein staining microscopic images of a large sample;
specifically, firstly, the stained sperm slide is placed under a microscope for photographing under a 100-time microscope, the photographed pictures are arranged, and the pictures with the photographing effect meeting the requirement are selected to be used as a data set, so that the screened samples (sperm nucleoprotein) are more diversified and more balanced.
S102: marking sperm nucleoprotein mature or immature sperm nucleoprotein on a sperm nucleoprotein staining microscopic image, and dividing the marked sperm nucleoprotein staining microscopic image into a training image set and a testing image set;
specifically, marking the sperm nucleoprotein on the image in the dataset, recording the left upper corner coordinates and the corresponding width and height of the nucleoprotein marking frame of each sperm, dividing the left upper corner coordinates and the corresponding width and height into two types of mature sperm nucleoprotein and immature sperm nucleoprotein, generating a data file in json format, and dividing the data set into two types, namely a training image set and a test image set; and finally, generating the marked file into tfreeord format data so as to facilitate training.
S103: under the TensorFlow framework, training an image set by adopting a Faster-RCNN+ResNet network model, verifying and updating Faster-RCNN+ResNet network model parameters by using a test image set, training, and cycling the process until a final sperm nucleoprotein detection model is obtained.
Specifically, the target detection is performed through the neural network, compared with a traditional target detection method, the network structure is deepened, extracted features are more, more useful information can be obtained, and the target detection effect is improved. The neural network structures currently used for target detection are the rcnn series, the ssd series and the YOLO series. Wherein the rcnn series is a two-step process and the ssd series and YOLO series are one-step processes, since the rcnn series is more accurate than the ssd series and YOLO series, the ssd series and YOLO series are Faster than the rcnn series, and we use the Faster-rcnn + ResNet50 network in the rcnn series to train sperm nuclear proteins under the tensorsurface framework for higher accuracy.
In S103, under the TensorFlow framework, the method for training the training image set by using the fast-rcnn+resnet network model includes:
adjusting the network structure of Faster-rcnn, replacing the basic network with a ResNet50 network, and adjusting corresponding training parameters: learning rate, number of training images, number of iterations, selection of optimizers, and selection of loss functions;
extracting the features of the training image set by using a group of basic conv+relu+pooling layers, and sharing the extracted features to a regional suggestion network layer and a full connection layer for processing;
the regional suggestion network layer is used for generating a regional suggestion frame, judging whether the suggestion frame is correct or not through a softmax loss function, and correcting the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame;
the interesting area pooling layer is used for collecting input characteristics and suggestion frames, synthesizing the information and then sending the information to the subsequent full-connection layer to judge the target category;
the full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and simultaneously obtains the final accurate position of the detection frame by using the bounding box regression again, wherein the formula is as follows:wherein x is i Input data for a Faster-RCNN+ResNet network, w ij B for the weight value of the input data to the output data i For the corresponding deviation value, y i Is the corresponding output value.
Specifically, the process of training sperm nuclear proteins under the tensorsurface framework by the Faster-rcnn+ResNet50 network (where ResNet50 is the base network) includes the primary content of Faster-rcnn and the image processing process within Faster-rcnn. The method specifically comprises the following steps: the network structure of Faster-rcnn is regulated, the basic network is replaced by a ResNet50 network, and corresponding training parameters such as learning rate, number of training images, iteration times, selection optimizer, selection loss function and the like are regulated. The learning rate of the method is 0.001, the number of training images is about 3000, wherein sperm samples are 10000 more, the iteration number is 10 ten thousand, the SGD is selected by the optimizer, and the loss function softmax is obtained. Among them, the fast-rcnn processes the main content of the image, namely, some main methods adopted include: 1) Extracting features of the image by using a group of basic conv+relu+pooling layers (namely a convolution layer, an activation layer and a pooling layer), and sharing the extracted features to an RPN (regional suggestion network) layer and a full-connection layer for processing; 2) The RPN network is used to generate the regional suggestion box. The layer judges whether the suggestion frame is correct or not through a softmax loss function, and then corrects the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame. 3) Pooling (roi pooling) of the region of interest. The layer is used for collecting input characteristics and suggestion frames, and after integrating the information, the information is sent to a subsequent full-connection layer to judge the target category; 4) And (5) classification. The full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and meanwhile, the final accurate position of the detection frame is obtained by carrying out bounding box regression again. The formula is as follows:wherein x is i Is input data of the neural network, w ij B for the weight value of the input data to the output data i For the corresponding deviation value, y i Is the corresponding output value.
It should be noted that, in S103, the method for verifying and updating the parameters of the fast-rcnn+res net network model by the test image set and training again, and cycling the process until the final sperm nucleoprotein detection model is obtained includes:
on the trained sperm nucleoprotein detection model, training a region suggestion network for the first time to obtain the region suggestion network trained for the first time;
collecting a suggestion frame by utilizing the first trained regional suggestion network and the test image set;
training a fast-rcnn network for the first time;
the second training area suggests a network to obtain a second trained RPN network;
collecting a suggestion frame by utilizing the region suggestion network trained for the second time and the test image set;
the fast-rcnn network is trained a second time.
Specifically, corresponding training codes are operated, the image set is tested for training, and a trained final sperm nucleoprotein detection model is finally obtained. The method comprises the following steps: 1) Training an RPN network on the trained model; 2) Collecting proposals (suggestion boxes) by utilizing the RPN network trained in the step 1); 3) Training a fast-rcnn network for the first time; 4) Training the RPN network for the second time; 5) Collecting proposals by using the RPN network trained in the step 4); 6) The fast-rcnn network is trained a second time. The above steps are the process of processing images inside the network during the training process by the fast-rcnn.
The principle of the application is to detect the sperm nucleoprotein based on a neural network model so as to identify the maturity of the sperm nucleoprotein. Firstly, marking sperm nucleoprotein in a photographed sperm nucleoprotein staining microscopic image, making a data format required by training, then training the marked data through a neural network model to obtain a final sperm nucleoprotein detection model, finally detecting sperm nucleoprotein through the sperm nucleoprotein detection model, and analyzing the maturity of the sperm nucleoprotein through a detection result. Compared with the traditional target detection algorithm, the recognition rate and accuracy of the sperm nucleoprotein maturity are greatly improved.
As shown in fig. 4, the present application further provides a sperm nucleoprotein maturity detection device, comprising:
a sperm image acquisition module 21 for acquiring sperm nucleoprotein staining microscopy images;
the nucleoprotein detection module 22 is used for passing the sperm nucleoprotein staining microscopic image through a sperm nucleoprotein detection model to obtain a sperm nucleoprotein image, position information of the sperm nucleoprotein in the image, and a detection result of sperm nucleoprotein maturation or immature.
The sperm nucleoprotein maturity detection device further comprises: a detection model generation module 20 for generating a sperm nucleoprotein detection model; the detection model generation module 20 includes:
the sample data screening unit is used for screening sperm nucleoprotein staining microscopic images of large samples;
the sample data labeling unit is used for labeling sperm nucleoprotein maturation or immature sperm nucleoprotein on the sperm nucleoprotein staining microscopic image and dividing the labeled sperm nucleoprotein staining microscopic image into a training image set and a testing image set;
the detection model training unit is used for training the training image set by adopting a Faster-RCNN+ResNet network model under the TensorFlow framework, verifying and updating Faster-RCNN+ResNet network model parameters by the test image set, training again, and cycling the process until a final sperm nucleoprotein detection model is obtained.
It should be noted that, the method for training the training image set by the detection model training unit by adopting the fast-RCNN+ResNet network model under the TensorFlow framework includes:
adjusting the network structure of Faster-rcnn, replacing the basic network with a ResNet50 network, and adjusting corresponding training parameters: learning rate, number of training images, number of iterations, selection of optimizers, and selection of loss functions;
extracting the features of the training image set by using a group of basic conv+relu+pooling layers, and sharing the extracted features to a regional suggestion network layer and a full connection layer for processing;
the regional suggestion network layer is used for generating a regional suggestion frame, judging whether the suggestion frame is correct or not through a softmax loss function, and correcting the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame;
the interesting area pooling layer is used for collecting input characteristics and suggestion frames, synthesizing the information and then sending the information to the subsequent full-connection layer to judge the target category;
the full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and simultaneously obtains the final accurate position of the detection frame by using the bounding box regression again, wherein the formula is as follows:wherein x is i Input data for a Faster-RCNN+ResNet network, w ij B for the weight value of the input data to the output data i For the corresponding deviation value, y i Is the corresponding output value.
It should be noted that, the test model training unit verifies and updates the parameters of the Faster-RCNN+ResNet network model through the test image set, and then trains, and the method for cycling the process until the final sperm nucleoprotein test model is obtained comprises the following steps:
on the trained sperm nucleoprotein detection model, training a region suggestion network for the first time to obtain the region suggestion network trained for the first time;
collecting a suggestion frame by utilizing the first trained regional suggestion network and the test image set;
training a fast-rcnn network for the first time;
the second training area suggests a network to obtain a second trained RPN network;
collecting a suggestion frame by utilizing the region suggestion network trained for the second time and the test image set;
the fast-rcnn network is trained a second time.
The application also provides a sperm nucleoprotein maturity detection device, comprising: a memory for storing a computer program; a processor for executing a computer program to perform the steps of the sperm nucleoprotein maturity detection method described above.
The description of the features of the embodiment corresponding to fig. 4 may be referred to the related description of the embodiment corresponding to fig. 1-3, and will not be repeated here.
The method and the device for detecting the maturity of the sperm nucleoprotein provided by the embodiment of the application are described in detail. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the application can be made without departing from the principles of the application and these modifications and adaptations are intended to be within the scope of the application as defined in the following claims.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Claims (8)
1. A method for detecting sperm nucleoprotein maturity comprising:
s10: screening out sperm nucleoprotein staining microscopic images of large samples, marking sperm nucleoprotein maturation or immature sperm nucleoprotein on the sperm nucleoprotein staining microscopic images, dividing the marked sperm nucleoprotein staining microscopic images into a training image set and a test image set, training the training image set by adopting a Faster-RCNN+ResNet network model under a TensorFlow framework, verifying and updating Faster-RCNN+ResNet network model parameters by the test image set, training again, and cycling the process until a final sperm nucleoprotein detection model is obtained;
s11: obtaining a sperm nucleoprotein staining microscopic image;
s12: and (3) passing the sperm nucleoprotein staining microscopic image through a sperm nucleoprotein detection model to obtain a sperm nucleoprotein image, position information of the sperm nucleoprotein in the image and a detection result of sperm nucleoprotein maturation or immature.
2. The method for detecting sperm nucleoprotein maturity according to claim 1, wherein said training image set using a fast-rcnn+res net network model under a TensorFlow framework comprises:
adjusting the network structure of Faster-rcnn, replacing the basic network with a ResNet50 network, and adjusting corresponding training parameters: learning rate, number of training images, number of iterations, selection of optimizers, and selection of loss functions;
extracting the features of the training image set by using a group of basic conv+relu+pool layers, and sharing the extracted features to a regional suggestion network layer and a full connection layer for processing;
the regional suggestion network layer is used for generating a regional suggestion frame, judging whether the suggestion frame is correct or not through a softmax loss function, and correcting the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame;
the interesting area pooling layer is used for collecting input characteristics and suggestion frames, synthesizing the information and then sending the information to the subsequent full-connection layer to judge the target category;
the full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and simultaneously obtains the final accurate position of the detection frame by using the bounding box regression again, wherein the formula is as follows:wherein x is i Input data for a Faster-RCNN+ResNet network, w ij B, for inputting data to corresponding weight value on output data j For the corresponding deviation value, y j I= (1, 2, 3 … … n), j= (1, 2) for the corresponding output value.
3. The method of claim 1, wherein the step of verifying and updating parameters of the fast-rcnn+resnet network model by using the test image set and training the parameters, and the step of cycling the process until a final sperm nucleoprotein detection model is obtained comprises:
on the trained sperm nucleoprotein detection model, training a region suggestion network for the first time to obtain the region suggestion network trained for the first time;
collecting a suggestion frame by utilizing the first trained regional suggestion network and the test image set;
training a fast-rcnn network for the first time;
the second training area suggests a network to obtain a second trained RPN network;
collecting a suggestion frame by utilizing the region suggestion network trained for the second time and the test image set;
the fast-rcnn network is trained a second time.
4. A sperm nucleoprotein maturity detection device, comprising:
the detection model generation module is used for screening out sperm nucleoprotein staining microscopic images of large samples, marking sperm nucleoprotein maturation or immature sperm nucleoprotein on the sperm nucleoprotein staining microscopic images, dividing the marked sperm nucleoprotein staining microscopic images into a training image set and a test image set, training the training image set by adopting a Faster-RCNN+ResNet network model under a TensorFlow framework, verifying and updating Faster-RCNN+ResNet network model parameters by the test image set, and training again, and cycling the process until a final sperm nucleoprotein detection model is obtained;
the sperm image acquisition module is used for acquiring sperm nucleoprotein staining microscopic images;
the nucleoprotein detection module is used for passing the sperm nucleoprotein staining microscopic image through a sperm nucleoprotein detection model to obtain a sperm nucleoprotein image, position information of the sperm nucleoprotein in the image and a detection result of sperm nucleoprotein maturation or immature.
5. The sperm nucleoprotein maturity detection device as described in claim 4, wherein said detection model generation module comprises:
the sample data screening unit is used for screening sperm nucleoprotein staining microscopic images of large samples;
the sample data labeling unit is used for labeling sperm nucleoprotein maturation or immature sperm nucleoprotein on the sperm nucleoprotein staining microscopic image and dividing the labeled sperm nucleoprotein staining microscopic image into a training image set and a testing image set;
the detection model training unit is used for training the training image set by adopting a Faster-RCNN+ResNet network model under the TensorFlow framework, verifying and updating Faster-RCNN+ResNet network model parameters by the test image set, training again, and cycling the process until a final sperm nucleoprotein detection model is obtained.
6. The sperm nucleoprotein maturity detection device as described in claim 5, wherein said detection model training unit is configured to train a training image set using a Faster-rcnn+resnet network model under a TensorFlow framework, said method comprising:
adjusting the network structure of Faster-rcnn, replacing the basic network with a ResNet50 network, and adjusting corresponding training parameters: learning rate, number of training images, number of iterations, selection of optimizers, and selection of loss functions;
extracting the features of the training image set by using a group of basic conv+relu+pool layers, and sharing the extracted features to a regional suggestion network layer and a full connection layer for processing;
the regional suggestion network layer is used for generating a regional suggestion frame, judging whether the suggestion frame is correct or not through a softmax loss function, and correcting the suggestion frame by utilizing bounding box regression to accurately acquire the suggestion frame;
the interesting area pooling layer is used for collecting input characteristics and suggestion frames, synthesizing the information and then sending the information to the subsequent full-connection layer to judge the target category;
the full-connection layer calculates the category of the suggestion frame by using the suggestion frame obtained by the interesting area pooling layer, namely the category of whether sperm nucleoprotein is mature or not, and simultaneously obtains the final accurate position of the detection frame by using the bounding box regression again, wherein the formula is as follows:wherein x is i Input data for a Faster-RCNN+ResNet network, w ij B, for inputting data to corresponding weight value on output data i For the corresponding deviation value, y i I= (1, 2, 3 … … n), j= (1, 2) for the corresponding output value.
7. The sperm nucleoprotein maturity detection device as described in claim 5, wherein said detection model training unit verifies and updates the parameters of the Faster-RCNN+ResNet network model by testing the image set and trains again, and the method of cycling the process until a final sperm nucleoprotein detection model is obtained comprises:
on the trained sperm nucleoprotein detection model, training a region suggestion network for the first time to obtain the region suggestion network trained for the first time;
collecting a suggestion frame by utilizing the first trained regional suggestion network and the test image set;
training a fast-rcnn network for the first time;
the second training area suggests a network to obtain a second trained RPN network;
collecting a suggestion frame by utilizing the region suggestion network trained for the second time and the test image set;
the fast-rcnn network is trained a second time.
8. A sperm nucleoprotein maturity detection device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the sperm nucleoprotein maturity detection method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011216686.XA CN112200803B (en) | 2020-11-04 | 2020-11-04 | Sperm nucleoprotein maturity detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011216686.XA CN112200803B (en) | 2020-11-04 | 2020-11-04 | Sperm nucleoprotein maturity detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112200803A CN112200803A (en) | 2021-01-08 |
| CN112200803B true CN112200803B (en) | 2023-10-10 |
Family
ID=74033201
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011216686.XA Active CN112200803B (en) | 2020-11-04 | 2020-11-04 | Sperm nucleoprotein maturity detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112200803B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861319A (en) * | 2023-02-28 | 2023-03-28 | 中国科学院长春光学精密机械与物理研究所 | Cumulus cell complex maturity analysis method, device, equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107492095A (en) * | 2017-08-02 | 2017-12-19 | 西安电子科技大学 | Medical image pulmonary nodule detection method based on deep learning |
| CN110458821A (en) * | 2019-08-07 | 2019-11-15 | 屈晨 | A kind of sperm morphology analysis method based on deep neural network model |
| WO2020090947A1 (en) * | 2018-10-31 | 2020-05-07 | 合同会社みらか中央研究所 | Program, learning model, information processing device, information processing method, information display method, and method for producing learning model |
| CN111402232A (en) * | 2020-03-16 | 2020-07-10 | 深圳市瑞图生物技术有限公司 | Method for detecting sperm aggregation in semen |
| WO2020181685A1 (en) * | 2019-03-12 | 2020-09-17 | 南京邮电大学 | Vehicle-mounted video target detection method based on deep learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7907769B2 (en) * | 2004-05-13 | 2011-03-15 | The Charles Stark Draper Laboratory, Inc. | Image-based methods for measuring global nuclear patterns as epigenetic markers of cell differentiation |
| CN108694401B (en) * | 2018-05-09 | 2021-01-12 | 北京旷视科技有限公司 | Target detection method, device and system |
-
2020
- 2020-11-04 CN CN202011216686.XA patent/CN112200803B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107492095A (en) * | 2017-08-02 | 2017-12-19 | 西安电子科技大学 | Medical image pulmonary nodule detection method based on deep learning |
| WO2020090947A1 (en) * | 2018-10-31 | 2020-05-07 | 合同会社みらか中央研究所 | Program, learning model, information processing device, information processing method, information display method, and method for producing learning model |
| WO2020181685A1 (en) * | 2019-03-12 | 2020-09-17 | 南京邮电大学 | Vehicle-mounted video target detection method based on deep learning |
| CN110458821A (en) * | 2019-08-07 | 2019-11-15 | 屈晨 | A kind of sperm morphology analysis method based on deep neural network model |
| CN111402232A (en) * | 2020-03-16 | 2020-07-10 | 深圳市瑞图生物技术有限公司 | Method for detecting sperm aggregation in semen |
Non-Patent Citations (3)
| Title |
|---|
| 人类精子核成熟度检测及其临床意义;谷龙杰;陈振文;;国际生殖健康/计划生育杂志(第04期);全文 * |
| 基于卷积神经网络的目标检测与识别;王高峰;徐子同;卢玮;王翠翠;高涛;;计算机与数字工程(第02期);全文 * |
| 精子功能检测与男性不育诊治的新进展;刘德一;H.W.Gordon Baker;;中华男科学杂志(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112200803A (en) | 2021-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111448582B (en) | System and method for single channel whole cell segmentation | |
| CN103528617B (en) | A kind of cockpit instrument identifies and detection method and device automatically | |
| CN107004123A (en) | Iterative defect filters out technique | |
| CN108615046A (en) | A kind of stored-grain pests detection recognition methods and device | |
| CN108830332A (en) | A kind of vision vehicle checking method and system | |
| CN110161233B (en) | Rapid quantitative detection method for immunochromatography test paper card | |
| CN110045015A (en) | A kind of concrete structure Inner Defect Testing method based on deep learning | |
| CN104680185B (en) | Hyperspectral image classification method based on boundary point reclassification | |
| CN112200803B (en) | Sperm nucleoprotein maturity detection method and device | |
| CN117152152A (en) | Production management system and method for detection kit | |
| US9785848B2 (en) | Automated staining and segmentation quality control | |
| CN113205511B (en) | Electronic component batch information detection method and system based on deep neural network | |
| CN108805181B (en) | Image classification device and method based on multi-classification model | |
| CN112304229A (en) | Automatic analysis method and system for textile fiber components | |
| CN117576195A (en) | Plant leaf morphology recognition method | |
| US11727673B1 (en) | Visual analysis method for cable element identification | |
| CN109886923A (en) | It is a kind of for internet detection in measurement detection system and method based on machine learning | |
| WO2018131091A1 (en) | Image processing device, image processing method, and image processing program | |
| CN115494062A (en) | Printing method for identifying defects based on machine vision | |
| CN113218998A (en) | Eddy current thermal imaging defect identification method based on global Moran index | |
| CN112184708B (en) | Sperm survival rate detection method and device | |
| CN111896456A (en) | Single cell analysis method based on micro-fluidic and hyperspectral imaging | |
| CN108764367A (en) | A kind of characteristic image extraction element and extracting method based on relationship regularization | |
| CN109543696A (en) | A kind of image-recognizing method neural network based and its application | |
| CN118052814B (en) | AI technology-based full-automatic specimen pretreatment system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |