CN111612744A - Curved chromosome image straightening model generation method, application of model, system, readable storage medium and computer equipment - Google Patents
Curved chromosome image straightening model generation method, application of model, system, readable storage medium and computer equipment Download PDFInfo
- Publication number
- CN111612744A CN111612744A CN202010360505.4A CN202010360505A CN111612744A CN 111612744 A CN111612744 A CN 111612744A CN 202010360505 A CN202010360505 A CN 202010360505A CN 111612744 A CN111612744 A CN 111612744A
- Authority
- CN
- China
- Prior art keywords
- chromosome
- image
- map
- curved
- skeleton
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000000349 chromosome Anatomy 0.000 title claims abstract description 362
- 238000000034 method Methods 0.000 title claims abstract description 84
- 238000010606 normalization Methods 0.000 claims abstract description 60
- 238000005452 bending Methods 0.000 claims abstract description 46
- 238000012549 training Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 17
- 230000006870 function Effects 0.000 claims description 64
- 239000012535 impurity Substances 0.000 claims description 18
- 230000003416 augmentation Effects 0.000 claims description 16
- 238000013507 mapping Methods 0.000 claims description 13
- 238000013434 data augmentation Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 7
- 230000003190 augmentative effect Effects 0.000 claims description 6
- 230000005489 elastic deformation Effects 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 230000003042 antagnostic effect Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000003321 amplification Effects 0.000 abstract description 3
- 230000008485 antagonism Effects 0.000 abstract description 3
- 238000003199 nucleic acid amplification method Methods 0.000 abstract description 3
- 238000013136 deep learning model Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 36
- 238000000605 extraction Methods 0.000 description 10
- 239000003550 marker Substances 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000004913 activation Effects 0.000 description 5
- 230000003247 decreasing effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 230000031864 metaphase Effects 0.000 description 3
- 210000004027 cell Anatomy 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 210000000805 cytoplasm Anatomy 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 210000003463 organelle Anatomy 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 208000031404 Chromosome Aberrations Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000003855 cell nucleus Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 230000002559 cytogenic effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000007850 fluorescent dye Substances 0.000 description 1
- 230000009395 genetic defect Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 210000003917 human chromosome Anatomy 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 210000004940 nucleus Anatomy 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
- G06T2207/10061—Microscopic image from scanning electron microscope
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10064—Fluorescence image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for generating a straightening model of a curved chromosome image, application of the model, a system, a readable storage medium and computer equipment, wherein the method for generating the straightening model comprises the following steps of obtaining an original curved chromosome image and processing the original curved chromosome image to obtain a chromosome image; generating a chromosome zoom map and a chromosome bending skeleton map according to the chromosome map; and carrying out data amplification on the chromosome zoom map and the chromosome curved skeleton map, and carrying out normalization processing on the amplified chromosome zoom map and the chromosome curved skeleton map to obtain a training data set, and inputting the training data set into a curved chromosome image straightening model to train until the model converges. According to the scheme, the chromosome image features are extracted on the premise of retaining chromosome detail information based on a deep learning model, local and global features of the image are self-learned by a antagonism training method, and a generation mode of a corresponding chromosome map is generated by a skeleton map, so that the chromosome is flexibly straightened, and a feasible processing tool with high accuracy is provided for straightening of a curved chromosome image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a curved chromosome image straightening model generation method based on a BagPix2Pix self-learning model, application of the model, a system, a readable storage medium and computer equipment.
Background
With the development of the electronic computer technology and the field of artificial intelligence, the recognition and segmentation of images by computer algorithms can be preliminarily realized, and the image analysis technology is generally used for detecting, separating and extracting objects in medical images medically. However, when the detected object is in a different form, such as a curve and an overlap, the graphics algorithm often cannot change the form of the object on a pixel level.
Human genetic information is carried on chromosomes, so that chromosome karyotyping is a basic method for cytogenetics research and is an important means for researching the connection between chromosome morphology and structure and functions and researching the association between chromosome abnormalities and genetic defects. However, due to the nature of the chromosome itself, the chromosome is often in a curved form. Compared with the chromosome in the vertical form, the chromosome in the curved form often brings obstacles to chromosome identification, classification and functional research, and manual analysis is time-consuming and labor-consuming. The existing chromosome straightening technology is based on algorithms of cutting, splicing and other graphics, and has the defects that the image cannot be flexibly straightened at a pixel level, and obvious cutting traces and blanks at the spliced position can be generated.
For example, document "a method for straightening highly curved chromosome map" Roshtkhari, m.j. and setarehdan, s.k.,2008.a novel algorithm for straightening highly curved image of human chromosome, pattern recognition readers, 29(9), pp.1208 — 1217 "discloses a means for straightening a chromosome by locating the maximum curve and rotating and splicing both ends after cutting at the maximum curve. The method is suitable for chromosome with only one large curve, and chromosome photos taken in different experiments and fineness often have three or more curves, so that the algorithm often has straightening errors, and the blank of cutting and splicing can also make chromosome information discontinuous, so that the use requirement can not be met.
Disclosure of Invention
One of the objectives of the present invention is to provide a curved chromosome image straightening model generation method based on a BagPix2Pix self-learning model, based on a convolutional neural network in deep learning, to extract a skeleton map of a chromosome, which is subjected to elastic deformation, random rotation, and other expansions, and high-altitude features of a corresponding chromosome map, and to learn a model to a mode for generating the corresponding chromosome map from the skeleton by using a countertraining method, thereby providing a feasible processing tool for flexibly generating a chromosome straightening map.
The purpose of the invention is realized by the following technical scheme:
a method for generating a curved chromosome image straightening model comprises the following steps,
s10, obtaining an original image of the bent chromosome, and preprocessing the original image to obtain a chromosome image with impurities removed;
s20, generating a chromosome zoom map according to the chromosome map;
s30, generating a chromosome bending skeleton map according to the chromosome map;
s40, carrying out data augmentation of the chromosome zoom map and the chromosome bending skeleton map;
s50, normalizing the chromosome zoom map and the chromosome bending skeleton map obtained in the step S40 to obtain a training data set;
and S60, inputting the training data set obtained in the step S50 into a curved chromosome image straightening model for training until the model converges.
Preferably, in the method for generating a curved chromosome image straightening model, the S10 includes:
s101, acquiring k curved chromosome images shot by a microscope, and converting the k curved chromosome images into a gray scale image;
s102, marking each gray scale image obtained in the S101, marking a chromosome part in the image as a 1-value pixel, and marking impurity and background parts as 0-value pixels to obtain a marked image;
s103, keeping the 1-value pixels in each label graph in the primary color, and turning the 0-value pixels into black.
Preferably, in the method for generating a curved chromosome image straightening model, the S20 includes:
s201, calculating the length and width of the chromosome part in the chromosome map and judging the length, width and the size of a set value a, if the length and width are smaller than the set value a, executing S202; if at least one of the length and the width is larger than a, executing S203;
s202, with the chromosome as the center, expanding the black area of the chromosome picture to the size of a multiplied by a pixels;
and S203, scaling the larger one of the length and the width to a, scaling the other one, and filling the part which is less than the length of a into black.
Preferably, in the method for generating a curved chromosome image straightening model, the S30 includes:
s301, recording a first pixel and a last pixel which belong to a chromosome part in each row of the chromosome map;
s302, calculating coordinates (X, Y) of a middle point of a first pixel and a last pixel belonging to a chromosome part of each line;
s303, averagely dividing all the solved Y values into n equal parts to form n equal division points;
s304, after the equal division, dividing the division points except the first division point and the last division point into skeleton anchor points of the bent chromosome;
s305, newly building a black background picture with the size of a multiplied by a, and positioning the skeleton anchor point determined in the step S304 on the picture;
and S306, sequentially connecting the skeleton anchor points in the S305 by using line segments with certain widths to obtain a chromosome bending skeleton map.
Preferably, in the method for generating a curved chromosome image straightening model, in S306, the pixel values of the plurality of line segments sequentially increase or decrease.
Preferably, in the method for generating the curved chromosome image straightening model, if the chromosome bending amplitude is too large or the skeleton anchor point is positioned outside the chromosome, the target number of points are artificially marked as the skeleton anchor point.
Preferably, in the method for generating a curved chromosome image straightening model, the S40 includes:
s401, carrying out chromosome zooming and chromosome bending skeleton map data augmentation by adopting an elastic deformation method and a random rotation method;
s402, repeating S401 to form m pairs of augmented chromosome zoom maps and chromosome bending skeleton maps.
Preferably, in the method for generating the curved chromosome image straightening model, the rotation angle of the random rotation method is between-45 ° and 45 °.
Preferably, in the method for generating a curved chromosome image straightening model, the S50 includes:
s501, normalizing the chromosome zoom map and the chromosome bending skeleton map obtained in the step S40 after the augmentation;
and S502, randomly dividing the data subjected to the normalization processing into two groups, wherein one group is used as a training set, and the other group is used as a verification set.
Preferably, in the method for generating a curved chromosome image straightening model, the S60 includes:
s601, extracting local and global characteristics of the picture by using a convolution kernel in the self-learning model in a sliding mode line by line on the input picture according to step length;
s602, carrying out batch normalization operation on the feature map generated in the S601;
s603, carrying out nonlinear mapping on the features extracted in the S602;
s604, adding random noise to the features extracted in the S603;
s605, keeping the same latitude of the feature graphs to be connected in series;
s606, deepening the depth of the input feature mapping image in an up-sampling mode;
s607, reducing the image resolution and detail features in a deconvolution mode, and connecting the reduced image resolution and detail features with the image features extracted by the convolution kernel under the same dimensionality in series;
s608, after the last layer of convolution layer of the model generator, using a tanh function to obtain an output graph generated after the input graph passes through the generator in a nonlinear mapping mode;
s609, the antagonistic neural network training is carried out by the method of the minimization generator (G) and the maximization discriminator (D),
wherein x isBAs an input skeleton diagram, yBFor the input chromosome map, z is the random noise input by the Dropout function.
Preferably, in the method for generating the curved chromosome image straightening model, the curved chromosome image straightening model includes a generator and a discriminator; the generator includes an encoding path and a decoding path.
Another objective of the present invention is to provide an application method of the above curved chromosome image straightening model, including the following steps:
a curved chromosome image straightening model obtained according to any one of the methods;
according to the received original image of the bent chromosome, a chromosome straightening skeleton map is constructed;
and inputting the chromosome straightening skeleton map into a bent chromosome image straightening model, and outputting a chromosome straightening image by the bent chromosome image straightening model.
It is still another object of the present invention to provide a generation system of a curved chromosome image straightening model for implementing the above generation method, which includes
The preprocessing unit is used for acquiring an original image of the bent chromosome and preprocessing the original image to obtain a chromosome image with impurities removed;
a zoom map generation unit for generating a chromosome zoom map from the chromosome map;
a curved skeleton map generation unit for generating a chromosome curved skeleton map from the chromosome map;
the augmentation unit is used for carrying out data augmentation of the chromosome zoom map and the chromosome bending skeleton map;
the normalization unit is used for carrying out normalization processing on the chromosome zoom map and the chromosome bending skeleton map after the augmentation, which are obtained by the augmentation unit, so as to obtain a training data set;
and the training unit is used for inputting the training data set into the bent chromosome image straightening model to train until the model converges.
A readable storage medium storing a program for implementing any one of the above-described curved chromosome image straightening model generation methods.
A final objective of the present solution is to provide a computer device, which includes a memory, a processor and a communication bus; the communication bus is used for realizing connection communication between the processor and the memory; the processor is configured to execute a program stored in the memory to implement any one of the above-described curved chromosome image straightening model generation methods.
The technical scheme of the invention has the advantages that:
according to the scheme, the augmentation data set can be accurately constructed through the single chromosome map, the straightened chromosome map is accurately generated through the straightening framework, and the detail information of the chromosome is reserved, so that the flexible straightening target of the human bent chromosome is finished at high quality.
The method is based on the deep learning model, can extract the chromosome image characteristics on the premise of retaining chromosome detail information, self-learns the local and global characteristics of the image and generates a corresponding chromosome map by a skeleton map by a antagonism training method, so as to flexibly straighten the chromosome, and provides a feasible and high-accuracy processing tool for straightening the curved chromosome image.
Compared with the original curved chromosome, the straightened chromosome can provide higher classification accuracy for a neural network, and assist a computer to construct a more accurate classification model and help medical staff to observe and distinguish abnormal and diseased regions in the chromosome more conveniently.
Drawings
FIG. 1 is a chromosome alignment map generated by a graphical algorithm described in the background art;
FIG. 2 is a schematic flow chart of a model application method of the present invention;
FIG. 3 is a labeled image of a chromosome which has been subjected to fluorescent staining under a metaphase microscope selected in S12 of the present invention;
fig. 4 is a chromosome image filtered out of the labeled image in S13;
FIG. 5 is a curved skeleton diagram of chromosome obtained in S2;
FIG. 6 is a skeleton diagram of chromosome straightening obtained in S3;
FIG. 7 is a curved chromosome straightening image obtained at S4;
FIG. 8 is a schematic flow chart of a model generation method of the present invention;
FIG. 9 is a zoom view of the chromosome obtained in S40 after the augmentation;
FIG. 10 is a curved skeleton diagram of the chromosome after the amplification obtained in S40;
FIG. 11 is a schematic diagram of a first embodiment of a generation system of the present solution;
fig. 12 is a schematic diagram of a second embodiment of the image straightening system according to the present scheme.
Detailed Description
Objects, advantages and features of the present invention will be illustrated and explained by the following non-limiting description of preferred embodiments. The embodiments are merely exemplary for applying the technical solutions of the present invention, and any technical solution formed by replacing or converting the equivalent thereof falls within the scope of the present invention claimed.
In the description of the schemes, it should be noted that the terms "center", "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the embodiment, the operator is used as a reference, and the direction close to the operator is a proximal end, and the direction away from the operator is a distal end.
The curved chromosome image straightening method based on the BagPix2Pix self-learning model disclosed by the invention is explained with reference to the accompanying drawings, as shown in FIG. 2, which comprises the following steps,
s1, receiving the original image of the bent chromosome, and processing the original image to obtain a labeled graph;
s2, generating a chromosome skeleton anchor point diagram according to the marker diagram obtained in S1;
s3, generating a chromosome straightening skeleton map according to the chromosome skeleton anchor point map;
and S4, inputting the chromosome straightening skeleton diagram into a bent chromosome image straightening model which is converged by training and can receive the skeleton diagram and output a matched prediction diagram, and generating a bent chromosome straightening image.
The method is based on the fact that a curved chromosome image straightening model is formed, a straightening skeleton diagram of a curved chromosome is obtained after a certain collected curved chromosome image is subjected to certain processing, then the straightening skeleton diagram is input into the curved chromosome image straightening model, and the model can restore the constructed straightening skeleton with the same length into the straightened chromosome through a generator of the model, so that the curved chromosome straightening diagram is obtained.
Specifically, the processing procedure of S1 is as follows:
and S11, acquiring the curved chromosome image shot by the microscope and converting the curved chromosome image into a gray scale image. In order to obtain a real chromosome image, chromosomes in the metaphase of cells need to be fluorescently labeled, the chromosomes are selected by a microscope under a proper multiple to be shot, the image is introduced into a computer, and the computer receives the curved chromosome image shot by the microscope and then converts the curved chromosome image into a gray scale image. If the chromosome is contacted or overlapped with other chromosomes, the chromosome is cut and extracted according to the original edge shape.
S12, labeling the chromosome, the impurity and the background in the gray-scale image obtained in S11, labeling the chromosome in the image as 1-value pixel, labeling the impurity and the background as 0-value pixel, and obtaining a labeled image, as shown in fig. 3, wherein the impurity includes but is not limited to cytoplasm, nucleus and other organelles.
Further, the step S12 may be followed by step S13, where the chromosome image is extracted according to the marker map of S12, that is, the marker map is subjected to a reverse color processing, 1-value pixels (chromosomes) in the marker map are kept in a primary color, 0-value pixels (impurities and background portions) are changed into black, so as to remove the impurities, and a chromosome map including only the chromosome portion and a single color background (black background) is extracted, as shown in fig. 4. This step can be used to subsequently develop the scaling map required to train the model.
After obtaining the marker map, a curved skeleton map of the curved chromosome needs to be constructed, and the method specifically comprises the following steps:
s21, recording the first 1-value pixel and the last 1-value pixel of each line of the label map obtained in S12 (i.e., respectively recording the first 1-value pixel and the last 1-value pixel of each line from the first line to the last line of an image).
S22, the coordinates (X, Y) of the intermediate point between the first 1-value pixel and the last 1-value pixel in each line are obtained, i.e., a plurality of intermediate points are obtained.
At S23, the Y values of the coordinates of all the obtained intermediate points are equally divided into n equal parts, for example, 12 equal parts, to obtain 12 equal divided points.
And S24, dividing the equally divided equipartition points except the first equipartition point and the last equipartition point into skeleton anchor points of the bent chromosome, namely removing the first equipartition point and the last equipartition point in the 12 points, and reserving 10 equipartition points between the first equipartition point and the last equipartition point as skeleton anchor points.
S25, creating a black background picture with the size of a multiplied by a (pixel value is 0), and positioning 10 skeleton anchor points determined in the step S24 on the picture to obtain a chromosome skeleton anchor point map.
The S2 may further include S26, connecting the skeleton anchors in S25 sequentially with line segments of a certain width to obtain a skeleton map of chromosome bending, preferably connecting the skeleton anchors with line segments of 10 pixels in width to finally form a skeleton map of chromosome bending with 9 line segments, as shown in fig. 5, and the pixel values of the 9 line segments gradually increase or gradually decrease, preferably sequentially increase from top to bottom, i.e. the pixel values of the 9 line segments are 46,69,92,115,138,161,184,207 sequentially, so as to record the bending and orientation information of the chromosome during model training as described below.
If the bending amplitude of the selected chromosome is too large or the skeleton anchor point is positioned outside the chromosome, the chromosome bending skeleton map is abandoned, the skeleton anchor point can be marked in an artificial marking mode, and then the chromosome bending skeleton map is constructed again.
After the chromosome skeleton anchor point diagram is constructed, a chromosome straightening skeleton diagram can be constructed according to the method, and the specific process is as follows:
said S3 includes
S31, creating a black background picture of a × a size;
s32, calculating the distance between two non-adjacent skeleton anchor points in the chromosome skeleton anchor point diagram, and taking the 10 skeleton anchor points as an example, finally obtaining 9 distance values.
S33, in the black background picture of S31, a plurality of line segments that are respectively equal in distance to a pair of adjacent skeleton anchor points and are centered in position are sequentially connected to form a chromosome straightening skeleton map, that is, 9 line segments that are in one-to-one correspondence with 9 distance values are sequentially connected to form a straightening skeleton map, and the width of each line segment of the chromosome straightening skeleton map is the same as the width of the line segment of the chromosome bending skeleton map and is 10 pixels, and the pixel values of the plurality of line segments of the chromosome straightening skeleton map can also be decreased or increased in a decreasing manner, preferably are in one-to-one correspondence with the pixel values of each line segment of the chromosome straightening skeleton map, as shown in fig. 6.
After the chromosome straightening skeleton map is obtained, the chromosome straightening skeleton map can be input into a curved chromosome image straightening model which is trained and converged, and for a newly input straightening skeleton, a generator of the convergence model predicts and generates a corresponding prediction picture of a true chromosome which is comparable to the reality, so that the curved chromosome straightening map is obtained, as shown in fig. 7.
From fig. 7 we can see that the straightening effect and local detail of the deeply learned model closely approximates the chromosome map as described in fig. 4. Compared with the prior straightening graph which is obtained by the method based on the image science and is shown in the attached figure 1, the method has the advantages that the result does not contain obvious fracture and tangent planes, the quality of the straightened chromosome image and the integrity and continuity of effective characteristics are greatly improved, and the method is not influenced by the number of bends and the positions of bends.
The scheme further discloses a method for generating the bent chromosome image straightening model and a method for generating the bent chromosome image straightening model, the bent chromosome image straightening model is based on a BagPix2Pix self-learning model, a convolutional neural network in deep learning is adopted to extract high-latitude characteristics of a chromosome skeleton diagram and a corresponding chromosome zoom diagram after being subjected to elastic deformation, random rotation and other augmentation, the model is made to learn a mode of generating the corresponding chromosome diagram from the skeleton through a countermeasure training method, and therefore the purpose that a corresponding prediction diagram of the flexible straightened chromosome is generated through a chromosome vertical skeleton is achieved.
The method for generating the curved chromosome image straightening model is specifically described as follows, and comprises the following steps:
s10, acquiring a set (e.g. k) of original images of the bent chromosomes, and processing each original image of the bent chromosomes to obtain k chromosome maps with impurities removed, which includes the following steps:
s101, acquiring k curved chromosome images shot by a microscope, and converting the k curved chromosome images into a gray-scale image. In order to obtain a real chromosome image, chromosomes in the metaphase of cells need to be fluorescently labeled, the chromosomes are selected by a microscope under a proper multiple to be shot, the image is introduced into a computer, and the computer receives the curved chromosome image shot by the microscope and then converts the curved chromosome image into a gray scale image. If the chromosome is contacted or overlapped with other chromosomes, the chromosome is cut and extracted according to the original edge shape.
And S102, in order to remove pollution of edge pixels, marking the k gray-scale maps obtained in the S101 respectively, marking chromosomes in each map as 1-value pixels, and marking impurities and background parts as 0-value pixels to obtain k marked maps (the marked maps can be used as chromosome maps in the process of generating chromosome bending skeleton maps, wherein the impurities comprise but are not limited to cytoplasm, cell nucleus and other organelles).
And S103, extracting the chromosome image according to the marker map obtained in S102, namely, keeping the chromosome part in the marker map in a primary color, converting the impurities and the background part into black, removing the impurities, obtaining a chromosome map (only containing the black background and the reverse color) only comprising the chromosome part and a single-color background (black), and obtaining k chromosome maps.
S20, generating k chromosome zoom maps according to each chromosome map, wherein the specific process of generating each chromosome zoom map is as follows:
s201, calculating the length and width of the chromosome part (non-black region) in the chromosome map obtained in S103, and determining the length, width and the magnitude of a set value a, where the set value a may be 256 or 512, and if the length and width of the chromosome part are both less than the set value a, executing S202; if at least one of the length or width of the chromosome portion is greater than a, S203 is performed.
S202, with the chromosome part as the center, the black area of the chromosome map is expanded to a pixel size of a x a, and an a x a image is obtained.
S203, zoom the larger one of the length and width to a, and the other one to scale, and fill the part less than a length to black to form an image of a × a.
S30, generating k chromosome bending skeleton maps according to the chromosome maps, wherein the specific process of generating each chromosome skeleton map is as follows:
s301, recording a first pixel and a last pixel which belong to a chromosome part in each row of the chromosome map; since the chromosome, the impurity and the background part are marked in S102, and the chromosome part can be easily distinguished, the first 1-value pixel and the last 1-value pixel of each line of the marked map obtained in S102 can be recorded here, that is, the positions of the first 1-value pixel and the last 1-value pixel of each line are recorded from the first line to the last line of a marked map respectively.
S302, coordinates (X, Y) of intermediate points of the first pixel and the last pixel belonging to the chromosome part in each line of the chromosome map recorded in S301 are obtained, and when the first 1-value pixel and the last 1-value pixel recorded in the marker map are recorded, coordinates (X, Y) of intermediate points of the first 1-value pixel and the last 1-value pixel in each line are obtained, and a plurality of intermediate points are finally obtained.
S303, equally divide the Y values of the coordinates of all the obtained intermediate points into n equal parts, for example, into 12 equal parts, to obtain 12 equal division points.
S304, equally dividing the division points except the first division point and the last division point into skeleton anchor points of the bent chromosome, namely dividing 10 division points of the 12 division points after the first division point and the last division point are removed into skeleton anchor points.
S305, creating a new black background picture with a × a size (pixel value is 0), and positioning the 10 skeletal anchor points determined in the step S304 on the new background picture.
S306, connecting the skeleton anchors in the S305 sequentially by using line segments with certain width to obtain a chromosome bending skeleton map, preferably connecting 10 skeleton anchors sequentially by using line segments with the width of 10 pixels to finally form the chromosome bending skeleton map with 9 line segments, wherein the pixel values of the 9 line segments are gradually increased or gradually decreased, preferably are gradually increased from top to bottom, and more preferably the pixel values of the 9 line segments are 46,69,92,115,138,161,184 and 207 sequentially.
However, if the bending amplitude of the selected chromosome is too large or the skeleton anchor point is positioned outside the chromosome, the chromosome bending skeleton map is discarded, the skeleton anchor point can be marked by means of artificial marking, and then the chromosome bending skeleton map is reconstructed.
Through S20 and S30, K pairs of chromosome zoom maps and chromosome bending skeleton maps are obtained, namely one chromosome zoom map and the chromosome bending skeleton map corresponding to the same bent chromosome are a pair.
S40, performing data augmentation on the chromosome zoom map and the chromosome bending skeleton map by the K pair, wherein the data augmentation process of the chromosome zoom map and the chromosome bending skeleton map is as follows:
s401, carrying out chromosome zooming and chromosome bending skeleton map data augmentation by adopting an elastic deformation method and a random rotation method; for the image with a being 512, the parameters are selected to be sigma 25 and points 4; for an image with a being 256, the parameters are selected to be sigma 18 and points 3; and the random rotation angle range is between-45 degrees and 45 degrees.
The chromosome bending skeleton with the width can carry data augmentation information of elastic deformation and random rotation, and is used for the subsequent training of a BagPix2Pix self-learning model.
S402, repeating S401 to form m pairs of augmented chromosome scale maps (as shown in fig. 9) and chromosome curvature skeleton maps (as shown in fig. 10), and forming a set of training data for the chromosome, where m is 1000, for example, repeating S401 1000 times to generate 1000 pairs of augmented chromosome scale maps and chromosome curvature skeleton maps.
And finally obtaining k sets of process data for model training through the step S40, wherein each set of process data comprises 1000 pairs of augmented chromosome zoom maps and chromosome bending skeleton maps.
S50, carrying out normalization processing on the chromosome zoom map and the chromosome bending skeleton map after the S40 augmentation to obtain a training data set, wherein the training data set specifically comprises the following steps:
and S501, normalizing each set of the chromosome zoom map and the chromosome bending skeleton map obtained in the S40 after the amplification.
And S502, randomly dividing each set of m pairs of normalized data into two groups, wherein one group is used as a training set, the other group is used as a verification set, taking 1000 pairs of augmented chromosome zoom map and chromosome bending skeleton map data as an example, 800 pairs of data are adopted in the training set, and 200 pairs of data are adopted in the verification set.
And S60, inputting the training data set into the bent chromosome image straightening model to train until the model converges, inputting the training data set into the self-learning model, and then extracting local and global information to calculate the loss between the generated chromosome and the real chromosome and to perform countertraining between the generator and the discriminator in the self-learning model, so that the converged model can generate a prediction picture of the real chromosome comparable to the situation only through a skeleton diagram. In a classification task taking 642 chromosome pictures respectively trained to be converged and subjected to model straightening as a data set (because chromosomes with shorter lengths are not easy to bend, only chromosome numbers 1-7 are selected for the verification), in a 3:1 random cross verification mode, under model training of vgg16 and densenet121, the average accuracy of classification performance of verification set pictures after the same chromosome is straightened respectively reaches 93.28% and 86.41%, and compared with the classification performance of original bent chromosome pictures, the accuracy is respectively averagely improved by 1.40% and 2.82%. Therefore, the method can preliminarily show that the straightened chromosome can enable the classified neural network to achieve a better training result.
The specific training process of the BagPix2Pix self-learning model (curved chromosome image straightening model) is as follows:
s601, extracting local and global features of the picture by using a convolution kernel in the BagPix2Pix self-learning model in a sliding mode line by line on the input picture according to step length so as to keep details and space information in the local and global features.
And S602, performing batch normalization operation on the feature map generated in the S601 by using a batch normalization function, including but not limited to, so as to improve the training effect.
S603, the features extracted in S602 are nonlinearly mapped using activation functions including, but not limited to, leakyreu and ReLU to enhance the extraction effect on the features.
S604, random noise is added to the features extracted in S603 using a function including, but not limited to, Dropout.
S605, the feature maps to be concatenated are maintained at the same latitude using a method including, but not limited to, a zero-padding function.
And S606, deepening the depth of the input feature mapping image by using an up-sampling mode.
S607, reducing the image resolution and detail features by using a deconvolution mode, and connecting the reduced image resolution and detail features with the image features extracted by the convolution kernel under the same dimension in series, so that the detail features of the image are reserved to the maximum extent, and the difference between a chromosome image generated from the skeleton image and a real image is reduced.
S608, using the following tanh function after the last convolution layer of the model generator,
and obtaining an output graph generated after the input graph passes through the generator in a nonlinear mapping mode.
S609, the antagonistic neural network training is carried out by the methods of the minimization generator (G) and the maximization discriminator (D) to reduce the loss function,
wherein x isBAs an input skeleton diagram, yBFor the input chromosome map, z is the random noise input by Dropout function, and the formula here is known in the art and is not described in detail.
The BagPix2Pix self-learning model can extract local and global characteristics of high latitude of the picture, and trains the mapping relation between the skeleton generation and the corresponding chromosome in a counterstudy mode, so that the ability of generating a prediction graph of the straightened chromosome by a newly input straightened skeleton is achieved.
The BagPix2Pix self-learning model is based on a learning mode of countermeasure training, and is composed of a generator and a discriminator, wherein the generator is composed of an encoding path and a decoding path. The generator and the arbiter perform the antagonism training through the loss function described in S49.
The encoding path of the generator comprises 17 convolutional layers, 12 batch normalization functions, 16 LeakyReLU activation functions and 10 Dropout functions; the decoding path contains 14 deconvolution layers, 8 Dropout functions, 14 batch normalization functions, 14 ReLU activation functions, 5 zero-padding functions, 1 convolution layer, one upsampling layer, and one tanh activation function.
The coding path of the generator comprises two branches, wherein the connection mode of different layers and functions in the first branch is as follows: convolutional layer 1-LeakyReLU function-convolutional layer 2-batch normalization function-LeakyReLU function-convolutional layer 3-batch normalization function-LeakyReLU function-convolutional layer 4-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 5-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 6-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 7-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 8-LeakyReLU function-Dropout function.
The connection mode of different layers and functions of the decoding path of the first generator branch is as follows: inverse convolutional layer 1-batch normalization function-ReLU function-Dropout function-inverse convolutional layer 2-batch normalization function-ReLU function-Dropout function-inverse convolutional layer 3-batch normalization function-ReLU function-zero fill function-Dropout function-inverse convolutional layer 4-batch normalization function-ReLU function-Dropout function-inverse convolutional layer 5-batch normalization function-ReLU function-zero fill function-inverse convolutional layer 6-batch normalization function-ReLU function-inverse convolutional layer 7-batch normalization function-ReLU function.
The convolution layers 1 and 8 of the branch-one coding path both adopt convolution kernels of 3 multiplied by 3, and information extraction is carried out on input characteristics in a mode that the step length is 1 and zero padding is 1; the convolutional layers 2, 4 and 6 adopt 3 x 3 convolutional kernels, and information extraction is carried out on input features in a mode that the step length is 2 and zero padding is 1; the convolutional layers 3, 5, and 7 each use a 1 × 1 convolutional kernel, and perform information extraction on input features in the form of a step size of 1 and zero padding of 0.
The deconvolution layer 1 in the branch-one decoding path adopts a 3 x 3 deconvolution kernel, and the information is restored on the input characteristics in a mode that the step length is 1 and the zero padding is 1; the deconvolution layers 3, 5 and 7 adopt 3 x 3 deconvolution kernels, and information is restored on input characteristics in a mode of step length of 2 and zero padding of 1; the deconvolution layer 2, the deconvolution layer 4 and the deconvolution layer 6 all adopt 1 x 1 deconvolution kernels, and information reduction is carried out on input characteristics in a mode that the step length is 1 and zero padding is 0; the convolutional layer 9 performs feature extraction on the input feature map in the form of step length of 1 and zero padding of 1 by using a convolution kernel of 3 × 3.
The generator branches an input characteristic diagram and an output characteristic diagram of the convolutional layer 8 in an encoding path to be connected in series to serve as the input of the deconvolution layer 1; the output characteristic diagram is connected with the input of the convolution layer 7 in series after passing through a batch normalization function, a ReLU function and a Dropout function and is used as the input of the deconvolution layer 2; the output characteristic diagram is connected with the input of the convolution layer 6 in series after passing through a batch normalization function, a ReLU function and a Dropout function and is used as the input of the deconvolution layer 3; the output characteristic diagram is connected with the input of the convolution layer 5 in series after passing through a batch normalization function, a ReLU function and a Dropout function and is used as the input of the deconvolution layer 4; the output characteristic graph of the convolution function is serially connected with the input of the convolution layer 4 through the output of the batch normalization function and the output of the ReLU function to be used as the input of the deconvolution layer 5; the output characteristic graph is connected with the input of the convolution layer 3 in series after passing through a batch normalization function and a ReLU function and is used as the input of the deconvolution layer 6; the output characteristic diagram is connected with the input of the convolution layer 2 in series after passing through the batch normalization function and the ReLU function and is used as the input of the deconvolution layer 7.
The connection mode of different layers and functions in the second coding path branch of the generator is as follows: convolutional layer 9-LeakyReLU function-convolutional layer 10-batch normalization function-LeakyReLU function-convolutional layer 11-batch normalization function-LeakyReLU function-convolutional layer 12-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 13-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 14-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 15-batch normalization function-LeakyReLU function-Dropout function-convolutional layer 16-LeakyReLU function-Dropout function.
The connection mode of different layers and functions of the decoding path of the second generator branch is as follows: the deconvolution layer 8-batch normalization function-ReLU function-Dropout function-deconvolution layer 9-batch normalization function-ReLU function-Dropout function-deconvolution layer 10-batch normalization function-ReLU function-zero fill function-Dropout function-deconvolution layer 11-batch normalization function-ReLU function-Dropout function-deconvolution layer 12-batch normalization function-ReLU function-zero fill function-deconvolution layer 13-batch normalization function-ReLU function-deconvolution layer 14-batch normalization function-ReLU function-upsampling layer-connected with the characteristic diagram obtained by branching one decoding path-zero fill function-convolution layer 17-tanh function.
The convolutional layers 9 to 16 of the two-branch coding path all adopt 4 multiplied by 4 convolutional kernels, and information extraction is carried out on input characteristics in a mode that the step length is 2 and zero padding is 1;
the deconvolution layers 8 to 14 in the two-branch decoding path adopt 4 x 4 deconvolution kernels, and information is restored on the input characteristics in a mode that the step length is 2 and zero padding is 1; the convolutional layer 17 adopts 4 × 4 convolutional kernels, and performs feature extraction on the input feature mapping graph in a mode of step length of 1 and zero padding of 1; the upsampling layer deepens the depth of the input feature map by a multiple of 2; and the tanh function carries out the final nonlinear mapping operation of the generator on the input feature graph.
The generator branches the input characteristic diagram and the output characteristic diagram of the convolutional layer 16 in the two coding paths to be connected in series to be used as the input of the deconvolution layer 8; the output characteristic diagram is connected with the input of the convolution layer 15 in series after passing through a batch normalization function, a ReLU function and a Dropout function and is used as the input of the deconvolution layer 9; the output characteristic diagram is connected with the input of the convolution layer 14 in series after passing through a batch normalization function, a ReLU function and a Dropout function and is used as the input of the deconvolution layer 10; the output characteristic diagram is connected with the input of the convolution layer 13 in series after passing through a batch normalization function, a ReLU function and a Dropout function and is used as the input of the deconvolution layer 11; the output characteristic graph of the convolution function is serially connected with the input of the convolution layer 12 through the output of the batch normalization function and the ReLU function to be used as the input of the deconvolution layer 12; the output characteristic graph is connected with the input of the convolution layer 11 in series after passing through a batch normalization function and a ReLU function and is used as the input of the deconvolution layer 13; the output characteristic diagram is connected with the input of the convolution layer 10 in series after passing through the batch normalization function and the ReLU function and is used as the input of the deconvolution layer 14.
The depth of the feature mapping graph obtained after one convolutional layer and one deconvolution layer of the generator branch is 64, 128, 256, 512, 512, 512, 512, 512, 256, 128, 64 and 128 in sequence; the depth of the feature mapping image obtained after the convolution layer and the deconvolution layer of the branch two is 64, 128, 256, 512, 512, 512, 512, 512, 512, 512, 512, 512, 256, 128, 64 and 128 in sequence; the feature maps obtained by concatenating the output feature maps of branch one and branch two are 256, 1 in sequence.
The discriminator comprises 6 convolution layers, 5 batch normalization functions and 5 LeakyReLU activation functions; the connection mode of different layers and functions of the discriminator is as follows: convolution layer 1-batch normalization function-LeakyReLU function-convolution layer 2-batch normalization function-LeakyReLU function-convolution layer 3-batch normalization function-LeakyReLU function-convolution layer 4-batch normalization function-LeakyReLU function-convolution layer 5-batch normalization function-LeakyReLU function-convolution layer 6.
The convolution layers 1 and 5 of the discriminator both adopt convolution kernels of 3 multiplied by 3, and information extraction is carried out on input characteristics in a mode that the step length is 1 and zero padding is 1; the convolution layers 2, 3 and 4 adopt 3 x 3 convolution kernels, and information extraction is carried out on input features in a mode that the step length is 2 and zero padding is 1; the convolution layer 6 adopts a 1 × 1 convolution kernel, and performs information extraction on input features in a mode that the step length is 1 and zero padding is 0;
the depth of the feature map obtained by the discriminator after convolution is 64, 128, 256, 512, 1.
The scheme also discloses a generation system of the curved chromosome image straightening model, as shown in figure 11, which comprises
The preprocessing unit is used for acquiring an original image of the bent chromosome and processing the original image to obtain a chromosome image only with impurities removed;
a zoom map generation unit for generating a chromosome zoom map from the chromosome map;
a curved skeleton map generation unit for generating a chromosome curved skeleton map from the chromosome map;
the augmentation unit is used for carrying out data augmentation of the chromosome zoom map and the chromosome bending skeleton map;
the normalization unit is used for preprocessing the augmentation graph obtained by the augmentation unit to obtain a training data set;
and the training unit is used for inputting the training data set into the bent chromosome image straightening model for training until the model converges to obtain the bent chromosome image straightening model.
As shown in FIG. 12, the present embodiment further discloses a curved chromosome image straightening system comprising the preprocessing unit, the curved skeleton map generating unit, the zoom map generating unit, the augmentation unit, the normalization unit and the training unit, and the system further comprises
A straightened skeleton map generating unit used for generating a chromosome straightened skeleton map according to the chromosome skeleton anchor point map generated by the bent skeleton map generating unit;
and the straightening chromosome map generating unit is used for inputting the chromosome straightening skeleton map into a bent chromosome image straightening model which is converged by training and can receive a prediction map which can be output by the skeleton map and matched, and generating a bent chromosome straightening image.
The present disclosure merely discloses a readable storage medium storing a program for implementing any one of the foregoing curved chromosome image straightening methods and/or curved chromosome image straightening model generation methods based on the BagPix2Pix self-learning model.
The scheme also discloses computer equipment which comprises a memory, a processor and a communication bus; the communication bus is used for realizing connection communication between the processor and the memory; the processor is used for executing a software program stored in the memory to realize any one of the above-mentioned curved chromosome image straightening method and/or curved chromosome image straightening model generation method based on the BagPix2Pix self-learning model.
The invention has various embodiments, and all technical solutions formed by adopting equivalent transformation or equivalent transformation are within the protection scope of the invention.
Claims (15)
1. The method for generating the curved chromosome image straightening model is characterized by comprising the following steps of: comprises the following steps of (a) carrying out,
s10, obtaining an original image of the bent chromosome, and preprocessing the original image to obtain a chromosome image with impurities removed;
s20, generating a chromosome zoom map according to the chromosome map;
s30, generating a chromosome bending skeleton map according to the chromosome map;
s40, carrying out data augmentation of the chromosome zoom map and the chromosome bending skeleton map;
s50, normalizing the chromosome zoom map and the chromosome bending skeleton map obtained in the step S40 to obtain a training data set;
and S60, inputting the training data set obtained in the step S50 into a curved chromosome image straightening model for training until the model converges.
2. The curved chromosome image straightening model generation method according to claim 1, characterized in that: the S10 includes:
s101, acquiring k curved chromosome images shot by a microscope, and converting the k curved chromosome images into a gray scale image;
s102, marking each gray scale image obtained in the S101, marking a chromosome part in the image as a 1-value pixel, and marking impurity and background parts as 0-value pixels to obtain a marked image;
s103, keeping the 1-value pixels in each label graph in the primary color, and turning the 0-value pixels into black.
3. The curved chromosome image straightening model generation method according to claim 1, characterized in that: the S20 includes:
s201, calculating the length and width of the chromosome part in the chromosome map and judging the length, width and the size of a set value a, if the length and width are both smaller than the set value a, executing S202, and if at least one of the length or the width is larger than a, executing S203;
s202, with the chromosome as the center, expanding the black area of the chromosome picture to the size of a multiplied by a pixels;
and S203, scaling the larger one of the length and the width to a, scaling the other one, and filling the part which is less than the length of a into black.
4. The curved chromosome image straightening model generation method according to claim 1, characterized in that: the S30 includes:
s301, recording a first pixel and a last pixel which belong to a chromosome part in each row of the chromosome map;
s302, calculating coordinates (X, Y) of a middle point of a first pixel and a last pixel belonging to a chromosome part of each line;
s303, averagely dividing all the solved Y values into n equal parts to form n equal division points;
s304, after the equal division, dividing the division points except the first division point and the last division point into skeleton anchor points of the bent chromosome;
s305, newly building a black background picture with the size of a multiplied by a, and positioning the skeleton anchor point determined in the step S304 on the picture;
and S306, sequentially connecting the skeleton anchor points in the S305 by using line segments with certain widths to obtain a chromosome bending skeleton map.
5. The curved chromosome image straightening model generation method according to claim 4, characterized in that: in S306, the pixel values of the line segments sequentially increase or decrease.
6. The curved chromosome image straightening model generation method according to claim 4, characterized in that: if the chromosome bending amplitude is too large or the skeleton anchor points are positioned outside the chromosome, artificially marking the target number of points as the skeleton anchor points.
7. The curved chromosome image straightening model generation method according to claim 1, characterized in that: the S40 includes:
s401, carrying out chromosome zooming and chromosome bending skeleton map data augmentation by adopting an elastic deformation method and a random rotation method;
s402, repeating S401 to form m pairs of augmented chromosome zoom maps and chromosome bending skeleton maps.
8. The curved chromosome image straightening model generation method according to claim 7, characterized in that: the rotation angle of the random rotation method is between 45 degrees below zero and 45 degrees below zero.
9. The curved chromosome image straightening model generation method according to claim 1, characterized in that: the S50 includes:
s501, normalizing the chromosome zoom map and the chromosome bending skeleton map obtained in the step S40 after the augmentation;
and S502, randomly dividing the data subjected to the normalization processing into two groups, wherein one group is used as a training set, and the other group is used as a verification set.
10. The curved chromosome image straightening model generation method according to any one of claims 1 to 9, characterized in that: said S60 includes
S601, extracting local and global characteristics of the picture by using a convolution kernel in a BagPix2Pix self-learning model in a line-by-line sliding mode on the input picture according to step length;
s602, carrying out batch normalization operation on the feature map generated in the S601;
s603, carrying out nonlinear mapping on the features extracted in the S602;
s604, adding random noise to the features extracted in the S603;
s605, keeping the same latitude of the feature graphs to be connected in series;
s606, deepening the depth of the input feature mapping image in an up-sampling mode;
s607, reducing the image resolution and detail features in a deconvolution mode, and connecting the reduced image resolution and detail features with the image features extracted by the convolution kernel under the same dimensionality in series;
s608, the last layer of convolution layer of the model generator is usedtanhA function, which obtains an output graph generated after an input graph passes through a generator in a nonlinear mapping mode;
and S609, training the antagonistic neural network by a method of a minimization generator (G) and a maximization discriminator (D).
11. The curved chromosome image straightening model generation method according to claim 1, characterized in that: the curved chromosome image straightening model comprises a generator and a discriminator; the generator includes an encoding path and a decoding path.
12. The application method of the curved chromosome image straightening model is characterized in that: the method comprises the following steps:
generating a curved chromosome image straightening model according to the method of any one of claims 1 to 11;
according to the received original image of the bent chromosome, a chromosome straightening skeleton map is constructed;
and inputting the chromosome straightening skeleton map into a bent chromosome image straightening model, and outputting a chromosome straightening image by the bent chromosome image straightening model.
13. A generation system of a curved chromosome image straightening model is characterized in that: comprises that
The preprocessing unit is used for acquiring an original image of the bent chromosome and preprocessing the original image to obtain a chromosome image with impurities removed;
a zoom map generation unit for generating a chromosome zoom map from the chromosome map;
a curved skeleton map generation unit for generating a chromosome curved skeleton map from the chromosome map;
the augmentation unit is used for carrying out data augmentation of the chromosome zoom map and the chromosome bending skeleton map;
the normalization unit is used for carrying out normalization processing on the chromosome zoom map and the chromosome bending skeleton map after the augmentation, which are obtained by the augmentation unit, so as to obtain a training data set;
and the training unit is used for inputting the training data set into the bent chromosome image straightening model to train until the model converges.
14. A readable storage medium, characterized in that: a program for realizing any of the above-described methods for generating a curved chromosome image straightening model is stored.
15. Computer apparatus, characterized by: the system comprises a memory, a processor and a communication bus; the communication bus is used for realizing connection communication between the processor and the memory; the processor is configured to execute a program stored in the memory to implement any one of the above-described curved chromosome image straightening model generation methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010360505.4A CN111612744A (en) | 2020-04-30 | 2020-04-30 | Curved chromosome image straightening model generation method, application of model, system, readable storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010360505.4A CN111612744A (en) | 2020-04-30 | 2020-04-30 | Curved chromosome image straightening model generation method, application of model, system, readable storage medium and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111612744A true CN111612744A (en) | 2020-09-01 |
Family
ID=72199763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010360505.4A Pending CN111612744A (en) | 2020-04-30 | 2020-04-30 | Curved chromosome image straightening model generation method, application of model, system, readable storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111612744A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114820596A (en) * | 2022-06-23 | 2022-07-29 | 西湖大学 | Curved chromosome image straightening method based on combined model |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140016843A1 (en) * | 2012-06-19 | 2014-01-16 | Health Discovery Corporation | Computer-assisted karyotyping |
| CN109300111A (en) * | 2018-08-27 | 2019-02-01 | 杭州德适生物科技有限公司 | A kind of chromosome recognition methods based on deep learning |
| CN110348376A (en) * | 2019-07-09 | 2019-10-18 | 华南理工大学 | A kind of pedestrian's real-time detection method neural network based |
-
2020
- 2020-04-30 CN CN202010360505.4A patent/CN111612744A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140016843A1 (en) * | 2012-06-19 | 2014-01-16 | Health Discovery Corporation | Computer-assisted karyotyping |
| CN109300111A (en) * | 2018-08-27 | 2019-02-01 | 杭州德适生物科技有限公司 | A kind of chromosome recognition methods based on deep learning |
| CN110348376A (en) * | 2019-07-09 | 2019-10-18 | 华南理工大学 | A kind of pedestrian's real-time detection method neural network based |
Non-Patent Citations (1)
| Title |
|---|
| 郭翔: ""染色体识别算法的研究"", 《硕士电子期刊》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114820596A (en) * | 2022-06-23 | 2022-07-29 | 西湖大学 | Curved chromosome image straightening method based on combined model |
| CN114820596B (en) * | 2022-06-23 | 2022-10-11 | 西湖大学 | Curved chromosome image straightening method based on combined model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109493346B (en) | Stomach cancer pathological section image segmentation method and device based on multiple losses | |
| CN108492281B (en) | Bridge crack image obstacle detection and removal method based on generation type countermeasure network | |
| CN109726627B (en) | Neural network model training and universal ground wire detection method | |
| CN107424161B (en) | Coarse-to-fine indoor scene image layout estimation method | |
| CN110334645B (en) | Moon impact pit identification method based on deep learning | |
| CN111476806B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN108009554A (en) | A kind of image processing method and device | |
| CN113763441B (en) | Medical image registration method and system without supervision learning | |
| CN109711268B (en) | Face image screening method and device | |
| US20220335600A1 (en) | Method, device, and storage medium for lesion segmentation and recist diameter prediction via click-driven attention and dual-path connection | |
| CN105574527A (en) | Quick object detection method based on local feature learning | |
| CN107490356B (en) | Non-cooperative target rotating shaft and rotation angle measuring method | |
| CN104867225A (en) | Banknote face orientation identification method and apparatus | |
| CN102236798A (en) | Image matching method and device | |
| CN104036479A (en) | Multi-focus image fusion method based on non-negative matrix factorization | |
| CN113808180B (en) | Heterologous image registration method, system and device | |
| CN109948575B (en) | Eyeball area segmentation method in ultrasonic image | |
| CN115147426B (en) | Model training and image segmentation method and system based on semi-supervised learning | |
| CN112712273A (en) | Handwritten Chinese character beauty evaluation method based on skeleton similarity | |
| CN104217459A (en) | Spherical feature extraction method | |
| CN113706562B (en) | Image segmentation method, device and system and cell segmentation method | |
| CN114663880A (en) | Three-dimensional target detection method based on multi-level cross-modal self-attention mechanism | |
| CN114677558A (en) | Target detection method based on direction gradient histogram and improved capsule network | |
| CN111612744A (en) | Curved chromosome image straightening model generation method, application of model, system, readable storage medium and computer equipment | |
| CN111612745A (en) | Curved chromosome image straightening method, system, storage medium and device based on BagPix2Pix self-learning model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200901 |