CN115410194A - Chromosome image analysis method and system based on progressive segmentation and focus classification - Google Patents

Abstract

The invention provides a chromosome image analysis method and system based on progressive segmentation and focus classification, which comprises the following steps: a gradual segmentation step: segmenting the metaphase chromosome image by utilizing progressive segmentation to obtain a single chromosome image set; a focusing classification step: and (3) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart. The invention combines the traditional chromosome processing method and the deep learning to form a progressive segmentation method to gradually and effectively separate the conglutinated chromosome clusters, and has high segmentation precision and no manual participation.

Description

Chromosome image analysis method and system based on progressive segmentation and focus classification

Technical Field

The invention relates to the technical field of full-automatic chromosome analysis, in particular to a chromosome image analysis method and system based on progressive segmentation and focus classification. In particular, the invention preferably relates to a full-automatic chromosome image analysis method research based on progressive segmentation and focus classification.

Background

Chromosome image analysis methods, i.e., chromosome karyotyping analysis algorithms, are common means for diagnosing diseases such as chromosome abnormalities, such as trisomy 21 syndrome and leukemia. Karyotyping plays a crucial role in clinical prenatal diagnosis for determining whether a fetus has a severe defect or a genetic disease. Normal humans include 46 chromosomes: 22 pairs of autosomes and 1 pair of sex chromosomes XX or XY. Early human karyotype analysis required professional medical workers to manually complete, and the requirements for operator knowledge storage and operation proficiency were high, and the work was time-consuming and labor-consuming and extremely inefficient. Therefore, many scholars have been devoted to the study of a fully automated chromosome analysis method in recent years.

The chromosome image analysis mainly comprises the following steps: the method comprises two steps of chromosome segmentation and chromosome classification. The accuracy of chromosome image segmentation has a non-negligible influence on the later classification accuracy, and the segmentation of touching and overlapping chromosomes is an important link influencing the segmentation accuracy. In addition, the classification precision of the chromosome determines whether the whole chromosome karyotype analysis method can be used clinically.

Chinese patent publication No. CN113781505A discloses a chromosome segmentation method, apparatus, chromosome analyzer and storage medium, wherein the method includes: performing image processing on an original image to obtain a mask image corresponding to the original image, wherein the original image comprises crossed chromosomes; performing image skeleton extraction on the mask image to obtain a skeleton image; determining a target pixel point on a skeleton according to the pixel value of each pixel point in the skeleton image, wherein the target pixel point corresponds to the intersection point of the crossed chromosomes in the original image; determining a plurality of contour key points corresponding to crossed chromosomes in the original image; determining division points of crossed chromosomes in the original image according to the crossed points and the plurality of contour key points; and segmenting the crossed chromosomes in the original image based on the segmentation points.

For the related technologies, the inventor considers that the traditional segmentation method is mainly threshold segmentation, and the final result of the segmentation method depends on image quality and manual intervention; most of the existing deep learning chromosome segmentation methods are specially carried out aiming at a synthetic sticky chromosome data set, and the effect is poor when the method is applied to real chromosome data. In the study of chromosome image classification methods, deep learning classification is mainly applied, but some students directly classify by using single chromosome data which is segmented manually, and other students have low classification accuracy although no manual interference exists.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a chromosome image analysis method and system based on progressive segmentation and focus classification.

The invention provides a chromosome image analysis method based on progressive segmentation and focus classification, which comprises the following steps:

a gradual segmentation step: segmenting the metaphase chromosome image by utilizing progressive segmentation to obtain a single chromosome image set;

a focusing classification step: and (3) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart.

Preferably, the step of progressively segmenting comprises the steps of:

a global threshold segmentation step: setting pixel points lower than the threshold value as a first preset value and pixel points higher than the threshold value as a second preset value by calculating the average value of all pixel point values in the whole metaphase chromosome image as the threshold value, and dividing the whole metaphase chromosome image into a foreground and a background according to the first preset value and the second preset value;

a chromosome cluster classification step: judging a real chromosome cluster in the foreground, and classifying the real chromosome cluster through a chromosome cluster classification network to obtain the category information of the chromosome cluster;

chromosome example segmentation step: and segmenting the chromosome cluster according to the category information of the chromosome cluster to obtain all single chromosome image sets in the metaphase chromosome image.

Preferably, the chromosome cluster classification step comprises the steps of:

and (3) foreground discrimination: before classification implementation, a single chromosome and a real chromosome cluster of a foreground object are distinguished by using a convex hull and a preset external graph, wherein the single chromosome is directly output to a single chromosome set, and the real chromosome cluster is continuously segmented through the following steps;

a multi-scale characteristic diagram obtaining step: inputting the real chromosome cluster into a residual error network to obtain a multi-scale characteristic diagram;

and (3) calibrating the multi-scale characteristic diagram: inputting the multi-scale characteristic diagram to the respective extrusion excitation module, and outputting a calibration multi-scale characteristic diagram;

multi-scale feature fusion: performing characteristic fusion on the calibration multi-scale characteristic diagram through a formula to obtain a multi-scale fusion characteristic diagram;

and a final characteristic diagram acquisition step: performing average pooling operation on the multi-scale fusion characteristic graphs, and performing characteristic connection to obtain final characteristic graphs of each chromosome cluster;

a category information acquisition step: after the final characteristic diagram passes through a plurality of layers of full connection layers and a normalized index function excitation function, obtaining category information of the chromosome cluster; the category information of the chromosome clusters is touch chromosome clusters, overlapping chromosome clusters and touch overlapping chromosome clusters.

Preferably, the chromosome instance segmentation step comprises the steps of:

touch segmentation: extracting a skeleton graph, a terminal point and a contour graph of the touch chromosome cluster to obtain a cutting point of the cross region;

connecting the pair-by-pair cutting points to obtain a guide line, performing expansion treatment on the guide line to obtain a connecting area, calculating the mean value of all pixel points in the connecting area as a threshold, reserving all pixel points in the connecting area which are lower than the threshold, processing the pixel points by using a minimum binary method, fitting a parting line between touch chromosomes, and dividing the touch chromosome cluster by using the parting line;

an overlapping and dividing step: extracting an overlapping region of the overlapping chromosome clusters by adopting U-Net;

splicing the images on the two sides of the overlapping area with the overlapping area respectively to separate out all single chromosomes in the overlapping chromosome cluster;

touch overlap segmentation step: firstly, an overlapping dividing step is adopted to divide an overlapping chromosome cluster in a touch overlapping chromosome cluster, so that a plurality of single chromosomes in the touch overlapping chromosome cluster and a set of touch chromosome clusters are obtained, a convex hull and a preset external graph are used for judging the touch chromosome cluster in the touch overlapping chromosome cluster, wherein the separated single chromosomes are directly output to the single chromosome set, and the touch chromosome cluster is continuously divided through the following steps;

dividing touch chromosome clusters in the touch overlapped chromosome clusters by adopting a touch dividing step;

single chromosome assembly step: through the touch segmentation step, the overlap segmentation step and the touch overlap segmentation step, the touch chromosome cluster, the overlap chromosome cluster and the touch overlap chromosome cluster are segmented, and all single chromosome image sets in the metaphase chromosome image are obtained.

Preferably, the focus classification step includes the steps of:

a focusing step: focusing a salient region in an input single chromosome image, and extracting the salient region;

local network step: inputting the significant region image obtained in the focusing step into a local network, and extracting a local feature map of a single chromosome;

a global network step: inputting the single chromosome image into a global network to obtain a global feature map;

classifying the network: connecting the local feature map and the global feature map, inputting the connected local feature map and the global feature map into a classification network, and outputting single chromosome category information;

an analysis chart generation step: and combining the single chromosome image set and the single chromosome class information to generate a karyotype analysis chart.

Preferably, the focusing step includes the steps of:

basic feature extraction: inputting a single chromosome image by adopting a basic feature extraction network, and extracting single chromosome feature maps with multiple scales;

a multidimensional attention feature generation step: inputting the single chromosome feature maps of multiple scales into a multidimensional attention mechanism network to generate a multidimensional attention feature map;

and (3) multi-dimensional attention feature fusion: inputting the generated multidimensional attention feature map into a feature fusion network to generate a final fusion feature map;

mask extraction: inputting the final fusion feature graph into a mask generation network, and extracting a mask of the salient region;

and mask processing: and generating a mask of the significant region, performing morphological processing, removing noise points, filling holes, and multiplying the mask with the input original image to obtain a single chromosome significant region image.

The invention provides a chromosome image analysis system based on progressive segmentation and focus classification, which comprises the following modules:

a progressive segmentation module: segmenting the metaphase chromosome image by utilizing progressive segmentation to obtain a single chromosome image set;

a focus classification module: and (4) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart.

Preferably, the progressive segmentation module includes the following modules:

a global threshold segmentation module: calculating the average value of all pixel point values in the whole metaphase chromosome image to be used as a threshold value, setting pixel points lower than the threshold value as a first preset value, setting pixel points higher than the threshold value as a second preset value, and dividing the whole metaphase chromosome image into a foreground and a background according to the first preset value and the second preset value;

a chromosome cluster classification module: determining a real chromosome cluster in the foreground, and classifying the real chromosome cluster through a chromosome cluster classification network to obtain the category information of the chromosome cluster;

chromosome instance segmentation module: and segmenting the chromosome cluster according to the category information of the chromosome cluster to obtain all single chromosome image sets in the metaphase chromosome image.

Preferably, the chromosome cluster classification module comprises the following modules:

and a foreground judging module: before classification implementation, a single chromosome and a real chromosome cluster of a foreground object are distinguished by using a convex hull and a preset external graph, wherein the single chromosome is directly output to a single chromosome set, and the real chromosome cluster is continuously segmented through the following modules;

a multi-scale feature map acquisition module: inputting the real chromosome cluster into a residual error network to obtain a multi-scale characteristic diagram;

the multi-scale feature map calibration module: inputting the multi-scale characteristic diagram into each extrusion excitation module, and outputting a calibration multi-scale characteristic diagram;

a multi-scale feature fusion module: performing characteristic fusion on the calibration multi-scale characteristic diagram through a formula to obtain a multi-scale fusion characteristic diagram;

a final feature map acquisition module: performing average pooling operation on the multi-scale fusion characteristic graphs, and performing characteristic connection to obtain final characteristic graphs of each chromosome cluster;

a category information acquisition module: after the final characteristic diagram passes through a plurality of layers of full connection layers and a normalized index function excitation function, obtaining category information of the chromosome cluster; the category information of the chromosome clusters is touch chromosome clusters, overlapping chromosome clusters and touch overlapping chromosome clusters.

Preferably, the chromosome instance segmentation module comprises the following modules:

touch segmentation module: extracting a skeleton graph, a terminal point and a contour graph of the touch chromosome cluster to obtain a cutting point of the cross region;

connecting the pair of cutting points one by one to obtain a guide line, performing expansion treatment on the guide line to obtain a connecting area, calculating the mean value of all pixel points in the connecting area as a threshold, reserving all pixel points lower than the threshold in the connecting area, processing the pixel points by using a minimum binary method, fitting a division line between touch chromosomes, and dividing the touch chromosome clusters by using the division line;

an overlap segmentation module: extracting an overlapping region of the overlapping chromosome clusters by adopting U-Net;

splicing the images on the two sides of the overlapping area with the overlapping area respectively to separate out all single chromosomes in the overlapping chromosome cluster;

touch overlap and divide the module: firstly, an overlapping division module is adopted to divide overlapping chromosome clusters in touch overlapping chromosome clusters, so that a plurality of single chromosomes in the touch overlapping chromosome clusters and a set of touch chromosome clusters are obtained, a convex hull and a preset external graph are used for distinguishing the touch chromosome clusters in the touch overlapping chromosome clusters, wherein the separated single chromosomes are directly output to the single chromosome set, and the touch chromosome clusters are continuously divided through the following modules;

a touch division module is adopted to divide touch chromosome clusters in the touch overlapped chromosome clusters;

single chromosome assembly module: through the touch segmentation module, the overlap segmentation module and the touch overlap segmentation module, the touch chromosome cluster, the overlap chromosome cluster and the touch overlap chromosome cluster are all segmented, and all single chromosome image sets in the metaphase chromosome image are obtained.

Compared with the prior art, the invention has the following beneficial effects:

1. according to the full-automatic chromosome image analysis method based on progressive segmentation and focusing classification, a traditional chromosome processing method and deep learning are combined to form a progressive segmentation method, adherent chromosome clusters are gradually and effectively separated, the segmentation precision is high, and no manual intervention is involved;

2. in order to better realize chromosome classification, the chromosome significance characteristics are extracted by adopting a focusing network and combined with global characteristics to generate enhanced chromosome characteristic representation, so that the classification precision is greatly improved;

3. the invention is a full-automatic chromosome image analysis method, can automatically generate a chromosome karyotype analysis chart on the basis of high-precision segmentation and classification, and lays a certain foundation for the development of a full-automatic chromosome image analysis system.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a general block diagram of a fully automated chromosome image analysis method based on progressive segmentation and focus classification according to the present invention;

FIG. 2 is a schematic diagram of a metaphase G band chromosome image in the present invention;

FIG. 3 is a binary graph of global threshold segmentation in accordance with the present invention;

FIG. 4 is a diagram of a classification network of chromosome clusters according to the present invention;

FIG. 5 is a chromosome cluster image class label map in the present invention;

FIG. 6 is a graph of touch chromosome segmentation results in accordance with the present invention;

FIG. 7 is a graph showing the result of segmentation of an overlapped staining cluster in the present invention;

FIG. 8 is a diagram of a clustering network structure in the present invention;

FIG. 9 is a schematic diagram of a pixel attention mechanism of the present invention;

FIG. 10 is a schematic view of the channel attention mechanism of the present invention;

FIG. 11 is a structural view of a spatial attention mechanism of the present invention;

FIG. 12 is a diagram of karyotype analysis generated in the present invention.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the invention.

The embodiment of the invention discloses a full-automatic chromosome image analysis method research based on progressive segmentation and focus classification, as shown in figure 1, comprising the following steps:

a gradual segmentation step: and (3) segmenting the metaphase chromosome image by using a progressive segmentation method to obtain a single chromosome image set.

The progressive segmentation method comprises three steps: (1) global threshold segmentation; (2) chromosome cluster classification; and (3) chromosome example segmentation. The specific segmentation process is as follows: namely, the progressive segmentation step comprises the following steps:

global thresholding step (first step): the average value of all pixel point values in the whole metaphase chromosome image is calculated to serve as a threshold, pixel points lower than the threshold are set as first preset values, pixel points higher than the threshold are set as second preset values, and the whole metaphase chromosome image is divided into a foreground and a background according to the first preset values and the second preset values.

Specifically, global threshold segmentation is performed, an average value of all pixel point values in the whole metaphase chromosome image is calculated to serve as a threshold, pixel points lower than the threshold are set to be 0, and pixel points higher than the threshold are set to be 1, so that the whole metaphase chromosome image is divided into a foreground and a background.

Chromosome cluster classification step (second step): and (4) judging the real chromosome cluster in the foreground, and classifying the real chromosome cluster through a chromosome cluster classification network to obtain the category information of the chromosome cluster. Namely, chromosome cluster classification, which is classified by a chromosome cluster classification network.

The specific implementation steps of the chromosome cluster classification are as follows: namely, the chromosome cluster classification step comprises the following steps:

and (3) foreground discrimination: before classification implementation, the single chromosome and the real chromosome cluster of the foreground object are distinguished by using the convex hull and the preset circumscribed graph. Specifically, before classification implementation, a single chromosome and a real chromosome cluster of a foreground object are distinguished by using a convex hull and a minimum circumscribed rectangle, wherein the single chromosome is directly output to a single chromosome set, and the real chromosome cluster is continuously segmented through the following steps.

A multi-scale characteristic diagram obtaining step: inputting the real chromosome cluster into a residual error network 50 (ResNet 50) to obtain a multi-scale feature map { F } ₂ ,F ₃ ,F ₄ ,F ₅ In which F ₂ ，F ₃ ，F ₄ ，F ₅ Feature images output at layers 10, 22, 40, 49 of the ResNet50 network, respectively.

And (3) calibrating the multi-scale feature map: input { F ₂ ,F ₃ ,F ₄ ,F ₅ Outputting a calibrated multi-scale feature map { F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ Of which is F' ₂ ，F′ ₃ ，F′ ₄ ，F′ ₅ Respectively represent F ₂ ，F ₃ ，F ₄ ，F ₅ And inputting the calibration multi-scale characteristic diagram into a squeezing excitation module to obtain a calibration multi-scale characteristic diagram. Specifically, because the weight coefficient of each channel of the convolution characteristic layer can be changed by the Squeezing and Excitation (SE) module, important characteristic information in a chromosome image can be learned, and the difference characteristics among chromosome clusters are mainly concentrated in the network bottom layer characteristic layer, the input multi-scale characteristics can be enabled to have rich bottom layer information, namely input { F } by adjusting the excitation parameters in the SE module ₂ ,F ₃ ,F ₄ ,F ₅ Outputting a calibrated multi-scale feature map { F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ }。

Multi-scale feature fusion: in order to make each layer feature representation contain richer underlying feature information, the following formula is used to couple F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ Performing feature fusion to obtain a multi-scale fusion feature map { A } ₂ ,A ₃ ,A ₄ ,A ₅ }。

Wherein, { A ₂ ,A ₃ ,A ₄ ,A ₅ The multi-scale fusion characteristic diagram obtained by the calculation formula is represented, A ₂ And F' ₂ 、A ₃ And F' ₃ 、A ₄ And F' ₄ 、A ₅ And F' ₅ Each having the same image scale; conv _1×1 () Representing a one-dimensional convolution calculation; down () denotes downsampling calculation; up () represents the upsampling calculation;

indicating a pixel-by-pixel addition.

And a final characteristic diagram obtaining step: and performing average pooling operation on the multi-scale fusion characteristic maps, and performing characteristic connection on the multi-scale fusion characteristic maps to obtain a final characteristic map of each chromosome cluster.

A category information acquisition step: after the final characteristic diagram passes through a plurality of layers of full-link layers and a normalized exponential function (Softmax) excitation function, obtaining category information of the chromosome cluster; the category information of the chromosome clusters is touch chromosome clusters, overlapping chromosome clusters and touch overlapping chromosome clusters. And obtaining the category information of the chromosome cluster after the final characteristic diagram passes through two layers of full connection layers and a Softmax excitation function.

After all real chromosome clusters are input into the chromosome cluster classification network, the chromosome clusters can be divided into three categories: touch chromosome clusters, overlapping chromosome clusters, touch and overlapping chromosome clusters.

Chromosome instance segmentation step (third step): and segmenting the chromosome cluster according to the category information of the chromosome cluster to obtain all single chromosome image sets in the metaphase chromosome image. Namely, the chromosome instance segmentation comprises a touch segmentation module, an overlap segmentation module, a touch and overlap segmentation module.

The specific segmentation step of the chromosome instance segmentation step comprises the following steps:

touch division step (touch division module): and extracting a skeleton graph, a terminal point and a contour graph touching the chromosome cluster to obtain a cutting point of the cross region. That is, (1.1) the touch segmentation module is used for segmenting the touch chromosome cluster, and in order to perform segmentation quickly and accurately, the skeleton map, the terminal points and the contour map are extracted first, so that the cut points of the cross region can be obtained.

Connecting the pair of cut points to obtain a guide line, performing expansion treatment on the guide line to obtain a connecting area, calculating the mean value of all pixel points in the connecting area as a threshold, reserving all pixel points in the connecting area which are lower than the threshold, processing the pixel points by using a minimum binary method, fitting a partition line between touch chromosomes, and segmenting the touch chromosome clusters by using the partition line. That is, (1.2) the division lines between touch chromosomes are calculated by pairs of cutting points and image basic operation, and the division lines can completely divide touch chromosome clusters.

Overlap division step (overlap division module): and extracting an overlapping region of the overlapped chromosome clusters by adopting a classical U-shaped network (U-Net). Specifically, (2.1) the overlap segmentation module is used for segmenting the overlap chromosome cluster, and because the overlap chromosome image is small in size and complex in structure, an overlap region is extracted by adopting simplified U-Net, wherein the simplified U-Net is a network structure formed by only adopting two times of downsampling and corresponding upsampling.

And respectively splicing the images on the two sides of the overlapping area with the overlapping area to separate out all single chromosomes in the overlapping chromosome cluster. Specifically, (2.2) the overlapping area can be approximated to be a rectangle, and then the images on two sides of the parallel sides of the rectangle and the overlapping area are spliced to obtain all single separated chromosomes.

Touch overlap division step (touch and overlap division module): firstly, an overlapping dividing step is adopted to divide overlapping chromosome clusters in touch overlapping chromosome clusters, so that a plurality of single chromosomes in the touch overlapping chromosome clusters and a set of touch chromosome clusters are obtained, a convex hull and a preset external graph are used for distinguishing the touch chromosome clusters in the touch overlapping chromosome clusters, wherein the separated single chromosomes are directly output to the single chromosome set without being processed, and the touch chromosome clusters are continuously divided through the following steps. Specifically, (3.1) the touch and overlap segmentation module is used for segmenting the chromosome clusters with both touch and overlap, and the overlap segmentation module is firstly adopted to segment the overlapped chromosome clusters. And (3.2) completely dividing the overlapped chromosome clusters after the operation of (3.1) to obtain a plurality of single chromosomes and a set of touch chromosome clusters, and distinguishing the touch chromosome clusters by using the convex hulls and the minimum circumscribed rectangles.

And a touch segmentation step is adopted to segment touch chromosome clusters in the touch overlapped chromosome clusters. Namely, (3.3) the touch chromosome cluster is segmented by adopting a touch segmentation module.

Single chromosome assembly step: through the touch segmentation step, the overlap segmentation step and the touch overlap segmentation step, the touch chromosome cluster, the overlap chromosome cluster and the touch overlap chromosome cluster are segmented, and all single chromosome image sets in the metaphase chromosome image are obtained. Through the 3 segmentation steps, all chromosome clusters can be segmented, and all single chromosome sets in the metaphase chromosome image are obtained.

A focusing classification step: and (3) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart. Namely, a focusing classification algorithm is adopted to identify the categories of all the single chromosomes, and finally, a karyotype analysis chart is generated. The Focus-sorting network (Focus-Net) comprises 4 modules: the device comprises a focusing module, a local network module, a global network module and a classification module.

The specific classification process of the focus classification step comprises the following steps:

a focusing step: the salient regions in the input single chromosome image are focused and extracted. Specifically, the focusing module is used for focusing and extracting a salient region in a single chromosome image, and the focusing module is composed of a feature extraction network, a multi-dimensional attention mechanism network, a feature fusion network and a mask generation network.

The salient region is generated specifically as follows: namely, the focusing step includes the steps of:

basic feature extraction: and (3) inputting a single chromosome image by adopting a basic feature extraction network, and extracting single chromosome feature maps with multiple scales. Specifically, (4.1) a combined residual error network 50 and a feature pyramid network (ResNet 50-FPN) are used as basic feature extraction networks, a single chromosome image is input, and a single chromosome feature map with 5 scales can be extracted.

A multi-dimensional attention feature generation step: and inputting the single chromosome feature map with multiple scales into the multidimensional attention mechanism network to generate the multidimensional attention feature map. Specifically, (4.2) the multi-scale feature map is input into a multi-dimensional attention mechanism network to generate a multi-dimensional attention feature map, wherein the multi-dimensional attention mechanism network comprises three attention mechanisms, namely pixel points, channels and spaces.

And (3) multi-dimensional attention feature fusion: and inputting the generated multidimensional attention feature map into a feature fusion network to generate a final fusion feature map. Specifically, (4.3) the generated multidimensional attention feature map is input into a feature fusion network, so that the attention feature maps are effectively combined through a reasonable fusion mode, firstly, the resolution of the multi-scale pixel point attention feature map is changed to be the same as that of an input image through upsampling processing, the processed pixel point attention feature maps are fused through an adding mode to obtain a pixel point attention fusion feature map, the channel attention feature map and the spatial attention feature map are processed in the same mode to obtain a channel attention fusion feature map and a spatial attention fusion feature map, and finally, the three types of fusion feature maps are fused again through weighted addition to generate a final feature map, wherein the weight coefficients are all set to be 1.

Mask extraction: and inputting the final fusion feature graph into a mask generation network, and extracting the mask of the salient region. Specifically, (4.4) the feature map generated in (4.3) is input to a mask generation network, which is formed by Conv, in order to extract a mask of a significant region ₁ × ₁ And a Softmax excitation function.

And mask processing: and generating a mask of the significant region, performing morphological processing, removing noise points, filling holes, and multiplying the mask with the input original image to obtain a single chromosome significant region image. Specifically, (4.5) the mask for generating the salient region is firstly subjected to morphological processing, noise points are removed, small holes are filled, and then multiplication processing is carried out on the mask and the input original image to obtain a single chromosome salient region image.

Local network step: and inputting the image of the salient region obtained in the focusing step into a local network, and extracting a local feature map of a single chromosome. Specifically, the local network module is configured to generate a local feature map, and the local feature map of a single chromosome can be extracted by inputting the saliency region image obtained in the focusing step into the local network module, where the local network module is formed by a Resnet50 network.

And global network step: and inputting the single chromosome image into a global network to obtain a global feature map. Specifically, the global network module is used for generating a global feature map, and the global feature map can be obtained by inputting a single chromosome image into the global network module, wherein the global network module is also formed by a Resnet50 network.

Classifying the network: and connecting the local feature map and the global feature map, inputting the connected feature maps into a classification network, and outputting single chromosome class information. Specifically, the classification network module is used for identifying the category of a single chromosome, firstly, local and global feature maps obtained in a local network step and a global network step are connected, and then the local and global feature maps are input into the classification network module to output the category of the chromosome, wherein the classification network module consists of a 2-layer full connection layer and 1 Softmax excitation function.

An analysis chart generation step: and generating a karyotype analysis chart by combining the single chromosome image set and the single chromosome class information. Finally, the single chromosome set and the category information are combined to generate a final karyotype analysis chart.

The embodiment of the invention also discloses a full-automatic chromosome image analysis system based on progressive segmentation and focus classification, which comprises the following modules as shown in figure 1:

a progressive segmentation module: and (4) segmenting the metaphase chromosome image by utilizing progressive segmentation to obtain a single chromosome image set.

The progressive segmentation module comprises the following modules:

a global threshold segmentation module: the average value of all pixel point values in the whole metaphase chromosome image is calculated to serve as a threshold, pixel points lower than the threshold are set as first preset values, pixel points higher than the threshold are set as second preset values, and the whole metaphase chromosome image is divided into a foreground and a background according to the first preset values and the second preset values.

A chromosome cluster classification module: and (4) judging the real chromosome cluster in the foreground, and classifying the real chromosome cluster through a chromosome cluster classification network to obtain the category information of the chromosome cluster.

The chromosome cluster classification module comprises the following modules:

and a foreground judging module: before classification is carried out, a single chromosome and a real chromosome cluster of the foreground object are distinguished by using the convex hull and a preset circumscribed graph, wherein the single chromosome is directly output to a single chromosome set, and the real chromosome cluster is continuously segmented through the following steps.

A multi-scale feature map acquisition module: inputting the real chromosome cluster into a residual error network 50 (ResNet 50) to obtain a multi-scale feature map { F } ₂ ,F ₃ ,F ₄ ,F ₅ In which F ₂ ，F ₃ ，F ₄ ，F ₅ Feature images output for layers 10, 22, 40, 49 of the ResNet50 network, respectively.

The multi-scale feature map calibration module: input { F ₂ ,F ₃ ,F ₄ ,F ₅ Outputting a calibrated multi-scale feature map { F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ }。

A multi-scale feature fusion module: by formula pair { F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ Performing feature fusion to obtain a multi-scale fusion feature map { A } ₂ ,A ₃ ,A ₄ ,A ₅ }。

Wherein, { A ₂ ,A ₃ ,A ₄ ,A ₅ The multi-scale fusion characteristic diagram obtained by the calculation formula is represented, A ₂ And F' ₂ 、A ₃ And F' ₃ 、A ₄ And F' ₄ 、A ₅ And F' ₅ Each having the same image scale; conv _1×1 () Representing a one-dimensional convolution calculation; down () represents the downsampling calculation; up () represents the upsampling calculation;

indicating a pixel-by-pixel addition.

A final feature map acquisition module: and performing average pooling operation on the multi-scale fusion characteristic maps, and performing characteristic connection to obtain a final characteristic map of each chromosome cluster.

A category information acquisition module: after the final characteristic diagram passes through a plurality of layers of full-link layers and a normalized exponential function (Softmax) excitation function, obtaining category information of the chromosome cluster; the category information of the chromosome clusters is touch chromosome clusters, overlapping chromosome clusters and touch overlapping chromosome clusters.

Chromosome instance segmentation module: and segmenting the chromosome cluster according to the category information of the chromosome cluster to obtain all single chromosome image sets in the metaphase chromosome image.

The chromosome instance segmentation module comprises the following modules:

touch segmentation module: and extracting a skeleton graph, a terminal point and a contour graph of the touch chromosome cluster to obtain a cutting point of the cross region.

And connecting the pair of cut points to obtain a guide line, performing expansion treatment on the guide line to obtain a connecting area, calculating the mean value of all pixel points in the connecting area as a threshold, reserving all pixel points in the connecting area which are lower than the threshold, processing the pixel points by using a minimum binary method, fitting a parting line between touch chromosomes, and parting the touch chromosome clusters by using the parting line.

An overlap segmentation module: and extracting an overlapping region of the overlapped chromosome clusters by adopting a classical U-shaped network (U-Net).

And respectively splicing the images on the two sides of the overlapping area with the overlapping area to separate out all single chromosomes in the overlapping chromosome cluster.

Touch overlap segmentation module: firstly, an overlapping division module is adopted to divide overlapping chromosome clusters in touch overlapping chromosome clusters, so that a plurality of single chromosomes in the touch overlapping chromosome clusters and a set of touch chromosome clusters are obtained, a convex hull and a preset external graph are used for distinguishing the touch chromosome clusters in the touch overlapping chromosome clusters, wherein the separated single chromosomes are directly output to the single chromosome set without being processed, and the touch chromosome clusters are continuously divided through the following steps.

And a touch division module is adopted to divide touch chromosome clusters in the touch overlapped chromosome clusters.

Single chromosome aggregation module: through the touch segmentation module, the overlap segmentation module and the touch overlap segmentation module, the touch chromosome cluster, the overlap chromosome cluster and the touch overlap chromosome cluster are all segmented, and all single chromosome image sets in the metaphase chromosome image are obtained.

A focus classification module: and (4) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart.

The focus classification module comprises the following modules:

a focusing module: and focusing the salient region in the input single chromosome image and extracting the salient region.

The focusing module comprises the following modules:

a basic feature extraction module: and (3) inputting a single chromosome image by adopting a basic feature extraction network, and extracting single chromosome feature maps with multiple scales.

A multidimensional attention feature generation module: and inputting the single chromosome feature maps of multiple scales into the multi-dimensional attention mechanism network to generate the multi-dimensional attention feature map.

A multidimensional attention feature fusion module: and inputting the generated multidimensional attention feature map into a feature fusion network to generate a final fusion feature map.

A mask extraction module: and inputting the final fusion feature map into a mask generation network, and extracting the mask of the salient region.

A mask processing module: and generating a mask of the significant region, performing morphological processing, removing noise points, filling holes, and multiplying the mask with the input original image to obtain a single chromosome significant region image.

A local network module: and inputting the image of the salient region obtained by the focusing module into a local network, and extracting a local feature map of a single chromosome.

A global network module: and inputting the single chromosome image into a global network to obtain a global feature map.

A classification network module: and connecting the local feature map and the global feature map, inputting the connected local feature map and global feature map into a classification network, and outputting single chromosome class information.

An analysis graph generation module: and combining the single chromosome image set and the single chromosome class information to generate a karyotype analysis chart.

The embodiment of the invention also provides a chromosome image analysis method based on progressive segmentation and focusing classification, wherein a metaphase chromosome image is segmented by using a progressive segmentation method to obtain a single chromosome set, and then the categories of all the single chromosomes are identified by using a focusing classification algorithm.

1. The metaphase G band chromosome image shown in fig. 2 is subjected to global threshold processing, an average value of all pixel point values in the whole metaphase chromosome image is calculated as a threshold, pixel points lower than the threshold are set to be 0, pixel points higher than the threshold are set to be 1, and thus the whole metaphase chromosome image is divided into a foreground and a background, and a processing result graph is shown in fig. 3.

2. The foreground comprises a single chromosome and a chromosome cluster, and before the chromosome cluster is classified, the real chromosome cluster is judged by using a convex hull and a minimum circumscribed rectangle.

3. The chromosome cluster classification network is trained by using the chromosome cluster image data set, and a chromosome cluster classification network structure diagram can be obtained, as shown in fig. 4.

4. And (3) classifying the chromosome cluster images obtained in the step (2) by the trained chromosome cluster classification network, wherein the specific classification steps are as follows:

(1) Inputting the real chromosome cluster in the step 2 into ResNet50 to obtain a multi-scale feature map { F } ₂ ,F ₃ ,F ₄ ,F ₅ From lower to higher level.

(2) Input { F ₂ ,F ₃ ,F ₄ ,F ₅ Get the calibrated multi-scale feature map { F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ }。

(3) In order to make each layer feature representation contain richer underlying feature information, the following formula is used for F' ₂ ,F′ ₃ ,F′ ₄ ,F′ ₅ Performing feature fusion:

wherein { A ₂ ,A ₃ ,A ₄ ,A ₅ The multi-scale fusion characteristic diagram obtained by the calculation formula is represented, A ₂ And F' ₂ 、A ₃ And F' ₃ 、A ₄ And F' ₄ 、A ₅ And F' ₅ Each having the same image scale, conv _1×1 () Representing a one-dimensional convolution calculation, down () and Up () representing a Down-sampling and an Up-sampling calculation, respectively,

indicating a pixel-by-pixel addition.

(4) And performing average pooling operation on the multi-scale fusion characteristic maps, and performing characteristic connection on the multi-scale fusion characteristic maps to obtain a final characteristic map of each chromosome cluster.

(5) And (4) obtaining the category information of the chromosome cluster after the final characteristic diagram passes through two full-connection layers and a Softmax excitation function.

(6) After all the real chromosome clusters in the step 2 are input into the chromosome cluster classification network, the chromosome cluster classification network can be divided into three types: touching a chromosome cluster, overlapping a chromosome cluster, touching and overlapping a chromosome cluster, as shown in fig. 5.

5. The chromosome example segmentation is to segment three types of chromosome clusters respectively, and the segmentation process is as follows:

(1) Dividing the touch chromosome cluster, wherein in order to quickly and accurately implement the division, a skeleton graph, a terminal point and a contour graph of the touch chromosome cluster are extracted, so that a cutting point of a cross region can be obtained; and connecting the pair of cut points to obtain a guide line, performing expansion processing on the guide line to obtain a connecting region, calculating the mean value of all pixel points in the connecting region as a threshold, reserving all pixel points in the connecting region which are lower than the threshold, processing the pixel points by using a minimum binary method, and fitting a dividing line between touch chromosomes, wherein the dividing line can completely divide the touch chromosome cluster, as shown in fig. 5.

(2) Dividing the overlapped chromosome cluster, and extracting an overlapped region by adopting simplified U-Net (U-Net) because the image size of the overlapped chromosome is small and the structure is complex, wherein the structure of the simplified U-Net is a network structure formed by only adopting two times of downsampling and corresponding upsampling; secondly, the overlapping region can be approximated to be rectangular, and the images on both sides of the parallel sides of the rectangular and the overlapping region are spliced to obtain a single separated chromosome, as shown in fig. 7.

(3) For touch and overlapped chromosome segmentation, firstly, extracting an overlapped region by adopting simplified U-Net, and separating an overlapped chromosome cluster by splicing; after the operation, the overlapped chromosome clusters are completely divided to obtain a plurality of single chromosomes and a set of touch chromosome clusters, and then the touch chromosome clusters are judged by utilizing the convex hulls and the minimum circumscribed rectangles, wherein the separated single chromosomes are directly output to the single chromosome set without being processed, and the touch chromosome clusters are continuously divided through the following steps; touch chromosome segregation is as described above in step (1).

Through the 3 segmentation steps, all chromosome clusters can be separated, and all single chromosome sets in the metaphase chromosome image are obtained.

6. Before carrying out chromosome classification and identification, a single chromosome data set is used for training a Focus network-Focus-Net, and a structure diagram of the trained Focus-Net network is shown in FIG. 8 and comprises four modules: the device comprises a clustering module, a local network module, a global network module and a classification module. PAB, CAB, and SAB represent pixel attention modules, channel attention modules, and spatial attention modules.

7. The method comprises the following steps of focusing and inputting a salient region in a single chromosome image by using a focusing module, wherein the focusing module is composed of a feature extraction network, a multi-dimensional attention mechanism network, a feature fusion network and a mask generation network, and the salient region is specifically generated in the following process:

(1) And a ResNet50-FPN is used as a basic feature extraction network, and a single chromosome image is input, so that a single chromosome feature map with 5 scales can be extracted.

(2) And inputting the multi-scale feature map into a multi-dimensional attention mechanism network to generate the multi-dimensional attention feature map, wherein the multi-dimensional attention mechanism network comprises three attention mechanisms, namely a pixel point, a channel and a space, the pixel point attention mechanism is shown in fig. 9, the channel attention mechanism is shown in fig. 10, and the space attention mechanism is shown in fig. 11.

(3) The generated multidimensional attention feature map is input into a feature fusion network, the aim is to effectively combine all attention feature maps through a reasonable fusion mode, firstly, the multi-scale pixel point attention feature map can be subjected to upsampling processing to enable the resolution ratio to be the same as that of an input image, the processed pixel point attention feature maps are fused in an adding mode to obtain a pixel point attention fusion feature map, a channel and space attention feature map are processed in the same mode to obtain a channel attention fusion feature map and a space attention fusion feature map, finally, the three types of fusion feature maps are fused again through weighted addition to generate a final feature map, and the weighting coefficients are all set to be 1.

(4) Inputting the feature map generated in (3) into a mask generation network, in order to extract a mask of the salient region, wherein the mask generation network is formed by Conv _1×1 And a Softmax excitation function.

(5) And performing morphological processing on the mask for generating the salient region, removing noise points, filling small holes, and multiplying the input original image to obtain a single chromosome salient region image.

8. And (4) generating a local feature map by using a local network module, inputting the salient region image obtained in the step (7) into the local network module, and extracting the local feature map of a single chromosome, wherein the local network module is formed by a Resnet50 network.

9. Meanwhile, a global feature map is generated by using a global network module, and the global feature map can be obtained by inputting a single chromosome image into the global network module, wherein the global network module is also composed of a Resnet50 network.

10. And identifying the category of a single chromosome by using a classification network module, connecting the local characteristic graph and the global characteristic graph obtained in the step 8 and the step 9, inputting the connected local characteristic graph and the global characteristic graph into the classification network module, and outputting the category of the chromosome, wherein the classification network module consists of a 2-layer full connection layer and 1 Softmax excitation function.

11. Finally, combining the extracted single chromosome and the corresponding category thereof to generate a karyotype analysis chart of the chromosome image in fig. 2, as shown in fig. 12.

According to the full-automatic chromosome image analysis method based on progressive segmentation and focusing classification, the traditional segmentation method and the deep learning method are combined to gradually and accurately segment touch and overlapped chromosome clusters in the middle-stage chromosome image, and meanwhile, the single chromosome salient region is automatically focused and effective local features are extracted, so that the classification accuracy is greatly improved by combining with the global features.

The invention discloses a full-automatic chromosome image analysis method based on progressive segmentation and focus classification, which comprises two stages: 1. in the progressive segmentation stage consisting of global threshold segmentation, a chromosome cluster classification network and chromosome example segmentation, a foreground target is extracted from a human metaphase chromosome image by adopting the global threshold segmentation, then the conglutinated chromosome clusters in the foreground are divided into three categories by utilizing the chromosome cluster classification network, and finally the chromosome example segmentation is carried out according to the characteristics of each chromosome cluster; 2. and in the focusing classification stage, a focusing network (Focus-Net) consisting of a focusing module, a global network module, a local network module and a classification module is used for automatically focusing the salient region, extracting global and local features and identifying chromosome categories, so that a chromosome karyotype analysis chart is finally generated. According to the method, complete segmentation of touched and overlapped chromosomes is gradually and effectively realized, and meanwhile, the method can automatically focus on the significant region of a single chromosome, so that the extracted characteristic information is richer, the segmentation and classification precision is greatly improved, and the whole chromosome analysis process has no manual interference.

Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, units provided by the present invention as pure computer readable program code, the system and its various devices, modules, units provided by the present invention can be fully implemented by logically programming method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices, modules and units thereof provided by the invention can be regarded as a hardware component, and the devices, modules and units included in the system for realizing various functions can also be regarded as structures in the hardware component; means, modules, units for realizing various functions can also be regarded as structures in both software modules and hardware components for realizing the methods.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims (10)

1. A chromosome image analysis method based on progressive segmentation and focus classification is characterized by comprising the following steps:

a gradual segmentation step: segmenting the metaphase chromosome image by utilizing progressive segmentation to obtain a single chromosome image set;

a focusing classification step: and (3) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart.

2. The chromosome image analysis method based on progressive segmentation and focus classification according to claim 1, wherein the progressive segmentation step comprises the steps of:

global threshold segmentation step: calculating the average value of all pixel point values in the whole metaphase chromosome image to be used as a threshold value, setting pixel points lower than the threshold value as a first preset value, setting pixel points higher than the threshold value as a second preset value, and dividing the whole metaphase chromosome image into a foreground and a background according to the first preset value and the second preset value;

a chromosome cluster classification step: determining a real chromosome cluster in the foreground, and classifying the real chromosome cluster through a chromosome cluster classification network to obtain the category information of the chromosome cluster;

chromosome example segmentation step: and segmenting the chromosome cluster according to the category information of the chromosome cluster to obtain all single chromosome image sets in the metaphase chromosome image.

3. The chromosome image analysis method based on progressive segmentation and focus classification according to claim 2, wherein the chromosome cluster classification step comprises the steps of:

and a foreground judging step: before classification implementation, a single chromosome and a real chromosome cluster of a foreground object are distinguished by using a convex hull and a preset external graph, wherein the single chromosome is directly output to a single chromosome set, and the real chromosome cluster is continuously segmented through the following steps;

a multi-scale characteristic diagram obtaining step: inputting the real chromosome cluster into a residual error network to obtain a multi-scale characteristic diagram;

and (3) calibrating the multi-scale feature map: inputting the multi-scale characteristic diagram to the respective extrusion excitation module, and outputting a calibration multi-scale characteristic diagram;

multi-scale feature fusion: performing feature fusion on the calibrated multi-scale feature map through a formula to obtain a multi-scale fusion feature map;

and a final characteristic diagram obtaining step: carrying out average pooling operation on the multi-scale fusion characteristic diagrams, and then carrying out characteristic connection to obtain a final characteristic diagram of each chromosome cluster;

a category information acquisition step: obtaining the category information of the chromosome cluster after the final characteristic diagram passes through a plurality of layers of full-link layers and a normalized index function excitation function; the category information of the chromosome clusters is touch chromosome clusters, overlapping chromosome clusters and touch overlapping chromosome clusters.

4. The chromosome image analysis method based on progressive segmentation and focus classification according to claim 3, wherein the chromosome instance segmentation step comprises the steps of:

touch segmentation step: extracting a skeleton graph, a terminal point and a contour map of the touch chromosome cluster to obtain a cutting point of the cross region;

connecting the pair-by-pair cutting points to obtain a guide line, performing expansion treatment on the guide line to obtain a connecting area, calculating the mean value of all pixel points in the connecting area as a threshold, reserving all pixel points in the connecting area which are lower than the threshold, processing the pixel points by using a minimum binary method, fitting a parting line between touch chromosomes, and dividing the touch chromosome cluster by using the parting line;

an overlapping and dividing step: extracting an overlapping region of the overlapping chromosome clusters by adopting U-Net;

splicing the images on the two sides of the overlapping area with the overlapping area respectively to separate out all single chromosomes in the overlapping chromosome cluster;

touch overlap segmentation step: firstly, an overlapping dividing step is adopted to divide an overlapping chromosome cluster in a touch overlapping chromosome cluster, so that a plurality of single chromosomes in the touch overlapping chromosome cluster and a set of touch chromosome clusters are obtained, a convex hull and a preset external graph are used for judging the touch chromosome cluster in the touch overlapping chromosome cluster, wherein the separated single chromosomes are directly output to the single chromosome set, and the touch chromosome cluster is continuously divided through the following steps;

dividing touch chromosome clusters in the touch overlapped chromosome clusters by adopting a touch dividing step;

single chromosome assembly step: through the touch segmentation step, the overlap segmentation step and the touch overlap segmentation step, the touch chromosome cluster, the overlap chromosome cluster and the touch overlap chromosome cluster are segmented, and all single chromosome image sets in the metaphase chromosome image are obtained.

5. The chromosome image analysis method based on progressive segmentation and focus classification according to claim 1, wherein the focus classification step comprises the steps of:

a focusing step: focusing a salient region in an input single chromosome image, and extracting the salient region;

local network step: inputting the significant region image obtained in the focusing step into a local network, and extracting a local feature map of a single chromosome;

and global network step: inputting the single chromosome image into a global network to obtain a global feature map;

classifying the network: connecting the local feature map and the global feature map, inputting the local feature map and the global feature map into a classification network, and outputting single chromosome category information;

an analysis chart generation step: and combining the single chromosome image set and the single chromosome class information to generate a karyotype analysis chart.

6. The chromosome image analysis method based on progressive segmentation and focus classification according to claim 5, wherein the focusing step comprises the steps of:

basic feature extraction: inputting a single chromosome image by adopting a basic feature extraction network, and extracting single chromosome feature maps with multiple scales;

a multi-dimensional attention feature generation step: inputting the single chromosome feature maps of multiple scales into a multi-dimensional attention mechanism network to generate a multi-dimensional attention feature map;

and (3) multi-dimensional attention feature fusion: inputting the generated multidimensional attention feature map into a feature fusion network to generate a final fusion feature map;

mask extraction: inputting the final fusion feature map into a mask generation network, and extracting a mask of the salient region;

and mask processing: and generating a mask of the significant region, performing morphological processing, removing noise points, filling holes, and multiplying the mask with the input original image to obtain a single chromosome significant region image.

7. A chromosome image analysis system based on progressive segmentation and focus classification is characterized by comprising the following modules:

a progressive segmentation module: segmenting the metaphase chromosome image by utilizing progressive segmentation to obtain a single chromosome image set;

a focus classification module: and (3) identifying the categories of all the single chromosomes in the single chromosome image set by adopting a focusing classification algorithm, and finally generating a karyotype analysis chart.

8. The system for chromosome image analysis based on progressive segmentation and focus classification according to claim 7, wherein the progressive segmentation module comprises the following modules:

a global threshold segmentation module: setting pixel points lower than the threshold value as a first preset value and pixel points higher than the threshold value as a second preset value by calculating the average value of all pixel point values in the whole metaphase chromosome image as the threshold value, and dividing the whole metaphase chromosome image into a foreground and a background according to the first preset value and the second preset value;

a chromosome cluster classification module: determining a real chromosome cluster in the foreground, and classifying the real chromosome cluster through a chromosome cluster classification network to obtain the category information of the chromosome cluster;

chromosome instance segmentation module: and segmenting the chromosome cluster according to the category information of the chromosome cluster to obtain all single chromosome image sets in the metaphase chromosome image.

9. The system for chromosome image analysis based on progressive segmentation and focus classification according to claim 8, wherein the chromosome cluster classification module comprises the following modules:

and a foreground judging module: before classification implementation, a single chromosome and a real chromosome cluster of a foreground object are distinguished by using a convex hull and a preset external graph, wherein the single chromosome is directly output to a single chromosome set, and the real chromosome cluster is continuously segmented through the following modules;

a multi-scale feature map acquisition module: inputting the real chromosome cluster into a residual error network to obtain a multi-scale characteristic diagram;

the multi-scale feature map calibration module: inputting the multi-scale characteristic diagram to the respective extrusion excitation module, and outputting a calibration multi-scale characteristic diagram;

a multi-scale feature fusion module: performing characteristic fusion on the calibration multi-scale characteristic diagram through a formula to obtain a multi-scale fusion characteristic diagram;

a final feature map acquisition module: performing average pooling operation on the multi-scale fusion characteristic graphs, and performing characteristic connection to obtain final characteristic graphs of each chromosome cluster;

a category information acquisition module: obtaining the category information of the chromosome cluster after the final characteristic diagram passes through a plurality of layers of full-link layers and a normalized index function excitation function; the category information of the chromosome clusters is touch chromosome clusters, overlapping chromosome clusters and touch overlapping chromosome clusters.

10. The progressive segmentation and focus classification based chromosome image analysis system of claim 9, wherein the chromosome instance segmentation module comprises:

touch segmentation module: extracting a skeleton graph, a terminal point and a contour graph of the touch chromosome cluster to obtain a cutting point of the cross region;

connecting the pair-by-pair cutting points to obtain a guide line, performing expansion treatment on the guide line to obtain a connecting area, calculating the mean value of all pixel points in the connecting area as a threshold, reserving all pixel points in the connecting area which are lower than the threshold, processing the pixel points by using a minimum binary method, fitting a parting line between touch chromosomes, and dividing the touch chromosome cluster by using the parting line;

an overlap segmentation module: extracting an overlapping region of the overlapping chromosome clusters by adopting U-Net;

splicing the images on the two sides of the overlapping area with the overlapping area respectively to separate out all single chromosomes in the overlapping chromosome cluster;

touch overlap and divide the module: firstly, an overlapping chromosome cluster in a touch overlapping chromosome cluster is divided by adopting an overlapping division module, so that a plurality of single chromosomes in the touch overlapping chromosome cluster and a set of touch chromosome clusters are obtained, the touch chromosome cluster in the touch overlapping chromosome cluster is distinguished by utilizing a convex hull and a preset external graph, wherein the separated single chromosomes are directly output to the single chromosome set, and the touch chromosome cluster is continuously divided by the following modules;

a touch division module is adopted to divide touch chromosome clusters in the touch overlapped chromosome clusters;

single chromosome assembly module: through the touch segmentation module, the overlap segmentation module and the touch overlap segmentation module, the touch chromosome cluster, the overlap chromosome cluster and the touch overlap chromosome cluster are all segmented, and all single chromosome image sets in the metaphase chromosome image are obtained.

Images