CN111476754B - Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method - Google Patents

Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method Download PDF

Info

Publication number
CN111476754B
CN111476754B CN202010126812.6A CN202010126812A CN111476754B CN 111476754 B CN111476754 B CN 111476754B CN 202010126812 A CN202010126812 A CN 202010126812A CN 111476754 B CN111476754 B CN 111476754B
Authority
CN
China
Prior art keywords
bone marrow
cell
cells
image
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010126812.6A
Other languages
Chinese (zh)
Other versions
CN111476754A (en
Inventor
张曦
张�诚
彭贤贵
杨武晨
张洪洋
墙星
李佳
刘思恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Second Affiliated Hospital Army Medical University
Original Assignee
Second Affiliated Hospital Army Medical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Second Affiliated Hospital Army Medical University filed Critical Second Affiliated Hospital Army Medical University
Priority to CN202010126812.6A priority Critical patent/CN111476754B/en
Publication of CN111476754A publication Critical patent/CN111476754A/en
Application granted granted Critical
Publication of CN111476754B publication Critical patent/CN111476754B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/69Microscopic objects, e.g. biological cells or cellular parts
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/40ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30024Cell structures in vitro; Tissue sections in vitro
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30204Marker
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30242Counting objects in image

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biophysics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Computing Systems (AREA)
  • Pathology (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)

Abstract

The invention relates to the technical field of medical auxiliary diagnosis, and discloses a marrow cell image artificial intelligence auxiliary grading diagnosis system and a marrow cell image artificial intelligence auxiliary grading diagnosis method. The invention can not only automatically identify, label, count and analyze and diagnose the marrow cell image, but also manually label the marrow cell image, and store the manually labeled result into the storage module, and the system memorizes the manually labeled information, thereby gradually improving the discrimination capability of the system to the marrow cell image.

Description

Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method
Technical Field
The invention relates to the technical field of medical auxiliary diagnosis, in particular to a marrow cell image artificial intelligence auxiliary grading diagnosis system and a marrow cell image artificial intelligence auxiliary grading diagnosis method.
Background
Research shows that the annual growth rate of medical image data in China is about 30%, and the annual growth rate of the number of image diagnosticians is only about 4.1%, which means that the image diagnosticians will work in an overload manner in the future; this will necessarily reduce the efficiency of diagnosis by the physician, or even reduce the accuracy of diagnosis; in addition, since medical image diagnosis has high requirements on the diagnosis experience of physicians, in areas with low development level, the diagnosis physician with rich experience has relatively low resources.
With the development of modern medical science and technology, various new technologies gradually permeate into the medical field, and in order to relieve the pressure of image diagnosticians to a certain extent, artificial intelligence auxiliary diagnostic systems for images are proposed. The artificial intelligence and the medical image are used for specifically applying an artificial intelligence technology to diagnosis of the medical image, and are mainly divided into two parts at present, namely image recognition and application to a perception link, and the method mainly aims to analyze non-mechanization data such as the image and acquire some meaningful information. And secondly, deep learning, which is applied to learning and analyzing links and is the most core link of artificial intelligence application, and continuously performs deep learning training on a neural network through a large amount of image data and diagnosis data to promote the neural network to master the capability of diagnosis.
Patent with application number CN 201810866088.3 discloses an artificial intelligence medical image tumor malignancy risk stratification auxiliary diagnosis system, which comprises: the system comprises a data acquisition module, a data preprocessing module, a model establishing module, a model verifying and optimizing module, a layered diagnosis module and a database platform. The tumor malignant risk layering auxiliary diagnosis system is based on an artificial intelligence technology, can realize successive layering of malignant risks of tumors, simulates a clinical diagnosis idea, and automatically diagnoses space-occupying lesions with clear image characteristics by high-precision benign lesion detection and malignant tumor detection capabilities of an artificial intelligence model, so that the clinical management decision of the space-occupying lesions can be substantially assisted, the existing working process of clinical diagnosis is improved, the diagnosis confidence of doctors is increased, the working pressure is reduced, the anxiety of low-malignant-risk lesion patients is also reduced, and the diagnosis rate of the benign lesions and the malignant tumors is greatly improved.
Although the above patent implements the intelligent auxiliary diagnosis of the image, it is only applicable to the intelligent auxiliary diagnosis of the cell image of lung cancer, liver cancer, etc., and for the image of the bone marrow cell, it is necessary to identify the measured values according to the size of each cell in the bone marrow cell, the complexity of the particle and the nucleus, etc., to preliminarily determine and identify the cell characteristics.
Disclosure of Invention
In view of the above, the present invention provides an artificial intelligence aided hierarchical diagnosis system and method for bone marrow cell images, which can not only perform automatic identification, labeling, counting analysis and hierarchical diagnosis on bone marrow cell images, but also perform manual labeling on bone marrow cell images, store the results of the manual labeling in a storage module, and gradually improve the ability of the system to identify bone marrow cell images by memorizing the information of the manual labeling.
The invention solves the technical problems through the following technical means:
an artificial intelligent auxiliary grading diagnosis system for bone marrow cell images comprises a data acquisition unit, an automatic identification and labeling unit, a manual labeling unit, a cell data statistics unit, a data analysis and grading unit, an image auxiliary grading diagnosis unit, a display, a storage module and a processor, wherein the data acquisition unit, the automatic identification and labeling unit, the manual labeling unit, the cell data statistics unit, the data analysis and grading unit, the image auxiliary grading diagnosis unit, the storage module and the display are all in communication connection with the processor; the data acquisition unit comprises an image acquisition module for reading information of bone marrow cell images, a clinical information acquisition module for inputting clinical information of patients and a gene information acquisition module for inputting gene information of patients; the automatic identification and marking unit comprises a cell form standard module for expressing standard cell forms, a bone marrow cell form extraction module for automatically extracting the forms of bone marrow cells, a cell form comparison module for comparing the extracted information of bone marrow cell images with the cell form standard module, and a cell form automatic marking module for automatically marking the identified bone marrow cell forms; the manual labeling unit comprises a cell morphology manual labeling module and a labeling storage module, wherein the cell morphology manual labeling module is used for manually labeling the bone marrow cell images by a doctor, and the labeling storage module is used for manually storing labeling information of the doctor; the data analysis unit comprises a granulocyte ratio analysis module for calculating the ratio of nucleated cells to mature red blood cells, and a cell ratio analysis module for calculating the ratio of the number of each cell to the total number of the whole bone marrow cells.
Further, the image-assisted hierarchical diagnosis unit comprises a first stage, a second stage, a third stage and a fourth stage, wherein the first stage comprises a normal bone marrow elephant and an abnormal bone marrow elephant, the second stage is a large class of diseases comprising an anemia bone marrow elephant and a proliferative anemia bone marrow elephant, the third stage is a certain disease comprising an IDA bone marrow elephant, a CL bone marrow elephant and an AL bone marrow elephant, and the fourth stage is a certain disease containing a specific gene comprising an APL bone marrow elephant and a CML bone marrow elephant.
The data preprocessing unit comprises a data denoising module for denoising and data normalization processing of the data acquired by the image acquisition module, a lesion labeling module for searching lesions corresponding to pathological results and labeling the images, and a lesion segmentation module.
Further, the cell data statistical unit comprises a cell classification module and a cell counting module.
Further, the cell counting module classifies bone marrow cells as: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid cells, monocytic cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, and other granulocytic cells.
Further, the cell data analysis unit further comprises an error calibration module for discharging useless cell interference and accurately and efficiently classifying the bone marrow cells.
A method of marrow cell image artificial intelligence auxiliary grading diagnosis system includes the following steps:
a1, an image acquisition module reads information of a bone marrow cell image, a clinical information acquisition module inputs clinical information of a patient, and a gene information acquisition module inputs gene information of the patient;
a2, the data denoising module performs denoising and data normalization processing on the data acquired by the image acquisition module, searches for a pathological change corresponding to a pathological result and performs pathological change labeling on the image;
a3, automatically extracting the morphology of the bone marrow cells by a cell morphology extraction module, comparing the extracted information of the bone marrow cell image with a cell morphology standard module by a cell morphology comparison module, and automatically labeling the recognized bone marrow cell morphology by a cell morphology automatic labeling module;
a4, a doctor performs manual operation, manually marks the bone marrow cell image by using a manual marking module, and manually stores marking information of the doctor by using a marking storage module;
a5, an error calibration module discharges useless cell interference, and bone marrow cells are accurately and efficiently classified;
a6, dividing the bone marrow cells into: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid cells, monocytic cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other myeloid cells;
a7, calculating the ratio of nucleated cells to mature red blood cells by a granulocyte ratio analysis module, and calculating the ratio of the number of each cell in the total number of the whole bone marrow cells by a cell ratio analysis module;
a8, dividing the bone marrow image into: primary, secondary, tertiary and quaternary, wherein the primary comprises normal myeloid elephant and abnormal myeloid elephant, the secondary comprises anemia myeloid elephant and proliferative anemia myeloid elephant, the tertiary comprises IDA myeloid elephant, CL myeloid elephant and AL myeloid elephant, and the quaternary comprises APL myeloid elephant and CML myeloid elephant.
Further, the error calibration method of the error calibration module comprises the following steps:
b1, obtaining a stained bone marrow cell image, mapping the bone marrow cell image to an HSV space, and separating an S channel;
b2, drawing a histogram of the S-channel image, binarizing the S-channel image according to a threshold range to obtain a binary image of the bone marrow cells, and performing morphological processing on the binary image;
b3, extracting edge pixel points of the bone marrow cell image after morphological processing by using a connected domain method, finding edge pixel points of the bone marrow cell in the upper, lower, left and right directions, and then dividing the bone marrow cell;
b4, selecting the images of the segmented bone marrow cells, inputting the cell images with obvious characteristics in each class as training cells into a depth residual error network, and training the network;
b5, taking the remaining cells after selection as test cells, and grading the test cells by using a softmax classifier; if the maximum score is greater than or equal to a set threshold, classifying the score into a certain class; if the maximum score is smaller than a set threshold, classifying the score into a sub-classification;
b6, taking any pixel point at the edge of the training cells and the cells in the sub-classification in the step S5 as a pole, establishing a polar coordinate system, and mapping all pixel points into a rectangular coordinate system one by one through polar coordinate transformation;
b7, traversing edge pixel points of the training cells, wherein each pixel point generates a converted image, and each image has n pixel points, namely n times of conversion; for cells in the sub-classification, each image is transformed only once;
b8, the image after the training cell transformation is used as the input of the sub-classification network, the deep residual error network is retrained, and the network parameters are stored; taking the image after cell transformation in the sub-classification as test data, and scoring again by using a softmax classifier: if the maximum score of the cell image is greater than or equal to a set threshold, classifying the cell image into a certain subcategory; if the maximum score is less than a set threshold, the cell image is classified as an unclassified cell.
Further, if the proportion of unclassified cells is greater than a set unclassified threshold, then the unclassified cells are segmented and classified, and the method comprises the following steps of C1, taking edge pixel points of a bone marrow cell image after morphological processing for the classified cells, finding the edge pixel points above, below, left and right the bone marrow cells, and segmenting the bone marrow cells;
c2, inputting the segmented marrow cell image as a training cell into a depth residual error network and training the network;
c3, scoring the test cells by using a softmax classifier, and if the maximum score is greater than or equal to a set threshold value, classifying the test cells into a certain class; if the maximum fraction is smaller than a set threshold value, classifying the cells into finally unclassified cells;
and C4, stopping if the final unclassified cell proportion is smaller than the set unclassified threshold, and continuing the step C1 until the final unclassified cell proportion is smaller than the set unclassified threshold if the final unclassified cell proportion is larger than the set unclassified threshold.
The invention has the beneficial effects that:
(1) The data acquisition unit is used for reading information of bone marrow cell images, inputting clinical information of patients and inputting gene information of the patients; then, the data noise reduction module is used for carrying out noise reduction and data normalization processing on the data acquired by the image acquisition module, searching for pathological changes corresponding to pathological results and carrying out pathological change marking on the images; then automatically extracting the morphology of the bone marrow cells, comparing the extracted information of the bone marrow cell images with a cell morphology standard module by a cell morphology comparison module, and automatically labeling the recognized bone marrow cell morphology by a cell morphology automatic labeling module; and the cell counting module divides the bone marrow cells into: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid cells, monocytic cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other myeloid cells; calculating the ratio of nucleated cells to mature red blood cells, and calculating the ratio of the number of each cell in the total number of the whole bone marrow cells by a cell ratio analysis module; and finally, dividing the marrow picture into: the whole system can carry out intelligent grading diagnosis on bone marrow cell images, and the workload of imaging doctors is reduced.
(2) According to the invention, through the arrangement of the manual labeling unit and the data storage unit, when the system is used, the accuracy of the auxiliary grading diagnosis system for identifying cells can be distinguished one by one, correct identification is confirmed and stored, and error correction and correction are carried out on wrong identification; therefore, in the process of using the invention, the identification capability of the bone marrow cell image can be gradually improved, and the aim of really and completely identifying the image by artificial intelligence is fulfilled.
Drawings
FIG. 1 is a schematic diagram of an artificial intelligence-aided hierarchical diagnosis system for bone marrow cell imaging according to the present invention;
FIG. 2 is a flow chart of an error calibration method of the error calibration module of the present invention.
Detailed Description
The invention will be described in detail below with reference to the following drawings:
as shown in fig. 1-2:
an artificial intelligence auxiliary grading diagnosis system for marrow cell images is shown in figure 1 and comprises a data acquisition unit, an automatic identification and labeling unit, a manual labeling unit, a cell data statistics unit, a data analysis and grading unit, an image auxiliary grading diagnosis unit, a display, a storage module and a processor, wherein the data acquisition unit, the automatic identification and labeling unit, the manual labeling unit, the cell data statistics unit, the data analysis and grading unit, the image auxiliary grading diagnosis unit, the storage module and the display are all in communication connection with the processor; the data acquisition unit comprises an image acquisition module for reading information of bone marrow cell images, a clinical information acquisition module for inputting clinical information of patients and a gene information acquisition module for inputting gene information of patients; the automatic identification and marking unit comprises a cell form standard module for representing standard cell forms, a bone marrow cell form extraction module for automatically extracting the forms of bone marrow cells, a cell form comparison module for comparing the extracted information of bone marrow cell images with the cell form standard module, and a cell form automatic marking module for automatically marking the identified bone marrow cell forms; the manual labeling unit comprises a cell morphology manual labeling module and a labeling storage module, wherein the cell morphology manual labeling module is used for manually labeling the bone marrow cell images by a doctor, and the labeling storage module is used for manually storing labeling information of the doctor; the data analysis unit comprises a granulocyte-erythrocyte ratio analysis module for calculating the ratio of nucleated cells to mature erythrocytes and a cell ratio analysis module for calculating the ratio of the number of each cell to the total number of the whole bone marrow cells; the image-assisted grading diagnosis unit comprises a first grade, a second grade, a third grade and a fourth grade, wherein the first grade comprises a normal bone marrow image and an abnormal bone marrow image, the second grade comprises an anemia bone marrow image and a hyperplastic anemia bone marrow image, the third grade comprises an IDA bone marrow image, a CL bone marrow image and an AL bone marrow image, and the fourth grade comprises an APL bone marrow image and a CML bone marrow image. The image acquisition module is used for acquiring image data, and the data preprocessing unit comprises a data denoising module for denoising and data normalization processing the data acquired by the image acquisition module, a lesion labeling module for searching lesions corresponding to pathological results and labeling the images with the lesions, and a lesion segmentation module. The cell data statistical unit comprises a cell classification module and a cell counting module. The cell counting module classifies bone marrow cells as: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid lineage cells, monocyte lineage cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other granulocytic lineage cells. The cell data analysis unit also comprises an error calibration module which is used for eliminating useless cell interference and accurately and efficiently classifying the bone marrow cells.
The invention relates to an auxiliary diagnosis method of a marrow cell image artificial intelligence auxiliary grading diagnosis system, which comprises the following steps:
a1, an image acquisition module reads information of a bone marrow cell image, a clinical information acquisition module inputs clinical information of a patient, and a gene information acquisition module inputs gene information of the patient;
a2, the data noise reduction module carries out noise reduction and data normalization processing on the data acquired by the image acquisition module, searches for pathological changes corresponding to pathological results and carries out pathological change marking on the image;
a3, automatically extracting the morphology of the bone marrow cells by a cell morphology extraction module, comparing the extracted information of the bone marrow cell image with a cell morphology standard module by a cell morphology comparison module, and automatically labeling the recognized bone marrow cell morphology by a cell morphology automatic labeling module;
a4, a doctor performs manual operation, manually marks the bone marrow cell image by using a manual marking module, and manually marks information for the doctor by using a marking storage module;
a5, an error calibration module discharges useless cell interference, and bone marrow cells are accurately and efficiently classified;
a6, dividing bone marrow cells into: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid cells, monocytic cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other myeloid cells;
a7, calculating the ratio of nucleated cells to mature red blood cells by a granulocyte ratio analysis module, and calculating the ratio of the number of each cell in the total number of the whole bone marrow cells by a cell ratio analysis module;
a8, dividing the bone marrow image into: primary, secondary, tertiary and quaternary, wherein the primary comprises normal myeloid elephant and abnormal myeloid elephant, the secondary comprises anemia myeloid elephant and proliferative anemia myeloid elephant, the tertiary comprises IDA myeloid elephant, CL myeloid elephant and AL myeloid elephant, and the quaternary comprises APL myeloid elephant and CML myeloid elephant.
As shown in fig. 2, the error calibration method of the error calibration module in the present system includes the following steps:
b1, obtaining a stained bone marrow cell image, mapping the bone marrow cell image to an HSV space, and separating an S channel;
b2, drawing a histogram of the S-channel image, binarizing the S-channel image according to a threshold range to obtain a binary image of the bone marrow cells, and performing morphological processing on the binary image;
b3, extracting edge pixel points of the bone marrow cell image after morphological processing by using a connected domain method, finding edge pixel points on the upper, lower, left and right sides of the bone marrow cell, and then segmenting the bone marrow cell;
b4, selecting the images of the segmented bone marrow cells, inputting the cell images with obvious characteristics in each class as training cells into a depth residual error network, and training the network;
b5, taking the selected residual cells as test cells, and scoring the test cells by using a softmax classifier; if the maximum score is greater than or equal to a set threshold, classifying the score into a certain class; if the maximum score is smaller than a set threshold, classifying the score into a sub-classification;
b6, taking any pixel point at the edge of the training cells and the cells in the sub-classification in the step S5 as a pole, establishing a polar coordinate system, and mapping all pixel points into a rectangular coordinate system one by one through polar coordinate transformation;
b7, traversing edge pixel points of the training cells, wherein each pixel point generates a converted image, and each image has n pixel points, namely n times of conversion; for cells in the sub-classification, each image is transformed only once;
b8, the image after the training cell transformation is used as the input of the sub-classification network, the deep residual error network is retrained, and the network parameters are stored; taking the image after cell transformation in the sub-classification as test data, and scoring again by using a softmax classifier: if the maximum score of the cell image is greater than or equal to a set threshold, classifying the cell image into a certain subcategory; if the maximum score is less than a set threshold, the cell image is classified as an unclassified cell. If the proportion of the unclassified cells is more than 5 percent of the set unclassified threshold value, then the unclassified cells are segmented and classified, comprising the following steps,
c1, taking edge pixel points of the bone marrow cell image after morphological processing for classifying cells, finding edge pixel points on the upper side, the lower side, the left side and the right side of the bone marrow cells, and segmenting the bone marrow cells;
c2, inputting the segmented marrow cell image as a training cell into a depth residual error network and training the network;
c3, scoring the test cells by using a softmax classifier, and if the maximum score is greater than or equal to a set threshold value, classifying the test cells into a certain class; if the maximum fraction is smaller than a set threshold value, classifying the cells into final unclassified cells;
and C4, stopping if the final unclassified cell proportion is less than 5 percent of the set unclassified threshold value, and continuing the step C1 if the final unclassified cell proportion is more than the set unclassified threshold value until the final unclassified cell proportion is less than the set unclassified threshold value.
The use process of the invention is as follows:
reading information of the bone marrow cell image by using a data acquisition unit, inputting clinical information of a patient, and inputting gene information of the patient; then, the data noise reduction module is used for carrying out noise reduction and data normalization processing on the data acquired by the image acquisition module, searching for pathological changes corresponding to pathological results and carrying out pathological change marking on the images; then automatically extracting the morphology of the bone marrow cells, comparing the extracted information of the bone marrow cell image with a cell morphology standard module by a cell morphology comparison module, and automatically labeling the recognized bone marrow cell morphology by a cell morphology automatic labeling module; and the cell counting module divides the bone marrow cells into: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid cells, monocytic cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other myeloid cells; calculating the ratio of nucleated cells to mature red blood cells, and calculating the ratio of the number of each cell in the total number of the whole bone marrow cells by a cell ratio analysis module; and finally, dividing the bone marrow image into: first grade, second grade, third grade, level four, entire system can carry out intelligent hierarchical diagnosis to the marrow cell image, reduces image doctor's work load.
Although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the true spirit and scope of the present invention, which is defined by the claims appended hereto. The techniques, shapes, and configurations not described in detail in the present invention are all known techniques.

Claims (5)

1. The artificial intelligence auxiliary grading diagnosis system for the bone marrow cell image is characterized in that: the cell analysis and classification system comprises a data acquisition unit, an automatic identification and labeling unit, a manual labeling unit, a cell data statistics unit, a data analysis and classification unit, an image auxiliary classification diagnosis unit, a display, a storage module and a processor, wherein the data acquisition unit, the automatic identification and labeling unit, the manual labeling unit, the cell data statistics unit, the data analysis and classification unit, the image auxiliary classification diagnosis unit, the storage module and the display are all in communication connection with the processor;
the data acquisition unit comprises an image acquisition module for reading information of bone marrow cell images, a clinical information acquisition module for inputting clinical information of patients and a gene information acquisition module for inputting gene information of patients;
the automatic identification and marking unit comprises a cell form standard module for expressing standard cell forms, a bone marrow cell form extraction module for automatically extracting the forms of bone marrow cells, a cell form comparison module for comparing the extracted information of bone marrow cell images with the cell form standard module, and a cell form automatic marking module for automatically marking the identified bone marrow cell forms;
the manual labeling unit comprises a cell morphology manual labeling module and a labeling storage module, wherein the cell morphology manual labeling module is used for manually labeling the bone marrow cell images by a doctor, and the labeling storage module is used for manually storing labeling information of the doctor;
the data analysis and grading unit comprises a granulocyte-erythrocyte ratio analysis module for calculating the ratio of nucleated cells to mature erythrocytes and a cell ratio analysis module for calculating the ratio of the number of each cell to the total number of the whole bone marrow cells;
the data preprocessing unit comprises a data denoising module for denoising and normalizing data acquired by the image acquisition module, a lesion labeling module for searching a lesion corresponding to a pathological result and labeling the image with the lesion and a lesion segmentation module; the cell data analysis and grading unit also comprises an error calibration module which is used for eliminating useless cell interference and accurately and efficiently classifying the bone marrow cells; the cell data statistical unit comprises a cell classification module and a cell counting module;
the marrow cell image artificial intelligence auxiliary grading diagnosis method includes the following steps:
a1, an image acquisition module reads information of a bone marrow cell image, a clinical information acquisition module inputs clinical information of a patient, and a gene information acquisition module inputs gene information of the patient;
a2, the data denoising module performs denoising and data normalization processing on the data acquired by the image acquisition module, searches for a pathological change corresponding to a pathological result and performs pathological change labeling on the image;
a3, automatically extracting the morphology of the bone marrow cells by a cell morphology extraction module, comparing the extracted information of the bone marrow cell image with a cell morphology standard module by a cell morphology comparison module, and automatically labeling the recognized bone marrow cell morphology by a cell morphology automatic labeling module;
a4, a doctor performs manual operation, manually marks the bone marrow cell image by using a manual marking module, and manually marks information for the doctor by using a marking storage module;
a5, an error calibration module discharges useless cell interference, and bone marrow cells are accurately and efficiently classified;
a6, dividing bone marrow cells into: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid cells, monocytic cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other myeloid cells;
a7, calculating the ratio of nucleated cells to mature red blood cells by a granulocyte ratio analysis module, and calculating the ratio of the number of each cell in the total number of the whole bone marrow cells by a cell ratio analysis module;
a8, dividing the bone marrow image into: the first grade comprises normal bone marrow elephant and abnormal bone marrow elephant, the second grade comprises anemia bone marrow elephant and hyperplastic anemia bone marrow elephant, the third grade comprises IDA bone marrow elephant, CL bone marrow elephant and AL bone marrow elephant, and the fourth grade comprises APL bone marrow and CML bone marrow elephant.
2. The system of claim 1, wherein the system comprises: the image-assisted grading diagnosis unit comprises a first grade, a second grade, a third grade and a fourth grade, wherein the first grade comprises a normal bone marrow elephant and an abnormal bone marrow elephant, the second grade is a large class of diseases comprising an anemia bone marrow elephant and a proliferative anemia bone marrow elephant, the third grade comprises an IDA bone marrow elephant, a CL bone marrow elephant and an AL bone marrow elephant, and the fourth grade comprises an APL bone marrow elephant and a CML bone marrow elephant.
3. The system of claim 2, wherein the bone marrow cell image artificial intelligence aided grading diagnosis system comprises: the cell counting module divides the bone marrow cells into: mesoblast, metablast, other erythroid cells, blasts, mature lymphocytes, other lymphoid lineage cells, monocyte lineage cells, promyelocytes, mesogranulocytes, metagranulocytes, rhabdocytes, subtenocytes, and other granulocytic lineage cells.
4. The system of claim 3, wherein the bone marrow cell image artificial intelligence aided grading diagnosis system comprises: the error calibration method of the error calibration module comprises the following steps:
b1, obtaining a stained bone marrow cell image, mapping the bone marrow cell image to an HSV space, and separating an S channel;
b2, drawing a histogram of the S-channel image, binarizing the S-channel image according to a threshold range to obtain a binary image of the bone marrow cells, and performing morphological processing on the binary image;
b3, extracting edge pixel points of the bone marrow cell image after morphological processing by using a connected domain method, finding edge pixel points on the upper, lower, left and right sides of the bone marrow cell, and then segmenting the bone marrow cell;
b4, selecting the images of the segmented bone marrow cells, inputting the cell images with obvious characteristics in each class as training cells into a depth residual error network, and training the network;
b5, taking the remaining cells after selection as test cells, and grading the test cells by using a softmax classifier; if the maximum score is greater than or equal to a set threshold, classifying the score as a certain class; if the maximum score is smaller than a set threshold, classifying the score into a sub-classification;
b6, taking any pixel point on the edge of the training cells and the cells in the sub-classification in the step B5 as a pole, establishing a polar coordinate system, and mapping all pixel points into a rectangular coordinate system one by one through polar coordinate transformation;
b7, traversing edge pixel points of the training cells, wherein each pixel point generates a converted image, and each image has n pixel points, namely n times of conversion; for cells in the sub-classification, each image is transformed only once;
b8, the image after the training cell transformation is used as the input of the sub-classification network, the deep residual error network is retrained, and the network parameters are stored; taking the image after cell transformation in the sub-classification as test data, and scoring again by using a softmax classifier: if the maximum score of the cell image is greater than or equal to a set threshold, classifying the cell image into a certain subcategory; if the maximum score is less than a set threshold, the cell image is classified as an unclassified cell.
5. The system of claim 4, wherein the system comprises: if the proportion of the unclassified cells is greater than the set unclassified threshold, then the unclassified cells are segmented and classified, and the method comprises the following steps of C1, taking edge pixel points of a bone marrow cell image after morphological processing for the classified cells, finding edge pixel points above, below, left and right the bone marrow cells, and segmenting the bone marrow cells;
c2, inputting the segmented marrow cell image as a training cell into a depth residual error network and training the network;
c3, scoring the test cells by using a softmax classifier, and if the maximum score is greater than or equal to a set threshold value, classifying the test cells into a certain class; if the maximum fraction is smaller than a set threshold value, classifying the cells into finally unclassified cells;
and C4, stopping if the final unclassified cell proportion is smaller than the set unclassified threshold, and continuing the step C1 until the final unclassified cell proportion is smaller than the set unclassified threshold if the final unclassified cell proportion is larger than the set unclassified threshold.
CN202010126812.6A 2020-02-28 2020-02-28 Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method Active CN111476754B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010126812.6A CN111476754B (en) 2020-02-28 2020-02-28 Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010126812.6A CN111476754B (en) 2020-02-28 2020-02-28 Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method

Publications (2)

Publication Number Publication Date
CN111476754A CN111476754A (en) 2020-07-31
CN111476754B true CN111476754B (en) 2022-12-09

Family

ID=71748011

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010126812.6A Active CN111476754B (en) 2020-02-28 2020-02-28 Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method

Country Status (1)

Country Link
CN (1) CN111476754B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112201336B (en) * 2020-09-28 2023-08-11 暨南大学 Remote consultation system and method based on AI diagnosis
CN113408480B (en) * 2021-07-13 2022-09-30 上海交通大学医学院附属瑞金医院 Artificial intelligent auxiliary diagnosis system for blood diseases based on bone marrow cell morphology
CN113945558B (en) * 2021-09-29 2024-05-24 杭州智微信息科技有限公司 Method for judging myeloproliferation degree
CN114121291A (en) * 2021-10-26 2022-03-01 泰康保险集团股份有限公司 Disease grading prediction method and device, electronic equipment and storage medium
CN114697547A (en) * 2022-02-14 2022-07-01 上海奕瑞光电子科技股份有限公司 Image diagnosis system and image risk processing method
CN115578598B (en) * 2022-10-26 2023-09-05 北京大学第三医院(北京大学第三临床医学院) Bone marrow cell identification method and system based on convolutional neural network

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1646910A (en) * 2002-04-09 2005-07-27 学校法人东海大学 Method of judging leukemia, pre-leukemia or aleukemic malignant blood disease and diagnostic therefor
CN101705302A (en) * 2009-11-05 2010-05-12 北京大学人民医院 Kit for assisting to diagnose multiple myeloma
CN102298700A (en) * 2011-06-09 2011-12-28 华东师范大学 Method for recognizing and positioning cells in bone marrow pathology image
CN105821002A (en) * 2016-02-16 2016-08-03 复旦大学附属华山医院 Highly aggressive human acute B lymphocytic leukemia cell strain with add(11)(q23) chromosome abnormality
WO2017046747A1 (en) * 2015-09-15 2017-03-23 Acerta Pharma B.V. Therapeutic combinations of a cd19 inhibitor and a btk inhibitor
WO2017126420A1 (en) * 2016-01-19 2017-07-27 コニカミノルタ株式会社 Image processing device and program
CN108038472A (en) * 2017-12-27 2018-05-15 杭州智微信息科技有限公司 A kind of exchange method of bone marrow examination domain software fast finding abnormal cell
CN108338990A (en) * 2018-04-11 2018-07-31 中国医学科学院基础医学研究所 Application of the water-soluble realgar solid dispersions in preparing hematopoietic progenitor erythroid differentiation derivant
CN108961208A (en) * 2018-05-21 2018-12-07 江苏康尚生物医疗科技有限公司 A kind of aggregation leucocyte segmentation number system and method
CN109166105A (en) * 2018-08-01 2019-01-08 中国人民解放军南京军区南京总医院 The malignancy of tumor risk stratification assistant diagnosis system of artificial intelligence medical image
WO2019016169A1 (en) * 2017-07-16 2019-01-24 Imec Vzw Cell recognition
CN109360198A (en) * 2018-10-08 2019-02-19 北京羽医甘蓝信息技术有限公司 Bone marrwo cell sorting method and sorter based on deep learning
CN109754398A (en) * 2019-01-14 2019-05-14 武汉兰丁医学高科技有限公司 A kind of computer based myeloplast automark method and system
CN110060229A (en) * 2019-05-29 2019-07-26 武汉兰丁医学高科技有限公司 A kind of cell automatic positioning dividing method of myeloplast

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160223554A1 (en) * 2011-08-05 2016-08-04 Nodality, Inc. Methods for diagnosis, prognosis and methods of treatment

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1646910A (en) * 2002-04-09 2005-07-27 学校法人东海大学 Method of judging leukemia, pre-leukemia or aleukemic malignant blood disease and diagnostic therefor
CN101705302A (en) * 2009-11-05 2010-05-12 北京大学人民医院 Kit for assisting to diagnose multiple myeloma
CN102298700A (en) * 2011-06-09 2011-12-28 华东师范大学 Method for recognizing and positioning cells in bone marrow pathology image
WO2017046747A1 (en) * 2015-09-15 2017-03-23 Acerta Pharma B.V. Therapeutic combinations of a cd19 inhibitor and a btk inhibitor
WO2017126420A1 (en) * 2016-01-19 2017-07-27 コニカミノルタ株式会社 Image processing device and program
CN105821002A (en) * 2016-02-16 2016-08-03 复旦大学附属华山医院 Highly aggressive human acute B lymphocytic leukemia cell strain with add(11)(q23) chromosome abnormality
WO2019016169A1 (en) * 2017-07-16 2019-01-24 Imec Vzw Cell recognition
CN108038472A (en) * 2017-12-27 2018-05-15 杭州智微信息科技有限公司 A kind of exchange method of bone marrow examination domain software fast finding abnormal cell
CN108338990A (en) * 2018-04-11 2018-07-31 中国医学科学院基础医学研究所 Application of the water-soluble realgar solid dispersions in preparing hematopoietic progenitor erythroid differentiation derivant
CN108961208A (en) * 2018-05-21 2018-12-07 江苏康尚生物医疗科技有限公司 A kind of aggregation leucocyte segmentation number system and method
CN109166105A (en) * 2018-08-01 2019-01-08 中国人民解放军南京军区南京总医院 The malignancy of tumor risk stratification assistant diagnosis system of artificial intelligence medical image
CN109360198A (en) * 2018-10-08 2019-02-19 北京羽医甘蓝信息技术有限公司 Bone marrwo cell sorting method and sorter based on deep learning
CN109754398A (en) * 2019-01-14 2019-05-14 武汉兰丁医学高科技有限公司 A kind of computer based myeloplast automark method and system
CN110060229A (en) * 2019-05-29 2019-07-26 武汉兰丁医学高科技有限公司 A kind of cell automatic positioning dividing method of myeloplast

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
"Automatic Bone Marrow Cell Identification and Classification By Deep Neural Network";Ta-ChuanYuMD 等;《Blood》;20191113;第134卷;2084 *
"CAP骨髓细胞形态学能力比对检验结果分析";赵瑾 等;《标记免疫分析与临床》;20180525;第25卷(第5期);718-721+726 *
"CD82在儿童急性白血病中异常表达及作用机制研究";纪红燕;《中国博士学位论文全文数据库 (医药卫生科技辑)》;20190915(第(2019)09期);E069-24 *
"Segmentation and Classification of Bone Marrow Cells Images Using Contextual Information for Medical Diagnosis of Acute Leukemias ";Reta C等;《PLoS One》;20150624;第10卷(第6期);1-18 *
"为你揭秘——检验科细胞形态学检查";吴胜;《健康人生》;20191015(第(2019)10期);39-40 *
"定量探讨双源双能量CT在正常椎体骨髓成像及白血病疗效评估中的价值研究";王红;《中国优秀硕士学位论文全文数据库 (医药卫生科技辑)》;20190915(第(2019)09期);E060-241 *
"面向急性白血病诊断的骨髓细胞形态学自动检验关键技术研究";师婷婷;《中国优秀硕士学位论文全文数据库 (信息科技辑)》;20181215(第(2018)12期);I138-1486 *

Also Published As

Publication number Publication date
CN111476754A (en) 2020-07-31

Similar Documents

Publication Publication Date Title
CN111476754B (en) Bone marrow cell image artificial intelligence auxiliary grading diagnosis system and method
CN108573490B (en) Intelligent film reading system for tumor image data
CN111986150B (en) The method comprises the following steps of: digital number pathological image Interactive annotation refining method
Dundar et al. Computerized classification of intraductal breast lesions using histopathological images
CN108464840B (en) Automatic detection method and system for breast lumps
CN112101451B (en) Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block
CN106248559B (en) A kind of five sorting technique of leucocyte based on deep learning
CN111028206A (en) Prostate cancer automatic detection and classification system based on deep learning
CN107280697A (en) Lung neoplasm grading determination method and system based on deep learning and data fusion
CN110335665A (en) It is a kind of applied to medical image auxiliary diagnosis analysis to scheme to search drawing method and system
CN108305249A (en) The quick diagnosis and methods of marking of full size pathological section based on deep learning
CN109635846A (en) A kind of multiclass medical image judgment method and system
CN109003269A (en) A kind of mark extracting method for the medical image lesion that can improve doctor's efficiency
Bai et al. Automatic segmentation of cervical region in colposcopic images using K-means
CN102737379A (en) Captive test (CT) image partitioning method based on adaptive learning
CN112085113B (en) Severe tumor image recognition system and method
CN110910371A (en) Liver tumor automatic classification method and device based on physiological indexes and image fusion
CN111986148B (en) Quick Gleason scoring system for digital pathology image of prostate
CN112767355A (en) Method and device for constructing thyroid nodule Tirads grading automatic identification model
CN112419396B (en) Automatic thyroid ultrasonic video analysis method and system
CN112579808B (en) Data annotation processing method, device and system
WO2013019856A1 (en) Automated malignancy detection in breast histopathological images
CN111738997A (en) Method for calculating new coronary pneumonia lesion area ratio based on deep learning
CN112116559A (en) Digital pathological image intelligent analysis method based on deep learning
CN112504947A (en) Morphological analysis and counting method for peripheral blood cells

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant