CN113205490A - Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method - Google Patents

Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method Download PDF

Info

Publication number
CN113205490A
CN113205490A CN202110418327.0A CN202110418327A CN113205490A CN 113205490 A CN113205490 A CN 113205490A CN 202110418327 A CN202110418327 A CN 202110418327A CN 113205490 A CN113205490 A CN 113205490A
Authority
CN
China
Prior art keywords
image
mask
information
cnn network
diagnosis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110418327.0A
Other languages
Chinese (zh)
Inventor
骆汉宾
张兆辉
张佳乐
聂淑科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202110418327.0A priority Critical patent/CN113205490A/en
Publication of CN113205490A publication Critical patent/CN113205490A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computed tomography [CT]
    • A61B6/032—Transmission computed tomography [CT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
    • A61B6/501—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of the head, e.g. neuroimaging or craniography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
    • A61B6/504—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of blood vessels, e.g. by angiography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30016—Brain

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Public Health (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Physics & Mathematics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Optics & Photonics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Computational Linguistics (AREA)
  • Databases & Information Systems (AREA)
  • Pulmonology (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)

Abstract

The invention discloses an auxiliary diagnosis system and an auxiliary diagnosis information generation method based on a Mask R-CNN network, belonging to the technical field of medical image processing and segmentation, wherein the system comprises a data uploading end, a data processing and segmentation end and a data processing and segmentation end, wherein the data uploading end is used for uploading a CT image of a patient and corresponding medical auxiliary information, and the CT image carries cerebral hemorrhage information of the patient; the image processor is connected with the data uploading end and is used for performing enhancement and graying on the CT image to obtain a gray image; the target detection module is connected with the image processor and is used for detecting the gray level image by utilizing the trained Mask R-CNN network model so as to identify and extract focus characteristic information; and the diagnosis analysis module is connected with the target detection module and used for matching the focus characteristic information with the structured medical record in the case database and synthesizing medical auxiliary information to generate auxiliary diagnosis information. The invention utilizes the trained Mask R-CNN network model to scan the brain CT image to generate auxiliary diagnosis information, thereby improving the efficiency of screening the cerebral hemorrhage in emergency treatment.

Description

Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method
Technical Field
The invention belongs to the technical field of medical image processing and segmentation, and particularly relates to a Mask R-CNN network-based auxiliary diagnosis system and an auxiliary diagnosis information generation method.
Background
Cerebral hemorrhage (cerebral hemorrhage) refers to hemorrhage caused by rupture of blood vessels in non-traumatic brain parenchyma, and accounts for 20% -30% of all cerebral apoplexy, and the acute stage fatality rate is 30% -40%, wherein about 80% of hemorrhage parts occur in cerebral hemisphere, and about 20% of hemorrhage occurs in brain stem and cerebellum. The disease type with the highest fatality rate in acute cerebrovascular diseases is mainly caused by cerebrovascular diseases, namely, the disease type is closely related to hyperlipidemia, diabetes, hypertension, vascular aging, smoking and the like.
The current domestic preferred imaging examination item is craniocerebral CT. Wherein, the CT flat scan can rapidly and accurately display the cerebral hemorrhage part, the hemorrhage amount and the space occupying effect, whether the cerebral hemorrhage part breaks into the ventricle or the subarachnoid space and whether the cerebral hemorrhage part damages to the peripheral brain tissue; enhanced CT scanning may reveal that the contrast agent spills into the hematoma, thus suggesting a risk of enlargement of the patient's hematoma; perfusion CT can reflect the hemodynamic change of brain tissue after cerebral hemorrhage, and can know the peripheral blood perfusion condition of hematoma.
The cerebral hemorrhage is typical on CT, generally, the cerebral hemorrhage is an ellipse, kidney-shaped or irregular high-density (75-80 Hu) shadow, and the boundary is clear. Edema around the hematoma may appear after several hours of bleeding, with the edema zone being of low density. Meanwhile, it is seen that the brain parenchyma is pressed, the sulci and the gyrus are shallow, and the medial line structure in the brain is displaced due to hematoma and edema. However, because cerebral hemorrhage is an acute disease, missed diagnosis or misdiagnosis is easy to occur due to the fatigue of doctors when patients are sent to doctors; or the experience of young doctors is insufficient, the problems are easy to occur, and the use efficiency is limited in a real emergency environment. The predicament is troubling the stroke medical field at home and abroad for a long time, namely the diagnosis efficiency of cerebral hemorrhage patients depends on the interpretation level of a doctor on CT images to a great extent.
Disclosure of Invention
Aiming at the defects or the improvement requirements of the prior art, the invention provides an auxiliary diagnosis system and an auxiliary diagnosis information generation method based on a Mask R-CNN network, and aims to utilize a trained Mask R-CNN network model to scan a brain CT image to generate auxiliary diagnosis information in the process of diagnosing a cerebral hemorrhage patient, so that the efficiency of screening the cerebral hemorrhage in emergency treatment is improved.
To achieve the above object, according to an aspect of the present invention, there is provided a Mask R-CNN network-based auxiliary diagnostic system, including:
the data uploading end is used for uploading a CT image of a patient and corresponding medical auxiliary information, and the CT image carries the cerebral hemorrhage information of the patient;
the image processor is connected with the data uploading end and used for performing enhancement and graying on the CT image to obtain a gray image;
the target detection module is connected with the image processor and used for detecting the gray level image by using the trained Mask R-CNN network model so as to identify and extract focus characteristic information;
and the diagnosis analysis module is connected with the target detection module and used for matching the focus characteristic information with the structured medical record in the case database and synthesizing the medical auxiliary information to generate auxiliary diagnosis information.
In one embodiment, the image processor is used for
Carrying out nonlinear smooth median filtering on the CT image, and then utilizing a cumulative distribution function:
Figure BDA0003026870440000021
Figure BDA0003026870440000022
carrying out histogram equalization on the filtered CT image, wherein n is the total number of pixels of the CT image, and L is the cumulative frequency;
and (3) establishing a weighted average of the brightness and R, G, B three color components by utilizing the RGB and YUV color space variation relation in the equalized CT image: the grayscale image was obtained when Color is 0.35R +0.59G + 0.11B.
In one embodiment, the system further comprises: a model construction module for
Creating a Mask R-CNN network model comprising an RPN network and a ROIPool layer by using the Mask R-CNN network, wherein the PN network is used for finishing FasterR-CNN extraction of an image candidate region, the ROIPool layer is used for detecting a candidate region target, and the RPN and the ROIPool layer are convolutional neural networks with different dimensions;
defining the Loss function target of the ROIPool layer as follows: l ═ LCLS+LBOX+LMASKTraining the Mask R-CNN network model; wherein L isCLSAnd LBOXClassification and regression losses, L, respectively, for Fast RCNNMAXIs lost as a mask branch.
In one embodiment, the object detection module is configured to,
inputting the gray level image into a pre-trained Mask R-CNN network model to obtain a corresponding characteristic diagram;
setting a predetermined number of ROI (regions of interest) for each point in the feature map to obtain a plurality of candidate ROI (regions of interest), and then sending the candidate ROI into an RPN (resilient packet network) to perform binary classification and BB regression to filter out partial candidate ROI;
roilign manipulation, classification, BB regression, and MASK were performed on the remaining ROIs to identify and extract the lesion feature information.
In one embodiment, the diagnostic module is further configured to,
indexing the focus characteristic information and a structured medical record in a case database to obtain a diagnosis prompt, wherein the focus characteristic information comprises a cerebral hemorrhage position and a cerebral hemorrhage volume;
synthesizing the diagnosis assistance information with the diagnosis prompt and the medical assistance information.
In one embodiment, the diagnosis module is further configured to store the lesion feature information in the case database to update case data in the case database.
According to another aspect of the present invention, there is provided a method for generating auxiliary diagnostic information based on a Mask R-CNN network, which is applied to an auxiliary diagnostic system based on the Mask R-CNN network, and includes:
s1: acquiring a CT image of a patient and corresponding medical auxiliary information, wherein the CT image carries cerebral hemorrhage information of the patient;
s2: performing enhancement and graying on the CT image to obtain a gray image;
s3: detecting the gray level image by using a trained Mask R-CNN network model to identify and extract focus characteristic information;
s4: matching the focus characteristic information with a structured medical record in a case database, and synthesizing the medical auxiliary information to generate auxiliary diagnosis information.
In one embodiment, the S2 includes:
carrying out nonlinear smooth median filtering on the CT image, and then utilizing a cumulative distribution function:
Figure BDA0003026870440000041
Figure BDA0003026870440000042
carrying out histogram equalization on the filtered CT image, wherein n is the total number of pixels of the CT image, and L is the cumulative frequency;
and (3) establishing a weighted average of the brightness and R, G, B three color components by utilizing the RGB and YUV color space variation relation in the equalized CT image: the grayscale image was obtained when Color is 0.35R +0.59G + 0.11B.
In one embodiment, the S3 includes:
inputting the gray level image into a pre-trained Mask R-CNN network model to obtain a corresponding characteristic diagram;
setting a predetermined number of ROI (regions of interest) for each point in the feature map to obtain a plurality of candidate ROI (regions of interest), and then sending the candidate ROI into an RPN (resilient packet network) to perform binary classification and BB regression to filter out partial candidate ROI;
roilign manipulation, classification, BB regression, and MASK were performed on the remaining ROIs to identify and extract the lesion feature information.
In one embodiment, the S4 includes:
indexing the focus characteristic information and a structured medical record in a case database to obtain a diagnosis prompt, wherein the focus characteristic information comprises a cerebral hemorrhage position and a cerebral hemorrhage volume;
synthesizing the diagnosis assistance information with the diagnosis prompt and the medical assistance information.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
1. the invention adopts the technologies of convolutional neural network, data matching and the like to process, detect and analyze the CT image of the patient, and utilizes the trained Mask R-CNN network to enable a doctor to generate auxiliary diagnosis information by utilizing the image characteristics and the structured case database (comprising diagnosis results of focuses at different parts of the brain and diagnosis cases of different doctors) within a short time only by scanning the CT image of the brain and inputting medical auxiliary information in the process of utilizing the CT image to diagnose the cerebral hemorrhage, thereby assisting the emergency diagnosis and screening of the cerebral hemorrhage. Meanwhile, the method can break through the capability limit of the existing medical personnel, is more accurate and efficient, gives consideration to instantaneity, accuracy and sensitivity, greatly improves the utilization rate and the circulation of the existing case data, and has the advantages of high accuracy, intelligent analysis and the like compared with the existing hospital imaging technology.
2. The invention uses the advantages of artificial intelligence to continuously update and optimize the Mask R-CNN model and fill the structured case database, thereby continuously improving the accuracy and the accuracy of diagnosis and treatment. The invention can be used as auxiliary equipment to provide experience reference for doctors, and improves the diagnosis and treatment accuracy.
Drawings
Fig. 1 is a schematic structural diagram of an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating the operation of an image processor in an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a target detection module in an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention;
fig. 4 is an overall architecture diagram of a Mask R-CNN network model according to an embodiment of the present invention;
FIG. 5 is a detailed architecture diagram of a Mask R-CNN network model according to an embodiment of the present invention;
fig. 6a is a login interface diagram of an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention;
fig. 6b is a working interface diagram of an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention;
fig. 7 is a flowchart of an auxiliary information generation method based on a Mask R-CNN network according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention provides an auxiliary diagnosis system based on a Mask R-CNN network, as shown in figure 1, the auxiliary diagnosis system based on the Mask R-CNN network comprises: a data uploading end 101, an image processor 102, a target detection module 103 and a diagnostic analysis module 104. The data uploading terminal 101 is used for uploading a CT image of a patient and corresponding medical auxiliary information, wherein the CT image carries cerebral hemorrhage information of the patient; the image processor 102 is in communication connection with the data uploading terminal 102 and is used for performing enhancement and graying on the CT image to obtain a grayscale image; the target detection module 103 is connected with the image processor 102 and is used for detecting a gray image by using the trained Mask R-CNN network model so as to identify and extract focus characteristic information; and the diagnosis analysis module 104 is connected with the target detection module 103 and is used for matching the focus characteristic information with the structured medical records in the case database and synthesizing medical auxiliary information to generate auxiliary diagnosis information.
Specifically, the auxiliary diagnosis system based on the Mask R-CNN network installs packaged auxiliary diagnosis software in a medical working computer in a consulting room; the auxiliary diagnosis software is used as a data uploading end 101 for importing a CT image of a patient and corresponding medical auxiliary information, and then the data uploading end 101 transmits the CT image to an image processor 102; the image processor 102 performs enhancement and graying on the collected CT image to obtain a grayscale image with stronger contrast; the target detection module 103 completes identification and extraction of focus characteristic information by using the established Mask R-CNN network, wherein the disease characteristic information comprises image information and character information; the diagnostic analysis module 104 matches the lesion feature information with a conventional structured medical record database, and then synthesizes medical auxiliary information to generate auxiliary diagnostic information. The auxiliary diagnostic information may be a medical diagnostic report. In one embodiment, the diagnosis analysis module can be further configured to store the data of the diagnosis and treatment in the structured medical record database, and update the content of the database. The structured medical record database in the diagnosis and analysis module consists of structured medical record character strings containing the information; the diagnosis analysis module at least comprises a characteristic index table, a characteristic library and a structured medical record database;
according to the Mask R-CNN network-based auxiliary diagnosis system provided by the invention, medical auxiliary information comprises patient personal data, disease information, combined disease information and other auxiliary data, and simultaneously comprises all diagnosis records of a patient after filing; can be directly called by using a mouse when needed.
In one embodiment, as shown in FIG. 2, the image processor is configured to apply a non-linear smooth median filter to the CT image, and to reuse a cumulative distribution function：
Figure BDA0003026870440000071
Carrying out histogram equalization on the filtered CT image, wherein n is the total number of CT image pixels, and L is the cumulative frequency; and (3) establishing a weighted average of the brightness and R, G, B three color components by utilizing the RGB and YUV color space variation relation in the equalized CT image: a Color of 0.35R +0.59G +0.11B yields a grayscale image.
In one embodiment, the system further comprises: the model construction module is used for creating a Mask R-CNN network model comprising an RPN network and an ROIPool layer by using the Mask R-CNN network, wherein the PN network is used for finishing fast R-CNN for extracting image candidate regions, the ROIPool layer is used for detecting targets of the candidate regions, and the RPN and the ROIPool layer are convolutional neural networks with different dimensions; defining the Loss function target of the ROIPool layer as: l ═ LCLS+LBOX+LMASKTraining a Mask R-CNN network model; wherein L isCLSAnd LBOXClass loss (cls loss) and regression loss (bbox regression loss), L, respectively, for Fast RCNNMAXAre lost for mask branches (masks).
Specifically, the Mask R-CNN network model manually adds labels and classifies to the processed brain CT gray level image, the bright part of the skull is labeled Normal _ external and Normal _ internal according to the shape, and the bright part of the intracranial hemorrhage is labeled Absormal according to the brightness; labeling completion is randomly selected 8: the 2-proportion data volume is used as a Mask R-CNN algorithm training and verification data set, and the RPN of the Faster R-CNN and the ROIPool layer for detecting the candidate region target are adopted during training to accelerate the detection speed of the convolutional neural network very quickly, so that the accuracy of the model is greatly improved.
In one embodiment, as shown in fig. 3, the target detection module is configured to input a gray image into a pre-trained Mask R-CNN network model to obtain a corresponding feature map; setting a predetermined number of ROI (regions of interest) at each point in the feature map to obtain a plurality of candidate ROI (regions of interest), and then sending the candidate ROI into an RPN (resilient packet network) to perform binary classification and BB regression to filter out partial candidate ROI; the remaining ROIs were subjected to ROIAlign manipulation, classification, BB regression, and MASK to identify and extract lesion feature information.
The invention finishes the reading-in of the CT image of the patient and the corresponding preprocessing; and inputting the data into a pre-trained deep convolutional network. The system extracts a candidate region by using an RPN network of fast R-CNN, and classifies and box regression on the extracted features of the candidate region by using ROIPool to complete target detection; and modifying by using a Mask R-CNN code packaged in the open source code, programming to complete corresponding Python code programming, assigning values and calling each control button to realize the function of the algorithm. The data selection amount of randomly extracting part of sample data during the training of the Mask R-CNN network model is more than 70 percent of the total amount of the corresponding extracted data; in the prior indexing process of the patient structured case database, a Boyer-Moore algorithm is adopted, and the matching is performed from the rightmost end of a character string to the left by utilizing a 'bad character + good suffix' rule in O (n) time complexity.
Fig. 4 is an overall architecture diagram of a Mask R-CNN network model according to an embodiment of the present invention; fig. 5 is a detailed architecture diagram of a Mask R-CNN network model according to an embodiment of the present invention. The system adopts Boyer-Moore algorithm to index focus information (position, volume and the like) in the CT image of the patient output by the model with a structured case database of the prior patient to obtain a diagnosis conclusion, and when the CT image and the diagnosis information for assisting diagnosis are synthesized, the right end of the character string is matched to the left by using a 'bad character + good suffix' rule within O (n) time complexity. As an improvement on the teaching mode of ' teacher with ' brother ' in clinic at present, the method structures the experience knowledge of a plurality of doctors and combines the experience knowledge with the artificial intelligence technology to form a comprehensive diagnosis and treatment system.
Fig. 6a is a login interface diagram of an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention, and fig. 6b is a working interface diagram of an auxiliary diagnostic system based on a Mask R-CNN network according to an embodiment of the present invention. The system working interface displays the diagnosis result and the suggested treatment scheme of the patient, and simultaneously, the interface also comprises a patient health evaluation board, an illness state detection board, latest information related to cerebral hemorrhage research at home and abroad and all treatment records of the patient after filing, and can be directly called by a mouse when needed. According to the interactive region selection of the system page, after the information such as the position, the volume and the like of the focus of different focuses is amplified and displayed by mouse operation or keyboard input, a preliminary diagnosis conclusion is generated and output by combining the information of the traditional structured medical record and expert experience. The multifunctional plate is helpful for improving the understanding of the doctor on the past medical history of the patient, thereby improving the diagnosis and treatment accuracy and efficiency. The system interface should include a plate for evaluating the health of the patient according to the patient's visit record; the system interface should include a board for detecting the patient's condition according to the patient's visit record; the system interface should also contain the latest information related to the study of cerebral hemorrhage at home and abroad.
In one embodiment, the diagnosis module is further configured to index lesion feature information with a structured medical record in a case database to obtain a diagnosis prompt, where the lesion feature information includes a location of cerebral hemorrhage and a volume of cerebral hemorrhage; and synthesizing the diagnosis prompt and the medical auxiliary information into diagnosis auxiliary information.
In one embodiment, the diagnosis module is further configured to store the lesion feature information in a case database to update case data in the case database.
In the construction process of a Mask R-CNN network model in the system, particularly during image annotation, manual intervention can be performed in real time, specifically, experts are invited to participate in the joint completion of the work, and the accuracy and the guidance value of the model are improved.
In practice, the medical examination of a patient suspected of having cerebral hemorrhage is often time-consuming and highly demanding for the physician. When the doctor is in a poor state or has insufficient experience, the system can transmit the data of the medical image to an auxiliary diagnosis system based on a Mask R-CNN network, a diagnosis analysis module calls a Mask R-CNN model and a conventional structured case database to study and judge the clinically acquired medical image information, a normalized medical diagnosis and treatment report is output through a medical end interface, and meanwhile, the data of the diagnosis and treatment is stored in the database system to update the structured case database. The invention can continuously update the learning in clinical use and greatly surpass the artificial learning ability.
In practical application, the invention can carry out linkage diagnosis on a plurality of diseases of the brain simultaneously to form an all-round comprehensive diagnosis and treatment result, breaks through the limitation that each department of the existing medicine carries out independent diagnosis and treatment and has a limited range, reduces the probability of repeated inquiry and examination, and has more accurate and comprehensive diagnosis.
The invention provides a method for generating auxiliary diagnosis information based on a Mask R-CNN network, which is applied to an auxiliary diagnosis system based on the Mask R-CNN network, and as shown in figure 7, the method comprises the following steps:
s1: acquiring a CT image of a patient and corresponding medical auxiliary information, wherein the CT image carries cerebral hemorrhage information of the patient;
s2: performing enhancement and graying on the CT image to obtain a gray image;
s3: detecting a gray image by using the trained Mask R-CNN network model to identify and extract focus characteristic information;
s4: matching the focus characteristic information with the structured medical records in the case database, and synthesizing medical auxiliary information to generate auxiliary diagnosis information.
In one embodiment, S2 includes: carrying out nonlinear smooth median filtering on the CT image, and then utilizing a cumulative distribution function:
Figure BDA0003026870440000101
carrying out histogram equalization on the filtered CT image, wherein n is the total number of CT image pixels, and L is the cumulative frequency; and (3) establishing a weighted average of the brightness and R, G, B three color components by utilizing the RGB and YUV color space variation relation in the equalized CT image: a Color of 0.35R +0.59G +0.11B yields a grayscale image.
In one embodiment, S3 includes: inputting the gray level image into a pre-trained Mask R-CNN network model to obtain a corresponding characteristic diagram; setting a predetermined number of ROI (regions of interest) at each point in the feature map to obtain a plurality of candidate ROI (regions of interest), and then sending the candidate ROI into an RPN (resilient packet network) to perform binary classification and BB regression to filter out partial candidate ROI; roilign manipulation, classification, BB regression, and MASK were performed on the remaining ROIs to identify and extract lesion feature information.
In one embodiment, S4 includes: indexing the focus characteristic information and a structured medical record in a case database to obtain a diagnosis prompt, wherein the focus characteristic information comprises a cerebral hemorrhage position and a cerebral hemorrhage volume; and synthesizing the diagnosis prompt and the medical auxiliary information into diagnosis auxiliary information.
It will be understood by those skilled in the art that the foregoing is merely a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included within the scope of the present invention.

Claims (10)

1. An auxiliary diagnosis system based on Mask R-CNN network is characterized by comprising:
the data uploading end is used for uploading a CT image of a patient and corresponding medical auxiliary information, and the CT image carries the cerebral hemorrhage information of the patient;
the image processor is connected with the data uploading end and used for performing enhancement and graying on the CT image to obtain a gray image;
the target detection module is connected with the image processor and used for detecting the gray level image by using the trained Mask R-CNN network model so as to identify and extract focus characteristic information;
and the diagnosis analysis module is connected with the target detection module and used for matching the focus characteristic information with the structured medical record in the case database and synthesizing the medical auxiliary information to generate auxiliary diagnosis information.
2. The Mask R-CNN network-based auxiliary diagnostic system as claimed in claim 1, wherein the image processor is used for
Carrying out nonlinear smooth median filtering on the CT image, and then utilizing a cumulative distribution function:
Figure FDA0003026870430000011
Figure FDA0003026870430000012
carrying out histogram equalization on the filtered CT image, wherein n is the total number of pixels of the CT image, and L is the cumulative frequency;
and (3) establishing a weighted average of the brightness and R, G, B three color components by utilizing the RGB and YUV color space variation relation in the equalized CT image: the grayscale image was obtained when Color is 0.35R +0.59G + 0.11B.
3. The Mask R-CNN network-based aided diagnosis system of claim 1, further comprising: a model construction module for
Creating a Mask R-CNN network model comprising an RPN network and an ROIPool layer by using the Mask R-CNN network, wherein the PN network is used for finishing fast R-CNN extraction of an image candidate region, the ROIPool layer is used for detecting a candidate region target, and the RPN and the ROIPool layer are convolutional neural networks with different dimensions;
defining the Loss function target of the ROIPool layer as follows: l ═ LCLS+LBOX+LMASKTraining the Mask R-CNN network model;
wherein L isCLSAnd LBOXClassification and regression losses, L, respectively, for Fast RCNNMAXIs lost as a mask branch.
4. The Mask R-CNN network-based auxiliary diagnostic system according to claim 3, wherein the target detection module is used for,
inputting the gray level image into a pre-trained Mask R-CNN network model to obtain a corresponding characteristic diagram;
setting a predetermined number of ROI (regions of interest) for each point in the feature map to obtain a plurality of candidate ROI (regions of interest), and then sending the candidate ROI into an RPN (resilient packet network) to perform binary classification and BB regression to filter out partial candidate ROI;
roilign manipulation, classification, BB regression, and MASK were performed on the remaining ROIs to identify and extract the lesion feature information.
5. The Mask R-CNN network-based auxiliary diagnostic system according to claim 1, wherein the diagnostic module is further configured to,
indexing the focus characteristic information and a structured medical record in a case database to obtain a diagnosis prompt, wherein the focus characteristic information comprises a cerebral hemorrhage position and a cerebral hemorrhage volume;
synthesizing the diagnosis assistance information with the diagnosis prompt and the medical assistance information.
6. The Mask R-CNN network-based aided diagnosis system according to claim 5, wherein the diagnosis module is further configured to store the lesion feature information in the case database to update case data in the case database.
7. A method for generating auxiliary diagnosis information based on a Mask R-CNN network is characterized in that the method is applied to an auxiliary diagnosis system based on the Mask R-CNN network, and comprises the following steps:
s1: acquiring a CT image of a patient and corresponding medical auxiliary information, wherein the CT image carries cerebral hemorrhage information of the patient;
s2: performing enhancement and graying on the CT image to obtain a gray image;
s3: detecting the gray level image by using a trained Mask R-CNN network model to identify and extract focus characteristic information;
s4: matching the focus characteristic information with a structured medical record in a case database, and synthesizing the medical auxiliary information to generate auxiliary diagnosis information.
8. The Mask R-CNN network-based auxiliary diagnostic information generating method according to claim 7, wherein the S2 includes:
carrying out nonlinear smooth median filtering on the CT image, and then utilizing a cumulative distribution function:
Figure FDA0003026870430000031
Figure FDA0003026870430000032
carrying out histogram equalization on the filtered CT image, wherein n is the total number of pixels of the CT image, and L is the cumulative frequency;
and (3) establishing a weighted average of the brightness and R, G, B three color components by utilizing the RGB and YUV color space variation relation in the equalized CT image: the grayscale image was obtained when Color is 0.35R +0.59G + 0.11B.
9. The Mask R-CNN network-based auxiliary diagnostic information generating method according to claim 7, wherein the S3 includes:
inputting the gray level image into a pre-trained Mask R-CNN network model to obtain a corresponding characteristic diagram;
setting a predetermined number of ROI (regions of interest) for each point in the feature map to obtain a plurality of candidate ROI (regions of interest), and then sending the candidate ROI into an RPN (resilient packet network) to perform binary classification and BB regression to filter out partial candidate ROI;
roilign manipulation, classification, BB regression, and MASK were performed on the remaining ROIs to identify and extract the lesion feature information.
10. The Mask R-CNN network-based auxiliary diagnostic information generating method according to claim 7, wherein the S4 includes:
indexing the focus characteristic information and a structured medical record in a case database to obtain a diagnosis prompt, wherein the focus characteristic information comprises a cerebral hemorrhage position and a cerebral hemorrhage volume;
synthesizing the diagnosis assistance information with the diagnosis prompt and the medical assistance information.
CN202110418327.0A 2021-04-19 2021-04-19 Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method Pending CN113205490A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110418327.0A CN113205490A (en) 2021-04-19 2021-04-19 Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110418327.0A CN113205490A (en) 2021-04-19 2021-04-19 Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method

Publications (1)

Publication Number Publication Date
CN113205490A true CN113205490A (en) 2021-08-03

Family

ID=77027408

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110418327.0A Pending CN113205490A (en) 2021-04-19 2021-04-19 Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method

Country Status (1)

Country Link
CN (1) CN113205490A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114067993A (en) * 2021-11-08 2022-02-18 杭州维纳安可医疗科技有限责任公司 Data processing method applied to remote auxiliary diagnosis and treatment system and related equipment
CN114359241A (en) * 2022-01-10 2022-04-15 北京富通东方科技有限公司 Brain injury detection auxiliary film reading method and system based on CT (computed tomography) image
CN115080782A (en) * 2022-06-14 2022-09-20 中山大学 Method and system for searching and matching similar focus
CN115187600A (en) * 2022-09-13 2022-10-14 杭州涿溪脑与智能研究所 Brain hemorrhage volume calculation method based on neural network
CN115375684A (en) * 2022-10-24 2022-11-22 南京邮电大学 Pulmonary tuberculosis image auxiliary diagnosis system based on machine learning
CN115831329A (en) * 2022-12-21 2023-03-21 青海大学附属医院 Infusorian classification model construction method, system and medium fusing doctor attention image
CN116798596A (en) * 2022-03-14 2023-09-22 数坤(北京)网络科技股份有限公司 Information association method, device, electronic equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110032617A (en) * 2019-03-05 2019-07-19 中电科软件信息服务有限公司 Rule-based stroke data pick-up management system and method
CN110755045A (en) * 2019-10-30 2020-02-07 湖南财政经济学院 Skin disease comprehensive data analysis and diagnosis auxiliary system and information processing method
CN112634239A (en) * 2020-12-25 2021-04-09 南京邮电大学 Cerebral hemorrhage detecting system based on deep learning

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110032617A (en) * 2019-03-05 2019-07-19 中电科软件信息服务有限公司 Rule-based stroke data pick-up management system and method
CN110755045A (en) * 2019-10-30 2020-02-07 湖南财政经济学院 Skin disease comprehensive data analysis and diagnosis auxiliary system and information processing method
CN112634239A (en) * 2020-12-25 2021-04-09 南京邮电大学 Cerebral hemorrhage detecting system based on deep learning

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114067993A (en) * 2021-11-08 2022-02-18 杭州维纳安可医疗科技有限责任公司 Data processing method applied to remote auxiliary diagnosis and treatment system and related equipment
CN114359241A (en) * 2022-01-10 2022-04-15 北京富通东方科技有限公司 Brain injury detection auxiliary film reading method and system based on CT (computed tomography) image
CN116798596A (en) * 2022-03-14 2023-09-22 数坤(北京)网络科技股份有限公司 Information association method, device, electronic equipment and storage medium
CN115080782A (en) * 2022-06-14 2022-09-20 中山大学 Method and system for searching and matching similar focus
CN115187600A (en) * 2022-09-13 2022-10-14 杭州涿溪脑与智能研究所 Brain hemorrhage volume calculation method based on neural network
CN115375684A (en) * 2022-10-24 2022-11-22 南京邮电大学 Pulmonary tuberculosis image auxiliary diagnosis system based on machine learning
CN115831329A (en) * 2022-12-21 2023-03-21 青海大学附属医院 Infusorian classification model construction method, system and medium fusing doctor attention image
CN115831329B (en) * 2022-12-21 2023-08-18 青海大学附属医院 Method, system and medium for constructing bag worm classification model fusing doctor focused image

Similar Documents

Publication Publication Date Title
CN113205490A (en) Mask R-CNN network-based auxiliary diagnosis system and auxiliary diagnosis information generation method
CN110060774B (en) Thyroid nodule identification method based on generative confrontation network
Sharif et al. A comprehensive review on multi-organs tumor detection based on machine learning
Chan et al. Texture-map-based branch-collaborative network for oral cancer detection
CN110689025B (en) Image recognition method, device and system and endoscope image recognition method and device
CN111292839B (en) Image processing method, image processing device, computer equipment and storage medium
Luo et al. Micro-vessel image segmentation based on the AD-UNet model
CN113420826B (en) Liver focus image processing system and image processing method
CN111899244A (en) Image segmentation method, network model training method, device and electronic equipment
CN111242921A (en) Method and system for automatically updating medical ultrasonic image auxiliary diagnosis system
CN114398979A (en) Ultrasonic image thyroid nodule classification method based on feature decoupling
Zang et al. Interpretable diabetic retinopathy diagnosis based on biomarker activation map
CN112990339B (en) Gastric pathological section image classification method, device and storage medium
CN111275720B (en) Full end-to-end small organ image identification method based on deep learning
CN116664592A (en) Image-based arteriovenous blood vessel separation method and device, electronic equipment and medium
CN116958537A (en) Lung nodule segmentation method based on U-Net model
Huang et al. Noise ECG generation method based on generative adversarial network
CN112949712B (en) Multi-mode-based thyroid nodule automatic identification model construction method
CN115910300A (en) Medical virtual platform system based on artificial intelligence and information processing method
CN114757894A (en) Bone tumor focus analysis system
CN113796850A (en) Parathyroid MIBI image analysis system, computer device, and storage medium
CN114612373A (en) Image identification method and server
Urina-Triana et al. Machine Learning and AI Approaches for Analyzing Diabetic and Hypertensive Retinopathy in Ocular Images: A Literature Review
Kher et al. DeB5-XNet: An Explainable Ensemble Model for Ocular Disease Classification using Transfer Learning and Grad-CAM
Doan et al. Implementation of complete glaucoma diagnostic system using machine learning and retinal fundus image processing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210803

RJ01 Rejection of invention patent application after publication