CN112288683A - Pulmonary tuberculosis judgment device and method based on multi-mode fusion - Google Patents

Pulmonary tuberculosis judgment device and method based on multi-mode fusion Download PDF

Info

Publication number
CN112288683A
CN112288683A CN202010611147.XA CN202010611147A CN112288683A CN 112288683 A CN112288683 A CN 112288683A CN 202010611147 A CN202010611147 A CN 202010611147A CN 112288683 A CN112288683 A CN 112288683A
Authority
CN
China
Prior art keywords
image
pathological
characteristic data
fusion
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010611147.XA
Other languages
Chinese (zh)
Inventor
李峰
权申文
范紫薇
刘旋
柯晓文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Zhiying Medical Technology Co ltd
Original Assignee
Shenzhen Zhiying Medical Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Zhiying Medical Technology Co ltd filed Critical Shenzhen Zhiying Medical Technology Co ltd
Priority to CN202010611147.XA priority Critical patent/CN112288683A/en
Publication of CN112288683A publication Critical patent/CN112288683A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Biology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Medical Informatics (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Quality & Reliability (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The application provides a pulmonary tuberculosis judgment device and method based on multi-mode fusion, and the system comprises: the first data acquisition module is used for carrying out feature extraction processing on an image so as to obtain corresponding image feature data from the image; the second data acquisition module is used for carrying out feature extraction processing on the pathological image so as to obtain corresponding pathological feature data from the pathological image; the fusion calculation module is used for performing fusion calculation on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis; and the result judging module is used for comparing the probability value with a preset probability threshold value to judge whether the tuberculosis exists, and if the probability value reaches the preset probability threshold value, the tuberculosis is judged to exist. The device realizes intelligent judgment of the pulmonary tuberculosis through fusion of the image characteristic data and the pathological characteristic data, has high accuracy, can reduce missed diagnosis and is high in speed.

Description

Pulmonary tuberculosis judgment device and method based on multi-mode fusion
Technical Field
The application belongs to the technical field of medical treatment, and particularly relates to a pulmonary tuberculosis judgment device and method based on multi-modal fusion, and also relates to electronic equipment and a storage medium for realizing corresponding functions of the pulmonary tuberculosis judgment device based on multi-modal fusion.
Background
With the rapid development and wide application of computer artificial intelligence technology, computer-aided diagnosis technology plays an increasingly important role in human health. For example, tuberculosis, a chronic infectious disease caused by mycobacterium tuberculosis, can invade many organs, and is the most common pulmonary tuberculosis infection. Currently, computer-aided diagnosis systems for tuberculosis generally perform intelligent identification and diagnosis on a patient's imaging image to determine whether the patient has tuberculosis. However, the computer-aided diagnosis system only performs intelligent diagnosis by a single imaging image processing technology, so that the reliability of the diagnosis result is low, diagnosis omission is easy, and a doctor is required to perform further manual diagnosis according to pathological data of a patient to confirm whether the patient has tuberculosis or not, so that the diagnosis speed is low, and the problem of intelligent identification and diagnosis of current tuberculosis with infection, morbidity, drug resistance and high fatality rate is difficult to solve.
Disclosure of Invention
In view of this, the embodiment of the present application provides a device and a method for determining pulmonary tuberculosis based on multimodal fusion, and an electronic device and a storage medium for implementing corresponding functions of the device for determining pulmonary tuberculosis based on multimodal fusion, which can implement intelligent determination of pulmonary tuberculosis by fusing image characteristic data and pathological characteristic data, and have the advantages of high accuracy, reduced missed diagnosis and high speed.
A first aspect of an embodiment of the present application provides a pulmonary tuberculosis determination apparatus based on multimodal fusion, where the apparatus includes:
the first data acquisition module is used for carrying out feature extraction processing on an image so as to obtain corresponding image feature data from the image;
the second data acquisition module is used for carrying out feature extraction processing on the pathological image so as to obtain corresponding pathological feature data from the pathological image;
the fusion calculation module is used for performing fusion calculation on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis;
and the result judging module is used for comparing the probability value with a preset probability threshold value to judge whether the tuberculosis exists, and if the probability value reaches the preset probability threshold value, the tuberculosis is judged to exist.
With reference to the first aspect, in a first possible implementation manner of the first aspect, the fusion calculation module further includes:
and the first fusion submodule is used for carrying out maximum value fusion and/or average value fusion on the obtained image characteristic data and pathological characteristic data.
With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the fusion calculation module further includes:
the first classification submodule is used for classifying the obtained image characteristic data and the pathological characteristic data through a decision tree classifier to obtain positive first image characteristic data and positive first pathological characteristic data;
the first configuration submodule is used for carrying out weight configuration on the first image characteristic data and the first pathological characteristic data so as to enable the first image characteristic data and the first pathological characteristic data to respectively have corresponding weight values;
and the first calculation submodule is used for combining the weighted value to perform fusion calculation on the first image characteristic data and the first pathological characteristic data so as to obtain a probability value for measuring the possibility of the pulmonary tuberculosis.
With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the first configuration sub-module further includes:
and the second configuration submodule is used for configuring the weighted value of the first image characteristic data to be smaller than the weighted value of the first pathological characteristic data.
With reference to the second possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the first classification sub-module further includes:
and the first dimension reduction submodule is used for carrying out dimension reduction processing on the obtained image characteristic data and the obtained pathological characteristic data.
With reference to the first aspect, in a fifth possible implementation manner of the first aspect, the first data obtaining module further includes:
the first receiving submodule is used for receiving an image of a patient to be diagnosed;
the first segmentation submodule is used for performing semantic segmentation processing on the image of the patient to be diagnosed;
and the first extraction submodule is used for inputting the image subjected to semantic segmentation processing into a first feature extraction model trained in advance for feature extraction so as to obtain corresponding image feature data from the image.
With reference to the first aspect, in a sixth possible implementation manner of the first aspect, the second data obtaining module further includes:
the second receiving submodule is used for receiving a pathological image of a patient to be diagnosed;
the second segmentation submodule is used for performing semantic segmentation processing on the pathological image of the patient to be diagnosed;
and the second extraction submodule is used for inputting the pathological image subjected to semantic segmentation processing into a second feature extraction model trained in advance for feature extraction so as to obtain corresponding pathological feature data from the pathological image.
A second aspect of the embodiments of the present application provides a method for determining pulmonary tuberculosis based on multimodal fusion, 8. the method for determining pulmonary tuberculosis based on multimodal fusion is implemented based on the device for determining pulmonary tuberculosis based on multimodal fusion as described in any one of the first aspect, and includes:
carrying out feature extraction processing on the image so as to obtain corresponding image feature data from the image;
carrying out feature extraction processing on a pathological image to obtain corresponding pathological feature data from the pathological image;
performing fusion calculation on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis;
and comparing the probability value with a preset probability threshold value to judge whether the patient has the pulmonary tuberculosis, and if the probability value reaches the preset probability threshold value, judging that the patient has the pulmonary tuberculosis.
A third aspect of embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, implements the steps of the apparatus for determining pulmonary tuberculosis based on multimodal fusion according to any one of the first aspect.
A fourth aspect of embodiments of the present application provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the steps of the apparatus for determining pulmonary tuberculosis based on multimodal fusion according to any one of the first aspect.
Compared with the prior art, the embodiment of the application has the advantages that:
the device for judging the pulmonary tuberculosis based on the multi-modal fusion obtains corresponding image characteristic data from an image by performing characteristic extraction processing on the image; carrying out feature extraction processing on a pathological image to obtain corresponding pathological feature data from the pathological image; performing decision fusion processing on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis; the probability value is compared with a preset probability threshold value to judge whether the pulmonary tuberculosis exists or not, if the probability value reaches the preset probability threshold value, the pulmonary tuberculosis is judged to exist, intelligent judgment of the pulmonary tuberculosis is achieved by fusing image characteristic data and pathological characteristic data, accuracy is high, missed diagnosis can be reduced, and speed is high.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a basic method of a tuberculosis determination apparatus based on multimodal fusion according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a method for performing decision fusion processing on feature data in a tuberculosis determination device based on multimodal fusion according to an embodiment of the present application;
fig. 3 is a schematic flow chart illustrating a method for extracting image feature data in a tuberculosis determination device based on multimodal fusion according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a method for extracting pathological feature data in a tuberculosis determination device based on multimodal fusion according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a tuberculosis determination apparatus based on multimodal fusion according to an embodiment of the present application;
fig. 6 is a schematic diagram of an electronic device for implementing a tuberculosis determination apparatus based on multimodal fusion according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
In order to explain the technical solution described in the present application, the following description will be given by way of specific examples.
The embodiment of the application aims to provide a pulmonary tuberculosis judgment device based on multi-mode fusion, which is applied to a computer-aided diagnosis system to realize intelligent judgment of pulmonary tuberculosis to be diagnosed by fusing image characteristic data and pathological characteristic data, improve the accuracy of system diagnosis and reduce missed diagnosis. It is understood that the computer-aided diagnosis system can be any electronic device that can implement the corresponding functions of the apparatus for determining pulmonary tuberculosis based on multimodal fusion.
In some embodiments of the present application, please refer to fig. 1, and fig. 1 is a schematic diagram of a basic structure of a tuberculosis determination device based on multimodal fusion provided in embodiment 1 of the present application. The details are as follows:
in this embodiment, the apparatus for determining pulmonary tuberculosis based on multimodal fusion according to the present application includes a first data obtaining module 10, a second data obtaining module 20, a fusion calculating module 30, and a result determining module 40. Wherein:
the first data acquisition module 10 is configured to perform feature extraction processing based on an image to obtain corresponding image feature data from the image. In some implementations, the image is a DR (Digital Radiography) image, which is a Digital image directly formed by the X-ray signal transmitted through the human body and acquired by the detector, and the Digital image may be transmitted to a computer for display and post-processing. Wherein, the DR image is stored in DICOM form. The computer-aided diagnosis system for executing the tuberculosis determination device based on multi-modal fusion of the embodiment can acquire the image of the patient to be diagnosed by connecting with the imaging equipment of the DR image. After the first data obtaining module 10 obtains the image, the computer-aided diagnosis system may extract corresponding image feature data from the image based on a pre-trained image feature data extraction model in an image modality. For example, for pulmonary tuberculosis, the image characteristics of the pulmonary tuberculosis often show that the real-variant shadow of the air cavity is involved in the real variant of the lower lobe of the left lung visible on the whole lung lobe chest orthotopic tablet, and a small amount of pleural effusion on the left side is accompanied, so that the image can be identified based on the characteristics, and the image characteristic data corresponding to the image and used for judging the pulmonary tuberculosis is obtained.
The second data acquisition module 20 is configured to perform feature extraction processing based on the pathological image to obtain corresponding pathological feature data from the pathological image. In some implementations, the pathological image is also called pathological electron section image, which refers to an image obtained by taking a lesion tissue with a certain size as a sample and placing the lesion tissue in an electron microscope for scanning and case analysis. After the second data obtaining module 20 obtains the pathological image, the computer-aided diagnosis system may extract corresponding pathological feature data from the pathological image based on the expected and trained pathological feature data extraction model in the pathological mode. For example, the pathological image for diagnosing the pulmonary tuberculosis is a pathological image obtained by sputum examination, and the pathological characteristics of the pathological image are represented by the presence of mycobacterium tuberculosis in sputum, so that the pathological image can be identified based on the characteristics of the mycobacterium tuberculosis, and the pathological characteristic data corresponding to the pathological image and used for judging the pulmonary tuberculosis is obtained.
The fusion calculation module 30 is configured to perform fusion calculation on the obtained image feature data and the pathological feature data to obtain a probability value for measuring the probability of the pulmonary tuberculosis. In some implementations, the fusion calculation module 30 may obtain one or more image feature data from the image, and similarly, may obtain one or more pathological feature data from the pathological image. The computer aided diagnosis system can lead the obtained characteristic data as influence factors into a preset fusion algorithm for calculation so as to obtain a probability value for measuring the probability of the pulmonary tuberculosis. Specifically, taking the diagnosis of pulmonary tuberculosis as an example, a weighting algorithm is adopted, and a correspondence table of characteristic data and weight values is set in the system, wherein the characteristic data in the correspondence table is characteristic data for diagnosing pulmonary tuberculosis. And according to the influence of each feature data on the size of the result of diagnosing the pulmonary tuberculosis, performing weight value configuration on the feature data in advance to generate a corresponding relation table of the feature data and the weight values. Therefore, after the image characteristic data and the pathological characteristic data are obtained, the corresponding weight value of each characteristic data is obtained by traversing the corresponding relation table, and then fusion calculation is carried out on the image characteristic data and the pathological characteristic data according to a preset fusion algorithm based on the weight value to obtain the probability value for measuring the probability of the pulmonary tuberculosis disease.
In this embodiment, the fusion calculation mode adopted by the computer-aided diagnosis system is a back-end fusion mode, where the back-end fusion is performed by respectively outputting and scoring (decision) data of different modalities, and includes maximum value fusion and/or average value fusion. In this embodiment, the fusion calculation module 30 is further configured with a first fusion sub-module, and the first fusion sub-module is configured to perform maximum value fusion and/or average value fusion on the obtained image feature data and the obtained pathological feature data. The data feature complementation can be realized by carrying out maximum value fusion on the data of different modes, and the relevance between the data features can be reduced by carrying out average value fusion on the data of different modes, so that the recognition rate of the data features can be improved by fusing the data of different modes.
The result determining module 40 is configured to compare the probability value with a preset probability threshold to determine whether the patient has the tuberculosis, and if the probability value reaches the preset probability threshold, the patient is determined to have the tuberculosis. In some implementations, the result determination module 40 sets a probability threshold by setting a criterion for determining whether tuberculosis is present or not based on the characteristic data in the system, and generates the result intelligently determined by the computer-aided diagnosis system according to the probability threshold. In particular, the probability threshold may be obtained by big data analysis. After the probability value for measuring the possibility of the pulmonary tuberculosis is calculated through the fusion algorithm, the probability value can be compared with a probability threshold value, if the probability value calculated by the system reaches a preset probability threshold value, the computer-aided diagnosis system is explained to perform fusion calculation based on the characteristics of two modalities of an image and a pathological image of a patient, so that the patient is judged to have the pulmonary tuberculosis, and the system outputs a result of yes. If the probability value obtained by the system calculation does not reach the preset probability threshold value, the computer-aided diagnosis system is explained to perform fusion calculation based on the characteristics of the image and the pathological image of the patient, so that the patient is judged not to suffer from the pulmonary tuberculosis, and the system outputs a result of no judgment.
The tuberculosis determination device based on multi-modal fusion provided by the embodiment performs feature extraction processing on the image to obtain corresponding image feature data from the image; carrying out feature extraction processing on a pathological image to obtain corresponding pathological feature data from the pathological image; performing decision fusion processing on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis; the probability value is compared with a preset probability threshold value to judge whether the pulmonary tuberculosis exists or not, if the probability value reaches the preset probability threshold value, the pulmonary tuberculosis is judged to exist, intelligent judgment of the pulmonary tuberculosis is achieved by fusing image characteristic data and pathological characteristic data, accuracy is high, missed diagnosis can be reduced, and speed is high.
In some embodiments of the present application, please refer to fig. 2, and fig. 2 is a schematic structural diagram of a tuberculosis determination apparatus based on multimodal fusion provided in embodiment 2 of the present application. The details are as follows:
in this embodiment, the apparatus for determining pulmonary tuberculosis based on multimodal fusion further includes a first classification submodule 31, a first configuration submodule 32, and a first calculation submodule 33. Wherein: the first classification submodule 31 is configured to perform classification processing on the obtained image feature data and the obtained pathological feature data through a decision tree classifier, and obtain positive first image feature data and positive first pathological feature data. The first configuration submodule 32 is configured to perform weight configuration on the first image feature data and the first pathological feature data, so that the first image feature data and the first pathological feature data respectively have corresponding weight values. The first calculating submodule 33 is configured to perform fusion calculation on the first image feature data and the first pathological feature data in combination with the weight value to obtain a probability value for measuring a probability of the pulmonary tuberculosis.
In this embodiment, the decision tree classifier may classify the data visually by making a series of decisions based on the attribute set according to the attributes of the data, each decision being represented by a node of the decision tree. In this embodiment, the obtained image feature data and the obtained pathological feature data may be classified by the decision tree classifier based on the first classification submodule 31, so as to obtain positive first image feature data and positive first pathological feature data. In some implementations, a first dimension reduction sub-module is configured in the first classification sub-module 31, and the first dimension reduction sub-module performs dimension reduction processing on the obtained image feature data and pathological feature data based on PCA (principal component analysis of data), and retains main feature data information by discarding the dimension with less information content, so as to obtain positive first image feature data and positive first pathological feature data. Therefore, the feature data under the image modality and the pathological modality are fused for the first time based on the decision classifier, and the feature data which are positive respectively under the image modality and the pathological modality, namely the first image feature data and the first pathological feature data, are determined. After the first image characteristic data and the first pathological characteristic data are obtained, weight configuration and fusion calculation are performed on the first image characteristic data and the first pathological characteristic data based on the first configuration submodule 32 and the first calculation submodule 33, so that second fusion is performed on the characteristic data which are respectively positive in the image modality and the pathological modality, and finally a probability value for measuring the possibility of the pulmonary tuberculosis is obtained. In this embodiment, since the influence of the pathological features on the pulmonary tuberculosis determination is greater than the influence of the image features on the pulmonary tuberculosis determination, when the first image feature data and the first pathological feature data are fused for the second time, based on the first configuration submodule 32, a second configuration submodule may be further configured, and the second configuration submodule configures that the weight value of the first image feature data is smaller than the weight value of the first pathological feature data, so that the system has higher accuracy when performing the pulmonary tuberculosis determination.
In some embodiments of the present application, please refer to fig. 3, and fig. 3 is a schematic structural diagram of a tuberculosis determination apparatus based on multimodal fusion provided in embodiment 3 of the present application. The details are as follows:
in this embodiment, the apparatus for determining pulmonary tuberculosis based on multimodal fusion further includes a first receiving submodule 11, a first segmenting submodule 12, and a first extracting submodule 13. Wherein: the first receiving submodule 11 is configured to receive an image of a patient to be diagnosed. The first segmentation submodule 12 is configured to perform semantic segmentation processing on the image of the patient to be diagnosed. The first extraction submodule 13 is configured to input the image subjected to the semantic segmentation processing into a first feature extraction model trained in advance for feature extraction, so as to obtain corresponding image feature data from the image.
In this embodiment, a data transmission channel between the computer-aided diagnosis system and the DR digital imaging device is established, and after an image of a patient to be diagnosed is captured and acquired by the DR digital imaging device, the image is transmitted to the computer-aided diagnosis system, so that the tuberculosis determination device based on multi-modal fusion in the computer-aided diagnosis system receives the image of the patient to be diagnosed through the first receiving sub-module 11. Then, after the computer aided diagnosis system receives the image of the patient to be diagnosed, the image can be semantically segmented at a pixel level based on an FCN (full convolution network). Specifically, the FCN can convert the fully-connected layers in CNN into convolutional layers, accepting video images of arbitrary size. The FCN performs upsampling on the two-dimensional picture (feature map) of the last convolutional layer of the video image by using the anti-convolutional layer to restore it to the same size as the video image, thereby enabling generation of a prediction for each pixel while preserving spatial information in the original input image, and thus, the semantic segmentation processing on the video image is performed by classifying the upsampled feature map pixel by the first segmentation sub-module 12 in the apparatus. The image obtained by semantic segmentation can display the content in the image, such as the position data and the focus data. After obtaining the image obtained after the semantic segmentation processing, the first extraction submodule 13 in the device inputs the image obtained after the semantic segmentation processing into a first feature extraction model trained in advance for feature extraction, and corresponding image feature data is obtained from the image. Therefore, the computer aided diagnosis system obtains the image characteristic data of the image modality.
In this embodiment, the first feature extraction model is a convolutional neural network model trained to a convergence state, and the convolutional neural network model is specifically trained to the convergence state through a large amount of sample data (for example, image images of different patients), so that the first feature extraction model has the capability of extracting image feature data for determining tuberculosis from the image images. For example, pulmonary tuberculosis, the image feature data for determining whether pulmonary tuberculosis exists can be extracted from the focus position of the image.
In some embodiments of the present application, please refer to fig. 4, and fig. 4 is a schematic structural diagram of a tuberculosis determination apparatus based on multimodal fusion provided in embodiment 4 of the present application. The details are as follows:
in this embodiment, the apparatus for determining pulmonary tuberculosis based on multimodal fusion further includes a second receiving submodule 21, a second segmentation submodule 22, and a second extraction submodule 23. Wherein: the second receiving submodule 21 is used for receiving the pathological image of the patient to be diagnosed; the second segmentation submodule 22 is configured to perform semantic segmentation processing on the pathological image of the patient to be diagnosed; the second extraction submodule 23 is configured to input the pathological image after the semantic segmentation processing into a second feature extraction model trained in advance for feature extraction, so as to obtain corresponding pathological feature data from the pathological image.
In this embodiment, a pathological image is obtained by extracting a lesion tissue of a patient to be diagnosed as a sample and scanning the lesion tissue in an electron microscope, and the pathological image is transmitted to a computer-aided diagnosis system, so that the tuberculosis determination device based on multi-modal fusion in the computer-aided diagnosis system receives the pathological image of the patient to be diagnosed through the second receiving sub-module 21. Then, after the computer aided diagnosis system receives the pathological image of the patient to be diagnosed, the second segmentation sub-module 22 performs semantic segmentation on the pathological image according to pixel level based on FCN (full convolution network), so as to obtain the pathological image obtained after the semantic segmentation. The specific semantic segmentation process and principle are basically consistent with the above-mentioned semantic segmentation process and principle for extracting image feature data, and are not described herein again. After obtaining the pathological image obtained through the semantic segmentation, the second extraction submodule 23 inputs the pathological image obtained through the semantic segmentation into a second feature extraction model trained in advance for feature extraction, and corresponding pathological feature data are obtained from the pathological image. The training mode of the second feature extraction model is consistent with that of the first feature extraction model, and details are not repeated here.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
In some embodiments of the present application, please refer to fig. 5, fig. 5 is a schematic flowchart of a basic method of a method for determining pulmonary tuberculosis based on multimodal fusion, which is provided in embodiment 5 of the present application, and is detailed as follows:
in step S51, feature extraction processing is performed on the video image to obtain corresponding video feature data from the video image
In this embodiment, the image is a Digital Radiography (DR) image, which is a Digital image directly formed by transmitting an X-ray signal through a human body and then being acquired by a detector, and the Digital image may be transmitted to a computer for display and post-processing. Wherein, the DR image is stored in DICOM form. The computer-aided diagnosis system for executing the tuberculosis determination device based on multi-modal fusion of the embodiment can acquire the image of the patient to be diagnosed by connecting with the imaging equipment of the DR image. After the image is obtained, the computer-aided diagnosis system can extract corresponding image feature data from the image based on a pre-trained image feature data extraction model in an image mode. For example, for pulmonary tuberculosis, the image characteristics of the pulmonary tuberculosis often show that the real-variant shadow of the air cavity is involved in the real variant of the lower lobe of the left lung visible on the whole lung lobe chest orthotopic tablet, and a small amount of pleural effusion on the left side is accompanied, so that the image can be identified based on the characteristics, and the image characteristic data corresponding to the image and used for judging the pulmonary tuberculosis is obtained.
In step S52, a feature extraction process is performed on the pathological image to obtain corresponding pathological feature data from the pathological image.
The pathological image is also called pathological electronic section image, and is an image for case analysis obtained by taking pathological tissues with a certain size as a sample and placing the sample in an electron microscope for scanning. After obtaining the pathological image, the computer-aided diagnosis system may extract corresponding pathological feature data from the pathological image based on the expected and trained pathological feature data extraction model in the pathological mode. For example, the pathological image for diagnosing the pulmonary tuberculosis is a pathological image obtained by sputum examination, and the pathological characteristics of the pathological image are represented by the presence of mycobacterium tuberculosis in sputum, so that the pathological image can be identified based on the characteristics of the mycobacterium tuberculosis, and the pathological characteristic data corresponding to the pathological image and used for judging the pulmonary tuberculosis is obtained.
In step S53, the obtained image feature data and the pathological feature data are subjected to fusion calculation to obtain a probability value for measuring the probability of the pulmonary tuberculosis.
In this embodiment, one or more image feature data may be obtained from the image, and one or more pathological feature data may be obtained from the pathological image. The computer aided diagnosis system can lead the obtained characteristic data as influence factors into a preset fusion algorithm for calculation so as to obtain a probability value for measuring the probability of the pulmonary tuberculosis. Specifically, taking the diagnosis of pulmonary tuberculosis as an example, a weighting algorithm is adopted, and a correspondence table of characteristic data and weight values is set in the system, wherein the characteristic data in the correspondence table is characteristic data for diagnosing pulmonary tuberculosis. And according to the influence of each feature data on the size of the result of diagnosing the pulmonary tuberculosis, performing weight value configuration on the feature data in advance to generate a corresponding relation table of the feature data and the weight values. Therefore, after the image characteristic data and the pathological characteristic data are obtained, the corresponding weight value of each characteristic data is obtained by traversing the corresponding relation table, and then fusion calculation is carried out on the image characteristic data and the pathological characteristic data according to a preset fusion algorithm based on the weight value to obtain the probability value for measuring the probability of the pulmonary tuberculosis disease.
In step S54, the probability value is compared with a preset probability threshold to determine whether tuberculosis exists, and if the probability value reaches the preset probability threshold, tuberculosis is determined to exist.
In the embodiment, a standard for judging whether tuberculosis exists or not based on the characteristic data is set in the system, namely a probability threshold value is set, and the result of intelligent judgment of the computer-aided diagnosis system is generated according to the probability threshold value. In particular, the probability threshold may be obtained by big data analysis. After the probability value for measuring the possibility of the pulmonary tuberculosis is calculated through the fusion algorithm, the probability value can be compared with a probability threshold value, if the probability value calculated by the system reaches a preset probability threshold value, the computer-aided diagnosis system is explained to perform fusion calculation based on the characteristics of two modalities of an image and a pathological image of a patient, so that the patient is judged to have the pulmonary tuberculosis, and the system outputs a result of yes. If the probability value obtained by the system calculation does not reach the preset probability threshold value, the computer-aided diagnosis system is explained to perform fusion calculation based on the characteristics of the image and the pathological image of the patient, so that the patient is judged not to suffer from the pulmonary tuberculosis, and the system outputs a result of no judgment.
According to the method for judging the pulmonary tuberculosis based on the multi-mode fusion, the image characteristic data and the pathological characteristic data are fused to realize intelligent judgment of the pulmonary tuberculosis, the accuracy is high, missed diagnosis can be reduced, and the speed is high.
It can be understood that the above method for determining pulmonary tuberculosis based on multimodal fusion corresponds to the above apparatus for determining pulmonary tuberculosis based on multimodal fusion one to one, and is not described herein again.
In some embodiments of the present application, please refer to fig. 6, and fig. 6 is a schematic diagram of an electronic device for implementing a tuberculosis determination apparatus based on multimodal fusion according to an embodiment of the present application. As shown in fig. 6, the electronic apparatus 6 of this embodiment includes: a processor 61, a memory 62 and a computer program 63 stored in said memory 62 and executable on said processor 61, such as a tuberculosis determination program based on multimodal fusion. The processor 61 implements the functions of the modules/units in the above-described device embodiments when executing the computer program 63. Alternatively, the processor 61 implements the method steps of the above-mentioned embodiment of the method for determining pulmonary tuberculosis based on multimodal fusion when executing the computer program 62.
Illustratively, the computer program 63 may be partitioned into one or more modules/units that are stored in the memory 62 and executed by the processor 61 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 63 in the electronic device 6. For example, the computer program 63 may be divided into:
the first data acquisition module is used for carrying out feature extraction processing on an image so as to obtain corresponding image feature data from the image;
the second data acquisition module is used for carrying out feature extraction processing on the pathological image so as to obtain corresponding pathological feature data from the pathological image;
the fusion calculation module is used for performing fusion calculation on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis;
and the result judging module is used for comparing the probability value with a preset probability threshold value to judge whether the tuberculosis exists, and if the probability value reaches the preset probability threshold value, the tuberculosis is judged to exist.
The electronic device may include, but is not limited to, a processor 61, a memory 62. Those skilled in the art will appreciate that fig. 6 is merely an example of an electronic device 6, and does not constitute a limitation of the electronic device 6, and may include more or fewer components than shown, or some components in combination, or different components, e.g., the electronic device may also include input-output devices, network access devices, buses, etc.
The Processor 61 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 62 may be an internal storage unit of the electronic device 6, such as a hard disk or a memory of the electronic device 6. The memory 62 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 6. Further, the memory 62 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 62 is used for storing the computer program and other programs and data required by the electronic device. The memory 62 may also be used to temporarily store data that has been output or is to be output.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.
The embodiments of the present application provide a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the above method embodiments when executed.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. . Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media which may not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims (10)

1. A pulmonary tuberculosis determination device based on multimodal fusion is characterized by comprising:
the first data acquisition module is used for carrying out feature extraction processing on an image so as to obtain corresponding image feature data from the image;
the second data acquisition module is used for carrying out feature extraction processing on the pathological image so as to obtain corresponding pathological feature data from the pathological image;
the fusion calculation module is used for performing fusion calculation on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis;
and the result judging module is used for comparing the probability value with a preset probability threshold value to judge whether the tuberculosis exists, and if the probability value reaches the preset probability threshold value, the tuberculosis is judged to exist.
2. The system of claim 1, wherein the fusion calculation module further comprises:
and the first fusion submodule is used for carrying out maximum value fusion and/or average value fusion on the obtained image characteristic data and pathological characteristic data.
3. A pulmonary tuberculosis determination apparatus based on multimodal fusion according to claim 1 or 2, wherein the fusion calculation module further comprises:
the first classification submodule is used for classifying the obtained image characteristic data and the pathological characteristic data through a decision tree classifier to obtain positive first image characteristic data and positive first pathological characteristic data;
the first configuration submodule is used for carrying out weight configuration on the first image characteristic data and the first pathological characteristic data so as to enable the first image characteristic data and the first pathological characteristic data to respectively have corresponding weight values;
and the first calculation submodule is used for combining the weighted value to perform fusion calculation on the first image characteristic data and the first pathological characteristic data so as to obtain a probability value for measuring the possibility of the pulmonary tuberculosis.
4. The apparatus of claim 3, wherein the first configuration submodule further comprises:
and the second configuration submodule is used for configuring the weighted value of the first image characteristic data to be smaller than the weighted value of the first pathological characteristic data.
5. The apparatus for determining pulmonary tuberculosis based on multimodal fusion as claimed in claim 3, wherein the first classification sub-module further comprises:
and the first dimension reduction submodule is used for carrying out dimension reduction processing on the obtained image characteristic data and the obtained pathological characteristic data.
6. The method for determining pulmonary tuberculosis based on multimodal fusion as recited in claim 1, wherein the first data obtaining module further comprises:
the first receiving submodule is used for receiving an image of a patient to be diagnosed;
the first segmentation submodule is used for performing semantic segmentation processing on the image of the patient to be diagnosed;
and the first extraction submodule is used for inputting the image subjected to semantic segmentation processing into a first feature extraction model trained in advance for feature extraction so as to obtain corresponding image feature data from the image.
7. The apparatus for determining pulmonary tuberculosis based on multimodal fusion as claimed in claim 1, wherein the second data obtaining module further comprises:
the second receiving submodule is used for receiving a pathological image of a patient to be diagnosed;
the second segmentation submodule is used for performing semantic segmentation processing on the pathological image of the patient to be diagnosed;
and the second extraction submodule is used for inputting the pathological image subjected to semantic segmentation processing into a second feature extraction model trained in advance for feature extraction so as to obtain corresponding pathological feature data from the pathological image.
8. A method for determining pulmonary tuberculosis based on multimodal fusion, which is implemented based on the apparatus for determining pulmonary tuberculosis based on multimodal fusion according to any one of claims 1-7, and comprises:
carrying out feature extraction processing on the image so as to obtain corresponding image feature data from the image;
carrying out feature extraction processing on a pathological image to obtain corresponding pathological feature data from the pathological image;
performing fusion calculation on the obtained image characteristic data and the pathological characteristic data to obtain a probability value for measuring the probability of the pulmonary tuberculosis;
and comparing the probability value with a preset probability threshold value to judge whether the patient has the pulmonary tuberculosis, and if the probability value reaches the preset probability threshold value, judging that the patient has the pulmonary tuberculosis.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the functions of the apparatus for determining pulmonary tuberculosis based on multimodal fusion as recited in claims 1-7.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the functions of the apparatus for determining pulmonary tuberculosis based on multimodal fusion as recited in claims 1-7.
CN202010611147.XA 2020-06-30 2020-06-30 Pulmonary tuberculosis judgment device and method based on multi-mode fusion Pending CN112288683A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010611147.XA CN112288683A (en) 2020-06-30 2020-06-30 Pulmonary tuberculosis judgment device and method based on multi-mode fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010611147.XA CN112288683A (en) 2020-06-30 2020-06-30 Pulmonary tuberculosis judgment device and method based on multi-mode fusion

Publications (1)

Publication Number Publication Date
CN112288683A true CN112288683A (en) 2021-01-29

Family

ID=74420100

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010611147.XA Pending CN112288683A (en) 2020-06-30 2020-06-30 Pulmonary tuberculosis judgment device and method based on multi-mode fusion

Country Status (1)

Country Link
CN (1) CN112288683A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114974518A (en) * 2022-04-15 2022-08-30 浙江大学 Multi-mode data fusion lung nodule image recognition method and device

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6032678A (en) * 1997-03-14 2000-03-07 Shraga Rottem Adjunct to diagnostic imaging systems for analysis of images of an object or a body part or organ
US20030060688A1 (en) * 2001-09-21 2003-03-27 Active Health Management Care engine
US20120256920A1 (en) * 2011-04-05 2012-10-11 Julian Marshall System and Method for Fusing Computer Assisted Detection in a Multi-Modality, Multi-Dimensional Breast Imaging Environment
CN107403201A (en) * 2017-08-11 2017-11-28 强深智能医疗科技（昆山）有限公司 Tumour radiotherapy target area and jeopardize that organ is intelligent, automation delineation method
CN109192306A (en) * 2018-09-21 2019-01-11 广东工业大学 A kind of judgment means of diabetes, equipment and computer readable storage medium
CN109907753A (en) * 2019-04-23 2019-06-21 杭州电子科技大学 A kind of various dimensions ECG signal intelligent diagnosis system
CN110111892A (en) * 2019-04-29 2019-08-09 杭州电子科技大学 A kind of postoperative short-term relapse and metastasis risk evaluating system of NSCLC patient
CN110309849A (en) * 2019-05-10 2019-10-08 腾讯医疗健康(深圳)有限公司 Blood-vessel image processing method, device, equipment and storage medium
CN110599451A (en) * 2019-08-05 2019-12-20 平安科技（深圳）有限公司 Medical image focus detection positioning method, device, equipment and storage medium
CN110969204A (en) * 2019-11-29 2020-04-07 中国科学院自动化研究所 Sample classification system based on fusion of magnetic resonance image and digital pathology image

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6032678A (en) * 1997-03-14 2000-03-07 Shraga Rottem Adjunct to diagnostic imaging systems for analysis of images of an object or a body part or organ
US20030060688A1 (en) * 2001-09-21 2003-03-27 Active Health Management Care engine
US20120256920A1 (en) * 2011-04-05 2012-10-11 Julian Marshall System and Method for Fusing Computer Assisted Detection in a Multi-Modality, Multi-Dimensional Breast Imaging Environment
CN107403201A (en) * 2017-08-11 2017-11-28 强深智能医疗科技（昆山）有限公司 Tumour radiotherapy target area and jeopardize that organ is intelligent, automation delineation method
CN109192306A (en) * 2018-09-21 2019-01-11 广东工业大学 A kind of judgment means of diabetes, equipment and computer readable storage medium
CN109907753A (en) * 2019-04-23 2019-06-21 杭州电子科技大学 A kind of various dimensions ECG signal intelligent diagnosis system
CN110111892A (en) * 2019-04-29 2019-08-09 杭州电子科技大学 A kind of postoperative short-term relapse and metastasis risk evaluating system of NSCLC patient
CN110309849A (en) * 2019-05-10 2019-10-08 腾讯医疗健康(深圳)有限公司 Blood-vessel image processing method, device, equipment and storage medium
CN110599451A (en) * 2019-08-05 2019-12-20 平安科技（深圳）有限公司 Medical image focus detection positioning method, device, equipment and storage medium
CN110969204A (en) * 2019-11-29 2020-04-07 中国科学院自动化研究所 Sample classification system based on fusion of magnetic resonance image and digital pathology image

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
周涛;陆惠玲;陈志强;马竟先;: "多模态医学影像融合识别技术研究进展", 生物医学工程学杂志, no. 05, pages 215 - 220 *
李帅: "基于深度学习的胸片辅助诊断算法", 中国优秀硕士学位论文全文数据库（医药卫生科技辑）, vol. 076, no. 2020, pages 076 - 8 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114974518A (en) * 2022-04-15 2022-08-30 浙江大学 Multi-mode data fusion lung nodule image recognition method and device

Similar Documents

Publication Publication Date Title
JP7297081B2 (en) Image classification method, image classification device, medical electronic device, image classification device, and computer program
US10810735B2 (en) Method and apparatus for analyzing medical image
WO2020125498A1 (en) Cardiac magnetic resonance image segmentation method and apparatus, terminal device and storage medium
CN111368849B (en) Image processing method, image processing device, electronic equipment and storage medium
US20220058821A1 (en) Medical image processing method, apparatus, and device, medium, and endoscope
US11967181B2 (en) Method and device for retinal image recognition, electronic equipment, and storage medium
CN109492547B (en) Nodule identification method and device and storage medium
CN111369562B (en) Image processing method, image processing device, electronic equipment and storage medium
CN110276408B (en) 3D image classification method, device, equipment and storage medium
Sun et al. Attention-embedded complementary-stream CNN for false positive reduction in pulmonary nodule detection
KR20200082660A (en) Pathological diagnosis method and apparatus based on machine learning
CN112288843A (en) Three-dimensional construction method and device of focus, terminal device and storage medium
WO2020109781A1 (en) Domain adaption
CN112634231A (en) Image classification method and device, terminal equipment and storage medium
Zhuang et al. Tumor classification in automated breast ultrasound (ABUS) based on a modified extracting feature network
Cai et al. Identifying architectural distortion in mammogram images via a se-densenet model and twice transfer learning
CN111325709A (en) Wireless capsule endoscope image detection system and detection method
CN112884702B (en) Polyp identification system and method based on endoscope image
CN112288683A (en) Pulmonary tuberculosis judgment device and method based on multi-mode fusion
CN112330787B (en) Image labeling method, device, storage medium and electronic equipment
KR101923962B1 (en) Method for facilitating medical image view and apparatus using the same
WO2022227193A1 (en) Liver region segmentation method and apparatus, and electronic device and storage medium
CN113658145B (en) Liver ultrasonic standard tangent plane identification method and device, electronic equipment and storage medium
CN115690486A (en) Method, device and equipment for identifying focus in image and storage medium
CN114360695A (en) Mammary gland ultrasonic scanning analysis auxiliary system, medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination