CN108806776A - A method of the Multimodal medical image based on deep learning - Google Patents
A method of the Multimodal medical image based on deep learning Download PDFInfo
- Publication number
- CN108806776A CN108806776A CN201810611120.3A CN201810611120A CN108806776A CN 108806776 A CN108806776 A CN 108806776A CN 201810611120 A CN201810611120 A CN 201810611120A CN 108806776 A CN108806776 A CN 108806776A
- Authority
- CN
- China
- Prior art keywords
- image
- deep learning
- medical image
- image based
- multimodal medical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Epidemiology (AREA)
- Biomedical Technology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Primary Health Care (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Pathology (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention discloses a kind of methods of the Multimodal medical image based on deep learning, include the following steps：S1：Image capturing：The medical image of patient's detection position, S2 are obtained using CT scan and mr imaging technique：Image processing：The two width images obtained using CT scan and mr imaging technique described in S1 are pre-processed, S3：Feature extraction：To treated described in S2, two width images carry out internal feature extraction and surface extraction, S4：Image alignment：The surface of the two width images extracted described in S3 is aligned, the internal feature that two width image of method pair is aligned using image segmentation alignment and pixel characteristic is aligned.The present invention is merged by the medical image to acquisition, and the image data of fusion is shown in the same three dimensions, forms 3-D view, more intuitively can be carried out comprehensive analysis to image, can be made diagnosis faster.
Description
Technical field
The present invention relates to Multimodal medical image technical field more particularly to a kind of multi-modal medicine based on deep learning
The method of image.
Background technology
Currently, with accurate medical treatment and the arriving in big data epoch, other than diagnosing text information, the analysis of image data
And application has become one of the link of clinical medicine more core.Medical staff is as needed to these medical images into pedestrian
Work identifies, to be diagnosed to corresponding sufferer.Since the daily medical image quantity of hospital is thousands of, workload is very big, examines
It is disconnected less efficient.In order to reduce the workload of medical staff, now it is badly in need of a kind of medical image recognition methods, through retrieval, application number
201710071847.2 patent document discloses a kind of Multimodal medical image recognition methods based on deep learning, including：
Multimodal medical image based on patient position to be detected is using method for registering by Multimodal medical image in same three dimensions
Interior display utilizes R-CNN based on Multimodal medical image, the lesion region in Multimodal medical image is identified, according to diseased region
Coordinate of the domain in Multimodal medical image lesions showed body in same three dimensions, it is corresponding according to the lesion region made a definite diagnosis
Image block obtains the probability of happening of each default disease using the dense method of sampling and CNN.The present invention provides a kind of based on depth
The Multimodal medical image identification device of habit, including：Multimodal medical image display module, lesion region detection module, lesion
Body display module presets disease probability of happening module.The present invention realizes the automatic identification of lesion region in medical image, and is
The further diagnosis of doctor provides effective reference data, but this recognition methods is not easy to merge image, no
More intuitively image can be observed, extend Diagnostic Time.
Invention content
Technical problems based on background technology, the present invention propose a kind of multi-modal medicine shadow based on deep learning
The method of picture.
A kind of method of Multimodal medical image based on deep learning proposed by the present invention, includes the following steps：
S1：Image capturing：Patient's detection position is obtained using CT scan and mr imaging technique
Medical image；
S2：Image processing：To what is obtained using CT scan and mr imaging technique described in S1
Two width images are pre-processed；
S3：Feature extraction：Internal feature extraction is carried out to treated described in S2 two width images and surface carries
It takes；
S4：Image alignment：The surface of the two width images extracted described in S3 is aligned, image point is utilized
The internal feature for cutting alignment and pixel characteristic alignment two width image of method pair is aligned；
S5：Visual fusion：After surface described in the S4 is aligned completion with internal feature, two width images are carried out
Fusion, the image data of fusion is shown in the same three dimensions, forms 3-D view；
S6：Image analysing computer：Comprehensive analysis is carried out to the 3-D view described in S5, makes diagnosis.
Preferably, in the S1, carry out CT scan when, using the X-ray beam of Accurate collimation, gamma-rays,
Ultrasonic wave makees profile scanning one by one together with detector around detection position.
Preferably, it in the S1, when carrying out magnetic resonance imaging, is obtained from the position to be detected of human body using electromagnetic induction phenomenon
Electromagnetic signal, and reconstruct human body information.
Preferably, in the S2, when being pre-processed to image, to image into row format, filtering, brightness processed.
Preferably, in the S3, internal feature extraction is mainly with human anatomic structure feature, such as：Skull, backbone, chest
Soft tissue under bone, rib cage, joint and diaphragm.
Preferably, in the S2, when handling image, initial error correction, geometric transformation school are carried out to image
Just.
Preferably, it in the S5, when carrying out visual fusion, needs to assess the error of fusion.
Preferably, it when being filtered to image, needs that signal is kept not suffer a loss, the profile of image cannot be damaged
With marginal information.
Beneficial effects of the present invention：It is merged by the medical image to acquisition, by the image data of fusion same
Display in a three dimensions, forms 3-D view, more intuitively can carry out comprehensive analysis to image, can make faster
Diagnosis.
Specific implementation mode
The present invention is made further to explain with reference to specific embodiment.
Embodiment
A kind of method of the Multimodal medical image based on deep learning is proposed in the present embodiment, is included the following steps：
S1：Image capturing：Patient's detection position is obtained using CT scan and mr imaging technique
Medical image；
S2：Image processing：To what is obtained using CT scan and mr imaging technique described in S1
Two width images are pre-processed；
S3：Feature extraction：Internal feature extraction is carried out to treated described in S2 two width images and surface carries
It takes；
S4：Image alignment：The surface of the two width images extracted described in S3 is aligned, image point is utilized
The internal feature for cutting alignment and pixel characteristic alignment two width image of method pair is aligned；
S5：Visual fusion：After surface described in the S4 is aligned completion with internal feature, two width images are carried out
Fusion, the image data of fusion is shown in the same three dimensions, forms 3-D view；
S6：Image analysing computer：Comprehensive analysis is carried out to the 3-D view described in S5, makes diagnosis.
In the present embodiment, in S1, carry out CT scan when, using the X-ray beam of Accurate collimation, gamma-rays,
Ultrasonic wave makees profile scanning one by one together with detector around detection position, in S1, when carrying out magnetic resonance imaging, and profit
Electromagnetic signal is obtained from the position to be detected of human body with electromagnetic induction phenomenon, and reconstructs human body information, in S2, is carried out to image
When pretreatment, to image into row format, filtering, brightness processed, in S3, internal feature extraction is mainly with human anatomic structure spy
Sign, such as：Soft tissue under skull, backbone, breastbone, rib cage, joint and diaphragm in S2, when handling image, carries out image
Initial error is corrected, geometric transformation correction, in S5, when carrying out visual fusion, needs to assess the error of fusion, right
When image is filtered, needs that signal is kept not suffer a loss, the profile and marginal information of image cannot be damaged, it is of the invention
Advantageous effect is merged by the medical image to acquisition, and the image data of fusion is shown in the same three dimensions
Show, form 3-D view, can comprehensive analysis more intuitively be carried out to image, diagnosis can be made faster.
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Any one skilled in the art in the technical scope disclosed by the present invention, according to the technique and scheme of the present invention and its
Inventive concept is subject to equivalent substitution or change, should be covered by the protection scope of the present invention.
Claims (8)
1. a kind of method of the Multimodal medical image based on deep learning, which is characterized in that include the following steps：
S1：Image capturing：The medicine of patient's detection position is obtained using CT scan and mr imaging technique
Image；
S2：Image processing：To two width obtained using CT scan and mr imaging technique described in S1
Image is pre-processed；
S3：Feature extraction：To treated described in S2, two width images carry out internal feature extraction and surface extraction；
S4：Image alignment：The surface of the two width images extracted described in S3 is aligned, image segmentation pair is utilized
The internal feature of neat and pixel characteristic alignment two width image of method pair is aligned；
S5：Visual fusion：After surface described in the S4 is aligned completion with internal feature, two width images are melted
It closes, the image data of fusion is shown in the same three dimensions, form 3-D view；
S6：Image analysing computer：Comprehensive analysis is carried out to the 3-D view described in S5, makes diagnosis.
2. a kind of method of Multimodal medical image based on deep learning according to claim 1, which is characterized in that institute
It states in S1, when carrying out CT scan, using the X-ray beam, gamma-rays, ultrasonic wave of Accurate collimation together with detector
Make profile scanning one by one around detection position.
3. a kind of method of Multimodal medical image based on deep learning according to claim 1, which is characterized in that institute
It states in S1, when carrying out magnetic resonance imaging, obtains electromagnetic signal from the position to be detected of human body using electromagnetic induction phenomenon, and reconstruct
Human body information.
4. a kind of method of Multimodal medical image based on deep learning according to claim 1, which is characterized in that institute
It states in S2, when being pre-processed to image, to image into row format, filtering, brightness processed.
5. a kind of method of Multimodal medical image based on deep learning according to claim 1, which is characterized in that institute
It states in S3, internal feature extraction is mainly with human anatomic structure feature, such as：Under skull, backbone, breastbone, rib cage, joint and diaphragm
Soft tissue.
6. a kind of method of Multimodal medical image based on deep learning according to claim 1, which is characterized in that institute
It states in S2, when handling image, initial error correction, geometric transformation correction is carried out to image.
7. a kind of method of Multimodal medical image based on deep learning according to claim 1, which is characterized in that institute
It states in S5, when carrying out visual fusion, needs to assess the error of fusion.
8. a kind of method of Multimodal medical image based on deep learning according to claim 4, which is characterized in that
When being filtered to image, needs that signal is kept not suffer a loss, the profile and marginal information of image cannot be damaged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810611120.3A CN108806776A (en) | 2018-06-14 | 2018-06-14 | A method of the Multimodal medical image based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810611120.3A CN108806776A (en) | 2018-06-14 | 2018-06-14 | A method of the Multimodal medical image based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108806776A true CN108806776A (en) | 2018-11-13 |
Family
ID=64086987
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810611120.3A Pending CN108806776A (en) | 2018-06-14 | 2018-06-14 | A method of the Multimodal medical image based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108806776A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859168A (en) * | 2018-12-28 | 2019-06-07 | 上海联影智能医疗科技有限公司 | A kind of X-ray rabat picture quality determines method and device |
| CN111358484A (en) * | 2020-03-23 | 2020-07-03 | 广州医科大学附属第一医院（广州呼吸中心） | Nuclear medicine lung perfusion imaging quantitative analysis method, analysis equipment and storage medium |
| CN111369546A (en) * | 2020-03-10 | 2020-07-03 | 北京邮电大学 | Neck lymph node image classification and identification device and method |
| CN111415727A (en) * | 2020-03-13 | 2020-07-14 | 广州医科大学附属肿瘤医院 | Multi-mode image data management system and analysis method |
| CN111863205A (en) * | 2020-07-23 | 2020-10-30 | 山东协和学院 | Accurate image recognition system and image recognition method |
| CN111914925A (en) * | 2020-07-28 | 2020-11-10 | 复旦大学 | Patient behavior multi-modal perception and analysis system based on deep learning |
| CN113384261A (en) * | 2021-05-28 | 2021-09-14 | 华南理工大学 | Centrum compression fracture multi-mode intelligent diagnosis system based on deep learning |
| US11436720B2 (en) | 2018-12-28 | 2022-09-06 | Shanghai United Imaging Intelligence Co., Ltd. | Systems and methods for generating image metric |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714550A (en) * | 2013-12-31 | 2014-04-09 | 鲁东大学 | Image registration automatic optimization algorithm based on matching of curve characteristic evaluation |
| US20160093050A1 (en) * | 2014-09-30 | 2016-03-31 | Samsung Electronics Co., Ltd. | Image registration device, image registration method, and ultrasonic diagnosis apparatus having image registration device |
| CN106909778A (en) * | 2017-02-09 | 2017-06-30 | 北京市计算中心 | A kind of Multimodal medical image recognition methods and device based on deep learning |
-
2018
- 2018-06-14 CN CN201810611120.3A patent/CN108806776A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714550A (en) * | 2013-12-31 | 2014-04-09 | 鲁东大学 | Image registration automatic optimization algorithm based on matching of curve characteristic evaluation |
| US20160093050A1 (en) * | 2014-09-30 | 2016-03-31 | Samsung Electronics Co., Ltd. | Image registration device, image registration method, and ultrasonic diagnosis apparatus having image registration device |
| CN106909778A (en) * | 2017-02-09 | 2017-06-30 | 北京市计算中心 | A kind of Multimodal medical image recognition methods and device based on deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 宁春玉: "医学影像后处理技术的研究及其在X线影像优化中的应用", 《中国优秀博硕士学位论文全文数据库(硕士) 信息科技辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859168A (en) * | 2018-12-28 | 2019-06-07 | 上海联影智能医疗科技有限公司 | A kind of X-ray rabat picture quality determines method and device |
| US11436720B2 (en) | 2018-12-28 | 2022-09-06 | Shanghai United Imaging Intelligence Co., Ltd. | Systems and methods for generating image metric |
| CN111369546A (en) * | 2020-03-10 | 2020-07-03 | 北京邮电大学 | Neck lymph node image classification and identification device and method |
| CN111369546B (en) * | 2020-03-10 | 2023-07-18 | 北京邮电大学 | Cervical lymph node image classification and identification device and method |
| CN111415727A (en) * | 2020-03-13 | 2020-07-14 | 广州医科大学附属肿瘤医院 | Multi-mode image data management system and analysis method |
| CN111358484A (en) * | 2020-03-23 | 2020-07-03 | 广州医科大学附属第一医院（广州呼吸中心） | Nuclear medicine lung perfusion imaging quantitative analysis method, analysis equipment and storage medium |
| CN111358484B (en) * | 2020-03-23 | 2021-12-24 | 广州医科大学附属第一医院（广州呼吸中心） | Nuclear medicine lung perfusion imaging quantitative analysis method, analysis equipment and storage medium |
| CN111863205A (en) * | 2020-07-23 | 2020-10-30 | 山东协和学院 | Accurate image recognition system and image recognition method |
| CN111914925A (en) * | 2020-07-28 | 2020-11-10 | 复旦大学 | Patient behavior multi-modal perception and analysis system based on deep learning |
| CN113384261A (en) * | 2021-05-28 | 2021-09-14 | 华南理工大学 | Centrum compression fracture multi-mode intelligent diagnosis system based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108806776A (en) | A method of the Multimodal medical image based on deep learning | |
| CN108324246B (en) | Medical diagnosis assisting system and method | |
| CN107920722B (en) | Reconstruction by object detection for images captured from a capsule camera | |
| CN105025799B (en) | Three-dimensional mapping display system for diagnostic ultrasound machine | |
| EP2705806B1 (en) | Advanced fiber tracking and medical navigation in a brain | |
| US20150245882A1 (en) | Systems for linear mapping of lumens | |
| CN106909778A (en) | A kind of Multimodal medical image recognition methods and device based on deep learning | |
| JP2001157675A (en) | Method and apparatus for displaying image | |
| CN111667447A (en) | Intravascular image fusion method and system and image acquisition device | |
| JP2001184492A (en) | Method and device for displaying image | |
| EP3242602B1 (en) | Ultrasound imaging apparatus and method for segmenting anatomical objects | |
| JP2003153082A (en) | Image aligner and image processor | |
| CN107220646B (en) | Medical image character recognition enhancing method for removing background interference | |
| KR20210014267A (en) | Ultrasound diagnosis apparatus for liver steatosis using the key points of ultrasound image and remote medical-diagnosis method using the same | |
| Patias | Medical imaging challenges photogrammetry | |
| CN111904379A (en) | Scanning method and device of multi-modal medical equipment | |
| Rajinikanth et al. | Hybrid image processing methods for medical image examination | |
| KR102242114B1 (en) | Oct medical image based artificial intelligence computer aided diagnosis system and its method | |
| Fleute | Shape reconstruction for computer assisted surgery based on non-rigid registration of statistical models with intra-operative point data and X-ray images | |
| EP2710553B1 (en) | Determination of a physically-varying anatomical structure | |
| KR102136107B1 (en) | Apparatus and method for alignment of bone suppressed chest x-ray image | |
| US10510145B2 (en) | Medical image comparison method and system thereof | |
| KR102495365B1 (en) | System for analysising body joint score of bone scan image using artificial intelligence and method thereof | |
| US20220405957A1 (en) | Computer Vision Systems and Methods for Time-Aware Needle Tip Localization in 2D Ultrasound Images | |
| US20210035274A1 (en) | Method and system for generating an enriched image of a target object and corresponding computer program and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181113 |