CN110517238A - CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system - Google Patents
CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system Download PDFInfo
- Publication number
- CN110517238A CN110517238A CN201910766934.9A CN201910766934A CN110517238A CN 110517238 A CN110517238 A CN 110517238A CN 201910766934 A CN201910766934 A CN 201910766934A CN 110517238 A CN110517238 A CN 110517238A
- Authority
- CN
- China
- Prior art keywords
- data
- medical image
- patient
- dimensional reconstruction
- stl
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
- G06T2207/30064—Lung nodule
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
Abstract
The present invention relates to a kind of CT medical image AI three-dimensional reconstructions and human-computer interaction visual network system.It is responsible for the segmentation of each anatomical organs of the CT image based on deep learning including AI server；Web server uploads for doctor's client, graphics workstation and downloads required file；Doctor's client downloads corresponding STL threedimensional model file for uploading patient's CT images data；Graphics workstation with Web server for realizing interacting and interactive medical image processing；Data archiving system for storage and management original CT data and the STL model data of generation, it can be achieved that patient is postoperative to include the function of qualitative assessment and course of disease tracking quantitative assessment, and can provide new training data, to regularly update AI model for AI server.The present invention can be effectively applied to course of disease tracking, preoperative accurate simulation is planned, navigation, postoperative qualitative assessment and follow-up in art, provide combined imaging application solution for modern times integrated form operating room.
Description
Technical field
The invention belongs to Signal and Information Processing fields, and in particular to a kind of CT medical image AI three-dimensional reconstruction and man-machine friendship
Mutual visual network system.
Background technique
With CT(Computed Tomograph, computed tomography) technology continuous development, multi-layer spiral CT
Scanning can real-time reconstruction obtain millimetre-sized high-resolution thin layer image, it has also become doctor is qualitative, human body is quantitatively evaluated respectively organizes
The important tool of function.By CT images can independently, intuitively, repeatably observe regional area, precise measurement volume, density
Etc. indexs, realize without invasive virtual endoscope detecting；Also it is able to guide operation, expansion disorder in screening etc.[1]。
But it may include several hundred tomographic images that CT scan is reconstructed at present, reading great amount of images is not only time-consuming, is also easy to cause
State of an illness mistaken diagnosis is failed to pinpoint a disease in diagnosis.Accurate structural parameters information can be obtained by computer aided detection technical treatment CT image, will mention
For strong auxiliary diagnosis foundation and three-dimensional visualization image, not only mitigate doctor's burden significantly, but also it is excellent to be conducive to performance equipment
Gesture.The anatomical structure that each tissue is partitioned into from CT image is most basic and most necessary link, has important theory significance
And clinical value.
For many years, domestic and foreign scholars are proposing the partitioning algorithm of many CT images, have threshold method, region-growing method and gather
The conventional segmentation methods such as class method and mathematics morphology, Active contour[2].In recent years, with artificial intelligence and depth
The progress of habit technology, deep learning method present advantage in CT images processing and analysis, will become following this field
Main stream approach.Currently, since initial data source lacks, and marking difficulty, also only in the application based on deep learning method
It is preferably to be in progress in the tissue of some marks for being easier to obtain, for example chest CT is mainly in Lung neoplasm[3,4]With
Lung qi pipe[5]On detection, and complicated blood vessel research is less, is in the ground zero stage.Due to the dissection knot of each tissue CT images
The diversity of the particularity of relevance, characteristics of image, the complexity of grayscale information and form between structure, the field work still
With many difficulty and challenge.
Domestic and foreign hospitals application at present is related to CT three-dimensional reconstruction with the related software interacted, there is MIMICS, is Belgium
A kind of medical image control system that Materialise company releases, is the software of modular construction.MIMICS is a set of height
Integration and easy-to-use 3D rendering generates and the editing and processing software software, it is powerful, but require doctor's participation high.Such as
An example completely three-dimensional lung anatomy is reconstructed, have been spent the several hours time of experienced clinician, majority doctor
Life can not be accomplished.
In addition, the system software that the EDDA scientific & technical corporation in the U.S. releases, provides for the diagnosis and treatment management complete period of major disease
The area of computer aided clinical solution of optimization, wherein navigation system in IQQA-Guide 3-dimensional image art, obtains U.S. FDA batch
Quasi- listing.There is still a need for experienced doctors to spend a lot of time carry out human-computer interaction for the software systems, could obtain complete three-dimensional
Anatomical structure.
Summary of the invention
The purpose of the present invention is to provide a kind of CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system,
It is driven by core of artificial intelligence, the complete period of disease control will be covered, can be effectively applied to course of disease tracking, preoperative accurate simulation
Navigation, postoperative qualitative assessment and follow-up in planning, art provide combined imaging application solution for modern times integrated form operating room.
To achieve the above object, the technical scheme is that a kind of CT medical image AI three-dimensional reconstruction and human-computer interaction
Visual network system, including AI server, Web server, doctor's client, graphics workstation and data archiving system；
The AI server is responsible for the segmentation of each anatomical organs of the CT image based on deep learning；
The Web server includes patient CT data's file management system and STL threedimensional model file management system, for doctor
Client, graphics workstation, which upload, downloads required file；
Doctor's client uploads patient's CT images data to patient CT data's file pipe by Web for registered doctor user
Reason system after handling via system, downloads corresponding STL threedimensional model file by STL threedimensional model file management system；
The graphics workstation includes Web client and interactive medical image processing software, the Web client for realizing
With the interaction of the STL threedimensional model file management system of Web server, it is described interactive mode medical image processing software for realizing
Interactive medical image processing；
The data archiving system is arranged in the graphics workstation backstage, for storage and management original CT data and generation
STL model data, it can be achieved that patient it is postoperative include being quantitatively evaluated and the function of course of disease tracking quantitative assessment, and can be AI server
New training data is provided, to regularly update AI model.
In an embodiment of the present invention, the AI server includes AI training module and AI test module, the AI training
Module can carry out the strategy of stage update to the AI test model in AI test module, as data are increasing, reconstruction
Good result continues to train AI test model as new training data.
In an embodiment of the present invention, patient CT data's file management system storage management is uploaded by doctor's client
Patient's CT images；STL threedimensional model file management system storage and management is by medical image interactive in graphics workstation
Manage the STL threedimensional model file of Software Create, doctor's client can download corresponding STL model, doctor's client can be into
Row interactive browser.
In an embodiment of the present invention, doctor's client uploads patient's CT images data to patient CT data's file pipe
When reason system, corresponding serial number can be automatically generated；The interactive mode medical image processing software is suitable according to serial number file
Sequence handles patient's CT images data.
In an embodiment of the present invention, the data archiving system is according to serial number storage and management original CT data and life
At STL model data.
In an embodiment of the present invention, the interactive medical image processing software realization process is as follows:
1) CT sequence D ICOM image data imports and exports；
2) CT sequence D ICOM pre-processing image data is generated into AI test data；
3) preprocessed data is uploaded to AI server, after AI server process, downloads AI segmentation result；
4) three-dimensional reconstruction is carried out to AI segmentation result；
5) tracking calibration caliber tracking and calibration: is carried out to the result of the vascular arteriovenous three-dimensional reconstruction of lung qi pipe and each tissue；
6) result of three-dimensional reconstruction is generated as STL threedimensional model file, and is transferred to STL threedimensional model file management system.
In an embodiment of the present invention, specific step is as follows for the step 2:
2.1) patient's CT images data of DICOM data format, i.e. patient CT sequence D ICOM image are obtained, by its most fine sequence
The interface that column are provided by ITK open source software packet, is converted into the CT body data of " .nii " format；
2.2) grey scale: the window width and window level of data is adjusted to the tonal range best to corresponding anatomical tissue contrast, no
Value of the same tissue or anatomical structure in CT is different, and CT body data normalization to 0 to 255 gray levels；
2.3) data normalization: acquisition normalization data is normalized by formula (1) in the CT body data after standardizationV norm , whereinVFor initial body data,V mean ForVThe mean value of figure；
(1)
2.4) data 3D standardizes: former data standard is turned into 1024*1024*320 three-dimensional data, then to volume data according to
128*128*64 carries out stripping and slicing, and every block number does not overlap between, obtains 320 individual data items blocks；Finally, using this data block as net
The input of network.
In an embodiment of the present invention, the AI server, which uses, is based on the improved deep learning network model of 3D U-Net
The data block of acquisition is handled, it is described that 22 3D convolution are included based on the improved deep learning network model of 3D U-Net
Layer, wherein 4 maximum corresponding 4 up-sampling layers of pond layer, and 4 articulamentums are set, the last one 3D convolutional layer can be according to not
Different classification settings is carried out with demand.
In an embodiment of the present invention, specific step is as follows for the step 5):
5.1) skeletal extraction is carried out to the blood vessel of system reconstructing or tracheae 3-D graphic, obtains skeletal point；
5.2) bifurcation detection is carried out to the skeletal point obtained using bifurcation detection algorithm, obtains bifurcation；Wherein, bifurcated
Point detection algorithm implementation process is as follows:
26 neighborhoods of skeletal point are counted, the number of 26 neighborhood middle skeleton points is counted；Under normal circumstances, the company of a pipeline
In logical domain, if it is bifurcation, it includes skeletal point number should be greater than 3；Therefore, comprising skeletal point number greater than 3 can
It is considered bifurcation；
5.3) using skeletal point direction tracking tracking skeletal point and bifurcation, hierarchical detection then is carried out to blood vessel or tracheae
With calibration.
Compared to the prior art, the invention has the following advantages: present system drives by core of artificial intelligence,
It is process object with CT sequential images, the complete period of disease control will be covered, can be effectively applied to course of disease tracking, preoperative accurate mould
Navigation, postoperative qualitative assessment and follow-up in quasi- planning, art provide combined imaging using solution party for modern times integrated form operating room
Case.
Detailed description of the invention
Fig. 1 is system the general frame.
Fig. 2 is interactive medical image processing software UI schematic diagram.
Fig. 3 is blood vessel and tracheae track algorithm flow diagram.
Fig. 4 is bifurcation detection signal.
Fig. 5 is the detection signal of skeleton spot moving direction.
Fig. 6 is the STL model interactive operation schematic diagram based on WebGL.
Specific embodiment
With reference to the accompanying drawing, technical solution of the present invention is specifically described.
The present invention provides a kind of CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system, by artificial intelligence
(AI) 5 server, Web server, doctor's client, graphics workstation and data archiving system components are constituted, the total frame of system
Figure is as shown in Figure 1.
1, artificial intelligence (AI) server
Artificial intelligence (AI) server is the core component of system algorithm, includes AI training module and AI test module.AI network
Relate generally to the improvement network based on 3D U-Net.For parenchymatous disease class using the network of more shallow-layer, network training speed is fast, and
Precision is met the requirements；Caliber class, such as lung qi pipe, the vascular arteriovenous respectively organized, then using the network of more deep layer, to obtain more
Good segmentation effect.The above network model can aid in corresponding pretreatment and network according to the CT images feature of different tissues
Improvement, particular content square method description.The present invention is quasi- to take the strategy that stage update is carried out to each test model of AI, with
Data (example) are increasing, and on the basis of ensuring patients' privacy, continue to instruct using the result rebuild as new training data
Practice AI model.
2, Web server
Web server is patient's CT images data and storage and the interaction node for rebuilding STL threedimensional model, by patient CT data's text
Part management system and STL model file management system composition.Patient CT data's file management system is stored and is managed by doctor visitor
Patient's CT images that family end uploads.STL model file management system storage and management is by medicine shadow interactive in graphics workstation
As the STL threedimensional model file that processing software generates, doctor can download corresponding STL model, interact in Web client
Browsing.
3, doctor's client
Doctor's client (including mobile terminal), is Web client.This system registered doctor user uploads patient CT shadow by Web
As data, steps are as follows:
(1) upload data before, must fill in the CT images of being passed relevant information (including patient's name, ID number, check number, inspection
Survey time, medical history etc.), system automatically generated serial number after filling in (this number guarantees the number of cases according to uniqueness in systems).
(2) after uploading, waiting system processing result.Doctor can check system processing result by Web, have letter after handling well
Breath prompt.
(3) after receiving information, the corresponding STL threedimensional model file of passed data can be downloaded.To protect patients' privacy, when
After stl file is downloaded successfully, the CT images data uploaded can be voluntarily deleted.
STL threedimensional model file browsed by Web client based on the software tool of WebGL exploitation, interactive operation.It hands over
The function such as display, rotation, color setting, transparency setting, the hiding, label of anatomical structure of the interoperability comprising each three-dimensional reconstruction
Can, combined imaging application solution is provided for navigation in preoperative accurate simulation planning, art.
4, graphics workstation
Graphics workstation is made of Web client, interactive medical image processing software two parts.Web client is realized and Web
The file system of server interacts, and by Web downloading patient CT sequence image, generates storage folder by serial number.In
It, can be the data sync storage to data archiving system under the license for uploading doctor.
Interactive medical image processing software is according to serial number file sequential processes patient's CT images data.Interactive mode doctor
Learning image processing software is the core component that system is realized.The software will complete following functions: 1) CT sequence D ICOM image imports
Export；2) pretreatment generates AI test data；3) human-computer interaction function；4) three-dimensional reconstruction carries out Three-dimensional Gravity to AI segmentation result
It builds；5) tracking and calibration of caliber (such as lung qi pipe and the vascular arteriovenous respectively organized), point of automatic marking to the sub- section of three-level
Crunode and branch；6) stl file is generated, web browsing is convenient for, and considers to adapt to the display and interaction of mobile terminal, generates a kind of pressure
Contracting version, and the data are stored by serial number to data archiving system.
5, data archiving system
Data archiving system is this system local datastore system, is stored by serial number, including patient's CT images initial data,
The STL threedimensional model file of each tissue anatomical structure after each patient's reconstruction.Every processing an example patient's CT images, in conjunction with mark
Tool can be generated as new training data, update AI training module and AI test model.In addition, in data archiving system
Setting one it is postoperative/course of disease be quantitatively evaluated module, can be provided for patient related disease early detection and diagnosis, the course of disease track,
The functions such as postoperative qualitative assessment and follow-up, and then realize the diagnosis and treatment and management in patient's related disease complete period.
In the present invention, since CT sequence original image is DICOM data format, in order to allow deep learning network model to obtain
Better characteristic needs to pre-process original CT image.Pre-treatment step is as follows:
1) patient's CT images data of DICOM data format, i.e. patient CT sequence D ICOM image are obtained, by its most fine sequence
(the three-dimensional coordinate x/y/z for representing each sequence size being multiplied, maximum value is desired sequence) passes through ITK open source software packet
The interface of offer is converted into the CT body data of " .nii " format；
2) window width and window level of data: being adjusted the tonal range best to corresponding anatomical tissue contrast by grey scale, different
Value in CT of tissue or anatomical structure it is different, and CT body data normalization to 0 to 255 gray levels；
3) data normalization: acquisition normalization data is normalized by formula (1) in the CT body data after standardizationV norm , whereinVFor initial body data,V mean ForVThe mean value of figure；
(1)
4) data 3D standardizes: the resolution sizes of the every tomographic image of CT sequential images are generally 1024*1024, the number of plies of adult
General 300 or more differ.Therefore, former data standard can be turned into 1024*1024*320 three-dimensional data, then volume data is pressed
Stripping and slicing is carried out according to 128*128*64, every block number does not overlap between, obtains 320 individual data items blocks；Finally, using this data block as
The input of network.
The improved deep learning network model of 3D U-Net is based in the present invention:
The 3D U-Net improved network model of the present invention that is based on joined a variety of rule on the basis of 3D U-Net model
Generalized means, the effect of Lai Gaishan U-Net application.Firstly, crowd standardization (Batch is added after every layer of 3D convolution
Normalization, BN) processing, can preferably processing feature spatial distribution change, and effectively accelerate training.Secondly, In
The constricted path of network is added depth supervision module and exports the volume of network in advance behind second up-sampling path of master network
Outside as a result, not only supervising the output of the last layer of network, output in advance is also supervised, this can improve well gradient and disappear
The phenomenon that mistake.The training process of network then uses the classification cross entropy loss function of binary_crossentropy bis-:
(2)
The network is as described in Table 1, is made of action type, convolution kernel, port number, input size and the column of output size 5, wherein grasping
Making type includes 3D convolution (Convolution3D), batch normalization (Batch Normalization, BN), maximum pond
(MaxPooling3D), the operations such as (UpSampling3D), connection (Concatenate), activation (Activation) are up-sampled.
The network shares 22 3D convolutional layers, 4 maximum corresponding 4 up-sampling layers of pond layer, and 4 articulamentums are arranged and (are separately connected
(conv11, conv8), (conv14, conv6), (conv17, conv4) and (conv20, conv2)), the last one volume
Lamination (22 layers) can carry out different classification settings according to different demands.The network is defeated with 128*128*64 3D data block
Enter, according to different tasks, output size is different, which is suitably for the situation of 2 classification, suitable for the defeated of single class anatomical structure
Out, such as lung qi pipe, tissue blood vessel.But also there are the situations of more classification outputs, such as lobe of the lung, publicly-owned 5 lobes of the lung of pulmo, on the left side
Lower two lobes of the lung, three lobe of the lung of the right upper, middle and lower just use 6 disaggregated model of 3D.
Interactive medical image processing software uses interactive medical image processing technology in the present invention:
The workspace of interactive medical image processing software is by four cross section, sagittal plane, frontal plane, three-dimensional reconstruction group of windows
At left-hand column and upper sidebar are toolbar, and right hand column is that column is arranged in attribute.Software UI signal is as shown in Figure 2.
Interactive mode medical image processing software of the invention is with the following functions:
(1) CT sequence D ICOM image imports and exports, import realize thickness it is optional, export can for sequence " .jpg ", " .png " or
" .nii " volume data.
(2) pretreatment generates AI test data.For different anatomical structures, different preprocess methods is had, i.e., not
With window width and window level adjustment and region segmentation method so that test data is closer to training data.
(3) human-computer interaction function can show cross section, sagittal plane and frontal plane sequence image, including display CT value, image
The functions such as zoom, translation, window width and window level adjusting, full screen display；Frontal plane, sagittal plane realize angle, distance on cross section
Measurement realizes that Freehandhand-drawing regional choice, area measurement, mean CT-number calculates, region histogram is shown；Freehandhand-drawing area is realized on cross section
Domain selection, and three-dimensional reconstruction and display are carried out to selection region.There are also Interactive Segmentation, the three-dimensional reconstruction of multiple objects and visual
Change, including multiple entity attributes setting (color, transparency, switch, additions and deletions) and pickup etc.；
(4) three-dimensional reconstruction carries out three-dimensional reconstruction to AI segmentation result, and needs to carry out certain manual intervention post-processing.
(5) caliber tracking and calibration.Since operation reference and navigation need, need the blood vessel to lung qi pipe and each tissue dynamic
The result of vein three-dimensional reconstruction carries out tracking calibration.
(6) stl file is generated, web browsing is convenient for, three-dimensional reconstruction result needs to be generated as STL threedimensional model file, do not examine
Consider the display and interaction for adapting to mobile terminal, generates a kind of compressed version, and the data are stored by serial number to data file
Filing system.
Caliber tracking and calibration are realized using the blood vessel based on skeleton topology/tracheae tracking and calibration technique in the present invention:
In view of each tissue blood vessel and tracheae show as three-dimensional tubulose connectivity structure in volume data, using based on skeleton topology
The method of blood vessel and tracheae tracking and calibration.Method flow block diagram is as shown in Figure 3.
(1) skeletal extraction is carried out to the blood vessel of system reconstructing or tracheae 3-D graphic, method, which uses, is based on Fast Marching side
Method optimizes dual range field and improves the full-automatic three-dimensional framework extraction method of SUSAN end-point detection[6], obtain skeletal point.
(2) bifurcation detection is carried out to the skeletal point obtained, obtains bifurcation.
Bifurcation detection algorithm: 26 neighborhoods (Fig. 4 (a) is 26 Neighborhood Graphs) of skeletal point are counted, 26 neighborhoods are counted
The number of middle skeleton point.Under normal circumstances, in the connected domain of a pipeline, if it is bifurcation, it includes skeletal point number
3 should be greater than.Therefore, bifurcation is regarded as greater than 3 comprising skeletal point number.As shown in figure 4, marking the point for being in figure
Indicate skeletal point, when neighborhood skeletal point number is greater than 3, such as Fig. 4 (c), central point is regarded as bifurcation.And the center Fig. 4 (b)
Point then thinks overstepping one's bounds crunode.
(3) then, skeletal point and bifurcation are tracked, hierarchical detection and calibration then are carried out to blood vessel or tracheae.
Skeletal point direction tracking: since bifurcation shows as the convergent point of skeletal point in 26 neighborhoods, it can be determined that
The direction of motion of skeletal point out.As shown in figure 5, wherein the point labeled as 1 is bifurcation, skeletal point is indicated labeled as 2 arrow
The direction of movement.
STL threedimensional model file browsed by doctor client based on the software tool of WebGL exploitation, interactive operation.
By taking lung as an example, interactive operation includes intratracheal, blood vessel, tubercle, the display of pulmonary parenchyma, rotation, color is arranged, transparency is set
The functions such as set, hide, marking.Schematic diagram is as shown in Figure 6.
[1]Ginneken V, Bram. Fifty years of computer analysis in chest
imaging: rule-based, machine learning, deep learning[J]. Radiological Physics
and Technology, 2017, 10(1): 23-32.
[2] Bian Zijian, Qin Wenjun, Liu Jiren wait the anatomical structure dividing method in lung CT image to summarize in [J]
State's image graphics journal, 2018,23 (10): 22-43.
[3]Setio A A A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in
CT images: false positive reduction using multi-view convolutional networks
[J]. IEEE transactions on medical imaging, 2016, 35(5): 1160-1169.
[4]Dou Q, Chen H, Yu L, et al. Multilevel contextual 3-D CNNs for false
positive reduction in pulmonary nodule detection[J]. IEEE Transactions on
Biomedical Engineering, 2016, 64(7): 1558-1567.
[5]Yun J, Park J, Yu D, et al. Improvement of fully automated airway
segmentation on volumetric computed tomographic images using a 2.5
dimensional convolutional neural net[J]. Medical image analysis, 2019, 51:
13-20.
[6] a kind of three-dimensional framework extraction algorithm towards human body tubular tissue of Geng Huan, Yang Jinzhu, Zhao great Zhe, et al.
[J] Chinese journal of scientific instrument, 2014,35 (4): 754-761..
The above are preferred embodiments of the present invention, all any changes made according to the technical solution of the present invention, and generated function is made
When with range without departing from technical solution of the present invention, all belong to the scope of protection of the present invention.
Claims (9)
1. a kind of CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system, which is characterized in that serviced including AI
Device, Web server, doctor's client, graphics workstation and data archiving system；
The AI server is responsible for the segmentation of each anatomical organs of the CT image based on deep learning；
The Web server includes patient CT data's file management system and STL threedimensional model file management system, for doctor
Client, graphics workstation, which upload, downloads required file；
Doctor's client uploads patient's CT images data to patient CT data's file pipe by Web for registered doctor user
Reason system after handling via system, downloads corresponding STL threedimensional model file by STL threedimensional model file management system；
The graphics workstation includes Web client and interactive medical image processing software, the Web client for realizing
With the interaction of the STL threedimensional model file management system of Web server, it is described interactive mode medical image processing software for realizing
Interactive medical image processing；
The data archiving system is arranged in the graphics workstation backstage, for storage and management original CT data and generation
STL model data, it can be achieved that patient it is postoperative include being quantitatively evaluated and the function of course of disease tracking quantitative assessment, and can be AI server
New training data is provided, to regularly update AI model.
2. CT medical image AI three-dimensional reconstruction according to claim 1 and human-computer interaction visual network system, feature
It is, the AI server includes AI training module and AI test module, and the AI training module can be in AI test module
AI test model carry out stage update strategy, as data are increasing, using the result rebuild as new training number
According to continue train AI test model.
3. CT medical image AI three-dimensional reconstruction according to claim 1 and human-computer interaction visual network system, feature
It is, patient's CT images that patient CT data's file management system storage management is uploaded by doctor's client；STL three-dimensional mould
Type file management system storage and management by medical image processing Software Create interactive in graphics workstation STL threedimensional model
File, doctor's client can download corresponding STL model, can interact browsing in doctor's client.
4. CT medical image AI three-dimensional reconstruction according to claim 1 and human-computer interaction visual network system, feature
It is, when doctor's client uploads patient's CT images data to patient CT data's file management system, can automatically generates pair
The serial number answered；The interactive mode medical image processing software is according to serial number file sequential processes patient's CT images data.
5. CT medical image AI three-dimensional reconstruction according to claim 4 and human-computer interaction visual network system, feature
It is, the data archiving system is according to the STL model data of serial number storage and management original CT data and generation.
6. CT medical image AI three-dimensional reconstruction according to claim 1 and human-computer interaction visual network system, feature
It is, the interactive mode medical image processing software realization process is as follows:
1) CT sequence D ICOM image data imports and exports；
2) CT sequence D ICOM pre-processing image data is generated into AI test data；
3) preprocessed data is uploaded to AI server, after AI server process, downloads AI segmentation result；
4) three-dimensional reconstruction is carried out to AI segmentation result；
5) tracking calibration caliber tracking and calibration: is carried out to the result of the vascular arteriovenous three-dimensional reconstruction of lung qi pipe and each tissue；
6) result of three-dimensional reconstruction is generated as STL threedimensional model file, and is transferred to STL threedimensional model file management system.
7. CT medical image AI three-dimensional reconstruction according to claim 6 and human-computer interaction visual network system, feature
It is, specific step is as follows for the step 2:
2.1) patient's CT images data of DICOM data format, i.e. patient CT sequence D ICOM image are obtained, by its most fine sequence
The interface that column are provided by ITK open source software packet, is converted into the CT body data of " .nii " format；
2.2) grey scale: the window width and window level of data is adjusted to the tonal range best to corresponding anatomical tissue contrast, no
Value of the same tissue or anatomical structure in CT is different, and CT body data normalization to 0 to 255 gray levels；
2.3) data normalization: acquisition normalization data is normalized by formula (1) in the CT body data after standardizationV norm , whereinVFor initial body data,V mean ForVThe mean value of figure；
(1)
2.4) data 3D standardizes: former data standard is turned into 1024*1024*320 three-dimensional data, then to volume data according to
128*128*64 carries out stripping and slicing, and every block number does not overlap between, obtains 320 individual data items blocks；Finally, using this data block as net
The input of network.
8. CT medical image AI three-dimensional reconstruction according to claim 7 and human-computer interaction visual network system, feature
Be, the AI server use based on the improved deep learning network model of 3D U-Net to the data block of acquisition at
Reason, it is described that 22 3D convolutional layers are included based on the improved deep learning network model of 3D U-Net, wherein 4 maximum pond layers
Corresponding 4 up-sampling layers, and 4 articulamentums are set, the last one 3D convolutional layer can carry out different classification according to different demands
Setting.
9. CT medical image AI three-dimensional reconstruction according to claim 6 and human-computer interaction visual network system, feature
It is, specific step is as follows for the step 5):
5.1) skeletal extraction is carried out to the blood vessel of system reconstructing or tracheae 3-D graphic, obtains skeletal point；
5.2) bifurcation detection is carried out to the skeletal point obtained using bifurcation detection algorithm, obtains bifurcation；Wherein, bifurcated
Point detection algorithm implementation process is as follows:
26 neighborhoods of skeletal point are counted, the number of 26 neighborhood middle skeleton points is counted；Under normal circumstances, the company of a pipeline
In logical domain, if it is bifurcation, it includes skeletal point number should be greater than 3；Therefore, comprising skeletal point number greater than 3 can
It is considered bifurcation；
5.3) using skeletal point direction tracking tracking skeletal point and bifurcation, hierarchical detection then is carried out to blood vessel or tracheae
With calibration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910766934.9A CN110517238B (en) | 2019-08-20 | 2019-08-20 | AI three-dimensional reconstruction and human-computer interaction visualization network system for CT medical image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910766934.9A CN110517238B (en) | 2019-08-20 | 2019-08-20 | AI three-dimensional reconstruction and human-computer interaction visualization network system for CT medical image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110517238A true CN110517238A (en) | 2019-11-29 |
| CN110517238B CN110517238B (en) | 2022-01-11 |
Family
ID=68626718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910766934.9A Active CN110517238B (en) | 2019-08-20 | 2019-08-20 | AI three-dimensional reconstruction and human-computer interaction visualization network system for CT medical image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110517238B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111553979A (en) * | 2020-05-26 | 2020-08-18 | 广州狄卡视觉科技有限公司 | Operation auxiliary system and method based on medical image three-dimensional reconstruction |
| CN111968070A (en) * | 2020-04-22 | 2020-11-20 | 深圳睿心智能医疗科技有限公司 | Blood vessel detection method and device based on three-dimensional modeling |
| CN112294434A (en) * | 2020-11-05 | 2021-02-02 | 辽宁省肿瘤医院 | Application of IQQA surgical planning platform in interventional radiology accurate volume spleen embolism |
| CN112967786A (en) * | 2021-02-26 | 2021-06-15 | 江南大学 | Construction method and system of anatomical navigation based on multimode image and interactive equipment |
| CN113223013A (en) * | 2021-05-08 | 2021-08-06 | 推想医疗科技股份有限公司 | Method, device, equipment and storage medium for pulmonary vessel segmentation positioning |
| CN113674279A (en) * | 2021-10-25 | 2021-11-19 | 青岛美迪康数字工程有限公司 | Coronary artery CTA image processing method and device based on deep learning |
| CN114171166A (en) * | 2021-01-20 | 2022-03-11 | 赛维森(广州)医疗科技服务有限公司 | Management system of model of visual digital pathological artificial intelligence |
| CN114266856A (en) * | 2021-10-08 | 2022-04-01 | 上海应用技术大学 | Portable CT visualization device |
| WO2022222458A1 (en) * | 2021-04-19 | 2022-10-27 | 温州医科大学 | Artificial intelligence-assisted diagnosis model construction system for medical images |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103136785A (en) * | 2013-02-01 | 2013-06-05 | 上海交通大学医学院附属第九人民医院 | Human body medical model three-dimensional visualization system used for mobile terminal and realizing method thereof |
| CN104318057A (en) * | 2014-09-25 | 2015-01-28 | 新乡医学院第一附属医院 | Medical image three-dimensional visualization system |
| CN105139443A (en) * | 2015-07-30 | 2015-12-09 | 芜湖卫健康物联网医疗科技有限公司 | Three-dimensional imaging system and method of diagnosis result |
| CN106898043A (en) * | 2017-02-08 | 2017-06-27 | 上海维尔盛视智能科技有限公司 | A kind of PACK based on virtual reality |
| CN107563383A (en) * | 2017-08-24 | 2018-01-09 | 杭州健培科技有限公司 | A kind of medical image auxiliary diagnosis and semi-supervised sample generation system |
| CN107610743A (en) * | 2017-11-09 | 2018-01-19 | 同心医联科技（北京）有限公司 | Medical imaging and the comprehensive solution system of diagnosis based on internet cloud technology |
| CN108694702A (en) * | 2018-03-30 | 2018-10-23 | 宁波宝略智能科技有限公司 | A kind of spatial coordinate system conversion method of oblique photograph outdoor scene threedimensional model |
| CN109584998A (en) * | 2018-11-01 | 2019-04-05 | 常州华森三维科技股份有限公司 | Medical image service management system and method |
| US10319476B1 (en) * | 2015-02-06 | 2019-06-11 | Brain Trust Innovations I, Llc | System, method and device for predicting an outcome of a clinical patient transaction |
-
2019
- 2019-08-20 CN CN201910766934.9A patent/CN110517238B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103136785A (en) * | 2013-02-01 | 2013-06-05 | 上海交通大学医学院附属第九人民医院 | Human body medical model three-dimensional visualization system used for mobile terminal and realizing method thereof |
| CN104318057A (en) * | 2014-09-25 | 2015-01-28 | 新乡医学院第一附属医院 | Medical image three-dimensional visualization system |
| US10319476B1 (en) * | 2015-02-06 | 2019-06-11 | Brain Trust Innovations I, Llc | System, method and device for predicting an outcome of a clinical patient transaction |
| CN105139443A (en) * | 2015-07-30 | 2015-12-09 | 芜湖卫健康物联网医疗科技有限公司 | Three-dimensional imaging system and method of diagnosis result |
| CN106898043A (en) * | 2017-02-08 | 2017-06-27 | 上海维尔盛视智能科技有限公司 | A kind of PACK based on virtual reality |
| CN107563383A (en) * | 2017-08-24 | 2018-01-09 | 杭州健培科技有限公司 | A kind of medical image auxiliary diagnosis and semi-supervised sample generation system |
| CN107610743A (en) * | 2017-11-09 | 2018-01-19 | 同心医联科技（北京）有限公司 | Medical imaging and the comprehensive solution system of diagnosis based on internet cloud technology |
| CN108694702A (en) * | 2018-03-30 | 2018-10-23 | 宁波宝略智能科技有限公司 | A kind of spatial coordinate system conversion method of oblique photograph outdoor scene threedimensional model |
| CN109584998A (en) * | 2018-11-01 | 2019-04-05 | 常州华森三维科技股份有限公司 | Medical image service management system and method |
Non-Patent Citations (1)
| Title |
|---|
| 耿欢: "基于MSCT的肺功能定量评估关键算法研究", 《医药卫生科技辑》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111968070A (en) * | 2020-04-22 | 2020-11-20 | 深圳睿心智能医疗科技有限公司 | Blood vessel detection method and device based on three-dimensional modeling |
| CN111968070B (en) * | 2020-04-22 | 2023-12-05 | 深圳睿心智能医疗科技有限公司 | Blood vessel detection method and device based on three-dimensional modeling |
| CN111553979A (en) * | 2020-05-26 | 2020-08-18 | 广州狄卡视觉科技有限公司 | Operation auxiliary system and method based on medical image three-dimensional reconstruction |
| CN111553979B (en) * | 2020-05-26 | 2023-12-26 | 广州雪利昂生物科技有限公司 | Operation auxiliary system and method based on three-dimensional reconstruction of medical image |
| CN112294434A (en) * | 2020-11-05 | 2021-02-02 | 辽宁省肿瘤医院 | Application of IQQA surgical planning platform in interventional radiology accurate volume spleen embolism |
| CN114171166A (en) * | 2021-01-20 | 2022-03-11 | 赛维森(广州)医疗科技服务有限公司 | Management system of model of visual digital pathological artificial intelligence |
| CN112967786A (en) * | 2021-02-26 | 2021-06-15 | 江南大学 | Construction method and system of anatomical navigation based on multimode image and interactive equipment |
| CN112967786B (en) * | 2021-02-26 | 2023-04-18 | 江南大学 | Construction method and system of anatomical navigation based on multimode image and interactive equipment |
| WO2022222458A1 (en) * | 2021-04-19 | 2022-10-27 | 温州医科大学 | Artificial intelligence-assisted diagnosis model construction system for medical images |
| CN113223013A (en) * | 2021-05-08 | 2021-08-06 | 推想医疗科技股份有限公司 | Method, device, equipment and storage medium for pulmonary vessel segmentation positioning |
| CN114266856A (en) * | 2021-10-08 | 2022-04-01 | 上海应用技术大学 | Portable CT visualization device |
| CN113674279A (en) * | 2021-10-25 | 2021-11-19 | 青岛美迪康数字工程有限公司 | Coronary artery CTA image processing method and device based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110517238B (en) | 2022-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110517238A (en) | CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system | |
| Ueda et al. | Technical and clinical overview of deep learning in radiology | |
| US10713856B2 (en) | Medical imaging system based on HMDS | |
| JP2022529682A (en) | Medical video splitting methods, devices, computer devices and computer programs | |
| CN106569673A (en) | Multi-media case report displaying method and displaying device for multi-media case report | |
| CN104586418B (en) | medical image data processing apparatus and medical image data processing method | |
| CN110163877A (en) | A kind of method and system of MRI ventricular structure segmentation | |
| CN107481326A (en) | A kind of anatomical structure VR display methods rendered based on CT images body | |
| CN112967786B (en) | Construction method and system of anatomical navigation based on multimode image and interactive equipment | |
| CN109727197A (en) | A kind of medical image super resolution ratio reconstruction method | |
| CN111383328A (en) | 3D visualization method and system for breast cancer focus | |
| US20240062498A1 (en) | Systems and methods for rendering models based on medical imaging data | |
| Rhee et al. | Scan-based volume animation driven by locally adaptive articulated registrations | |
| CN112967254A (en) | Lung disease identification and detection method based on chest CT image | |
| US20230054394A1 (en) | Device and system for multidimensional data visualization and interaction in an augmented reality virtual reality or mixed reality image guided surgery | |
| Abbasov | Artificial intelligence in medical imaging | |
| WO2022163513A1 (en) | Learned model generation method, machine learning system, program, and medical image processing device | |
| WO2022163402A1 (en) | Learned model generation method, machine learning system, program, and medical image processing device | |
| Zhao et al. | Medical images super resolution reconstruction based on residual network | |
| CN109377477A (en) | A kind of method, apparatus and computer readable storage medium of image classification | |
| Poh et al. | Addressing the future of clinical information systems—Web-based multilayer visualization | |
| CN108573514A (en) | A kind of three-dimensional fusion method and device of image, computer storage media | |
| Kunert et al. | An interactive system for volume segmentation in computer-assisted surgery | |
| Behrendt et al. | 2.5 D Geometric Mapping of Aortic Blood Flow Data for Cohort Visualization. | |
| Zhang et al. | The Application of 3D Virtual Technology in the Teaching of Clinical Medicine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |