CN112086174B - Three-dimensional knowledge diagnosis model construction method and system - Google Patents
Three-dimensional knowledge diagnosis model construction method and system Download PDFInfo
- Publication number
- CN112086174B CN112086174B CN202011015038.8A CN202011015038A CN112086174B CN 112086174 B CN112086174 B CN 112086174B CN 202011015038 A CN202011015038 A CN 202011015038A CN 112086174 B CN112086174 B CN 112086174B
- Authority
- CN
- China
- Prior art keywords
- dimensional knowledge
- medical image
- training
- image data
- probability value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003745 diagnosis Methods 0.000 title claims abstract description 56
- 238000010276 construction Methods 0.000 title claims description 7
- 238000012549 training Methods 0.000 claims abstract description 61
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000013528 artificial neural network Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 41
- 238000003860 storage Methods 0.000 claims description 16
- 230000002776 aggregation Effects 0.000 claims description 9
- 238000004220 aggregation Methods 0.000 claims description 9
- 201000010099 disease Diseases 0.000 claims description 9
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 7
- 230000035945 sensitivity Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000009795 derivation Methods 0.000 claims description 2
- 230000008685 targeting Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 12
- 238000013527 convolutional neural network Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 206010028980 Neoplasm Diseases 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 201000011510 cancer Diseases 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000009966 trimming Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Epidemiology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Primary Health Care (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Pathology (AREA)
- Databases & Information Systems (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Image Analysis (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention discloses a method and a system for constructing a three-dimensional knowledge diagnosis model. The method comprises the following steps: creating a training set based on the known medical image data, the training set comprisingAndtwo parts, wherein X m Representing the medical image data of the mth example, Y m Label, X representing mth example medical image data i Representing the i-th example of unlabeled medical image data, M and N being the number of corresponding samples respectively; training the constructed three-dimensional knowledge diagnosis model based on the neural network with the set loss function as the target, wherein X m ，Y m Input and label of supervised training as main branch of the three-dimensional knowledge diagnostic model, X i As input to the unsupervised training of the main branch of the three-dimensional knowledge diagnostic model,as input to the unsupervised training of the auxiliary branches of the three-dimensional knowledge diagnostic model,is to X i And carrying out data after various disturbances. The invention can realize high-efficiency intelligent diagnosis on medical images and is used for clinical indication.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a method and a system for constructing a three-dimensional knowledge diagnosis model.
Background
In recent years, with the rapid development of deep learning, students at home and abroad diagnose diseases through computer simulation doctors, and intelligent diagnosis indexes of the diseases are extracted and diagnosed and analyzed by using the deep learning. The current achievements of scientific research have reached expert levels of disease diagnosis and even have superior performance in some respects to experienced clinicians. Although deep learning-based disease diagnosis techniques have achieved great success, these deep learning-based diagnosis methods rely on medical image data of a large number of real label samples, and it is very difficult to acquire a large number of clinical medical image data with real label samples. The most common approach to address this deficiency is regularization, which is the dominant regularization at present, since its globally optimal solution can be computed efficiently. However, the L1 norm has no bias, which prevents consistency in variable selection. To ensure statistical performance of variable selection, strong non-representable conditions are required. In addition, the acquired clinical medical image data may have quality problems such as data loss due to imaging apparatuses, individual differences, and the like. The data is usually removed by means of data cleaning and the like to ensure standardization of the data, which certainly wastes a large amount of clinical data and is not in line with the actual clinical diagnosis condition.
The drawbacks of the prior art, analyzed, were mainly:
1) The real label data volume of clinical medicine is very limited, and when the dimension P is far greater than the sample volume, the over fitting phenomenon can occur when the model parameters are solved. To prevent overfitting, a certain regularization approach is typically used to limit the parameter space to a certain range. L1 norm regularization is a relatively common regularization method. However, L1 norm regularization is unbiased, which prevents consistency in variable selection. To ensure statistical performance of variable selection, strong non-representable conditions are required.
2) Because of the quality problem of the acquired data and the limited real tag data, the accuracy of the probability value of the real standard generated by predicting the sample data by the network model is not high.
3) The network architecture commonly used at present is based on a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network does not fully utilize space information.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a three-dimensional knowledge diagnosis model construction method and a system, and is a novel technical scheme for realizing efficient and intelligent diagnosis.
According to a first aspect of the present invention, a method of constructing a three-dimensional knowledge diagnostic model is provided. The method comprises the following steps:
creating a training set based on the known medical image data, the training set comprisingAndtwo parts, wherein X m Representing the medical image data of the mth example, Y m Label, X representing mth example medical image data i Representing the i-th example of unlabeled medical image data, M and N being the number of corresponding samples respectively;
training the constructed three-dimensional knowledge diagnosis model based on the neural network with the set loss function as the target, wherein X m ，Y m Input and label of supervised training as main branch of the three-dimensional knowledge diagnostic model, X i As input to the unsupervised training of the main branch of the three-dimensional knowledge diagnostic model,as input for unsupervised training of auxiliary branches of the three-dimensional knowledge diagnostic model,/for diagnosis of a disease>Is to X i And carrying out data after various disturbances.
According to a second aspect of the present invention, a three-dimensional knowledge diagnostic model building system is provided. The system comprises:
data acquisitionTaking a unit: for creating a training set based on known medical image data, the training set comprisingAnd->Two parts, wherein X m Representing the medical image data of the mth example, Y m Label, X representing mth example medical image data i Representing the i-th example of unlabeled medical image data, M and N being the number of corresponding samples respectively;
model training unit: for training a three-dimensional knowledge diagnosis model based on a neural network and built by taking a set loss function as a target, wherein X is m ，Y m Input and label of supervised training as main branch of the three-dimensional knowledge diagnostic model, X i As input to the unsupervised training of the main branch of the three-dimensional knowledge diagnostic model,as input for unsupervised training of auxiliary branches of the three-dimensional knowledge diagnostic model,/for diagnosis of a disease>Is to X i And carrying out data after various disturbances.
According to a third aspect of the present invention, a medical image recognition system is provided. The system comprises:
a data acquisition unit: the method comprises the steps of acquiring medical image data to be detected;
an image recognition unit: and the medical image data are input into a main branch of the three-dimensional knowledge diagnosis model constructed by the invention to obtain a classification result.
Compared with the prior art, the method has the advantages that the sample data are input into the three-dimensional knowledge diagnosis model based on the convolutional neural network, and the probability value of the real standard is generated through prediction by the network model. Then, the probability value of the initial prediction is optimized, so that the difficulty of high-dimensional learning caused by limited real tag data is solved, and the recognition efficiency and recognition accuracy of the medical image are improved.
Other features of the present invention and its advantages will become apparent from the following detailed description of exemplary embodiments of the invention, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.
FIG. 1 is a flow chart of a method of three-dimensional knowledge diagnostic model construction, in accordance with an embodiment of the invention;
FIG. 2 is a specific example of a three-dimensional knowledge diagnostic model building process, in accordance with an embodiment of the invention;
FIG. 3 is a schematic diagram of a three-dimensional knowledge diagnostic model based on a convolutional neural network, in accordance with an embodiment of the invention;
FIG. 4 is a schematic diagram of an optimization process based on maximum expectation theory according to one embodiment of the invention.
Detailed Description
Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.
The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the invention, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
Referring to fig. 1 and 2, the provided three-dimensional knowledge diagnosis model construction method includes the following steps:
step S110, acquiring medical image data.
In this step, known medical image data is acquired for subsequent training. For example, medical image data containing tumor and normal expert labeling and non-expert labeled medical image data may be collected, and the image data may be labeled as normal, cancer, or other types.
Step S120, a training set and a test set are established based on the medical image data.
A dataset is established from the collected medical image data, e.g., the dataset is divided into a training set and a testing set. The training set comprisesAnd->Two parts, where m+n=k, X m The method comprises the steps of obtaining medical image data of an mth example; y is Y m A label of the mth medical image data marked by an expert; x is X i The obtained i-th example of the label-free medical image data. Test set is->
And step S130, constructing a three-dimensional knowledge diagnosis model, and training by using a training set to obtain a classification probability value of the input data.
For example, a three-dimensional knowledge diagnostic model based on a convolutional neural network is constructed, as shown in fig. 3, which includes a main branch and an auxiliary branch from the overall structure, functionally including a feature extraction function (i.e., a three-dimensional convolutional neural network feature extractor) and a classification function (i.e., a full connection layer & softmax), wherein the main branch and the auxiliary branch share the three-dimensional convolutional neural network feature extractor.
Medical image data X in training set m ，Y m ，X i A kind of electronic devicej=1, … p, the input model performs network training, and the probability value is obtained through softmax. Wherein X is m ，Y m Input and label of supervised training as main branch of model, X i Input of unsupervised training as main branch of model, < ->Is to X i The data after various perturbations are performed as input for unsupervised training of the auxiliary branches of the model. Herein, X is m Obtaining probability value through network and marking the probability value as p m ，X i Obtaining probability value through network and marking the probability value as p i ，/>The probability value obtained through the network is marked as +.>
In a preferred embodiment, step S130 specifically includes:
step S131: a three-dimensional knowledge diagnostic model, or referred to as a three-dimensional convolutional network, is designed based on acceptance v 3.
For example, a three-dimensional knowledge diagnostic model designed based on existing acceptance v3 includes 5 3D convolutional layers, 9 block structures, 2 3D pooling layers, one fully connected layer and one softmax. The 3D convolutional layer and block structure are used for feature extraction, the pooling layer is used for data compression, and either maximum pooling or average pooling can be employed. The output channel is set to 2, e.g., normal and cancer, corresponding to the constructed training set.
Step S132: medical image data X m ，X i ，And inputting a model, and acquiring the characteristics of the medical image through the 3D convolution layer and the block structure.
Step S133: x is to be m ，X i And (3) performing feature attribute combination through the full connection layer of the feature input main branch obtained in the step S132. Will beAnd (3) performing feature attribute combination through the full connection layer of the feature input auxiliary branch obtained in the step S132.
Step S134: three-dimensional knowledge diagnosis classifier constructed by feature input after full connection layer, and predicted probability value p is obtained through softmax m ，p i And
the output result of the jth auxiliary branch of the ith image corresponds to the diagnosis result of the jth doctor of the ith image.
In this step S130, a three-dimensional convolutional neural network structure having a branched structure is constructed, which can simulate a plurality of doctors. In addition, the data with labels, the data without labels and the data after disturbance are learned, and the semi-supervised learning mode can reduce data labeling and improve the prediction accuracy of learning.
In step S140, the obtained probability value is optimized by using the maximum expected theory in the training process, so as to obtain a more accurate predicted probability value.
For example, the initial probability value p acquired in step S130 m ，p i ，Inputting an aggregation layer (representing a layer optimized by utilizing the maximum expected theory), and acquiring a probability value mu by the aggregation layer through a maximum voting method i Mu using maximum expected theory i Proceeding withOptimizing and obtaining more accurate predicted value mu' i . The contents of step S140 will be described in more detail below.
And step S150, solving parameters of the three-dimensional knowledge diagnosis model by using the set loss function and the propagation mechanism in the training process to obtain the optimized three-dimensional knowledge diagnosis model.
Preferably, the aggregation layer loss function is combined with the supervised loss function and the folded concave regularized loss function is reversely transmitted, model parameters are solved through chain rule derivation and reverse transmission, and an optimized three-dimensional knowledge diagnosis model is obtained.
For example, Y m As a model supervised training label, the model supervised training label is used for obtaining a predicted value p in the step S130 m A supervised loss function calculation was performed, expressed as:
wherein J (w, b) is a supervised class loss function, w m ,b m Weights and offsets. L (mu) i ,p i ) To overcome the high-dimensional learning problem, to aggregate the loss function of the layer, it is preferable to add a fold concave regularization to the loss function, the total loss function expressed as:
P λ (. Cndot.) is a fold concave regularization penalty function,a, lambda is the trimming parameter.
In this embodiment, by adding a folded concave regularization penalty function as a strong irreducible condition to the loss function, the statistical performance of variable selection is ensured, training errors and model complexity can be balanced, and overfitting is avoided. In addition, the back propagation algorithm can conveniently calculate the derivative of the loss function on each parameter, and the obtained derivative is used for the gradient descent method to perform model training optimization.
And (3) training the steps S130-S150 until the model converges, so as to obtain a trained three-dimensional knowledge diagnosis model or an optimized three-dimensional knowledge diagnosis model.
Step S160, evaluating the trained three-dimensional knowledge diagnostic model using the test set.
To evaluate training effects, the test set may be further populatedAnd inputting a trained three-dimensional knowledge diagnosis model to evaluate the prediction accuracy of the model.
In a preferred embodiment, as shown in connection with fig. 4, the step S140 includes:
step S141: determining initial judgment probability value mu by maximum voting method i By mu i Calculation of initial sensitivity alpha j And specificity beta j (where j.epsilon.P is the jth physician).
Wherein the method comprises the steps of
Step S142: calculation condition expectations (E-step):
where D is the set of observations and, g is an implicit variable (true tag value), -j>Is the currently estimated parameter.
Step S143: deducing mu using Bayesian theory i And alpha j And beta j The relationship is expressed as:
step S144: sensitivity and specificity (M-step) were calculated.
For example, μ obtained by the given observation set D and step S143 i Deriving the expected value, and setting the derivative to 0 to calculate alpha j And beta j Expressed as:
step S145: binding sensitivity alpha j And specificity beta j Parameter learning softmax classifier, obtaining predicted probability value μ' i 。
In one embodiment, the loss function of the aggregate layer is defined as:
further, after obtaining the trained three-dimensional knowledge diagnostic model, the medical image data to be detected is taken as input, for example, input to the main branch, and a corresponding output, for example, belonging to a normal or cancer, can be obtained, and the output result can be used for clinical indication.
Correspondingly, the invention also provides a three-dimensional knowledge diagnosis model construction system and a medical image recognition system, which are used for realizing one aspect or more aspects of the method.
For example, a medical image recognition system includes: a data acquisition unit for acquiring medical image data to be detected; and the image recognition unit is used for inputting the medical image data into a main branch of the optimized three-dimensional knowledge diagnosis model so as to obtain a classification result.
For example, a three-dimensional knowledge diagnostic model building system includes: a data acquisition unit for creating a training set based on the known medical image data, the training set comprisingAnd->Two parts, wherein X m Representing the medical image data of the mth example, Y m Label, X representing mth example medical image data i Representing the i-th example of unlabeled medical image data, M and N being the number of corresponding samples respectively; a model training unit for training the constructed three-dimensional knowledge diagnosis model based on the neural network with the set loss function as a target, wherein X m ，Y m Input and label of supervised training as main branch of the three-dimensional knowledge diagnostic model, X i As input for unsupervised training of the main branch of the three-dimensional knowledge diagnostic model,/for example>As input for unsupervised training of auxiliary branches of the three-dimensional knowledge diagnostic model,/for diagnosis of a disease>Is to X i And carrying out data after various disturbances.
In conclusion, the three-dimensional knowledge diagnosis model provided by the invention effectively solves the difficult problem of high-dimensional learning caused by limited real tag data of the diagnosis model, and simultaneously overcomes the defect of low accuracy of the network model predicted value caused by quality problems such as data loss by adopting a maximum expected theory and a semi-supervision algorithm. Moreover, the knowledge diagnosis model is optimized based on the folding concave regularization, and compared with the L1 norm regularization, the folding concave regularization can obtain better statistical performance by using fewer theoretical rules. Compared with a common deep learning framework, the method adopts the maximum expected theory to simulate doctor consultation to optimize judgment accuracy, thereby solving the defect that the initial predicted value is not accurate enough caused by quality problems such as data loss and the like. In addition, in order to overcome the problem of insufficient two-dimensional space information, all calculation processes of the constructed model adopt a three-dimensional convolutional neural network structure, so that efficient and intelligent diagnosis is realized.
It should be noted that the present invention is applicable to analysis and recognition of various medical images, and the recognition result is not limited to tumor, cancer, and the like. Those skilled in the art may make appropriate modifications or alterations to the above described embodiments, such as designing more or fewer 3D convolutional layers, block structures, or employing other classification methods than softmax, without departing from the spirit and scope of the invention.
The present invention may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present invention.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for carrying out operations of the present invention may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information for computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.
The foregoing description of embodiments of the invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.
Claims (8)
1. A three-dimensional knowledge diagnosis model construction method comprises the following steps:
creating a training set based on the known medical image data, the training set comprisingAnd->Two parts, wherein X m Representing the medical image data of the mth example, Y m Label, X representing mth example medical image data i Representing the i-th example of unlabeled medical image data, M and N being the number of corresponding samples respectively;
neural network-based constructed for training with set loss function as targetThree-dimensional knowledge diagnostic model of a complex, wherein X m ，Y m Input and label of supervised training as main branch of the three-dimensional knowledge diagnostic model, X i As input to the unsupervised training of the main branch of the three-dimensional knowledge diagnostic model,as input for unsupervised training of auxiliary branches of the three-dimensional knowledge diagnostic model,/for diagnosis of a disease>Is to X i Performing data after various disturbance;
the three-dimensional knowledge diagnosis model based on the neural network and constructed by training with the set loss function as a target comprises the following steps:
designing a three-dimensional knowledge diagnosis model, wherein a main branch and an auxiliary branch of the three-dimensional knowledge diagnosis model share the same feature extractor, and the main branch and the auxiliary branch respectively comprise an independent full-connection layer and a classifier;
image data X of medical image m ，X i ，Inputting the three-dimensional knowledge diagnosis model, and extracting the characteristics of the medical image through the characteristic extractor;
extracting X m ，X i Inputting the corresponding features into the full connection layer of the main branch of the three-dimensional knowledge diagnosis model to perform feature attribute combination, and extracting the featuresInputting the corresponding features into a full connection layer of an auxiliary branch of the three-dimensional knowledge diagnosis model to perform feature attribute combination;
inputting the features after the full connection layer into corresponding classifiers to obtain a predicted probability value p m ，p i Andwherein (1)>Representing the output result of the jth auxiliary branch of the ith image, p m X represents m Corresponding probability value, p i X represents i Corresponding probability value>Representation->A corresponding probability value;
the acquired initial probability value p m ，p i ，Inputting the aggregation layer to obtain initial judgment probability value mu i And optimizing to obtain more accurate predicted value mu' i ；
The set loss function is used as a target, and the reverse propagation is derived through a chain rule, so that an optimized three-dimensional knowledge diagnosis model is obtained;
wherein the acquired initial probability value p m ，p i ，Inputting the aggregation layer to obtain initial judgment probability value mu i And optimizing to obtain more accurate predicted value mu' i Comprising the following steps:
determining initial judgment probability value mu by maximum voting method i By mu i Calculation of initial sensitivity alpha j And specificity beta j Where j ε P is considered the jth physician:
wherein,
the calculation conditions are as follows:
wherein D is the set of observations, g is an implicit variable, ">Is the currently estimated parameter;
inferring μ using Bayesian theory i And alpha j 、β j Is expressed as:
by a given observation set D and the μ obtained i Deriving the expected value, and making the derivative be 0 to calculate alpha j And beta j Expressed as:
binding sensitivity alpha j And specificity beta j Parameter learning classifier, obtaining predicted value mu' i 。
2. The method of claim 1, wherein targeting the set loss function, deriving back propagation by chain law, obtaining an optimized three-dimensional knowledge diagnostic model comprises:
and combining the aggregation layer loss function with the supervised loss function and the folded concave regularized loss function back transmission, solving the parameters of the three-dimensional knowledge diagnosis model through the chain rule derivation back transmission, and obtaining the optimized three-dimensional knowledge diagnosis model.
3. The method of claim 2, wherein the loss function is set to:
P λ (. Cndot.) is a fold concave regularization penalty function,a, lambda is the tuning parameter, J (w, b) is the supervised class loss function,/->w m ,b m For weight and bias, L (μ) i ,p i ) Is the loss function of the aggregate layer.
4. A method according to claim 3, wherein the loss function of the aggregation layer is expressed as:
5. the method of claim 1, wherein the three-dimensional knowledge diagnostic model is designed based on acceptance v3, wherein the feature extractor comprises a plurality of 3D convolutional layers and a plurality of block structures, and wherein the classifier is a softmax classifier.
6. A three-dimensional knowledge diagnostic model building system, comprising:
a data acquisition unit: for creating a training set based on known medical image data, the training set comprisingAnd->Two parts, wherein X m Representing the medical image data of the mth example, Y m Label, X representing mth example medical image data i Representing the i-th example of unlabeled medical image data, M and N being the number of corresponding samples respectively;
model training unit: for training a three-dimensional knowledge diagnosis model based on a neural network and built by taking a set loss function as a target, wherein X is m ，Y m Input and label of supervised training as main branch of the three-dimensional knowledge diagnostic model, X i As input to the unsupervised training of the main branch of the three-dimensional knowledge diagnostic model,as input for unsupervised training of auxiliary branches of the three-dimensional knowledge diagnostic model,/for diagnosis of a disease>Is to X i Performing data after various disturbance;
the three-dimensional knowledge diagnosis model based on the neural network and constructed by training with the set loss function as a target comprises the following steps:
designing a three-dimensional knowledge diagnosis model, wherein a main branch and an auxiliary branch of the three-dimensional knowledge diagnosis model share the same feature extractor, and the main branch and the auxiliary branch respectively comprise an independent full-connection layer and a classifier;
image data X of medical image m ，X i ，Inputting the three-dimensional knowledge diagnosis model, and extracting the characteristics of the medical image through the characteristic extractor;
extracting X m ，X i Inputting the corresponding features into the full connection layer of the main branch of the three-dimensional knowledge diagnosis model to perform feature attribute combination, and extracting the featuresInputting the corresponding features into a full connection layer of an auxiliary branch of the three-dimensional knowledge diagnosis model to perform feature attribute combination;
inputting the features after the full connection layer into corresponding classifiers to obtain a predicted probability value p m ，p i Andwherein (1)>Representing the output result of the jth auxiliary branch of the ith image, p m X represents m Corresponding probability value, p i X represents i Corresponding probability value>Representation->A corresponding probability value;
the acquired initial probability value p m ，p i ，Inputting the aggregation layer to obtain initial judgment probability value mu i And optimizing to obtain more accurate predicted value mu' i ；
The set loss function is used as a target, and the reverse propagation is derived through a chain rule, so that an optimized three-dimensional knowledge diagnosis model is obtained;
wherein the acquired initial probability value p m ，p i ，Inputting the aggregation layer to obtain initial judgment probability value mu i And optimizing to obtain more accurate predicted value mu' i Comprising the following steps:
determining initial judgment probability value mu by maximum voting method i By mu i Calculation of initial sensitivity alpha j And specificity beta j Where j ε P is considered the jth physician:
wherein,
the calculation conditions are as follows:
wherein D is the set of observations, g is an implicit variable, ">Is the currently estimated parameter;
inferring μ using Bayesian theory i And alpha j 、β j Is expressed as:
by a given observation set D and the μ obtained i Deriving the expected value, and making the derivative be 0 to calculate alpha j And beta j Expressed as:
binding sensitivity alpha j And specificity beta j Parameter learning classifier, obtaining predicted value mu' i 。
7. A medical image recognition system, comprising:
a data acquisition unit: the method comprises the steps of acquiring medical image data to be detected;
an image recognition unit: a main branch for inputting the medical image data into a three-dimensional knowledge diagnostic model constructed according to the method of any one of claims 1 to 5, obtaining a classification result.
8. A computer readable storage medium having stored thereon a computer program, wherein the program when executed by a processor realizes the steps of the method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011015038.8A CN112086174B (en) | 2020-09-24 | 2020-09-24 | Three-dimensional knowledge diagnosis model construction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011015038.8A CN112086174B (en) | 2020-09-24 | 2020-09-24 | Three-dimensional knowledge diagnosis model construction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112086174A CN112086174A (en) | 2020-12-15 |
| CN112086174B true CN112086174B (en) | 2023-12-12 |
Family
ID=73738700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011015038.8A Active CN112086174B (en) | 2020-09-24 | 2020-09-24 | Three-dimensional knowledge diagnosis model construction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112086174B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112734906B (en) * | 2020-12-30 | 2022-08-19 | 华东师范大学 | Three-dimensional reconstruction method of ultrasonic or CT medical image based on knowledge distillation |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109034205A (en) * | 2018-06-29 | 2018-12-18 | 西安交通大学 | Image classification method based on the semi-supervised deep learning of direct-push |
| KR20190117969A (en) * | 2018-04-09 | 2019-10-17 | 주식회사 뷰노 | Method for semi supervised reinforcement learning using data with label and data without label together and apparatus using the same |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA3061717A1 (en) * | 2018-11-16 | 2020-05-16 | Royal Bank Of Canada | System and method for a convolutional neural network for multi-label classification with partial annotations |
-
2020
- 2020-09-24 CN CN202011015038.8A patent/CN112086174B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20190117969A (en) * | 2018-04-09 | 2019-10-17 | 주식회사 뷰노 | Method for semi supervised reinforcement learning using data with label and data without label together and apparatus using the same |
| CN109034205A (en) * | 2018-06-29 | 2018-12-18 | 西安交通大学 | Image classification method based on the semi-supervised deep learning of direct-push |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112086174A (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Castiglioni et al. | AI applications to medical images: From machine learning to deep learning | |
| Neal Joshua et al. | 3D CNN with Visual Insights for Early Detection of Lung Cancer Using Gradient‐Weighted Class Activation | |
| Shanmugam et al. | Alzheimer’s disease classification using pre-trained deep networks | |
| Khan et al. | Intelligent fusion-assisted skin lesion localization and classification for smart healthcare | |
| Wu et al. | Cascaded fully convolutional networks for automatic prenatal ultrasound image segmentation | |
| CN111488921B (en) | Intelligent analysis system and method for panoramic digital pathological image | |
| CN113421652A (en) | Method for analyzing medical data, method for training model and analyzer | |
| Althubiti et al. | Ensemble learning framework with GLCM texture extraction for early detection of lung cancer on CT images | |
| Wankhade et al. | A novel hybrid deep learning method for early detection of lung cancer using neural networks | |
| Li et al. | Hierarchical deep network with uncertainty-aware semi-supervised learning for vessel segmentation | |
| Bashir-Gonbadi et al. | Brain tumor classification using deep convolutional autoencoder-based neural network: Multi-task approach | |
| Barbano et al. | Uncertainty quantification in medical image synthesis | |
| Lukyanenko et al. | Developmental stage classification of embryos using two-stream neural network with linear-chain conditional random field | |
| CN114580501A (en) | Bone marrow cell classification method, system, computer device and storage medium | |
| CN112086174B (en) | Three-dimensional knowledge diagnosis model construction method and system | |
| Zhong et al. | Dilated and soft attention‐guided convolutional neural network for breast cancer histology images classification | |
| Zheng et al. | Integrating semi-supervised label propagation and random forests for multi-atlas based hippocampus segmentation | |
| Jin et al. | Metadata and image features co-aware personalized federated learning for smart healthcare | |
| CN116759076A (en) | Unsupervised disease diagnosis method and system based on medical image | |
| Marini et al. | Semi-supervised learning with a teacher-student paradigm for histopathology classification: a resource to face data heterogeneity and lack of local annotations | |
| WO2024006572A1 (en) | Apparatus and a method for detecting associations among datasets of different types | |
| Sajiv et al. | Predicting Breast Cancer Risk from Histopathology Images using Hybrid Deep Learning Classifier | |
| Zhang et al. | Semi‐supervised graph convolutional networks for the domain adaptive recognition of thyroid nodules in cross‐device ultrasound images | |
| Mir et al. | Artificial intelligence-based techniques for analysis of body cavity fluids: a review | |
| Hao et al. | Radiographs and texts fusion learning based deep networks for skeletal bone age assessment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |