CN110310740A - Based on see a doctor again information forecasting method and the system for intersecting attention neural network - Google Patents
Based on see a doctor again information forecasting method and the system for intersecting attention neural network Download PDFInfo
- Publication number
- CN110310740A CN110310740A CN201910300329.2A CN201910300329A CN110310740A CN 110310740 A CN110310740 A CN 110310740A CN 201910300329 A CN201910300329 A CN 201910300329A CN 110310740 A CN110310740 A CN 110310740A
- Authority
- CN
- China
- Prior art keywords
- information
- patient
- treatment information
- module
- diagnostic message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
Abstract
The present disclosure proposes based on see a doctor again information forecasting method and the system for paying attention to neural network is intersected, the historical electronic health records data of patient are obtained；The primary medical treatment information record of the time flag sequence of the multivariable of each patient is split as diagnostic message and treatment information, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；Diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding diagnostic message hidden state, and the treatment information after dimensionality reduction is handled using two way blocks obtains corresponding treatment Information hiding state；Historical diagnostic information and historical therapeutic information are integrated into the context vector that can indicate patient's current state using intersection attention mechanism；After the medical diagnosis on disease information and treatment information for having obtained patient, by context vector, this two parts information is merged into the expression vector for indicating patient's Global Information by the way of connection；It will indicate that vector is put into final output layer to predict medical treatment information.
Description
Technical field
This disclosure relates to technical field of information processing, more particularly to based on the information of seeing a doctor again for intersecting attention neural network
Prediction technique and system.
Background technique
The health and fitness information of analysis patient is to help people and prevents disease, guiding treatment method as early as possible.Therefore, pass through trouble
Medical treatment coding (the packet of historical electronic health records (Electronic Health Record, EHR) the data prediction of person next time
Include disease and therapeutic modality) it is a critically important task.How to the timeliness of continuous EHR data and high-dimensional to model
And interpretation prediction is the result is that complete the critical problem of this task.
Inventor has found that existing method is by using Recognition with Recurrent Neural Network (Recurrent Neural under study for action
Network, RNN) it is solved these problems to EHR data modeling and using attention mechanism offer interpretation.Previous mould
Type will treat and diagnostic message is mixed into medical treatment information, and two kinds of information are indiscriminate as input, it means that diagnose and control
The content for the treatment of is of a sort.
In practice, the information of diagnosis is disease, and the information for the treatment of is drug etc., and the two is different content.And disease
The validity of people's therapeutic modality is usually to judge by the way that whether disease next time improves, and the severity of disease can also lead to
The difference for crossing therapeutic modality is embodied.If merged, two parts information can be interfered with each other, cannot analyze well and
Utilize this correlation.In addition, may also include the therapeutic agent for the disease that certain patients once suffered from treatment information.
Hybrid diagnosis and treatment information can interfere the judgement to this disease of patient, influence prediction effect.
Summary of the invention
The purpose of this specification embodiment is to provide based on the information forecasting method of seeing a doctor again for intersecting attention neural network,
The disclosure separately handles diagnostic message and drug information, ensure that the integrality and independence of two parts information, utilizes two
The correlation of point information improves the accuracy for information prediction result of seeing a doctor again.
This specification embodiment is provided based on the information forecasting method of seeing a doctor again for paying attention to neural network is intersected, by following
Technical solution is realized:
Include:
The historical electronic health records data for obtaining patient, by the historical electronic health records tables of data of each patient
It is shown as the time flag sequence of multivariable；
The primary medical treatment information record of the time flag sequence of the multivariable of each patient is split as diagnostic message and is controlled
Information is treated, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding diagnostic message hidden state, using double
Corresponding treatment Information hiding state is obtained to the treatment information after Processing with Neural Network dimensionality reduction；
Being integrated into historical diagnostic information and historical therapeutic information using intersection attention mechanism can indicate that patient is current
The context vector of state；
After the medical diagnosis on disease information that has obtained patient and treatment information, by the way of connection by context vector this
Two parts information is merged into the expression vector of expression patient's Global Information；
It will indicate that vector is put into final output layer to predict medical treatment information.
This specification embodiment is provided based on the information prediction system of seeing a doctor again for paying attention to neural network is intersected, by following
Technical solution is realized:
Include:
Data Dimensionality Reduction layer, is configured as: obtaining the historical electronic health records data of patient, will go through described in each patient
History electric health record data are expressed as the time flag sequence of multivariable；
The primary medical treatment information record of the time flag sequence of the multivariable of each patient is split as diagnostic message and is controlled
Information is treated, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Two-way RNN process layer, is configured as: the diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding
Diagnostic message hidden state, the treatment information after dimensionality reduction is handled using two way blocks obtain corresponding treatment Information hiding shape
State；
Intersect attention mechanism layer, be configured as: being believed historical diagnostic information and historical therapeutic using attention mechanism is intersected
Breath is integrated into the context vector that can indicate patient's current state；
Pooling information layer, is configured as: after the medical diagnosis on disease information and treatment information for having obtained patient, using connection
Mode context vector this two parts information is merged into the expression vector of expression patient's Global Information；
Prediction interval is configured as: will indicate that vector is put into final output layer to predict medical treatment information.
This specification embodiment is provided based on the information prediction device of seeing a doctor again for paying attention to neural network is intersected, by following
Technical solution is realized:
Include: data acquisition module, dimensionality reduction module, two-way RNN module, intersect attention mechanism module, information merging module,
Prediction module；
Data acquisition module is configured as: the historical electronic health records data of patient are obtained, it will be described in each patient
Historical electronic health records data are expressed as the time flag sequence of multivariable；
Dimensionality reduction module, is configured as: the primary medical treatment information of the time flag sequence of the multivariable of each patient is recorded
It is split as diagnostic message and treatment information, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Two-way RNN module, is configured as: the diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding examine
Disconnected Information hiding state, the treatment information after dimensionality reduction is handled using two way blocks obtain corresponding treatment Information hiding shape
State；
Intersect attention mechanism module, be configured as: utilizing and intersect attention mechanism for historical diagnostic information and historical therapeutic
Information is integrated into the context vector that can indicate patient's current state；
Information merging module, is configured as: after the medical diagnosis on disease information and treatment information for having obtained patient, using even
This two parts information of context vector is merged into the expression vector for indicating patient's Global Information by the mode connect；
Prediction module is configured as: will indicate that vector is put into final output layer to predict medical treatment information.
Compared with prior art, the beneficial effect of the disclosure is:
The disclosure can accurately predict the medical treatment information next time of patient, and have good interpretation.
The disclosure can separately be handled diagnostic message and drug information, ensure that the integrality and independence of two parts information
Property, forecasting accuracy is improved using the correlation of two parts information.
Detailed description of the invention
The Figure of description for constituting a part of this disclosure is used to provide further understanding of the disclosure, and the disclosure is shown
Meaning property embodiment and its explanation do not constitute the improper restriction to the disclosure for explaining the disclosure.
Fig. 1 is that the higher-order logic of embodiment of the present disclosure summarizes figure；
Fig. 2 is the structure chart of the RNNd and RNNt of embodiment of the present disclosure；
Fig. 3 is the internal LSTM cellular construction figure of each of embodiment of the present disclosure.
Specific embodiment
It is noted that following detailed description is all illustrative, it is intended to provide further instruction to the disclosure.Unless another
It indicates, all technical and scientific terms used herein has usual with disclosure person of an ordinary skill in the technical field
The identical meanings of understanding.
It should be noted that term used herein above is merely to describe specific embodiment, and be not intended to restricted root
According to the illustrative embodiments of the disclosure.As used herein, unless the context clearly indicates otherwise, otherwise singular
Also it is intended to include plural form, additionally, it should be understood that, when in the present specification using term "comprising" and/or " packet
Include " when, indicate existing characteristics, step, operation, device, component and/or their combination.
Diagnosing and treating is separated to the change of illness state that can preferably hold patient, analyzes therapeutic process.It can also assist simultaneously
Doctor is helped to find out diagnosis important in historical information and effective therapeutic modality.Therefore, diagnosing and treating is separated into preferably handle
The change of illness state of patient is held, therapeutic process is analyzed, assists a physician and finds out diagnosis important in historical information and effective treatment side
Formula becomes the urgent problem of this field one.
Examples of implementation one
The examples of implementation are directed to the change of illness state for separating diagnosing and treating and preferably holding patient, analyze therapeutic process,
Assist a physician find out important diagnosis and effective therapeutic modality in historical information propose it is a kind of based on intersecting attention mechanism
Medical information prediction technique, specifically:
First is that data will be handled, the data after the dimensionality reduction needed.
Second is that being performed corresponding processing by two-way RNN to data.
Third is that the data obtained after processing to be carried out to the result for integrating to the end.
Specifically, including:
The historical electronic health records data for obtaining patient, by the historical electronic health records tables of data of each patient
It is shown as the time flag sequence of multivariable；
The primary medical treatment information record of the time flag sequence of the multivariable of each patient is split as diagnostic message and is controlled
Information is treated, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding diagnostic message hidden state, using double
Corresponding treatment Information hiding state is obtained to the treatment information after Processing with Neural Network dimensionality reduction；
Being integrated into historical diagnostic information and historical therapeutic information using intersection attention mechanism can indicate that patient is current
The context vector of state；
After the medical diagnosis on disease information that has obtained patient and treatment information, by the way of connection by context vector this
Two parts information is merged into the expression vector of expression patient's Global Information；
It will indicate that vector is put into final output layer to predict medical treatment information.
About data collected, the history medical treatment information of each patient can be expressed as the medical treatment being sequentially arranged
Information sequence.Assuming that Cd is the quantity of exclusive diagnosis code, Ct is the quantity of sole therapy code.For one there is T (n) to visit
It asks the patient of record, can indicate x by a series of access1, x2..., xT(n).Every time access xi can be divided into two to
Amount, diaiAnd trei。diaiAnd treiIt is binary vector (diai∈ { 0,1 }|Cd|, trei∈ { 0,1 }|Ct|).If VTInclude
Diagnostic code cdi, i-th bit diagnostic code is 1；If VTComprising treating code ctj, it is 1 that code is treated in jth position.
In the examples of implementation, in Data Dimensionality Reduction, x is recorded respectively for primary medical treatment informationiDiagnostic message diaiWith
Treat information treiDimensionality reduction is expressed as d by the following methodi∈RmAnd ti∈Rm, m is that the state being manually set indicates the big of vector
It is small.
Specifically,
di=ReLU (Wddiai+bd)
ti=ReLU (Wttrei+bt)。
Due to the information record x that once sees a doctoriIn have two parts relatively independent information diAnd ti, so using RNNdAnd RNNt
Two parts information is handled respectively.RNNdHandle diagnostic message d1To dn, obtain corresponding diagnostic message hidden state
{h1..., hn}(hi∈Rp), p is the dimension of hidden state).RNNtProcessing treatment information t1To tn, obtain corresponding treatment letter
Cease hidden state { g1..., gn}(gi∈Rp)。
Historical diagnostic information and historical therapeutic information are integrated into the context vector that can indicate patient's current stateWithThe process including the following steps:
(1) pass through hidden state h respectivelyiAnd giCalculate weight:
(2) softmax function is used to obtained weight vectors α and β, specific as follows:
(3) by weight vectors cross action, i.e.,As treatment informationWeight vectors,As diagnostic messageWeight vectors.Finally summation obtains patient's current disease state respectivelyAnd therapeutic stateCalculation method is as follows:
When information merges, after the medical diagnosis on disease information and treatment information for having obtained patient, using connection
(concatenation) this two parts information is merged into the expression vector p of expression patient's Global Information by mode.It uses
One articulamentum obtains integrated information to combine two parts information；
Specifically, its calculation formula is:
When prediction: the final expression p of patient being put into final output layer to predict n+1 medical treatment information.We adopt
Use softmax as the activation primitive of output layer.Calculation formula is as follows:
Finally, calculating the difference between the model prediction result y^ of all patients and legitimate reading y using cross entropy.
Examples of implementation two
The examples of implementation are disclosed based on the information prediction system of seeing a doctor again for paying attention to neural network is intersected, referring to 1 institute of attached drawing
Show, comprising: Data Dimensionality Reduction layer 101, intersects attention mechanism layer 105, pooling information layer 106, prediction interval at two-way RNN process layer 103
107。
Data Dimensionality Reduction layer 101: for the information record x that once sees a doctori∈ { 0,1 }|C|It can be split as diagnostic message diaiWith
Treat information trei, i.e. xi={ diai；trei}.Respectively to diaiAnd treiDimensionality reduction is expressed as d by the following methodi∈RmAnd ti
∈Rm:
di=ReLU (Wddiai+bd)
ti=ReLU (Wttrei+bt)
Two-way RNN process layer 103: due to the information record x that once sees a doctoriIn have two parts relatively independent information diAnd ti,
So we use RNNdAnd RNNtTwo parts information is handled respectively.RNNdHandle diagnostic message d1To dn, corresponded to
Diagnostic message hidden state { h1..., hn}(hi∈Rp), p is the dimension of hidden state).RNNtProcessing treatment information t1It arrives
tn, obtain corresponding treatment Information hiding state { g1..., gn}(gi∈Rp)。
Intersect attention mechanism layer 105: being to be integrated into historical diagnostic information and historical therapeutic information to indicate that patient works as
The context vector of preceding stateWithProcess including the following steps:
Step by step 1: passing through hidden state h respectivelyiAnd giCalculate weight:
Step by step 2: softmax function is used to obtained weight vectors α and β, specific as follows:
Step by step 3: by weight vectors cross action, i.e.,As treatment informationWeight vectors,As diagnosis
InformationWeight vectors.Finally summation obtains patient's current disease state respectivelyAnd therapeutic stateCalculation method is such as
Under:
Pooling information layer 106: after the medical diagnosis on disease information and treatment information for having obtained patient, using connection
(concatenation) this two parts information is merged into the expression vector p of expression patient's Global Information by mode.It uses
One articulamentum obtains integrated information to combine two parts information；
Specifically, its calculation formula is:
The final expression p of patient is put into final output layer to predict medical treatment information next time by prediction interval 107.I
Using activation primitive of the softmax as output layer.Calculation formula is as follows:
In conclusion by the embodiment of the present disclosure, by based on the information prediction side of seeing a doctor again for intersecting attention neural network
Method realizes isolation diagnostic and treatment information, and keeps prediction more accurate using the relationship between this two parts.
Examples of implementation three
The examples of implementation disclose a kind of computer equipment, including memory, processor and storage are on a memory and can
The computer program run on a processor, which is characterized in that the processor is realized a kind of based on friendship when executing described program
The step of pitching the medical information prediction technique of attention mechanism.
This it is a kind of based on intersect attention mechanism medical information prediction technique specific steps referring to examples of implementation one, herein
No longer it is described in detail.
Examples of implementation four
The examples of implementation disclose a kind of computer readable storage medium, are stored thereon with computer program, and feature exists
In, the program realized when being executed by processor it is a kind of based on the medical information prediction technique for intersecting attention mechanism the step of.
This it is a kind of based on intersect attention mechanism medical information prediction technique specific steps referring to examples of implementation one, herein
No longer it is described in detail.
In the present embodiment, computer program product may include computer readable storage medium, containing for holding
The computer-readable program instructions of row various aspects of the disclosure.Computer readable storage medium, which can be, can keep and store
By the tangible device for the instruction that instruction execution equipment uses.Computer readable storage medium for example can be-- but it is unlimited
In-- storage device electric, magnetic storage apparatus, light storage device, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned
Any appropriate combination.
Examples of implementation four
The examples of implementation are disclosed based on the information prediction device of seeing a doctor again for intersecting attention neural network, comprising: data are adopted
Collect module, dimensionality reduction module, two-way RNN module, intersect attention mechanism module, information merging module, prediction module and loss function
Module.
In data acquisition module, the history medical treatment information of each patient can be expressed as the medical treatment being sequentially arranged letter
Cease sequence.Assuming that Cd is the quantity of exclusive diagnosis code, Ct is the quantity of sole therapy code.For one there is T (n) to access
The patient of record can indicate x by a series of access1, x2..., xT(n).Access xi can be divided into two vectors every time,
diaiAnd trei。diaiAnd treiIt is binary vector (diai∈ { 0,1 }|Cd|, trei∈ { 0,1 }|Ct|).If VTComprising examining
Division of history into periods code cdi, i-th bit diagnostic code is 1；If VTComprising treating code ctj, it is 1 that code is treated in jth position.
Dimensionality reduction module records x respectively for primary medical treatment informationiDiagnostic message diaiWith treatment information treiBy such as
Lower method dimensionality reduction is expressed as di∈RmAnd ti∈Rm, m is that the state being manually set indicates the size of vector.
Specifically,
di=ReLU (Wddiai+bd)
ti=ReLU (Wttrei+bt)
Wherein Wd∈Rm×|Cd|, Wt∈Rm×|Ct|And bd∈Rm, bt∈RmIt is the parameter for needing to learn.
Two-way RNN module, due to the information di and ti for having two parts relatively independent in the information record xi that once sees a doctor, so
Two parts information is handled respectively using RNNd and RNNt.The structure of RNNd and RNNt is as shown in Fig. 2, respectively to di and ti
Handled twice with inverse time sequencing by LSTM in chronological order.Each internal LSTM cellular construction is as shown in Figure 3.Specific place
Reason process is as follows,
It is to forget door, the working memory h of upper LSTM unit firsti-1(if it is first LSTM unit be then zero to
Amount) and current input siRespectively multiplied by weight matrix UfAnd Wf, in addition biasing bf, sigmoid (σ) activation primitive is then used, is obtained
Belong to the output between [0,1] to an element.
Forget door: fi=σ (Wfsi+Ufhi-1+bf)；
Next current input s is determinediIn which information can be input in neural metamemory.Wherein input gate iiCertainly
It is fixed that we those of will update in neural metamemory information；Candidate's memory C*It is then to merge historical trace and current information.
Input gate: ii=σ(Wisi+Uihi-1+bi)；
Candidate's memory: C*=tanh (Wcsi+Uchi-1+bc)；
It then is exactly to update Current neural metamemory.Neural metamemory C beforei-1With forgetting door fiIt is multiplied, forgetting is fallen
The information for needing to forget；Candidate is remembered into C*With input gate iiIt is multiplied, the determination information to be updated.The two sums to obtain current mind
Through metamemory Ci。
Current neural metamemory: Ci=fi*Ci-1+iiC*；
It is final we determined that LSTM unit to export what value (only have the last one LSTM unit to have in model to export,
It is only used as hidden state in remaining LSTM unit to play a role).We are by calculating out gate oiDetermine which partially has
With information, need to export.
Out gate: oi=σ (Wosi+Uohi-1+bo)；
Hidden state: hi=oi*tanh(Ci)；
Wherein Wf, Uf, bfForget the network parameter of door, Wi, Ui, biIt is the network parameter of input gate, Wo, Uo, boIt is out gate
Network parameter, all-network parameter passes through trained acquisition.
Each input siBy being spliced to obtain hidden state most for the vector expression of forward and reverse LSTM unit
Whole vector indicates.In this way, passing through RNNdHandle diagnostic message d1To dn, obtain corresponding diagnostic message hidden state { h1...,
hn}(hi∈Rp, p is the dimension of hidden state)；RNNtProcessing treatment information t1To tn, obtain corresponding treatment Information hiding state
{ g1 ..., gn } (gi ∈ Rp).Using the hidden state of two kinds of information, by intersecting attention mechanism, to obtain final patient current
State indicates.
Intersect attention mechanism module, is to be integrated into historical diagnostic information and historical therapeutic information to indicate that patient is current
The context vector of stateWithThe process of the module including the following steps:
(4) weight is calculated by hidden state hi and gi respectively:
Wherein Wα, bα, Wβ, bβNetwork parameter is belonged to, is obtained by training, αiAnd βiFor calculating final weight vector.
(5) softmax function is used to obtained vector α and β, specific as follows:
(6) by weight vectors cross action, i.e.,As treatment informationWeight vectors,As diagnostic messageWeight vectors.Finally summation obtains patient's current disease state respectivelyAnd therapeutic stateCalculation method is as follows:
Information merging module, after the medical diagnosis on disease information and treatment information for having obtained patient, using connection
(concatenation) this two parts information is merged into the expression vector p of expression patient's Global Information by mode.It uses
One articulamentum obtains integrated information to combine two parts information；
Specifically, its calculation formula is:
Wherein WconAnd bconIt is parameter to be trained, p indicates patient's Global Information
The final expression p of patient is put into final output layer to predict n+1 medical treatment information by prediction module.Using
Activation primitive of the softmax as output layer.Calculation formula is as follows:
Wherein WsAnd bsIt is parameter to be trained,Indicate model prediction result.
Loss function module is calculated between the model prediction result y^ of all patients and legitimate reading y using cross entropy
Difference, and the method by minimizing loss function is to carrying out training pattern parameter.
The disclosure is to access possible medical treatment next time by the historical information prediction of patient to the prediction of calibration-based hearing loss evaluation
Information.The interpretation of prediction model is of great significance in health care.For isolation diagnostic and treatment information, and utilize
Relationship between this two parts proposes a kind of entitled new mould based on the information prediction of seeing a doctor again for intersecting attention neural network
Type.Notice that the information forecasting method of seeing a doctor again of neural network respectively models diagnosing and treating information based on intersecting, and makes
The accuracy and interpretation of predictive information are improved with intersection attention mechanism.
It is understood that in the description of this specification, reference term " embodiment ", " another embodiment ", " other
The description of embodiment " or " first embodiment~N embodiment " etc. means specific spy described in conjunction with this embodiment or example
Sign, structure, material or feature are included at least one embodiment or example of the invention.In the present specification, to above-mentioned
The schematic representation of term may not refer to the same embodiment or example.Moreover, the specific features of description, structure, material
Person's feature can be combined in any suitable manner in any one or more of the embodiments or examples.
The foregoing is merely preferred embodiment of the present disclosure, are not limited to the disclosure, for the skill of this field
For art personnel, the disclosure can have various modifications and variations.It is all within the spirit and principle of the disclosure, it is made any to repair
Change, equivalent replacement, improvement etc., should be included within the protection scope of the disclosure.
Claims (10)
1. based on the information forecasting method of seeing a doctor again for paying attention to neural network is intersected, characterized in that include:
The historical electronic health records data for obtaining patient, the historical electronic health records data of each patient are expressed as
The time flag sequence of multivariable；
The primary medical treatment information record of the time flag sequence of the multivariable of each patient is split as diagnostic message and treatment letter
Breath, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding diagnostic message hidden state, utilizes two-way mind
Treatment information after network processes dimensionality reduction obtains corresponding treatment Information hiding state；
Patient's current state can be indicated by being integrated into historical diagnostic information and historical therapeutic information using intersection attention mechanism
Context vector；
After the medical diagnosis on disease information that has obtained patient and treatment information, by context vector this two by the way of connection
Point information is merged into the expression vector of expression patient's Global Information；
It will indicate that vector is put into final output layer to predict medical treatment information.
2. based on the information prediction system of seeing a doctor again for paying attention to neural network is intersected, characterized in that include:
Data Dimensionality Reduction layer, is configured as: the historical electronic health records data of patient is obtained, by the history electricity of each patient
Sub- health records data are expressed as the time flag sequence of multivariable；
The primary medical treatment information record of the time flag sequence of the multivariable of each patient is split as diagnostic message and treatment letter
Breath, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Two-way RNN process layer, is configured as: the diagnostic message after handling dimensionality reduction using two way blocks obtains corresponding diagnosis
Information hiding state, the treatment information after dimensionality reduction is handled using two way blocks obtain corresponding treatment Information hiding state；
Intersect attention mechanism layer, be configured as: using intersecting, attention mechanism is whole by historical diagnostic information and historical therapeutic information
It is combined into the context vector that can indicate patient's current state；
Pooling information layer, is configured as: after the medical diagnosis on disease information and treatment information for having obtained patient, using the side of connection
This two parts information of context vector is merged into the expression vector for indicating patient's Global Information by formula；
Prediction interval is configured as: will indicate that vector is put into final output layer to predict medical treatment information.
3. based on the information prediction device of seeing a doctor again for paying attention to neural network is intersected, characterized in that include: data acquisition module, drop
It ties up module, two-way RNN module, intersect attention mechanism module, information merging module, prediction module；
Data acquisition module is configured as: the historical electronic health records data of patient is obtained, by the history of each patient
Electric health record data are expressed as the time flag sequence of multivariable；
Dimensionality reduction module, is configured as: the primary medical treatment information of the time flag sequence of the multivariable of each patient being recorded and is split
For diagnostic message and information is treated, and dimensionality reduction expression is carried out to diagnostic message and treatment information respectively；
Two-way RNN module, is configured as: the diagnostic message after being handled dimensionality reduction using two way blocks is obtained corresponding diagnosis and believed
Hidden state is ceased, the treatment information after dimensionality reduction is handled using two way blocks obtains corresponding treatment Information hiding state；
Intersect attention mechanism module, be configured as: utilizing and intersect attention mechanism for historical diagnostic information and historical therapeutic information
It is integrated into the context vector that can indicate patient's current state；
Information merging module, is configured as: after the medical diagnosis on disease information and treatment information for having obtained patient, using connection
This two parts information of context vector is merged into the expression vector for indicating patient's Global Information by mode；
Prediction module is configured as: will indicate that vector is put into final output layer to predict medical treatment information.
4. as claimed in claim 3 based on the information prediction device of seeing a doctor again for intersecting attention neural network, characterized in that dimensionality reduction
Module, for information record xi ∈ { 0,1 } of once seeing a doctor | C | it can be split as diagnostic message diai and treatment information trei, i.e.,
Xi={ diai；trei}.To diai and trei, dimensionality reduction is expressed as di ∈ Rm and ti ∈ Rm by the following method respectively:
Specifically,
di=ReLU (Wddiai+bd)
ti=ReLU (Wttrei+bt)。
5. as claimed in claim 3 based on the information prediction device of seeing a doctor again for intersecting attention neural network, characterized in that two-way
RNN module, due to the information di and ti for having two parts relatively independent in the information record xi that once sees a doctor, so we use RNNd
Two parts information is handled respectively with RNNt, RNNd handles diagnostic message d1 to dn, obtains corresponding diagnostic message and hides
State { h1 ..., hn } (hi ∈ Rp), p is the dimension of hidden state), RNNt processing treatment information t1 to tn is obtained corresponding
It treats Information hiding state { g1 ..., gn } (gi ∈ Rp).
6. as claimed in claim 3 based on the information prediction device of seeing a doctor again for intersecting attention neural network, characterized in that intersect
Attention mechanism module is that historical diagnostic information and historical therapeutic information are integrated into the context that can indicate patient's current state
VectorWithThe process of the module including the following steps:
Weight is calculated by hidden state hi and gi respectively:
Softmax function is used to obtained weight vectors α and β, specific as follows:
By weight vectors cross action, i.e.,As treatment informationWeight vectors,As diagnostic message's
Weight vectors.Finally summation obtains patient's current disease state respectivelyAnd therapeutic stateCalculation method is as follows:
7. as claimed in claim 3 based on the information prediction device of seeing a doctor again for intersecting attention neural network, characterized in that information
Merging module is believed this two parts by the way of connection after the medical diagnosis on disease information and treatment information for having obtained patient
Breath be merged into one expression patient's Global Information expression vector p, combined using an articulamentum two parts information obtain it is comprehensive
Close information；
Specifically, its calculation formula is:
8. as claimed in claim 3 based on the information prediction device of seeing a doctor again for intersecting attention neural network, characterized in that prediction
The final expression p of patient is put into final output layer to predict n+1 medical treatment information by module.We are made using softmax
For the activation primitive of output layer.Calculation formula is as follows:
9. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor is realized described in claim 1 a kind of based on intersection note when executing described program
The step of medical information prediction technique of meaning mechanism.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The step of a kind of medical information prediction technique based on intersection attention mechanism described in claim 1 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910300329.2A CN110310740B (en) | 2019-04-15 | 2019-04-15 | Re-hospitalizing information prediction method and system based on cross attention neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910300329.2A CN110310740B (en) | 2019-04-15 | 2019-04-15 | Re-hospitalizing information prediction method and system based on cross attention neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110310740A true CN110310740A (en) | 2019-10-08 |
| CN110310740B CN110310740B (en) | 2020-11-20 |
Family
ID=68074436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910300329.2A Active CN110310740B (en) | 2019-04-15 | 2019-04-15 | Re-hospitalizing information prediction method and system based on cross attention neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110310740B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110880362A (en) * | 2019-11-12 | 2020-03-13 | 南京航空航天大学 | Large-scale medical data knowledge mining and treatment scheme recommending system |
| CN111192693A (en) * | 2019-12-19 | 2020-05-22 | 山东大学 | Method and system for correcting diagnosis codes based on medicine combination |
| CN111462897A (en) * | 2020-04-01 | 2020-07-28 | 山东大学 | Patient similarity analysis method and system based on improved heterogeneous information network |
| CN112017776A (en) * | 2020-10-27 | 2020-12-01 | 平安科技（深圳）有限公司 | Disease prediction method based on dynamic graph and medical knowledge map and related equipment |
| CN113192627A (en) * | 2021-04-13 | 2021-07-30 | 山东大学 | Patient and disease bipartite graph-based readmission prediction method and system |
| CN114255350A (en) * | 2021-12-23 | 2022-03-29 | 四川大学 | Method and system for measuring thickness of soft and hard tissues of palate part |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977361A (en) * | 2017-12-06 | 2018-05-01 | 哈尔滨工业大学深圳研究生院 | The Chinese clinical treatment entity recognition method represented based on deep semantic information |
| US20190034589A1 (en) * | 2017-07-28 | 2019-01-31 | Google Inc. | System and Method for Predicting and Summarizing Medical Events from Electronic Health Records |
| CN109616205A (en) * | 2018-12-04 | 2019-04-12 | 泰康保险集团股份有限公司 | Data processing method, device, medium and electronic equipment |
-
2019
- 2019-04-15 CN CN201910300329.2A patent/CN110310740B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190034589A1 (en) * | 2017-07-28 | 2019-01-31 | Google Inc. | System and Method for Predicting and Summarizing Medical Events from Electronic Health Records |
| CN107977361A (en) * | 2017-12-06 | 2018-05-01 | 哈尔滨工业大学深圳研究生院 | The Chinese clinical treatment entity recognition method represented based on deep semantic information |
| CN109616205A (en) * | 2018-12-04 | 2019-04-12 | 泰康保险集团股份有限公司 | Data processing method, device, medium and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| DEHONG MA等: "Interactive attention networks for aspect-level sentiment classification", 《IJCAI"17: PROCEEDINGS OF THE 26TH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110880362A (en) * | 2019-11-12 | 2020-03-13 | 南京航空航天大学 | Large-scale medical data knowledge mining and treatment scheme recommending system |
| CN110880362B (en) * | 2019-11-12 | 2022-10-11 | 南京航空航天大学 | Large-scale medical data knowledge mining and treatment scheme recommending system |
| CN111192693A (en) * | 2019-12-19 | 2020-05-22 | 山东大学 | Method and system for correcting diagnosis codes based on medicine combination |
| CN111462897A (en) * | 2020-04-01 | 2020-07-28 | 山东大学 | Patient similarity analysis method and system based on improved heterogeneous information network |
| CN112017776A (en) * | 2020-10-27 | 2020-12-01 | 平安科技（深圳）有限公司 | Disease prediction method based on dynamic graph and medical knowledge map and related equipment |
| CN113192627A (en) * | 2021-04-13 | 2021-07-30 | 山东大学 | Patient and disease bipartite graph-based readmission prediction method and system |
| CN114255350A (en) * | 2021-12-23 | 2022-03-29 | 四川大学 | Method and system for measuring thickness of soft and hard tissues of palate part |
| CN114255350B (en) * | 2021-12-23 | 2023-08-04 | 四川大学 | Method and system for measuring thickness of soft and hard tissues of palate |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110310740B (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110310740A (en) | Based on see a doctor again information forecasting method and the system for intersecting attention neural network | |
| Zhang | ATTAIN: Attention-based time-aware LSTM networks for disease progression modeling. | |
| CN1234092C (en) | Predictive modelling method application to computer-aided medical diagnosis | |
| Moujahid et al. | Combining CNN and Grad-Cam for COVID-19 Disease Prediction and Visual Explanation. | |
| Kamble et al. | Applications of artificial intelligence in human life | |
| CN109636061A (en) | Training method, device, equipment and the storage medium of medical insurance Fraud Prediction network | |
| Moein | Medical diagnosis using artificial neural networks | |
| Zhang et al. | KnowRisk: an interpretable knowledge-guided model for disease risk prediction | |
| Harjai et al. | An intelligent clinical decision support system based on artificial neural network for early diagnosis of cardiovascular diseases in rural areas | |
| CN110111885A (en) | Attribute forecast method, apparatus, computer equipment and computer readable storage medium | |
| CN109887606B (en) | Attention-based diagnosis and prediction method for bidirectional recurrent neural network | |
| Zhang et al. | An interpretable fast model for predicting the risk of heart failure | |
| Zhang et al. | New method for dependence assessment in human reliability analysis based on linguistic hesitant fuzzy information | |
| Ulloa | Applied biomedical engineering using artificial intelligence and cognitive models | |
| Leonardi et al. | Explainable process trace classification: An application to stroke | |
| Tiwari et al. | Effective analysis of lung infection using fuzzy rules | |
| Walia et al. | A survey on applications of adaptive neuro fuzzy inference system | |
| US11250723B1 (en) | Visuospatial disorders detection in dementia using a computer-generated environment based on voting approach of machine learning algorithms | |
| CN112329921B (en) | Diuretic dose reasoning equipment based on deep characterization learning and reinforcement learning | |
| CN115240873A (en) | Medicine recommendation method based on machine learning, electronic equipment and computer-readable storage medium | |
| Yadav et al. | Machine learning in expert systems for disease diagnostics in human healthcare | |
| Nurmaini et al. | Breast cancer classification using deep learning | |
| Walia et al. | Effective analysis of lung infection using fuzzy rules | |
| CN113436743B (en) | Representation learning-based multi-outcome efficacy prediction method, device and storage medium | |
| Antonio et al. | PNS2 FLUID BALANCE OR BODY WEIGHT CHANGES AS ROUTINE NURSING ACTIVITY: IS THERE A FAVORITE? |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210927 Address after: 250101 No. 1502, Bojing building, No. 933, Shuntai North Road, Jinan area, China (Shandong) pilot Free Trade Zone, Jinan City, Shandong Province Patentee after: Haorui Zhiyuan (Shandong) artificial intelligence Co.,Ltd. Address before: 250101 no.1500 Shunhua Road, high tech Zone, Jinan City, Shandong Province Patentee before: SHANDONG University |