CN111462841A - Depression intelligent diagnosis device and system based on knowledge graph - Google Patents
Depression intelligent diagnosis device and system based on knowledge graph Download PDFInfo
- Publication number
- CN111462841A CN111462841A CN202010170779.7A CN202010170779A CN111462841A CN 111462841 A CN111462841 A CN 111462841A CN 202010170779 A CN202010170779 A CN 202010170779A CN 111462841 A CN111462841 A CN 111462841A
- Authority
- CN
- China
- Prior art keywords
- data
- model
- knowledge
- entity
- depression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003745 diagnosis Methods 0.000 title claims abstract description 41
- 208000020401 Depressive disease Diseases 0.000 claims abstract description 22
- 230000009471 action Effects 0.000 claims description 19
- 238000000034 method Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000005516 engineering process Methods 0.000 claims description 10
- 238000003058 natural language processing Methods 0.000 claims description 10
- 238000013145 classification model Methods 0.000 claims description 9
- 230000008909 emotion recognition Effects 0.000 claims description 9
- 230000008451 emotion Effects 0.000 claims description 8
- 238000013500 data storage Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 238000010801 machine learning Methods 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 208000001431 Psychomotor Agitation Diseases 0.000 description 2
- 206010038743 Restlessness Diseases 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000019771 cognition Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002996 emotional effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 208000020016 psychiatric disease Diseases 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 206010023644 Lacrimation increased Diseases 0.000 description 1
- 238000012300 Sequence Analysis Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004317 lacrimation Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/70—Multimodal biometrics, e.g. combining information from different biometric modalities
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/20—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for electronic clinical trials or questionnaires
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Pathology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Animal Behavior & Ethology (AREA)
- Child & Adolescent Psychology (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- Medical Treatment And Welfare Office Work (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention provides a knowledge graph-based depression intelligent diagnosis device and system, wherein the device comprises: the data acquisition module is used for acquiring human body data of a user; the human body data comprises video data, audio data, electroencephalogram data and heart rate data; the entity attribute value acquisition module is used for acquiring an entity and a corresponding entity attribute value from human body data by adopting a trained learning model; and the knowledge map module is used for connecting the entity and the entity attribute value to form a knowledge map so as to obtain a depression diagnosis result. The invention can intelligently output the depression diagnosis result and assist doctors in diagnosing the depression.
Description
Technical Field
The invention relates to the technical field of depression intelligent diagnosis devices, in particular to a depression intelligent diagnosis device and system based on a knowledge graph.
Background
In different forms of psychological and mental diseases, depressive disorder is the most extensive, but at present, due to the deviation of cognition on depression, many people are shy of seeing a doctor and miss the opportunity of seeing a doctor, and part of the people enter a hospital for diagnosis and treatment due to the fact that the cognition on depression is very deviated, the diagnosis of depression depends on the experience level of doctors seriously, the levels of doctors in various places are uneven, and the lives of doctors are seriously insufficient, so that part of patients cannot be diagnosed and treated in time. The failure of timely diagnosis and treatment of depression can bring huge social losses, so that the utilization of scientific and technological means to assist doctors in diagnosing depression becomes an important research topic in the health field. The method has very important significance for improving the health level of people and social stability.
The knowledge graph is essentially a semantic network with nodes representing entities or concepts and edges representing semantic relationships between entities/concepts. Knowledge-graphs provide a better ability to organize and manage information. At present, the application of knowledge maps in the medical field is mainly a question-and-answer system, which is not suitable for the diagnosis of mental diseases due to the complexity of the diagnosis, so a knowledge map device and a knowledge map system are urgently needed to be designed for the diagnosis of depression.
Disclosure of Invention
In order to overcome the defects and shortcomings in the prior art, the invention aims to provide a depression intelligent diagnosis device and system based on a knowledge graph, which can intelligently output depression diagnosis results and assist doctors in diagnosing depression.
In order to achieve the purpose, the invention is realized by the following technical scheme: a depression intelligent diagnosis device based on knowledge graph is characterized in that: the method comprises the following steps:
the data acquisition module is used for acquiring human body data of a user; the human body data comprises video data, audio data, electroencephalogram data and heart rate data;
the entity attribute value acquisition module is used for acquiring an entity and a corresponding entity attribute value from human body data by adopting a learning model;
and the knowledge map module is used for connecting the entity and the entity attribute value to form a knowledge map so as to obtain a depression diagnosis result.
Preferably, in the entity attribute value obtaining module, the learning model includes an expression recognition model, an action recognition model, a dressing form recognition model, a speech rate and intonation calculation model, a text analysis model, an emotion recognition model and a stress classification model;
acquiring a picture sequence and pictures from video data; an entity obtained from the picture sequence by adopting an expression recognition model is an expression; the entity obtained from the picture sequence by adopting the action recognition model comprises an action and a reaction; an entity acquired from the picture by adopting the dressing form identification model is the dressing form;
entities obtained from audio data by adopting a speech speed and tone calculation model comprise speech speed and tone; entities obtained from audio data by adopting a text analysis model comprise semantic information; acquiring entities from the electroencephalogram data by adopting an emotion recognition model as emotion information; the entity obtained from the heart rate data using the stress classification model is stress information.
Preferably, the expression recognition model, the action recognition model, the dressing form recognition model, the speech rate and intonation calculation model and the text analysis model respectively adopt a convolutional neural network model or a cyclic neural network model;
the emotion recognition model and the pressure classification model adopt a machine learning model.
Preferably, in the entity attribute value acquisition module, an entity and a corresponding entity attribute value are acquired from medical data by a natural language processing technology to train a learning model; in the knowledge graph module, entities and corresponding entity attribute values are obtained from medical data through a natural language processing technology, and then a knowledge graph is constructed through the relationship between the entities.
Preferably, the knowledge graph module is constructed by adopting a knowledge graph, the association relation among the entities is extracted, and the constructed knowledge graph is subjected to knowledge reasoning to obtain a deeper entity relation, so that an expanded knowledge graph is obtained; the constructed knowledge-graph is then stored in a Neo4j database.
Preferably, the knowledge-graph module further realizes iteration and perfection through a closed-loop system.
A system comprising the intelligent depression diagnosis device based on the knowledge map is characterized in that: the method comprises the following steps:
the client layer comprises a data acquisition module, a scale generation module for generating a response scale, a scale response module for triggering emotion of a user and filling in the scale, a report module for displaying a depression diagnosis result and a labeling module for constructing a knowledge map;
the data storage layer is used for storing the data and the knowledge graph transmitted by the client layer and transmitting the data to the data processing layer;
the data processing layer is used for preprocessing, extracting features and classifying the received data collected by the client to obtain each entity and corresponding entity attribute value in the knowledge graph, then calculating the intensity index among different nodes, and further constructing the knowledge graph to obtain a depression diagnosis result; the entity attribute value acquisition module and the knowledge graph module are respectively positioned in the data processing layer.
Compared with the prior art, the invention has the following advantages and beneficial effects:
compared with the existing depression recognition device and system, the invention extracts entities and entity attribute values in medical data by using a natural language processing technology, then establishes a learning model to calculate the entity attribute values to obtain the entity and entity attribute values, and calculates the relationship among the entities to construct a knowledge graph so as to simulate the diagnosis process of a doctor and realize intelligent and comprehensive depression diagnosis.
Drawings
FIG. 1 is a block diagram of the intelligent depression diagnosis device based on knowledge map;
fig. 2 is a block diagram of the architecture of the system of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
Example one
The intelligent depression diagnosis device based on the knowledge graph comprises a data acquisition module, an entity attribute value acquisition module and a knowledge graph module, as shown in fig. 1.
The data acquisition module is used for acquiring human body data of a user; the human body data includes video data, audio data, electroencephalogram data and heart rate data. Collecting video data by using a camera and collecting audio data by using a microphone; in order to judge the stress and emotion conditions, a multi-lead physiological instrument can be used for collecting stimulation electroencephalogram data and heart rate data. The video data is divided into image sequences by frames, the image sequences and the images are respectively stored, and other data are stored after being cleaned.
In the entity attribute value acquisition module, acquiring entities and corresponding entity attribute values from medical data through a natural language processing technology to train a learning model; in the knowledge graph module, entities and corresponding entity attribute values are obtained from medical data through a natural language processing technology, and then a knowledge graph is constructed through the relationship between the entities.
The entity attribute value acquisition module is used for acquiring an entity and a corresponding entity attribute value from human body data by adopting a learning model;
in the entity attribute value acquisition module, the learning model comprises an expression recognition model, an action recognition model, a dressing form recognition model, a speed and tone calculation model, a text analysis model, an emotion recognition model and a stress classification model.
Acquiring a picture sequence and pictures from video data; an entity obtained from the picture sequence by adopting an expression recognition model is an expression; the entity obtained from the picture sequence by adopting the action recognition model comprises an action and a reaction; an entity acquired from the picture by adopting the dressing form identification model is the dressing form;
entities obtained from audio data by adopting a speech speed and tone calculation model comprise speech speed and tone; entities obtained from audio data by adopting a text analysis model comprise semantic information; acquiring entities from the electroencephalogram data by adopting an emotion recognition model as emotion information; the entity obtained from the heart rate data using the stress classification model is stress information.
The entity and entity attribute values are as follows:
1) wear patterns-sloppy, odd wear, irregular wear, normal;
2) speech speed-fast, slow, normal;
3) intonation-high, low, normal;
4) reaction-too fast, too slow, normal;
5) expression-anger, disgust, fear, happiness, sadness, surprise, neutrality, lacrimation;
6) action-restlessness, rich body language, small actions and normal;
7) pressure information-neutral, pressure, pleasure;
8) emotional information — emotional classification level.
The expression recognition model, the action recognition model, the dressing form recognition model, the speech rate and tone calculation model and the text analysis model respectively adopt a convolutional neural network model or a cyclic neural network model, wherein the convolutional neural network model comprises a 2D convolutional neural network model and a 3D convolutional neural network model, the 2D convolutional neural network model is commonly used for processing two-dimensional information such as pictures and the like, the 3D convolutional neural network model is commonly used for processing three-dimensional information such as videos and the like, the convolutional neural network model generally comprises a convolutional layer, a pooling layer and a full-connection layer, the common convolutional neural network models comprise L eNet, AlexNet, VggNet, ResNet and the like, the cyclic neural network model has the main function of processing and predicting sequence data, the conventional convolutional neural network model can memorize the previous information, the current output can be influenced by the output in the previous period of time and is commonly used for voice processing, and can also be used for video classification by being combined with the convolutional neural network model, and the two deep network models can finish two tasks of feature extraction and classification, so that the preprocessed data are directly input into the deep network model, and the super-parameter training of the deep network model can be obtained.
For signal processing of pictures, videos and the like, the mainstream methods at present are divided into two methods, one method is to manually extract features and then input the features into a classifier for classification, the other method is to directly input data into a deep network model to complete the tasks of feature extraction and classification to obtain a classification result, a deep neural network is rapidly developed in recent years and obtains a better result than the traditional method in the fields of computer vision, natural language processing and the like, so that the deep network model method is mainly adopted for establishing the picture and video analysis model.
Because rich backgrounds contained in pictures can have negative influence on analysis, face detection algorithms are needed to detect faces from the pictures and cut and store face picture sequences. In deep learning, CNN is often used for extracting spatial features of images, RNN is often used for extracting temporal features due to the fact that RNN has time sequence analysis capability, therefore, spatiotemporal features can be extracted by combining the characteristics of CNN and RNN, a CNN network trained by large-scale data can be finely adjusted by adopting a similar data set when the data set is small, the finely adjusted model is used for extracting spatial features, then the spatial features with a certain length are input into the RNN network to extract the spatial features, and finally classification is carried out.
The attribute values related to the actions comprise restlessness, rich body language, more small actions and the like, namely the frequency of the actions is concerned more, a video-based action recognition deep network model is constructed, and the times of various actions in unit time are counted. The attribute values of the dressing form contain sloppy, odd-clothing is irregular and normal, and only contain spatial information based on picture analysis, so that the dressing form can be classified by using a convolutional neural network.
The intonation is the arrangement and the change of the tone height in a sentence, the word meaning and the intonation meaning of a sentence are the pitch characteristics in the complete meaning audio, namely the pitch information is included, and the intonation information can be obtained by acquiring the pitch information.
Speech rate is the speed of the vocabulary presented by a human-expressed-meaning token in a unit of time. The recognition of the speed of speech can be represented by the number of characters contained in unit time from the beginning to the end of a section of speech, and when the number of characters reaches a small threshold value, the speed of speech is considered to be slow, and when the number of characters reaches a large threshold value, the speed of speech is considered to be fast.
The speech recognition technology can convert speech signals into character signals, a large amount of data is needed for training a speech recognition model, speech recognition can be carried out by using a public speech recognition interface under the condition of insufficient data quantity, and keyword extraction and semantic understanding are carried out on obtained text information by using a natural language processing technology.
The emotion recognition model and the pressure classification model adopt a machine learning model. The electroencephalogram data and the heart rate data have small information content, so that a good classification effect can be obtained by adopting a machine learning model. The treatment process can be divided into three steps: firstly, preprocessing data, and removing interference noise such as current, other physiological signals and the like in signals by adopting a conventional operation removing method to obtain relatively pure electroencephalogram signals and heart rate signals; then extracting features, and extracting linear and nonlinear features aiming at the characteristics of electroencephalogram signals and heart rate signals by using a traditional signal processing method; and inputting the features into common classifiers such as SVM, KNN or random forest and the like to obtain a classification result.
The knowledge map module is used for connecting the entity and the entity attribute value to form a knowledge map so as to obtain a depression diagnosis result, and iteration and perfection of the whole system are realized through a closed loop system.
The knowledge graph module is constructed by adopting a knowledge graph, the association relation among all entities is extracted, and the constructed knowledge graph is subjected to knowledge reasoning to obtain a deeper entity relation so as to obtain an expanded knowledge graph; the constructed knowledge-graph is then stored in a Neo4j database.
A knowledge graph is a knowledge network that contains entities, entity attribute values, and relationships between the entities. The expression of the knowledge graph is as follows:
G＝(E，R，S)
where E ═ E1, E2, E3., en } represents a set of entities, R ═ R1, R2, R3., rn } represents a set of relationships, S ∈ E R E represents a triplet of (entities, relationships, entities), and the entities are linked to form a web-like knowledge structure.
The module establishes the calculation process of the entities and the entity attribute values, takes the entities and the entity attribute values as nodes in the knowledge graph, extracts the relationships among the entities by using a relationship inference model to obtain the knowledge graph, and then can obtain a conclusion from the knowledge graph through knowledge retrieval.
The constructed knowledge graph is stored in a Neo4j database. Neo4j is an embedded, disk-based, java persistence engine with full transactional features that can store structured data on a network (mathematically called a graph) instead of tables, and subsequently derive depression diagnosis from a knowledge map by knowledge retrieval. When new data is generated, the entity and the entity attribute value can be expanded according to the voice keywords, the attribute calculation model is supplemented, the attribute calculation model is further trained by using the new data, and the relationship inference model is trained, so that the knowledge graph is updated, and the recognition rate of the depression knowledge graph is higher.
Example two
The embodiment describes a system comprising the intellectual map-based depression intelligent diagnosis device of the embodiment, as shown in fig. 2, comprising a client layer, a data storage layer and a data processing layer.
The client layer comprises a data acquisition module, a scale generation module for generating a response scale, a scale response module for triggering emotion of a user and filling in the scale, a report module for displaying a depression diagnosis result, and a labeling module for constructing a knowledge map.
The diagnosis process of the depression mainly depends on the judgment of doctors on the condition of questions in answer scales of patients, so a scale generation module is established, various scales are conveniently established and converted into an xml form, and the scales are displayed to users in a proper form in the scale answering module, wherein the scales comprise subjects of one channel, applicable diseases, scale types and the like, and the subjects comprise questions, optional answers, corresponding audio and video data and the like, and the scales are not limited to the traditional mini scale, the depression self-measuring scale, the SC L90 scale and the like, but also can be 'scales' consisting of a plurality of different scenes or videos used when pressure and emotion are induced.
And the gauge answering module is used for displaying the gauge to the user by using a web browser. In the report module, different visual data information is displayed according to different user roles.
The data marking module is mainly used for marking the acquired signals, displaying the reasoning result of each model and correcting the wrong result by professional personnel so as to further train the model. The data marking module is constructed based on the entities of the knowledge graph, the entities comprise personal information, dressing forms, voice, behavior actions, reactions, physiological signals and the like, and the module can mark the attribute values of all the entities by professional doctors and experts in the stage of training the models required by a data processing layer so as to construct the calculation models of all the entity attributes; after the model training is completed, the attribute values of each entity obtained through the model analysis are displayed, so that the professional can correct the error result and further train the model conveniently.
The reporting module is used for displaying the attribute values of the entities and the final result. Different roles have different contents and different emphasis, so different roles are shown with different contents. And displaying the result in a more underscore manner by using a visualization means.
Front-end pages are written based on a contact framework, a BFF layer is written based on an egg framework, and background codes are written by java. Data visualization may be implemented using javascript library Data-Driven Documents, which uses the SVG format, allowing rendered shapes to be scaled up or down without degrading quality.
The data storage layer is used for storing the data and the knowledge graph transmitted by the client layer and transmitting the data to the data processing layer; and the function of linking the client layer and the data processing layer is realized. Basic information and the like of a user can be saved by using a relational database mysql, a knowledge map can be saved by using a map database Neo4j, and audio, video and picture data used in a scale and collected physiological signals, audio and video information can be stored in an OSS cloud storage.
The data processing layer is used for preprocessing, extracting features and classifying the received data collected by the client to obtain each entity in the knowledge graph and corresponding entity attribute values, then calculating the intensity indexes among different nodes, and further constructing the knowledge graph to obtain a depression diagnosis result; the entity attribute value acquisition module and the knowledge graph module are respectively positioned in the data processing layer. The data processing layer calls each model interface to acquire each piece of information from the oss end, calculates the information to obtain a conclusion, returns to the data storage layer and displays the conclusion in the marking module; and retrieving the knowledge graph to obtain a final conclusion, returning the result to the data storage layer, and then displaying the result in the report module.
Compared with the existing depression recognition method and system, the method utilizes the natural language processing technology to extract the entities and the entity attribute values in the medical data, then establishes an algorithm model to calculate the entity attribute values to obtain the entity and the entity attribute values, calculates the relationship among the entity groups to construct a knowledge graph, and realizes intelligent and comprehensive depression diagnosis.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (7)
1. A depression intelligent diagnosis device based on knowledge graph is characterized in that: the method comprises the following steps:
the data acquisition module is used for acquiring human body data of a user; the human body data comprises video data, audio data, electroencephalogram data and heart rate data;
the entity attribute value acquisition module is used for acquiring an entity and a corresponding entity attribute value from human body data by adopting a trained learning model;
and the knowledge map module is used for connecting the entity and the entity attribute value to form a knowledge map so as to obtain a depression diagnosis result.
2. The intellectual diagnosis device for depression based on knowledge-graph according to claim 1, wherein: in the entity attribute value acquisition module, the learning model comprises an expression recognition model, an action recognition model, a dressing form recognition model, a speed and tone calculation model, a text analysis model, an emotion recognition model and a pressure classification model;
acquiring a picture sequence and pictures from video data; an entity obtained from the picture sequence by adopting an expression recognition model is an expression; the entity obtained from the picture sequence by adopting the action recognition model comprises an action and a reaction; an entity acquired from the picture by adopting the dressing form identification model is the dressing form;
entities obtained from audio data by adopting a speech speed and tone calculation model comprise speech speed and tone; entities obtained from audio data by adopting a text analysis model comprise semantic information; acquiring entities from the electroencephalogram data by adopting an emotion recognition model as emotion information; the entity obtained from the heart rate data using the stress classification model is stress information.
3. The intellectual property map-based depression intelligent diagnosis device according to claim 2, wherein: the expression recognition model, the action recognition model, the dressing form recognition model, the speech rate and intonation calculation model and the text analysis model respectively adopt a convolutional neural network model or a cyclic neural network model;
the emotion recognition model and the pressure classification model adopt a machine learning model.
4. The intellectual diagnosis device for depression based on knowledge-graph according to claim 1, wherein: in the entity attribute value acquisition module, acquiring entities and corresponding entity attribute values from medical data through a natural language processing technology to train a learning model; in the knowledge graph module, entities and corresponding entity attribute values are obtained from medical data through a natural language processing technology, and then a knowledge graph is constructed through the relationship between the entities.
5. The intellectual diagnosis device for depression based on knowledge-graph according to claim 1, wherein: the knowledge graph module is constructed by adopting a knowledge graph, the association relation among all entities is extracted, and the constructed knowledge graph is subjected to knowledge reasoning to obtain a deeper entity relation so as to obtain an expanded knowledge graph; the constructed knowledge-graph is then stored in a Neo4j database.
6. The intellectual diagnosis device for depression based on knowledge-graph according to claim 1, wherein: the knowledge graph module realizes iteration and perfection through a closed loop system.
7. A system comprising the intellectual property map based depression intelligent diagnosis apparatus according to any one of claims 1 to 6, wherein: the method comprises the following steps:
the client layer comprises a data acquisition module, a scale generation module for generating a response scale, a scale response module for triggering emotion of a user and filling in the scale, a report module for displaying a depression diagnosis result and a labeling module for constructing a knowledge map;
the data storage layer is used for storing the data and the knowledge graph transmitted by the client layer and transmitting the data to the data processing layer;
the data processing layer is used for preprocessing, extracting features and classifying the received data collected by the client to obtain each entity and corresponding entity attribute value in the knowledge graph, then calculating the intensity index among different nodes, and further constructing the knowledge graph to obtain a depression diagnosis result; the entity attribute value acquisition module and the knowledge graph module are respectively positioned in the data processing layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010170779.7A CN111462841B (en) | 2020-03-12 | 2020-03-12 | Intelligent depression diagnosis device and system based on knowledge graph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010170779.7A CN111462841B (en) | 2020-03-12 | 2020-03-12 | Intelligent depression diagnosis device and system based on knowledge graph |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111462841A true CN111462841A (en) | 2020-07-28 |
| CN111462841B CN111462841B (en) | 2023-06-20 |
Family
ID=71684240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010170779.7A Active CN111462841B (en) | 2020-03-12 | 2020-03-12 | Intelligent depression diagnosis device and system based on knowledge graph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111462841B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897972A (en) * | 2020-08-06 | 2020-11-06 | 南方电网科学研究院有限责任公司 | Data track visualization method and device |
| CN112037911A (en) * | 2020-08-28 | 2020-12-04 | 北京万灵盘古科技有限公司 | Machine learning-based mental assessment screening system and training method thereof |
| CN112148884A (en) * | 2020-08-21 | 2020-12-29 | 北京阿叟阿巴科技有限公司 | Systems and methods for autism intervention |
| CN112668384A (en) * | 2020-08-07 | 2021-04-16 | 深圳市唯特视科技有限公司 | Knowledge graph construction method and system, electronic equipment and storage medium |
| CN112925918A (en) * | 2021-02-26 | 2021-06-08 | 华南理工大学 | Question-answer matching system based on disease field knowledge graph |
| WO2022102721A1 (en) * | 2020-11-11 | 2022-05-19 | Ａｓｓｅｓｔ株式会社 | Depression-state-determining program |
| CN115399773A (en) * | 2022-09-14 | 2022-11-29 | 山东大学 | Depression state identification system based on deep learning and pulse signals |
| CN115630697A (en) * | 2022-10-26 | 2023-01-20 | 泸州职业技术学院 | Knowledge graph construction method and system capable of distinguishing single-phase and double-phase affective disorder |
| CN117056536A (en) * | 2023-10-10 | 2023-11-14 | 湖南创星科技股份有限公司 | Knowledge graph driving-based virtual doctor system and operation method thereof |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407733A (en) * | 2016-12-12 | 2017-02-15 | 兰州大学 | Depression risk screening system and method based on virtual reality scene electroencephalogram signal |
| CN106725532A (en) * | 2016-12-13 | 2017-05-31 | 兰州大学 | Depression automatic evaluation system and method based on phonetic feature and machine learning |
| EP3223177A1 (en) * | 2016-03-24 | 2017-09-27 | Fujitsu Limited | System and method to aid diagnosis of a patient |
| CN109171769A (en) * | 2018-07-12 | 2019-01-11 | 西北师范大学 | It is a kind of applied to depression detection voice, facial feature extraction method and system |
| CN109346169A (en) * | 2018-10-17 | 2019-02-15 | 长沙瀚云信息科技有限公司 | A kind of artificial intelligence assisting in diagnosis and treatment system and its construction method, equipment and storage medium |
| CN109545373A (en) * | 2018-11-08 | 2019-03-29 | 新博卓畅技术（北京）有限公司 | A kind of automatic abstracting method of human body diseases symptom characteristic, system and equipment |
| CN109697233A (en) * | 2018-12-03 | 2019-04-30 | 中电科大数据研究院有限公司 | A kind of knowledge mapping system building method |
| CN110083708A (en) * | 2019-04-26 | 2019-08-02 | 常州市贝叶斯智能科技有限公司 | A kind of medical bodies association analysis method of knowledge based map |
| CN110675951A (en) * | 2019-08-26 | 2020-01-10 | 北京百度网讯科技有限公司 | Intelligent disease diagnosis method and device, computer equipment and readable medium |
-
2020
- 2020-03-12 CN CN202010170779.7A patent/CN111462841B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3223177A1 (en) * | 2016-03-24 | 2017-09-27 | Fujitsu Limited | System and method to aid diagnosis of a patient |
| CN106407733A (en) * | 2016-12-12 | 2017-02-15 | 兰州大学 | Depression risk screening system and method based on virtual reality scene electroencephalogram signal |
| CN106725532A (en) * | 2016-12-13 | 2017-05-31 | 兰州大学 | Depression automatic evaluation system and method based on phonetic feature and machine learning |
| CN109171769A (en) * | 2018-07-12 | 2019-01-11 | 西北师范大学 | It is a kind of applied to depression detection voice, facial feature extraction method and system |
| CN109346169A (en) * | 2018-10-17 | 2019-02-15 | 长沙瀚云信息科技有限公司 | A kind of artificial intelligence assisting in diagnosis and treatment system and its construction method, equipment and storage medium |
| CN109545373A (en) * | 2018-11-08 | 2019-03-29 | 新博卓畅技术（北京）有限公司 | A kind of automatic abstracting method of human body diseases symptom characteristic, system and equipment |
| CN109697233A (en) * | 2018-12-03 | 2019-04-30 | 中电科大数据研究院有限公司 | A kind of knowledge mapping system building method |
| CN110083708A (en) * | 2019-04-26 | 2019-08-02 | 常州市贝叶斯智能科技有限公司 | A kind of medical bodies association analysis method of knowledge based map |
| CN110675951A (en) * | 2019-08-26 | 2020-01-10 | 北京百度网讯科技有限公司 | Intelligent disease diagnosis method and device, computer equipment and readable medium |
Non-Patent Citations (2)
| Title |
|---|
| 丁汉青;刘念;: "情绪识别研究的学术场域――基于CiteSpace的科学知识图谱分析" * |
| 董丽丽等: "融合知识图谱与深度学习的疾病诊断方法研究", 《计算机科学与探索》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111897972A (en) * | 2020-08-06 | 2020-11-06 | 南方电网科学研究院有限责任公司 | Data track visualization method and device |
| CN111897972B (en) * | 2020-08-06 | 2023-10-17 | 南方电网科学研究院有限责任公司 | Data track visualization method and device |
| CN112668384A (en) * | 2020-08-07 | 2021-04-16 | 深圳市唯特视科技有限公司 | Knowledge graph construction method and system, electronic equipment and storage medium |
| CN112668384B (en) * | 2020-08-07 | 2024-05-31 | 深圳市唯特视科技有限公司 | Knowledge graph construction method, system, electronic equipment and storage medium |
| CN112148884A (en) * | 2020-08-21 | 2020-12-29 | 北京阿叟阿巴科技有限公司 | Systems and methods for autism intervention |
| CN112148884B (en) * | 2020-08-21 | 2023-09-22 | 北京阿叟阿巴科技有限公司 | Systems and methods for autism intervention |
| CN112037911A (en) * | 2020-08-28 | 2020-12-04 | 北京万灵盘古科技有限公司 | Machine learning-based mental assessment screening system and training method thereof |
| CN112037911B (en) * | 2020-08-28 | 2024-03-05 | 北京万灵盘古科技有限公司 | Screening system for mental assessment based on machine learning and training method thereof |
| WO2022102721A1 (en) * | 2020-11-11 | 2022-05-19 | Ａｓｓｅｓｔ株式会社 | Depression-state-determining program |
| CN112925918A (en) * | 2021-02-26 | 2021-06-08 | 华南理工大学 | Question-answer matching system based on disease field knowledge graph |
| CN115399773A (en) * | 2022-09-14 | 2022-11-29 | 山东大学 | Depression state identification system based on deep learning and pulse signals |
| CN115630697A (en) * | 2022-10-26 | 2023-01-20 | 泸州职业技术学院 | Knowledge graph construction method and system capable of distinguishing single-phase and double-phase affective disorder |
| CN117056536A (en) * | 2023-10-10 | 2023-11-14 | 湖南创星科技股份有限公司 | Knowledge graph driving-based virtual doctor system and operation method thereof |
| CN117056536B (en) * | 2023-10-10 | 2023-12-26 | 湖南创星科技股份有限公司 | Knowledge graph driving-based virtual doctor system and operation method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111462841B (en) | 2023-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111462841B (en) | Intelligent depression diagnosis device and system based on knowledge graph | |
| CN110148318B (en) | Digital teaching assistant system, information interaction method and information processing method | |
| CN112863630A (en) | Personalized accurate medical question-answering system based on data and knowledge | |
| CN110532912B (en) | Sign language translation implementation method and device | |
| CN112667799B (en) | Medical question-answering system construction method based on language model and entity matching | |
| CN113506624B (en) | Autism children cognitive ability evaluation intervention system based on hierarchical generalization push logic | |
| CN109448851A (en) | A kind of cognition appraisal procedure and device | |
| CN112766173A (en) | Multi-mode emotion analysis method and system based on AI deep learning | |
| CN113380271B (en) | Emotion recognition method, system, device and medium | |
| JP2022146884A (en) | Method and device for evaluating difficult airway based on artificial intelligence | |
| CN112133406B (en) | Multi-mode emotion guidance method and system based on emotion maps and storage medium | |
| CN111920420A (en) | Patient behavior multi-modal analysis and prediction system based on statistical learning | |
| CN115064246A (en) | Depression evaluation system and equipment based on multi-mode information fusion | |
| CN115713875A (en) | Virtual reality simulation teaching method based on psychological analysis | |
| Zhao et al. | Research and development of autism diagnosis information system based on deep convolution neural network and facial expression data | |
| CN116883608B (en) | Multi-mode digital person social attribute control method and related device | |
| CN117454217A (en) | Deep ensemble learning-based depression emotion recognition method, device and system | |
| CN115810424A (en) | Cognitive behavioral assessment and treatment system | |
| CN114974254A (en) | Child depression emotion recognition method based on multi-mode artificial intelligence technology | |
| Younis et al. | Machine learning for human emotion recognition: a comprehensive review | |
| Tan et al. | Towards automatic engagement recognition of autistic children in a machine learning approach | |
| CN110289065A (en) | A kind of auxiliary generates the control method and device of medical electronic report | |
| CN114595731B (en) | Semantic segmentation method of nonlinear medical sensor data based on continuous learning | |
| Zhang et al. | Construction method of college students' depression knowledge map based on education big data | |
| CN117976192A (en) | Multimode-based sleep disorder and depression evaluation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |