CN117056536B - Knowledge graph driving-based virtual doctor system and operation method thereof - Google Patents

Knowledge graph driving-based virtual doctor system and operation method thereof Download PDF

Info

Publication number
CN117056536B
CN117056536B CN202311301744.2A CN202311301744A CN117056536B CN 117056536 B CN117056536 B CN 117056536B CN 202311301744 A CN202311301744 A CN 202311301744A CN 117056536 B CN117056536 B CN 117056536B
Authority
CN
China
Prior art keywords
module
user
emotion
intervention
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311301744.2A
Other languages
Chinese (zh)
Other versions
CN117056536A (en
Inventor
龙海
文建全
彭炜
黄刊迪
杨文君
肖媛
甘元茂
任强
文舸扬
虞敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Trasen Technology Co ltd
Original Assignee
Hunan Trasen Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Trasen Technology Co ltd filed Critical Hunan Trasen Technology Co ltd
Priority to CN202311301744.2A priority Critical patent/CN117056536B/en
Publication of CN117056536A publication Critical patent/CN117056536A/en
Application granted granted Critical
Publication of CN117056536B publication Critical patent/CN117056536B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/335—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/194—Calculation of difference between files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G06F40/295—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention discloses a knowledge graph driving-based virtual doctor system and an operation method thereof, wherein the system comprises the following steps: the system comprises a natural language processing module, a multi-mode data acquisition module, a knowledge graph module, a question-answering system module, a traditional Chinese medicine syndrome differentiation module, a psychological assessment module, a emotion recognition module and a digital intervention management module; the natural language processing module is used for receiving and converting data input by a user; the multi-mode data acquisition module is used for acquiring behavior characteristics of a user in the interaction process; the knowledge graph module is used for storing disease knowledge; the question and answer system module is used for interviewing question and answer with the user; the traditional Chinese medicine dialectical module is used for identifying traditional Chinese medicine symptoms of the user; the psychological assessment module is used for assessing the psychological state of the user; the emotion recognition module is used for recognizing and judging emotion types of the user; the digital intervention management module is used for pushing a digital intervention management scheme to a user; the invention provides intelligent medical consultation, diagnosis advice and intervention service for the depression user.

Description

Knowledge graph driving-based virtual doctor system and operation method thereof
Technical Field
The invention relates to the field of artificial intelligence and medical treatment, in particular to a virtual doctor system driven by a knowledge graph and an operation method thereof.
Background
Depression is a common mental disorder, and is mainly represented by continuous mood depression, interest loss, self-detraction, sleep disorder, anorexia and the like, and seriously affects the life quality and social functions of patients. About 2.64 million people worldwide have depression, which is one of the leading causes of disability and death, as counted by the world health organization.
The treatment method of the depression mainly comprises medication, psychological treatment, social support and the like, wherein the psychological treatment is one of effective non-medication treatment means, and can help patients to improve psychological states, enhance self-regulation capacity and improve life satisfaction. However, many patients with depression cannot obtain timely and appropriate psychotherapy services due to the lack of psychotherapy resources, imbalance, and accessibility of the resources.
To solve this problem, virtual doctor systems based on artificial intelligence technology have appeared in recent years, which provide personalized medical consultation, diagnosis, evaluation and intervention services to patients by simulating dialogue interactions between human doctors and patients. These systems utilize techniques such as natural language processing, machine learning, knowledge representation, etc., to extract useful information from text or sound signals input by the patient, and to generate appropriate answers or advice based on a built-in or external medical knowledge base or rules base, and to output the same to the patient in natural language.
However, existing virtual doctor systems still suffer from several drawbacks, such as:
the construction and maintenance cost of the knowledge base or rule base is high, and the integrity, accuracy and timeliness of knowledge are difficult to ensure; the system lacks support for the diagnosis and treatment method of traditional Chinese medicine, and cannot fully utilize the abundant experience and effective scheme of the traditional Chinese medicine field about depression; the system lacks the processing capability of multi-modal data (such as facial expressions, limb actions and the like) and cannot comprehensively capture psychological characteristics and emotional states of a patient; the system lacks the ability to push digital intervention management schemes (e.g., medication reminders) that are not effective in facilitating patient execution of treatment plans.
Therefore, there is a need to design a new knowledge-graph-driven virtual doctor system to solve the above-mentioned problems.
Disclosure of Invention
Aiming at the problems, the invention provides a knowledge-graph-driven virtual doctor system and an operation method thereof, which are used for providing intelligent medical consultation, diagnosis suggestion and intervention service for depression users.
In order to achieve the above purpose, the invention adopts the following technical scheme:
comprising the following steps:
the natural language processing module is used for receiving text or sound signals input by a user and converting the text or sound signals into data input which can be recognized and processed by the computer system, and the data input comprises entities, relations and semantics;
The multi-mode data acquisition module is used for acquiring behavior characteristics, including facial micro-expressions, limb actions and voice intonation, of a user in the interaction process with the system;
the knowledge map module is used for storing and managing disease knowledge about depression in the field of Chinese and Western medicine, and comprises semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological states and intervention schemes and semantic relations among the semantic types, wherein each semantic type has a corresponding description and is used for explaining concepts and features of the semantic type;
the question-answering system module is used for generating a content expression of questions and answers between a doctor and a user according to the data converted by the natural language processing module and disease knowledge in the knowledge graph module, converting the content expression into natural language and outputting the natural language to the user;
the traditional Chinese medicine syndrome differentiation module is used for identifying traditional Chinese medicine syndrome features of a depressive disorder user by matching with corresponding syndrome patterns in the knowledge graph module through entity alignment and semantic search according to the data converted by the natural language processing module and the syndrome description in the knowledge graph module;
the psychological assessment module is used for comprehensively assessing the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content in the data converted by the natural language processing module, the behavioral characteristics acquired by the multi-modal data acquisition module and the psychological state description in the knowledge graph module;
The emotion recognition module is used for recognizing and judging the emotion type of the user according to the data converted by the natural language processing module and interview question and answer contents in the question and answer system module;
the digital intervention management module is used for pushing a personalized digital intervention management scheme to the user according to the traditional Chinese medicine syndrome characteristics, the psychological state evaluation result and the emotion recognition result of the user.
In a preferred implementation case, a user feedback module is provided, and the user feedback module is used for collecting information and data fed back by a user after the user finishes the pushed intervention management scheme according to the instruction of the digital intervention management module, sending the information and data to the natural language module, and optimizing or adjusting the output and performance of each related module based on the user feedback information and data.
In a preferred embodiment, the natural language processing module adopts a deep neural network model, and includes:
the text analysis sub-module is used for performing text analysis tasks of word segmentation, part-of-speech tagging, named entity identification, relation extraction and semantic role tagging on a text signal input by a user, so that entities, relations and semantics in the text are extracted;
and the voice recognition sub-module is used for carrying out acoustic feature extraction, acoustic modeling, language modeling and decoding on the voice signal input by the user so as to convert the voice signal into a text signal, and inputting the text signal into the text analysis sub-module for further processing.
In a preferred embodiment, the knowledge graph module adopts an ontology modeling method, which includes:
the ontology construction submodule is used for defining a set of ontology concepts and attributes according to disease knowledge about depression in the field of Chinese and Western medicine, and representing an ontology structure by using an ontology representation language, wherein the ontology structure comprises classification and definition description of an intervention scheme;
the ontology storage submodule is used for storing the constructed ontology structure in an ontology database and providing an ontology query language to query ontology data;
and the ontology reasoning sub-module is used for logically reasoning the data in the ontology database according to the ontology reasoning rule, so as to obtain some implicit knowledge and relations, wherein the ontology reasoning rule is a rule for reasoning and deducing the knowledge in the ontology.
In a preferred implementation case, the question answering system module adopts a question answering method based on a knowledge graph, and the question answering method comprises the following steps:
the problem analysis sub-module is used for analyzing the problems in the data converted by the natural language processing module, including problem classification, problem intention recognition and problem entity linking tasks, so as to obtain the types, intention and related entities of the problems;
The answer generation sub-module is used for generating a content expression of a question and answer between a doctor and a user according to the question information obtained by the question analysis sub-module and the knowledge of the diseases in the knowledge graph module, and converting the content expression into a natural language and outputting the natural language to the user; the answer generation submodule selects different answer generation strategies according to the question types and intentions;
the answer evaluation sub-module is used for evaluating the answers generated by the answer generation sub-module, including answer correctness, answer completeness, answer consistency and answer credibility indexes, so as to obtain quality scores of the answers;
and the answer optimizing sub-module is used for optimizing or adjusting the answer generating strategy and parameters in the answer generating sub-module according to the answer quality scores obtained by the answer evaluating sub-module and the feedback data in the user feedback module, so that the answer quality and the user satisfaction are improved.
Under the preferred implementation condition, the traditional Chinese medicine syndrome differentiation module adopts a traditional Chinese medicine syndrome differentiation method based on a knowledge graph, and comprises the following steps:
the entity alignment sub-module is used for aligning the entity in the data converted by the natural language processing module, namely, matching and mapping the entity with the ontology concept in the knowledge graph module so as to obtain the ontology type and attribute of the entity;
The semantic search sub-module is used for finding out candidate answers of the syndromes matched with or close to the entity by calculating the semantic similarity or the semantic distance between the entity and the syndromes according to the entity information obtained by the entity alignment sub-module and the syndrome description in the knowledge graph module;
and the syndrome identification sub-module is used for determining traditional Chinese medicine syndrome characteristics of the depressive disorder user by applying a syndrome identification rule or algorithm according to the semantic relation between the syndrome candidate answers obtained by the semantic search sub-module and the syndromes in the knowledge graph module.
In a preferred embodiment, the psychological assessment module adopts a psychological assessment method based on multi-modal data fusion, and the psychological assessment method comprises the following steps:
the text emotion analysis submodule is used for carrying out emotion analysis on text content in the data converted by the natural language processing module, wherein the text content comprises emotion tendency and emotion strength, so that emotion characteristics of a user reflected by the text content are obtained;
the psychological characteristic analysis sub-module is used for analyzing the behavioral characteristics of the user acquired by the multi-mode data acquisition module so as to obtain psychological characteristics of the user reflected by the behavioral characteristics;
the data fusion sub-module is used for carrying out data fusion on the emotion characteristics and the psychological characteristics of the user, which are obtained by the text emotion analysis sub-module and the psychological characteristic analysis sub-module, and comprises data alignment, data fusion and data mapping, so that the comprehensive psychological characteristics of the user are obtained;
And the psychological state evaluation sub-module is used for determining the psychological state of the user by calculating the semantic similarity or the semantic distance between the comprehensive psychological features of the user and the psychological state description according to the comprehensive psychological features of the user and the psychological state description in the knowledge graph module obtained by the data fusion sub-module.
In a preferred implementation case, the emotion recognition module adopts an emotion recognition method based on deep learning, and the emotion recognition method comprises the following steps:
the emotion classification sub-module is used for performing emotion classification on the text content in the data converted by the natural language processing module and the interview question and answer content in the question and answer system module, wherein the emotion classification sub-module comprises emotion intensity and emotion category, so that emotion types of the users reflected by the text content and the interview question and answer content are obtained;
the emotion recognition network is used for recognizing and judging the emotion type of the user obtained by the emotion classification submodule and comprises a convolutional neural network layer, a cyclic neural network layer and a full-connection layer; the convolution neural network layer is used for extracting local features of the emotion types of the users, the circulation neural network layer is used for capturing time sequence features of the emotion types of the users, and the full-connection layer is used for outputting probability distribution of the emotion types of the users according to the local features of the emotion types and the time sequence features of the emotion types;
And the emotion recognition result output sub-module is used for selecting the emotion type with the highest probability as the emotion recognition result of the user according to the probability distribution of the emotion type of the user obtained by the emotion recognition network and transmitting the emotion type to the digital intervention management module.
In a preferred embodiment, the digital intervention management module adopts a digital intervention management method based on multi-factor decision, and the method comprises the following steps:
the intervention scheme generation sub-module is used for finding out the intervention scheme which is most matched or closest to the user by calculating the semantic similarity or the semantic distance between the traditional Chinese medicine syndrome features of the user and the intervention scheme classification and definition description according to the traditional Chinese medicine syndrome features of the user and the intervention scheme classification and definition description in the knowledge graph module, which are obtained by the traditional Chinese medicine syndrome differentiation module, wherein the most matched or closest intervention scheme is calculated by adopting the following formula:
，
，
wherein, ，is a set of traditional Chinese medicine syndrome feature vectors, psychological state assessment result vectors and emotion recognition result vectors,is a set of intervention plan classification and definition description vectors,representing the semantic similarity or semantic distance between the traditional Chinese medicine syndrome feature vector, the psychological state assessment result vector or the emotion recognition result vector and the intervention scheme classification and definition description vector, Representing the best matching or closest intervention plan;
the intervention scheme pushing sub-module is used for generating a proper natural language expression based on the emotion recognition result of the user obtained by the emotion recognition module according to the intervention scheme of the user obtained by the intervention scheme generating sub-module and pushing the proper natural language expression to the user; the intervention scheme pushing submodule selects the most suitable pushing strategy and mode according to the emotion type and strength of the user, and the most suitable pushing strategy and mode are calculated by adopting the following formula:
，
wherein,is the emotion type and intensity vector of the user,is a collection of push strategies and modes,among all possible push strategies and modesThe strategy or manner that maximizes the conditional probability,representing emotion type and intensity vector at a given userUnder the condition of selecting a certain push strategy or modeIs a function of the probability of (1),is the most suitable push strategy and mode;
and the intervention effect evaluation sub-module is used for evaluating and optimizing the pushing strategies and modes in the intervention scheme generation sub-module and the intervention scheme pushing sub-module according to the user feedback information and the data obtained by the user feedback module, so that the effect and quality of the intervention scheme are improved.
An operation method for driving a virtual doctor based on a knowledge graph comprises the following steps:
step 101, receiving text or sound signals input by a user, and converting the text or sound signals into data input which can be recognized and processed by a computer system, wherein the data input comprises entities, relations and semantics;
step 102, collecting behavior characteristics of a user in the interaction process with a system, wherein the behavior characteristics comprise facial micro-expressions, limb actions and voice intonation;
step 103, constructing and storing a knowledge graph according to the disease knowledge about depression in the field of Chinese and Western medicine, wherein the knowledge graph comprises semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological states and intervention schemes and semantic relations among the semantic types, and each semantic type has a corresponding description for explaining the concept and the characteristics of the semantic type;
step 104, according to the converted data and disease knowledge in the knowledge graph, generating a question-answer content expression between the doctor and the user, and converting the question-answer content expression into natural language to be output to the user;
step 105, according to the converted data and the syndrome description in the knowledge graph, matching with the corresponding syndrome pattern in the knowledge graph through entity alignment and semantic search, and identifying the traditional Chinese medicine syndrome features of the depressive disorder user;
Step 106, comprehensively evaluating the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content in the converted data, the behavioral characteristics of the user in the system interaction process and the psychological state description in the knowledge graph module;
step 107, identifying and judging the emotion type of the user according to the converted data and interview question and answer contents between doctors and the user;
step 108, calculating a digital intervention management scheme which is most suitable for personalized needs and preferences of the user according to the traditional Chinese medicine syndrome features, psychological states and emotion types of the user obtained in the step, and pushing the digital intervention management scheme to the user;
step 109, after the user completes the pushed intervention management scheme according to the instruction, information and data fed back by the user are collected, and the output and performance of each step are optimized or adjusted.
The invention has the beneficial effects that:
1) The system adopts a knowledge graph as a core knowledge representation and management mode, can effectively store and organize the disease knowledge about depression in the field of Chinese and Western medicine, and improves the integrity, accuracy and timeliness of the knowledge;
2) The system adopts a question and answer method based on the knowledge graph, can generate the content expression of questions and answers between doctors and users according to the questions input by the users and the knowledge of diseases in the knowledge graph, and converts the content expression into natural language to be output to the users, thereby improving the quality of the questions and answers and the satisfaction degree of the users;
3) The system adopts a traditional Chinese medicine syndrome differentiation method based on a knowledge graph, can be matched with corresponding syndrome patterns in the knowledge graph through entity alignment and semantic search according to information input by a user and syndrome descriptions in the knowledge graph, and can identify traditional Chinese medicine syndrome features of a depressive user, thereby improving the accuracy and the effectiveness of traditional Chinese medicine syndrome differentiation;
4) The system adopts a psychological assessment method based on multi-mode data fusion, and can comprehensively assess the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content input by the user, the behavioral characteristics (such as facial micro-expressions, limb actions, voice tones and the like) and the psychological state description in the knowledge graph module shown in the system interaction process, so that the comprehensiveness and objectivity of the psychological assessment are improved;
5) The system adopts the emotion recognition method based on deep learning, can recognize and judge the emotion type of the user according to the text content input by the user and the interview question-answering content in the question-answering system module, and improves the accuracy and sensitivity of emotion recognition;
6) The system adopts the digital intervention management method based on multi-factor decision, can push a personalized digital intervention management scheme to the user according to the traditional Chinese medicine symptom characteristics and emotion recognition results of the user, and improves the suitability and the effectiveness of the digital intervention management scheme;
7) The system adopts a user feedback method based on machine learning, can collect information and data fed back by a user after the user finishes the pushed intervention management scheme according to the instruction of the digital intervention management module, optimizes or adjusts the output and performance of each step, and improves the self-adaptability and the sustainability of the system.
Drawings
Fig. 1 is a system diagram of the present invention.
Fig. 2 is a method diagram of the present invention.
Detailed Description
In order that those skilled in the art will better understand the technical solutions, the following detailed description of the technical solutions is provided with reference to examples, which are exemplary and explanatory only and should not be construed as limiting the scope of the invention in any way.
Example 1:
as shown in fig. 1, the present invention provides a knowledge-graph-based driving virtual doctor system, which comprises the following main modules:
a knowledge-graph-driven virtual doctor system, comprising:
the natural language processing module is used for receiving text or sound signals input by a user and converting the text or sound signals into data input which can be recognized and processed by the computer system, and the data input comprises entities, relations and semantics;
The multi-mode data acquisition module is used for acquiring behavior characteristics, including facial micro-expressions, limb actions and voice intonation, of a user in the interaction process with the system;
the knowledge map module is used for storing and managing disease knowledge about depression in the field of Chinese and Western medicine, and comprises semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological states and intervention schemes and semantic relations among the semantic types, wherein each semantic type has a corresponding description and is used for explaining concepts and features of the semantic type;
the question-answering system module is used for generating a content expression of questions and answers between a doctor and a user according to the data converted by the natural language processing module and disease knowledge in the knowledge graph module, converting the content expression into natural language and outputting the natural language to the user;
the traditional Chinese medicine syndrome differentiation module is used for identifying traditional Chinese medicine syndrome features of a depressive disorder user by matching with corresponding syndrome patterns in the knowledge graph module through entity alignment and semantic search according to the data converted by the natural language processing module and the syndrome description in the knowledge graph module;
the psychological assessment module is used for comprehensively assessing the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content in the data converted by the natural language processing module, the behavioral characteristics acquired by the multi-modal data acquisition module and the psychological state description in the knowledge graph module;
The emotion recognition module is used for recognizing and judging the emotion type of the user according to the data converted by the natural language processing module and interview question and answer contents in the question and answer system module;
the digital intervention management module is used for pushing a personalized digital intervention management scheme to the user according to the traditional Chinese medicine syndrome characteristics, the psychological state evaluation result and the emotion recognition result of the user.
In a preferred implementation case, a user feedback module is provided, and the user feedback module is used for collecting information and data fed back by a user after the user finishes the pushed intervention management scheme according to the instruction of the digital intervention management module, sending the information and data to the natural language module, and optimizing or adjusting the output and performance of each related module based on the user feedback information and data.
In a preferred embodiment, the natural language processing module adopts a deep neural network model, and includes:
the text analysis sub-module is used for performing text analysis tasks of word segmentation, part-of-speech tagging, named entity identification, relation extraction and semantic role tagging on a text signal input by a user, so that entities, relations and semantics in the text are extracted;
and the voice recognition sub-module is used for carrying out acoustic feature extraction, acoustic modeling, language modeling and decoding on the voice signal input by the user so as to convert the voice signal into a text signal, and inputting the text signal into the text analysis sub-module for further processing.
In a preferred embodiment, the knowledge graph module adopts an ontology modeling method, which includes:
the ontology construction submodule is used for defining a set of ontology concepts and attributes according to disease knowledge about depression in the field of Chinese and Western medicine, and representing an ontology structure by using an ontology representation language, wherein the ontology structure comprises classification and definition description of an intervention scheme;
the ontology storage submodule is used for storing the constructed ontology structure in an ontology database and providing an ontology query language to query ontology data;
the ontology reasoning sub-module is used for logically reasoning the data in the ontology database according to an ontology reasoning rule, so as to obtain some implicit knowledge and relations, wherein the ontology reasoning rule is a rule for reasoning and deducing the knowledge in the ontology;
specifically, the ontology-building sub-module defines the following ontology concepts and attributes:
disorders: depression is a psychological disease with the attributes of etiology, symptoms and the like;
the syndrome: depression is a condition of traditional Chinese medicine, and has the properties of definition, symptoms and the like;
the intervention scheme is as follows: psychological consultation is a method for intervention of depression, and has the properties of scheme purpose, scheme content, form and the like;
The ontology storage sub-module stores the ontology structure in an ontology database, and provides a structured query language similar to SQL to query the definition and symptoms of the depression and the content and form of psychological consultation;
the ontology reasoning submodule logically reasoning the data in the ontology database according to the ontology reasoning rule, so as to obtain some implicit knowledge and relations, for example:
if the user has depression, the user has depression;
if the user has depression, the user can accept psychological consultation;
if the user can accept psychological consultation, the user can select different content and forms.
In a preferred implementation case, the question answering system module adopts a question answering method based on a knowledge graph, and the question answering method comprises the following steps:
the problem analysis sub-module is used for analyzing the problems in the data converted by the natural language processing module, including problem classification, problem intention recognition and problem entity linking, so as to obtain the types, intention and related entities of the problems;
the answer generation sub-module is used for generating a content expression of a question and answer between a doctor and a user according to the question information obtained by the question analysis sub-module and the knowledge of the diseases in the knowledge graph module, and converting the content expression into a natural language and outputting the natural language to the user; the answer generation submodule selects different answer generation strategies according to the question types and intentions;
The answer evaluation sub-module is used for evaluating the answers generated by the answer generation sub-module, including answer correctness, answer completeness, answer consistency and answer credibility indexes, so as to obtain quality scores of the answers;
the answer optimizing sub-module is used for optimizing or adjusting the answer generating strategy and parameters in the answer generating sub-module according to the answer quality score obtained by the answer evaluating sub-module and the feedback data in the user feedback module, so that the answer quality and the user satisfaction are improved;
the question and answer based on the knowledge graph is a method for answering natural language questions by using structured data and semantic relations in the knowledge graph. Knowledge-graph-based questions and answers can be expressed as:
Q → P → A ，
wherein Q is a question in the data converted by the natural language processing module, P is a query sentence written by the ontology query language in the knowledge graph module, and A is an answer; the problem analysis submodule is a process of converting Q into P and comprises the following steps:
problem classification: q is divided into different types, such as a real type, a definition type, a column phenotype and the like;
problem intent recognition: mapping the keywords or phrases in the Q to concepts or attributes in the knowledge graph, so as to obtain the intention of the Q;
Problem entity linking: matching and linking the named entity in the Q with the instance in the knowledge graph, so as to obtain a related entity of the Q;
the answer generation sub-module is a process of converting P into A, and comprises the following steps:
answer retrieval: inquiring in the knowledge graph according to the P, and returning data or knowledge matched with the P as a first candidate answer;
answer reasoning: carrying out logic reasoning on the first candidate answer according to the P and the reasoning rules or algorithms in the knowledge graph, and returning the data or knowledge after reasoning as a second candidate answer;
answer generation: and generating a proper natural language expression as a final answer according to the second candidate answer and the question type and intention.
Under the preferred implementation condition, the traditional Chinese medicine syndrome differentiation module adopts a traditional Chinese medicine syndrome differentiation method based on a knowledge graph, and comprises the following steps:
the entity alignment sub-module is used for aligning the entity in the data converted by the natural language processing module, namely, matching and mapping the entity with the ontology concept in the knowledge graph module so as to obtain the ontology type and attribute of the entity;
the semantic search sub-module is used for finding out candidate answers of the syndromes matched with or close to the entity by calculating the semantic similarity or the semantic distance between the entity and the syndromes according to the entity information obtained by the entity alignment sub-module and the syndrome description in the knowledge graph module;
The syndrome identification sub-module is used for determining traditional Chinese medicine syndrome characteristics of the depressive disorder user by applying a syndrome identification rule or algorithm according to the semantic relation between the syndrome candidate answers obtained by the semantic search sub-module and the syndromes in the knowledge graph module;
the traditional Chinese medicine syndrome differentiation based on the knowledge graph is a method for carrying out traditional Chinese medicine syndrome differentiation by utilizing structured data and semantic relations in the knowledge graph; the traditional Chinese medicine syndrome differentiation based on the knowledge graph can be expressed as follows:
N → S → Y
wherein E is an entity set in the data converted by the natural language processing module, S is a syndrome set matched with or close to the entity, and Z is a first traditional Chinese medicine syndrome feature of the user.
The entity alignment sub-module is a process of converting N into S, and includes the following steps:
entity classification: n is classified into different types such as symptoms, emotion, behavior, etc.;
entity mapping: and matching and mapping each entity in the N with the ontology concept in the knowledge graph, so as to obtain the ontology type and attribute of the entity.
The semantic search sub-module is a process of converting S into Y and comprises the following steps:
semantic similarity calculation: according to the syndrome classification and definition description in the knowledge graph, calculating the semantic similarity or semantic distance between each entity and each syndrome in the E, thereby obtaining a similarity or distance matrix;
Semantic ordering: according to the similarity or distance matrix, sequencing the syndromes corresponding to each entity, and selecting a plurality of syndromes which are the most similar or closest to each other as a candidate set;
the syndrome recognition submodule is a process of converting Y into a second Chinese medical syndrome feature Y', and comprises the following steps:
semantic relationship analysis: according to semantic relations among the syndromes in the knowledge graph, analyzing whether the relations such as logical compatibility, conflict or inclusion exist between every two syndromes in the candidate set;
and (3) screening syndrome: screening and adjusting the candidate set according to the semantic relation analysis result, and removing the syndrome which does not accord with logic or is irrelevant;
syndrome combination: and combining one or more traditional Chinese medicine syndromes according with the disease characteristics of the user according to the screened candidate set.
In a preferred embodiment, the psychological assessment module adopts a psychological assessment method based on multi-modal data fusion, and the psychological assessment method comprises the following steps:
the text emotion analysis submodule is used for carrying out emotion analysis on text content in the data converted by the natural language processing module, wherein the text content comprises emotion tendency and emotion strength, so that emotion characteristics of a user reflected by the text content are obtained;
The psychological characteristic analysis sub-module is used for analyzing the behavioral characteristics of the user acquired by the multi-mode data acquisition module so as to obtain psychological characteristics of the user reflected by the behavioral characteristics;
the data fusion sub-module is used for carrying out data fusion on the emotion characteristics and the psychological characteristics of the user, which are obtained by the text emotion analysis sub-module and the psychological characteristic analysis sub-module, and comprises data alignment, data fusion and data mapping, so that the comprehensive psychological characteristics of the user are obtained;
and the psychological state evaluation sub-module is used for determining the psychological state of the user by calculating the semantic similarity or the semantic distance between the comprehensive psychological features of the user and the psychological state description according to the comprehensive psychological features of the user and the psychological state description in the knowledge graph module obtained by the data fusion sub-module.
In a preferred implementation case, the emotion recognition module adopts an emotion recognition method based on deep learning, and the emotion recognition method comprises the following steps:
the emotion classification sub-module is used for performing emotion classification on the text content in the data converted by the natural language processing module and the interview question and answer content in the question and answer system module, wherein the emotion classification sub-module comprises emotion intensity and emotion category, so that emotion types of the users reflected by the text content and the interview question and answer content are obtained;
The emotion recognition network is used for recognizing and judging the emotion type of the user obtained by the emotion classification submodule and comprises a convolutional neural network layer, a cyclic neural network layer and a full-connection layer; the convolution neural network layer is used for extracting local features of the emotion types of the users, the circulation neural network layer is used for capturing time sequence features of the emotion types of the users, and the full-connection layer is used for outputting probability distribution of the emotion types of the users according to the local features of the emotion types and the time sequence features of the emotion types;
and the emotion recognition result output sub-module is used for selecting the emotion type with the highest probability as the emotion recognition result of the user according to the probability distribution of the emotion type of the user obtained by the emotion recognition network and transmitting the emotion type to the digital intervention management module.
In a preferred embodiment, the digital intervention management module adopts a digital intervention management method based on multi-factor decision, and the method comprises the following steps:
the intervention scheme generation sub-module is used for finding out the intervention scheme which is most matched or closest to the user by calculating the semantic similarity or the semantic distance between the traditional Chinese medicine syndrome features of the user and the intervention scheme classification and definition description according to the traditional Chinese medicine syndrome features of the user and the intervention scheme classification and definition description in the knowledge graph module, which are obtained by the traditional Chinese medicine syndrome differentiation module, wherein the most matched or closest intervention scheme is calculated by adopting the following formula:
，
，
Wherein, ，is a set of traditional Chinese medicine syndrome feature vectors, psychological state assessment result vectors and emotion recognition result vectors,is a set of intervention plan classification and definition description vectors,representing the semantic similarity or semantic distance between the traditional Chinese medicine syndrome feature vector, the psychological state assessment result vector or the emotion recognition result vector and the intervention scheme classification and definition description vector,representing the best matching or closest intervention plan;
the intervention scheme pushing sub-module is used for generating a proper natural language expression based on the emotion recognition result of the user obtained by the emotion recognition module according to the intervention scheme of the user obtained by the intervention scheme generating sub-module and pushing the proper natural language expression to the user; the intervention scheme pushing submodule selects the most suitable pushing strategy and mode according to the emotion type and strength of the user, and the most suitable pushing strategy and mode are calculated by adopting the following formula:
，
wherein,is the emotion type and intensity vector of the user,is a collection of push strategies and modes,among all possible push strategies and modesThe strategy or manner that maximizes the conditional probability,representing emotion type and intensity vector at a given user Under the condition of selecting a certain push strategy or modeIs a function of the probability of (1),is the most suitable push strategy and mode;
and the intervention effect evaluation sub-module is used for evaluating and optimizing the pushing strategies and modes in the intervention scheme generation sub-module and the intervention scheme pushing sub-module according to the user feedback information and the data obtained by the user feedback module, so that the effect and quality of the intervention scheme are improved.
The digital intervention management based on the knowledge graph is a method for carrying out digital intervention management by utilizing structured data and semantic relations in the knowledge graph. Knowledge-graph-based digital intervention management can be expressed as:
Z → G → P ，
wherein Z is a set of traditional Chinese medicine syndrome features, psychological state assessment results and emotion recognition results of a user, G is an intervention scheme, and P is an intervention effect of the user; the intervention scheme generation submodule is a process of converting Z into G and comprises the following steps:
semantic similarity calculation: according to the intervention scheme classification and definition description in the knowledge graph, calculating the semantic similarity or semantic distance between Z and each intervention scheme, thereby obtaining a similarity or distance matrix;
semantic ordering: according to the similarity or distance matrix, sorting the intervention schemes corresponding to each syndrome, and selecting a plurality of intervention schemes which are most similar or closest to each other as a candidate set;
And (3) selecting an intervention scheme: and selecting one or more intervention schemes meeting the requirements and conditions of the user according to the candidate set and factors such as personal preference, illness degree and living habit of the user.
The intervention scheme pushing submodule is a process of converting G into P and comprises the following steps:
and (3) generating intervention content: generating proper natural language expression according to the specific content and steps contained in G, and adding some necessary description and prompt;
and (3) selecting an intervention mode: according to the emotion type and intensity of the user obtained by the emotion recognition module, different pushing strategies and modes are selected, such as active pushing, passive pushing, timing pushing or real-time pushing;
and (3) feedback of intervention effect: and collecting feedback information and data of the user according to the actions of the user on reading, clicking, commenting and the like of the intervention content, and transmitting the feedback information and data to a user feedback module.
In a preferred embodiment, the user feedback module adopts a user feedback method based on machine learning, including:
the feedback data acquisition sub-module is used for acquiring information and data fed back by the user after the user finishes the pushed intervention management scheme according to the instruction of the digital intervention management module, wherein the information and data comprise satisfaction degree of the user on the intervention scheme, evaluation of the user on a system, illness state change of the user and psychological state change of the user;
The feedback data analysis sub-module is used for analyzing the information and the data fed back by the user, which are obtained by the feedback data acquisition sub-module, and comprises feedback data cleaning, feedback data statistics, feedback data mining and feedback data visualization, so that the characteristics and rules of the feedback of the user are obtained;
the feedback data application sub-module is used for optimizing or adjusting the output and performance of each related module according to the characteristics and rules of the user feedback obtained by the feedback data analysis sub-module, and comprises a natural language processing module, a knowledge graph module, a question-answering system module, a traditional Chinese medicine syndrome differentiation module, a psychological assessment module, a emotion recognition module and a digital intervention management module; the feedback data application sub-module improves the performance and user experience of the system according to different optimization or adjustment targets and methods.
Example 2:
as shown in fig. 2, an operation method for driving a virtual doctor based on a knowledge graph includes the following steps:
step 101, receiving text or sound signals input by a user, and converting the text or sound signals into data input which can be recognized and processed by a computer system, wherein the data input comprises entities, relations and semantics;
the user first accesses the virtual doctor system through the mobile phone or the computer and the like, and inputs the problem that the user wants to consult or seek help. For example, he has entered "how is i always frustrated recently, has little power? "
The system receives the user's text signal and converts it into data input, including entities (e.g., "me," "depression," "power," etc.), relationships (e.g., "feel," "none," etc.), and semantics (e.g., "help," "depression," etc.).
Step 102, collecting behavior characteristics of a user in the interaction process with a system, wherein the behavior characteristics comprise facial micro-expressions, limb actions and voice intonation;
the system captures the user's facial expression through a camera or other sensor as sad, eye-less, shoulder-sagged, gesture-weak, etc., and analyzes and identifies it.
Step 103, constructing and storing a knowledge graph according to the disease knowledge about depression in the field of Chinese and Western medicine, wherein the knowledge graph comprises semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological states and intervention schemes and semantic relations among the semantic types, and each semantic type has a corresponding description for explaining the concept and the characteristics of the semantic type;
the system constructs and stores a knowledge graph according to the prior knowledge of the disease of the traditional Chinese and Western medicine field about depression, wherein the knowledge graph comprises the following contents:
classification of syndrome: the traditional Chinese medicine symptoms of depression can be divided into qi depression, liver depression, spleen deficiency, heart blood deficiency, kidney yin deficiency and the like, and each symptom has corresponding symptoms and treatment methods;
Semantic types of mental states: the psychological states of depression can be categorized into sadness, anxiety, fear, anger, guilt, self-responsibility, sped, forfex, etc., each psychological state having its corresponding psychological state description for describing the characteristics and effects of the psychological state;
semantic relationship: various semantic relationships exist between entities and semantics in the knowledge graph, such as belonging to, containing, causing, affecting, similar, etc., for expressing logical relationships and inference rules in the knowledge graph.
Step 104, according to the converted data and disease knowledge in the knowledge graph, generating a question-answer content expression between the doctor and the user, and converting the question-answer content expression into natural language to be output to the user;
step 105, according to the converted data and the syndrome description in the knowledge graph, matching with the corresponding syndrome pattern in the knowledge graph through entity alignment and semantic search, and identifying the traditional Chinese medicine syndrome features of the depressive disorder user;
when the information input by the user comprises entities and semantics such as insomnia, depression, mood swings, anorexia and the like;
one pattern of symptoms in the knowledge graph is 'qi depression', which is described as 'depression caused by qi movement disorder', and the corresponding symptoms include insomnia, chest distress, hypochondriac pain and the like;
The system discovers that the information input by the user is partially matched with the pattern of the 'qi depression' syndrome through entity alignment and semantic search;
the pattern of the knowledge pattern is also known as liver depression, which is described as "emotional disorder caused by liver qi disorder", and the corresponding symptoms include emotional fluctuation, irritability, depressed emotion, depression and the like;
the system discovers that the information input by the user has higher matching degree with the syndrome pattern of liver depression through entity alignment and semantic search, and completely accords with the pattern of liver depression;
the syndrome recognition module comprehensively compares the matching degree of the two syndrome patterns and judges that the traditional Chinese medicine syndrome features of the user are liver depression.
Step 106, comprehensively evaluating the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content in the converted data, the behavioral characteristics of the user in the system interaction process and the psychological state description in the knowledge graph module;
the text content input by the user contains the semantics of psychological states such as sadness, forfex, self-responsibility and the like;
the facial micro-expressions of the user in the interaction process with the system are sadness, pain, fear and the like;
the limb movements exhibited by the user in the process of interacting with the system are low head, shoulder shrinking, fist tightening, etc.;
The voice intonation of the user in the process of interacting with the system is low, slow, weak and the like;
the knowledge map module has a psychological state of depression, which is described as negative emotion generated by excessively negative views of the knowledge map module and the world, and the corresponding characteristics of the knowledge map module include low emotion, lack of interest, self-responsibility and the like;
by analyzing the data, the system comprehensively evaluates the psychological health index of the user to 40 (which is divided into 100), which means that the psychological health condition of the user is poor and needs to be improved.
Step 107, identifying and judging the emotion type of the user according to the converted data and interview question and answer contents between doctors and the user;
the data and the question-answering contents input by the user contain the semantics of emotion types such as sadness, anxiety, fear, anger and the like;
the system judges the emotion type of the user to be negative emotion through semantic analysis and emotion recognition technology, and gives out corresponding emotion intensity and duration;
the emotion judging module gives corresponding emotion management advice, such as respiration adjustment, relaxation training, positive thinking and the like, according to the emotion type and intensity of the user.
Step 108, calculating a digital intervention management scheme which is most suitable for personalized needs and preferences of the user according to the traditional Chinese medicine syndrome features, psychological states and emotion types of the user obtained in the step, and pushing the digital intervention management scheme to the user;
The emotion of the user is characterized by 'depression', the corresponding intervention method is 'improving active emotion', and the corresponding digital intervention scheme is 'music relaxation';
the psychological health index of the user is 40, the corresponding intervention method is 'changing negative thinking', and the corresponding digital intervention scheme is 'positive training';
the intervention scheme recommending module calculates an optimized combination scheme by combining two digital intervention schemes according to the personalized requirements and preferences of the user, and pushes the optimized combination scheme to the user.
Step 109, after the user completes the pushed intervention management scheme according to the instruction, collecting information and data fed back by the user, and optimizing or adjusting the output and performance of each step;
after the music relaxing application program is finished, the user feedback says that he likes to listen to classical music and feels relaxed and comfortable;
after the user finishes the positive-concept training application program, the user feeds back that he learns some cognitive reconstruction skills and feels more positive to himself and the world;
the system stores the information and data fed back by the user in a database, and analyzes and processes the information and the data;
the system optimizes or adjusts each step according to the information and data fed back by the user, such as adding or deleting some entities, relations, semantics, syndromes, psychological states and the like; modifying or updating some question and answer content expressions, syndrome descriptions, psychological state descriptions and the like; improving or optimizing some matching algorithms, evaluation methods, recommendation strategies, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The principles and embodiments of the present invention are described herein by applying specific examples, and the above examples are only used to help understand the method and core idea of the present invention. The foregoing is merely illustrative of the preferred embodiments of this invention, and it is noted that there is objectively no limit to the specific structure disclosed herein, since numerous modifications, adaptations and variations can be made by those skilled in the art without departing from the principles of the invention, and the above-described features can be combined in any suitable manner; such modifications, variations and combinations, or the direct application of the inventive concepts and aspects to other applications without modification, are contemplated as falling within the scope of the present invention.

Claims (8)

1. A knowledge-graph-driven virtual doctor system, comprising:
the natural language processing module is used for receiving text or sound signals input by a user and converting the text or sound signals into data input which can be recognized and processed by the computer system, and the data input comprises entities, relations and semantics;
the multi-mode data acquisition module is used for acquiring behavior characteristics, including facial micro-expressions, limb actions and voice intonation, of a user in the interaction process with the system;
the knowledge map module is used for storing and managing disease knowledge about depression in the field of Chinese and Western medicine, and comprises semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological states and intervention schemes and semantic relations among the semantic types, wherein each semantic type has a corresponding description and is used for explaining concepts and features of the semantic type;
the knowledge graph module adopts an ontology modeling method, and comprises the following steps:
the ontology construction submodule is used for defining a set of ontology concepts and attributes according to disease knowledge about depression in the field of Chinese and Western medicine, and representing an ontology structure by using an ontology representation language, wherein the ontology structure comprises classification and definition description of an intervention scheme;
The ontology storage submodule is used for storing the constructed ontology structure in an ontology database and providing an ontology query language to query ontology data;
the ontology reasoning sub-module is used for logically reasoning the data in the ontology database according to an ontology reasoning rule, so as to obtain some implicit knowledge and relations, wherein the ontology reasoning rule is a rule for reasoning and deducing the knowledge in the ontology;
the question-answering system module is used for generating a content expression of questions and answers between a doctor and a user according to the data converted by the natural language processing module and disease knowledge in the knowledge graph module, converting the content expression into natural language and outputting the natural language to the user;
the traditional Chinese medicine syndrome differentiation module is used for identifying traditional Chinese medicine syndrome features of a depressive disorder user by matching with corresponding syndrome patterns in the knowledge graph module through entity alignment and semantic search according to the data converted by the natural language processing module and the syndrome description in the knowledge graph module;
the psychological assessment module is used for comprehensively assessing the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content in the data converted by the natural language processing module, the behavioral characteristics acquired by the multi-modal data acquisition module and the psychological state description in the knowledge graph module;
The emotion recognition module is used for recognizing and judging the emotion type of the user according to the data converted by the natural language processing module and interview question and answer contents in the question and answer system module;
the digital intervention management module is used for pushing a personalized digital intervention management scheme to the user according to the traditional Chinese medicine syndrome characteristics, the psychological state assessment result and the emotion recognition result of the user;
the digital intervention management module adopts a digital intervention management method based on multi-factor decision, and comprises the following steps:
the intervention scheme generation sub-module is used for finding out the intervention scheme which is most matched or closest to the user by calculating the semantic similarity or the semantic distance between the traditional Chinese medicine syndrome features of the user and the intervention scheme classification and definition description according to the traditional Chinese medicine syndrome features of the user and the intervention scheme classification and definition description in the knowledge graph module, which are obtained by the traditional Chinese medicine syndrome differentiation module, wherein the most matched or closest intervention scheme is calculated by adopting the following formula:
，
，
wherein, ，/>is the set of Chinese medicine syndrome feature vector, psychological state evaluation result vector and emotion recognition result vector, and is->Is a set of intervention plan classification and definition description vectors, < > >Representing the characteristic vector of the syndrome of Chinese medicineSemantic similarity or semantic distance between mental state assessment result vector or emotion recognition result vector and intervention plan classification and definition description vector, +.>Representing the best matching or closest intervention plan;
the intervention scheme pushing sub-module is used for generating a proper natural language expression based on the emotion recognition result of the user obtained by the emotion recognition module according to the intervention scheme of the user obtained by the intervention scheme generating sub-module and pushing the proper natural language expression to the user; the intervention scheme pushing submodule selects the most suitable pushing strategy and mode according to the emotion type and strength of the user, and the most suitable pushing strategy and mode are calculated by adopting the following formula:
，
wherein,is the emotion type and intensity vector of the user, +.>Is a set of push policies and modes, +.>Represent +.>To find out which strategy or way to maximize conditional probability, +.>Representing the emotion type and intensity vector of a given user +.>Under the condition of (1) selecting a certainPersonal push strategy or mode->Probability of->Is the most suitable push strategy and mode;
and the intervention effect evaluation sub-module is used for evaluating and optimizing the pushing strategies and modes in the intervention scheme generation sub-module and the intervention scheme pushing sub-module according to the user feedback information and the data obtained by the user feedback module, so that the effect and quality of the intervention scheme are improved.
2. The knowledge-graph-driven virtual doctor system according to claim 1, wherein a user feedback module is provided for collecting information and data fed back by a user after the user indicates completion of the pushed intervention management scheme according to the digital intervention management module, and sending the information and data to the natural language module, and optimizing or adjusting the output and performance of each relevant module based on the user feedback information and data.
3. The knowledge-based on-graph-driven virtual doctor system according to claim 1, wherein the natural language processing module adopts a deep neural network model, and the system comprises:
the text analysis sub-module is used for performing text analysis tasks of word segmentation, part-of-speech tagging, named entity identification, relation extraction and semantic role tagging on a text signal input by a user, so that entities, relations and semantics in the text are extracted;
and the voice recognition sub-module is used for carrying out acoustic feature extraction, acoustic modeling, language modeling and decoding on the voice signal input by the user so as to convert the voice signal into a text signal, and inputting the text signal into the text analysis sub-module for further processing.
4. The knowledge-based on-graph-driven virtual doctor system according to claim 2, wherein the question-answering system module adopts a knowledge-graph-based question-answering method, comprising:
the problem analysis sub-module is used for analyzing the problems in the data converted by the natural language processing module, including problem classification, problem intention recognition and problem entity linking tasks, so as to obtain the types, intention and related entities of the problems;
the answer generation sub-module is used for generating a content expression of a question and answer between a doctor and a user according to the question information obtained by the question analysis sub-module and the knowledge of the diseases in the knowledge graph module, and converting the content expression into a natural language and outputting the natural language to the user; the answer generation submodule selects different answer generation strategies according to the question types and intentions;
the answer evaluation sub-module is used for evaluating the answers generated by the answer generation sub-module, including answer correctness, answer completeness, answer consistency and answer credibility indexes, so as to obtain quality scores of the answers;
and the answer optimizing sub-module is used for optimizing or adjusting the answer generating strategy and parameters in the answer generating sub-module according to the answer quality scores obtained by the answer evaluating sub-module and the feedback data in the user feedback module, so that the answer quality and the user satisfaction are improved.
5. The knowledge-based driving virtual doctor system according to claim 1, wherein the diagnosis module of the traditional Chinese medicine adopts a diagnosis method of the traditional Chinese medicine based on the knowledge spectrum, and the method comprises the following steps:
the entity alignment sub-module is used for aligning the entity in the data converted by the natural language processing module, namely, matching and mapping the entity with the ontology concept in the knowledge graph module so as to obtain the ontology type and attribute of the entity;
the semantic search sub-module is used for finding out candidate answers of the syndromes matched with or close to the entity by calculating the semantic similarity or the semantic distance between the entity and the syndromes according to the entity information obtained by the entity alignment sub-module and the syndrome description in the knowledge graph module;
and the syndrome identification sub-module is used for determining traditional Chinese medicine syndrome characteristics of the depressive disorder user by applying a syndrome identification rule or algorithm according to the semantic relation between the syndrome candidate answers obtained by the semantic search sub-module and the syndromes in the knowledge graph module.
6. The knowledge-based on-graph-driven virtual doctor system according to claim 1, wherein the psychological assessment module adopts a psychological assessment method based on multi-modal data fusion, and the method comprises the steps of:
The text emotion analysis submodule is used for carrying out emotion analysis on text content in the data converted by the natural language processing module, wherein the text content comprises emotion tendency and emotion strength, so that emotion characteristics of a user reflected by the text content are obtained;
the psychological characteristic analysis sub-module is used for analyzing the behavioral characteristics of the user acquired by the multi-mode data acquisition module so as to obtain psychological characteristics of the user reflected by the behavioral characteristics;
the data fusion sub-module is used for carrying out data fusion on the emotion characteristics and the psychological characteristics of the user, which are obtained by the text emotion analysis sub-module and the psychological characteristic analysis sub-module, and comprises data alignment, data fusion and data mapping, so that the comprehensive psychological characteristics of the user are obtained;
and the psychological state evaluation sub-module is used for determining the psychological state of the user by calculating the semantic similarity or the semantic distance between the comprehensive psychological features of the user and the psychological state description according to the comprehensive psychological features of the user and the psychological state description in the knowledge graph module obtained by the data fusion sub-module.
7. The knowledge-graph-based driving virtual doctor system according to claim 1, wherein the emotion recognition module adopts a deep learning-based emotion recognition method, comprising:
The emotion classification sub-module is used for performing emotion classification on the text content in the data converted by the natural language processing module and the interview question and answer content in the question and answer system module, wherein the emotion classification sub-module comprises emotion intensity and emotion category, so that emotion types of the users reflected by the text content and the interview question and answer content are obtained;
the emotion recognition network is used for recognizing and judging the emotion type of the user obtained by the emotion classification submodule and comprises a convolutional neural network layer, a cyclic neural network layer and a full-connection layer; the convolution neural network layer is used for extracting local features of the emotion types of the users, the circulation neural network layer is used for capturing time sequence features of the emotion types of the users, and the full-connection layer is used for outputting probability distribution of the emotion types of the users according to the local features of the emotion types and the time sequence features of the emotion types;
and the emotion recognition result output sub-module is used for selecting the emotion type with the highest probability as the emotion recognition result of the user according to the probability distribution of the emotion type of the user obtained by the emotion recognition network and transmitting the emotion type to the digital intervention management module.
8. The operation method for driving the virtual doctor based on the knowledge graph is characterized by comprising the following steps of:
Step 101, receiving text or sound signals input by a user, and converting the text or sound signals into data input which can be recognized and processed by a computer system, wherein the data input comprises entities, relations and semantics;
step 102, collecting behavior characteristics of a user in the interaction process with a system, wherein the behavior characteristics comprise facial micro-expressions, limb actions and voice intonation;
step 103, constructing and storing a knowledge graph according to the disease knowledge about depression in the field of Chinese and Western medicine, wherein the knowledge graph comprises semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological states and intervention schemes and semantic relations among the semantic types, and each semantic type has a corresponding description for explaining the concept and the characteristics of the semantic type;
step 103 adopts an ontology modeling method, which comprises the following steps:
defining a set of ontology concepts and attributes according to disease knowledge about depression in the field of Chinese and Western medicine, and representing an ontology structure by using an ontology representation language, wherein the ontology structure comprises classification and definition description of an intervention scheme;
storing the constructed ontology structure in an ontology database, and providing an ontology query language to query the ontology data;
carrying out logical reasoning on the data in the ontology database according to the ontology reasoning rule, so as to obtain some implicit knowledge and relations, wherein the ontology reasoning rule is a rule for reasoning and deducing the knowledge in the ontology;
Step 104, according to the converted data and disease knowledge in the knowledge graph, generating a question-answer content expression between the doctor and the user, and converting the question-answer content expression into natural language to be output to the user;
step 105, according to the converted data and the syndrome description in the knowledge graph, matching with the corresponding syndrome pattern in the knowledge graph through entity alignment and semantic search, and identifying the traditional Chinese medicine syndrome features of the depressive disorder user;
step 106, comprehensively evaluating the psychological state of the user by analyzing the psychological characteristics of the user reflected by the text content in the converted data and the behavioral characteristics and psychological state description in the knowledge graph module shown by the user in the system interaction process;
step 107, identifying and judging the emotion type of the user according to the converted data and interview question and answer contents between doctors and the user;
step 108, calculating a digital intervention management scheme which is most suitable for personalized needs and preferences of the user according to the traditional Chinese medicine syndrome features, psychological states and emotion types of the user obtained in the step, and pushing the digital intervention management scheme to the user;
step 108 adopts a digital intervention management method based on multi-factor decision, and comprises the following steps:
According to the traditional Chinese medicine syndrome features and the intervention scheme classification and definition description of the user, the semantic similarity or semantic distance between the traditional Chinese medicine syndrome features and the intervention scheme classification and definition description of the user is calculated, so that the intervention scheme which is the best match or closest to the user is found out, and the best match or closest intervention scheme is calculated by adopting the following formula:
，
，
wherein, ，/>is the set of Chinese medicine syndrome feature vector, psychological state evaluation result vector and emotion recognition result vector, and is->Is a set of intervention plan classification and definition description vectors, < >>Representing semantic similarity or semantic distance between traditional Chinese medicine syndrome feature vector, psychological state evaluation result vector or emotion recognition result vector and intervention plan classification and definition description vector, +.>Representing the best matching or closest intervention plan;
according to the intervention scheme of the user, based on the emotion recognition result of the user, generating a proper natural language expression and pushing the natural language expression to the user; the method comprises the steps of selecting the most suitable pushing strategy and mode according to the emotion type and strength of a user, wherein the most suitable pushing strategy and mode are calculated by adopting the following formula:
，
wherein,is the emotion type and intensity vector of the user, +. >Is a set of push policies and modes, +.>Represent +.>To find out which strategy or way to maximize conditional probability, +.>Representing the emotion type and intensity vector of a given user +.>Under the condition of (1) selecting a certain push strategy or mode->Probability of->Is the most suitable push strategy and mode;
according to the feedback information and data of the user, the pushing strategy and mode are evaluated and optimized, so that the effect and quality of the intervention scheme are improved;
step 109, after the user completes the pushed intervention management scheme according to the instruction, information and data fed back by the user are collected, and the output and performance of each step are optimized or adjusted.
CN202311301744.2A 2023-10-10 2023-10-10 Knowledge graph driving-based virtual doctor system and operation method thereof Active CN117056536B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311301744.2A CN117056536B (en) 2023-10-10 2023-10-10 Knowledge graph driving-based virtual doctor system and operation method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311301744.2A CN117056536B (en) 2023-10-10 2023-10-10 Knowledge graph driving-based virtual doctor system and operation method thereof

Publications (2)

Publication Number Publication Date
CN117056536A CN117056536A (en) 2023-11-14
CN117056536B true CN117056536B (en) 2023-12-26

Family

ID=88662882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311301744.2A Active CN117056536B (en) 2023-10-10 2023-10-10 Knowledge graph driving-based virtual doctor system and operation method thereof

Country Status (1)

Country Link
CN (1) CN117056536B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111462841A (en) * 2020-03-12 2020-07-28 华南理工大学 Depression intelligent diagnosis device and system based on knowledge graph
CN111916181A (en) * 2020-07-23 2020-11-10 南京汇心创悦心理咨询服务有限公司 Psychological construction system and device based on artificial intelligence
CN112732923A (en) * 2020-11-13 2021-04-30 哈尔滨工业大学 Express mail logistics service semantic extraction method based on knowledge graph
CN113571184A (en) * 2021-07-20 2021-10-29 武汉情智感知科技有限公司 Dialogue interaction design method and system for mental health assessment
CN114334163A (en) * 2022-01-25 2022-04-12 思必驰科技股份有限公司 Depression diagnosis dialogue data set generation method, electronic device and storage medium
WO2022174161A1 (en) * 2021-02-14 2022-08-18 Lan Zou Systems and methods for psychotherapy using artificial intelligence
CN116616770A (en) * 2023-04-18 2023-08-22 厦门诸格量科技有限公司 Multimode depression screening and evaluating method and system based on voice semantic analysis

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180107940A1 (en) * 2010-04-27 2018-04-19 Jeremy Lieberman Artificial intelligence method and apparatus
WO2017112813A1 (en) * 2015-12-22 2017-06-29 Sri International Multi-lingual virtual personal assistant
EP3223180A1 (en) * 2016-03-24 2017-09-27 Fujitsu Limited A system and a method for assessing patient risk using open data and clinician input
WO2018205224A1 (en) * 2017-05-11 2018-11-15 Microsoft Technology Licensing, Llc Assisting psychological cure in automated chatting
US20220261668A1 (en) * 2021-02-12 2022-08-18 Tempus Labs, Inc. Artificial intelligence engine for directed hypothesis generation and ranking

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111462841A (en) * 2020-03-12 2020-07-28 华南理工大学 Depression intelligent diagnosis device and system based on knowledge graph
CN111916181A (en) * 2020-07-23 2020-11-10 南京汇心创悦心理咨询服务有限公司 Psychological construction system and device based on artificial intelligence
CN112732923A (en) * 2020-11-13 2021-04-30 哈尔滨工业大学 Express mail logistics service semantic extraction method based on knowledge graph
WO2022174161A1 (en) * 2021-02-14 2022-08-18 Lan Zou Systems and methods for psychotherapy using artificial intelligence
CN113571184A (en) * 2021-07-20 2021-10-29 武汉情智感知科技有限公司 Dialogue interaction design method and system for mental health assessment
CN114334163A (en) * 2022-01-25 2022-04-12 思必驰科技股份有限公司 Depression diagnosis dialogue data set generation method, electronic device and storage medium
CN116616770A (en) * 2023-04-18 2023-08-22 厦门诸格量科技有限公司 Multimode depression screening and evaluating method and system based on voice semantic analysis

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
基于知识图谱的抑郁症自动问答系统研究;田迎;单娅辉;王时绘;;湖北大学学报(自然科学版)(第05期);全文 *
基于道路法规知识图谱的多轮自动问答研究;陈金菊;王义真;欧石燕;;现代情报(第08期);全文 *
情绪识别研究的学术场域――基于CiteSpace的科学知识图谱分析;丁汉青;刘念;;新闻大学(第02期);全文 *

Also Published As

Publication number Publication date
CN117056536A (en) 2023-11-14

Similar Documents

Publication Publication Date Title
JP7199451B2 (en) Emotional interaction system, device and method based on emotional computing user interface
CN112863630A (en) Personalized accurate medical question-answering system based on data and knowledge
CN111415740A (en) Method and device for processing inquiry information, storage medium and computer equipment
CN110110169A (en) Man-machine interaction method and human-computer interaction device
Teng et al. Automatic medical code assignment via deep learning approach for intelligent healthcare
Han et al. A review on sentiment discovery and analysis of educational big‐data
CN112667799B (en) Medical question-answering system construction method based on language model and entity matching
KR20210015010A (en) System and Method for Analyzing Emotion in Text using Psychological Counseling data
CN113871020B (en) Health management method and system after critical illness diagnosis based on AI machine learning
CN112164477A (en) Intelligent question-answering system for hypertension patient based on knowledge graph and establishing method thereof
Gudivada et al. A literature review on machine learning based medical information retrieval systems
Zhang et al. Holistic affect recognition using panda: Paralinguistic non-metric dimensional analysis
CN116910172A (en) Follow-up table generation method and system based on artificial intelligence
Zhou et al. Tamfn: Time-aware attention multimodal fusion network for depression detection
Mao et al. An LSTM&Topic-CNN model for classification of online Chinese medical questions
Livermore et al. Law search as prediction
CN117056536B (en) Knowledge graph driving-based virtual doctor system and operation method thereof
Alibasa et al. Predicting Mood from Digital Footprints Using Frequent Sequential Context Patterns Features
CN116956934A (en) Task processing method, device, equipment and storage medium
US20230031514A1 (en) Methods and systems for arranging and displaying guided recommendations via a graphical user interface based on biological extraction
Nielsen Embodied, embedded, and enactive psychopathology: Reimagining mental disorder
Huang et al. Exploring the effect of emotions in human–machine dialog: an approach toward integration of emotional and rational information
CN114496231A (en) Constitution identification method, apparatus, equipment and storage medium based on knowledge graph
Chawda et al. Learning to recommend descriptive tags for health seekers using deep learning
Das et al. Application of neural network and machine learning in mental health diagnosis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant