CN111597308A - Knowledge graph-based voice question-answering system and application method thereof - Google Patents
Knowledge graph-based voice question-answering system and application method thereof Download PDFInfo
- Publication number
- CN111597308A CN111597308A CN202010422420.4A CN202010422420A CN111597308A CN 111597308 A CN111597308 A CN 111597308A CN 202010422420 A CN202010422420 A CN 202010422420A CN 111597308 A CN111597308 A CN 111597308A
- Authority
- CN
- China
- Prior art keywords
- information
- voice
- data
- voice recognition
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3343—Query execution using phonetics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
Abstract
The invention discloses a knowledge graph-based voice question-answering system and an application method thereof.A terminal application module is used for recording voice and displaying acquired data; the voice recognition module is responsible for monitoring and receiving audio information sent by the terminal application module in real time, and performs analog-to-digital conversion by using a voice recognition engine to generate text sentence pattern information; the semantic understanding module receives the text sentence pattern information generated by the voice recognition module in real time and extracts the keyword information by utilizing a semantic understanding engine; the data retrieval module is responsible for receiving the keyword information generated by the semantic understanding module, acquiring information actually expected by a user through accessing the knowledge map database, and finally returning the information to the terminal application module to be displayed on an interface. The invention improves the input efficiency, improves the freedom and reliability of the question-answer sentence pattern retrieval, effectively saves the storage space and improves the retrieval efficiency.
Description
Technical Field
The invention relates to a data retrieval system and an application method, in particular to a high-efficiency and high-flexibility voice question-answering system supporting voice input based on a knowledge graph and an application method thereof.
Background
Through the construction of a multi-year combat command information system, various army, people, vehicles, projects, equipment, materials and other information are collected and stored in a data information center, and the effect of data information aggregation is preliminarily embodied. However, the existing operational data gathering and retrieving means cannot meet the actual requirements of the existing system, the relevance among data and the accuracy of data retrieval capability are weak, and knowledge-graph voice retrieval research work based on the existing operational data is urgently needed under the condition of relative weakness of relevant theories, methods and research.
Knowledge-graph is a knowledge representation technology that can describe the concept, entity, event and their relationship in the objective world. Essentially, a knowledge graph is a network of semantic relationships that can be described in a refined way about anything and the relationships between them. A knowledge graph can be viewed as a large graph, where nodes represent entities or concepts and edges are formed by attributes or relationships. At present, knowledge maps are also widely used in various fields. Meanwhile, with the increasing expansion of military data information in various fields of our army, the construction of the knowledge graph is imperative.
The speech recognition technology is a technology for realizing man-machine interaction, and has the function of enabling a computer to complete the conversion of information from 'sound' to 'text', and directly converting human speech into corresponding text or commands. In terms of system construction, a speech recognition system is an application software system based on a certain hardware platform and an operating system. In terms of system structure, a speech recognition system is generally mainly composed of modules such as preprocessing, feature extraction, acoustic model, recognition decoding, language model, recognition result processing and the like. Compared with the traditional manual text input and manual command click, the voice recognition can more effectively and quickly complete text input and instruction operation, so that the research work of the voice recognition technology is imperative.
Semantics can be regarded as meaning of concepts represented by objects in the real world corresponding to data, and relationships among the meaning are interpretation and logical representation of data in a certain field. Semantic understanding, i.e. so-called knowing the meaning of a word or a sentence. It contains two layers of meanings: firstly, the conversion between different symbols can be realized; secondly, reasoning can be carried out. The semantic understanding is wide in actual related range and many in content, and generally comprises the fields of text classification, element extraction, semantic understanding, intelligent customer service, machine translation and the like. Compared with the traditional retrieval mode, the user needs to analyze and summarize the keywords by himself, and the semantic understanding user only needs to give out spoken question-answer sentence patterns, so that the thinking of the user is greatly simplified, the time is saved, the input is more natural through voice recognition, and the purpose of communication between people and a computer is achieved, so that the research work of the semantic understanding technology is also imperative.
Disclosure of Invention
The purpose of the invention is as follows: it is an object of the present invention to provide a knowledge-graph based, efficient, flexible voice question-answering system that supports voice input.
The invention also aims to provide an application method of the knowledge-graph-based voice question-answering system.
The technical scheme is as follows: the knowledge graph-based voice question-answering system comprises a terminal application module, a voice recognition module, a semantic understanding module and a data retrieval module, wherein the terminal application module is used for recording voice and displaying acquired data; the voice recognition module is responsible for monitoring and receiving audio information sent by the terminal application module in real time, and performs analog-to-digital conversion by using a voice recognition engine to generate text sentence pattern information; the semantic understanding module receives the text sentence pattern information generated by the voice recognition module in real time and extracts the keyword information by utilizing a semantic understanding engine; the data retrieval module is responsible for receiving the keyword information generated by the semantic understanding module, acquiring information actually expected by a user through accessing the knowledge map database, and finally returning the information to the terminal application module to be displayed on an interface.
Preferably, the voice recognition module comprises a voice recognition client and a voice recognition server, wherein the voice recognition client is mainly responsible for recording the questioning voice of the user and recording the questioning voice into a pcm format audio file to be sent to the voice recognition server; the voice recognition server executes the main functions of voice recognition, pre-processing and feature extraction are carried out firstly, noise on a frequency spectrum is distinguished from a target section through anti-aliasing filtering, useful information is extracted, conversion from an analog signal to a digital signal is completed, then the digital signal is sent to a decoder to carry out mode matching according to an acoustic model and a language model which are trained in advance, and finally, the voice content of a user is taken as a recognition result to be returned to a voice recognition client.
Preferably, the semantic understanding module mainly provides a semantic understanding server, the semantic understanding server provides an http access interface, the command system accesses the semantic understanding server by taking text information obtained by the voice recognition module as a parameter through an http protocol, the semantic understanding server matches the text information with a template sentence pattern trained in advance, finds the template sentence pattern with the highest likelihood, obtains a keyword preset in the sentence pattern and type information thereof, and returns the keyword and the type information to the command system in the form of a character string.
Preferably, the keywords correspond to name fields, attribute names or relationship names of data tables in the knowledge map database, and the types correspond to data table names, attributes or relationships, and information that the user wants to retrieve is obtained through table query.
The application method of the voice question-answering system comprises the following steps:
(1) configuring structural information of a table in a knowledge graph database, constructing an ontology model through field association of the table in the knowledge graph database, constructing a data relation model by using associated fields among the models, and storing the data relation model in the knowledge graph database;
(2) a user inputs contents to be retrieved in a spoken language mode through a recording device, generates an audio file and sends the audio file to a voice recognition server, and the voice recognition server analyzes the audio file into text information and returns the text information to a voice recognition client; the audio is recorded to generate an audio file, the audio file is processed and decoded through modules such as signal preprocessing, voice feature extraction, training and recognition and the like, and finally the audio file is converted into text information, so that the method has high precision and real-time performance.
(3) Constructing a question-answer sentence pattern model so as to select corresponding keyword information by matching a semantic understanding module;
the keywords and the types of the keywords of the semantic features are combined, and the keywords and the types of the keywords are classified and recombined to form the semantic template sentence pattern, so that the application range is wider, the information acquisition is more flexible, and the accuracy and the universality of the whole question-answer retrieval are improved.
(4) The voice recognition client accesses the text information as a parameter to an http port provided by the semantic understanding service, and the semantic understanding service extracts the text information and returns the extracted keywords to the voice recognition client;
the method comprises the steps of generating a text corpus through corpus collection and semantic modeling, training and testing the corpus to generate a feature template, analyzing a given text sentence pattern, extracting keywords in the text sentence pattern through matching the feature template, and labeling the types of the keywords.
(5) And the data retrieval module queries the relation or attribute information from the knowledge map database through the keyword information and displays the relation or attribute information to the user through the terminal application module.
The step matches the corresponding data relation model through the keywords and the types thereof obtained by the semantic function, searches the corresponding attributes or relation contents in the database, and displays the information on the page after extracting the information characteristics.
Further, the method for constructing the data relationship model in the step (1) comprises the following steps:
(11) constructing an ontology model according to a data source, wherein each data table can be used as an ontology;
(12) constructing a body model relationship, and if an associated field exists between the two data tables, constructing the body model relationship;
(13) constructing an entity relationship, namely taking each piece of data in a data table as an entity, taking each field of each piece of data as the attribute of the entity, and constructing the entity relationship if the values of the associated attributes are equal or equal after calculation by a specific formula;
(14) and storing the generated data into a graph database and establishing an index to complete the construction of a data relation model.
Further, the speech recognition implementation method in the step (2) is as follows:
(21) the voice recognition client generates a pcm format audio file after receiving audio recorded by an operator in a command system and sends the pcm format audio file to the voice recognition server;
(22) after receiving the audio file, the voice recognition server preprocesses the voice signal, firstly carries out end point detection and recognizes the initial and end positions of the voice; then adding weight to the voice frequency part to increase the voice resolution; finally, windowing is carried out on the voice signals, voice waveforms are emphasized, and other waveforms are weakened, so that the voice quality is improved, and the preprocessing effect is achieved;
(23) extracting characteristics, namely filtering a voice signal, and taking the energy of the output signal as the basic characteristics of the signal;
(24) decoding after feature extraction, matching according to the trained acoustic model and language model, and outputting a word sequence with the highest likelihood as a recognition result; the acoustic model is mainly responsible for matching the audio signal with basic acoustic units and converting the audio signal into a sequence set of the acoustic units; the language model is mainly responsible for recording the combination probability relation among different words and outputting the combination of the acoustic units into sentences which are closer to natural texts; finally, a decoder scores and screens out possible recognition results;
(25) and the server generates a text message from the recognition result and returns the text message to the command system.
Further, the construction method of the question-answer sentence pattern in the step (3) is as follows:
(31) writing different types of data into the documents named by the types respectively;
(32) filling the structure of the question-answer sentence pattern completely according to the requirement according to the format of the grammar design;
(33) compiling the document in step (31) into a grammar resource file using a grammar compilation tool;
(34) and (4) if the question-answer sentence pattern is subjected to adding, deleting or modifying operation, directly modifying the document in the step (31), and generating and replacing the grammar resource file.
Further, the implementation method of semantic understanding in the step (4) is as follows:
(41) after receiving text data information returned by the voice recognition module, the command system accesses and sends the text data to a semantic understanding server through an http protocol in a parameter form;
(42) after receiving the text data, the semantic understanding service end preprocesses the text data, removes interfering characters, extracts useful information in the interfering characters, and then maps the information into a characteristic sequence;
(43) matching the text characteristic sequence with a characteristic template trained in advance to obtain characteristic vector information of the text data, and introducing the characteristic vector into a semantic model trained in advance to obtain a group of model sentence patterns matched with the text data;
(44) scoring different model sentence patterns through the semantics pk and arranging the sentence patterns in a descending order, and selecting a sentence pattern template with the highest matching degree and reliability as a basis for extracting semantic information;
(45) and converting the semantic information into final keywords and types thereof, and returning the final keywords and types thereof to the command system.
Further, the keyword search analysis method in the step (5) is as follows:
(51) after receiving keywords and type information thereof returned by a semantic understanding server, a command system distinguishes entity keywords and associated keywords;
(52) according to the type of the entity key word, finding a corresponding data table and finding out the information corresponding to the entity;
(53) according to the type of the associated keywords, finding out the attribute or the relationship corresponding to the entity and finding out the complete content corresponding to the attribute or the relationship;
(54) and (5) displaying the content obtained in the step (53) as a retrieval result on a command system terminal interface.
The invention establishes the relation among the data by establishing a data relation model through the knowledge map, quickly and conveniently retrieves the corresponding content by utilizing the voice recognition and semantic analysis capabilities, and provides technical support for the data retrieval capability of the command information system.
Has the advantages that: compared with the prior art, the invention has the following advantages:
(1) the retrieval efficiency is improved by utilizing the efficient index, and the storage space is saved.
(2) The voice recognition function frees up the hands of the user and provides a more efficient way of interaction.
(3) The semantic understanding function can rapidly extract key information from the complex question-answer sentence pattern, and the retrieval efficiency is improved.
(4) The constructable function of the question-answer sentence pattern can increase the accuracy and universality of the question-answer retrieval.
(5) The keyword retrieval function can more specifically obtain the retrieval result expected by the user.
Drawings
FIG. 1 is a system architecture diagram of the present invention;
FIG. 2 is a schematic diagram of a data relationship model construction;
FIG. 3 is a schematic diagram of a speech recognition module;
FIG. 4 is a schematic diagram of question and answer sentence construction;
FIG. 5 is a schematic diagram of a semantic understanding module;
FIG. 6 is a schematic diagram of keyword search analysis.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments.
As shown in fig. 1, the voice question-answering system of the present invention mainly includes a terminal application module, a voice recognition module, a semantic understanding module, and a data retrieval module. The terminal application module is a specific use scene of the voice question-answering system in the command information system, and provides an entrance for recording voice and a display interface for acquiring data; the voice recognition module is responsible for monitoring and receiving audio information sent by the terminal application in real time, and performs analog-to-digital conversion by using a voice recognition engine to generate text information; the semantic understanding module provides an http service interface, receives text sentence pattern information generated by the voice recognition module in real time, and extracts keyword information by using a semantic understanding engine; the data retrieval module is responsible for receiving the keyword information generated by the semantic understanding module, acquiring information actually expected by a user through accessing the knowledge map database, and finally returning to the terminal application to be displayed on the interface.
The application method of the voice question-answering system comprises the following steps:
(1) establishing a data relation model between the entity and the relation and storing the data relation model in a knowledge map database for subsequent retrieval and reference;
establishing entity model according to the field content of each table in the database, establishing relation model according to the field relation between tables, and storing the relation model into the knowledge map database
The data relationship model is mainly a method for establishing a large amount of data in a system, the data have a large amount of invisible associations, but the associations are difficult to be obviously presented only by storing the data in a database, as shown in fig. 2, a method for establishing the data relationship model is provided, the data are classified, refined and associated step by step according to hierarchical division, and the associations between the data are presented more intuitively, and the specific establishing method is as follows:
(11) and constructing an ontology model according to the data source, wherein each data sheet can be used as an ontology (the ontology model comprises personnel, troops, regions and the like).
(12) And (4) constructing the ontology model relationship, wherein if the association fields exist between the two data tables, the ontology model relationship can be constructed (such as the relationship between personnel and troops and the relationship between troops and regions).
(13) And (3) constructing an entity relationship, wherein each piece of data in the data table is used as an entity, each field of each piece of data is used as the attribute of the entity, and if the values of the associated attributes are equal or equal after being calculated by a specific formula, the entity relationship can be constructed (for example, the army of Zhang three people is 2 links).
(14) And storing the generated data into a graph database and establishing an index to complete the construction of a data relation model.
(2) A user inputs contents to be retrieved in a spoken language mode through a recording device, generates an audio file and sends the audio file to a voice recognition server, and the voice recognition server analyzes the audio file into text information and returns the text information to a client;
the voice recognition mainly provides a function of replacing manual input with voice operation, and can be more suitable for bumpy surrounding environments in actual combat, as shown in fig. 3, a voice recognition module is divided into a client and a server to be executed, and the client is mainly responsible for recording question voice of a user and recording the question voice into a pcm format audio file to be sent to the server; the server executes the main functions of voice recognition, pre-processing and feature extraction are carried out firstly, noise on a frequency spectrum is distinguished from a target section through anti-aliasing filtering, useful information is extracted, conversion from an analog signal to a digital signal is completed, then the digital signal is sent to a decoder to carry out mode matching according to an acoustic model and a language model which are trained in advance, and finally, the voice content of a user is returned to a client as a recognition result. The specific implementation method comprises the following steps:
(21) and the voice recognition client generates a pcm format audio file after receiving the audio recorded by the operator in the command system and sends the pcm format audio file to the voice recognition server.
(22) After receiving the audio file, the server side preprocesses the voice signal, firstly detects an end point and identifies the initial position and the end position of the voice; then adding weight to the voice frequency part to increase the voice resolution; finally, windowing is carried out on the voice signals, voice waveforms are emphasized, and other waveforms are weakened, so that the voice quality is improved, and the preprocessing effect is achieved.
(23) And (4) feature extraction, namely filtering the voice signals, and taking the output signal energy as the basic features of the signals.
(24) And decoding after feature extraction, matching according to the trained acoustic model and language model, and outputting the word sequence with the highest likelihood as a recognition result. The acoustic model is mainly responsible for matching the audio signal with basic acoustic units and converting the audio signal into a sequence set of the acoustic units; the language model is mainly responsible for recording the combination probability relation among different words and outputting the combination of the acoustic units into sentences which are closer to natural texts; and finally, the decoder scores and screens out possible recognition results.
(25) And the server generates a text message from the recognition result and returns the text message to the command system.
(3) Constructing a question-answer sentence pattern model so as to select corresponding keyword information by matching a semantic understanding module;
taking the entity, the attribute name and the relationship name as keywords; the table name, attribute and relation corresponding to the entity are used as the type of the key word, the possible sentence format is listed and the template sentence format is trained, and the sentence format can be flexibly designed and supports free modification.
The question-answer sentence pattern construction mainly provides a grammar design function, which can disassemble various commonly used sentence patterns into a plurality of vocabularies including key information and provide important supports such as a semantic model for the following semantic understanding function, the function can enable the semantic understanding and identifying range to be deeper and wider, simultaneously can enable the sentence pattern input of the whole voice question-answer system to be more flexible, and is convenient to modify, increases the operation space and improves the fault tolerance, as shown in figure 4, the key information can be rapidly obtained once the question-answer sentence pattern is matched with the design template by classifying the information and combining the grammar logic forming the question-answer sentence pattern, and the concrete realization method is as follows:
(31) different types of data are written into the documents named by the types respectively (for example, three Zhang, Liqu and the like are contained in a personnel document, and 1 Zhang, 2 Zhang and the like are contained in a troop document).
(32) The structure of the question-answer sentence pattern is completely filled according to the requirement of the format designed by the grammar (for example, $ person belongs to. $ army, can match with \ "Zhang three belongs to 1 link \", etc.).
(33) And compiling the document into a grammar resource file by using a grammar compiling tool as an important basis for semantic analysis.
(34) If the question-answer sentence pattern is added, deleted or modified, the above-mentioned file can be directly modified, and the grammar resource file can be generated and substituted.
(4) The client accesses the text information as a parameter to an http port provided by the semantic understanding service, and the semantic understanding service extracts the keywords from the text information and returns the keywords to the client;
the semantic understanding module mainly provides a semantic understanding server, the server provides an http access interface, the command system accesses the semantic understanding server by taking text information obtained by the language recognition module as a parameter through an http protocol, the server matches the text information with template sentence patterns trained in advance, the template sentence pattern with the highest likelihood is found, keywords preset by the sentence pattern and type information of the keywords are obtained, and the keywords are returned to the command system in the form of character strings.
Semantic understanding mainly provides a question-answer sentence pattern parsing function, and can match text information generated by a voice recognition module with a semantic template to obtain important information concerned by a user, as shown in fig. 5, a semantic understanding module mainly provides a semantic understanding server, and obtains key information and information types in the text information by parsing a question-answer sentence pattern, and the specific implementation method is as follows:
(41) and after receiving the text information returned by the voice recognition server, the command system accesses and sends the text data to the semantic understanding server in a parameter form through an http protocol.
(42) After receiving the text data, the server side preprocesses the text data, removes the interfering characters, extracts useful information in the text data, and then maps the information into a characteristic sequence to facilitate semantic recognition.
(43) And matching the text characteristic sequence with a characteristic template trained in advance to obtain characteristic vector information of the text, and importing the characteristic vector into a semantic model trained in advance to obtain a group of model sentence patterns matched with the text.
(44) And (4) scoring and sequencing different model sentence patterns in a descending order through the semantics pk, and selecting the sentence pattern template with the highest matching degree and reliability as a basis for extracting the semantic information.
(45) And converting the semantic information into final keywords and types thereof (such as keywords: Zhang III, types: personnel) and returning the final keywords and types to the command system.
(5) Searching and analyzing the keywords, and inquiring the relation or attribute information from a knowledge map database through the keywords and displaying the relation or attribute information to a user by the client;
and (4) finding a corresponding entity and a corresponding relation model in the knowledge graph database according to the returned result in the step (4), wherein the keyword corresponds to the name field, the attribute name or the relation name of the data table, and the type corresponds to the name, the attribute or the relation of the data table, and the information which the user wants to retrieve can be obtained by inquiring the table.
The keyword retrieval and analysis function is mainly to classify the keyword information obtained from the semantic understanding module, as shown in fig. 6, the keyword type corresponds to a specific entity and its attribute or name of relationship (for example, "zhang san" is a human entity, "zhang san name" is an attribute of zhang san, "zhang san and lie four are friends," zhang san relationship), and matches with the corresponding entity model, and finally, corresponding information is queried and displayed in the database, and the specific analysis method is as follows:
(51) after receiving the keywords and the type information thereof returned by the semantic understanding service end, the command system firstly distinguishes entity keywords and associated keywords (such as characters: xu peishan; attributes: arbitrary units, wherein the xu peishan is the entity keywords, and the arbitrary units are the associated keywords).
(52) And finding out the corresponding data table and finding out the information corresponding to the entity according to the type of the entity key word.
(53) And according to the type of the associated key words, finding out the attribute or the relationship corresponding to the entity and finding out the complete content corresponding to the attribute or the relationship.
(54) And (5) displaying the content obtained in the step (53) as a retrieval result on a command system terminal interface.
The invention mainly starts from the angle of improving the data retrieval efficiency in the existing command information system, converts the voice into characters through a voice recognition method, then matches question-answer sentence patterns through a semantic understanding function, extracts key words in the question-answer sentence patterns, then searches corresponding information in a graph database by utilizing the key words, and finally obtains a desired result. The method firstly uses a voice input mode to replace manual input, then uses a voice recognition method to convert voice into text information, and the voice recognition module improves the input efficiency compared with manual character input; and then matching the text information with the semantic understanding template sentence pattern, and analyzing to obtain the keywords and the types thereof, wherein the semantic understanding module improves the freedom and reliability of retrieving the question-answer sentence pattern. And finally, the information is utilized to search in a knowledge graph system to obtain a result, and the knowledge graph module effectively saves the storage space and improves the searching efficiency.
Claims (10)
1. A knowledge-graph-based voice question-answering system, comprising:
the terminal application module is used for recording voice and displaying the acquired data;
the voice recognition module is responsible for monitoring and receiving the audio information sent by the terminal application module in real time, and performing analog-to-digital conversion by using a voice recognition engine to generate text sentence pattern information;
the semantic understanding module receives the text sentence pattern information generated by the voice recognition module in real time and extracts the keyword information by utilizing a semantic understanding engine;
and the data retrieval module is responsible for receiving the keyword information generated by the semantic understanding module, acquiring the information actually expected by the user through accessing the knowledge map database, and finally returning the information to the terminal application module to display on the interface.
2. The knowledge-graph-based voice question-answering system according to claim 1, wherein the voice recognition module comprises a voice recognition client and a voice recognition server, and the voice recognition client is mainly responsible for receiving and recording the question voice of the user and recording an audio file in a pcm format to send the voice recognition server; the voice recognition server executes the main functions of voice recognition, pre-processing and feature extraction are carried out firstly, noise on a frequency spectrum is distinguished from a target section through anti-aliasing filtering, useful information is extracted, conversion from an analog signal to a digital signal is completed, then the digital signal is sent to a decoder to carry out mode matching according to an acoustic model and a language model which are trained in advance, and finally, the voice content of a user is taken as a recognition result to be returned to a voice recognition client.
3. The knowledge-graph-based voice question-answering system according to claim 1, wherein the semantic understanding module mainly provides a semantic understanding server, the semantic understanding server provides an http access interface, the command system accesses the semantic understanding server by taking text information obtained by the voice recognition module as a parameter through an http protocol, the semantic understanding server matches the text information with template sentence patterns trained in advance, finds the template sentence pattern with the highest likelihood, obtains preset keywords and type information of the sentence pattern, and returns the preset keywords and type information to the command system in the form of character strings.
4. The knowledge-graph-based voice question-answering system according to claim 1, wherein the keywords correspond to name fields, attribute names or relationship names of data tables in the knowledge-graph database, and the types correspond to data table names, attributes or relationships, and information that a user wants to retrieve is obtained through table query.
5. An application method of a knowledge graph-based voice question-answering system is characterized by comprising the following steps:
(1) configuring structural information of a table in a knowledge graph database, constructing an ontology model through field association of the table in the knowledge graph database, constructing a data relation model by using associated fields among the models, and storing the data relation model in the knowledge graph database;
(2) a user inputs contents to be retrieved in a spoken language mode through a recording device, generates an audio file and sends the audio file to a voice recognition server, and the voice recognition server analyzes the audio file into text information and returns the text information to a voice recognition client;
(3) constructing a question-answer sentence pattern model so as to select corresponding keyword information by matching a semantic understanding module;
(4) the voice recognition client accesses the text information as a parameter to an http port provided by the semantic understanding service, and the semantic understanding service extracts the text information and returns the extracted keywords to the voice recognition client;
(5) and the data retrieval module queries the relation or attribute information from the knowledge map database through the keyword information and displays the relation or attribute information to the user through the terminal application module.
6. The application method of the knowledge-graph-based voice question-answering system according to claim 5, wherein the construction method of the data relation model in the step (1) is as follows:
(11) constructing an ontology model according to a data source, wherein each data table can be used as an ontology;
(12) constructing a body model relationship, and if an associated field exists between the two data tables, constructing the body model relationship;
(13) constructing an entity relationship, namely taking each piece of data in a data table as an entity, taking each field of each piece of data as the attribute of the entity, and constructing the entity relationship if the values of the associated attributes are equal or equal after calculation by a specific formula;
(14) and storing the generated data into a graph database and establishing an index to complete the construction of a data relation model.
7. The method for applying the knowledge-graph-based voice question-answering system according to claim 5, wherein the voice recognition implementation method in the step (2) is as follows:
(21) the voice recognition client generates a pcm format audio file after receiving audio recorded by an operator in a command system and sends the pcm format audio file to the voice recognition server;
(22) after receiving the audio file, the voice recognition server preprocesses the voice signal, firstly carries out end point detection and recognizes the initial and end positions of the voice; then adding weight to the voice frequency part to increase the voice resolution; finally, windowing is carried out on the voice signals, voice waveforms are emphasized, and other waveforms are weakened, so that the voice quality is improved, and the preprocessing effect is achieved;
(23) extracting characteristics, namely filtering a voice signal, and taking the energy of the output signal as the basic characteristics of the signal;
(24) decoding after feature extraction, matching according to the trained acoustic model and language model, and outputting a word sequence with the highest likelihood as a recognition result; the acoustic model is mainly responsible for matching the audio signal with basic acoustic units and converting the audio signal into a sequence set of the acoustic units; the language model is mainly responsible for recording the combination probability relation among different words and outputting the combination of the acoustic units into sentences which are closer to natural texts; finally, a decoder scores and screens out possible recognition results;
(25) and the server generates a text message from the recognition result and returns the text message to the command system.
8. The method for applying the knowledge-graph-based voice question-answer system according to claim 5, wherein the question-answer sentence pattern in the step (3) is constructed by the following steps:
(31) writing different types of data into the documents named by the types respectively;
(32) filling the structure of the question-answer sentence pattern completely according to the requirement according to the format of the grammar design;
(33) compiling the document in step (31) into a grammar resource file using a grammar compilation tool;
(34) and (4) if the question-answer sentence pattern is subjected to adding, deleting or modifying operation, directly modifying the document in the step (31), and generating and replacing the grammar resource file.
9. The method for applying the knowledge-graph-based voice question-answering system according to claim 5, wherein the semantic understanding in the step (4) is realized by:
(41) after receiving text data information returned by the voice recognition module, the command system accesses and sends the text data to a semantic understanding server through an http protocol in a parameter form;
(42) after receiving the text data, the semantic understanding service end preprocesses the text data, removes interfering characters, extracts useful information in the interfering characters, and then maps the information into a characteristic sequence;
(43) matching the text characteristic sequence with a characteristic template trained in advance to obtain characteristic vector information of the text data, and introducing the characteristic vector into a semantic model trained in advance to obtain a group of model sentence patterns matched with the text data;
(44) scoring different model sentence patterns through the semantics pk and arranging the sentence patterns in a descending order, and selecting a sentence pattern template with the highest matching degree and reliability as a basis for extracting semantic information;
(45) and converting the semantic information into final keywords and types thereof, and returning the final keywords and types thereof to the command system.
10. The application method of the knowledge-graph-based voice question-answering system according to claim 5, wherein the keyword retrieval analysis method in the step (5) is as follows:
(51) after receiving keywords and type information thereof returned by a semantic understanding server, a command system distinguishes entity keywords and associated keywords;
(52) according to the type of the entity key word, finding a corresponding data table and finding out the information corresponding to the entity;
(53) according to the type of the associated keywords, finding out the attribute or the relationship corresponding to the entity and finding out the complete content corresponding to the attribute or the relationship;
(54) and (5) displaying the content obtained in the step (53) as a retrieval result on a command system terminal interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010422420.4A CN111597308A (en) | 2020-05-19 | 2020-05-19 | Knowledge graph-based voice question-answering system and application method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010422420.4A CN111597308A (en) | 2020-05-19 | 2020-05-19 | Knowledge graph-based voice question-answering system and application method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111597308A true CN111597308A (en) | 2020-08-28 |
Family
ID=72185715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010422420.4A Pending CN111597308A (en) | 2020-05-19 | 2020-05-19 | Knowledge graph-based voice question-answering system and application method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111597308A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069299A (en) * | 2020-09-03 | 2020-12-11 | 三一专用汽车有限责任公司 | Response method for question-answer interaction, server, engineering vehicle and readable storage medium |
| CN112417102A (en) * | 2020-11-26 | 2021-02-26 | 中国科学院自动化研究所 | Voice query method, device, server and readable storage medium |
| CN112435512A (en) * | 2020-11-12 | 2021-03-02 | 郑州大学 | Voice behavior assessment and evaluation method for rail transit simulation training |
| CN112465144A (en) * | 2020-12-11 | 2021-03-09 | 北京航空航天大学 | Multi-modal demonstration intention generation method and device based on limited knowledge |
| CN112506405A (en) * | 2020-12-03 | 2021-03-16 | 浪潮云信息技术股份公司 | Artificial intelligent voice large screen command method based on Internet supervision field |
| CN112508388A (en) * | 2020-12-02 | 2021-03-16 | 唐旸 | Method and system for inputting product quality detection data, server side and storage medium |
| CN112528041A (en) * | 2020-12-17 | 2021-03-19 | 贵州电网有限责任公司 | Scheduling phrase specification verification method based on knowledge graph |
| CN112735475A (en) * | 2020-12-25 | 2021-04-30 | 北京博瑞彤芸科技股份有限公司 | Method and system for searching disease knowledge through voice |
| CN112818096A (en) * | 2021-01-26 | 2021-05-18 | 广州三星通信技术研究有限公司 | Dialog generating method and device |
| CN112950326A (en) * | 2021-03-17 | 2021-06-11 | 南通大学 | Artificial intelligence data analysis system supporting deep learning working principle |
| CN112966119A (en) * | 2021-02-25 | 2021-06-15 | 青岛海信网络科技股份有限公司 | Information acquisition method, equipment and medium |
| CN113053387A (en) * | 2021-02-26 | 2021-06-29 | 上海声通信息科技股份有限公司 | Voice input system supporting semantic understanding |
| CN113255361A (en) * | 2021-05-19 | 2021-08-13 | 平安科技（深圳）有限公司 | Automatic detection method, device, equipment and storage medium of voice content |
| CN113345430A (en) * | 2021-06-25 | 2021-09-03 | 上海适享文化传播有限公司 | Multi-field query method based on voice fixed condition |
| CN113407667A (en) * | 2021-06-11 | 2021-09-17 | 上海适享文化传播有限公司 | Query method based on voice fixed condition |
| CN113420124A (en) * | 2021-06-25 | 2021-09-21 | 上海适享文化传播有限公司 | Method for solving conflict under voice retrieval multi-condition |
| CN113593530A (en) * | 2021-07-26 | 2021-11-02 | 国网安徽省电力有限公司建设分公司 | Safety helmet system based on NLP technology and operation method |
| CN113641833A (en) * | 2021-08-17 | 2021-11-12 | 同济大学 | Service requirement matching method and device |
| CN113836280A (en) * | 2021-08-31 | 2021-12-24 | 上海欣能信息科技发展有限公司 | Power grid operation detection voice question-answer method and system based on knowledge graph technology |
| WO2022204937A1 (en) * | 2021-03-30 | 2022-10-06 | 江苏树实科技有限公司 | Text input system based on speech recognition device, and method therefor |
| CN115827848A (en) * | 2023-02-10 | 2023-03-21 | 天翼云科技有限公司 | Method, device, equipment and storage medium for extracting knowledge graph events |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180308487A1 (en) * | 2017-04-21 | 2018-10-25 | Go-Vivace Inc. | Dialogue System Incorporating Unique Speech to Text Conversion Method for Meaningful Dialogue Response |
| CN109255113A (en) * | 2018-09-04 | 2019-01-22 | 郑州信大壹密科技有限公司 | Intelligent critique system |
| CN110827801A (en) * | 2020-01-09 | 2020-02-21 | 成都无糖信息技术有限公司 | Automatic voice recognition method and system based on artificial intelligence |
-
2020
- 2020-05-19 CN CN202010422420.4A patent/CN111597308A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180308487A1 (en) * | 2017-04-21 | 2018-10-25 | Go-Vivace Inc. | Dialogue System Incorporating Unique Speech to Text Conversion Method for Meaningful Dialogue Response |
| CN109255113A (en) * | 2018-09-04 | 2019-01-22 | 郑州信大壹密科技有限公司 | Intelligent critique system |
| CN110827801A (en) * | 2020-01-09 | 2020-02-21 | 成都无糖信息技术有限公司 | Automatic voice recognition method and system based on artificial intelligence |
Non-Patent Citations (1)
| Title |
|---|
| 李泽宇 等: "《一种智能语音技术在指挥系统中的应用方法》", 《通信电源技术》 * |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069299A (en) * | 2020-09-03 | 2020-12-11 | 三一专用汽车有限责任公司 | Response method for question-answer interaction, server, engineering vehicle and readable storage medium |
| CN112435512A (en) * | 2020-11-12 | 2021-03-02 | 郑州大学 | Voice behavior assessment and evaluation method for rail transit simulation training |
| CN112417102A (en) * | 2020-11-26 | 2021-02-26 | 中国科学院自动化研究所 | Voice query method, device, server and readable storage medium |
| CN112417102B (en) * | 2020-11-26 | 2024-03-22 | 中国科学院自动化研究所 | Voice query method, device, server and readable storage medium |
| CN112508388B (en) * | 2020-12-02 | 2022-08-19 | 唐旸 | Method and system for inputting product quality detection data, server side and storage medium |
| CN112508388A (en) * | 2020-12-02 | 2021-03-16 | 唐旸 | Method and system for inputting product quality detection data, server side and storage medium |
| CN112506405A (en) * | 2020-12-03 | 2021-03-16 | 浪潮云信息技术股份公司 | Artificial intelligent voice large screen command method based on Internet supervision field |
| CN112465144A (en) * | 2020-12-11 | 2021-03-09 | 北京航空航天大学 | Multi-modal demonstration intention generation method and device based on limited knowledge |
| CN112528041A (en) * | 2020-12-17 | 2021-03-19 | 贵州电网有限责任公司 | Scheduling phrase specification verification method based on knowledge graph |
| CN112735475A (en) * | 2020-12-25 | 2021-04-30 | 北京博瑞彤芸科技股份有限公司 | Method and system for searching disease knowledge through voice |
| CN112735475B (en) * | 2020-12-25 | 2023-02-21 | 北京博瑞彤芸科技股份有限公司 | Method and system for searching disease knowledge through voice |
| CN112818096A (en) * | 2021-01-26 | 2021-05-18 | 广州三星通信技术研究有限公司 | Dialog generating method and device |
| CN112966119B (en) * | 2021-02-25 | 2022-11-25 | 青岛海信网络科技股份有限公司 | Information acquisition method, equipment and medium |
| CN112966119A (en) * | 2021-02-25 | 2021-06-15 | 青岛海信网络科技股份有限公司 | Information acquisition method, equipment and medium |
| CN113053387A (en) * | 2021-02-26 | 2021-06-29 | 上海声通信息科技股份有限公司 | Voice input system supporting semantic understanding |
| CN112950326A (en) * | 2021-03-17 | 2021-06-11 | 南通大学 | Artificial intelligence data analysis system supporting deep learning working principle |
| WO2022204937A1 (en) * | 2021-03-30 | 2022-10-06 | 江苏树实科技有限公司 | Text input system based on speech recognition device, and method therefor |
| CN113255361A (en) * | 2021-05-19 | 2021-08-13 | 平安科技（深圳）有限公司 | Automatic detection method, device, equipment and storage medium of voice content |
| CN113255361B (en) * | 2021-05-19 | 2023-12-22 | 平安科技（深圳）有限公司 | Automatic voice content detection method, device, equipment and storage medium |
| CN113407667B (en) * | 2021-06-11 | 2024-03-22 | 上海适享文化传播有限公司 | Query method based on voice fixed condition |
| CN113407667A (en) * | 2021-06-11 | 2021-09-17 | 上海适享文化传播有限公司 | Query method based on voice fixed condition |
| CN113420124A (en) * | 2021-06-25 | 2021-09-21 | 上海适享文化传播有限公司 | Method for solving conflict under voice retrieval multi-condition |
| CN113345430A (en) * | 2021-06-25 | 2021-09-03 | 上海适享文化传播有限公司 | Multi-field query method based on voice fixed condition |
| CN113420124B (en) * | 2021-06-25 | 2024-03-22 | 上海适享文化传播有限公司 | Method for resolving conflict under multiple conditions of voice retrieval |
| CN113593530A (en) * | 2021-07-26 | 2021-11-02 | 国网安徽省电力有限公司建设分公司 | Safety helmet system based on NLP technology and operation method |
| CN113641833A (en) * | 2021-08-17 | 2021-11-12 | 同济大学 | Service requirement matching method and device |
| CN113641833B (en) * | 2021-08-17 | 2024-04-09 | 同济大学 | Service demand matching method and device |
| CN113836280A (en) * | 2021-08-31 | 2021-12-24 | 上海欣能信息科技发展有限公司 | Power grid operation detection voice question-answer method and system based on knowledge graph technology |
| CN115827848A (en) * | 2023-02-10 | 2023-03-21 | 天翼云科技有限公司 | Method, device, equipment and storage medium for extracting knowledge graph events |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111597308A (en) | Knowledge graph-based voice question-answering system and application method thereof | |
| CN110399457B (en) | Intelligent question answering method and system | |
| CN106874378B (en) | Method for constructing knowledge graph based on entity extraction and relation mining of rule model | |
| CN111753099B (en) | Method and system for enhancing relevance of archive entity based on knowledge graph | |
| KR102094934B1 (en) | Natural Language Question-Answering System and method | |
| CN102262634B (en) | Automatic questioning and answering method and system | |
| RU2488877C2 (en) | Identification of semantic relations in indirect speech | |
| CN110765257A (en) | Intelligent consulting system of law of knowledge map driving type | |
| CN109947921B (en) | Intelligent question-answering system based on natural language processing | |
| CN103810998B (en) | Based on the off-line audio recognition method of mobile terminal device and realize method | |
| KR20050036541A (en) | Semi-automatic construction method for knowledge of encyclopedia question answering system | |
| CN109766417A (en) | A kind of construction method of the literature annals question answering system of knowledge based map | |
| CN105045852A (en) | Full-text search engine system for teaching resources | |
| CN109947952B (en) | Retrieval method, device, equipment and storage medium based on English knowledge graph | |
| CN106446162A (en) | Orient field self body intelligence library article search method | |
| CN111190920B (en) | Data interaction query method and system based on natural language | |
| CN108664599A (en) | Intelligent answer method, apparatus, intelligent answer server and storage medium | |
| CN111782800B (en) | Intelligent conference analysis method for event tracing | |
| CN109271459A (en) | Chat robots and its implementation based on Lucene and grammer networks | |
| CN113569023A (en) | Chinese medicine question-answering system and method based on knowledge graph | |
| CN117290489B (en) | Method and system for quickly constructing industry question-answer knowledge base | |
| CN111061828B (en) | Digital library knowledge retrieval method and device | |
| CN105760462A (en) | Man-machine interaction method and device based on associated data query | |
| JP2013190985A (en) | Knowledge response system, method and computer program | |
| CN114647719A (en) | Question-answering method and device based on knowledge graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200828 |