CN111324689A - Index updating method, device, equipment and storage medium of question-answering system - Google Patents
Index updating method, device, equipment and storage medium of question-answering system Download PDFInfo
- Publication number
- CN111324689A CN111324689A CN202010138638.7A CN202010138638A CN111324689A CN 111324689 A CN111324689 A CN 111324689A CN 202010138638 A CN202010138638 A CN 202010138638A CN 111324689 A CN111324689 A CN 111324689A
- Authority
- CN
- China
- Prior art keywords
- index
- question
- answering
- updating
- engine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
Abstract
The invention relates to the technical field of big data, and discloses an index updating method of a question-answering system, which comprises the following steps: receiving an index updating request, and determining a target question-answering engine to be updated according to configuration information preset by a question-answering system; setting a target question-answering engine as an index updating state based on the index updating request, and reading an index serial number in a preset Redis to obtain a first index name of a corresponding index; based on the index name, acquiring a corresponding Annoy index file from Redis and loading the Annoy index file to a memory; taking the index name as a search formula, inquiring a second index name of the search formula from the Elasticissearch, and taking the second index name as the latest ES index of the target question-answering engine; and resetting the index updating state of the target question-answering engine and determining the next target question-answering engine to be updated according to the configuration information. The invention also discloses an index updating device, equipment and a computer readable storage medium of the question answering system. The invention provides more accurate index updating service for the user and ensures the correct operation of the index updating process.
Description
Technical Field
The invention relates to the technical field of big data, in particular to an index updating method, device and equipment of a question-answering system and a computer readable storage medium.
Background
In recent years, with the continuous development of artificial intelligence, in the existing index updating technology, the general architecture of the question-answering system in the intelligent dialogue system is mainly literal semantic recall, the indexing large-data-volume retrieval usually uses the Annoy algorithm, and the indexing uses the Elasticsearch as the literal recall, and the results of the literal recall and the semantic recall are rearranged to obtain the final result. Retrieval recall by the Annoy algorithm relies on semantic Annoy indexing, and literal recall by the ES index is based on the data to be retrieved that the Elasticsearch has stored.
However, because the generation of the current index depends on an offline manual processing mode, the whole index updating process is completed by restarting the service, which results in excessive cost of index updating. Therefore, it is desirable to provide an index updating method that does not depend on offline manual processing to solve the above problems.
Disclosure of Invention
The invention mainly aims to provide an index updating method, device and equipment of a question-answering system and a computer readable storage medium, aiming at solving the technical problem that the updating cost is overlarge due to the fact that the current index generation depends on offline manual processing.
In order to achieve the above object, the present invention provides an index updating method for a question answering system, which comprises the following steps:
receiving an index updating request, and determining a target question-answering engine to be updated based on preset configuration information in the question-answering system;
setting the target question-answering engine to be in an index updating state based on the index updating request, reading an index sequence number in a preset Redis, and acquiring a first index name of an index corresponding to the index sequence number;
based on the first index name, acquiring a corresponding Annoy index file from Redis and loading the Annoy index file into a memory;
taking the first index name as a search formula, inquiring a second index name matched with the search formula from an Elasticissearch, and taking the second index name as the latest ES index of the target question-answering engine;
and resetting the index updating state of the target question-answering engine and determining the target question-answering engine to be updated next to update the index based on the configuration information.
Optionally, before the step of receiving an index update request and determining a target question-answering engine to be updated based on configuration information preset in the question-answering system, the method includes:
acquiring an original corpus;
performing corpus collection and labeling on the original corpus, and outputting labeled corpus;
verifying the labeled corpus and outputting a verification result;
based on the verification result, storing the labeled corpus meeting the qualification evaluation to a corpus management platform;
based on a preset interface, calling a question-answering engine to acquire the labeled corpus in the corpus management platform and storing the labeled corpus to Redis;
acquiring the markup corpus from Redis, and preprocessing the markup corpus to obtain a semantic vector of the markup corpus;
and generating an Annoy index file based on the semantic vector of the labeled corpus.
Optionally, after generating the Annoy index file based on the semantic vector of the corpus, the method further includes:
writing the Annoy index file into Redis, and storing an intermediate corpus used when the Annoy index file is generated;
constructing request parameter information of an ES index calling interface based on the Annoy index file and the intermediate corpus;
based on the request parameter information, calling the ES index calling interface to generate an ES index;
storing the Annoy index file to Redis, and storing the ES index to an Elasticissearch.
Optionally, the step of receiving an index update request and determining a target question-answering engine to be updated based on configuration information preset in the question-answering system includes:
receiving an index updating request and acquiring IP list information corresponding to the question answering engine;
and determining each target question-answering engine to be updated based on each IP address in the IP list information corresponding to the question-answering engine.
Optionally, after the step of resetting the index update state of the target question and answer engine and determining the next target question and answer engine to be updated based on the configuration information to perform index update, the method further includes:
if the target question-answering engine index fails to be updated, setting the target question-answering engine to be in an index updating failure state;
detecting whether a question answering engine in an index updating failure state exists or not based on a health detection mechanism of the question answering engine;
if so, sending the index updating request to the question answering engine in the index updating failure state again;
and if the index updating times of the target question-answering engine exceed the preset times and the index updating is not finished, sending a reminding mail to a preset receiver mailbox.
Optionally, after the step of sending a reminding mail to a preset recipient mailbox if the index update times of the target question answering engine exceed the preset times and the index update is still not completed, the method further includes:
judging whether the index version numbers corresponding to the question and answer engines are consistent or not based on the IP list information corresponding to the question and answer engines;
if the index version numbers corresponding to the question-answering engines are not consistent, respectively counting the index version numbers corresponding to the question-answering engines;
and determining the question-answering engines with the corresponding index version numbers occupying a few number based on the index version numbers corresponding to the question-answering engines, and setting the question-answering engines with the corresponding index version numbers occupying a few number to be in an unavailable state.
Further, the present invention provides an index updating apparatus for a question answering system, the index updating apparatus for a question answering system comprising:
the receiving module is used for receiving an index updating request and determining a target question-answering engine to be updated based on preset configuration information in the question-answering system;
the acquisition module is used for setting the target question-answering engine to be in an index updating state based on the index updating request, reading an index serial number in preset Redis and acquiring an index name of a latest first index corresponding to the index serial number;
the loading module is used for acquiring a corresponding Annoy index file from Redis based on the first index name and loading the Annoy index file into a memory;
the updating module is used for inquiring a second index name of the index from an elastic search by taking the first index name as a search formula, and taking the second index name as the latest ES index of the target question-answering engine;
and the resetting module is used for resetting the index updating state of the target question-answering engine and determining the next target question-answering engine to be updated to update the index based on the configuration information.
The index updating device of the question-answering system further comprises:
the corpus storage module is used for acquiring original corpuses, collecting and labeling the corpuses of the original corpuses, and outputting labeled corpuses; verifying the labeled corpus, outputting a verification result, and storing the labeled corpus meeting the qualification evaluation to a corpus management platform based on the verification result;
the calling module is used for calling a question and answer engine based on a preset interface to acquire the labeled corpus in the corpus management platform and store the labeled corpus in Redis;
the preprocessing module is used for acquiring the labeling linguistic data from Redis and preprocessing the labeling linguistic data to obtain semantic vectors of the labeling linguistic data;
and the Annoy index generation module is used for generating an Annoy index file based on the semantic vector of the corpus.
The index updating device of the question-answering system further comprises:
the writing module is used for writing the Annoy index file into Redis and storing an intermediate corpus used when the Annoy index file is generated;
the construction module is used for constructing request parameter information of an ES index calling interface based on the Annoy index file and the intermediate corpus;
the ES index generation module is used for calling the ES index calling interface based on the request parameter information to generate an ES index;
and the index storage module is used for storing the Annoy index file to Redis and storing the ES index to an Elasticissearch.
Optionally, the receiving module is specifically configured to:
receiving an index updating request and acquiring IP list information corresponding to the question answering engine; and determining each target question-answering engine to be updated based on each IP address in the IP list information corresponding to the question-answering engine.
Optionally, the update module is further configured to:
when the index of the target question-answering engine fails to be updated, setting the target question-answering engine to be in an index updating failure state; detecting whether a question answering engine in an index updating failure state exists or not based on a health detection mechanism of the question answering engine; when the question answering engine is in the index updating failure state, the index updating request is sent to the question answering engine in the index updating failure state again; and when the index updating times of the target question-answering engine exceed the preset times and the index updating is not completed, sending a reminding mail to a preset receiver mailbox.
Further, the index updating device of the question-answering system further includes:
the version number unifying module is used for judging whether the index version numbers corresponding to the question answering engines are consistent or not based on the IP list information corresponding to the question answering engines; if the index version numbers corresponding to the question-answering engines are not consistent, respectively counting the index version numbers corresponding to the question-answering engines; and determining the question-answering engines with the corresponding index version numbers occupying a few number based on the index version numbers corresponding to the question-answering engines, and setting the question-answering engines with the corresponding index version numbers occupying a few number to be in an unavailable state.
Further, in order to achieve the above object, the present invention further provides an index updating apparatus of a question-answering system, where the index updating apparatus of the question-answering system includes a memory, a processor, and an index updating program of the question-answering system stored in the memory and operable on the processor, and when executed by the processor, the index updating program of the question-answering system implements the steps of the index updating method of the question-answering system as described in any one of the above items.
Further, to achieve the above object, the present invention further provides a computer readable storage medium, where an index updating program of a question-answering system is stored, and when the index updating program of the question-answering system is executed by a processor, the steps of the index updating method of the question-answering system as described in any one of the above are implemented.
The method comprises the steps that a target question-answering engine to be updated is determined through preset configuration information in a question-answering system, when the target question-answering engine receives an index updating request, the corresponding target question-answering engine is set to be in an index updating state, and index serial numbers in preset Redis are read to obtain a corresponding Annoy index file; and then, taking the index name of the index file as a search formula, searching the latest ES index corresponding to the target question-answering engine from the elastic search, resetting the state of the corresponding target question-answering engine after the index is updated, determining the next question-answering engine to be updated according to the configuration information, and completing the updating of all the question-answering engine indexes. The invention reduces the dependency of the current index updating on the offline manual processing, ensures the correct operation of the index updating process and further reduces the development and maintenance cost.
Drawings
FIG. 1 is a schematic structural diagram of a hardware operating environment of a device according to an embodiment of an index updating device of the question-answering system of the present invention;
FIG. 2 is a schematic flow chart of a first embodiment of an index updating method for the question-answering system according to the present invention;
FIG. 3 is a flowchart illustrating a second embodiment of an index updating method for the question-answering system according to the present invention;
FIG. 4 is a flowchart illustrating a third embodiment of an index updating method for the question-answering system according to the present invention;
FIG. 5 is a schematic view of a detailed process of step S110 in FIG. 2;
FIG. 6 is a schematic flow chart of an index updating method for a question-answering system according to a fourth embodiment of the present invention;
FIG. 7 is a flowchart illustrating a fifth embodiment of the index updating method of the question-answering system according to the present invention;
FIG. 8 is a functional block diagram of an index updating apparatus of the question-answering system according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides an index updating device of a question-answering system.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a hardware operating environment of a device according to an embodiment of the index updating device of the question-answering system of the present invention.
As shown in fig. 1, the index updating apparatus of the question-answering system may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the hardware configuration of the index update apparatus of the question-answering system shown in fig. 1 does not constitute a limitation of the index update apparatus of the question-answering system, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer-readable storage medium, may include therein an index update program of an operating system, a network communication module, a user interface module, and a question-and-answer system. The operating system is a program for managing and controlling the index updating equipment and the software resources of the question-answering system, and supports the operation of the network communication module, the user interface module, the index updating program of the question-answering system and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.
In the hardware structure of the index updating device of the question answering system shown in fig. 1, the network interface 1004 is mainly used for connecting to a system background and performing data communication with the system background; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; the index updating apparatus of the question-answering system calls the index updating program of the question-answering system stored in the memory 1005 by the processor 1001 and performs the operations of the following embodiments of the index updating method of the question-answering system.
Based on the hardware structure of the index updating device of the question-answering system, the invention provides various embodiments of the index updating method of the question-answering system.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the index updating method of the question-answering system according to the present invention. In this embodiment, the index updating method of the question-answering system includes the following steps:
step S110, receiving an index updating request, and determining a target question answering engine to be updated based on preset configuration information in the question answering system;
in this embodiment, the index is a database term, and the foreign language is named index. In a relational database, an index is a single, physical storage structure that orders values of one or more columns in a database table, which is a collection of values of one or more columns in a table. For example, the index functions as a directory of a book, and the required content can be quickly found according to the page number in the directory.
In this embodiment, the index update request is an index update instruction. And according to the configuration information in the question-answering system, acquiring the IP information of each question-answering engine, and the question-answering engines receive an index updating request sent by the system and start to update the indexes.
In this embodiment, the question answering system is mainly divided into four parts: faq-Index, question and answer engine (faq-engine), persistent storage Redis and inverted Index elastic search (abbreviated). Wherein faq-Index generates indexes online; the question answering engine (faq-engine) supports online matching index; persistent storage Redis saves Annoy indexes; the Elasticsearch holds literal indexes.
Step S120, based on the index updating request, setting the target question answering engine to be in an index updating state, reading an index sequence number in a preset Redis, and acquiring a first index name of an index corresponding to the index sequence number;
in this embodiment, when a question-answering engine (faq-engine) receives an index update request, it will set itself to be in an index update state to suspend receiving a new index update request, thereby completing the current index update operation. When a new index update request is received, a status bit is set to false, the status bit is queried each time a new index update request comes, and an http 500 error is returned when the status bit is false.
In this embodiment, the http 500 is an error code of the http protocol, which means an internal error of the server, and further, after the current index update is completed, the status bit is set to tube, and a new index update request is received to complete the index update.
In this embodiment, Redis a high-performance database. When the index is notified to be updated, the faq-index writes the index to Redis and Elasticissearch, wherein Redis has an increasing sequence number, and the index is generated and then added to the back end of the index according to the remainder of dividing the increasing sequence number by 2. For example: if the index name is elife and the sequence number is 3, the index name of the generated Annoy is elife _1, and the index name in es is elife _ 1. When the index is generated again, the sequence number becomes 4, and the generated Annoy index name is elife _0, and the index name in es is elife _ 0.
In this embodiment, the preset logic is Faq-index logic, that is, the "incremented sequence number" is only used to distinguish two versions, and may also use two values, i.e. 0 and 1, when the current value is 0, the update completion request is set to 1, and the next time the update completion request is updated to 0.
Step S130, acquiring a corresponding Annoy index file from Redis based on the first index name and loading the Annoy index file into a memory;
in this embodiment, the index file is composed of data files, and is a sequential file with an index. The records are randomly accessed in the file, requiring knowledge of the addresses of the records. For example, a customer may want to access a bank account, neither the customer nor the teller may know the address of the customer's record, and the customer may only give the teller's own account number (key). Here, the index file may associate an account number and a record address.
In this embodiment, the index name refers to the name of the index in a popular way, wherein the index is explicit according to a certain rule, and the purpose is to distinguish the indexes of the respective versions. For example, if the index name is elife and the sequence number is 3, the index name of the generated Annoy is elife _1, and the index name in es is elife _ 1. When the index is generated again, the sequence number becomes 4, and the generated Annoy index name is elife _0, and the index name in es is elife _ 0. And according to the index name, inquiring the index name from the Redis to acquire a corresponding index file.
Step S140, using the first index name as a search formula, searching a second index name matched with the search formula from an elastic search, and using the second index name as a latest ES index of the target question-answering engine;
in the present embodiment, the search formula is an instruction issued by the searcher to the computer, and is also a language of a human-computer conversation, and the search formula expresses the search intention of the searcher. Typically consisting of terms, logical operators, wildcards, etc.
The search term is a collective representation of the search intention of the searcher, and is also a constituent body of the search language. The search term is a generic term, and in the topic search, the search term category includes a free term, a keyword, a subject term, and the like. When more than two search terms are used for searching, the relation between the terms is connected by a logic operator to express the search intention of a searcher. Logical operations in a database typically use "or", "and" or "not" to denote relationships between terms. The three logic operators can be assembled according to the retrieval requirements to form a complex retrieval formula.
The search expression includes parentheses indicating the combination first operation in the parentheses. For example, looking up the literature on the processing of hawthorn foods, the formula can be found as follows: (Hawthorn fruit, haw and Chinese hawthorn) 4. In the search formula without priority, some search systems perform operations from left to right in word order, and some systems artificially specify the operation order by priority.
The wildcard characters mainly comprise word interceptors, substitutes and position operators. Wherein, the keyword is marked with "" or "? "is used for representing. During retrieval, word interceptors are added at different positions of the word stem to represent variable parts of the word, so that the input of the retrieval words with the same word stem is reduced, and the recall ratio is improved. Meanwhile, English vocabulary often causes the phenomenon of one word and multiple characters due to different English spelling and American spelling, and the changes are caused by different pronunciations. And in order to prevent the search word from being missed, a substitute character is marked at the changed position to replace the changed letter. Further, the position operator has the function of indicating the position relation of two search terms.
In this embodiment, the index file is read into the memory through an Annoy interface, the Elasticsearch searches through the corresponding index name, a new index name of the index file in the Elasticsearch is retrieved, and the new index name is replaced as the index name of the index file, so as to complete the updating of the index.
In this embodiment, the loading literally means increasing the loading amount, and in this embodiment, the loading of the file or information when the program is started is indicated, that is, the required file and information are read into the memory.
In this embodiment, the memory is one of important components in the computer, and is also called a storage, and is used for temporarily storing operation data in the CPU.
In this embodiment, the elastic search is a distributed, highly-extended, and highly-real-time search and data analysis engine.
Step S150, resetting the index update state of the target question and answer engine and determining the target question and answer engine to be updated next based on the configuration information to perform index update.
In this embodiment, the target question-answering engine refers to a question-answering engine that receives the index update request and completes the index update. When the question-answering engine receives the question-answering updating request, the new index updating request can not be received in order to complete the updating of the current index, so that the state of the question-answering engine is set to be an index updating state, and the host corresponding to the current question-answering engine is in an unavailable state to refuse to receive the new index updating request.
If and only if the current index update is completed, the question answering engine can be restored to a normal state and waits for receiving the next index update request. After the current question-answering engine is updated successfully, notifying other question-answering engines according to the IP list information of the question-answering engine in the configuration information, acquiring the latest index from Redis and Elasticissearch, receiving an index updating request, and finishing index updating.
In this embodiment, the reset refers to resetting, which can be extended to resetting and restoring to the original state. After the target question-answering engine completes the index update, the state is restored from the index update state to the original state to receive a new index update request.
In this embodiment, a target question-answering engine to be updated is determined according to configuration information preset in a question-answering system, when the target question-answering engine receives an index updating request, the corresponding target question-answering engine is set to be in an index updating state, and an index serial number in preset Redis is read to obtain a corresponding Annoy index file; and then, taking the index name of the index file as a search formula, searching the latest ES index corresponding to the target question-answering engine from the elastic search, resetting the state of the corresponding target question-answering engine after the index is updated, determining the next question-answering engine to be updated according to the configuration information, and completing the updating of all the question-answering engine indexes. The invention reduces the dependency of the current index updating on the offline manual processing, ensures the correct operation of the index updating process and further reduces the development and maintenance cost.
Referring to fig. 3, fig. 3 is a flowchart illustrating an index updating method of the question-answering system according to a second embodiment of the present invention. In this embodiment, before the step S110, the method further includes:
step S210, obtaining an original corpus;
in this embodiment, the corpus is all collected question-answer pairs, which may be understood as all historical question-answer record sets.
In this embodiment, when there is a need to update the corpus, the original text captured from the related website or the manually collected text information is converted into a uniform electronic text format, so as to obtain the original corpus.
Step S220, performing corpus collection and labeling on the original corpus, and outputting a labeled corpus;
in this embodiment, the collected original corpus is analyzed, entities, relationships, and events are sequentially selected, the specific categories to which the entities, relationships, and events belong are determined and labeled, and the labeled contents are automatically stored in a uniform format according to a predefined coding mode, so that the parsed corpus is obtained.
Step S230, verifying the labeled corpus and outputting a verification result;
in this embodiment, the corpus can be entered into the corpus management platform only when the corpus meets the entry specification of the corpus management platform, so before entry, the corpus to which the labeling is completed is subjected to eligibility verification, and a verification result is output. The eligibility check of the corpus includes but is not limited to: whether the characters are repeated or not, whether the characters contain special characters or not and whether the input format meet the specification of the corpus management platform or not. For example, whether the special characters are included or not is judged, wherein the special characters are processed in a blacklist mode, only Chinese characters, English letters, numbers and common punctuation marks are allowed, and all characters except the Chinese characters, the English letters, the numbers and the common punctuation marks are special characters; or, format check: the corpus entry of the corpus management platform must be in a question-answer pair form, and the entry requirement can be met only in the question-answer pair form.
Step S240, based on the verification result, storing the labeled corpus meeting the qualification evaluation to a corpus management platform;
in this embodiment, the corpus management platform is a platform for corpus entry and management of entered corpora.
In the embodiment, according to the verification result, if the verification result is qualified, the labeled corpus meeting the qualification evaluation is stored in the expected management platform; and if the verification result is unqualified, rejecting the labeled corpus.
In this embodiment, the storage location of the labeled corpus is not limited, and the corpus may be stored in the corpus management platform or in a local database.
Step S250, based on a preset interface, calling a question and answer engine to acquire the labeled corpus in the corpus management platform and storing the labeled corpus to Redis;
in this embodiment, the account interface is an interface of the corpus management platform, and the tagged corpus can be synchronously transmitted to the question-answering engine through the interface. After the markup language materials of the language material management platform are input, the system sends a language material synchronization instruction to the question and answer engine, further, a preset interface capable of synchronously transmitting the language materials is called, the markup language materials of the language material management platform are received through the interface, and reading and writing of the language materials by the question and answer system are completed.
In this embodiment, after the corpus is successfully entered, when the index module of the question and answer engine is started, the index module registers the notification update interface address to the corpus management platform, notifies the question and answer engine to update the corpus synchronously, stores the corpus to Redis, and pulls data in batches in pages according to the timestamp of the history pulled corpus to complete the synchronization of the corpus. The timestamp of the historical pulled corpus refers to the maximum value of the last updating time of the last pulled corpus.
In this embodiment, Redis is a high-performance memory, i.e., a database, and the query-answering engine stores the corpus to Redis after completing the synchronization of the corpus
Step S260, acquiring the markup corpus from Redis, and preprocessing the markup corpus to obtain semantic vectors of the markup corpus;
in this embodiment, as mentioned above, Redis is a high performance memory, i.e., a database. And acquiring updated linguistic data from Redis, and preprocessing the linguistic data to acquire semantic vectors of the linguistic data.
In this embodiment, the semantic vector has two layers of meanings: one is a vector, which is to identify the text by the vector; the semantic vector refers to a vector with certain meaning, which is mainly expressed as: semantically similar words are close in distance on the vector. Such as: the two sentences of the phrase "can not drink water" and the phrase "can not drink water" are the same, in this case, the semantic similarity is higher, and the distance on the vector is similar.
Step S270, generating an Annoy index file based on the semantic vector of the labeled corpus.
In this embodiment, according to the semantic vector of the corpus, a dynamic link library of the Annoy is called, and an Annoy index file is generated.
In this embodiment, a Dynamic Link Library (DLL) is a program module that can be shared by other applications, and some routines and resources that can be shared are packaged in the program module.
Referring to fig. 4, fig. 4 is a flowchart illustrating an index updating method of the question-answering system according to a third embodiment of the present invention. In this embodiment, after the step S270, the method further includes:
step S310, writing the Annoy index file into Redis, and saving an intermediate corpus used when the Annoy index file is generated;
in this embodiment, the generated Annoy index file is stored in the Redis, and the Annoy index file has an increasing sequence number in the Redis. For example: if the index name is elife and the sequence number is 3, the index name of the generated Annoy is elife _1, and the index name in es is elife _ 1. When the index is generated again, the sequence number becomes 4, and the generated Annoy index name is elife _0, and the index name in es is elife _ 0.
In this embodiment, the corpus sequence used when the Annoy index file is generated is the intermediate corpus.
In this embodiment, the Annoy algorithm search only returns the position in the Annoy index, for example, the Annoy index has 100 pieces of data in total, the Annoy algorithm search only returns the position information, for example, 10, and in order to find the corresponding text information through the position information, the corpus sequence used in generating the Annoy index file, that is, the intermediate corpus, is obtained.
Step S320, constructing request parameter information of an ES index calling interface based on the Annoy index file and the intermediate corpus;
in this embodiment, relevant data that needs to be stored in the elastic search is supplemented according to the Annoy index file and the corpus sequence used when the file is generated. Since the answer pairs are in the Annoy index file, only text information exists, and the ES index has more associated information, such as fields of mechanisms, objects and the like.
In this embodiment, request parameter information of a call interface of the ES index is constructed, and the interface is called according to the request parameter information to generate the ES index.
Step S330, based on the request parameter information, calling the ES index calling interface to generate an ES index;
in this embodiment, the ES index uses an elastic search (search server) as a database, stores data in the elastic search according to rules of the elastic search, and then retrieves a stored data set through the elastic search.
In this embodiment, a single question-answering engine corresponds to a plurality of knowledge bases, and the whole updating process is executed in parallel; that is, the two sections of parts for updating the corpus and generating the index file are performed in parallel, and through a message notification mechanism, the processes can be ensured to be performed sequentially, but the processes are not influenced mutually.
It should be noted that the updating of the Annoy index file is successful, and the Elasticissearch index can be notified, so that the consistency of the data Elasticissearch and the Annoy index is ensured.
Step S340, storing the Annoy index file to Redis, and storing the ES index to an elastic search.
In this embodiment, the ES index is a data set that uses an elastic search as a database, stores data to the elastic search according to a storage rule of the elastic search, and then retrieves the data through the elastic search.
In this embodiment, Redis is a high-performance memory, i.e., a database.
In this embodiment, the elastic search is a distributed, highly-extended, and highly-real-time search and data analysis engine. Although not a database, in many cases, it is used as a document-type database.
In this embodiment, the announce index and the ES index are stored in the Redis and the elastic search, where the announce index file has an incremented sequence number in the Redis. For example: if the index name is elife and the sequence number is 3, the index name of the generated Annoy is elife _1, and the index name in the ES is elife _ 1. When the index is generated again, the sequence number becomes 4, the generated announce index name is elife _0, and the index name in the ES is elife _ 0.
In this embodiment, the incremented sequence number is only for distinguishing two versions, and two values, i.e. 0 and 1, may also be used, where when the current value is 0, the update completion request is set to 1, and the next update is 0. The latest index name is obtained based on the incremented sequence number and the same logic as the faq-index module.
Referring to fig. 5, fig. 5 is a schematic view of a detailed flow of the step S110 in fig. 2. Based on the foregoing embodiment, in this embodiment, the foregoing step S110 further includes:
step S1101, receiving an index updating request and acquiring IP list information corresponding to the question answering engine;
in this embodiment, because of the distributed service, there are multiple question-answering engines to obtain the IP information of each question-answering engine.
Step S1102, determining each target question-answering engine to be updated based on each IP address in the IP list information corresponding to the question-answering engine.
In this embodiment, after generating an index file according to the IP information of each question and answer engine, sequentially invoking interfaces of each question and answer engine, and depending on the health detection mechanism of springcluster, when receiving an index update request, first reading an incremental sequence number in Redis, then using logic as the faq-index module to obtain an index name of a latest index, and then obtaining an index file from Redis through the index name if the index is at the end of 0 or 1, saving the index file to the local, reloading the index file into a memory, completing update, and switching logic for matching word and face to query the index name of the index from the Elasticsearch as the latest index name.
Referring to fig. 6, fig. 6 is a schematic flow chart of an index updating method of the question-answering system according to a fourth embodiment of the present invention. In this embodiment, after the step S160, the method further includes:
step S410, if the index of the target question answering engine fails to be updated, setting the target question answering engine to be in an index updating failure state;
in this embodiment, if the index of the target question-answering engine fails to be updated, the target question-answering area engine is set to be in the index update failure state, so as to receive the index update request again to update the index. For example, when question answering engine No. 1 receives an index update request to update the index, question answering engine No. 1 is in the index update state. At this point, question and answer engine number 1 no longer receives a new index update request. If the index updating fails due to the fault, the question answering engine No. 1 is set to be in an index updating failure state, and therefore the index updating request is redistributed to the host corresponding to the question answering engine No. 1.
In this embodiment, when the question-answering engine fails to update, the faq-index will try to update again until the question-answering engine completes the update. The retry interval waiting time is increased with the number of updating attempts, such as: when the question-answering engine fails to update, faq-index tries to update again, the first try fails, the retry is carried out after 5 seconds, the retry is carried out after 10 seconds after the second failure, the retry is carried out after 15 seconds after the third failure, the fourth time and the fifth time are separated by 20 seconds, and the waiting time of each interval of the whole process is increased.
Step S420, based on the health detection mechanism of the question answering engine, detecting whether the question answering engine in the index updating failure state exists;
in this embodiment, the health detection of the question-answering engine is a detection method that can ensure whether the target question-answering engine is available. And determining whether the question-answering engine in the index updating failure state exists at present or not by detecting the health condition of the question-answering engine corresponding to each service. The detection modes of the health monitoring mechanism are mainly divided into three types:
reporting: reporting whether the current service state is available or not every service interval;
active detection: the registration center actively detects whether the service is normally available at a certain time interval;
the detection and reporting modes are all through interfaces.
The supplementary measures are to judge whether the returned result of the service is normal or not, and the mode is to judge whether the service is normal or not.
Step S430, if yes, sending an index updating request to the question answering engine in the index updating failure state again;
in this embodiment, the registry may perform health detection on the health status of the question-answering engine corresponding to each service, and detect whether there is a question-answering engine in an index update failure state currently. And if the question-answering engine in the index updating failure state exists, the index updating request is retransmitted to the question-answering engine, and the index updating is completed. For example, when it is detected that the question-answering engine No. 1 is in the update failure state, the index update request is retransmitted to the question-answering engine No. 1, and the index update is continuously completed.
Step S440, if the index updating times of the target question answering engine exceed the preset times and the index updating is not completed, sending a reminding mail to a preset receiver mailbox.
In this embodiment, if the index of the target question-answering engine fails to be updated, the index update request is received again to complete the update of the index, and if the retry number exceeds the preset number, the index cannot be updated, the email is sent to the corresponding mailbox of the responsible person to remind the question-answering engine that the index is updated abnormally. For example, after the index update of the question and answer engine No. 1 fails, the index update request is received again, the index update is attempted to be completed, but if the index update is not completed after all the retry times, the update is not attempted any more, the system directly sends an email to the service system responsible person and the faq engine responsible person to remind the service system responsible person of the index update failure, and the relevant responsible person is asked to check the index update process.
In this embodiment, when the index of one question-answering engine fails to be updated, the question-answering engine retries the updating of the index. However, as the number of updating attempts increases, the time interval of retry increases, for example, the first attempt fails, and the retry is performed for the second time after 5 seconds, the retry is performed for the second time after 10 seconds, the retry is performed for the third time after 15 seconds, the fourth time and the fifth time are separated by 20 seconds, and the waiting time of each interval of the whole process increases.
Referring to fig. 7, fig. 7 is a schematic flow chart of an index updating method of the question-answering system according to a fifth embodiment of the present invention. In this embodiment, after step S440, the method further includes:
step S510, judging whether index version numbers corresponding to the question answering engines are consistent or not based on the IP list information corresponding to the question answering engines;
in this embodiment, the IP list information corresponding to the question and answer engine refers to a logical address corresponding to each question and answer engine.
In this embodiment, since the system has a plurality of question-answering engines, and the update of each question-answering engine is not successful, there may be a situation that indexes of faq engines under a plurality of machines are inconsistent, and at this time, the system may determine the index version corresponding to each question-answering engine at present, and output the determination result.
In this embodiment, the index inconsistency refers to that there is a part of the question answering engine that uses the index of the latest version, and another part that uses the index of the old version due to the failure of notification, the new and old index versions are inconsistent, and the content is inconsistent, that is, the indexes are inconsistent.
Step S520, if the index version numbers corresponding to the question answering engines are not consistent, respectively counting the index version numbers corresponding to the question answering engines;
in this embodiment, since the system has a plurality of question-answering engines, and not all question-answering engines will successfully complete the updating of the index, which results in that a part of question-answering engines use the index of the new version, and another part of question-answering engines use the index of the old version, it is determined whether the index versions corresponding to each question-answering engine are consistent. For example, there are 50 question-answering engines in the system, wherein only 35 question-answering engines successfully complete the updating of the index, which results in that 35 question-answering engines of the 50 question-answering engines use the index of the new version, and the remaining 15 question-answering engines use the index of the old version, and the index versions corresponding to the 50 question-answering engines are inconsistent.
Step S530, determining the question-answering engines with the corresponding index version numbers in a minority based on the index version numbers corresponding to the question-answering engines, and setting the question-answering engines with the corresponding index version numbers in a minority to be in an unavailable state.
In this embodiment, if the index versions corresponding to each question-answering engine are not consistent, the index of which version is the majority is seen according to the judgment result, and then the question-answering engines which are the minority are set to be in the unavailable state, so that the question-answering engines which are the minority are removed from the load through a distributed health detection mechanism.
In this embodiment, the method for setting the question-answering engine corresponding to the service to be in the unavailable state is as follows:
because all services depend on the framework, the framework is provided with a registration center, and the services developed by using the framework are registered in the registration center;
the registry detects the health condition of the question-answering engines corresponding to each service, and when a question-answering engine is detected to be unavailable, unhealthy question-answering engines in the service are eliminated.
Realizing the calling among the services depending on the framework, and acquiring an IP list of a question answering engine of the called service depending on a registration center;
when a certain question-answering engine corresponding to a certain service is unavailable, other services call the service through a framework, and the registration center only returns the service in a healthy state to a calling party, so that the calling process is normal.
In this embodiment, there are three detection modes of the health monitoring mechanism:
(1) reporting, wherein each service reports whether the current service state is available or not at intervals;
(2) detecting, namely actively detecting whether the service is normally available by a registration center at a certain time interval;
the detection and report modes are all through interfaces;
(3) and judging whether the service is normally available or not according to the returned result of the service.
Referring to fig. 8, fig. 8 is a functional block diagram of an embodiment of an index updating apparatus of the question-answering system according to the present invention. In this embodiment, the index updating apparatus of the question answering system includes:
a receiving module 10, configured to receive an index update request, and determine a target question-answering engine to be updated based on configuration information preset in the question-answering system;
the obtaining module 20, based on the index updating request, sets the target question-answering engine to be in an index updating state, reads an index sequence number in a preset Redis, and obtains an index name of a latest first index corresponding to the index sequence number;
a loading module 30, configured to obtain a corresponding Annoy index file from the Redis based on the first index name, and load the Annoy index file into the memory;
an updating module 40, configured to query, using the first index name as a search formula, a second index name of the index from an elastic search, and use the second index name as a latest ES index of the target question-answering engine;
and the resetting module 50 is used for resetting the index updating state of the target question-answering engine and determining the next target question-answering engine to be updated to update the index based on the configuration information.
Optionally, in a specific embodiment, the index updating apparatus of the question-answering system further includes:
the corpus storage module is used for acquiring original corpuses, collecting and labeling the corpuses of the original corpuses, and outputting labeled corpuses; verifying the labeled corpus, outputting a verification result, and storing the labeled corpus meeting the qualification evaluation to a corpus management platform based on the verification result
The calling module is used for calling a question and answer engine based on a preset interface to acquire the labeled corpus in the corpus management platform and store the labeled corpus in Redis;
the preprocessing module is used for acquiring the labeling linguistic data from Redis and preprocessing the labeling linguistic data to obtain semantic vectors of the labeling linguistic data;
and the Annoy index file generation module is used for generating an Annoy index file based on the semantic vector of the labeled corpus.
Optionally, in a specific embodiment, the index updating apparatus of the question-answering system further includes: :
the writing module is used for writing the Annoy index file into Redis and storing an intermediate corpus used when the Annoy index file is generated;
the calling module is used for constructing request parameter information of an ES index calling interface based on the Annoy index file and the intermediate corpus;
the ES index generation module is used for calling the ES index calling interface based on the request parameter information to generate an ES index;
and the index storage module is used for storing the Annoy index file to Redis and storing the ES index to an Elasticissearch.
Optionally, in a specific embodiment, the receiving module is specifically configured to:
and receiving an index updating request, acquiring IP list information corresponding to the question and answer engines, and determining each target question and answer engine to be updated based on each IP address in the IP list information corresponding to the question and answer engines.
Further, the index updating device of the question-answering system further includes:
the sending module is used for setting the target question-answering engine to be in an index updating failure state when the index of the target question-answering engine fails to be updated; detecting whether a question answering engine in an index updating failure state exists or not based on a health detection mechanism of the question answering engine; when the question answering engine is in the index updating failure state, the index updating request is sent to the question answering engine in the index updating failure state again; and when the index updating times of the target question-answering engine exceed the preset times and the index updating is not completed, sending a reminding mail to a preset receiver mailbox.
Further, the index updating device of the question-answering system further includes:
the version number unifying module is used for judging whether the index version numbers corresponding to the question answering engines are consistent or not based on the IP list information of the question answering engines; when the index version numbers corresponding to the question and answer engines are not consistent, respectively counting the version numbers of the indexes corresponding to the question and answer engines; and determining the question-answering engines with the corresponding index versions in a minority based on the version numbers of the indexes corresponding to the question-answering engines, and setting the question-answering engines with the index versions in the minority to be in an unavailable state.
In this embodiment, a target question-answering engine to be updated is determined according to configuration information preset in a question-answering system, when the target question-answering engine receives an index updating request, the corresponding target question-answering engine is set to be in an index updating state, and an index serial number in preset Redis is read to obtain a corresponding Annoy index file; and then, taking the index name of the index file as a search formula, searching the latest ES index corresponding to the target question-answering engine from the elastic search, resetting the state of the corresponding target question-answering engine after the index is updated, determining the next question-answering engine to be updated according to the configuration information, and completing the updating of all the question-answering engine indexes. The invention reduces the dependency of the current index updating on the offline manual processing, ensures the correct operation of the index updating process and further reduces the development and maintenance cost.
Based on the same description content of the embodiment as the above-mentioned index updating method of the question-answering system, the contents of the embodiment of the index updating apparatus of the question-answering system are not described in detail in this embodiment.
The invention also provides a computer readable storage medium.
In this embodiment, the computer-readable storage medium stores an index updating program of the question-answering system, and the index updating program of the question-answering system, when executed by the processor, implements the steps of the index updating method of the question-answering system described in any one of the above embodiments. The method implemented by the index updating program of the question-answering system when executed by the processor may refer to various embodiments of the index updating method of the question-answering system, and thus, will not be described in detail.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM), and includes instructions for causing a terminal (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The present invention is described in connection with the accompanying drawings, but the present invention is not limited to the above embodiments, which are only illustrative and not restrictive, and those skilled in the art can make various changes without departing from the spirit and scope of the invention as defined by the appended claims, and all changes that come within the meaning and range of equivalency of the specification and drawings that are obvious from the description and the attached claims are intended to be embraced therein.
Claims (10)
1. An index updating method of a question-answering system is characterized by comprising the following steps of:
receiving an index updating request, and determining a target question-answering engine to be updated based on preset configuration information in the question-answering system;
setting the target question-answering engine to be in an index updating state based on the index updating request, reading an index sequence number in a preset Redis, and acquiring a first index name of an index corresponding to the index sequence number;
based on the first index name, acquiring a corresponding Annoy index file from Redis and loading the Annoy index file into a memory;
taking the first index name as a search formula, inquiring a second index name matched with the search formula from an Elasticissearch, and taking the second index name as the latest ES index of the target question-answering engine;
and resetting the index updating state of the target question-answering engine and determining the target question-answering engine to be updated next to update the index based on the configuration information.
2. The index updating method of the question-answering system according to claim 1, wherein before the step of receiving an index update request and determining a target question-answering engine to be updated based on configuration information preset in the question-answering system, comprising:
acquiring an original corpus;
performing corpus collection and labeling on the original corpus, and outputting labeled corpus;
verifying the labeled corpus and outputting a verification result;
based on the verification result, storing the labeled corpus meeting the qualification evaluation to a corpus management platform;
based on a preset interface, calling a question-answering engine to acquire the labeled corpus in the corpus management platform and storing the labeled corpus to Redis;
acquiring the markup corpus from Redis, and preprocessing the markup corpus to obtain a semantic vector of the markup corpus;
and generating an Annoy index file based on the semantic vector of the labeled corpus.
3. The index updating method of the question-answering system according to claim 2, wherein after the step of generating the Annoy index file based on the semantic vectors of the corpus, further comprising:
writing the Annoy index file into Redis, and storing an intermediate corpus used when the Annoy index file is generated;
constructing request parameter information of an ES index calling interface based on the Annoy index file and the intermediate corpus;
based on the request parameter information, calling the ES index calling interface to generate an ES index;
storing the Annoy index file to Redis, and storing the ES index to an Elasticissearch.
4. The index updating method of the question-answering system according to claim 1, wherein the receiving an index update request and determining a target question-answering engine to be updated based on configuration information preset in the question-answering system comprises:
receiving an index updating request and acquiring IP list information corresponding to the question answering engine;
and determining each target question-answering engine to be updated based on each IP address in the IP list information corresponding to the question-answering engine.
5. The index updating method of the question-answering system according to claim 1, wherein after the step of resetting the index updating state of the target question-answering engine and determining the target question-answering engine to be updated next for index updating based on the configuration information, further comprising:
if the target question-answering engine index fails to be updated, setting the target question-answering engine to be in an index updating failure state;
detecting whether a question answering engine in an index updating failure state exists or not based on a health detection mechanism of the question answering engine;
if so, sending the index updating request to the question answering engine in the index updating failure state again;
and if the index updating times of the target question-answering engine exceed the preset times and the index updating is not finished, sending a reminding mail to a preset receiver mailbox.
6. The index updating method of a question-answering system according to claim 5, wherein after the step of sending a reminding mail to a preset recipient mailbox if the index updating times of the target question-answering engine exceed the preset times and the index updating is not completed, further comprising:
judging whether the index version numbers corresponding to the question and answer engines are consistent or not based on the IP list information corresponding to the question and answer engines;
if the index version numbers corresponding to the question-answering engines are not consistent, respectively counting the index version numbers corresponding to the question-answering engines;
and determining the question-answering engines with the corresponding index version numbers occupying a few number based on the index version numbers corresponding to the question-answering engines, and setting the question-answering engines with the corresponding index version numbers occupying a few number to be in an unavailable state.
7. An index updating apparatus of a question-answering system, characterized by comprising:
the receiving module is used for receiving an index updating request and determining a target question-answering engine to be updated based on preset configuration information in the question-answering system;
an obtaining module, configured to set the target question-answering engine to an index update state based on the index update request, read an index sequence number in a preset Redis, and obtain an index name of a latest first index corresponding to the index sequence number;
the loading module is used for acquiring a corresponding Annoy index file from Redis based on the first index name and loading the Annoy index file into a memory;
the updating module is used for inquiring a second index name of the index from an elastic search by taking the first index name as a search formula, and taking the second index name as the latest ES index of the target question-answering engine;
and the resetting module is used for resetting the index updating state of the target question-answering engine and determining the next target question-answering engine to be updated to update the index based on the configuration information.
8. The index updating apparatus of the question-answering system according to claim 7, characterized in that the index updating apparatus of the question-answering system further comprises:
the corpus storage module is used for acquiring original corpuses, collecting and labeling the corpuses of the original corpuses, and outputting labeled corpuses; verifying the labeled corpus, outputting a verification result, and storing the labeled corpus meeting the qualification evaluation to a corpus management platform based on the verification result;
the calling module is used for calling a question and answer engine based on a preset interface to acquire the labeled corpus in the corpus management platform and store the labeled corpus in Redis;
the preprocessing module is used for acquiring the labeling linguistic data from Redis and preprocessing the labeling linguistic data to obtain semantic vectors of the labeling linguistic data;
and the Annoy index file generation module is used for generating an Annoy index file based on the semantic vector of the corpus.
9. An index updating apparatus of a question-answering system, characterized in that the index updating apparatus of the question-answering system comprises a memory, a processor and an index updating program of the question-answering system stored on the memory and operable on the processor, the index updating program of the question-answering system realizing the steps of the index updating method of the question-answering system according to any one of claims 1 to 6 when executed by the processor.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon an index updating program of a question-answering system, which when executed by a processor, implements the steps of the index updating method of a question-answering system according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010138638.7A CN111324689A (en) | 2020-03-03 | 2020-03-03 | Index updating method, device, equipment and storage medium of question-answering system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010138638.7A CN111324689A (en) | 2020-03-03 | 2020-03-03 | Index updating method, device, equipment and storage medium of question-answering system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111324689A true CN111324689A (en) | 2020-06-23 |
Family
ID=71173040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010138638.7A Pending CN111324689A (en) | 2020-03-03 | 2020-03-03 | Index updating method, device, equipment and storage medium of question-answering system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111324689A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112214571A (en) * | 2020-10-10 | 2021-01-12 | 中国平安人寿保险股份有限公司 | KV storage-based index generation method, device, equipment and medium |
| CN112463944A (en) * | 2020-12-22 | 2021-03-09 | 安徽商信政通信息技术股份有限公司 | Retrieval type intelligent question-answering method and device based on multi-model fusion |
| CN112966007A (en) * | 2021-04-02 | 2021-06-15 | 新华智云科技有限公司 | Search terminal control method and search terminal |
| CN113031992A (en) * | 2021-04-27 | 2021-06-25 | 中国平安人寿保险股份有限公司 | Annoy hot update method, device, equipment and medium |
| CN113569022A (en) * | 2021-06-30 | 2021-10-29 | 四川新网银行股份有限公司 | Method for realizing dialogue robot response engine based on cascading search |
| CN115186679A (en) * | 2022-07-15 | 2022-10-14 | 广东广信通信服务有限公司 | Intelligent response method, device, computer equipment and storage medium |
| CN116578666A (en) * | 2023-07-12 | 2023-08-11 | 拓尔思信息技术股份有限公司 | Segment sentence position inverted index structure design and limited operation full text retrieval method thereof |
-
2020
- 2020-03-03 CN CN202010138638.7A patent/CN111324689A/en active Pending
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112214571A (en) * | 2020-10-10 | 2021-01-12 | 中国平安人寿保险股份有限公司 | KV storage-based index generation method, device, equipment and medium |
| CN112214571B (en) * | 2020-10-10 | 2024-02-20 | 中国平安人寿保险股份有限公司 | Index generation method, device, equipment and medium based on KV storage |
| CN112463944A (en) * | 2020-12-22 | 2021-03-09 | 安徽商信政通信息技术股份有限公司 | Retrieval type intelligent question-answering method and device based on multi-model fusion |
| CN112463944B (en) * | 2020-12-22 | 2023-10-24 | 安徽商信政通信息技术股份有限公司 | Search type intelligent question-answering method and device based on multi-model fusion |
| CN112966007A (en) * | 2021-04-02 | 2021-06-15 | 新华智云科技有限公司 | Search terminal control method and search terminal |
| CN112966007B (en) * | 2021-04-02 | 2022-06-17 | 新华智云科技有限公司 | Search terminal control method and search terminal |
| CN113031992A (en) * | 2021-04-27 | 2021-06-25 | 中国平安人寿保险股份有限公司 | Annoy hot update method, device, equipment and medium |
| CN113569022A (en) * | 2021-06-30 | 2021-10-29 | 四川新网银行股份有限公司 | Method for realizing dialogue robot response engine based on cascading search |
| CN113569022B (en) * | 2021-06-30 | 2023-06-09 | 四川新网银行股份有限公司 | Method for realizing dialogue robot response engine based on cascade search |
| CN115186679A (en) * | 2022-07-15 | 2022-10-14 | 广东广信通信服务有限公司 | Intelligent response method, device, computer equipment and storage medium |
| CN116578666A (en) * | 2023-07-12 | 2023-08-11 | 拓尔思信息技术股份有限公司 | Segment sentence position inverted index structure design and limited operation full text retrieval method thereof |
| CN116578666B (en) * | 2023-07-12 | 2023-09-22 | 拓尔思信息技术股份有限公司 | Segment sentence position inverted index structure design and limited operation full text retrieval method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111324689A (en) | Index updating method, device, equipment and storage medium of question-answering system | |
| US7043690B1 (en) | Method, system, and program for checking contact information | |
| JP4093012B2 (en) | Hypertext inspection apparatus, method, and program | |
| US7003725B2 (en) | Method and system for normalizing dirty text in a document | |
| US8112436B2 (en) | Semantic and text matching techniques for network search | |
| US9524291B2 (en) | Visual display of semantic information | |
| US7814097B2 (en) | Discovering alternative spellings through co-occurrence | |
| US8892580B2 (en) | Transformation of regular expressions | |
| US9032031B2 (en) | Apparatus, method and computer program product for processing email, and apparatus for searching email | |
| US8407781B2 (en) | Information providing support device and information providing support method | |
| US20080104037A1 (en) | Automated scheme for identifying user intent in real-time | |
| US20150095320A1 (en) | Apparatus, systems and methods for scoring the reliability of online information | |
| CN107577755B (en) | Searching method | |
| CN109656999B (en) | Method, device, storage medium and apparatus for synchronizing large data volume data | |
| CN109597874B (en) | Information recommendation method, device and server | |
| JPH10501912A (en) | System and method for portable document indexing using N-gram word decomposition | |
| US10394939B2 (en) | Resolving outdated items within curated content | |
| CN110874403B (en) | Question answering system, question answering processing method, and question answering integration system | |
| WO2010015068A1 (en) | Topic word generation method and system | |
| US20200334639A1 (en) | Email content modification system | |
| US20040158558A1 (en) | Information processor and program for implementing information processor | |
| US10936638B2 (en) | Random index pattern matching based email relations finder system | |
| WO2012047214A2 (en) | Visual display of semantic information | |
| WO2011044025A2 (en) | Method and system for taking actions based on analysis of enterprise communication messages | |
| CN111737443B (en) | Answer text processing method and device and key text determining method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |