CN115859998A

CN115859998A - Problem data processing method and device, computer equipment and storage medium

Info

Publication number: CN115859998A
Application number: CN202211560633.9A
Authority: CN
Inventors: 曹贵邦
Original assignee: Ping An Property and Casualty Insurance Company of China Ltd
Current assignee: Ping An Property and Casualty Insurance Company of China Ltd
Priority date: 2022-12-07
Filing date: 2022-12-07
Publication date: 2023-03-28

Abstract

The embodiment of the application belongs to the field of artificial intelligence and relates to a problem data processing method, which comprises the following steps: judging whether a standard question adding request triggered by a user is received or not; if yes, analyzing a target standard question from the standard question adding request; determining a target similarity algorithm from a plurality of similarity algorithms; similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question answering system based on a target similarity calculation method, obtaining a plurality of similarity calculation results; judging whether a target similarity calculation result larger than a similarity threshold exists in all the similarity calculation results; if not, the target standard question is added into the question-answering system as a new standard question. The application also provides a processing device, computer equipment and storage medium of the problem data. In addition, the present application also relates to a block chain technology, and the target standard questions can be stored in the block chain. The method and the device can avoid the problem that the knowledge points are repeatedly constructed in the question-answering system, and ensure the construction quality of the question-answering system.

Description

Problem data processing method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for processing problem data, a computer device, and a storage medium.

Background

The question-answering system based on the FAQ is one of common technical forms of industrial application of the intelligent question-answering system, and is widely applied to various robots in the financial field due to accurate, flexible and controllable question answering. The FAQ knowledge base is used as important bottom-layer knowledge in the FAQ question-answering system, and the quality and the quantity of the knowledge base directly influence the answer coverage rate and the accuracy rate of the FAQ question-answering system. Therefore, the quality of knowledge base maintenance is particularly critical.

The maintenance of the knowledge base in the traditional question answering system usually takes manual maintenance as the main part, and has the defect of heavy task in view of the manual maintenance. When the standard questions in the question-answering system are preset, because knowledge understanding by different people is thousands of people, the phenomenon that similar question sentences construct multiple pieces of knowledge is easy to occur in the question-answering system, so that the question-answering system has the problem that knowledge points are repeatedly constructed, and the construction quality of the question-answering system is influenced.

Disclosure of Invention

The embodiment of the application aims to provide a problem data processing method, a problem data processing device, computer equipment and a storage medium, so that the technical problems that when the existing standard questions in a question-answering system are preset, since different people understand thousands of knowledge, similar question sentences easily construct multiple pieces of knowledge in the question-answering system, knowledge points are repeatedly constructed in the question-answering system, and the construction quality of the question-answering system is affected are solved.

In order to solve the above technical problem, an embodiment of the present application provides a method for processing problem data, which adopts the following technical solutions:

judging whether a standard question adding request triggered by a user is received or not; wherein the standard question adding request carries a target standard question to be added;

if yes, analyzing the target standard question from the standard question adding request;

determining a target similarity algorithm from a plurality of preset similarity algorithms;

similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question answering system based on the target similarity calculation method, and a plurality of corresponding similarity calculation results are obtained; each question sentence data comprises a standard question and one or more similar sentences corresponding to the standard question;

judging whether a target similarity calculation result larger than a preset similarity threshold exists in all the similarity calculation results;

and if the target similarity calculation result does not exist, adding the target standard question as a new standard question into the question-answering system.

Further, the step of determining a target similarity algorithm from a plurality of preset similarity algorithms specifically includes:

calling a preset similarity algorithm statistical data table;

acquiring the processing efficiency, accuracy and release time of each similarity algorithm from the similarity algorithm statistical data table;

determining a first preset weight, a second preset weight and a third preset weight which respectively correspond to the processing efficiency, the accuracy and the release time;

calculating the processing efficiency, accuracy and release time of each similarity algorithm based on the first preset weight, the second preset weight and the third preset weight to generate a processing comprehensive score of each similarity algorithm;

screening out a designated similarity algorithm with the maximum processing comprehensive score from all the similarity algorithms;

and taking the specified similarity algorithm as the target similarity algorithm.

Further, after the step of adding the target standard question as a new standard question to the question-answering system, the method further comprises the following steps:

calling a pre-trained problem generation model;

inputting the target standard question into the question generation model, and generating a first similar question corresponding to the target standard question based on the question generation model;

establishing an association relation between the target standard question and the first similar question;

and storing the first similar question into the question system based on the incidence relation.

Further, before the step of invoking the pre-trained problem generation model, the method further includes:

acquiring preset standard question sample data;

acquiring similar problem data corresponding to the standard question sample data from a preset data platform;

constructing training data based on the standard question sentence sample data and the similar question data;

calling a preset deep learning model;

and training the deep learning model based on the training data to generate the problem generation model.

determining a target field corresponding to the target standard question;

acquiring a target data source corresponding to the target field;

collecting a first statement from the target data source; wherein the number of the first sentences comprises a plurality;

respectively calculating the similarity between the target standard question and each first statement;

screening out second sentences of which the similarity meets a preset condition from all the first sentences on the basis of the similarity;

and generating a second similar question corresponding to the target standard question based on the second statement.

Further, the step of generating a second similar question corresponding to the target standard question based on the second sentence specifically includes:

acquiring a pre-stored question length limiting rule and a similar question number limiting rule;

screening out a third sentence which meets the question sentence length limiting rule from all the second sentences;

acquiring a quantity threshold corresponding to the similar question quantity limiting rule;

judging whether the number of the third sentences is larger than the number threshold value;

if yes, screening out fourth sentences which are the same as the quantity threshold value and have the highest similarity degree value from all the third sentences;

taking the fourth statement as the second similarity question.

Further, after the step of determining whether there is a target similarity calculation result greater than a preset similarity threshold in all the similarity calculation results, the method further includes:

if the similarity calculation results of the targets are larger than the similarity threshold value, refusing to add the target standard question in the question system;

generating failure reminding information corresponding to the target standard questions;

and displaying the failure reminding information.

In order to solve the above technical problem, an embodiment of the present application further provides a device for processing problem data, which adopts the following technical solutions:

the first judgment module is used for judging whether a standard question adding request triggered by a user is received or not; wherein the standard question adding request carries a target standard question to be added;

the analysis module is used for analyzing the target standard question from the standard question adding request if the target standard question is in the standard question adding request;

the first determining module is used for determining a target similarity algorithm from a plurality of preset similarity algorithms;

the calculation module is used for carrying out similarity calculation on the target standard question and each question sentence data stored in a preset question-answering system based on the target similarity calculation method to obtain a plurality of corresponding similarity calculation results; each question sentence data comprises a standard question and one or more similar sentences corresponding to the standard question;

the second judgment module is used for judging whether a target similarity calculation result larger than a preset similarity threshold exists in all the similarity calculation results or not;

and the adding module is used for adding the target standard question as a new standard question into the question-answering system if the target similarity calculation result does not exist.

In order to solve the above technical problem, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:

if yes, the target standard question is analyzed from the standard question adding request;

In order to solve the above technical problem, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:

judging whether a standard question adding request triggered by a user is received or not; the standard question adding request carries a target standard question to be added;

Compared with the prior art, the embodiment of the application mainly has the following beneficial effects:

when a standard question adding request triggered by a user is received, firstly, the target standard question is analyzed from the standard question adding request; then determining a target similarity algorithm from a plurality of preset similarity algorithms; subsequently, similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question-answering system based on the target similarity calculation method to obtain a plurality of corresponding similarity calculation results, and whether a target similarity calculation result larger than a preset similarity threshold exists in all the similarity calculation results is judged; and if the target similarity calculation result does not exist, adding the target standard question as a new standard question into the question-answering system. According to the method and the device, the similarity calculation result of the target standard question and each piece of question data stored in the preset question-answering system can be quickly and accurately calculated based on the use of a target similarity algorithm, further, whether standard question data similar to the target standard question to be added exist in the question-answering system can be accurately judged according to the numerical comparison result between the similarity calculation result and the preset similarity threshold, and the target standard question can be added into the question-answering system as a new standard question only if the standard question data similar to the target standard question to be added do not exist in the question-answering system, so that the problem that knowledge points are repeatedly constructed in the question-answering system can be effectively avoided, and the construction quality of the question-answering system is guaranteed.

Drawings

In order to more clearly illustrate the solution of the present application, the drawings needed for describing the embodiments of the present application will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow diagram of one embodiment of a method of processing issue data in accordance with the present application;

FIG. 3 is a schematic block diagram of one embodiment of an apparatus for processing problem data according to the present application;

FIG. 4 is a schematic block diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "including" and "having," and any variations thereof, in the description and claims of this application and the description of the above figures are intended to cover non-exclusive inclusions. The terms "first," "second," and the like in the description and claims of this application or in the above-described drawings are used for distinguishing between different objects and not for describing a particular order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The

terminal devices

101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Mov I picture Experts G roup Aud I o Layer I, mpeg compression standard audio Layer 3), MP4 players (Mov I ng P I ctu Experts G roup Aud I o Layer I V, mpeg compression standard audio Layer 4), laptop portable computers, desktop computers, and the like.

The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the

terminal devices

101, 102, 103.

It should be noted that the problem data processing method provided in the embodiments of the present application is generally executed by a server/terminal device, and accordingly, the problem data processing apparatus is generally disposed in the server/terminal device.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continuing reference to FIG. 2, a flow diagram of one embodiment of a method for processing issue data in accordance with the present application is shown. The problem data processing method comprises the following steps:

step S201, judging whether a standard question adding request triggered by a user is received; and the standard question adding request carries a target standard question to be added.

In this embodiment, the electronic device (for example, the server/terminal device shown in fig. 1) on which the processing method of the question data operates may acquire the target standard question to be added in a wired connection manner or a wireless connection manner. It should be noted that the wireless connection means may include, but is not limited to, a 3G/4G/5G connection, a wifi connection, a bluetooth connection, a wimax connection, a Z i gbee connection, a UWB (u l t ra W i deband) connection, and other wireless connection means now known or developed in the future. The standard question adding request is a request triggered by a user and used for adding the target standard question in the question answering system. The question-answering system can be FAQ question-answering system

And step S202, if yes, analyzing the target standard question from the standard question adding request.

In this embodiment, the target standard question may be parsed from the standard question adding request by performing information parsing on the standard question adding request.

Step S203, a target similarity algorithm is determined from a plurality of preset similarity algorithms.

In this embodiment, a specific implementation process of determining the target similarity algorithm from the multiple preset similarity algorithms is described in further detail in the following specific embodiments, and will not be described in detail herein.

Step S204, similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question answering system based on the target similarity algorithm, and a plurality of corresponding similarity calculation results are obtained; each question sentence data comprises a standard question and one or more similar sentences corresponding to the standard question.

In this embodiment, the first similarity between the target standard question and each standard question in the question-answering system is calculated by using a target similarity calculation method, and the second similarity between the target standard question and each similar sentence in the question-answering system is calculated by using a target similarity calculation method, and the first similarity and the second similarity are used as the similarity calculation result.

Step S205, determining whether a target similarity calculation result greater than a preset similarity threshold exists in all the similarity calculation results.

In this embodiment, the value of the similarity threshold is not specifically limited, and may be set according to actual service use requirements.

Step S206, if the target similarity calculation result does not exist, adding the target standard question as a new standard question into the question-answering system.

In this embodiment, when the target similarity calculation result larger than the similarity threshold does not exist in all the similarity calculation results, it indicates that standard question data similar to the target standard question to be added does not exist in the question-answering system, and at this time, new knowledge can be constructed in the question-answering system, so that the target standard question is added in the question-sentence system.

When a standard question adding request triggered by a user is received, the target standard question is firstly analyzed from the standard question adding request; then determining a target similarity algorithm from a plurality of preset similarity algorithms; subsequently, similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question-answering system based on the target similarity calculation method to obtain a plurality of corresponding similarity calculation results, and whether a target similarity calculation result larger than a preset similarity threshold exists in all the similarity calculation results is judged; and if the target similarity calculation result does not exist, adding the target standard question as a new standard question into the question-answering system. According to the method, the similarity calculation result of the target standard question and each question data stored in a preset question-answering system can be quickly and accurately calculated based on the use of a target similarity calculation method, further, whether standard question data similar to the target standard question to be added exist in the question-answering system or not can be accurately judged according to the numerical comparison result between the similarity calculation result and a preset similarity threshold, and the target standard question can be added into the question-answering system as a new standard question only if standard question data similar to the target standard question to be added do not exist in the question-answering system, so that the problem that knowledge points of the question-answering system are repeatedly constructed can be effectively avoided, and the construction quality of the question-answering system is guaranteed.

In some optional implementations, step S203 includes the following steps:

and calling a preset similarity algorithm statistical data table.

In this embodiment, the similarity algorithm statistical data table is a pre-constructed data table in which algorithm test data of a plurality of similarity algorithms are stored, and the algorithm test data may be data obtained by performing data test on each similarity algorithm by a related developer. Wherein the algorithmic test data includes at least processing efficiency, accuracy, and release time. In addition, the similarity algorithm may be various existing similarity algorithms, and for example, may include a semantic similarity calculation method based on a hamming distance, a cosine similarity calculation method, a calculation method based on a vector space model, and the like.

And acquiring the processing efficiency, accuracy and release time of each similarity algorithm from the similarity algorithm statistical data table.

And determining a first preset weight, a second preset weight and a third preset weight which respectively correspond to the processing efficiency, the accuracy and the release time.

In this embodiment, the values of the first preset weight, the second preset weight, and the third preset weight are not specifically limited, and may be set according to actual service usage requirements, and preferably, the sum of the first preset weight, the second preset weight, and the third preset weight is 1.

And calculating the processing efficiency, accuracy and release time of each similarity algorithm based on the first preset weight, the second preset weight and the third preset weight to generate a processing comprehensive score of each similarity algorithm.

In this embodiment, a weighted summation calculation manner may be adopted to calculate the processing efficiency, accuracy and release time of each similarity algorithm based on the first preset weight, the second preset weight and the third preset weight, so as to generate a processing comprehensive score of each similarity algorithm.

And screening out the designated similarity algorithm with the maximum processing comprehensive score from all the similarity algorithms.

The method comprises the steps of counting a data table by calling a preset similarity algorithm; then, acquiring the processing efficiency, accuracy and release time of each similarity algorithm from the similarity algorithm statistical data table; then, determining a first preset weight, a second preset weight and a third preset weight which respectively correspond to the processing efficiency, the accuracy and the release time; subsequently calculating the processing efficiency, accuracy and release time of each similarity algorithm based on the first preset weight, the second preset weight and the third preset weight to generate a processing comprehensive score of each similarity algorithm; and finally, screening out the specified similarity algorithm with the maximum processing comprehensive score from all the similarity algorithms, and taking the specified similarity algorithm as the target similarity algorithm. According to the method and the device, the processing comprehensive scores of various similarity algorithms can be quickly and accurately generated based on the processing efficiency, accuracy and issuing time of each similarity algorithm, and then the similarity algorithm with the maximum processing comprehensive score is used as the target similarity algorithm for subsequently calculating the similarity of the target standard question.

In some optional implementations of this embodiment, after step S206, the electronic device may further perform the following steps:

and calling a pre-trained problem generation model.

In the present embodiment, the above-described problem generation model is a model that is constructed in advance for generating similar sentences of an input sentence. The training generation process for the problem generation model will be explained in the following embodiments.

Inputting the target standard question into the question generation model, and generating a first similar question corresponding to the target standard question based on the question generation model.

And establishing an association relation between the target standard questions and the first similar questions.

In this embodiment, the association relationship between the target standard question and the first similarity question may refer to a mapping relationship between the target standard question and the first similarity question.

And storing the first similar questions into the question system based on the incidence relation.

In this embodiment, before the first similar questions are stored in the question system, the first similar questions may be subjected to knowledge constraint processing based on a preset question length limiting rule and a preset similar question number limiting rule to obtain specific similar questions, and then the specific similar questions are stored in the question system. The process of performing knowledge constraint processing on the first similar question based on the preset question length limiting rule and the similar question number limiting rule may refer to a process of generating a second similar question corresponding to the target standard question by referring to the subsequent question length limiting rule and the similar question number limiting rule.

The method comprises the steps of calling a pre-trained question generation model, inputting a target standard question into the question generation model, generating a first similar question corresponding to the target standard question based on the question generation model, establishing an association relationship between the target standard question and the first similar question, and storing the first similar question into a question system based on the association relationship. After the target standard question is added in the question answering system, the first similar question corresponding to the target standard question can be automatically, quickly and accurately generated based on the use of a pre-trained question generation model, so that the similar question data of the target standard question can be enriched by auxiliary knowledge expansion, and the robustness of the question answering system can be enhanced.

In some optional implementations, before the step of invoking the pre-trained problem generation model, the electronic device may further perform the following steps:

and acquiring preset standard question sample data.

In this embodiment, the standard question sample data may be constructed by collecting an FAQ dataset including historical standard question data, and specifically, the FAQ dataset may be analyzed and preprocessed to screen out the standard question sample data. The preprocessing can comprise data cleaning, duplication removal and the like, so that the generated standard question sample data is in a format of the adaptive model training data set.

And acquiring similar question data corresponding to the standard question sample data from a preset data platform.

In this embodiment, the data platform may be a big data platform, and a crawler means may be used to crawl similar problem data matched with standard question sample data on the big data platform, so as to obtain a large amount of similar problem data for subsequent model training.

And constructing training data based on the standard question sample data and the similar question data.

And calling a preset deep learning model.

In this embodiment, the selection of the deep learning model is not particularly limited, and a bert model or the like may be used, for example.

In the present embodiment, the training generation process of the problem generation model may refer to the generation process of the existing deep learning model. Specifically, parameters such as a learning rate and iteration times can be set, standard question sample data is used as input of a deep learning model, similar problem data is used as output of the deep learning model for model training, generation effect verification is performed on similar problem generation models in different rounds, and a model with the best generation effect is selected as the problem generation model.

According to the method, preset standard question sample data are obtained, then similar problem data corresponding to the standard question sample data are obtained from a preset data platform, and then training data are constructed on the basis of the standard question sample data and the similar problem data; subsequently calling a preset deep learning model; and finally, training the deep learning model based on the training data to generate the problem generation model. According to the method and the device, the required question generation model can be quickly trained and generated based on the training data constructed by the standard question sample data and the similar question data, after the target standard question is added in the question answering system, the first similar question corresponding to the target standard question can be automatically, quickly and accurately generated based on the trained question generation model, so that the similar question data of the target standard question can be enriched by auxiliary knowledge expansion, and the robustness of the question answering system can be enhanced.

In some optional implementations, after step S206, the electronic device may further perform the following steps:

and determining a target field corresponding to the target standard questions.

In this embodiment, the target standard question pair may be subjected to keyword extraction to extract a specified keyword related to the service field, and then the target field corresponding to the target standard question may be determined based on the specified keyword.

And acquiring a target data source corresponding to the target field.

In this embodiment, different domains may have different data sources, which may refer to network data sources.

Collecting a first statement from the target data source; wherein the number of the first sentences comprises a plurality.

In this embodiment, the target data source may be crawled with similar question data matching the target standard question by using a crawler means to obtain the first sentence.

And respectively calculating the similarity between the target standard question and each first sentence.

In this embodiment, the process of calculating the similarity between the target standard question and each of the first sentences may refer to the process of calculating the similarity between the target standard question and each of the question sentence data stored in the preset question-answering system based on the target similarity calculation method to obtain a plurality of corresponding similarity calculation results, which is not described herein in detail.

And screening out second sentences of which the similarity meets the preset condition from all the first sentences based on the similarity.

In this embodiment, the step of screening out the second sentences, of which the similarity satisfies the preset condition, from all the first sentences means that the second sentences, of which the similarity is smaller than a specified similarity threshold, are screened out from all the first sentences. The value of the specified similarity threshold is not specifically limited, and may be set according to actual service use requirements.

In this embodiment, the above-mentioned specific implementation process of generating the second similar question corresponding to the target standard question based on the second statement is further described in detail in the following specific embodiments, and is not set forth herein in any more detail.

Determining a target field corresponding to the target standard question; then acquiring a target data source corresponding to the target field; then, collecting first sentences from the target data source, and respectively calculating the similarity between the target standard questions and each first sentence; subsequently, based on the similarity, screening out second sentences of which the similarity meets a preset condition from all the first sentences; and finally, generating a second similar question corresponding to the target standard question based on the second statement. After the addition of the target standard question is completed in the question answering system, the second similar question corresponding to the target standard question can be automatically, quickly and accurately generated by screening the similar question from the target data source corresponding to the target field based on the target field corresponding to the target standard question, so that the similar question data of the target standard question can be enriched by auxiliary knowledge expansion, and the robustness of the question answering system can be enhanced.

In some optional implementation manners of this embodiment, the electronic device may further perform the following steps, in which the second similar question corresponding to the target standard question is generated based on the second sentence:

and acquiring a pre-stored question length limiting rule and a similar question number limiting rule.

In this embodiment, the question length limiting rule and the similar question number limiting rule are rules that are written and generated in advance according to actual service requirements. Wherein, the question length limiting rule is a rule for limiting the sentence length of the similar questions in a standard range; the above-mentioned similar question sentence number limitation rule is a rule for limiting the number of sentences of the similar question within a standard number threshold.

And screening out a third sentence which meets the question sentence length limiting rule from all the second sentences.

In this embodiment, the corresponding standard question length numerical range may be obtained in the question length limiting rule, and then the sentences whose sentence lengths are in the standard question length numerical range may be screened out from all the second sentences to obtain the third sentence.

And acquiring a quantity threshold corresponding to the similar question quantity limiting rule.

And judging whether the number of the third sentences is larger than the number threshold value.

In this embodiment, if the number of the third sentences is not greater than the number threshold, the third sentences may be directly used as the second similar questions corresponding to the target standard questions.

And if so, screening out fourth sentences which are the same as the quantity threshold value and have the highest similarity value from all the third sentences.

Taking the fourth statement as the second similarity question.

The method comprises the steps of obtaining a pre-stored question length limiting rule and a similar question number limiting rule; then, screening out a third sentence which meets the question sentence length limiting rule from all the second sentences; then obtaining a quantity threshold corresponding to the similar question sentence quantity limiting rule, and judging whether the quantity of the third sentence is larger than the quantity threshold; if yes, screening out fourth sentences which are the same as the quantity threshold value and have the highest similarity degree value from all the third sentences, and taking the fourth sentences as the second similarity questions. In the process of constructing the similar questions of the target standard questions, the fourth sentences which accord with the rules are screened out from the second sentences as corresponding similar questions intelligently based on the use of the question length limiting rules and the similar question quantity limiting rules, and the quality and the accuracy of the generated similar questions of the target standard questions are ensured by performing quality constraint and mathematical constraint on the similar questions.

In some optional implementation manners of this embodiment, after step S205, the electronic device may further perform the following steps:

and if the target similarity calculation results which are larger than the similarity threshold exist in all the similarity calculation results, refusing to add the target standard question in the question system.

In this embodiment, when the target similarity calculation result greater than the similarity threshold exists in all the similarity calculation results, it indicates that standard question data similar to the target standard question to be added exists in the question-answering system, and at this time, it is not necessary to construct duplicate knowledge in the question-answering system, and thus, the target standard question is rejected from being added in the question-answering system.

And generating failure reminding information corresponding to the target standard questions.

In this embodiment, a pre-stored failure reminding information template may be obtained in advance, and then a target standard question is filled into a corresponding position in the failure reminding information template to generate failure reminding information corresponding to the target standard question. The content of the failure reminding information template can be written and generated according to the actual business requirements, and for example, the content can include that a standard question matched with the "XXX" already exists in a question answering system.

And displaying the failure reminding information.

In this embodiment, the display mode of the failure reminding information is not limited, and may be displayed on the current interface, for example.

When the target similarity calculation result which is larger than the similarity threshold value exists in all the similarity calculation results, the target standard question is refused to be added in the question system, so that the problem of repeated knowledge construction in the question-answering system is avoided, and the construction quality of a knowledge base is improved. And generating failure reminding information corresponding to the target standard questions, and displaying the failure reminding information to remind a user that standard question data similar to the target standard questions to be added exist in the question answering system, so that the user can know the reason for the target standard question addition failure, and the use experience of the user is improved.

It should be emphasized that, in order to further ensure the privacy and security of the target standard questions, the target standard questions may also be stored in a node of a block chain.

The block chain referred by the application is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The block chain (B l ockcha i n), which is essentially a decentralized database, is a string of data blocks associated by using cryptography, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating the next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. The artificial intelligence (Art I f I c I a l I nte l I gene, ai) is a theory, method, technology and application system for simulating, extending and expanding human intelligence, sensing environment, acquiring knowledge and obtaining optimal results by using knowledge by using a digital computer or a machine controlled by the digital computer.

The artificial intelligence base technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a computer readable storage medium, and when executed, the processes of the embodiments of the methods described above can be included. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-only Memory (ROM), or a Random Access Memory (RAM).

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of an apparatus for processing problem data, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be applied to various electronic devices.

As shown in fig. 3, the processing apparatus 300 for problem data according to the present embodiment includes: a first judging module 301, a parsing module 302, a first determining module 303, a calculating module 304, a second judging module 305 and an adding module 306. Wherein:

a first determining module 301, configured to determine whether a standard question adding request triggered by a user is received; the standard question adding request carries a target standard question to be added;

the parsing module 302 is configured to parse the target standard question from the standard question adding request if the target standard question is found;

a first determining module 303, configured to determine a target similarity algorithm from multiple preset similarity algorithms;

a calculating module 304, configured to perform similarity calculation on the target standard question and each question data stored in a preset question-answering system based on the target similarity calculation method, so as to obtain a plurality of corresponding similarity calculation results; each question sentence data comprises a standard question and one or more similar sentences corresponding to the standard question;

a second determining module 305, configured to determine whether a target similarity calculation result greater than a preset similarity threshold exists in all the similarity calculation results;

an adding module 306, configured to add the target standard question as a new standard question to the question-answering system if the target similarity calculation result does not exist.

In this embodiment, the operations executed by the modules or units respectively correspond to the steps of the problem data processing method of the foregoing embodiment one by one, and are not described herein again.

In some optional implementations of this embodiment, the first determining module 303 includes:

the calling submodule is used for calling a preset similarity algorithm statistical data table;

the first obtaining sub-module is used for obtaining the processing efficiency, the accuracy and the release time of each similarity algorithm from the similarity algorithm statistical data table;

the first determining submodule is used for determining a first preset weight, a second preset weight and a third preset weight which respectively correspond to the processing efficiency, the accuracy and the release time;

the calculation submodule is used for calculating the processing efficiency, accuracy and release time of each similarity algorithm based on the first preset weight, the second preset weight and the third preset weight, and generating a processing comprehensive score of each similarity algorithm;

the first screening submodule is used for screening out the specified similarity algorithm with the maximum processing comprehensive score from all the similarity algorithms;

a second determination submodule for taking the specified similarity algorithm as the target similarity algorithm.

In some optional implementations of this embodiment, the apparatus for processing issue data further includes:

the first calling module is used for calling a pre-trained problem generation model;

the first generation module is used for inputting the target standard question into the question generation model and generating a first similar question corresponding to the target standard question based on the question generation model;

the establishing module is used for establishing an association relationship between the target standard question and the first similar question;

and the storage module is used for storing the first similar questions into the question system based on the incidence relation.

the first acquisition module is used for acquiring preset standard question sample data;

the second acquisition module is used for acquiring similar question data corresponding to the standard question sample data from a preset data platform;

the construction module is used for constructing training data based on the standard question sample data and the similar question data;

the second calling module is used for calling a preset deep learning model;

and the second generation module is used for training the deep learning model based on the training data to generate the problem generation model.

In some optional implementations of this embodiment, the processing apparatus of the question data further includes:

the second determining module is used for determining a target field corresponding to the target standard question;

the third acquisition module is used for acquiring a target data source corresponding to the target field;

the acquisition module is used for acquiring a first statement from the target data source; wherein the number of the first sentences comprises a plurality;

the calculation module is used for calculating the similarity between the target standard question and each first statement;

the screening module is used for screening out second sentences of which the similarity meets a preset condition from all the first sentences based on the similarity;

and the third generation module is used for generating a second similar question corresponding to the target standard question based on the second statement.

In some optional implementations of this embodiment, the third generating module includes:

the second obtaining submodule is used for obtaining a pre-stored question length limiting rule and a similar question quantity limiting rule;

the second screening submodule is used for screening out a third sentence which accords with the question sentence length limiting rule from all the second sentences;

a third obtaining submodule, configured to obtain a quantity threshold corresponding to the similar question sentence quantity limiting rule;

the judging submodule is used for judging whether the number of the third sentences is larger than the number threshold value;

a third screening submodule, configured to screen out, if yes, fourth sentences that are the same as the quantity threshold and have the highest similarity values from all the third sentences;

a third determining submodule, configured to use the fourth sentence as the second similarity question.

the processing module is used for refusing to add the target standard question in the question system if a target similarity calculation result which is larger than the similarity threshold exists in all the similarity calculation results;

the fourth generation module is used for generating failure reminding information corresponding to the target standard question;

and the display module is used for displaying the failure reminding information.

In order to solve the technical problem, an embodiment of the present application further provides a computer device. Referring to fig. 4, fig. 4 is a block diagram of a basic structure of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only the computer device 4 with the components 41-43 is shown in the figure, but it should be understood that, not all illustrated components may be required to be implemented, and more or fewer components may be alternatively implemented. AS will be understood by those skilled in the art, the computer device herein is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an application specific integrated circuit (App I cat I on Spec I C I integrated C I rcu I, AS ic), a programmable Gate array (F I l D-programmable ab l Gate Ar ray, FPGA), a digital Processor (D I ta l S I gna l Processor, DSP), an embedded device, and the like.

The computer device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The computer equipment can carry out man-machine interaction with a user in a keyboard mode, a mouse mode, a remote controller mode, a touch panel mode or a voice control equipment mode.

The memory 41 includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure digital (Secure D i g i ta l, SD) Card, a flash memory Card (F l ash Card), and the like, which are provided on the computer device 4. Of course, the memory 41 may also include both internal and external storage devices of the computer device 4. In this embodiment, the memory 41 is generally used for storing an operating system installed in the computer device 4 and various types of application software, such as computer readable instructions of a processing method of problem data. Further, the memory 41 may also be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions or process data stored in the memory 41, such as computer readable instructions to execute a method of processing the issue data.

The network interface 43 may comprise a wireless network interface or a wired network interface, and the network interface 43 is generally used for establishing communication connection between the computer device 4 and other electronic devices.

in the embodiment of the application, when a standard question adding request triggered by a user is received, the target standard question is firstly analyzed from the standard question adding request; then determining a target similarity algorithm from a plurality of preset similarity algorithms; subsequently, similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question-answering system based on the target similarity calculation method to obtain a plurality of corresponding similarity calculation results, and whether a target similarity calculation result larger than a preset similarity threshold exists in all the similarity calculation results is judged; and if the target similarity calculation result does not exist, adding the target standard question as a new standard question into the question-answering system. According to the method and the device, the similarity calculation result of the target standard question and each question data stored in the preset question-answering system can be quickly and accurately calculated based on the target similarity calculation method, further, whether the standard question data similar to the target standard question to be added exists in the question-answering system or not can be accurately judged according to the numerical comparison result between the similarity calculation result and the preset similarity threshold, and the target standard question can be added into the question-answering system as a new standard question only if the standard question data similar to the target standard question to be added does not exist in the question-answering system, so that the problem that knowledge points of the question-answering system are repeatedly constructed can be effectively avoided, and the construction quality of the question-answering system is guaranteed.

The present application provides yet another embodiment, which is to provide a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the method for processing problem data as described above.

in the embodiment of the application, when a standard question adding request triggered by a user is received, the target standard question is firstly analyzed from the standard question adding request; then determining a target similarity algorithm from a plurality of preset similarity algorithms; subsequently, similarity calculation is carried out on the target standard question and each question sentence data stored in a preset question-answering system based on the target similarity calculation method to obtain a plurality of corresponding similarity calculation results, and whether a target similarity calculation result larger than a preset similarity threshold exists in all the similarity calculation results is judged; and if the target similarity calculation result does not exist, adding the target standard question as a new standard question into the question-answering system. According to the method and the device, the similarity calculation result of the target standard question and each piece of question data stored in the preset question-answering system can be quickly and accurately calculated based on the use of a target similarity algorithm, further, whether standard question data similar to the target standard question to be added exist in the question-answering system can be accurately judged according to the numerical comparison result between the similarity calculation result and the preset similarity threshold, and the target standard question can be added into the question-answering system as a new standard question only if the standard question data similar to the target standard question to be added do not exist in the question-answering system, so that the problem that knowledge points are repeatedly constructed in the question-answering system can be effectively avoided, and the construction quality of the question-answering system is guaranteed.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.

It should be understood that the above-described embodiments are merely exemplary of some, and not all, embodiments of the present application, and that the drawings illustrate preferred embodiments of the present application without limiting the scope of the claims appended hereto. This application is capable of embodiments in many different forms and is provided for the purpose of enabling a thorough understanding of the disclosure of the application. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that the present application may be practiced without modification or with equivalents of some of the features described in the foregoing embodiments. All equivalent structures made by using the contents of the specification and the drawings of the present application are directly or indirectly applied to other related technical fields, and all the equivalent structures are within the protection scope of the present application.

Claims

1. A method for processing problem data, comprising the steps of:

2. The method for processing problem data according to claim 1, wherein the step of determining the target similarity algorithm from a plurality of preset similarity algorithms specifically comprises:

calling a preset similarity algorithm statistical data table;

3. The method for processing question data according to claim 1, characterized by further comprising, after said step of adding said target standard question as a new standard question into said question-answering system:

calling a pre-trained problem generation model;

4. The method of claim 3, wherein before the step of invoking the pre-trained problem generation model, the method further comprises:

acquiring preset standard question sample data;

calling a preset deep learning model;

5. The method for processing question data according to claim 1, characterized by further comprising, after said step of adding said target standard question as a new standard question into said question-answering system:

determining a target field corresponding to the target standard question;

acquiring a target data source corresponding to the target field;

and generating a second similar question corresponding to the target standard question based on the second sentence.

6. The method for processing question data according to claim 5, wherein the step of generating a second similar question corresponding to the target standard question based on the second sentence specifically includes:

screening out a third sentence which accords with the question length limiting rule from all the second sentences;

if yes, screening out fourth sentences which are the same as the quantity threshold value and have the highest similarity value from all the third sentences;

taking the fourth sentence as the second similarity question.

7. The method for processing problem data according to claim 1, further comprising, after the step of determining whether there is a target similarity calculation result that is greater than a preset similarity threshold among all the similarity calculation results:

if the target similarity calculation result which is larger than the similarity threshold exists in all the similarity calculation results, refusing to add the target standard question in the question system;

and displaying the failure reminding information.

8. An apparatus for processing issue data, comprising:

the calculation module is used for carrying out similarity calculation on the target standard question and each piece of question sentence data stored in a preset question answering system based on the target similarity algorithm to obtain a plurality of corresponding similarity calculation results; each question sentence data comprises a standard question and one or more similar sentences corresponding to the standard question;

9. A computer device comprising a memory having computer readable instructions stored therein and a processor which when executed implements the steps of a method of processing issue data according to any one of claims 1 to 7.

10. A computer-readable storage medium, having computer-readable instructions stored thereon, which, when executed by a processor, implement the steps of the method of processing issue data according to any one of claims 1 to 7.