CN112035325B

CN112035325B - Text robot automatic monitoring method and device

Info

Publication number: CN112035325B
Application number: CN202010914806.7A
Authority: CN
Inventors: 梁雨霏; 李永亮; 李凤亭; 杨玉; 刘晓刚
Original assignee: Bank of China Ltd
Current assignee: Bank of China Ltd
Priority date: 2020-09-01
Filing date: 2020-09-01
Publication date: 2023-08-18
Anticipated expiration: 2040-09-01
Also published as: CN112035325A

Abstract

The embodiment of the application provides a text robot automatic monitoring method and a text robot automatic monitoring device, wherein the method comprises the following steps: automatically acquiring business index data of the text robot during online operation of the text robot for processing the problem response data; judging whether the business index data of the text robot meets the corresponding business standard, if not, carrying out model training on the text robot based on the original corpus data and the newly added corpus data to obtain an updated text robot; and replacing the text robots with the updated text robots, wherein the service index data of the text robots do not meet the service standard, so as to process the problem response data based on the updated text robots. The application can automatically realize the service state monitoring and system upgrading of the text robot, further can effectively improve the updating efficiency and the automation degree of the text robot, and can effectively ensure the accuracy and the reliability of the response service of the text robot to the user.

Description

Text robot automatic monitoring method and device

Technical Field

The application relates to the technical field of data processing, in particular to an automatic text robot monitoring method and device.

Background

In order to provide a more convenient and better question-answering service to users, enterprises or institutions generally adopt a machine learning model capable of automatically generating answer text data according to question text data presented by users, and such a machine learning model may be called a text robot from the viewpoint of providing a service function. Text robots used in the professional field often have higher requirements on response effects, and a large number of business personnel actively monitor system conditions in the past to manually inspect the response conditions of the text robots. With the online of the system, newly-increased business scenes in the field are increasingly complex, professional knowledge is gradually increased, and the problem of clients needing the response of the robot is also continuously increased, so that the response effect of the text robot to the clients is a great test.

Currently, in order to maintain the service capability of a text robot to a user, service personnel are required to manually detect the running condition of the system, collect user experience, give task demands and provide latest user corpus data, and after a developer manually imports data to complete model training and tuning, a tester designs a test scheme, writes a test case to cover a service scene and evaluates the model modification effect. And if the transformed response effect meets the service requirement, the maintainer deploys the version and the parameters provided by development. However, the system upgrading and reforming mode is too long in period, cannot quickly respond to market demands, and is unfavorable for improving user viscosity, that is, the existing text robot monitoring mode cannot simultaneously ensure the application accuracy and updating efficiency of the text robot.

Disclosure of Invention

Aiming at the problems in the prior art, the application provides the automatic text robot monitoring method and the automatic text robot monitoring device, which can automatically monitor the service state of the text robot and upgrade the system, further effectively improve the updating efficiency and the automation degree of the text robot, and effectively ensure the accuracy and the reliability of the response service of the text robot to the user.

In order to solve the technical problems, the application provides the following technical scheme:

in a first aspect, the present application provides a text robot automatic monitoring method, including:

automatically acquiring business index data of the text robot during online operation of the text robot for processing the problem response data;

judging whether the business index data of the text robot in the online running state meets the corresponding business standard, if not, carrying out model training on the text robot based on pre-stored original corpus data and newly-added corpus data generated by the text robot in the running period to obtain an updated text robot;

and replacing the text robots with the updated text robots, wherein the service index data of the text robots do not meet the service standard, so as to process the problem response data based on the updated text robots.

Further, the method further comprises the following steps:

acquiring the problem response data generated by the text robot during the operation in real time during the online operation of the text robot for processing the problem response data;

performing data deduplication processing on the data corresponding to the existing business scene in the problem response data to update the original corpus data corresponding to the existing business scene;

performing newly added service scene recognition on the data which do not correspond to the existing service scene in the problem response data;

and updating and expanding corpus data corresponding to the newly added business scene based on the existing business scene by applying a natural language processing technology, and associating scene characteristics of the newly added business scene with a preset knowledge graph so as to generate the newly added corpus data corresponding to the newly added business scene.

Further, the training of the text robot to obtain an updated text robot based on the pre-stored original corpus data and the newly-added corpus data generated by the text robot during the operation process includes:

outputting early warning information of the text robot aiming at the business index data which does not meet the business standard;

If a text robot upgrading instruction generated according to the early warning information is received, pre-stored original corpus data and newly-added corpus data generated during the operation of the text robot are obtained;

and training a model of the text robot of which the business index data does not meet the business standard according to the original corpus data and the newly added corpus data to obtain an updated text robot.

Further, the training of the model for the text robot whose business index data does not meet the business standard according to the original corpus data and the newly added corpus data to obtain an updated text robot includes:

generating a training set and a testing set based on the original corpus data and the newly added corpus data;

performing model training on the text robots with the business index data not meeting the business standards by using the training set, and performing model test on the text robots obtained by training by using the test set;

and determining the text robot passing the model test as an updated text robot.

Further, after the text robot with updated text robot replacement business index data not meeting the business standard is applied to process the problem response data based on the updated text robot, the method further comprises:

And during the online running of the updated text robot, acquiring service index data of the updated text robot in a preset period, judging whether the service index data of the updated text robot in the preset period meets corresponding quality post-evaluation standards, and if not, carrying out version rollback processing on the updated text robot.

Further, the performing data deduplication processing on the data corresponding to the existing business scenario in the problem response data to update the original corpus data corresponding to the existing business scenario includes:

according to the type of the existing service scene, data corresponding to the existing service scene in the problem response data are classified;

and carrying out data deduplication processing on response data in a plurality of corpus data corresponding to each existing business scene.

Further, the identifying the newly added service scene for the data which does not correspond to the existing service scene in the problem response data includes:

clustering data which do not correspond to the existing service scene in the problem response data to obtain each group of clustered data;

and respectively extracting data elements in each group of clustered data to respectively form corresponding newly added business scenes based on each data element.

Further, the applying a natural language processing technology, updating and expanding corpus data corresponding to the new business scene based on the existing business scene, and associating scene features of the new business scene with a preset knowledge graph to generate the new corpus data corresponding to the new business scene, includes:

performing word segmentation on corpus data corresponding to the existing business scene by applying a preset natural language processing mode, and determining language rules of the existing business scene according to a corresponding word segmentation result;

according to the language rule of the existing service scene, updating and expanding corpus data of a newly added service scene related to the existing service scene;

and associating scene characteristics of the newly added business scene with a preset knowledge graph to generate the newly added corpus data corresponding to the newly added business scene.

In a second aspect, the present application provides an automatic text robot monitoring apparatus, comprising:

the automatic monitoring module is used for automatically acquiring service index data of the text robot during the online operation of the text robot for processing the problem response data;

the risk assessment and model updating module is used for judging whether the business index data of the text robot in the online running state meets the corresponding business standard, if not, model training is carried out on the text robot based on the pre-stored original corpus data and the newly-added corpus data generated by the text robot in the running period, and the updated text robot is obtained;

The model deployment module is used for replacing the text robots of which the business index data does not meet the business standards by using the updated text robots so as to process the problem response data based on the updated text robots.

Further, the method further comprises the following steps:

the data acquisition module is used for acquiring the problem response data generated by the text robot in real time during the running period during the online running period of the text robot for processing the problem response data;

the original corpus updating module is used for carrying out data deduplication processing on the data corresponding to the existing business scene in the problem response data so as to update the original corpus data corresponding to the existing business scene;

the new scene identification module is used for carrying out new business scene identification on the data which do not correspond to the existing business scene in the problem response data;

the new corpus generation module is used for applying a natural language processing technology, updating and expanding corpus data corresponding to the new business scene based on the existing business scene, and associating scene characteristics of the new business scene with a preset knowledge graph so as to generate the new corpus data corresponding to the new business scene.

Further, the risk assessment and model update module includes:

the early warning information output unit is used for outputting early warning information of the text robot aiming at the business index data which does not meet the business standard;

the corpus data acquisition unit is used for acquiring prestored original corpus data and newly-increased corpus data generated during the operation of the text robot if a text robot upgrading instruction generated according to the early warning information is received;

and the model training unit is used for carrying out model training on the text robots of which the business index data does not meet the business standard according to the original corpus data and the newly added corpus data to obtain updated text robots.

Further, the model training unit is configured to perform the following:

and determining the text robot passing the model test as an updated text robot.

Further, the method further comprises the following steps:

the quality post-evaluation module is used for acquiring business index data of the updated text robot in a preset period during online operation of the updated text robot, judging whether the business index data of the updated text robot in the preset period meets corresponding quality post-evaluation standards, and if not, carrying out version rollback processing on the updated text robot.

Further, the original corpus updating module is configured to execute the following contents:

Further, the added scene recognition module is configured to execute the following contents:

Further, the new corpus generation module is configured to execute the following contents:

In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the text robot automatic monitoring method when executing the program.

In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the text robot automatic monitoring method.

According to the technical scheme, the automatic text robot monitoring method and device provided by the application comprise the following steps: automatically acquiring business index data of the text robot during online operation of the text robot for processing the problem response data; judging whether the business index data of the text robot in the online running state meets the corresponding business standard, if not, carrying out model training on the text robot based on pre-stored original corpus data and newly-added corpus data generated by the text robot in the running period to obtain an updated text robot; the updated text robot is applied to replace the text robot with the business index data which does not meet the business standard, so that the problem response data processing is performed based on the updated text robot, the service and running state monitoring of the text robot can be automatically realized, whether the text robot needs to be updated or not can be automatically determined, the text robot can be automatically subjected to model training update and system update, the intelligent degree of automatic detection and update control is high, the update efficiency and the automation degree of the text robot can be effectively improved, the labor cost and the time cost are effectively saved, the accuracy and the reliability of the response service of the text robot to a user can be effectively ensured, the user experience is effectively improved, the market demand is rapidly responded, and the viscosity of the user is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a first method for automatically monitoring a text robot according to an embodiment of the present application.

Fig. 2 is a schematic diagram of a specific flow of steps 010 to 040 in the text robot automatic monitoring method according to an embodiment of the present application.

Fig. 3 is a schematic flowchart of step 200 in the text robot automatic monitoring method according to the embodiment of the present application.

Fig. 4 is a schematic flowchart of step 240 in the text robot automatic monitoring method according to the embodiment of the present application.

Fig. 5 is a schematic diagram of a second flow of the text robot automatic monitoring method in the embodiment of the application.

Fig. 6 is a schematic diagram of a specific flow of step 020 in the text robot automatic monitoring method according to an embodiment of the present application.

Fig. 7 is a schematic diagram of a specific flow of step 030 in the text robot automatic monitoring method according to an embodiment of the present application.

Fig. 8 is a schematic diagram of a specific flow of step 040 in the text robot automatic monitoring method according to an embodiment of the present application.

Fig. 9 is a schematic structural view of an automatic text robot monitoring device in an embodiment of the present application.

Fig. 10 is a logic flow diagram of an application text robot automatic monitoring system provided by the application example of the present application to execute a text robot automatic monitoring method.

Fig. 11 is a schematic structural diagram of an electronic device in an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In consideration of the problems that the existing text robot system upgrading and reforming mode is overlong in period and cannot quickly respond to market demands and is not beneficial to improving user viscosity, that is, the existing text robot monitoring mode cannot simultaneously ensure application accuracy and updating efficiency of the text robot, the application provides an automatic text robot monitoring method, an automatic text robot monitoring device, electronic equipment and a computer readable storage medium.

Based on the foregoing, the present application further provides a text robot automatic monitoring system for implementing the text robot automatic monitoring method provided in one or more embodiments of the present application, where in an example of the text robot automatic monitoring system, the text robot automatic monitoring system may be respectively in communication connection with a text robot online operation system and a client device of a person in a financial enterprise, and the text robot automatic monitoring system may apply its own application server to access an application server and the client device of the text robot online operation system, respectively.

The text robot automatic monitoring system automatically acquires service index data of the text robot from the text robot online operation system; judging whether the business index data of the text robot in the online running state meets the corresponding business standard, if not, carrying out model training on the text robot based on the prestored original corpus data and the newly-added corpus data generated by the text robot in the running period to obtain an updated text robot; and then the updated text robot is used for replacing the text robot with the business index data not meeting the business standard, so that the problem response data processing is performed based on the updated text robot, and the processing content, the result and the like can be sent to the client equipment of the financial enterprise internal personnel.

It is understood that the client device may include a smart phone, a tablet electronic device, a network set top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), an in-vehicle device, a smart wearable device, etc. Wherein, intelligent wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..

In a practical application scenario, the foregoing text robot automatic monitoring system text robot automatic monitoring portion may be executed by the text robot automatic monitoring system as described in the foregoing, or all operations may be completed in the client device. Specifically, the selection may be made according to the processing capability of the client device, and restrictions of the use scenario of the user. The application is not limited in this regard. If all operations are completed in the client device, the client device may further include a processor for performing specific processing of text robot automatic monitoring.

The client device may have a communication module (i.e. a communication unit) and may be connected to a remote server in a communication manner, so as to implement data transmission with the server. For example, the communication unit may transmit, to the server, service index data automatically acquired by the text robot for performing the processing of the problem response data during the online operation of the text robot. The communication unit may also receive updated text robots returned by the server. The server may include a server on the side of the task scheduling center, and in other implementations may include a server of an intermediate platform, such as a server of a third party server platform having a communication link with the task scheduling center server. The server may include a single computer device, a server cluster formed by a plurality of servers, or a server structure of a distributed device.

Any suitable network protocol may be used between the server and the client device, including those not yet developed on the filing date of the present application. The network protocols may include, for example, TCP/IP protocol, UDP/IP protocol, HTTP protocol, HTTPS protocol, etc. Of course, the network protocol may also include, for example, RPC protocol (Remote Procedure Call Protocol ), REST protocol (Representational State Transfer, representational state transfer protocol), etc. used above the above-described protocol.

The text robot automatic monitoring method, the text robot automatic monitoring device, the electronic equipment and the computer readable storage medium automatically acquire business index data of the text robot during online operation of the text robot for processing the problem response data; judging whether the business index data of the text robot in the online running state meets the corresponding business standard, if not, carrying out model training on the text robot based on pre-stored original corpus data and newly-added corpus data generated by the text robot in the running period to obtain an updated text robot; the updated text robot is applied to replace the text robot with the business index data which does not meet the business standard, so that the problem response data processing is performed based on the updated text robot, the service and running state monitoring of the text robot can be automatically realized, whether the text robot needs to be updated or not can be automatically determined, the text robot can be automatically subjected to model training update and system update, the intelligent degree of automatic detection and update control is high, the update efficiency and the automation degree of the text robot can be effectively improved, the labor cost and the time cost are effectively saved, the accuracy and the reliability of the response service of the text robot to a user can be effectively ensured, the user experience is effectively improved, the market demand is rapidly responded, and the viscosity of the user is improved.

In one or more embodiments provided by the present application, the text robot for performing the processing of the response data to the question may specifically refer to a learning model of a learning machine for performing the processing of the response data to the question, such as a sequence-to-sequence model Seq2Seq model, etc., where the Seq2Seq model is composed of two parts of an encoder and a decoder, and the encoder and the decoder may be composed of any one of three structures of a convolutional neural network CNN (Convolutional Neural Networks), a recurrent neural network RNN (Recurrent Neural Networks), and a transducer. For example, both the encoder and the decoder employ RNN series models, and a gate loop unit GRU, a Long Short-Term Memory (LSTM), and the like may be employed.

In one or more embodiments of the present application, a process of online operation of a text robot for processing response data of a problem refers to a process in which an online operation system of the text robot receives the problem data sent by a user online, inputs the problem data into a preset text robot, uses output of the text robot as response data corresponding to the problem data, and sends the response data to the user online.

The following embodiments and application examples are described in detail.

In order to automatically execute the upgrading and reconstruction process for the text robot, the application provides an embodiment of an automatic text robot monitoring method, referring to fig. 1, wherein the automatic text robot monitoring method specifically comprises the following steps:

step 100: during the online operation of the text robot for processing the problem response data, the business index data of the text robot is automatically acquired.

It can be appreciated that the business index data includes customer satisfaction and response accuracy; the service standard corresponding to the service index data comprises: a satisfaction threshold corresponding to the customer satisfaction and an accuracy threshold corresponding to the response accuracy.

In step 100, by monitoring the current situation of the system in real time, the system can early warn in time when the text robot cannot answer the service problem or the customer satisfaction is continuously reduced to a certain extent, inform service personnel to discover the service quality risk as soon as possible, and avoid the situations of too lag of manual detection and continuous reduction of the customer satisfaction.

Step 200: judging whether the business index data of the text robot in the online running state meets the corresponding business standard, if not, carrying out model training on the text robot based on the prestored original corpus data and the newly-added corpus data generated by the text robot in the running period to obtain the updated text robot.

In step 200, new corpus data and corresponding scenes generated from the last training in the data warehouse can be intercepted, the key problems focused by the current clients or the service fields which cannot be responded by the text robot can be positioned according to the new scenes, and the system upgrading direction is defined. And acquiring a training set and a testing set of the new and original service scenes, starting the training of a system core model, comparing and selecting an optimal model, and storing model codes, parameters and data.

Step 300: and replacing the text robots with the updated text robots, wherein the service index data of the text robots do not meet the service standard, so as to process the problem response data based on the updated text robots.

In step 300, the code, parameters, data, etc. of the optimal model may be synthesized to form a production version to complete deployment.

From the above description, it can be seen that the automatic text robot monitoring method provided by the embodiment of the application can automatically monitor the service and the running state of the text robot, automatically determine whether the text robot needs to be updated, automatically update the model training and upgrade the system of the text robot, and automatically detect and update the text robot with high intelligent degree, thereby effectively improving the update efficiency and the automation degree of the text robot, effectively saving the labor cost and the time cost, and effectively ensuring the accuracy and the reliability of the response service of the text robot to the user.

In order to automatically generate training data of the updated text robot, in one embodiment of the text robot automatic monitoring method provided by the application, referring to fig. 2, the text robot automatic monitoring method further specifically includes the following contents:

step 010: during online operation of a text robot for performing problem response data processing, problem response data generated during operation of the text robot is acquired in real time.

Step 020: and carrying out data deduplication processing on the data corresponding to the existing business scene in the problem response data so as to update the original corpus data corresponding to the existing business scene.

Step 030: and performing newly added service scene recognition on the data which do not correspond to the existing service scene in the problem response data.

Step 040: and updating and expanding corpus data corresponding to the newly added business scene based on the existing business scene by applying a natural language processing technology, and associating scene characteristics of the newly added business scene with a preset knowledge graph so as to generate the newly added corpus data corresponding to the newly added business scene.

Specifically, after the text robot is online, the online actual customer data can be obtained in real time and preprocessed, corpus pair data are formed through operations such as duplication removal and slicing and stored in a data warehouse, each corpus pair only corresponds to a unique business scene, and a plurality of corpus data can be associated with the same scene.

As can be seen from the above description, the text robot automatic monitoring method provided by the embodiment of the application uses the natural language processing technology and the clustering algorithm to sort the customer data and identify the newly added service scene, combines the knowledge graph technology to correlate the structured data, realizes scene coverage, can effectively shorten the time required for manually filtering the customer problem and identifying the service scene, and can avoid the situation that the development and testing personnel are not familiar with the service due to the influence of personal experience skills to miss the scene. In addition, in the past, the developer needs to manually import training data when upgrading the system, and a great deal of time is spent waiting for model training and test implementation.

In order to improve the reliability of updating the text robot model, in one embodiment of the text robot automatic monitoring method provided by the application, referring to fig. 3, step 200 in the text robot automatic monitoring method specifically includes the following steps:

step 210: judging whether the service index data of the text robot in the online running state meets the corresponding service standard, if not, executing step 220; if yes, go back to execute step 100.

Step 220: and outputting the early warning information of the text robot aiming at the business index data which does not meet the business standard.

Step 230: and if a text robot upgrading instruction generated according to the early warning information is received, obtaining pre-stored original corpus data and newly-added corpus data generated during the operation of the text robot.

Step 240: and training a model of the text robot of which the business index data does not meet the business standard according to the original corpus data and the newly added corpus data to obtain an updated text robot.

Specifically, the early warning information can be sent to business personnel according to the threshold value setting, and whether the system needs to be upgraded or not is evaluated. The service personnel can manually judge the system condition, and if the judgment is carried out temporarily without upgrading the system, the upgrading program can be ended to terminate the early warning. If the system operation condition is not good, the system continues to operate the upgrade program.

In order to automatically update the model of the text robot, in one embodiment of the text robot automatic monitoring method provided by the present application, referring to fig. 4, step 240 in the text robot automatic monitoring method specifically includes the following:

step 241: generating a training set and a testing set based on the original corpus data and the newly added corpus data.

Step 242: and performing model training on the text robots with the business index data not meeting the business standards by using the training set, and performing model test on the text robots obtained by training by using the test set.

Step 243: and determining the text robot passing the model test as an updated text robot.

Specifically, new corpus data and corresponding scenes generated from the last training in the data warehouse can be intercepted, the key problems focused by the current clients or the service field which cannot be responded by the text robot can be positioned according to the new scenes, and the system upgrading direction is defined. And acquiring a training set and a testing set of the new and original service scenes, starting the training of a system core model, comparing and selecting an optimal model, and storing model codes, parameters and data.

In order to improve the operation reliability of the updated text robot, in one embodiment of the text robot automatic monitoring method provided by the present application, referring to fig. 5, step 300 in the text robot automatic monitoring method further specifically includes the following:

step 400: and during the online running of the updated text robot, acquiring service index data of the updated text robot in a preset period, judging whether the service index data of the updated text robot in the preset period meets corresponding quality post-evaluation standards, and if not, carrying out version rollback processing on the updated text robot.

Specifically, a final version package can be obtained to complete automatic deployment. And in a certain time window, the risk monitoring and evaluating module continuously collects service index related information such as customer satisfaction, response accuracy and the like, if the quality post-evaluation meets the service requirement, the upgrading is finished, and if the quality post-evaluation does not meet the evaluation requirement, version rollback can be started.

In order to provide a preferred way of data deduplication, in one embodiment of the text robot automatic monitoring method provided by the present application, referring to fig. 6, step 020 in the text robot automatic monitoring method specifically includes the following:

step 021: and according to the types of the existing service scenes, classifying the data corresponding to the existing service scenes in the problem response data.

Step 022: and carrying out data deduplication processing on response data in a plurality of corpus data corresponding to each existing business scene.

Specifically, there are now a plurality of pieces of unprocessed customer corpus data { [ Q ] _i ,A _i ],[Q _j ,A _j ]...,[Q _n ,A _n ]And (Q is a customer question and a is an answer). By classification processing, find Q ₁ 、Q ₂ 、Q ₃ If the client questions are the same class of client questions, the client questions are arranged in the same group, and the client answers A ₁ 、A ₂ 、A ₃ And (3) merging, namely if the answer A source is the latest manual answer content, replacing the original stored answer content with the latest answer. Finally, all the classified corpora are arranged into { [ Q ] ₁₁ ,Q ₁₂ ,Q ₁₃ ,...Q _1n ]],A ₁ }(Q _1i For the same kind of customer problems, A ₁ Corresponding answer) and with a unique business scenario S ₁ And (5) association. And if the content of the current answer A is NULL, displaying the content to service personnel at a front-end interface, and manually supplementing the content of the answer. The data module automatically processes corpus which cannot acquire answers and is not manually supplemented by service personnel, and the corpus is not acquired by the model training and testing module.

In order to provide a preferred manner of identifying a new service scenario, in one embodiment of the text robot automatic monitoring method provided by the present application, referring to fig. 7, step 030 in the text robot automatic monitoring method specifically includes the following:

step 031: and clustering the data which do not correspond to the existing service scene in the problem response data to obtain each group of clustered data.

Step 032: and respectively extracting data elements in each group of clustered data to respectively form corresponding newly added business scenes based on each data element.

Specifically, if there is unclassified corpus data { [ M ] _i ,N _i ],[M _j ,N _j ]...,[M _o ,N _o ]Clustering unclassified data, extracting key elements to form new service scene S _i 、S _j 、S _o . And associating the key dimension characteristic information of the service scene with the knowledge graph. All the newly added corpus data The business scenario and KNOWLEDGE graph information are respectively stored in a CORPUS_ SHEET, SCENARIO _ SHEET, MAPPING _KNOWLEDCE_DOMAIN_SHEET incremental data table.

In order to provide a preferred manner of generating new corpus data corresponding to a new business scenario, in one embodiment of the text robot automatic monitoring method provided by the present application, referring to fig. 8, step 040 in the text robot automatic monitoring method specifically includes the following contents:

step 041: and performing word segmentation on the corpus data corresponding to the existing business scene by applying a preset natural language processing mode, and determining the language rule of the existing business scene according to the corresponding word segmentation result.

Step 042: and updating and expanding corpus data of the newly added business scene related to the existing business scene according to the language rule of the existing business scene.

Step 043: and associating scene characteristics of the newly added business scene with a preset knowledge graph to generate the newly added corpus data corresponding to the newly added business scene.

Specifically, the new added service scene and the original service scene can be compared, and language rules corresponding to statistics in the word segmentation process are utilized by utilizing a natural language processing technology, so that the corpus is constructed in a reverse recombination mode. For example: in the banking field, for the existing business scenario "ATM debit card balance query", the corpus of customer questions that the corresponding text robotic system can answer includes: how to query debit card accounts through ATM; how to query the debit card balance through the ATM; ATM can check the balance of the debit card; inquiring balance information of the ATM debit card; i want to ask how to query the debit card foreign exchange balance "through the ATM. If a new business scene of 'ATM no-card withdrawal number inquiry' appears, and if the corresponding corpus data only has 'how to know the no-card withdrawal number', then constructing a corpus supplementary data warehouse such as 'how to inquire the no-card withdrawal number by the ATM', 'how to inquire the no-card withdrawal number by the ATM' according to the language rules counted by the existing corpus, and the like, for training and optimizing a robot model.

In order to solve the problem that the existing text robot monitoring method cannot simultaneously ensure the application accuracy and the update efficiency of the text robot, the application provides an embodiment of a text robot automatic monitoring device for executing all or part of the content in the text robot automatic monitoring method, referring to fig. 9, the text robot automatic monitoring device specifically comprises the following contents:

the automatic monitoring module 10 is configured to automatically acquire business index data of a text robot for performing problem response data processing during online operation of the text robot.

The automatic monitoring module 10 monitors the current situation of the system in real time, and can early warn timely when the text robot cannot answer the service problem or the customer satisfaction degree is continuously reduced to a certain degree, so as to inform service personnel to discover the service quality risk as soon as possible, and avoid the situations of too lag of manual detection and continuous reduction of the customer satisfaction degree.

And the risk assessment and model update module 20 is configured to determine whether the business index data of the text robot in the online running state meets the corresponding business standard, and if not, perform model training on the text robot based on the pre-stored original corpus data and the newly-increased corpus data generated during the running process of the text robot, so as to obtain an updated text robot.

The risk assessment and model updating module 20 can intercept newly added corpus data and corresponding scenes generated from the last training in the data warehouse, locate key problems focused by the current clients or service fields which cannot be responded by the text robot according to the newly added scenes, and determine the system upgrading direction. And acquiring a training set and a testing set of the new and original service scenes, starting the training of a system core model, comparing and selecting an optimal model, and storing model codes, parameters and data.

The model deployment module 30 is configured to apply the updated text robot to replace the text robot whose business index data does not meet the business standard, so as to perform the problem response data processing based on the updated text robot.

The model deployment module 30 can synthesize the code, parameters, data and other contents of the optimal model to form a production version to complete deployment.

The embodiment of the text robot automatic monitoring device provided by the application can be particularly used for executing the processing flow of the embodiment of the text robot automatic monitoring method in the embodiment, and the functions of the embodiment of the text robot automatic monitoring device are not repeated herein, and can be referred to in the detailed description of the embodiment of the method.

From the above description, it can be seen that the automatic text robot monitoring device provided by the embodiment of the application can automatically monitor the service and the running state of the text robot, automatically determine whether the text robot needs to be updated, automatically update the model training and upgrade the system of the text robot, and automatically detect and update the text robot with high intelligent degree, thereby effectively improving the update efficiency and the automation degree of the text robot, effectively saving the labor cost and the time cost, and effectively ensuring the accuracy and the reliability of the response service of the text robot to the user.

In order to automatically generate training data of the updated text robot, in one embodiment of the text robot automatic monitoring device provided by the application, the text robot automatic monitoring device further specifically comprises the following contents:

and the data acquisition module is used for acquiring the problem response data generated by the text robot during the operation in real time during the online operation of the text robot for processing the problem response data.

And the original corpus updating module is used for carrying out data deduplication processing on the data corresponding to the existing business scene in the problem response data so as to update the original corpus data corresponding to the existing business scene.

And the new scene identification module is used for carrying out new business scene identification on the data which do not correspond to the existing business scene in the problem response data.

As can be seen from the above description, the text robot automatic monitoring device provided by the embodiment of the application uses the natural language processing technology and the clustering algorithm to sort the customer data and identify the newly added service scene, combines the knowledge graph technology to correlate the structured data, realizes scene coverage, can effectively shorten the time required for manually filtering the customer problem and identifying the service scene, and can avoid the situation that the development and testing personnel are not familiar with the service due to the influence of personal experience skills to miss the scene. In addition, in the past, the developer needs to manually import training data when upgrading the system, and a great deal of time is spent waiting for model training and test implementation.

In order to improve the reliability of the upgrade and update of the text robot model, in one embodiment of the text robot automatic monitoring device provided by the present application, the risk assessment and model update module 20 in the text robot automatic monitoring device specifically includes the following:

and the early warning information output unit is used for outputting early warning information of the text robot aiming at the business index data which does not meet the business standard.

And the corpus data acquisition unit is used for acquiring prestored original corpus data and newly-increased corpus data generated during the operation of the text robot if a text robot upgrading instruction generated according to the early warning information is received.

In order to automatically update the model of the text robot, in one embodiment of the text robot automatic monitoring device provided by the application, a model training unit in the text robot automatic monitoring device is used for executing the following contents:

In order to improve the operation reliability of the updated text robot, in one embodiment of the text robot automatic monitoring device provided by the application, the text robot automatic monitoring device further specifically comprises the following contents:

In order to provide a preferred mode of data deduplication, in one embodiment of the text robot automatic monitoring device provided by the application, an original corpus update module in the text robot automatic monitoring device is used for executing the following contents:

In order to provide a preferred mode of identifying a new service scene, in one embodiment of the text robot automatic monitoring device provided by the application, a new scene identification module in the text robot automatic monitoring device is used for executing the following contents:

In order to provide a preferred manner of generating new corpus data corresponding to a new business scenario, in one embodiment of the text robot automatic monitoring device provided by the present application, the new corpus generating module 40 in the text robot automatic monitoring device is configured to execute the following:

In order to further explain the scheme of the text, the application also provides a specific application example of the text robot automatic monitoring method, and aiming at the construction, test and deployment of the text robot at present, the system upgrading and reconstruction requirements are put forward mainly depending on the condition of a business monitoring system. After receiving the service requirement, development, test and operation and maintenance personnel respectively manually push the work, and the problem of longer system upgrading flow is solved, namely, the construction, test and deployment method of the text robot adopts the traditional software development operation and maintenance mode; the system upgrade requires longer development, test and application deployment preparation time, and cannot effectively cope with the current situations that the service volume in the professional field is continuously increased and the demands of clients are increasingly diversified. The application example is based on the DevOps development, test and operation integrated thought, and utilizes the technologies of natural language processing, knowledge graph and the like to realize automation of data processing, model training, test evaluation, version synthesis and deployment, so that the labor cost can be reduced, the system transformation period can be shortened, the smooth upgrading of the system can be realized, and better customer experience can be provided.

Based on the implementation scheme provided by the application example of the application, the current situation of the system is monitored in real time through the risk monitoring and evaluating module, and when the text robot cannot answer the service problem or the customer satisfaction is continuously reduced to a certain extent, early warning is timely carried out, service personnel are informed of finding the service quality risk as soon as possible, and the situations that manual detection is too lagged and the customer satisfaction is continuously reduced are avoided. The customer data are arranged and new business scenes are identified by using a natural language processing technology and a clustering algorithm, and structured data are associated by combining a knowledge graph technology, so that scene coverage is realized, the time required for manually filtering customer problems and identifying business scenes can be effectively shortened, and the situation that development and testing personnel are not familiar with the business due to the influence of personal experience skills can be avoided. In addition, in the past, the developer needs to manually import training data when upgrading the system, and a great deal of time is spent waiting for model training and test implementation. By applying the DevOps development, test, operation and maintenance integrated ideas and related technical flows, business scenes are automatically mined from client data, automation of data preparation, model training, test implementation, version synthesis and deployment is realized, the service quality of a system can be pertinently improved, human errors possibly occurring in the process of synthesizing versions, parameters and data are reduced, and the upgrading and transformation period of the system is shortened.

From the development perspective, the technician needs to send out early warning and trigger the upgrading program by the risk monitoring and evaluating module when the customer satisfaction degree, response accuracy rate and the like of the text robot reach a certain threshold value for customer service indexes. And cleaning and processing mass client data collected by different service channels to form a data warehouse, identifying a newly added service scene by using a natural language processing technology, updating and expanding corpus data by using the existing service scene, associating service scene characteristics with a knowledge graph, and providing text robot model training basic data. After model training and comparison, the system selects optimal model parameters, and automatic synthesis and deployment of the versions are completed when the model parameters meet the requirements through test evaluation. After online, continuously collecting customer feedback for quality post-evaluation, and realizing version rollback when customer experience cannot be effectively improved.

The method is most important for the application example of the application, and a set of upgrading program which can automatically monitor the system performance, reduce the manual processing and intervention links in the system reconstruction process and realize the system construction and deployment automation is established for the existing text robot system.

Referring to fig. 10, the text robot automatic monitoring system for performing the text robot automatic monitoring method mainly includes the following modules:

(1) Risk monitoring and evaluation module: and monitoring business indexes such as customer satisfaction and response accuracy, and when the text robot has more business problems which cannot be responded and the customer satisfaction continuously drops, sending early warning information to business personnel by the risk monitoring and evaluating module according to the threshold setting, and evaluating whether the system needs to be upgraded or not. The service personnel can manually judge the system condition, and if the judgment is carried out temporarily without upgrading the system, the upgrading program can be ended to terminate the early warning. If the system operation condition is not good, the system continues to operate the upgrade program.

(2) Data module (including data warehouse, knowledge graph): corpus data (including robot response data and artificial customer service response data generated from various business channels) generated by a text robot during customer service in the professional field is collected, cleaned and removed to form basic corpus data, and the basic corpus data is stored in a data warehouse. Corpus data exists in pairs, including customer questions and answers. The effective corpus data has corresponding service scenes, a plurality of pieces of corpus data can be associated with the same service scene, and different client questions corresponding to the same service scene only correspond to unique answers.

The data module applies a classification algorithm to process the newly added corpus data and associates the newly added corpus data with the existing business scene. The corpus data which is not classified is further divided by adopting a clustering algorithm, and key elements are extracted to form a newly added business scene. And for the newly added business scene, the corpus data is reversely formed by utilizing a natural language processing technology, and the corpus data and the actual online corpus generated in the customer service process form a corpus set corresponding to the business scene together.

The data module provides a front-end interface and displays the corpus, the corresponding business scene and the associated knowledge graph label. If only the client questions exist in the corpus data corresponding to the business scene, the response information to the client is NULL, and the business personnel manually supplement answer content to the data warehouse to perfect the corpus data. Marking the business scene according to dimension characteristics of the classification, and storing the business scene and the map label in the knowledge map as structural data if no dimension corresponding to the corresponding feature map label can be marked.

Specifically, after the text robot is online, the data module acquires online actual customer data in real time and performs preprocessing, and corpus pair data are formed through operations such as duplication removal and slicing and stored in a data warehouse, each corpus pair only corresponds to a unique business scene, and a plurality of corpus data can be associated with the same scene.

For example, there are now a plurality of unprocessed customer corpus data { [ Q ] _i ,A _i ],[Q _j ,A _j ]...,[Q _n ,A _n ]And (Q is a customer question and a is an answer). By classification processing, find Q ₁ 、Q ₂ 、Q ₃ If the client questions are the same class of client questions, the client questions are arranged in the same group, and the client answers A ₁ 、A ₂ 、A ₃ And (3) merging, namely if the answer A source is the latest manual answer content, replacing the original stored answer content with the latest answer. Finally, all the classified corpora are arranged into { [ Q ] ₁₁ ,Q ₁₂ ,Q ₁₃ ,...Q _1n ],A ₁ }(Q _1i For the same kind of customer problems, A ₁ Corresponding answer) and with a unique business scenario S ₁ And (5) association. And if the content of the current answer A is NULL, displaying the content to service personnel at a front-end interface, and manually supplementing the content of the answer. The data module automatically processes corpus which cannot acquire answers and is not manually supplemented by service personnel, and the corpus is not acquired by the model training and testing module.

If there is unclassified corpus data { [ M ] _i ,N _i ],[M _j ,N _j ]...,[M _o ,N _o ]Clustering unclassified data, extracting key elements to form new service scene S _i 、S _j 、S _o . And associating the key dimension characteristic information of the service scene with the knowledge graph. All the newly added CORPUS data, business scenes and KNOWLEDGE graph information are respectively stored in a CORPUS_ SHEET, SCENARIO _ SHEET, MAPPING _KNOWLEDGE_DOMAIN_SHEET incremental data table.

And comparing the newly added service scene with the original service scene, and reversely recombining and constructing the corpus by utilizing a statistical language rule in the word segmentation process by using a natural language processing technology. For example: in the banking field, for the existing business scenario "ATM debit card balance query", the corpus of customer questions that the corresponding text robotic system can answer includes: how to query debit card accounts through ATM; how to query the debit card balance through the ATM; ATM can check the balance of the debit card; inquiring balance information of the ATM debit card; i want to ask how to query the debit card foreign exchange balance "through the ATM. If a new business scene of 'ATM no-card withdrawal number inquiry' appears, and if the corresponding corpus data only has 'how to know the no-card withdrawal number', then constructing a corpus supplementary data warehouse such as 'how to inquire the no-card withdrawal number by the ATM', 'how to inquire the no-card withdrawal number by the ATM' according to the language rules counted by the existing corpus, and the like, for training and optimizing a robot model.

The risk monitoring and evaluating module monitors business indexes such as customer satisfaction and response accuracy in real time, and informs business personnel when the risk continuously rises to a certain threshold value, and the business personnel can terminate automatic upgrading through manual judgment according to the evaluation starting system upgrading program.

After the system upgrading program is started, the model training and testing module acquires new and original business scenes, corpus data and atlas labels from the data module, and divides the new and original business scenes, corpus data and atlas labels into a training set and a testing set according to preset sample set dividing rules, and robot model training and testing verification are started. And selecting an optimal version of model training, and storing the current version, parameters and data content after the optimal version passes the service index through verification of the test set.

(3) Model training and testing module: the module intercepts new corpus data and corresponding scenes generated from the last training in the data warehouse, positions key problems focused by current clients or service fields which cannot be responded by the text robot according to the new scenes, and determines the system upgrading direction. And acquiring a training set and a testing set of the new and original service scenes, starting the training of a system core model, comparing and selecting an optimal model, and storing model codes, parameters and data.

(4) Automatic version synthesis deployment module: and synthesizing the code, parameters, data and other contents of the optimal model to form a production version to finish deployment. After the system is reformed and online, the risk monitoring and evaluating module is used for checking the actual application effect of the system in a certain time window, and if the service requirement is not met, version rollback can be realized.

Based on the above, the text robot automatic monitoring method provided by the application example of the application provides a method for automatically constructing, testing and deploying text robots in the professional field, which can monitor the running condition of a system, automatically early warn and start an upgrading program, acquire corpus data of customers so as to identify newly added business scenes, take the business as a guide to improve the system function, complete the training, tuning and testing and evaluating of a robot model, and realize the synthesis and automatic deployment of version programs, parameters and data information. Through setting up risk control and evaluation module, carry out real-time supervision to business index such as customer satisfaction and response accuracy of text robot, when the system running condition can not satisfy market demand, initiatively early warning and trigger the upgrading procedure. The data module acquires response data to the customer, completes data cleaning and processing and stores effective corpus data. And identifying new business scenes related to the corpus data by using a clustering algorithm, extracting language rules by using a natural language processing technology, and expanding a training set and a testing set through text recombination. The model training and testing module is in butt joint with the data module, acquires new and original corpus data, corresponding business scenes and knowledge maps to perform model training and tuning, and stores parameters and data of an optimal model. The version synthesis and deployment module completes the packaging and automatic deployment of the version, the parameters and the data. And establishing a post-evaluation mechanism of version quality, and monitoring the system condition in real time by a risk monitoring and evaluating module, wherein version rollback can be realized when the requirements are not met.

From the above description, the text robot automatic monitoring method provided by the application example of the application can automatically monitor the system condition, send out early warning information in real time when the market demand is not satisfied, start the upgrading process, greatly reduce the manpower cost required by manual monitoring and evaluating the system operation condition in the past, and simultaneously shorten the stand, evaluation and implementation period required by upgrading and reconstruction of the system. The method can also automatically screen client data, extract key elements such as corpus data, business scenes and knowledge patterns, form training set data and testing set data, train and tune a system model, reduce the inefficiency of manually processing data, possibly cause data omission and repetition, and avoid the situation that newly added business scenes cannot be accurately positioned, and avoid the influence of insufficient personnel experience on system transformation and testing.

In order to solve the problem that the existing text robot monitoring mode cannot simultaneously ensure the application accuracy, updating efficiency and the like of the text robot, the application provides an embodiment of electronic equipment for realizing all or part of contents in the text robot automatic monitoring method, wherein the electronic equipment specifically comprises the following contents:

Fig. 11 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 11, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 11 is exemplary; other types of structures may also be used in addition to or in place of the structures to implement telecommunications functions or other functions.

In an embodiment, the text robot auto-monitoring function may be integrated into the central processor. Wherein the central processor may be configured to control:

From the above description, it can be seen that the electronic device provided by the embodiment of the application can automatically monitor the service and the running state of the text robot, automatically determine whether the text robot needs to be upgraded, automatically update the model training and upgrade the system of the text robot, and automatically detect and update the text robot with high intelligent degree, thereby effectively improving the update efficiency and the automation degree of the text robot, effectively saving the labor cost and the time cost, and effectively ensuring the accuracy and the reliability of the response service of the text robot to the user.

In another embodiment, the text robot automatic monitoring device may be configured separately from the central processor 9100, for example, the text robot automatic monitoring device may be configured as a chip connected to the central processor 9100, and the text robot automatic monitoring function is implemented by control of the central processor.

As shown in fig. 11, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 need not include all of the components shown in fig. 11; in addition, the electronic device 9600 may further include components not shown in fig. 11, and reference may be made to the related art.

As shown in fig. 11, the central processor 9100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, which central processor 9100 receives inputs and controls the operation of the various components of the electronic device 9600.

The memory 9140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information about failure may be stored, and a program for executing the information may be stored. And the central processor 9100 can execute the program stored in the memory 9140 to realize information storage or processing, and the like.

The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, but not limited to, an LCD display.

The memory 9140 may be a solid state memory such as Read Only Memory (ROM), random Access Memory (RAM), SIM card, etc. But also a memory which holds information even when powered down, can be selectively erased and provided with further data, an example of which is sometimes referred to as EPROM or the like. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 storing application programs and function programs or a flow for executing operations of the electronic device 9600 by the central processor 9100.

The memory 9140 may also include a data store 9143, the data store 9143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, address book applications, etc.).

The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. A communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, as in the case of conventional mobile communication terminals.

Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, etc., may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132 to implement usual telecommunications functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100 so that sound can be recorded locally through the microphone 9132 and sound stored locally can be played through the speaker 9131.

An embodiment of the present application also provides a computer-readable storage medium capable of implementing all steps in the text robot automatic monitoring method in the above embodiment, the computer-readable storage medium storing thereon a computer program which, when executed by a processor, implements all steps in the text robot automatic monitoring method in which an execution subject in the above embodiment is a server or a client, for example, the processor implements the following steps when executing the computer program:

As can be seen from the above description, the computer readable storage medium provided by the embodiments of the present application can automatically monitor the service and the running state of the text robot, automatically determine whether the text robot needs to be updated, automatically update the model training and upgrade the system of the text robot, and automatically detect and update the text robot with high intelligent degree, thereby effectively improving the update efficiency and the automation degree of the text robot, effectively saving the labor cost and the time cost, and effectively ensuring the accuracy and the reliability of the response service of the text robot to the user.

It will be apparent to those skilled in the art that embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The principles and embodiments of the present invention have been described in detail with reference to specific examples, which are provided to facilitate understanding of the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims

1. An automatic text robot monitoring method, comprising:

Replacing the text robots with updated text robots, wherein the service index data of the text robots do not meet the service standard, so as to process the problem response data based on the updated text robots;

the text robot automatic monitoring method further comprises the following steps:

2. The automatic text robot monitoring method according to claim 1, wherein the training the text robot to obtain the updated text robot based on the pre-stored original corpus data and the new corpus data generated by the text robot during the operation includes:

3. The automatic text robot monitoring method according to claim 2, wherein the training the text robot with the business index data not meeting the business standard according to the original corpus data and the newly added corpus data to obtain the updated text robot comprises:

and determining the text robot passing the model test as an updated text robot.

4. The automatic text robot monitoring method according to claim 1, further comprising, after the application of the updated text robot to replace the text robot whose business index data does not satisfy the business criterion to perform the problem response data processing based on the updated text robot:

5. The automatic text robot monitoring method according to claim 1, wherein the performing data deduplication processing on the data corresponding to the existing business scenario in the question response data to update the original corpus data corresponding to the existing business scenario includes:

6. The text robot automatic monitoring method according to claim 1, wherein the performing the new business scenario recognition on the data that does not correspond to the existing business scenario in the question response data includes:

7. The text robot automatic monitoring method according to claim 1, wherein the applying a natural language processing technology to update and expand corpus data corresponding to the new business scene based on the existing business scene, and associating scene features of the new business scene with a preset knowledge graph to generate the new corpus data corresponding to the new business scene includes:

8. An automatic text robot monitoring device, comprising:

the model deployment module is used for replacing the text robots of which the business index data do not meet the business standards by using the updated text robots so as to process the problem response data based on the updated text robots;

The text robot automatic monitoring device further comprises:

9. The automated text robot monitoring device of claim 8, wherein the risk assessment and model update module comprises:

10. The text robot automatic monitoring device of claim 9, wherein the model training unit is configured to perform the following:

and determining the text robot passing the model test as an updated text robot.

11. The text robot automatic monitoring device of claim 8, further comprising:

12. The text robot automatic monitoring device of claim 8, wherein the original corpus update module is configured to perform the following:

13. The automated text robot monitoring device of claim 8, wherein the add-on scene recognition module is configured to:

14. The automated text robot monitoring device of claim 8, wherein the new corpus generation module is configured to perform the following:

15. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the text robot automatic monitoring method of any of claims 1 to 7 when executing the program.

16. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the text robot automatic monitoring method of any one of claims 1 to 7.