CN117993381A

CN117993381A - Information processing method, information processing device, computer equipment and storage medium

Info

Publication number: CN117993381A
Application number: CN202410255822.8A
Authority: CN
Inventors: 薛晓舟; 蔡承蒙; 薛研歆
Original assignee: Hangzhou Ansiyuan Technology Co ltd
Current assignee: Hangzhou Ansiyuan Technology Co ltd
Priority date: 2024-03-06
Filing date: 2024-03-06
Publication date: 2024-05-07

Abstract

The disclosure relates to the technical field of internet, and discloses an information processing method, an information processing device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving demand information of a target user; generating a work plan corresponding to the demand information according to the demand information by using a large language model, wherein the work plan corresponding to the demand information indicates a plurality of operations aiming at the demand information and having sequence; using the large language model, the following target operations among the plurality of operations are performed: carrying out intention recognition and pre-operation on the demand information, determining whether the intention recognized by the intention recognition meets the intention reasonable condition, and determining whether a pre-operation result obtained by the pre-operation meets the result reasonable condition corresponding to the pre-operation result; and executing at least part of subsequent operations in the plurality of operations when the identified intention meets the intention reasonable condition and the front operation result meets the result reasonable condition corresponding to the front operation result.

Description

Information processing method, information processing device, computer equipment and storage medium

Technical Field

The disclosure relates to the technical field of internet, and in particular relates to an information processing method, an information processing device, computer equipment and a storage medium.

Background

Currently, a large language model (LargeLanguageModel, abbreviated as LLM) is applied in the processing of the demands of users, such as the processing of the refund demands of users, to replace some of the customer service personnel, saving labor costs. In the related art, a workflow (workflow) for information indicating a user's demand is generated by a large language model from received information indicating a user's demand, the workflow for information indicating a user's demand including all operations for information indicating a user's demand, the operations for information indicating a user's demand being sequentially performed in order. However, large language models sometimes fail to meet the needs of users due to model illusions that result in erroneous final results after all operations in the execution workflow have been performed. How to improve the accuracy of the final result obtained when processing a process for the needs of the user using a large language model becomes a problem to be solved.

Disclosure of Invention

In view of this, embodiments of the present disclosure provide an information processing method, apparatus, computer device, and storage medium.

In a first aspect, an embodiment of the present disclosure provides an information processing method, including:

receiving demand information of a target user, wherein the demand information indicates the demand of the target user;

Generating a work plan corresponding to the demand information according to the demand information by using a large language model, wherein the work plan corresponding to the demand information indicates a plurality of operations aiming at the demand information and having a sequence;

Performing, with the large language model, each of the following target operations of the plurality of operations: performing intention recognition, pre-operation on the demand information, determining whether the intention recognized by the intention recognition meets an intention reasonable condition, and determining whether a pre-operation result obtained by the pre-operation meets a result reasonable condition corresponding to the pre-operation result;

and when the identified intention meets the intention reasonable condition and the result of the pre-operation meets the result reasonable condition corresponding to the result of the pre-operation, executing at least part of subsequent operations in the plurality of operations, wherein the subsequent operations are operations except for each target operation in the plurality of operations, and the position of each target operation in the sequence is before the position of the subsequent operation in the sequence.

In a second aspect, an embodiment of the present disclosure provides an information processing apparatus including:

A receiving unit, configured to receive requirement information of a target user, where the requirement information indicates a requirement of the target user;

A generation unit, configured to generate a work plan corresponding to the demand information according to the demand information using a large language model, where the work plan corresponding to the demand information indicates a plurality of operations with order for the demand information;

A first execution unit configured to execute each of the following target operations among the plurality of operations using the large language model: performing intention recognition, pre-operation on the demand information, determining whether the intention recognized by the intention recognition meets an intention reasonable condition, and determining whether a pre-operation result obtained by the pre-operation meets a result reasonable condition corresponding to the pre-operation result;

And a second execution unit configured to execute at least a part of subsequent operations among the plurality of operations when it is determined that the identified intention satisfies an intention reasonable condition and the result of the preceding operation satisfies a result reasonable condition corresponding to the result of the preceding operation, wherein the subsequent operations are operations other than the each target operation among the plurality of operations, and a position of each target operation in the sequence is before a position of the subsequent operation in the sequence.

In a third aspect, embodiments of the present disclosure provide a computer device comprising: the memory and the processor are in communication connection, the memory stores computer instructions, and the processor executes the computer instructions to perform the method of the first aspect or any implementation manner corresponding to the first aspect.

In a fourth aspect, the disclosed embodiments provide a computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the method of the first aspect or any of its corresponding embodiments.

The information processing method provided by the embodiment of the disclosure judges whether the identified intention is reasonable or not and whether the front-end operation result is reasonable or not when the processing process aiming at the demands of the user is performed by utilizing the large language model, namely, determines that the identified intention meets the intention reasonable condition and determines that the front-end operation result meets the result reasonable condition corresponding to the front-end operation result, the identified intention can be determined to be reasonable and the front operation result can be determined to be reasonable, the subsequent operation can be performed under the condition that the intention is reasonable and the front operation result is reasonable, and the situation that the operation result of the operation which is caused by the model illusion to be unreasonable and the operation result of the subsequent operation which is caused by the operation result of the operation which is caused by the front position is wrong and the obtained final result is wrong is avoided. And the accuracy of the final result obtained when the processing process aiming at the user requirement is carried out by utilizing the large language model is improved.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the prior art, the drawings that are required in the detailed description or the prior art will be briefly described, it will be apparent that the drawings in the following description are some embodiments of the present disclosure, and other drawings may be obtained according to the drawings without inventive effort for a person of ordinary skill in the art.

Fig. 1 is a flow chart of an information processing method provided in an embodiment of the present disclosure;

FIG. 2 is a flow chart of another information processing method provided by an embodiment of the present disclosure;

FIG. 3 is a flowchart of one example of an information processing method provided by an embodiment of the present disclosure;

Fig. 4 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some embodiments of the present disclosure, but not all embodiments. Based on the embodiments in this disclosure, all other embodiments that a person skilled in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.

Referring to fig. 1, an example flowchart of an information processing method provided by an embodiment of the present disclosure is shown.

In step S101, demand information of a target user is received.

Wherein the demand information of the target user indicates the demand of the target user.

In the embodiment of the present disclosure, the target user may be any user that may interact by using the method provided by the embodiment of the present disclosure.

The demand information of the target user may be generated by an operation of the target user on a page of the application performed with the user when the user uses the application running on the user's terminal. The system for executing the method provided by the embodiment of the disclosure can receive the requirement information of the target user from the terminal used by the user. A system for performing the methods provided by embodiments of the present disclosure includes a large language model.

As one example, a user may desire to refund a purchased item using an e-commerce application. The user inputs information indicating that the user needs refund, such as 'i want refund', in a page of the e-commerce application, which interacts with the user, or clicks a button representing refund in the page of the e-commerce application, which interacts with the user, and generates demand information of the target user, which indicates that refund is needed. The terminal running the e-commerce application transmits the requirement information of the target user to a system executing the information processing method provided by the embodiment of the disclosure.

In step S102, a work plan corresponding to the demand information of the target user is generated according to the demand information of the target user using the large language model.

Wherein the work plan (workplan) corresponding to the demand information of the target user indicates a plurality of operations with order for the demand information of the target user.

As one example, the plurality of operations that the large language model determines that the work plan indicates that the demand information of the target user indicates "i want refund" that a refund is required include: the method includes the steps of carrying out intention recognition on demand information of a target user, acquiring an order number, determining whether intention recognized through the intention recognition meets an intention reasonable condition and determining whether the acquired order number meets a result reasonable condition corresponding to the acquired order number, generating codes for acquiring detailed information of an order to which the order number belongs when the recognized intention meets the intention reasonable condition and determining that the acquired order number meets the result reasonable condition corresponding to the acquired order number, acquiring the detailed information of the order to which the order number belongs, confirming whether a form including the detailed information of the order to which the acquired order number belongs is correct or not by the user, generating codes for calling an API of an express system to carry out express delivery on the form including the detailed information of the order to which the acquired order number belongs after the user confirms that the form including the detailed information of the order to which the acquired order number belongs, carrying out express delivery on the form including the detailed information of the order to which the acquired order number belongs, acquiring goods related to an express delivery person, and transporting the goods related to refund to an update database. Updating the database includes adding return information corresponding to the acquired order number to the database, and updating the status recorded in the database corresponding to the acquired order number to a successful refund.

In the embodiment of the present disclosure, part of operations of a plurality of operations indicated by a work plan corresponding to demand information of a target user may be performed by a large language model. Wherein,

Each of the partial operations of the plurality of operations indicated by the work plan corresponding to the demand information of the target user may be respectively performed by an Agent (Agent) of the large language model. The large language model may operate in a Multi-Agent mode. An Agent of the large language model may autonomously execute chained calls and access external tools. Multiple agents of the large language model share a part of memory and cooperate with each other independently.

For example, the code that performs intention recognition on demand information of a target user, acquires an order number, determines whether the intention recognized by the intention recognition satisfies an intention reasonable condition, determines whether the acquired order number satisfies a result reasonable condition corresponding to the acquired order number, generates detailed information of an order to which the acquired order number belongs when it is determined that the recognized intention satisfies the intention reasonable condition, and determines that the acquired order number satisfies the result reasonable condition corresponding to the acquired order number, and the like may be performed by a large language model. The operations such as the courier obtaining the commodity related to refund, and transporting the commodity related to refund to the merchant are executed by the corresponding executor.

In step S103, each of the following target operations among the plurality of operations indicated by the work plan corresponding to the demand information of the target user is executed using the large language model: performing intention recognition, pre-operation on the demand information, determining whether the intention recognized by the intention recognition meets an intention reasonable condition, and determining whether a pre-operation result obtained by the pre-operation meets a result reasonable condition corresponding to the pre-operation result.

The method comprises the steps of carrying out intention recognition, pre-operation on the requirement information, determining whether the intention recognized by the intention recognition meets an intention reasonable condition, and determining whether a pre-operation result obtained by the pre-operation meets a result reasonable condition corresponding to the pre-operation result, wherein the result reasonable condition is the first three operations in the sequence indicated by the work plan corresponding to the requirement information of a target user.

The work plan instruction corresponding to the demand information of the target user sequentially performs three operations of performing intention recognition on the demand information, performing a pre-operation, determining whether the intention recognized by the intention recognition satisfies an intention reasonable condition, and determining whether a result of the pre-operation obtained by the pre-operation satisfies a result reasonable condition corresponding to the result of the pre-operation. First, the demand information is subjected to intention recognition, then a pre-operation is performed, and then, it is determined whether the intention recognized by the intention recognition satisfies an intention reasonable condition and whether a result of the pre-operation obtained by the pre-operation satisfies a result reasonable condition corresponding to the result of the pre-operation.

As one example, the plurality of operations by which the large language model determines the work plan indication for the "i want refund" indicating that refund is required include: the method includes the steps of carrying out intention recognition on demand information of a target user, acquiring an order number, determining whether intention recognized through the intention recognition meets an intention reasonable condition and determining whether the acquired order number meets a result reasonable condition corresponding to the acquired order number, generating codes for acquiring detailed information of an order to which the order number belongs when the recognized intention meets the intention reasonable condition and determining that the acquired order number meets the result reasonable condition corresponding to the acquired order number, acquiring the detailed information of the order to which the order number belongs, confirming whether a form including the detailed information of the order to which the acquired order number belongs is correct or not by the user, generating codes for calling an API of an express system to carry out express delivery on the form including the detailed information of the order to which the acquired order number belongs after the user confirms that the form including the detailed information of the order to which the acquired order number belongs, carrying out express delivery on the form including the detailed information of the order to which the acquired order number belongs, acquiring goods related to an express delivery person, and transporting the goods related to refund to an update database. Updating the database includes adding return information corresponding to the acquired order number to the database, and updating the status recorded in the database corresponding to the acquired order number to a successful refund.

In this example, the target operation is performed by performing intention recognition on demand information of the target user, acquiring an order number, determining whether the intention recognized by the intention recognition satisfies an intention reasonable condition, and determining whether the acquired order number satisfies a result reasonable condition corresponding to the acquired order number. The method comprises the steps of acquiring an order number as a pre-operation, wherein the acquired order number is a pre-operation result.

In this example, the order instruction of the work plan instruction corresponding to the demand information of the target user first performs three target operations of performing intention recognition on the demand information of the target user, acquiring an order number, determining whether the intention recognized by the intention recognition satisfies the intention reasonable condition, and determining whether the acquired order number satisfies the result reasonable condition corresponding to the acquired order number.

In the embodiment of the present disclosure, the intention reasonable condition is a condition for judging whether the identified intention is reasonable. The result reasonable condition corresponding to the pre-operation result is a condition for judging whether the pre-operation result is reasonable or not.

The identified intent indicates an operation that the target user desires to perform through the application. The target user inputs the requirement information of the target user in this application. Whether the identified intent is reasonable may be measured in terms of whether the operation that the identified intent indicates is expected to be performed by the application when the user is using the application. For this reason, a plurality of operations that the user may desire to perform by the application when the user uses the application may be set in advance. The intent rationale condition may be: whether the identified intent indicates that the desired operation is a preset operation that the user may desire to be performed by the application while using the application.

As one example, a user may desire to refund a purchased item using an e-commerce application. The demand information of the target user is "i want refund" indicating that refund is required. If the identified intention of the user is refund, presetting a plurality of operations which the user may expect to perform through the application when the user uses the e-commerce application, including refund operation, and then enabling the identified intention to meet the intention reasonable condition, wherein the identified intention is reasonable. If the identified intention of the user is tax refund, the e-commerce application has no tax refund function, and the tax refund is not an operation which the user may expect to perform through the application when using the e-commerce application, the identified intention does not meet the reasonable intention condition, and the identified intention is unreasonable.

In the embodiment of the disclosure, whether the pre-operation result is reasonable can be measured according to whether the target user has authority to check the information comprising the pre-operation result and whether the format of the pre-operation result is one of a plurality of preset formats. The result reasonable condition corresponding to the pre-operation result may be: the target user has authority to view information including the result of the pre-operation and the format of the result of the pre-operation is one of a plurality of preset formats.

In step S104, when it is determined that the identified intention satisfies the intention reasonable condition and it is determined that the result of the pre-operation satisfies the result reasonable condition corresponding to the result of the pre-operation, at least part of the subsequent operations among the operations indicated by the work plan corresponding to the demand information of the target user are performed.

And the subsequent operations are operations except for each target operation, wherein the position of each target operation in the sequence indicated by the work plan corresponding to the requirement information of the target user is before the position of any subsequent operation in the sequence.

At least part of the subsequent operations in the plurality of operations indicated by the work plan corresponding to the demand information of the target user are sequentially executed according to the sequence.

The method comprises the steps of sequentially executing three target operations of intention recognition on the requirement information, pre-operation, determining whether the intention recognized by the intention recognition meets an intention reasonable condition and determining whether a pre-operation result obtained by the pre-operation meets a result reasonable condition corresponding to the pre-operation result according to the sequence instruction of work plan instructions corresponding to the requirement information of a target user.

In this example, the target operation is performed by performing intention recognition on demand information of the target user, acquiring an order number, determining whether the intention recognized by the intention recognition satisfies an intention reasonable condition, and determining whether the acquired order number satisfies a result reasonable condition corresponding to the acquired order number. Acquiring the order number as a pre-operation, and acquiring the order number as a pre-operation result.

In this example, the code that obtains the detailed information of the order to which the order number belongs is generated when it is determined that the identified intention satisfies the intention reasonable condition and that the obtained order number satisfies the result reasonable condition corresponding to the obtained order number. In this example, after the code for acquiring the detailed information of the order to which the order number belongs is generated, the detailed information of the order to which the order number belongs may be acquired. After the detailed information of the order to which the order number belongs is acquired, whether or not the form including the detailed information of the order to which the acquired order number belongs is correct may be confirmed by the user. The code for calling the API of the express delivery system to place the form including the detailed information of the order to which the acquired order number belongs may be generated after the user confirms that the form including the detailed information of the order to which the acquired order number belongs is correct. After the code for calling the API of the express system to express the form including the detailed information of the order to which the acquired order number belongs is generated, the code for calling the API of the express system to express the form including the detailed information of the order to which the acquired order number belongs may be executed to express the form including the detailed information of the order to which the acquired order number belongs. After the form including the detailed information of the order to which the acquired order number belongs is placed in the express delivery, the courier obtains the commodity related to the refund. After the courier obtains the goods related to the refund, the goods related to the refund are transported to the merchant. After shipping the goods involved in the refund to the merchant, the database is updated. Updating the database includes adding return information corresponding to the acquired order number to the database, and updating the status recorded in the database corresponding to the acquired order number to a successful refund.

In the embodiments of the present disclosure, a large language model is trained using training data of the large language model before the information processing method provided by the embodiments of the present disclosure is performed. The training data of one large language model includes demand information for training and a plurality of operations having a sequence that are labeled in advance and can successfully satisfy the demand indicated by the demand information for training. By training the large language model with the training data, such that after the training of the large language model is completed, the large language model can learn, for the needs indicated by the need information for training, which operation-including work plans should be generated for the needs indicated by the need information for training. A work plan with sequential operations that includes pre-labeled requirements that can successfully meet the requirements indicated by the requirement information for training can be understood as a work plan that a large language model should generate for the requirements indicated by the requirement information for training.

Referring to fig. 2, a flow diagram of another information processing method provided by an embodiment of the present disclosure is shown.

In step S201, demand information of a target user is received.

Process parameters step S101 of step S201.

In step S202, a work plan corresponding to the demand information of the target user is generated according to the demand information of the target user using the large language model.

Process parameters step S102 of step S202.

In step S203, each of the following target operations among the plurality of operations indicated by the work plan corresponding to the demand information of the target user is executed using the large language model: performing intention recognition, pre-operation on the demand information, determining whether the intention recognized by the intention recognition meets an intention reasonable condition, and determining whether a pre-operation result obtained by the pre-operation meets a result reasonable condition corresponding to the pre-operation result.

Process parameters step S103 of step S203.

At step S204, when it is determined that the identified intention satisfies the intention reasonable condition and the result of the pre-operation satisfies the result reasonable condition corresponding to the result of the pre-operation, performing at least part of the subsequent operations among the plurality of operations indicated by the work plan corresponding to the requirement information of the target user, wherein after performing the first subsequent operation and before performing the second subsequent operation corresponding to the first subsequent operation, it is determined whether the code of the second subsequent operation generated by the first subsequent operation is legal; and when the code of the second subsequent operation is determined to be legal, executing the code of the second subsequent operation to execute the second subsequent operation.

The first subsequent operation is an operation for generating code among the plurality of operations. The second subsequent operation corresponding to the first subsequent operation is the next operation of the first subsequent operation indicated by the sequence.

Wherein the first subsequent operation may be performed by an Agent of the large language model.

For a subsequent operation of the plurality of operations, if the subsequent operation is used to generate code and there is a next operation of the subsequent operation indicated by the sequence, the subsequent operation may be referred to as a first subsequent operation.

In the disclosed embodiments, consider the case where an illegitimate code generated when performing a process for a user's needs using a large language model would cause an unsafe system with a large language model to interact with the user. When the code of the second subsequent operation is determined to be legal, the code of the second subsequent operation is executed to execute the second subsequent operation, so that the condition that a system with a large language model and interacting with a user is unsafe due to the fact that the code of the second subsequent operation is executed when the code of the second subsequent operation is illegal can be avoided.

As one example, the plurality of operations that the large language model determines that the work plan indicates that the demand information of the target user indicates "i want refund" that a refund is required include: performing intention recognition on demand information of a target user, acquiring an order number, determining whether the intention recognized by the intention recognition satisfies an intention reasonable condition and determining whether the acquired order number satisfies a result reasonable condition corresponding to the acquired order number, generating a code for acquiring detailed information of an order to which the order number belongs when determining that the recognized intention satisfies the intention reasonable condition and determining that the acquired order number satisfies the result reasonable condition corresponding to the acquired order number, determining whether the code for generating the detailed information of the order to which the order number belongs is legal, acquiring the detailed information of the order to which the order number belongs when determining that the code for generating the detailed information of the order to which the order number belongs is legal, and determining whether a form including the detailed information of the order to which the acquired order number belongs is correct after confirming that the form comprising the detailed information of the order to which the acquired order number belongs is correct, the user generates a code for calling an API of the express delivery system to express the form comprising the detailed information of the order to which the acquired order number belongs, determines whether the code for calling the API of the express delivery system to express the form comprising the detailed information of the order to which the acquired order number belongs is legal, and when determining that the code for calling the API of the express delivery system to express the form comprising the detailed information of the order to which the acquired order number belongs is legal, express the form comprising the detailed information of the order to which the acquired order number belongs, express a commodity to which a refund relates, and conveys the commodity to which the refund relates to a merchant and updates a database. Updating the database includes adding return information corresponding to the acquired order number to the database, and updating the status recorded in the database corresponding to the acquired order number to a successful refund.

In this example, the code for generating the detailed information of the order to which the order number belongs, the code for generating the API for calling the express delivery system to place the express delivery order on the form including the detailed information of the order to which the order number belongs, are all the first subsequent operations. And generating a second subsequent operation corresponding to the code for acquiring the detailed information of the order number, wherein the second subsequent operation corresponds to the code for acquiring the detailed information of the order number, and executing the code for acquiring the detailed information of the order number, thereby acquiring the detailed information of the order number. Generating a second subsequent operation corresponding to the code for calling the API of the express system to express the form comprising the detailed information of the order to which the acquired order number belongs, wherein the second subsequent operation corresponds to the code for calling the API of the express system to express the form comprising the detailed information of the order to which the acquired order number belongs, and executing the code for calling the API of the express system to express the form comprising the detailed information of the order to which the acquired order number belongs, so that the form comprising the detailed information of the order to which the acquired order number belongs is expressed.

In the embodiment of the disclosure, any means for detecting whether the code is legal or not can be used to detect whether the generated code is legal or not. When the generated code is detected to be illegal by at least one means for detecting whether the code is legal, the code is determined to be illegal. And when the detection result of each detection means indicates that the code is legal, determining that the generated code is legal.

As one example, it is detected whether the generated code satisfies at least one of: the code is for modifying data in the data source, the code is for deleting data from the data source, the code is for transmitting secure data to an external network, and the code is for saving the secure data locally. If yes, determining that the code is illegal. If not, determining that the code is legal.

In the embodiment of the disclosure, determining whether a code of a second subsequent operation corresponding to the first subsequent operation generated through the first subsequent operation is legally included; at least one of a plurality of detections of code of a second subsequent operation corresponding to the first subsequent operation generated by the first subsequent operation is performed, the plurality of detections including: detecting whether the code is used for modifying data in the data source, detecting whether the code is used for deleting data from the data source, detecting whether the code accords with a preset grammar rule, detecting whether the code is used for sending the secret data to an external network, and detecting whether the code is used for storing the secret data to a local area; when the detection result of each detection indicates that the code is legal, determining that the code is legal; when the detection result of the at least one detection indicates that the code is illegal, it is determined that the code is illegal.

Wherein when the code is detected to be used for modifying data in the data source, a detection result of the detection of whether the code is used for modifying data in the data source indicates that the code is illegal. When it is detected that the code is not used to modify data in the data source, a detection result of the detection of whether the code is used to modify data in the data source indicates that the code is legitimate.

When the code is detected for deleting data from the data source, a detection result of the detection of whether the code is used for deleting data from the data source indicates that the code is illegal. When it is detected that the code is not used to delete data from the data source, then a detection result of the detection of whether the code is used to delete data from the data source indicates that the code is legitimate.

When the code is detected to be in accordance with the preset grammar rule, detecting whether the code is in accordance with the detection result of the preset grammar rule indicates that the code is legal. And when the code is detected to be not in accordance with the preset grammar rule, detecting whether the code is detected to be in accordance with the preset grammar rule or not, wherein a detection result of detecting whether the code is in accordance with the preset grammar rule indicates that the code is illegal.

When the code is detected for transmitting the secret data to the external network, a detection result of detecting whether the code is used for transmitting the secret data to the external network indicates that the code is illegal. When it is detected that the code is not used for transmitting the secret data to the external network, a detection result of detecting whether the code is used for transmitting the secret data to the external network indicates that the code is legal.

When the code is detected to be used for storing the secret data locally, a detection result of the detection of whether the code is used for storing the secret data locally indicates that the code is illegal. When it is detected that the code is not used for saving the secret data to the local, a detection result of the detection of whether the code is used for saving the secret data to the local indicates that the code is legal.

In the embodiment of the present disclosure, when it is determined that the code of the second subsequent operation corresponding to the first subsequent operation is not legal, a preset code corresponding to the second subsequent operation may be executed, where the preset code corresponding to the second subsequent operation is predetermined to be legal.

In the embodiment of the disclosure, the accuracy of work plans for large language model production is improved. A plurality of first training data may be acquired. The first training data may be obtained from log data recording data produced during interactions of the large model with the user after the large model is online. The first training data includes: the method comprises the steps of receiving demand information indicating the demands of a user in the completed processing process of successfully meeting the demands of the user by using a large language model, and generating a work plan meeting the demands indicated by the received demand information by using the large language model in the completed processing process of successfully meeting the demands of the user by using the large language model; the large language model is fine-tuned using a plurality of first training data.

In the embodiment of the disclosure, in order to improve the accuracy of the code generated by the large language model, a plurality of second training data may be acquired. The second training data may be obtained from log data recording data produced during interactions of the large model with the user after the large model is online. The second training data includes: the method comprises the steps of receiving demand information indicating the demands of users in the completed processing process of successfully meeting the demands of users by using the large language model, and generating codes by using the large language model in the completed processing process of successfully meeting the demands of users by using the large language model; the large language model is fine-tuned using a plurality of second training data. In addition, the user privacy information can be removed from the code, summarizing the generic code pattern. The large language model is fine-tuned using the generic code pattern and the received demand information indicating the user's demand.

Referring to fig. 3, a flowchart of one example of an information processing method provided by an embodiment of the present disclosure is shown.

In this example, the target user desires to apply for refunds to goods purchased by the target user using an e-commerce application. The e-commerce application used by the target user sends demand information "i want refund" indicating the demand of the target user to the large language model inference service. The large language model reasoning service interacts with the user through the large language model. And the large language model performs intention recognition and order number extraction on the requirement information of the target user at the same time to acquire the order number. Checkpoint 1 may be an Agent of a large language model. Checkpoint 1 checks whether the identified intent is reasonable and checks whether the acquired order number is reasonable. If the identified intention is not reasonable and/or the acquired order number is not reasonable, the check fails, and step 3a is performed to send a failure message to the target user. If the identified intention is checked to be reasonable and/or the acquired order number is checked to be reasonable, the check is successful, and the step 3b is continued to be executed. An Agent for generating a large language model of codes generates codes that query for detailed information of outstanding orders. This code for inquiring about the detailed information of the unfinished order refers to the code for acquiring the detailed information of the order to which the order number belongs. Checkpoint 2 may be an Agent of a large language model. The check point 2 checks the validity of the code for inquiring the detailed information of the incomplete order, and after confirming that the code for inquiring the detailed information of the incomplete order is valid, the step 5a is performed to execute the code for inquiring the detailed information of the incomplete order, and inquires the detailed information of the incomplete order. If the code to query the detailed information of the incomplete order is not legal, step 5b may be performed to re-execute the intent recognition and generate code flow. After step 5a is performed, a refund order is confirmed by the user including detailed information of the unfinished order. After confirming the refund order, the Agent of the large language model for generating the code for calling the API of the express delivery system to make the express delivery order generates the code for calling the API of the express delivery system. Checkpoint 3 may be an Agent of a large language model. And checking the validity of the generated codes calling the API of the express system, and if the API calling the express system is illegal, calling the API of the express system to carry out express order by using the preset codes calling the API of the express system to carry out express order, so as to call the API of the express system to carry out express order. The courier takes the goods related to refund and transports the goods related to refund to the merchant. And the merchant receives the commodity related to refund and performs quality inspection. The database is updated after the quality inspection is passed. After updating the database, the information of successful refund is pushed to the Agent responsible for notifying the user of the large language model, and the Agent responsible for notifying the user of the large language model notifies the user of the result, i.e., the refund success.

The embodiment of the disclosure provides an information processing device. The device is used for realizing the above embodiments and preferred embodiments, and is not described in detail. As used below, the term "unit" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The information processing apparatus includes:

In an optional implementation manner, the second execution unit is further configured to determine, after executing a first subsequent operation and before executing a second subsequent operation corresponding to the first subsequent operation, whether code of the second subsequent operation generated by the first subsequent operation is legal, where the first subsequent operation is an operation for generating code in the plurality of operations, and the second subsequent operation corresponding to the first subsequent operation is a next operation of the first subsequent operation indicated by the sequence; and when the code of the second subsequent operation is determined to be legal, executing the code of the second subsequent operation to execute the second subsequent operation.

In an alternative embodiment, the information processing apparatus further includes:

And the third execution unit is used for executing the preset code corresponding to the second subsequent operation when the code of the second subsequent operation is determined to be illegal, wherein the preset code corresponding to the second subsequent operation is the code which is determined to be legal in advance.

In an alternative embodiment, the first subsequent operation is performed by an agent of the large language model.

In an alternative embodiment, the second execution unit is further configured to perform at least one of a plurality of detections of the code of the second subsequent operation generated by the first subsequent operation, where the plurality of detections includes: detecting whether the code is used for modifying data in a data source, detecting whether the code is used for deleting data from the data source, detecting whether the code accords with a preset grammar rule, detecting whether the code is used for sending secret data to an external network, and detecting whether the code is used for storing the secret data locally; when the detection result of each detection indicates that the code is legal, determining that the code is legal; and determining that the code is illegal when the detection result of the at least one detection indicates that the code is illegal.

The first fine tuning unit is used for acquiring a plurality of first training data, wherein the first training data comprises: the method comprises the steps of receiving demand information indicating the demands of users in the completed processing process of successfully meeting the demands of users by utilizing the large language model, and generating a work plan meeting the demands indicated by the received demand information by utilizing the large language model in the completed processing process of successfully meeting the demands of users by utilizing the large language model; the large language model is fine-tuned using a plurality of first training data.

The second fine tuning unit is configured to obtain a plurality of second training data, where the second training data includes: the method comprises the steps of receiving demand information indicating the demands of users in the completed processing process of successfully meeting the demands of users by using the large language model, and generating codes by using the large language model in the completed processing process of successfully meeting the demands of users by using the large language model; and fine tuning the large language model by using a plurality of second training data.

The apparatus in this embodiment is presented in the form of functional units, where the units refer to ASIC circuits, processors and memories executing one or more software or firmware programs, and/or other devices that can provide the functionality described above.

Further functional descriptions of the above units are the same as those of the above corresponding embodiments, and are not repeated here.

Referring to fig. 4, there is shown a schematic structural diagram of a computer device provided in an embodiment of the present disclosure, the computer device including: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system).

The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.

Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform the methods shown in implementing the above embodiments.

The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.

The computer device further comprises input means 30 and output means 40. The processor 10, memory 20, input device 30, and output device 40 may be connected by a bus or other means.

The input device 30 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointer stick, one or more mouse buttons, a trackball, a joystick, and the like. The output means 40 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. Such display devices include, but are not limited to, liquid crystal displays, light emitting diodes, displays and plasma displays. In some alternative implementations, the display device may be a touch screen.

The presently disclosed embodiments also provide a computer readable storage medium, and the methods described above according to the presently disclosed embodiments may be implemented in hardware, firmware, or as recordable storage medium, or as computer code downloaded over a network that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and is to be stored in a local storage medium, such that the methods described herein may be stored on such software processes on a storage medium using a general purpose computer, special purpose processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments. Although embodiments of the present disclosure have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the disclosure, and such modifications and variations are within the scope defined by the appended claims.

Claims

1. An information processing method, characterized in that the method comprises:

2. The method of claim 1, wherein performing at least a portion of the subsequent operations of the plurality of operations comprises:

After a first subsequent operation is performed and before a second subsequent operation corresponding to the first subsequent operation is performed, determining whether a code of the second subsequent operation generated by the first subsequent operation is legal, wherein the first subsequent operation is an operation for generating a code in the plurality of operations, and the second subsequent operation corresponding to the first subsequent operation is a next operation of the first subsequent operation indicated by the sequence;

and when the code of the second subsequent operation is determined to be legal, executing the code of the second subsequent operation to execute the second subsequent operation.

3. The method according to claim 2, wherein the method further comprises:

And when the code of the second subsequent operation is determined to be illegal, executing the preset code corresponding to the second subsequent operation, wherein the preset code corresponding to the second subsequent operation is the code which is determined to be legal in advance.

4. The method of claim 2, wherein the first subsequent operation is performed by an agent of the large language model.

5. The method of claim 1, wherein determining whether the code of the second subsequent operation generated by the first subsequent operation is legal comprises;

performing at least one of a plurality of detections of code generated by the first subsequent operation and the second subsequent operation, the plurality of detections including: detecting whether the code is used for modifying data in a data source, detecting whether the code is used for deleting data from the data source, detecting whether the code accords with a preset grammar rule, detecting whether the code is used for sending secret data to an external network, and detecting whether the code is used for storing the secret data locally;

when the detection result of each detection indicates that the code is legal, determining that the code is legal;

and determining that the code is illegal when the detection result of the at least one detection indicates that the code is illegal.

6. The method according to any one of claims 1-5, further comprising:

Acquiring a plurality of first training data, the first training data comprising: the method comprises the steps of receiving demand information indicating the demands of users in the completed processing process of successfully meeting the demands of users by utilizing the large language model, and generating a work plan meeting the demands indicated by the received demand information by utilizing the large language model in the completed processing process of successfully meeting the demands of users by utilizing the large language model;

The large language model is fine-tuned using a plurality of first training data.

7. The method of claim 6, wherein the method further comprises:

acquiring a plurality of second training data, the second training data comprising: the method comprises the steps of receiving demand information indicating the demands of users in the completed processing process of successfully meeting the demands of users by using the large language model, and generating codes by using the large language model in the completed processing process of successfully meeting the demands of users by using the large language model;

And fine tuning the large language model by using a plurality of second training data.

8. An information processing apparatus, characterized in that the apparatus comprises:

9. A computer device, comprising:

a memory and a processor in communication with each other, the memory having stored therein computer instructions which, upon execution, cause the processor to perform the method of any of claims 1 to 7.

10. A computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1 to 7.