CN116880269A - Intelligent body control method and device, electronic equipment and readable storage medium - Google Patents

Abstract

The application provides an agent control method, an agent control device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a user command and visual information of an external environment; performing target detection on the visual information to obtain target detection information; splicing the user command and the target detection information to obtain a first instruction; using a first instruction to instruct the trained LLM to select an action to be executed from all primitive actions in the primitive action set; and sending the action to be executed to the intelligent agent so that the intelligent agent executes the action to be executed. The application realizes the interaction between the LLM and the external environment in the reasoning process of adding the visual information into the LLM, and can select the most favorable decision for the intelligent agent based on the environment.

Description

An intelligent body control method, device, electronic equipment and readable storage medium

Technical field

This application belongs to the field of intelligent control technology, and in particular relates to an intelligent body control method, device, electronic equipment and readable storage medium.

Background technique

Currently, a Large Language Model (LLM) is an artificial intelligence model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more.

However, large language models lack real-world experience and cannot provide the most powerful decisions to agents in specific environments.

Contents of the invention

Embodiments of the present application provide an intelligent agent control method, device, electronic device, readable storage medium and computer program product, which can solve the problem that large language models cannot provide the most powerful decision-making for intelligent agents in specific environments.

In the first aspect, embodiments of the present application provide an intelligent agent control method, including:

Obtain user commands and visual information from the external environment;

Perform target detection on the visual information to obtain target detection information;

Splice the user command and the target detection information to obtain the first instruction;

Using the first instruction, instruct the trained LLM to select an action to be executed from each primitive action in the primitive action set;

The action to be executed is sent to the intelligent agent, so that the intelligent agent executes the action to be executed.

In one embodiment, the method further includes:

Input the first instruction to the trained action preliminary screening model to obtain a relevant action set output by the trained action preliminary screening model, where the relevant action set includes the primitive action related to the user command;

Using the trained evaluation model, score each of the primitive actions in the relevant action set to obtain the first scoring result;

Using the first instruction, instruct the trained LLM to score each of the primitive actions in the relevant action set, and obtain a second scoring result;

The action to be performed is determined based on the first scoring result and the second scoring result.

In one embodiment, before inputting the first instruction to the trained action preliminary screening model, the method further includes:

Obtain positive samples and negative samples, the positive samples include actions selected by the trained LLM for the task sample, and the negative samples include actions not selected by the trained LLM for the task sample;

The positive sample and the negative sample are used to train the action preliminary screening model, and the trained action preliminary screening model is obtained.

In one embodiment, using the first instruction to instruct the trained LLM to score each of the primitive actions in the relevant action set to obtain a second scoring result includes:

Input the first instruction and each of the primitive actions in the related action set to the trained LLM, and obtain the second scoring result output by the trained LLM;

After the agent executes the action to be executed, the following steps are executed in a loop until the user instruction is completed;

The first instruction, the executed action of the agent, and each of the primitive actions in the related action set are input to the trained LLM, and the second scoring result output by the trained LLM is obtained.

In one embodiment, determining the action to be performed based on the first scoring result and the second scoring result includes:

Determine the score of each primitive action in the relevant action set according to the first scoring result and the second scoring result;

Select the primitive action corresponding to the target score, the primitive action corresponding to the target score is the action to be executed, and the target score is among the scores of each primitive action in the relevant action set highest score.

In one embodiment, inputting each primitive action of the first instruction, the executed action of the agent, and the related action set to the trained LLM includes:

Splice the first instruction, the executed action of the agent, and the primitive actions in the related action set to obtain a reconstruction instruction;

The reconstruction instructions are input to the trained LLM.

In one embodiment, using the trained evaluation model to score each of the primitive actions in the relevant action set to obtain the first scoring result includes:

Each primitive action of the relevant action set is input to the trained evaluation model, and a probability value of each primitive action in the relevant action set output by the trained evaluation model is obtained, and the probability value is used for Characterizing the probability of successful execution of the primitive action in the external environment, the first scoring result includes the probability value of each primitive action in the relevant action set.

In a second aspect, embodiments of the present application provide an intelligent control device, including:

Acquisition module, used to obtain user commands and visual information of the external environment;

A target detection module, used to perform target detection on the visual information and obtain target detection information;

A splicing module, used to splice the user command and the target detection information to obtain the first instruction;

The execution module is used to use the first instruction to instruct the trained LLM to select an action to be executed from each primitive action in the primitive action set;

It is also used to send the action to be executed to the intelligent agent, so that the intelligent agent executes the action to be executed.

In a third aspect, embodiments of the present application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program Implement the method described in any one of the above first aspects.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the computer program implements any of the above-described first aspects. method described.

In a fifth aspect, embodiments of the present application provide a computer program product, which when the computer program product is run on an electronic device, causes the electronic device to execute the method described in any one of the above first aspects.

Compared with the prior art, the beneficial effects of the embodiments of the present application are:

The embodiment of the present application acquires user commands and visual information of the external environment; performs target detection on the visual information to obtain target detection information; splices user commands and target detection information to obtain the first instruction; and uses the first instruction to indicate that training has been performed LLM selects the action to be executed from each primitive action in the primitive action set; sends the action to be executed to the agent, so that the agent executes the action to be executed, and adds visual information to the reasoning process of LLM, allowing LLM to interact with the external environment Interact and be able to select the most beneficial decision for the agent based on the environment. +-

It can be understood that the beneficial effects of the above-mentioned second aspect to the fifth aspect can be referred to the relevant description in the above-mentioned first aspect, and will not be described again here.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the purpose of the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a first flow diagram of an intelligent agent control method provided by an embodiment of the present application;

Figure 2 is a second flow diagram of an intelligent agent control method provided by an embodiment of the present application;

Figure 3 is a schematic structural diagram of an intelligent control device provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures and technologies are provided to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and/or components but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or collections thereof.

It will also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. ". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, to mean "once determined" or "in response to a determination" or "once the [described condition or event] is detected ]" or "in response to detection of [the described condition or event]".

In addition, in the description of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms “including,” “includes,” “having,” and variations thereof all mean “including but not limited to,” unless otherwise specifically emphasized.

A Large Language Model (LLM) is an artificial intelligence model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. LLM has good reasoning ability and can act as a "brain", able to decompose high-level language instructions into low-level language instructions that can be executed by the "cerebellum". For example, the high-level language instruction is {I want to eat a sandwich}, and the corresponding decomposed low-level instruction is {1. Pick up the plate, 2. Put down the plate, 3. Pick up lettuce, 4. Put down lettuce,...}.

However, LLM does not have "eyes" and "hands" and cannot interact well with the environment, so it cannot participate in the environmental information in the reasoning process of LLM. This makes LLM lack experience in the real world and cannot provide the best solutions to agents in specific environments. Powerful decision-making.

In this regard, this application provides an intelligent agent control method, which includes obtaining user commands and visual information of the external environment; performing target detection on the visual information to obtain target detection information; splicing user commands and target detection information to obtain the first Instruction; use the first instruction to instruct the trained LLM to select an action to be executed from each primitive action in the primitive action set; send the action to be executed to the agent, so that the agent executes the action to be executed, thereby adding visual information to the LLM During the reasoning process, LLM interacts with the external environment and can select the most beneficial decision for the agent based on the environment.

Figure 1 is a first flow diagram of an intelligent agent control method provided by an embodiment of the present application. As shown in Figure 1, the method includes:

S11: Obtain user commands and visual information of the external environment.

In the application, when the user needs the intelligent agent to provide services, user commands are input to the electronic device. User commands can be input by voice or in text form.

Electronic devices collect visual information of the external environment through sensors or camera devices. Among them, the external environment is the environment where the target operated by the agent is located. Visual information can be captured images and videos, etc.

S12: Perform target detection on visual information to obtain target detection information.

Among them, the target detection information includes the detection frame of the object and the location of the object.

In the example, the visual information is an image. YoloV5 (You Only Look Once, single-stage target detection algorithm) is used to detect targets in the image through attention and convolution, and the target detection information is output in the form of text. The target detection information is expressed as {apple (x1, y1), banana (x2, y2), plate (x3, y3)}.

S13: Splice the user command and target detection information to obtain the first instruction.

For example, the user command is {Make me a salad}, the target detection information is {Apple (x1, y1), Banana (x2, y2), Plate (x3, y3)}, and the first command after splicing is {Give me Make a salad. The following are the positions of the objects in the image, apple (x1, y1), banana (x2, y2), plate (x3, y3)}.

S14: Use the first instruction to instruct the trained LLM to select an action to be executed from each primitive action in the primitive action set.

In a possible implementation, the input format of the LLM is {user command + target detection information + primitive action in the primitive action set}, and the first instruction is {user command + target detection information}.

Example, based on the input format of LLM, the input of LLM is {Make me a salad, the following is the position of the object in the image, apple (x1, y1), banana (x2, y2), plate (x3, y3), Take the plate, ..., take the lettuce, put the plate down}.

In the application, in the electronic device, the LLM selects an action to be executed from each primitive action in the input primitive action set based on the first instruction.

S15: Send the action to be executed to the agent, so that the agent can execute the action to be executed.

In applications, agents are intelligent devices that can perform actions, such as robotic arms, robots, etc.

It is understandable that the visual information of the environment is first processed to obtain target detection information in the form of text description, and then the target detection information is added to the user command to obtain the first instruction. Then LLM generates the most executable instruction in the current environment based on the first instruction to complete the user command. This eliminates the need for LLM to perform visual language alignment and does not damage LLM's reasoning capabilities.

Embodiments of the present application include obtaining user commands and visual information of the external environment; performing target detection on the visual information to obtain target detection information; splicing user commands and target detection information to obtain the first instruction; using the first instruction to indicate that training has been performed LLM selects the action to be executed from each primitive action in the primitive action set; sends the action to be executed to the agent, so that the agent executes the action to be executed, and adds visual information to the reasoning process of LLM, allowing LLM to communicate with the external environment Interact and be able to select the most beneficial decision for the agent based on the environment.

Figure 2 is a second schematic flowchart of an intelligent agent control method provided by an embodiment of the present application. As shown in Figure 2, the method also includes:

S21: Input the first instruction to the trained action preliminary screening model, and obtain the relevant action set output by the trained action preliminary screening model.

Among them, the relevant action set includes primitive actions related to user commands.

In the application, in the electronic device, the trained action preliminary screening model selects the primitive actions that may occur during the execution of the user command from all primitive actions according to the first instruction, and excludes irrelevant primitive actions. Use the trained action preliminary screening model to filter out primitive actions related to user commands, reduce the impact of irrelevant primitive actions on the inference of the trained LLM, and reduce the selection of irrelevant primitives by the trained LLM due to certain factors. action.

In one possible implementation, the primitive actions screened by the LLM model and the excluded primitive actions are used as samples to train the action preliminary screening model.

Before step S21, it also includes:

S31: Obtain positive samples and negative samples. Positive samples include actions selected by the trained LLM for task samples, and negative samples include actions not selected by the trained LLM for task samples.

In the application, the trained LLM model is used in advance to screen the primitive actions that occur during the execution of the task sample, and the primitive actions screened by the trained LLM model are used as positive samples and the primitive actions selected by the trained LLM model are used as negative samples.

S32: Use positive samples and negative samples to train the action preliminary screening model and obtain the trained action preliminary screening model.

In the application, when the loss value of the model is less than the preset loss value, the preliminary screening model of the trained actions is obtained.

S22: Use the trained evaluation model to score each primitive action in the relevant action set, and obtain the first scoring result.

In a possible implementation, step S22 includes:

Input each primitive action of the relevant action set into the trained evaluation model, and obtain the probability value of each primitive action in the relevant action set output by the trained evaluation model. The probability value is used to represent the probability of successful execution of the primitive action in the external environment. , the first scoring result includes the probability value of each primitive action in the relevant action set.

For example, the trained evaluation model is a model trained using reinforcement learning. Specifically, primitive actions are evaluated under a given environment. If the primitive action is successfully executed in this environment, the evaluation model will be rewarded, otherwise it will be punished. When the loss value of the evaluation model is less than the preset loss value, the trained evaluation model is obtained.

S23: Use the first instruction to instruct the trained LLM to score each primitive action in the relevant action set, and obtain the second scoring result.

The score of the primitive action in the second scoring result indicates how beneficial the primitive action is to the execution of the user command.

In a possible implementation, step S23 includes:

S231: Input the first instruction and each primitive action in the related action set to the trained LLM, and obtain the second scoring result output by the trained LLM.

Among them, the input format of LLM is {user command + target detection information + related action set primitive action}, and the first instruction is {user command + target detection information}.

In the application, the trained LLM scores each primitive action in the relevant action set according to the first instruction, and obtains the second scoring result.

S232: After the agent executes the action to be executed, the following steps are executed in a loop until the user command is completed.

S233: Input the first instruction, the executed action of the agent, and each primitive action in the set of related actions to the trained LLM, and obtain the second scoring result output by the trained LLM.

In the application, if the agent performs an action and then completes the user command, steps S232 and S233 will not be executed. If the agent fails to complete the user command after performing an action, steps S232 and S233 need to be performed.

Specifically, the first instruction, the executed actions of the agent, and each primitive action in the set of related actions are input to the trained LLM, including:

S41: Splice the primitive actions of the first instruction, the executed action of the agent, and the related action set to obtain the reconstruction instruction.

Among them, the reconstruction instruction is expressed as {user command + target detection information + 1. The action has been performed by the agent + 2. The action has been performed by the agent +... + the primitive action in the relevant action set}, and the first instruction is {user command + target Detection information}.

S42: Input the reconstruction instructions to the trained LLM.

In the application, the reconstruction instruction is input to the trained LLM, and the trained LLM scores each primitive action in the relevant action set based on the reconstruction instruction and the executed action, and obtains the second scoring result.

S24: Determine the action to be executed based on the first scoring result and the second scoring result.

In a possible implementation, step S24 includes:

S241: Determine the score of each primitive action in the relevant action set based on the first scoring result and the second scoring result.

S242: Select the primitive action corresponding to the target score.

Among them, the primitive action corresponding to the target score is the action to be executed, and the target score is the highest score among the scores of each primitive action in the relevant action set.

In the application, after the integrated model receives the first scoring result and the second scoring result, it uses the hyperparameters of the model to integrate the first scoring result and the second scoring result, and then selects the primitive action corresponding to the target score. Among them, the hyperparameters can be set manually or trained.

For example, the user command to obtain is {I want to eat a sandwich}. The image is processed to obtain target detection information {ham (x1, y1), plate (x2, y2), lettuce (x3, y3), fried egg (x4, y4), human (x5, y5)}. Splicing user commands and target detection information, the first instruction obtained is {I want to eat a sandwich, the following is the position of the object in the image, ham (x1, y1), plate (x2, y2), lettuce (x3, y3), fried Egg(x4,y4), human(x5,y5)}.

Assume that there are 1,000 primitive actions. The trained action preliminary screening model receives the input primitive actions, marks each primitive action with a probability of occurring during the execution of the user command, and obtains relevant actions by selecting primitive actions whose probability is greater than the threshold. set.

By splicing the primitive actions of the first instruction and related action sets, we obtain {I want to eat a sandwich, the following is the position of the object in the image, ham (x1, y1), plate (x2, y2), lettuce (x3, y3), Omelette (x4, y4), human (x5, y5), please rate the following action primitives [get plate, get sandwich, get ham, get bowl,...]}.

Input the above spliced instructions into the trained evaluation model to obtain the first scoring result output by the trained evaluation model. An example of the scoring process is as follows: In the current environment, the evaluation model has been trained to judge that if fetching the plate is performed, a collision may occur or the robot arm cannot reach the position of the plate, but it can fetch the bowl. The trained evaluation model scores the fetching of the bowl better than fetching the plate. The score is high, that is, the probability value of taking the bowl is higher than the probability value of taking the plate.

Input the above spliced instructions into the trained LLM to obtain the second scoring result output by the trained LLM. An example of the scoring process is as follows: In the current environment, the trained LLM gives a low score to taking a sandwich because there is no sandwich in the environment, a high score to taking a plate, and a lower score to taking a bowl than taking a plate.

Combining the first scoring result and the second scoring result, it is determined that the action to be performed is to take the bowl. Send the action to be executed to the agent to get the bowl.

Since there is no sandwich in the environment, multiple actions need to be executed to complete the user command, so the steps of selecting the action to be executed are repeated until the user command is completed:

Splicing the first instruction, the executed action of the agent and the primitive actions of the related action set, we obtain {I want to eat a sandwich, the following is the position of the object in the image, ham (x1, y1), plate (x2, y2), Lettuce (x3, y3), fried egg (x4, y4), human (x5, y5), executed 1. Get the bowl, please rate the following action primitives [get the plate, get the sandwich, get the ham, get the bowl, …]}. Input it into the trained LLM to obtain the second scoring result output by the trained LLM. Among them, Putting Down the Bowl received the highest rating.

According to the second scoring result and the first scoring result, it is determined that the action to be performed is to put down the bowl.

Splicing the first instruction, the executed action of the agent and the primitive actions of the related action set, we obtain {I want to eat a sandwich, the following is the position of the object in the image, ham (x1, y1), plate (x2, y2), Lettuce (x3, y3), fried egg (x4, y4), human (x5, y5), executed 1. Take the bowl, 2. Put down the bowl, please rate the following action primitives [take the plate, take the sandwich, take Ham, get the bowl,…]}. Input it into the trained LLM to obtain the second scoring result output by the trained LLM. Among them, lettuce has the highest score.

According to the second scoring result and the first scoring result, it is determined that the action to be performed is to get lettuce. And so on until the task is successful.

In the embodiment of the present application, the first instruction is input into the trained action preliminary screening model to obtain the relevant action set output by the trained action preliminary screening model. The relevant action set includes primitive actions related to user commands, reducing irrelevant primitives. Actions have an impact on the inference of the trained LLM, reducing the probability that the trained LLM selects irrelevant primitive actions due to certain factors.

and by using the trained evaluation model to score each primitive action in the relevant action set to obtain the first scoring result; using the first instruction to instruct the trained LLM to score each primitive action in the relevant action set to obtain the second scoring result ; Determine the action to be executed based on the first scoring result and the second scoring result, and determine the action to be executed by combining the two scoring results, so that the action to be executed can be accurately selected.

It should be understood that the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

Corresponding to the methods described in the above embodiments, for convenience of explanation, only the parts related to the embodiments of the present application are shown.

Figure 3 is a schematic structural diagram of an intelligent agent control device provided by an embodiment of the present application. As shown in Figure 3, the device includes:

The acquisition module 10 is used to acquire user commands and visual information of the external environment;

The target detection module 11 is used to perform target detection on visual information and obtain target detection information;

The splicing module 12 is used to splice the user command and the target detection information to obtain the first instruction;

The execution module 13 is configured to use the first instruction to instruct the trained LLM to select an action to be executed from each primitive action in the primitive action set;

It is also used to send actions to be executed to the agent so that the agent can execute the action to be executed.

In one embodiment, the device further includes:

The action preliminary screening module is used to input the first instruction to the trained action preliminary screening model and obtain the relevant action set output by the trained action preliminary screening model. The relevant action set includes primitive actions related to the user command.

The evaluation module is used to use the trained evaluation model to score each primitive action in the relevant action set and obtain the first scoring result.

The execution module is also used to use the first instruction to instruct the trained LLM to score each primitive action in the relevant action set to obtain the second scoring result; and determine the action to be executed based on the first scoring result and the second scoring result.

In one embodiment, the device further includes:

The training module is used to obtain positive samples and negative samples. Positive samples include actions selected by the trained LLM for task samples, and negative samples include actions not selected by the trained LLM for task samples. Using positive samples and negative samples, training actions are initially screened. model to obtain a preliminary screening model of trained actions.

In one embodiment, the evaluation module is specifically configured to input each primitive action of the relevant action set to the trained evaluation model, and obtain the probability value of each primitive action in the relevant action set output by the trained evaluation model. The probability value is used for It represents the probability of successful execution of primitive actions in the external environment. The first scoring result includes the probability value of each primitive action in the relevant action set.

In one embodiment, the execution module is specifically configured to input the first instruction and each primitive action in the related action set to the trained LLM, and obtain the second scoring result output by the trained LLM; after the agent executes the action to be executed, The following steps are executed in a loop until the user instruction is completed; input the first instruction, the executed actions of the agent, and each primitive action in the set of related actions into the trained LLM to obtain the second scoring result output by the trained LLM.

In one embodiment, the execution module is specifically configured to determine the score of each primitive action in the relevant action set based on the first scoring result and the second scoring result; select the primitive action corresponding to the target score, and select the primitive action corresponding to the target score. For an action to be executed, the target score is the highest score among the scores of each primitive action in the relevant action set.

In one embodiment, the execution module is specifically configured to splice the first instruction, the executed action of the agent, and the primitive actions in the related action set to obtain the reconstruction instruction; and input the reconstruction instruction into the trained LLM.

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 4 , the electronic device 2 of this embodiment includes: at least one processor 20 (only one is shown in FIG. 4 ), a memory 21 and data stored in the memory 21 and available in the at least one processor 20 The computer program 22 running on the processor 20 implements the steps in any of the above method embodiments when the processor 20 executes the computer program 22 .

The electronic device 2 may be a computing device such as a desktop computer or a cloud server. The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21 . Those skilled in the art can understand that FIG. 4 is only an example of the electronic device 2 and does not constitute a limitation on the electronic device 2. It may include more or fewer components than shown in the figure, or some components may be combined, or different components may be used. , for example, it may also include input and output devices, network access devices, etc.

The so-called processor 20 may be a central processing unit (Central Processing Unit, CPU). The processor 20 may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit). , ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

The memory 21 may be an internal storage unit of the electronic device 2 in some embodiments, such as a hard disk or memory of the electronic device 2 . In other embodiments, the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk, a smart memory card (SMC), or a secure digital card equipped on the electronic device 2. (Secure Digital, SD) card, Flash Card, etc. Further, the memory 21 may also include both an internal storage unit of the electronic device 2 and an external storage device. The memory 21 is used to store operating systems, application programs, boot loaders, data and other programs, such as program codes of the computer programs. The memory 21 can also be used to temporarily store data that has been output or is to be output.

It should be noted that the information interaction, execution process, etc. between the above-mentioned devices/units are based on the same concept as the method embodiments of the present application. For details of their specific functions and technical effects, please refer to the method embodiments section. No further details will be given.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional units and modules is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs. Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be hardware-based. It can also be implemented in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present application. For the specific working processes of the units and modules in the above system, please refer to the corresponding processes in the foregoing method embodiments, and will not be described again here.

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps in each of the above method embodiments can be implemented.

Embodiments of the present application provide a computer program product. When the computer program product is run on an electronic device, the steps in each of the above method embodiments can be implemented when the electronic device is executed.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, this application can implement all or part of the processes in the methods of the above embodiments by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program When executed by a processor, the steps of each of the above method embodiments may be implemented. Wherein, the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code to the camera device/terminal device, recording media, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electrical carrier signals, telecommunications signals, and software distribution media. For example, U disk, mobile hard disk, magnetic disk or CD, etc. In some jurisdictions, subject to legislation and patent practice, computer-readable media may not be electrical carrier signals and telecommunications signals.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or documented in a certain embodiment, please refer to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed devices/network devices and methods can be implemented in other ways. For example, the apparatus/network equipment embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components can be combined or can be integrated into another system, or some features can be omitted, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above-described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still implement the above-mentioned implementations. The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of this application, and should be included in within the protection scope of this application.

Claims (10)

1. An agent control method, comprising:

acquiring a user command and visual information of an external environment;

performing target detection on the visual information to obtain target detection information;

splicing the user command and the target detection information to obtain a first instruction;

using a first instruction to instruct the trained LLM to select an action to be executed from all primitive actions in the primitive action set;

and sending the action to be executed to the intelligent agent so that the intelligent agent executes the action to be executed.

2. The method as recited in claim 1, further comprising:

inputting the first instruction into a trained action prescreening model to obtain a related action set output by the trained action prescreening model, wherein the related action set comprises the primitive actions related to the user command;

scoring each primitive action in the related action set by using a trained evaluation model to obtain a first scoring result;

using a first instruction to instruct the trained LLM to score each primitive action in the related action set, and obtaining a second scoring result;

and determining the action to be executed according to the first scoring result and the second scoring result.

3. The method of claim 2, wherein before inputting the first instruction into the trained motion screening model, further comprising:

obtaining a positive sample and a negative sample, wherein the positive sample comprises actions selected by the trained LLM for a task sample, and the negative sample comprises actions not selected by the trained LLM for the task sample;

and training an action preliminary screening model by utilizing the positive sample and the negative sample to obtain the trained action preliminary screening model.

4. The method of claim 2, wherein the instructing the trained LLM to score each of the primitive actions in the set of related actions with a first instruction to obtain a second scoring result comprises:

inputting the first instruction and each primitive action in the related action set to the trained LLM to obtain a second scoring result output by the trained LLM;

after the intelligent agent executes the action to be executed, the following steps are executed in a circulating way until the user instruction is completed;

and inputting the first instruction, the executed actions of the intelligent agent and the primitive actions in the related action set to the trained LLM to obtain the second scoring result output by the trained LLM.

5. The method according to any of claims 2 to 4, wherein said determining said action to be performed based on said first scoring result and said second scoring result comprises:

determining the score of each primitive action in the related action set according to the first scoring result and the second scoring result;

and selecting the primitive actions corresponding to the target scores, wherein the primitive actions corresponding to the target scores are the actions to be executed, and the target scores are the highest scores among the scores of the primitive actions in the related action set.

6. The method of claim 4, wherein inputting the first instruction, the executed actions of the agent, and the respective primitive actions in the set of related actions into the trained LLM comprises:

splicing the first instruction, the executed actions of the intelligent agent and the primitive actions in the relevant action set to obtain a reconstruction instruction;

and inputting the reconfiguration instruction to the trained LLM.

7. The method of claim 2, wherein scoring each of the primitive actions in the set of related actions using a trained evaluation model to obtain a first scoring result comprises:

inputting each primitive action of the related action set into the trained evaluation model to obtain a probability value of each primitive action in the related action set output by the trained evaluation model, wherein the probability value is used for representing the probability of successful execution of the primitive action in the external environment, and the first scoring result comprises the probability value of each primitive action in the related action set.

8. An agent control device, comprising:

the acquisition module is used for acquiring a user command and visual information of an external environment;

the target detection module is used for carrying out target detection on the visual information to obtain target detection information;

the splicing module is used for splicing the user command and the target detection information to obtain a first instruction;

the execution module is used for indicating the trained LLM to select actions to be executed from all primitive actions in the primitive action set by using the first instruction;

and the method is also used for sending the action to be executed to the intelligent agent so that the intelligent agent executes the action to be executed.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method according to any one of claims 1 to 7.