CN111267086A

CN111267086A - Action task creating and executing method and device, equipment and storage medium

Info

Publication number: CN111267086A
Application number: CN201811474344.0A
Authority: CN
Inventors: 杜坤
Original assignee: Beijing Orion Star Technology Co Ltd
Current assignee: Beijing Orion Star Technology Co Ltd
Priority date: 2018-12-04
Filing date: 2018-12-04
Publication date: 2020-06-12

Abstract

The application provides a method, a device, equipment and a storage medium for creating and executing an action task, wherein the method for creating the action task comprises the following steps: receiving an adding instruction of an action unit required by an action task to be created; the type of the action unit comprises at least one of an action atom and an action molecule; generating the action units according to the adding instruction to obtain a sixth action unit sequence; and establishing a corresponding relation between the name identifier of the action task to be created and the sixth action unit sequence, and creating the action task to be created according to the name identifier. Compared with the prior art, the action task is created, so that the robot is controlled to move to complete corresponding actions, the difficulty of controlling the robot to execute the task by a user can be reduced, and the interaction capacity and flexibility of the robot are improved.

Description

Action task creating and executing method and device, equipment and storage medium

Technical Field

The present application relates to the field of robot control technologies, and in particular, to a method, an apparatus, a device, and a storage medium for creating and executing an action task.

Background

Over the past half century or more, many researchers in the world have worked on the relevant techniques for robotic applications, with much success. Such as an industrial robot, or a robot that performs a specific or dangerous task, etc. These robots typically operate in a particular mode in a structural environment. A robot arm is an example of a very typical application of an industrial robot. With the development of technology and the demand of human daily life, robots are challenged by a series of problems such as unstructured and complicated. In order to enable the mechanical arm to better assist human labor and serve people in daily life, the mechanical arm has more flexible and intelligent interaction capacity.

In the prior art, for the control of the mechanical arm, one mode is to control the mechanical arm through complex programming, and the threshold for common users is high; one method is to set various parameters on the teaching board and control the mechanical arm by matching with manual dragging, and the method is not flexible enough, cannot complete more complex task actions and has high operation difficulty.

Disclosure of Invention

In view of this, embodiments of the present application provide a method, an apparatus, a device, and a storage medium for creating and executing an action task, so as to solve technical defects in the prior art.

The embodiment of the application discloses a method for creating an action task, which comprises the following steps:

receiving an adding instruction of an action unit required by an action task to be created; the type of the action unit comprises at least one of an action atom and an action molecule;

generating the action units according to the adding instruction to obtain a sixth action unit sequence;

and establishing a corresponding relation between the name identifier of the action task to be created and the sixth action unit sequence, and creating the action task to be created according to the name identifier.

In an exemplary embodiment of the present application, creating the action task to be created according to the name identifier includes:

and creating the action task to be created according to the name identification and the default type value.

In an exemplary embodiment of the present application, generating the action unit according to the add instruction includes: acquiring an action unit template of the action unit according to the adding instruction;

and generating the action unit according to the action unit template and the parameter value of the action unit.

In an exemplary embodiment of the present application, if the acting unit is an acting atom,

the action atom template of the action atom comprises a type value, and the type value of the action atom template corresponds to at least one execution function and execution logic of the at least one execution function;

the parameter value of the action atom is used for determining an execution function to be executed from the at least one execution function;

generating the action unit according to the action unit template and the parameter value of the action unit, comprising:

and generating the action atom according to the action atom template and the parameter value of the action atom.

In an exemplary embodiment of the present application, before generating the action atom according to the action atom template and the parameter value of the action atom, the method further includes:

acquiring a parameter value of the action atom based on user operation; or

And acquiring the parameter value of the action atom based on a preset acquisition mode.

In an exemplary embodiment of the present application, generating the action atom according to the action atom template and the parameter value of the action atom includes:

and generating the action atom according to the action atom template, the parameter value of the action atom and the name identification of the action atom.

In an exemplary embodiment of the present application, before generating the action atom according to the action atom template, the parameter value of the action atom, and the name identification of the action atom, the method further includes: and acquiring the name identification of the action atom based on user operation.

In an exemplary embodiment of the present application, the action atom template includes a default name identifier, and the name identifier of the action atom is the default name identifier.

In an exemplary embodiment of the present application, if the action unit is an action molecule,

the action molecule template of the action molecule comprises a name identifier, and the name identifier of the action molecule template corresponds to the seventh action unit sequence;

the parameter value of the action numerator is used for determining a seventh action unit sequence to be executed from the seventh action unit sequence;

generating the action unit according to the action unit template of the action unit and the parameter value of the action unit, comprising:

and generating the action molecule according to the action molecule template and the parameter value of the action molecule.

In an exemplary embodiment of the present application, before generating the action molecule according to the action molecule template and the parameter value of the action molecule, the method further includes:

acquiring a parameter value of the action molecule based on user operation; or

And acquiring the parameter value of the action molecule based on a preset acquisition mode.

In an exemplary embodiment of the present application, generating the action molecule based on the action molecule template and a parameter value of the action molecule comprises:

generating the action molecule according to the action molecule template, the parameter value of the action molecule and the type value of the action molecule.

In an exemplary embodiment of the present application, the action molecule template includes a default type value, and the type value of the action molecule to be generated is the default type value.

In an exemplary embodiment of the present application, the parameter values of the action molecule comprise a start point and an end point.

In an exemplary embodiment of the present application, the parameter value of the action molecule further comprises an execution mode; wherein the execution mode indicates an acquisition manner of an execution offset corresponding to the action molecule.

The embodiment of the application discloses an execution method of an action task, which comprises the following steps:

analyzing the action task to be executed, and acquiring a name identifier of the action task to be executed; the name identification of the action task to be executed corresponds to an eighth action unit sequence to be executed; the type of the action unit comprises at least one of an action atom and an action molecule;

and sequentially executing each action unit in the eighth action unit sequence to be executed, and generating at least one group of robot action control parameters.

In an exemplary embodiment of the present application, before obtaining the name identifier of the action task to be performed, the method further includes:

acquiring a type value of the action task to be executed;

and according to the type value of the action task to be executed, confirming that the type of the action task to be executed is the action task.

In an exemplary embodiment of the present application, sequentially executing each action unit in the eighth action unit sequence to be executed, and generating at least one set of robot action control parameters includes:

analyzing the action atom aiming at each action atom in the eighth action unit sequence to be executed, and acquiring the type value of the action atom and the parameter value of the action atom; the type value of the action atom corresponds to at least one execution function and execution logic of the at least one execution function;

determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom and the execution logic of the at least one execution function;

and generating at least one group of robot action control parameters according to the execution function to be executed.

In an exemplary embodiment of the present application, before determining, according to the parameter value of the action atom and the execution logic of the at least one execution function, an execution function to be executed from the at least one execution function, the method further includes:

acquiring sensor data;

determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom and the execution logic of the at least one execution function, including:

determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom, the execution logic of the at least one execution function, and the sensor data.

analyzing the action molecule aiming at each action molecule in the eighth action unit sequence to be executed, and acquiring the name identification of the action molecule and the parameter value of the action molecule; the name identification of the action molecule corresponds to a ninth action unit sequence;

determining a ninth action unit sequence to be executed from the ninth action unit sequence according to the parameter value of the action molecule;

and sequentially executing each action unit in the ninth action unit sequence to be executed to generate at least one group of robot action control parameters.

In an exemplary embodiment of the present application, before obtaining the name identifier of the action molecule and the parameter value of the action molecule, the method further includes:

obtaining a type value of the action molecule;

and confirming that the type of the action molecule is the action molecule according to the type value of the action molecule.

In an illustrative embodiment of the present application, the parameter values of the action molecule comprise a start point and an end point;

determining a ninth sequence of action units to be executed from the ninth sequence of action units according to the parameter values of the action numerator, comprising:

and determining a ninth action unit sequence to be executed from the ninth action unit sequence according to the starting point and the end point in the parameter values of the action molecules.

In an exemplary embodiment of the present application, the parameter value of the action molecule further comprises an execution mode; the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule;

sequentially executing each action unit in the ninth action unit sequence to be executed, and generating at least one group of robot action control parameters, wherein the method comprises the following steps:

sequentially executing each action unit in the ninth action unit sequence to be executed to generate at least one group of robot action control parameters to be adjusted;

and adjusting the at least one group of robot motion control parameters to be adjusted according to the execution offset to obtain the at least one group of robot motion control parameters.

In an exemplary embodiment of the present application, when the execution mode is an adaptive mode, the execution offset corresponding to the action molecule is obtained as follows:

and acquiring the execution offset corresponding to the action molecule which is determined in advance.

In an exemplary embodiment of the present application, when the execution mode is a multiplexing mode, the execution offset corresponding to the action molecule is obtained as follows:

acquiring a current pose value of the robot;

acquiring a pose value in the robot action control parameter to be adjusted generated by executing the first action unit in the ninth action unit sequence to be executed;

and generating an execution offset corresponding to the action molecule according to the current pose value of the robot and the pose value in the robot action control parameter to be adjusted generated by executing the first action unit.

In an exemplary embodiment of the present application, sequentially executing each action unit in the ninth action unit sequence to be executed to generate at least one set of robot motion control parameters to be adjusted, includes:

analyzing each action atom in the ninth action unit sequence to be executed to obtain a type value of the action atom and a parameter value of the action atom; the type value of the action atom corresponds to at least one execution function and execution logic of the at least one execution function;

and generating at least one group of robot motion control parameters to be adjusted according to the execution function to be executed.

analyzing the action molecule aiming at each action molecule in the ninth action unit sequence to be executed, and acquiring the name identification of the action molecule and the parameter value of the action molecule; the name identification of the action molecule corresponds to a tenth action unit sequence;

determining a tenth action unit sequence to be executed from the tenth action unit sequence according to the parameter value of the action molecule;

and sequentially executing each action unit in the tenth action unit sequence to be executed to generate at least one group of robot action control parameters to be adjusted.

In an exemplary embodiment of the present application, the robot motion control parameters include: at least one of robot arm pose control parameters and robot arm end-of-arm tool control parameters.

The embodiment of the application discloses a device for creating an action task, which comprises:

the task instruction receiving module is used for receiving an adding instruction of an action unit required by an action task to be created; the type of the action unit comprises at least one of an action atom and an action molecule;

the task instruction execution module is used for generating the action units according to the adding instruction to obtain a sixth action unit sequence;

and the task creating module is used for creating a corresponding relation between the name identifier of the action task to be created and the sixth action unit sequence and creating the action task to be created according to the name identifier.

The embodiment of the application discloses an actuating device of action molecules, the device includes:

the task analysis module is used for analyzing the action task to be executed and acquiring the name identifier of the action task to be executed; the name identification of the action task to be executed corresponds to an eighth action unit sequence to be executed; the type of the action unit comprises at least one of an action atom and an action molecule;

and the task execution module is used for sequentially executing each action unit in the eighth action unit sequence to be executed and generating at least one group of robot action control parameters.

The embodiment of the application discloses an electronic device, which comprises a memory, a processor and computer instructions stored on the memory and capable of running on the processor, and is characterized in that the processor implements the creating method of the action task when executing the instructions.

The embodiment of the application discloses a master controller, which comprises a memory, a processor and computer instructions stored on the memory and capable of running on the processor, and is characterized in that the execution method of the action task is realized when the processor executes the instructions.

The embodiment of the application discloses a robot, which comprises the main controller.

The embodiment of the application discloses a nonvolatile computer storage medium, wherein the computer storage medium stores computer instructions, and the computer instructions are executed by a processor to realize the creating method of the action task.

The embodiment of the application discloses a nonvolatile computer storage medium, wherein the computer storage medium stores computer instructions, and the computer instructions are executed by a processor to realize the execution method of the action task.

The action task creating and executing method, device, equipment and storage medium enable development of a mechanical arm control system to be highly extensible. And performing motion control on the robot based on the generated robot motion control parameters so that the robot completes the motion corresponding to the motion task, thereby controlling the robot to move and complete the corresponding motion. By implementing the scheme provided by the application, the difficulty of controlling the robot to execute the task by the user can be reduced, and the interaction capacity and flexibility of the robot are improved.

Drawings

FIG. 1 is a flow chart of a method for generating an action atom according to an embodiment of the present application;

FIG. 2 is a flow chart of another method for generating an action atom according to an embodiment of the present application;

FIG. 3 is a flow chart of another method for generating an action atom according to an embodiment of the present application;

2-1 to 2-4 are schematic GUI interfaces for acquiring an action atom template based on a user operation in the action atom generation method according to the embodiment of the present application;

3-1 to 3-3 are schematic GUI interfaces for acquiring parameter values of an action atom to be generated based on a user operation in the action atom generation method according to the embodiment of the present application;

4-1 and 4-2 are schematic diagrams of GUI interfaces for modifying the name identifier of an action atom in the action atom generation method according to the embodiment of the present application;

FIG. 5 is a flow chart illustrating a method for executing an action atom according to an embodiment of the present application;

FIG. 6 is a flow chart illustrating another method for executing an action atom according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an apparatus for generating action atoms according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an apparatus for performing an action atom according to an embodiment of the present application;

FIG. 9 is a flow chart of a method for creating an action molecule template according to an embodiment of the present disclosure;

FIG. 10 is a flow chart diagram illustrating another method for creating an action molecule template according to an embodiment of the present application;

FIG. 11 is a flow chart diagram illustrating another method for creating an action molecule template according to an embodiment of the present application;

FIG. 12 is a flow chart illustrating another method for creating an action molecule template according to an embodiment of the present application;

FIG. 13 is a schematic flow chart diagram illustrating another method for creating an action molecule template according to an embodiment of the present application;

FIG. 14 is a flow chart diagram of another method for creating an action molecule template according to an embodiment of the present application;

FIG. 15 is a schematic diagram of an apparatus for creating an action molecule template according to an embodiment of the present application;

FIG. 16 is a schematic flow chart diagram illustrating a method for generating an action molecule according to an embodiment of the present disclosure;

FIG. 17 is a schematic flow chart diagram illustrating a method for performing an action molecule in accordance with an embodiment of the present application;

FIG. 18 is a schematic flow chart diagram of another method of performing an action molecule in accordance with an embodiment of the present application;

FIG. 19 is a schematic flow chart diagram of another method of performing an action molecule in accordance with an embodiment of the present application;

FIG. 20 is a schematic flow chart diagram illustrating another method for performing an action molecule in accordance with an embodiment of the present application;

FIG. 21 is a schematic view of an apparatus for generating an action molecule according to an embodiment of the present application;

FIG. 22 is a schematic diagram of an actuator for an action molecule according to an embodiment of the present application;

FIG. 23 is a flowchart illustrating a method for creating an action task according to an embodiment of the present application;

FIG. 24 is a flow chart diagram of another method for creating an action task according to an embodiment of the present application;

FIG. 25 is a flow chart illustrating a method for performing an action task according to an embodiment of the present application;

FIG. 26 is a flow chart illustrating another method for performing an action task according to an embodiment of the present application;

FIG. 27 is a flow chart diagram illustrating another method for performing an action task according to an embodiment of the present application;

FIG. 28 is a flow chart illustrating another method for performing an action task according to an embodiment of the present application;

fig. 29 is a schematic diagram of a creating apparatus of an action task according to an embodiment of the present application;

FIG. 30 is a schematic diagram of an apparatus for performing an action task according to an embodiment of the present application;

fig. 31 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 32 is a schematic structural diagram of a master according to an embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

In order to solve the problems in the prior art, embodiments of the present application provide a method, an apparatus, a device, and a storage medium for creating and executing an action task, which are used to perform action control on a robot.

First, a robot in the embodiment of the present application will be described, and the robot in the present application refers to an electronic device capable of operating, and specifically, may be an intelligent robot having a function of acquiring environmental information, performing continuous trajectory motion or performing servo control from point-to-point trajectory motion, and the like, and includes, but is not limited to, a robot arm.

The robot, as a complete system, may include various components:

the robot main body is composed of various structural components. Such as links, bar knuckles, moving chassis, etc.

Drivers, among robots, common drivers include servo motors, stepping motors, air cylinders, hydraulic cylinders, etc., and servo motors are the most commonly used robot drivers. The driver is controlled by a controller, which transmits a control signal to the driver, which controls the movement of the actuator.

The sensor is used for collecting internal state information and external environment information of the robot or interacting with the external environment and the like. Wherein sensors integrated within the robot send information for each actuator to the controller to facilitate the controller in determining the current configuration state of the robot. In addition, the robot may be provided with an external sensor, such as a vision system, a touch sensor, a range finder, an olfactory sensor, a taste sensor, a voice recognition device, a voice synthesizer, etc., so that the robot can acquire external environment information.

And the controller acquires data from the main controller, controls the action of the driver by utilizing the information of the sensor and coordinates the robot to move.

The main controller is used for calculating the motion of each execution component, determining how each execution component can move to reach a preset speed and position, acquiring the current configuration state information of the robot through the controller, and supervising the coordination action of the sensor and the controller. When the master controller is an external device for controlling the robot, it may be a computer, which includes an operating system, an application program, and an electronic screen. In the robot system, a master controller realizes the operation of the robot by sending control parameter data to a robot controller and analyzing, processing, displaying and the like functions of sensor data.

In some robot systems, the master controller and the controller may be integrated into one unit, and in some systems, they may be separated, which is not limited in the embodiments of the present application.

In the following embodiments of the present application, a robot arm having a complicated motion control will be described in detail as an example.

As described above, the constituent elements of the robot system of the arm also include a main body part, a driver, a sensor, a controller, and a master controller. Wherein, the main body part of the mechanical arm consists of a movable joint part and other structural parts. The joints of the mechanical arm are execution parts comprising drivers, the mechanical arm robot needs to have multiple joints to realize multi-degree-of-freedom motion to complete complex actions, and the motion of each joint depends on a respective driver, such as a servo motor. The controller of the robot can control the mechanical device of each joint to move in freedom degree through each joint driver.

The robot arm includes, in addition to the above-described components, an end tool component connected to the last joint of the robot arm.

The end tools move in the space by relying on the mechanical arm and perform physical interaction with the surrounding space, wherein the end tools comprise but are not limited to cameras, mechanical claws, calligraphy pens, cutting tools and the like, and the achievable physical interaction comprises but is not limited to sensing images and light rays and emitting light waves such as infrared rays, grabbing and moving objects, writing and cutting objects and the like. In the embodiment of the present application, a specific implementation form of the end tool is not limited, and different end tools may be configured based on different application scenarios.

It will be appreciated that when the end tool comprises an actuator, the action may be controlled directly by the controller of the robotic arm or by the control means of the end tool itself (e.g. a programmable logic controller), in which case the control signal is transmitted from the robot controller to the control means of the end tool itself. The robot controller may thus control the end-tool directly or indirectly, in the following embodiments the movable end-tool is considered as an execution component like all joints in the robot arm, i.e. the master controller sends control parameter data to the robot arm controller, which controls the actuators of the end-tool to execute commands for controlling the parameters, whereby the end-tool of the robot arm performs the corresponding movement actions.

It should be noted that, the components included in the robot are described as examples, and are not intended to limit the robot in the present application, and any electronic device capable of acting may be used as the robot in the present application, and the solution proposed in the present application is used to implement the action control.

The following describes in detail a method for generating and executing an action atom, a method for creating an action molecule template, a method for generating and executing an action molecule, and a method for creating and executing an action task, which are disclosed in the embodiments of the present application, with reference to the accompanying drawings.

The method for generating action atoms disclosed in the embodiment of the present application can be operated in the master controller operating system of various robots such as a robotic arm robot, and can also be operated in the terminal electronic device communicatively connected to the master controller, as shown in fig. 1, the method includes:

s101: and acquiring an action atom template of the action atom to be generated.

The action atom template of the action atom to be generated comprises a type value, and the type value corresponds to at least one execution function and execution logic of the at least one execution function.

First, the action atom and the object related to the action atom described in the embodiments of the present application will be described.

The action atom proposed in the embodiment of the present application refers to a prepackaged data structure, which contains at least two data item values, i.e. a type value of the type of the action atom and a parameter value of a parameter of the action atom.

The type value of the action atom type is used for distinguishing different action atoms and functions for representing different action atoms, the type value of each action atom type is mapped to a function sequence, the sequence is composed of at least one function, and controllable execution logic is arranged between the functions. In other words, a piece of executable program code can be determined by the type value of each action atom, and the program code includes at least one pre-written execution function. The execution function can be implemented by different functions, and the code segment of the execution function can be stored in the master controller, or can be stored in other devices communicatively connected to the master controller, which is not limited specifically.

It should be noted that the type value of each action atom reflects the action that can be implemented by the action atom to control the robot to perform a motion, because the control parameters output by each pre-programmed execution function when the action atom is executed are different, and the execution logic of the execution function mapped by the type value in each action atom is different from that of the execution function. The execution of the action atom will be described further below.

The parameter value of the action atom is used for the execution function mapped by the type value in the action atom, and the parameter value of each action atom parameter is input into the execution function corresponding to the type value of the action atom type, so that the robot executes the corresponding action. That is, the order of execution of the execution functions and the determination of which execution function needs to be executed from among the execution functions mapped to each action atom depend on the parameter values of the action atom parameters. The parameter values of the action atoms further determine the execution logic of each execution function in the function sequence, and clarify the data of the executable program codes, thereby determining the behavior of the robot.

In one implementation of the present application, the data item of the action atom may be described and stored by using a data structure in the form of a key-value pair. In this specific implementation, the static data structure of the action atom is as follows:

{“type”：“value1”；“params”：{value21，value22，…}}

wherein the key name "type" refers to the action atom type, and the key value "value 1" is a type value of the action atom type. Similarly, the key name "params" refers to an action atom parameter, and the key values { value21, value22, … } are parameter values of the action atom parameter.

It should be noted that the type value "value 1" of the action atom type is a certain data value, may be a character string such as "gradp", "wait", or "poitlist", may be a real number "001" or "002", may be another type of data value, and is not particularly limited.

Correspondingly, according to the functional characteristics of the execution function mapped by the type value of the action atom type, the parameter value of the action atom parameter required by the execution function can be flexibly designed. Specifically, the parameter values { value21, value22, … } of the action atom parameters may be one or a specific set of values, and may also be one or a set of key-value pair data { "key 1": "value 21", "key 2": "value 22" … }. The parameter values belong to the action atoms and are input to the function sequences corresponding to the action atoms, the execution functions mapped by the type values of each action atom type are different, and the required parameters are also different, so the specific forms of the parameter values of different action atoms can be the same or different, and the embodiment of the application does not specifically limit the forms of the parameter values.

In implementations of the present application, an action atom template refers to a pre-packaged action atom. The type value of the action atom template and the program code entry and exit mapped by the type value are determined, namely, where and when to start execution, and finally, where and when to issue the control parameters to the corresponding components of the robot, and the interior of the code segment is dynamically adjusted through fixed logic execution or peripheral physical environment and state feedback of the robot during execution according to the design of different atom types, so as to realize controllable robot action.

In one implementation of the present application, the parameter value of the action atom template is nullable or a default value.

In one implementation of the present application, an action atom template is first obtained. Specifically, the required action atom template may be determined according to the type value of the action atom template, and one action atom template may be acquired as required.

In an implementation manner of the present application, an action atom template of an action atom to be generated may be obtained by a declaration calling manner. Specifically, in a program developed for a robot for a specific task, the data structure of an action atom template is directly called.

In an implementation manner of the present application, an action atom template of an action atom to be generated may be obtained based on a user operation. Specifically, at least one action atom template image is displayed on a display of the electronic device, wherein each action atom template image is a graphical interactive user interface object, and a user interacts with the graphical interactive user interface object to obtain an action atom template.

Taking a robot arm as an example, in the GUI interface displayed on the screen of the electronic device as shown in fig. 2-1 to 2-4, a process of acquiring a motion atom template grapp based on a user operation is shown, where grapp represents a type value of the motion atom template.

The user interface includes a first mechanical arm 200 and a second mechanical arm 202, and a motion atom image 204 and a motion atom image 206 have been added to the task display areas of the first mechanical arm 200 and the second mechanical arm 202, respectively. Each action atom image is also a graphical interactive user interface object, and a user can interact with each action atom image to complete operations such as editing and deleting the action atom images.

The motion atom image 204 and the motion atom image 206 refer to already generated motion atoms added based on a user operation, and include motion atom parameter values. The user interface further includes a play button 210 and a speed adjustment button 212, which are used to control the execution of the action atom in the robot task and the movement speed of the robot arm when executing the action control parameter generated by the action atom when the action atom is executed.

Specifically, the process of obtaining the action atom template grapp by the user includes, in fig. 2-1, selecting an action atom template image 208 corresponding to the grapp from a row of action atom template images below the interface by the user. In fig. 2-2, when the distance between the action atom template image 208 dragged by the user and the next action atom image in the task of the first robot arm 200 in fig. 2-3 satisfies a threshold, the slot position image 214 is displayed at the position of the next action atom image, and at this time, the user stops the dragging action, and the action atom template image 208 is overlaid on the slot position image 214, as shown in fig. 2-4. The user thus completes the operation of acquiring the action atom template grapp, and the action atom template image 208 is added to the task of the first robot arm 200.

It will be appreciated by those skilled in the art that in the above-described embodiment of obtaining the action atom template by user operation, each action atom image corresponds to one action atom data structure, and each action atom template image corresponds to one action atom template data structure. The process of obtaining the action atom template image through the user operation is also a process of obtaining an action atom template data structure, and the action atom data structure and the action atom template data structure obtained according to the user operation can be stored in the main controller or can be correspondingly stored in other equipment in communication connection with the main controller.

Fig. 2-1 to 2-4 show an embodiment of obtaining an action atom template through a user operation, and a specific method for further generating an action atom according to the action atom template and a parameter value of the action atom to be generated is described in detail below.

The display and arrangement mode of the motion atom template image and the operation mode of the motion atom template image are only used as a specific implementation mode for obtaining the motion atom template through user operation, and the skilled person can realize the operation through other user interface interaction modes in the prior art. For example, the action-atom template image is displayed with a square, oval, graphical interactive user interface object containing other pattern elements; displaying the action atom template images at different positions of the display area interface; and operating the action atom template image by using other interaction modes such as touch gestures or voice. The embodiment of the application does not specifically limit the form of the user graphical interface operated by the user and the graphical interaction mode.

S102: and generating the action atoms to be generated according to the action atom template and the parameter values of the action atoms to be generated.

Wherein the parameter value of the action atom to be generated is used for determining an execution function to be executed from the at least one execution function.

It is understood that the action atom to be generated is obtained on the basis of the action atom template that has been acquired. As can be seen from the above, the type value in the obtained action atom template and the program code (including the execution function sequence and the execution logic for executing the execution function in the function sequence) to which the type value is mapped are determined, so that the action atom to be generated can be further generated according to the action atom template and the parameter value of the action atom to be generated.

In an implementation manner of the present application, before the action atom to be generated is generated according to the action atom template and the parameter value of the action atom to be generated, the parameter value of the action atom to be generated may be obtained based on a user operation.

Taking a robot arm as an example, fig. 3-1 to 3-3 show GUI interfaces of screens of electronic devices for setting parameter values of an action atom to be generated based on user operations.

In the user interface shown in fig. 3-1, after the action atom template image 214 is acquired, the parameter value editing page is entered, or the user clicks the action atom template image 214 to enter the action atom parameter value editing page. In the embodiment shown in fig. 3-1, the motion atom parameter values corresponding to the motion atom template image 214 include the target position to which the robot arm moves, the moving speed, and the moving path mode, and at this time, the motion atom parameter values may be obtained based on the user's operation performed on each displayed motion atom parameter value. Specifically, the moving speed value and the moving path mode of the mechanical arm can be directly obtained based on the editing of the user on the parameter value editing page; the target position to which the robot arm is to be moved may be acquired in another position editing page based on user teaching.

In the user interface shown in fig. 3-2, the waiting time value can be directly obtained based on the input editing operation used in the action atom parameter value editing page corresponding to the action atom template image 216.

In the user interface shown in fig. 3-3, the motion speed values of the object for grasping an article and the robot arm may be determined based on the user selection operation in the action atom parameter value editing page corresponding to the action atom template image 218.

Fig. 3-1 to 3-3 show several specific implementation manners for obtaining the parameter value of the action atom through the user operation, and it can be understood that different types and forms of the parameter value of the action atom are different, and the graphical interaction manner for obtaining the parameter value through the user operation is also different, and the embodiments of the present application are not listed in the following.

In an implementation manner of the present application, before the action atom to be generated is generated according to the action atom template and the parameter value of the action atom to be generated, the parameter value of the action atom to be generated may be further obtained based on a preset obtaining manner.

Corresponding to the obtaining of the action atom parameter value based on the user operation, the parameter value of the action atom to be generated is determined based on the preset obtaining mode, and the action atom parameter value can be obtained without the user operation.

In an optional implementation manner of the present application, determining a parameter value of an action atom to be generated based on a preset obtaining manner includes directly obtaining a parameter value required by an execution function of the action atom from a program code mapped by an action atom type value, for example, obtaining a distance of an opening/closing claw of 10 centimeters given in the program code as a parameter value of the grabbing action atom in the action atom for realizing a grabbing function; in the action atom that implements the wait function, a 5-second wait time given in the program code is acquired as a parameter value of the wait action atom.

In another alternative embodiment, determining the parameter value of the action atom to be generated based on the preset obtaining manner includes obtaining, through calculation by other function functions or function modules, the parameter value of the action atom to be generated, which is related to the design of each action atom function and parameter value, for example, in a designed action atom, a temperature value or a humidity value obtained through an environment analysis function is taken as the parameter value of the action atom; in another designed action atom, a captured object is recognized by an image recognition function or a voice recognition module as a parameter value of the action atom.

Particularly, in an implementation manner of the present application, the action atom template of the action atom to be generated includes a default parameter value, and the parameter value of the action atom to be generated is the default parameter value.

The default parameter value of the action atom parameter is preset in the action atom template, so that the default value can be directly used as the parameter value of the action atom to be generated.

Further, the embodiment of obtaining the parameter value based on the user operation may be further adopted to obtain a new parameter value, and update the default parameter value in the action atom template.

When the action atom template is obtained in step S101, the type value is already determined, the parameter value determined by the implementation in this step is used by the execution function mapped by the type value in the action atom template, and the execution function to be executed can be determined from at least one execution function of the action atom template, so that the program code mapped by the type value of the action atom template can run normally, and an executable action atom with a determined function is generated according to the obtained action atom template and the parameter value of the action atom to be generated.

It will be appreciated that the action atoms generated according to the above embodiments are data objects having particular data structures, and may be stored directly at the master or at other devices communicatively coupled to the master, as long as the master is capable of fetching and executing.

As can be seen from the above, in the solution provided in the embodiment of the present application, the action atom template packaged in advance defines the actions that can be completed by the robot, and after the action atom template is acquired and the parameter values of the action atoms are determined, the action atoms that can control the robot to move are generated. Compared with the prior art, the user does not need to master a complex robot programming control technology and learn to use the functions of various complex demonstrator, and can achieve the purpose of using and controlling the robot only by matching the task defined by the user with the action atom function, selecting a proper action atom template and further determining the action atom parameter value.

In an embodiment of the present application, referring to fig. 2, a flowchart of another method for generating an action atom is provided, and a difference compared with the foregoing embodiment shown in fig. 1 is that step S102 generates the action atom to be generated according to the action atom template and a parameter value of the action atom to be generated, which specifically includes:

S102B: and generating the action atom to be generated according to the action atom template, the parameter value of the action atom to be generated and the name identifier of the action atom to be generated.

In an implementation manner of the present application, the data structure of the action atom further includes a data item, that is, a name identifier of a name of the action atom.

In an implementation manner of the present application, when describing and storing the action atom data item by using a data structure in a key-value pair form, a static structure of an action atom including a name identification data item is as follows:

{“type”：“value1”；“params”：{value21，value22，…}；“name”：“value3”}

wherein, compared with the data structure of the foregoing embodiment, the added key name "refers to the name of the action atom, and the key value" value3 "is the name identification of the action atom.

As can be seen from the foregoing embodiments, the type value of the action atom type is used to distinguish between different action atoms and is not alterable. By adding the name identification data item, the user can distinguish conveniently when using the name identification data item. In an implementation manner of the application, when a user operates through a graphical user interface, the name identifier of the action atom can be displayed below the action atom image, so that the user can remember and recognize the name identifier conveniently.

In an implementation manner of the application, the action atom template of the action atom to be generated includes a default name identifier, and the name identifier of the action atom to be generated is the default name identifier.

Specifically, since the action atom template includes the default name identifier, the name identifier of the action atom to be generated may be obtained when the action atom template of the action atom to be generated is obtained in step S101.

In an implementation manner of the present application, the name identifier of the action atom to be generated may be a default, or may be user-defined.

Generally, when an action atom template is packaged in advance, a default name identifier of the action atom template is determined, such as an action atom template with a type value of "gradp", and the name identifier of the action atom template is defaulted to "gradp" or "visual grab". The name identification of the action atom template may be renamed for ease of recognition and use.

In an embodiment of the present application, referring to fig. 3, a flowchart of another method for generating an action atom is provided, which is different from the foregoing embodiment shown in fig. 2 in that before the step S102B generates the action atom to be generated according to the action atom template, the parameter value of the action atom to be generated, and the name identifier of the action atom to be generated, the method further includes:

S102A: and acquiring the name identification of the action atom to be generated based on user operation.

Specifically, in the case of modification in the graphical user interface, as shown in fig. 4-1, the default name of the action atom 220 with the type value of "gradp" is identified as "visual grabbing", and the user modifies the action atom to "milk grabbing jar", as shown in fig. 4-2.

By setting the name identification, if a user encounters the situation of repeatedly using the same action atom for many times when designing a complicated robot control scheme, the user can name a plurality of action atoms with the same function according to own needs, so that confusion is avoided.

In an embodiment of the present application, an execution order of the step S101 and the step S102A is not limited to be sequential, and the steps may be executed sequentially or simultaneously, so as to obtain an action atom template and an action atom name identifier of an action atom to be generated, and the execution order of the step S101 and the step S102A does not affect subsequent steps.

In step S102B of the embodiment of the present invention, an action atom can be generated by acquiring all three data items included in the action atom to be generated and storing the data items according to a predefined data structure.

On the basis of the above embodiment, the scheme provided by this embodiment can display the name identifier of the action atom in the user image interface, and allow the user to modify the name identifier, thereby further improving the usability and flexibility of robot control and improving the efficiency of the user in controlling the robot to complete tasks.

In summary, the data structure of the action atom provided in the embodiment of the present application enables development of a robot control system to have high scalability. With the development of sensor technology and intelligent algorithms, more and more easily-used action atoms with various functions can be developed based on various execution functions and algorithm modules, the developed action atoms are packaged into action atom templates and provided for common users, and the users can conveniently control and use the robot.

After generating the action atoms of different functions based on the action atom template, the generated action atoms need to be loaded into the master for execution, and how to execute the generated action atoms is described in detail below.

In an implementation manner of the present application, as shown in fig. 5, a flow diagram of an action atom execution method is shown, where the method includes:

s501: analyzing the atom of the action to be executed, and acquiring the type value of the atom of the action to be executed and the parameter value of the atom of the action to be executed.

Wherein the type value corresponds to at least one execution function and execution logic of the at least one execution function.

As can be seen from the above embodiments of the method for generating an action atom, the generated action atom is stored as a specific data structure, the action atom at least includes two data items, i.e., a type value and a parameter value, and the following description will take a data structure in the form of a bond value pair as an example.

Based on the generation method of the action atom, the generated action atom may be stored in the master, or may be stored in another device communicatively connected to the master, and the generated action atom is generally executed in the master.

Generally, data of an action atom is stored in a storage area of a master, and when an execution command is received, the action atom is read and analyzed to obtain a type value of the action atom and a parameter value of the action atom.

In an implementation manner of the application, execution of an action atom is triggered by receiving a user operation, for example, when a user clicks a play button in a user interface, a process of executing the action atom is started. Wherein the user interface is displayed in the master or in an electronic device communicatively connected to the master.

In one implementation of the present application, after receiving a command to execute an action atom, the action atom is loaded into a control program of a master, and the control program parses a data structure of the action atom.

According to the data structure of the action atom, the type value of the action atom is obtained through a key value corresponding to the key name "type", and then one or more execution functions and execution logic among all the execution functions can be obtained. And acquiring parameter values used for executing the function through key values corresponding to the key names 'params'.

S502: and determining an execution function to be executed from the at least one execution function according to the parameter value and the execution logic of the at least one execution function.

The parameter value for an action atom is for the execution function to which the type value in the action atom is mapped. In the program code mapped by the action atom, which execution function needs to be executed and the execution order of the execution function can be determined by the execution logic in the code and the obtained parameter value.

For example, the data structure in which an action atom template exists is:

{ "type": "grasp"; "params": { "object": "value 1" }; "name": "visual grabbing" }

The program code corresponding to the grasp is described as follows:

generating an action atom A according to the action atom template:

{ "type": "grasp"; "params": { "object": "milk tank" }; "name": 'milk grabbing cylinder' }

And regenerating an action atom B according to the action atom template:

{ "type": "grasp"; "params": { "object": "cup" }; "name": "grab water cup" }

In gradps, functions 1 through 4 are well-defined execution functions, and if, else, loop, error, etc. are execution logic between execution functions.

Obviously, in one implementation of the present application, the execution functions to be executed are determined to be functions 1 and 2 according to the execution logic of the execution function in the grasp and the parameter value "milk container" of the action atom a.

In another implementation manner of the present application, the execution functions to be executed are determined to be functions 3 and 4 according to the execution logic of the execution function in the grasp and the parameter value "cup" of the action atom B.

As can be seen from the above, there may be one or more functions to be executed, at least one function exists for generating the control parameter, and other functions are in the execution logic and provide corresponding support conditions for the function that can generate the control parameter.

S503: and generating at least one group of robot action control parameters according to the execution function to be executed.

From the above, the controller of the robot needs to send control parameters to all the actuators, so that the actuators can drive the actuators to perform corresponding actions. The control parameter is information for describing a movement locus or a configuration state of the robot.

The robot arm must have multiple degrees of freedom if it is to perform a complicated motion. According to engineering mechanics knowledge, the mechanical arm robot with 6 degrees of freedom can place objects according to any expected pose and posture. Robots using 4 or 5 degrees of freedom are very common in industry. In the embodiment of the present application, the degree of freedom of the robot arm is not specifically limited.

In one implementation of the present application, the motion control parameters of the robot arm comprise at least one of robot arm pose control parameters and end-of-arm-tool control parameters.

The mechanical arm pose control parameters comprise mechanical arm tail end pose information or angle value information of respective freedom degree joints.

For the motion control of the robot arm, when the angle values of the respective joints of all robots are known, the pose of the end tool can be determined using a forward kinematics algorithm, and when the end tool is placed at a specific spatial point and has a specific pose, the angle values of the joints of each degree of freedom are generally calculated using an inverse kinematics algorithm.

Therefore, no matter how the number and the sequence of the joints of the mechanical arm are designed, the tail end pose of the mechanical arm and one of the joint angle values of the respective degrees of freedom are used as control parameters, and the tail end track of the mechanical arm can be controlled.

Specifically, the robot arm pose control parameters are exemplified, such as control parameters (X, Y, Z, Rx, Ry, Rz) representing pose information of the robot arm tip; and control parameters (30 degrees, 60 degrees, 70 degrees, 180 degrees, 30 degrees, 90 degrees and 60 degrees) representing the degree of freedom values of all joints of the 7-degree-of-freedom mechanical arm. In the arm robots of different configurations, the structural parts of the respective arms are different, and the expression forms of the motion control parameters are also different, and the embodiments of the present invention do not limit the specific expression forms of the attitude control parameters.

It will be understood by those skilled in the art that when the robotic arm robot is provided with an end tool, the motion control of the robotic arm also includes the motion control of the end tool, and the motion control parameters of the robotic arm generated according to the execution function to be executed include end tool control parameters, wherein the specific form of the end tool differs and the control parameters differ. For example, one common end tool is a gripper, and the control parameter may be the gripping distance.

The embodiment of the present application further provides another schematic flow chart of an action atom execution method shown in fig. 6, which is different from the method shown in fig. 5 in that:

the step S501 of analyzing the to-be-executed action atom, and before acquiring the type value of the to-be-executed action atom and the parameter value of the to-be-executed action atom, includes S500: sensor data is acquired.

The step S502 specifically includes: S502A: determining an executed execution function from the at least one execution function based on the parameter value, the execution logic of the at least one execution function, and the sensor data.

As described above, the robot arm robot includes sensors disposed inside the robot, for example, on each joint, and can be used to collect and feed back state information inside the robot. In addition to the foregoing, the robot in the embodiments of the present application may also include various external sensors including, but not limited to, light sensitive sensors, sound sensors, temperature sensors, pressure sensors, and the like. Various information in the robot working environment can be acquired in time through the external sensors so as to determine the action control parameters of the robot to complete the interaction with the external environment.

In one implementation of the present application, the values of some sensors may be directly used to determine the execution function to be executed by the action atom, for example, by reading the values of a temperature sensor and a humidity sensor, the values may be directly used to determine whether to execute the execution function that adjusts the temperature button or the comfort button.

In an implementation manner of the present application, after analyzing and calculating the acquired sensor data through an algorithm or a preprocessing function, the execution function to be executed by the action atom is determined, as in the above-mentioned rasp in the embodiment, the image information obtained by the vision sensor is analyzed and identified by using the image identification module, and when the image identification module identifies a different target object from the image information, the corresponding execution function is executed.

With the continuous improvement of the computing power, the processing power of the algorithm is continuously enhanced, and the motion parameters of the robot can be accurately output in real time through the analysis and operation of the data of the robot sensor, so that the intelligent motion capability of the robot is effectively improved.

It should be noted that, in the embodiment of the present application, since the execution function in each action atom is different, an action atom may output an action control parameter value only once or may output action control parameter values multiple times during execution, and each time the control parameter is output, the action control parameter value is sent to the controller for processing in real time. For example, when a target object identified by the image identification module is captured, in a specific embodiment, the execution function may output a primary motion control parameter according to a capture point identified by the object, so that the mechanical arm directly captures the object; in another specific embodiment, the execution function may output the motion control parameter once per object recognition, so that the robot arm moves to a position closer to the target object, and thus the robot arm performs multiple motions according to the multiple output control parameters and then performs grabbing according to the motion control parameter when reaching the grabbing point.

In addition, the motion planning algorithm of the robot controller of the mechanical arm can simultaneously control the position, the speed and the acceleration of the robot to ensure that the motion of the mechanical arm is smooth and accurate. Correspondingly, the motion control parameters of the robot comprise motion speed control parameters besides pose control parameters and pose control parameters of the end tool, wherein the motion speed values can be output by an execution function, can be set by a user, and can be determined according to motion atom parameter values.

The embodiment of the application also provides an action atom generating device and an action atom executing device, which are adaptive to the action atom generating and executing method.

Fig. 7 is a schematic diagram of an apparatus for generating an action atom according to an embodiment of the present application, including:

an atom template obtaining module 701, configured to obtain an action atom template of an action atom to be generated; the action atom template of the action atom to be generated comprises a type value, and the type value corresponds to at least one execution function and execution logic of the at least one execution function;

an atom generation module 702, configured to generate the action atom to be generated according to the action atom template and the parameter value of the action atom to be generated; wherein the parameter value of the action atom to be generated is used for determining an execution function to be executed from the at least one execution function.

In an implementation manner of the present application, the atom template obtaining module 701 is specifically configured to obtain an action atom template of an action atom to be generated based on a user operation.

In an implementation manner of the present application, the atom template obtaining module 701 is further configured to obtain a parameter value of an action atom to be generated based on a user operation; or

The atom template obtaining module 701 is further configured to obtain a parameter value of an action atom to be generated based on a preset obtaining manner.

In an implementation manner of the application, the action atom template of the action atom to be generated includes a default parameter value, and the parameter value of the action atom to be generated is the default parameter value.

In an implementation manner of the present application, the atom generation module 702 is specifically configured to generate the action atom to be generated according to the action atom template, the parameter value of the action atom to be generated, and the name identifier of the action atom to be generated.

In an implementation manner of the present application, the atom template obtaining module 701 is further configured to obtain a name identifier of an action atom to be generated based on a user operation.

For specific functional implementation of each module, reference may be made to the foregoing embodiment of the method for generating an action atom, which is not described herein again.

Fig. 8 is a schematic diagram of an apparatus for executing an action atom according to an embodiment of the present application, where the apparatus for executing an action atom includes:

an atom analysis module 801, configured to analyze an action atom to be executed, and obtain a type value of the action atom to be executed and a parameter value of the action atom to be executed; the type value corresponds to at least one execution function and execution logic of the at least one execution function;

an execution function determining module 802, configured to determine, according to the parameter value and the execution logic of the at least one execution function, an execution function to be executed from the at least one execution function;

and the execution function execution module 803 is configured to generate at least one set of robot motion control parameters according to the execution function to be executed.

In an implementation manner of the present application, the atom parsing module 801 is further configured to obtain sensor data;

the execution function determining module 802 is specifically configured to determine an execution function to be executed from the at least one execution function according to the parameter value, the execution logic of the at least one execution function, and the sensor data.

In an implementation manner of the present application, the robot motion control parameters include: at least one of robot arm pose control parameters and robot arm end-of-arm tool control parameters.

For specific functional implementation of each module, reference may be made to the embodiment of the method for executing the action atom, which is not described herein again.

The embodiment of the application discloses a method for creating an action molecule template, which can be operated in a master controller operating system of various robots such as a mechanical arm robot, and can also be operated in a terminal electronic device in communication connection with a master controller, as shown in fig. 9, the method comprises the following steps:

901. and receiving an adding instruction of an action unit required by the action molecule template to be created.

Wherein the type of action unit includes at least one of an action atom and an action molecule.

For the data structure of the action atom, reference is made to the foregoing embodiments, and details are not repeated in this embodiment. The structure of the action molecule mentioned in the present application is explained below. In one implementation of the present application, the data structure of the action molecule is as follows:

{“type”：“molecule”；“params”：{“start”：xx，“end”：yy，“mode”：zz}；

“name”：“value3”}

wherein, the key name "type" refers to the action molecule type, and the key value "menu" is the type value of the action molecule type. The bond name "params" refers to the action molecule parameter, the key value { "start": xx, "end": yy, "mode": zz is the parameter value of the motion molecule parameter. The bond name "refers to the action molecule name, and the key value" value3 "is the name identification of the action molecule. Because the action molecules are all stored in the configuration file of the system, the corresponding action molecules can be found according to the name identification of the action molecules.

It should be noted that the type value "menu" of the action molecule type is a certain data value, and since the action molecule and the action atom have the same structure, when the key value of "type" is "menu", it is determined that the currently parsed action molecule is.

Further, { "start": xx, "end": yy, "mode": zz) is the parameter value of the action molecule parameter, where "start" is the starting point, "end" is the ending point, and "mode" is the execution mode. The structure of the motion molecule parameter value is complex and the types are various. In the case where the start point and the end point of the action molecule are different, in the case where each action molecule in the configuration file corresponds to an action unit sequence of at least two action units arranged in order, the action unit sequence to be executed can be determined from the action unit sequence corresponding to the action molecule by the start point and the end point. In case the starting point and the end point of the action molecule are the same, the sequence of action units to be performed comprises one action unit.

The execution mode may be a first execution mode and a second execution mode, and the acquisition mode is used for indicating the execution offset corresponding to the motion numerator. In this embodiment, the first execution mode is an adaptive mode, and the second execution mode is a multiplexing mode.

To understand the adaptive mode and the multiplexing mode, the embodiment will also describe an important parameter of the motion numerator, namely, the execution offset.

The action molecules are stored in the storage area and correspond to an action unit sequence formed by at least one action unit. During execution, the robot motion control parameters to be adjusted generated by executing each motion unit need to be acquired, and then the robot motion control parameters to be adjusted are adjusted according to the execution offset to obtain the robot motion control parameters.

In the multiplexing mode, the execution offset is obtained by: acquiring a current pose value of the robot; acquiring a pose value in the robot motion control parameter to be adjusted generated by executing the first action unit in the action unit sequence to be executed; and generating an execution offset corresponding to the action molecule according to the current pose value of the robot and the pose value in the robot action control parameter to be adjusted generated by executing the first action unit.

In the adaptive mode, the execution offset corresponding to the action molecule is obtained by the following method: and acquiring the execution offset corresponding to the action molecule which is determined in advance. In the adaptive mode, the execution offset may be determined in advance according to the environment, stored in a configuration file of the system as a predetermined parameter value, and loaded by the control process.

Take the pattern drawing action molecule of the mechanical arm robot to make coffee as an example.

In the multiplexing mode, a current pose value Q 'of the mechanical arm is acquired, a pose value Q in the mechanical arm motion control parameter to be adjusted generated by the first action unit in the action unit sequence to be executed corresponding to the garland action molecule is acquired, differences (Δ X, Δ Y, Δ Z) between Q and Q' are calculated, and the differences are used as an execution offset. And sequentially executing each action unit in the action unit sequence to be executed corresponding to the garland action molecule to generate a mechanical arm action control parameter to be adjusted, and adjusting the pose value (Q, W, E) in the mechanical arm action control parameter to be adjusted according to the execution offset to obtain the pose value (Q ', W ', E ') in the mechanical arm action control parameter.

In the adaptive mode, the execution offset amount offset (Δ X, Δ Y, Δ Z) is stored in a configuration file of the system as a predetermined parameter value. And sequentially executing each action unit in the action unit sequence to be executed corresponding to the garland action molecule to generate a mechanical arm action control parameter to be adjusted, and adjusting the pose value (Q, W, E) in the mechanical arm action control parameter to be adjusted according to the execution offset to obtain the pose value (Q ', W ', E ') in the mechanical arm action control parameter.

902. And generating the action units according to the adding instruction to obtain a first action unit sequence.

Optionally, in an implementation manner of the present application, the first action unit of the first action unit sequence may be a preset action unit.

The preset action unit can be specifically an action unit for setting the initial pose of the robot.

Optionally, in this step, the generating the action unit according to the add instruction, referring to fig. 10, specifically includes:

1041. and acquiring an action unit template of the action unit according to the adding instruction.

1042. And generating the action unit according to the action unit template and the parameter value of the action unit.

Optionally, in an implementation manner of the present application, if the action unit is an action atom, an action atom template of the action atom includes a type value, and the type value included in the action atom template corresponds to at least one execution function and execution logic of the at least one execution function;

the parameter value of the action atom is used for determining an execution function to be executed from at least one execution function;

referring to fig. 11, step 902 specifically includes:

1041a, obtaining the action atom template of the action atom according to the adding instruction.

1042a, generating the action atom according to the action atom template and the parameter value of the action atom.

Optionally, before the step 1042a, further comprising:

acquiring a parameter value of the action atom based on user operation; or

Optionally, in another implementation manner of the present application, referring to fig. 12, the step 902 specifically includes:

1041b, obtaining the action atom template of the action atom according to the adding instruction.

1042b, generating action atoms according to the action atom templates, the parameter values of the action atoms and the name identifications of the action atoms.

Optionally, before the step 1042b, further comprising: and acquiring the name identification of the action atom based on user operation.

Optionally, in an implementation manner of the present application, the action atom template includes a default name identifier, and the name identifier of the action atom is the default name identifier.

Specifically, since the action atom template includes the default name identifier, when the action atom template of the action atom is acquired, the default name identifier can be acquired as the name identifier of the action atom.

In implementations of the application, the name identification of the action atom may be user-defined, in addition to being default.

The above describes the specific execution of step 902 in the case where the action unit is an action atom. In an implementation manner of the present application, if the action unit is an action molecule, an action molecule template of the action molecule includes a name identifier, and the name identifier included in the action molecule template corresponds to a second action unit sequence;

the parameter value of the action molecule is used for determining a second action unit sequence to be executed from the second action unit sequence;

referring to fig. 13, the step 902 specifically includes:

1041c, obtaining the action molecule template of the action molecule according to the adding instruction.

1042c, generating an action molecule based on the action molecule template and the parameter value of the action molecule.

Optionally, before the step 1042c, further comprising:

acquiring a parameter value of the action molecule based on user operation; or

In another implementation manner of the present application, referring to fig. 14, the step 902 specifically includes:

1041d, obtaining the action molecule template of the action molecule according to the adding instruction.

1042d, generating the action molecule according to the action molecule template, the parameter value of the action molecule and the type value of the action molecule.

In implementations of the present application, the action molecule template includes a default type value, and the type value of the action molecule is the default type value. Since the action molecule template includes a default type value, the default type value may be obtained as the type value of the action molecule when the action molecule template of the action molecule is obtained.

903. And establishing a corresponding relation between the name identifier of the action molecule template to be created and the first action unit sequence, and creating the action molecule template to be created according to the name identifier.

Optionally, before establishing the correspondence between the name identifier of the action molecule template to be created and the first action unit sequence, the method further includes:

receiving a deleting instruction of an action unit in the first action unit sequence;

and deleting the corresponding action unit in the first action unit sequence according to the deletion instruction.

The action molecule template creating method allows a user to create the action molecule template, so that the action molecule is generated to control the robot, and the method is flexible to operate and easy to use by common users.

The embodiment of the present application further discloses an apparatus for creating an action molecule template, referring to fig. 15, the apparatus includes:

a molecule template instruction receiving module 1501, configured to receive an adding instruction of an action unit required by an action molecule template to be created; the type of the action unit includes at least one of an action atom and an action molecule.

The molecular template instruction execution module 1502 is configured to generate the action unit according to the add instruction, so as to obtain a first action unit sequence.

Optionally, the first action unit of the first action unit sequence is a preset action unit.

The molecule template creating module 1503 is configured to create a correspondence between a name identifier of the action molecule template to be created and the first action unit sequence, and create the action molecule template to be created according to the name identifier.

In a specific implementation manner of this embodiment, the molecular template creating module 1503 is specifically configured to create the action molecular template to be created according to the name identifier and the default type value; wherein the default type value indicates that the type of action unit generated based on the action molecule template to be created is an action molecule.

In a specific implementation manner of this embodiment, the molecular template instruction executing module 1502 is specifically configured to obtain an action unit template of the action unit according to the adding instruction; and generating the action unit according to the action unit template and the parameter value of the action unit.

In one embodiment of this embodiment, if the action unit is an action atom,

the action atom template of the action atom comprises a type value, and the type value of the action atom template corresponds to at least one execution function and execution logic of the at least one execution function; the parameter value of the action atom is used for determining an execution function to be executed from the at least one execution function;

the molecular template instruction execution module 1502 is specifically configured to: and generating the action atom according to the action atom template and the parameter value of the action atom.

In a specific implementation manner of this embodiment, the molecular template instruction execution module 1502 is further configured to: acquiring a parameter value of the action atom based on user operation; or acquiring the parameter value of the action atom based on a preset acquisition mode.

In a specific implementation manner of this embodiment, the molecular template instruction executing module 1502 is specifically configured to: and generating the action atom according to the action atom template, the parameter value of the action atom and the name identification of the action atom.

In a specific implementation manner of this embodiment, the molecular template instruction execution module 1502 is further configured to: and acquiring the name identification of the action atom based on user operation.

Optionally, the action atom template includes a default name identifier, and the name identifier of the action atom is the default name identifier.

In one embodiment of this embodiment, if the action unit is an action molecule,

the action molecule template of the action molecule comprises a name identifier, and the name identifier of the action molecule template corresponds to the second action unit sequence; the parameter value of the action molecule is used for determining a second action unit sequence to be executed from the second action unit sequence;

the molecular template instruction execution module 1502 is specifically configured to: and generating the action molecule according to the action molecule template and the parameter value of the action molecule.

In a specific implementation manner of this embodiment, the molecular template instruction execution module 1502 is further configured to: acquiring a parameter value of the action molecule based on user operation; or is also used for acquiring the parameter value of the action molecule based on a preset acquisition mode.

In a specific implementation manner of this embodiment, the molecular template instruction executing module 1502 is specifically configured to: generating the action molecule according to the action molecule template, the parameter value of the action molecule and the type value of the action molecule.

Optionally, the action molecule template comprises a default type value, and the type value of the action molecule is the default type value.

Optionally, the parameter values of the action molecule comprise a start point and an end point.

Optionally, the parameter value of the action molecule further comprises an execution mode; the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule generated based on the action molecule template to be created.

In a specific implementation manner of this embodiment, the molecular template instruction receiving module 1501 is further configured to receive a delete instruction for an action unit in the first action unit sequence;

the molecular template instruction execution module 1502 is further configured to delete a corresponding action unit in the first action unit sequence according to the deletion instruction.

The above is a schematic scheme of a device for creating an action molecule template of the present embodiment. It should be noted that the technical solution of the apparatus for creating an action molecule template is the same as that of the method for creating an action molecule template described above, and for details not described in detail, reference may be made to the description of the technical solution of the method for creating an action molecule template described above.

The embodiment of the application discloses a method for generating action molecules, which can be operated in a master controller operating system of various robots such as a mechanical arm robot and the like, and can also be operated in terminal electronic equipment in communication connection with a master controller, referring to fig. 16, and the method comprises the following steps:

1601. obtaining an action molecule template of an action molecule to be generated; wherein the action molecule template comprises a name identifier, and the name identifier corresponds to a third action unit sequence.

The type of the action unit includes at least one of an action atom and an action molecule.

For the data structures of the action atoms and the action molecules mentioned in this embodiment, reference may be made to the description of the foregoing embodiments, and the description of this embodiment is not repeated.

Optionally, the obtaining of an action molecule template of an action molecule to be generated specifically includes: and acquiring an action molecule template of the action molecule to be generated based on user operation.

1602. Generating the action molecules to be generated according to the action molecule template and the parameter values of the action molecules to be generated; wherein the parameter value is used to determine a third sequence of action units to be performed from the third sequence of action units.

The parameter value of the action molecule to be generated may be obtained in the process of generating the action molecule, or may be a default value.

Optionally, for the case that the parameter value of the action molecule to be generated is obtained in the process of generating the action molecule, before step 1602, further includes:

acquiring a parameter value of an action molecule to be generated based on user operation; or

And acquiring the parameter value of the action molecule to be generated based on a preset acquisition mode.

Optionally, for a case that the parameter value of the action molecule is a default value, the action molecule template includes the default parameter value, and the parameter value of the action molecule to be generated is the default parameter value.

The parameter values of the action molecules to be generated comprise a starting point and an end point, and further comprise an execution mode; and the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule to be generated.

Specifically, in this step 1602, generating the action molecule to be generated according to the action molecule template and the parameter value of the action molecule to be generated specifically includes:

generating the action molecules to be generated according to the action molecule template, the parameter values of the action molecules to be generated and the type values of the action molecules to be generated; wherein the type value of the action molecule to be generated indicates that the type of the action molecule to be generated is an action molecule.

Optionally, the action molecule template includes a default type value, and the type value of the action molecule to be generated is the default type value.

In the scheme provided by the embodiment of the application, the action molecules to be generated are generated according to the action molecule template and the parameter values of the action molecules to be generated. The data structure of the action molecules provided in the embodiment of the application enables the development of the robot control system to have high expandability. With the development of sensor technology and intelligent algorithms, more and more easily-used action molecules with various functions can be developed based on various execution functions and algorithm modules, the developed action molecules are packaged into action atom templates to be provided for common users, and the users can conveniently control and use the robot.

The embodiment of the application also discloses an execution method of the action molecule, and the method comprises the following steps:

1701. analyzing the action molecules to be executed, and acquiring the name identification of the action molecules to be executed and the parameter values of the action molecules to be executed; the name identification of the action molecule to be executed corresponds to a fourth action unit sequence; the type of the action unit includes at least one of an action atom and an action molecule.

For the data structures of the action atoms and the action molecules, reference is made to the foregoing embodiments, which are not repeated in this embodiment.

Optionally, in this step 1701, before obtaining the name identifier of the action molecule to be executed and the parameter value of the action molecule to be executed, the method further includes:

acquiring a type value of the action molecule to be executed;

and confirming that the type of the action molecule to be executed is an action molecule according to the type value of the action molecule to be executed.

1702. And determining a fourth action unit sequence to be executed from the fourth action unit sequence according to the parameter value of the action molecule to be executed.

The parameter values of the action molecules to be executed comprise a starting point and an end point, and the parameter values further comprise an execution mode, wherein the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecules to be executed.

Step 1702 specifically includes: and determining a fourth action unit sequence to be executed from the fourth action unit sequence according to the starting point and the ending point in the parameter values of the action molecule to be executed.

1703. And sequentially executing each action unit in the fourth action unit sequence to be executed to generate at least one group of robot action control parameters.

Referring to fig. 18, sequentially executing each action unit in the fourth action unit sequence to be executed, and generating at least one set of robot action control parameters specifically includes:

1801. and sequentially executing each action unit in the fourth action unit sequence to be executed, and generating at least one group of robot action control parameters to be adjusted.

1802. And adjusting the at least one group of robot motion control parameters to be adjusted according to the execution offset to obtain the at least one group of robot motion control parameters.

When the execution mode is the self-adaptive mode, acquiring the execution offset corresponding to the action molecule to be executed by adopting the following mode:

and acquiring the execution offset corresponding to the action molecule to be executed which is determined in advance.

When the execution mode is a multiplexing mode, acquiring an execution offset corresponding to the action molecule to be executed by adopting the following mode:

acquiring a current pose value of the robot;

acquiring a pose value in the robot action control parameter to be adjusted generated by executing the first action unit in the fourth action unit sequence to be executed;

and generating an execution offset corresponding to the action molecule to be executed according to the current pose value of the robot and the pose value in the action control parameter of the robot to be adjusted generated by executing the first action unit.

Specifically, in the case where the action unit is an action atom, referring to fig. 19, step 1801 includes:

1901. analyzing the action atom aiming at each action atom in the fourth action unit sequence to be executed, and acquiring the type value of the action atom and the parameter value of the action atom.

The type value of the action atom corresponds to at least one execution function and execution logic of the at least one execution function.

1902. And determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom and the execution logic of the at least one execution function.

Prior to step 1902, further comprising:

acquiring sensor data;

step 1902 specifically includes:

1903. And generating at least one group of robot motion control parameters to be adjusted according to the execution function to be executed.

The robot motion control parameters include: at least one of robot arm pose control parameters and robot arm end-of-arm tool control parameters.

Specifically, in the case where the action unit is an action molecule, referring to fig. 20, step 1801 includes:

2001. and analyzing the action molecule aiming at each action molecule in the fourth action unit sequence to be executed, and acquiring the name identification of the action molecule and the parameter value of the action molecule.

The name identifier of the action molecule corresponds to a fifth sequence of action units.

2002. And determining a fifth action unit sequence to be executed from the fifth action unit sequence according to the parameter value of the action molecule.

2003. And sequentially executing each action unit in the fifth action unit sequence to be executed, and generating at least one group of robot action control parameters to be adjusted.

The present embodiment performs motion control on the robot based on the generated robot motion control parameters, so that the robot completes the motion corresponding to the motion molecule. Compared with the prior art, the robot generates the action molecules according to the pre-packaged action molecule template and the parameter values of the action molecules, and then the robot can generate the control parameters by executing the generated action molecules, so that the robot is controlled to move to complete corresponding actions. By implementing the scheme provided by the application, the difficulty of controlling the robot to execute the task by the user can be reduced, and the interaction capacity and flexibility of the robot are improved.

The embodiment of the present application also discloses an apparatus for generating an action molecule, referring to fig. 21, the apparatus includes:

a molecule template obtaining module 2101, configured to obtain an action molecule template of an action molecule to be generated; the action molecule template comprises a name identifier, and the name identifier corresponds to a third action unit sequence; the type of the action unit comprises at least one of an action atom and an action molecule;

a molecule generating module 2102, configured to generate the action molecule to be generated according to the action molecule template and the parameter value of the action molecule to be generated; wherein the parameter value is used to determine a third sequence of action units to be performed from the third sequence of action units.

Optionally, the molecule template acquiring module 2101 is specifically configured to acquire an action molecule template of an action molecule to be generated based on a user operation.

Optionally, the molecule template obtaining module 2101 is further configured to obtain a parameter value of an action molecule to be generated based on a user operation; or the parameter value of the action molecule to be generated is also acquired based on a preset acquisition mode.

The action molecule template comprises default parameter values, and the parameter values of the action molecules to be generated are the default parameter values.

The parameter values of the action molecules to be generated comprise a starting point and an end point, and in addition, the parameter values of the action molecules to be generated also comprise an execution mode; and the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule to be generated.

Optionally, the molecule generating module 2102 is specifically configured to generate the action molecule to be generated according to the action molecule template, the parameter value of the action molecule to be generated, and the type value of the action molecule to be generated; wherein the type value of the action molecule to be generated indicates that the type of the action molecule to be generated is an action molecule.

The action molecule template comprises a default type value, and the type value of the action molecule to be generated is the default type value.

The above is a schematic configuration of an apparatus for generating an action molecule of the present embodiment. The technical means of the apparatus for generating an action molecule is the same as the technical means of the method for generating an action molecule described above, and for details not described in detail, reference may be made to the description of the technical means of the method for generating an action molecule described above.

The embodiment of the present application further discloses an actuating device of the action molecule, referring to fig. 22, the device includes:

a molecule analyzing module 2201, configured to analyze a to-be-executed action molecule, and obtain a name identifier of the to-be-executed action molecule and a parameter value of the to-be-executed action molecule; the name identification of the action molecule to be executed corresponds to a fourth action unit sequence; the type of the action unit comprises at least one of an action atom and an action molecule;

an action unit determining module 2202, configured to determine a fourth action unit sequence to be executed from the fourth action unit sequence according to the parameter value of the action molecule to be executed;

an action unit executing module 2203, configured to sequentially execute each action unit in the fourth action unit sequence to be executed, and generate at least one set of robot action control parameters.

Optionally, the molecule parsing module 2201 is further configured to obtain a type value of the to-be-performed action molecule; and confirming that the type of the action molecule to be executed is an action molecule according to the type value of the action molecule to be executed.

Optionally, the parameter values of the action molecule to be executed include a starting point and an end point, and additionally include an execution mode; the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule to be executed.

An action unit determining module 2202, configured to determine a fourth action unit sequence to be executed from the fourth action unit sequence according to a starting point and an ending point in the parameter values of the action molecule to be executed.

An action unit executing module 2203, specifically configured to sequentially execute each action unit in the fourth action unit sequence to be executed, and generate at least one set of robot action control parameters to be adjusted; and adjusting the at least one group of robot motion control parameters to be adjusted according to the execution offset to obtain the at least one group of robot motion control parameters.

When the execution mode is the adaptive mode, the action unit execution module 2203 obtains the execution offset corresponding to the action molecule to be executed by adopting the following method:

When the execution mode is the multiplexing mode, the action unit executing module 2203 acquires the execution offset corresponding to the action molecule to be executed by adopting the following method:

acquiring a current pose value of the robot; acquiring a pose value in the robot action control parameter to be adjusted generated by executing the first action unit in the fourth action unit sequence to be executed; and generating an execution offset corresponding to the action molecule to be executed according to the current pose value of the robot and the pose value in the action control parameter of the robot to be adjusted generated by executing the first action unit.

Optionally, the action unit executing module 2203 is specifically configured to:

analyzing the action atom aiming at each action atom in the fourth action unit sequence to be executed, and acquiring the type value of the action atom and the parameter value of the action atom; the type value of the action atom corresponds to at least one execution function and execution logic of the at least one execution function;

Further, the action unit executing module 2203 is also used for acquiring sensor data;

the action unit executing module 2203 is specifically configured to determine, according to the parameter value of the action atom, the execution logic of the at least one execution function, and the sensor data, an execution function to be executed from the at least one execution function.

analyzing the action molecule aiming at each action molecule in the fourth action unit sequence to be executed, and acquiring the name identification of the action molecule and the parameter value of the action molecule; the name identification of the action molecule corresponds to a fifth action unit sequence;

determining a fifth action unit sequence to be executed from the fifth action unit sequence according to the parameter value of the action molecule;

and sequentially executing each action unit in the fifth action unit sequence to be executed, and generating at least one group of robot action control parameters to be adjusted.

The above is a schematic scheme of an actuator of an action molecule according to the present embodiment. It should be noted that the technical solution of the actuator of the action molecule and the technical solution of the method for actuating the action molecule described above belong to the same concept, and for details that are not described in detail, reference may be made to the description of the technical solution of the method for actuating the action molecule described above.

The embodiment of the application also discloses a method for creating an action task, and referring to fig. 23, the method comprises the following steps:

2301. receiving an adding instruction of an action unit required by an action task to be created; the type of the action unit includes at least one of an action atom and an action molecule.

For a detailed description of the data structure of the action atom and the action molecule, refer to the foregoing embodiments, and are not repeated herein.

2302. And generating the action units according to the adding instruction to obtain a sixth action unit sequence.

Specifically, referring to fig. 24, the step 2302 of generating the action unit according to the add instruction includes the following steps 2401 to 2402:

2401. and acquiring an action unit template of the action unit according to the adding instruction.

2402. And generating the action unit according to the action unit template and the parameter value of the action unit.

As described above, an action unit may include both types of action atoms and action molecules.

If the action unit is an action atom, the action atom template of the action atom comprises a type value, and the type value of the action atom template corresponds to at least one execution function and execution logic of the at least one execution function; the parameter value of the action atom is used for determining an execution function to be executed from the at least one execution function;

step 2402 specifically includes: and generating the action atom according to the action atom template and the parameter value of the action atom.

Optionally, before step 2402, further comprising:

acquiring a parameter value of the action atom based on user operation; or

Optionally, the action atom further comprises: the name identification of the action atom generates the action atom according to the action atom template and the parameter value of the action atom, and comprises the following steps:

The name identifier of the action atom may be obtained before the action atom is generated, or may be a default name identifier of the action atom template.

In one embodiment, before generating the action atom according to the action atom template, the parameter value of the action atom, and the name identifier of the action atom, the method further includes:

and acquiring the name identification of the action atom based on user operation.

In another embodiment, the action atom template includes a default name identifier, and the name identifier of the action atom is the default name identifier.

If the action unit is an action molecule, an action molecule template of the action molecule comprises a name identifier, and the name identifier included by the action molecule template corresponds to a seventh action unit sequence;

generating the action unit according to the action unit template of the action unit and the parameter value of the action unit, comprising: and generating the action molecule according to the action molecule template and the parameter value of the action molecule.

Before generating the action molecule according to the action molecule template and the parameter value of the action molecule, the method further comprises:

acquiring a parameter value of the action molecule based on user operation; or

In addition to the parameter values, the action molecule further comprises: a type value;

generating the action molecule according to the action molecule template and the parameter value of the action molecule, comprising: generating the action molecule according to the action molecule template, the parameter value of the action molecule and the type value of the action molecule.

The parameter values of the action molecule include a start point and an end point. In addition, the parameter value of the action molecule further comprises an execution mode; the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule.

2303. And establishing a corresponding relation between the name identifier of the action task to be created and the sixth action unit sequence, and creating the action task to be created according to the name identifier.

In step 2303, creating the action task to be created according to the name identifier includes:

In one embodiment, the data structure for the action task is as follows:

{“type”：“mission”；“name”：“value3”}

wherein "type" is an action task type; the "permission" is a type value of the action task type, is a determined data value, and when the "type" value is analyzed to be the "permission", the currently analyzed action task can be determined; "name" is the action task name; the value3 is an action task name identifier, and a corresponding target action task can be found according to the action task name identifier.

The data structure of the action task provided by the embodiment of the application ensures that the development of the robot control system has high expandability, and is convenient for a user to control and use the robot.

Referring to fig. 25, an embodiment of the present application further discloses an execution method of an action task, where the method includes:

2501. analyzing the action task to be executed, and acquiring a name identifier of the action task to be executed; and the name identification of the action task to be executed corresponds to an eighth action unit sequence to be executed.

For the data structures of the action atoms and the action molecules, refer to the foregoing embodiments, and the description of the embodiment is omitted.

Optionally, in step 2501, before obtaining the name identifier of the action task to be executed, the method further includes:

acquiring a type value of the action task to be executed;

2502. And sequentially executing each action unit in the eighth action unit sequence to be executed, and generating at least one group of robot action control parameters.

Specifically, referring to fig. 26, step 2502 includes:

2601. analyzing the action atom aiming at each action atom in the eighth action unit sequence to be executed, and acquiring the type value of the action atom and the parameter value of the action atom; the type value of the action atom corresponds to at least one execution function and execution logic of the at least one execution function.

2602. And determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom and the execution logic of the at least one execution function.

In one embodiment, prior to step 2602, further comprising: acquiring sensor data;

step 2602 includes: determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom, the execution logic of the at least one execution function, and the sensor data.

2603. And generating at least one group of robot action control parameters according to the execution function to be executed.

Specifically, referring to fig. 27, step 2502 includes:

2701. analyzing the action molecule aiming at each action molecule in the eighth action unit sequence to be executed, and acquiring the name identification of the action molecule and the parameter value of the action molecule; the name identification of the action molecule corresponds to a ninth action unit sequence.

Specifically, before obtaining the name identifier of the action molecule and the parameter value of the action molecule, the method further includes:

obtaining a type value of the action molecule;

2702. And determining a ninth action unit sequence to be executed from the ninth action unit sequence according to the parameter value of the action molecule.

The parameter values of the action molecule comprise a starting point and an end point; the parameter values of the action molecule further comprise an execution mode; the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule.

2703. And sequentially executing each action unit in the ninth action unit sequence to be executed to generate at least one group of robot action control parameters.

Specifically, referring to fig. 28, step 2703 comprises:

2801. and sequentially executing each action unit in the ninth action unit sequence to be executed to generate at least one group of robot action control parameters to be adjusted.

Specifically, step 2801 includes:

2802. And adjusting the at least one group of robot motion control parameters to be adjusted according to the execution offset to obtain the at least one group of robot motion control parameters.

When the execution mode is the self-adaptive mode, acquiring the execution offset corresponding to the action molecule by adopting the following mode: and acquiring the execution offset corresponding to the action molecule which is determined in advance.

When the execution mode is a multiplexing mode, acquiring an execution offset corresponding to the action molecule by adopting the following mode: acquiring a current pose value of the robot; acquiring a pose value in the robot action control parameter to be adjusted generated by executing the first action unit in the ninth action unit sequence to be executed; and generating an execution offset corresponding to the action molecule according to the current pose value of the robot and the pose value in the robot action control parameter to be adjusted generated by executing the first action unit.

In the method for executing the action task, based on the generated robot action control parameter, the robot is subjected to action control, so that the robot completes an action corresponding to the action task, and the robot is controlled to move to complete a corresponding action. By implementing the scheme provided by the application, the difficulty of controlling the robot to execute the task by the user can be reduced, and the interaction capacity and flexibility of the robot are improved.

The following takes an action task as an example to exemplarily explain the creating and executing method of the action task of the present embodiment.

For example, the motion task of a robotic arm is to perform the function of "making coffee".

The action unit sequence corresponding to the action task of "making coffee" may include 4 action molecules, for example, including "making espresso", "whipping", "garbling", "cleaning" action molecules. The action unit sequence corresponding to each action molecule may include a plurality of action atoms, for example, action atoms including "move", "open and close claws", "wait", "visual grasp", "horizontal movement", "vertical movement", and the like.

The creation process of the action task of making coffee comprises the following steps:

1) receiving an adding instruction of action molecules required by the action task of 'making coffee'.

2) And generating an action molecule according to the adding instruction to obtain an action molecule sequence.

That is, the generated sequence of action molecules includes 4 action molecules: making concentrated coffee, making milk foam, making flower and cleaning.

The generation process of each action molecule comprises the following steps:

21) acquiring an action molecule template according to the adding instruction;

22) and generating the action molecule according to the action molecule template and the parameter value of the action molecule.

The action molecule template comprises a name identifier, and in an action task of 'making coffee', the name identifier included by the action molecule template corresponds to an action atom sequence.

3) And establishing a corresponding relation between the name identification of the action task of making coffee and the action molecule sequence, and establishing the action task of making coffee according to the name identification.

The execution process of the action task of making coffee comprises the following steps:

1) analyzing the action task of making coffee, and acquiring the name identifier of the action task of making coffee; the name identification of an action task corresponds to the sequence of action molecules to be performed.

I.e. the sequence of action molecules to be performed comprises 4 action molecules: making concentrated coffee, making milk foam, making flower and cleaning.

2) Analyzing the action molecules aiming at each action molecule in the action molecule sequence to be executed, and acquiring name identifications of the action molecules and parameter values of the action molecules; in the action task of 'making coffee', the name identification of an action molecule corresponds to an action atomic sequence;

3) determining an action atomic sequence to be executed from the action atomic sequence according to the parameter value of the action molecule;

4) and sequentially executing each action atom in the action atom sequence to be executed to generate at least one group of mechanical arm action control parameters.

The mechanical arm action control parameters obtained by executing the action task of making coffee can control the mechanical arm to realize the function of making coffee.

An embodiment of the present application discloses an action task creating device, see fig. 29, including:

a task instruction receiving module 2901, configured to receive an add instruction of an action unit required by an action task to be created; the type of the action unit comprises at least one of an action atom and an action molecule;

a task instruction execution module 2902, configured to generate the action unit according to the add instruction, and obtain a sixth action unit sequence;

a task creating module 2903, configured to create a corresponding relationship between the name identifier of the action task to be created and the sixth action unit sequence, and create the action task to be created according to the name identifier.

Optionally, the task creating module 2903 is specifically configured to create the action task to be created according to the name identifier and the default type value.

Optionally, the task instruction execution module 2902 is specifically configured to:

acquiring an action unit template of the action unit according to the adding instruction; and generating the action unit according to the action unit template and the parameter value of the action unit.

Alternatively, if the action unit is an action atom,

the task instruction execution module 2902 is specifically configured to: and generating the action atom according to the action atom template and the parameter value of the action atom.

Optionally, before the task instruction execution module 2902 generates the action atom according to the action atom template and the parameter value of the action atom, it is further configured to:

and acquiring the parameter value of the action atom based on user operation or acquiring the parameter value of the action atom based on a preset acquisition mode.

Optionally, the task instruction execution module 2902 is specifically configured to: and generating the action atom according to the action atom template, the parameter value of the action atom and the name identification of the action atom.

Optionally, before the task instruction execution module 2902 generates the action atom, it is further configured to: and acquiring the name identification of the action atom based on user operation.

Alternatively, if the action unit is an action molecule,

task instruction execution module 2902 is specifically configured to:

Optionally, before the task instruction execution module 2902 generates the action molecule according to the action molecule template and the parameter value of the action molecule, it is further configured to: and acquiring the parameter value of the action molecule based on user operation or acquiring the parameter value of the action molecule based on a preset acquisition mode.

Optionally, task instruction execution module 2902 is specifically configured to: generating the action molecule according to the action molecule template, the parameter value of the action molecule and the type value of the action molecule.

Optionally, the parameter value of the action molecule comprises a starting point and an end point, and in addition, the parameter value of the action molecule further comprises an execution mode; wherein the execution mode indicates an acquisition manner of an execution offset corresponding to the action molecule.

The above is an exemplary scheme of a creating apparatus of an action task of the present embodiment. It should be noted that the technical solution of the device for creating an action task is the same as that of the method for creating an action task, and for details that are not described in detail, reference may be made to the description of the technical solution of the method for creating an action task.

Referring to fig. 30, an embodiment of the present application further discloses an apparatus for executing an action task, where the apparatus includes:

the task analysis module 3001 is configured to analyze an action task to be executed, and obtain a name identifier of the action task to be executed; the name identification of the action task to be executed corresponds to an eighth action unit sequence to be executed; the type of the action unit comprises at least one of an action atom and an action molecule;

and the task execution module 3002 is configured to sequentially execute each action unit in the eighth action unit sequence to be executed, and generate at least one set of robot action control parameters.

Optionally, before the task resolution module 3001 obtains the name identifier of the action task to be executed, the method is further configured to:

acquiring a type value of the action task to be executed; and according to the type value of the action task to be executed, confirming that the type of the action task to be executed is the action task.

Optionally, the task execution module 3002 is specifically configured to:

Optionally, before the task execution module 3002 determines, according to the parameter value of the action atom and the execution logic of the at least one execution function, an execution function to be executed from the at least one execution function, further:

acquiring sensor data;

the task execution module 3002 is specifically configured to: determining an execution function to be executed from the at least one execution function according to the parameter value of the action atom, the execution logic of the at least one execution function, and the sensor data.

Optionally, the task execution module 3002 is specifically configured to:

Optionally, before the task execution module 3002 obtains the name identifier of the action molecule and the parameter value of the action molecule, it is further configured to:

obtaining a type value of the action molecule; and confirming that the type of the action molecule is the action molecule according to the type value of the action molecule.

Optionally, the parameter values of the action molecule comprise a start point and an end point;

the task execution module 3002 is specifically configured to:

Optionally, the parameter value of the action molecule further comprises an execution mode; the execution mode indicates an acquisition mode of an execution offset corresponding to the action molecule;

the task execution module 3002 is specifically configured to:

Optionally, when the execution mode is the adaptive mode, the task execution module 3002 obtains the execution offset corresponding to the action molecule as follows:

Optionally, when the execution mode is the multiplexing mode, the task execution module 3002 obtains the execution offset corresponding to the action molecule as follows:

acquiring a current pose value of the robot;

Optionally, the task execution module 3002 is specifically configured to:

analyzing the action molecule aiming at each action molecule in the ninth action unit sequence to be executed, and acquiring the name identification of the action molecule and the parameter value of the action molecule; the name identification of the action molecule corresponds to a seventh action unit sequence;

The above is a schematic scheme of an execution device of an action task according to the embodiment. It should be noted that the technical solution of the device for executing the action task belongs to the same concept as the technical solution of the method for executing the action task, and the details of the technical solution of the device for executing the action task, which are not described in detail, can be referred to the description of the technical solution of the method for executing the action task.

Fig. 31 is a block diagram illustrating a structure of an electronic device 3100 according to an embodiment of the present application. The components of the electronic device include, but are not limited to, the memory 3110 and the processor 3120, with the processor 3120 being coupled to the memory 3110. The processor may implement the method for generating an action atom, the method for generating an action molecule, the method for creating an action molecule template, and the method for creating an action task, as described above, when executing the instructions.

It should be appreciated that the electronic device may also include a network interface that enables the electronic device to communicate via one or more networks. Examples of such networks include a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The network interface may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In an embodiment of the application, the above-mentioned and other components not shown in fig. 31 of the electronic device may also be connected to each other, for example by a bus. It should be understood that the block diagram of the electronic device shown in fig. 31 is for exemplary purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as desired.

The electronic device may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The terminal may also be a mobile or stationary server.

The above is a schematic scheme of the electronic device of the present embodiment. It should be noted that the technical solutions of the electronic device are the same as those of the above-mentioned method for generating an action atom, method for generating an action molecule, method for creating an action molecule template, and method for creating an action task, and the details that are not described in detail can be referred to the above description.

The embodiment of the present application further discloses a master controller 3200, see fig. 32, which includes a memory 3210, a processor 3220, and computer instructions stored in the memory 3210 and executable on the processor 3220, where when the processor 3220 executes the instructions, the method for executing the action atom, the method for executing the action molecule, and the method for executing the action task may be implemented as described above.

In the embodiments of the present application, the aforementioned master may be any terminal device, including a mobile computer or mobile computing device, a mobile phone, a wearable computing device or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC.

The above is a schematic scheme of the master of the present embodiment. It should be noted that the technical solution of the master is the same as the technical solutions of the above-mentioned action atom executing method, action molecule executing method, and action task executing method, and the details that are not described in detail can be referred to the above description.

The embodiment of the application also provides a robot, which comprises the master controller provided by the embodiment of the application.

In an embodiment of the present invention, the main controller may be a main controller of a robot arm, and the robot may be a robot arm including the main controller.

The Memory mentioned above may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

The embodiment of the present application further provides a non-volatile computer storage medium, where the computer storage medium stores computer instructions, and the computer instructions are executed by a processor, so as to implement the method for generating an action atom, the method for generating an action molecule, the method for creating an action molecule template, and the method for creating an action task provided in the embodiment of the present application.

The embodiment of the present application further provides a non-volatile computer storage medium, where the computer storage medium stores computer instructions, and the computer instructions are executed by a processor, so as to implement the method for executing the action atom, the method for executing the action molecule, and the method for executing the action task provided by the embodiment of the present application.

The embodiment of the application also discloses a computer program, which comprises computer instructions, and the computer instructions are executed by a processor, so that the method for generating the action atom, the method for executing the action atom, the method for generating the action molecule, the method for executing the action molecule, the method for creating the action molecule template, the method for creating the action task or the method for executing the action task can be realized.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, to thereby enable others skilled in the art to best understand and utilize the application. The application is limited only by the claims and their full scope and equivalents.

Claims

1. A method for creating an action task, the method comprising:

2. The method for creating an action task according to claim 1, wherein creating the action task to be created according to the name identifier comprises:

3. The method for creating an action task according to claim 1, wherein generating the action unit according to the addition instruction includes:

acquiring an action unit template of the action unit according to the adding instruction;

4. The method for creating an action task according to claim 3, wherein if the action unit is an action atom,

5. The method for creating an action task according to claim 4,

before generating the action atom according to the action atom template and the parameter value of the action atom, the method further comprises the following steps:

acquiring a parameter value of the action atom based on user operation; or

6. The method for creating an action task according to claim 4,

generating the action atom according to the action atom template and the parameter value of the action atom, wherein the generating of the action atom comprises the following steps:

7. The method for creating an action task according to claim 6, wherein before generating the action atom according to the action atom template, the parameter value of the action atom, and the name identifier of the action atom, the method further comprises:

8. The method of creating an action task of claim 6, wherein the action atom template includes a default name identification, the name identification of the action atom being the default name identification.

9. The method for creating an action task according to claim 3, wherein if the action unit is an action molecule,

10. The action task creation method according to claim 9, further comprising, before generating the action molecule based on the action molecule template and a parameter value of the action molecule:

acquiring a parameter value of the action molecule based on user operation; or