CN113190012B - Robot task autonomous planning method and system - Google Patents
Robot task autonomous planning method and system Download PDFInfo
- Publication number
- CN113190012B CN113190012B CN202110506117.7A CN202110506117A CN113190012B CN 113190012 B CN113190012 B CN 113190012B CN 202110506117 A CN202110506117 A CN 202110506117A CN 113190012 B CN113190012 B CN 113190012B
- Authority
- CN
- China
- Prior art keywords
- task
- planning
- article
- action
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000009471 action Effects 0.000 claims abstract description 55
- 230000003068 static effect Effects 0.000 claims abstract description 55
- 230000010391 action planning Effects 0.000 claims abstract description 30
- 230000008569 process Effects 0.000 claims abstract description 29
- 238000004590 computer program Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 235000013336 milk Nutrition 0.000 description 2
- 239000008267 milk Substances 0.000 description 2
- 210000004080 milk Anatomy 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 235000016795 Cola Nutrition 0.000 description 1
- 244000228088 Cola acuminata Species 0.000 description 1
- 235000011824 Cola pachycarpa Nutrition 0.000 description 1
- 244000141359 Malus pumila Species 0.000 description 1
- 235000021016 apples Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Manipulator (AREA)
Abstract
The invention belongs to the field of robots, and provides a robot task autonomous planning method and system. The method comprises the steps of obtaining a semantic position of a static article and a position relation between the static article and a dynamic article based on a family environment semantic knowledge model; based on the semantic position of the static article and the position relation between the static article and the dynamic article, executing action planning according to the hybrid task planner until the task sequence executed by the robot completes the task; the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain.
Description
Technical Field
The invention belongs to the field of robots, and particularly relates to a robot task autonomous planning method and system.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
With the development of the robot technology, the service robot gradually moves into the family to provide various services for the human, and becomes a good helper and even a good partner for improving the quality of life of the human. Although service robots have great potential in the field of home applications, many problems remain to be solved in terms of mission planning and the like. The diversity of service tasks and uncertainty of target location, especially in unstructured, dynamic home environments, increases the complexity of service robot task planning. How to carry out reasonable task planning according to the family environment information is an urgent problem to be solved. For a service robot, task planning is to plan a complete action sequence to guide the robot to complete a given task. The entire sequence of actions is expressed in high level semantic form. Such as moving, grasping, etc.
PPDDL (probabilistic planar domain definition) and POMDP (probabilistic Observable Markov Decision Process) are the two most commonly used task planning methods. PPDDL can quickly generate an operational sequence for a given task based on state transitions. However, PPDDL lacks a reliable reasoning for unreliable observations. It generates a linear and static task execution sequence, and only after the current action is finished, the next action can be executed in sequence. If an action fails to perform, it means that the task fails, the flexibility is poor, and it is difficult to adapt to a dynamic, unstructured environment. Therefore, a dynamic planning strategy in robot sequence generation needs to be given by considering the problem of target occlusion in a complex dynamic environment. When there is uncertainty, we consider using POMDP to judge the execution of an action until the goal is completed. POMDP is a planning method in an uncertain environment, and generally refers to planning with probabilistic effect. The goal of the POMDP planner is to increase the probability of planning success. The complexity of the home environment and the variety of objects make it more difficult for the robot to perform tasks. It is difficult to interact with the environment through individual mission planning. The inventors have found that although POMDP can interact with the environment in real time, as space increases, the state dimension also increases.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a method and a system for independently planning tasks of a mobile home service robot, which are based on the object level semantic graph and the probabilistic reasoning, can improve the autonomy of task planning of the service robot.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a mobile home service robot task autonomous planning method.
A robot task autonomous planning method, comprising:
based on a family environment semantic knowledge model, obtaining a semantic position of a static article and a position relation between the static article and a dynamic article;
based on the semantic position of the static article and the position relation between the static article and the dynamic article, executing action planning according to the hybrid task planner until the task sequence executed by the robot completes the task;
the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain.
Further, the process of obtaining the semantic position of the static article and the position relationship between the static article and the dynamic article is as follows:
based on a home environment semantic knowledge model and autonomous movement of the robot in the home environment, relationships among object instance levels, map levels and symbol levels are established and mapped into an article position body, and semantic position relationships among home scenes, static articles and dynamic articles are inferred.
Further, in the process of performing online action planning, an article shielding calculation model is designed, confidence state information required by the online action planning is generated, and a corresponding execution state is generated according to the sub-targets of task execution, so that an online task execution sequence is generated.
Further, the building process of the item occlusion calculation model comprises the following steps: and calculating the shielding rate by using a plane formed by projecting the 3D rectangular frame of the object on an x-y plane in a camera coordinate system, and further constructing an object shielding calculation model.
Further, when no target object is detected through object detection in the process of executing the offline task planning, switching to online action planning, and generating an online execution strategy by using the sub-object state at the moment.
Further, the hybrid mission planner includes an offline mission planner and an online mission planner.
Furthermore, in the process of off-line task planning, a problem domain file and a planning domain file of off-line task planning are automatically generated according to the preconditions of the execution of the designed actions and the influence on the environment after the execution by combining task targets.
A second aspect of the invention provides a mobile home service robot task autonomous planning system.
A robotic task autonomous planning system, comprising:
the article position determining module is used for obtaining a semantic position of a static article and a position relation between the static article and a dynamic article based on the family environment semantic knowledge model;
the task planning module is used for executing action planning according to the hybrid task planner based on the semantic position of the static article and the position relation between the static article and the dynamic article until the task sequence executed by the robot completes the task;
the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain.
A third aspect of the invention provides a computer-readable storage medium.
A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for mobile home service robot task autonomous planning as described above.
A fourth aspect of the invention provides a computer apparatus.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the mobile home service robot task autonomous planning method as described above when executing the program.
Compared with the prior art, the invention has the beneficial effects that:
the invention establishes the probability relation between the dynamic object and the static object, deduces the dynamic object based on the semantic mapping of the static object level, then uses the target position information obtained by the method as the input of the hybrid task planner to generate the off-line and on-line action sequence, the switching mechanism can realize the free switching of the off-line and on-line task planning, and finally, the execution monitoring and re-planning mechanism is designed to process the task failure, thereby improving the intelligence of the robot.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a schematic diagram of a task autonomous planning method for a mobile home service robot according to an embodiment of the present invention;
FIG. 2 is an object level semantic representation of an embodiment of the invention;
FIG. 3 is an illustration of the effects of parameters, preconditions and action execution on part of the atomic task skills of an embodiment of the invention;
FIG. 4 is a schematic diagram of automatic generation of planning domain and problem domain files according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example one
In order to cope with complex home environment, the embodiment establishes an object level semantic graph and a probabilistic reasoning relationship, the object level semantic graph is used for providing semantic positions of static objects which are not easy to change in position, such as a refrigerator, a washing machine, a dining table and the like, and the probabilistic relationship is mainly used for providing semantic relationships between the static objects and dynamic objects, such as cups, apples, colas, milk and the like. As can be seen, a symbiotic relationship exists between dynamic and static objects. For example, milk is often placed in a refrigerator, and possibly on a table. Thus, a probabilistic relationship between the dynamic object and the static object is established, with the dynamic object being inferred based on the static object-level semantic mapping. Then, the target position information obtained by the method is used as the input of the hybrid task planner to generate an off-line action sequence and an on-line action sequence. The switching mechanism can realize free switching of offline and online task planning. Finally, an execution monitoring and re-planning mechanism is designed to process task failure, and the intelligence of the robot is improved.
As shown in fig. 1 and fig. 2, the method for autonomous planning of a robot task of the present embodiment includes:
s101: and obtaining the semantic position of the static article and the position relation between the static article and the dynamic article based on the family environment semantic knowledge model.
As shown in fig. 2, a physical map and ontology knowledge, and a bayesian probability model are used for establishing a semantic model of the home environment, i.e. an object-level semantic graph.
Firstly, a physical map is established by utilizing an automatic navigation technology or user control, static articles are identified by utilizing an article detection method, and semantic information is given to the map.
Secondly, establishing a position ontology of the family scene and the articles by using ontology knowledge, dividing the articles into static articles and dynamic articles, and establishing a probability relation model between the static articles and the dynamic articles
Wherein o is i ,o j Respectively representing static articles and dynamic articles, the number of times of the common appearance of the static articles and the dynamic articles in the camera vision is N (o) i |o j ) N represents the number of detected articles, and tau plays a smoothing role and takes a value of 0.5. In the case of dynamic item discovery, the probability model of the presence of a static item is
Where Θ represents a collection of items.
Finally, the relationship model is inferred through probability, namely theta * ＝argmax[ρ(o j |o T )]The co-occurrence relationship between the static and solid objects is known, and then the semantic position relationship between the static and dynamic objects can be obtained by searching the semantic map.
Specifically, the process of obtaining the semantic position of the static article and the position relationship between the static article and the dynamic article is as follows:
based on a home environment semantic knowledge model and autonomous movement of the robot in the home environment, relationships among object instance levels, map levels and symbol levels are established and mapped into an article position body, and semantic position relationships among home scenes, static articles and dynamic articles are inferred.
And respectively obtaining the semantic positions of the static articles and the semantic positions of the dynamic articles and the static articles by using the article level semantic graph and the probabilistic semantic reasoning.
S102: based on the semantic position of the static article and the position relation between the static article and the dynamic article, executing action planning according to the hybrid task planner until the task sequence executed by the robot completes the task;
the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain.
In a specific implementation, the hybrid mission planner includes an offline mission planner PPDDL and an online mission planner POMDP.
The PPDDL planner is a probabilistic planner that generates a sequence of actions that may have a deterministic or non-deterministic effect, the non-deterministic action effect being (probabilistic ρ) 1 e 1 ρ 2 e 2 … ρ k e k ) Wherein e is k Representing the influence of motion, and p k Representing the probability of the effect of the action.
In this embodiment, in the process of performing offline task planning, in combination with a task target, a problem domain file and a planning domain file for offline task planning are automatically generated according to preconditions for executing a designed action and an influence on an environment after execution.
As shown in fig. 4, the automatic generation manner of the PPDDL planner problem domain text and plan domain file of the present embodiment is shown. Through atomic action extraction, the initial state of an object is obtained from a semantic knowledge model, and the object is extracted from a user command, so that six elements required by a PPDDL planner, namely type (types), predicate (predicates), action (action), object (objects), initial state (initial state) and object (goal), are obtained. And finally, obtaining a standard form of a planning domain and a problem domain file required by the planner through information state conversion, and using the standard form for generating an action sequence. Atomic actions are required in the planning domain file, and the atomic actions plus the execution objects form atomic task skills. The following table lists the atomic actions and their descriptions.
TABLE 1 atomic actions and descriptions thereof
As shown in fig. 3, the parameters of partial atomic task skills and preconditions are the influences generated after the action is executed, which are the basis for the automatic generation of the planning domain file, and the generation of the problem domain file is generated according to the initial state of the target object, the initial state of the robot and the task goal. And generating an off-line task execution sequence by using a quick search rule.
In the specific implementation, in the process of performing online action planning, an article shielding calculation model is designed, confidence state information required by the online action planning is generated, and a corresponding execution state is generated according to sub-targets of task execution, so that an online task execution sequence is generated.
POMDP is a partially observable Markov decision process, is an ideal model for sequential decisions in a partially known and dynamically uncertain environment of environmental conditions, and takes action over a potentially long time horizon taking into account the uncertainties in observations and actions. The model may take action at a certain state. The invention designs a scene model, an action model, a perception model and a reward model aiming at the POMDP.
And (3) scene model: the components of the scene are represented as states S ═ S rob ,s obj Where the robot state (x, y, z, θ), which includes the robot's three-dimensional pose (x, y, z) and direction θ. The state of each object is represented as (x) i ,y i ,z i ,
) Wherein (x) i ,y i ,z i ) And
representing the 3D position and orientation of the ith object in a world coordinate system, t i Representing whether the ith object is the target object.
An action model: we design 4 motion models, moveBase, that allow the robot to adjust the robot body and change its position. moveObject (o) i ) The action indicates that the item may be moved out of the grid area of the work scene and placed in a designated placement area. publishGraspSuccess (o) i ) The action representation indicates that the target item was found and successfully grabbed. publishFailure (o) i ) The action indicates that the target item is not found in the operation area.
A perception model: let z ═ o rob ,o statei Is a robot and item viewing set. Wherein o is rob Is a state that can be obtained from observation, which is completely observable. o statei Is the state in which the ith object is partially observed, including the estimated item position (x) i ,y i ,z i ) Object type t i And occlusion ratio occl i
The observation function O (s ', a, z) may obtain the uncertainty of the current object type due to partial observation, and when action a is performed in state s, the probability of the next state s' may be observed. The accuracy of the target type estimation depends on the degree of occlusion. The greater the degree of occlusion, the less accurate the estimation of the target type. When an operation is performed in state s, a new state s' will appear. The robot captures the spatial relationship between objects by observation, which can be measured by occlusion rate. The degree of occlusion between objects, called the occlusion rate, can be estimated by the three-dimensional bounding box of the objects. We can use a three-dimensional bounding box to obtain the shape parameters of the object: height, width, length. The observation effect of the robot is affected by the visual angle of the camera, and therefore the estimation of the target occlusion is affected. Thus, given the pose and type of all object instances, their three-dimensional bounding box will be projected onto the x-y plane in the camera coordinate system, generating a set of two-dimensional rectangles R ═ rect j }. In addition, according to the size of the objectThe probability of the object class size is defined, and the problem of the shielding of the object caused by the size difference of the object is solved. Combining the probability of the object size and the projected 2D rectangular frame, the occlusion rate is determined as follows:
an observation function of
Where Max is the number of objects.
And (3) reward model: the design of the reward function is consistent with the goals of the task: the target object is found and then selected. This goal is achieved by operating the various objects in the cluster environment through four actions in action modeling. The reward for running the action moveBase is-250. If the robot successfully moves the object by performing the action moveObject, a reward of 150 will be obtained. Otherwise, if the object cannot be moved, a penalty of-2000 is assigned. If the robot successfully executes publishGraspsumes (o) i )(o i Is the target object) a reward of 200 would be achieved. Otherwise, a reward of-1000 is provided. If the target is not in the operating area, the action publishFailure is performed, then 200 is awarded. Otherwise, the reward is-2000.
The extension of Monte Carlo Tree Search (Monte Carlo Tree Search) to partially observable Monte Carlo (Partial observer Monte Carlo Projec) is used as the problem solver for POMDP.
And if the influence of a certain action is determined in the offline task planning process, continuously executing the offline task execution sequence until the task is completed. On the contrary, if the action influence is uncertain in probability, namely the target object is blocked, the current robot subtask target is obtained, and the environment state of the current target object is obtained through the perception and object blocking model.
And realizing online action planning based on the POMDP model, and interacting with the environment in real time until a subtask target is reached. And returning to the off-line task execution sequence, and continuing to execute the task until the final target of the task is reached.
And when the robot fails to execute the task, re-planning the task. Here, the task failure means that the semantic position of the dynamic article obtained by probabilistic reasoning is incorrect in the family environment, and the semantic position of the suboptimal dynamic article needs to be inferred again. Meanwhile, a problem domain file is generated again according to the semantic position of the robot in the family environment and the original task target. And repeating the process until the task is successfully executed.
Example two
The embodiment provides a robot task autonomous planning system, which includes:
the article position determining module is used for obtaining a semantic position of a static article and a position relation between the static article and a dynamic article based on the family environment semantic knowledge model;
the task planning module is used for executing action planning according to the hybrid task planner based on the semantic position of the static article and the position relation between the static article and the dynamic article until the task sequence executed by the robot completes the task;
the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain.
It should be noted that, each module in the autonomous robot task planning system of the embodiment corresponds to each step in the autonomous robot task planning method of the first embodiment one by one, and the specific implementation process is the same, which will not be described herein again.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the mobile home service robot task autonomous planning method as described above.
Example four
The embodiment provides a computer device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the mobile home service robot task autonomous planning method as described above when executing the program.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A robot task autonomous planning method is characterized by comprising the following steps:
based on a family environment semantic knowledge model, obtaining a semantic position of a static article and a position relation between the static article and a dynamic article;
based on the semantic position of the static article and the position relation between the static article and the dynamic article, executing action planning according to the hybrid task planner until the task sequence executed by the robot completes the task;
the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain;
the hybrid mission planner includes an offline mission planner PPDDL and an online mission planner POMDP.
2. The method for autonomous planning of a robot task according to claim 1, wherein the process of obtaining the semantic locations of the static items and the positional relationships between the static items and the dynamic items is:
based on a home environment semantic knowledge model and autonomous movement of the robot in the home environment, relationships among object instance levels, map levels and symbol levels are established and mapped into an article position body, and semantic position relationships among home scenes, static articles and dynamic articles are inferred.
3. The autonomous planning method for robot tasks according to claim 1, wherein in the process of performing online action planning, an item occlusion computation model is designed, confidence state information required by the online action planning is generated, and a corresponding execution state is generated according to sub-targets of task execution, thereby generating an online task execution sequence.
4. The method for autonomous planning of a robot task according to claim 3, wherein the building process of the item occlusion computation model is as follows: and calculating the shielding rate by using a plane formed by projecting the 3D rectangular frame of the object on an x-y plane in a camera coordinate system, and further constructing an object shielding calculation model.
5. The method as claimed in claim 1, wherein when no target object is detected by object detection during the execution of the off-line mission planning, the method switches to on-line action planning, and generates an on-line execution strategy by using the sub-object status.
6. The autonomous planning method for robot task according to claim 1, wherein in the process of performing offline task planning, in combination with task objectives, problem domain files and planning domain files for offline task planning are automatically generated according to preconditions for execution of designed actions and influence on environment after execution.
7. A robotic task autonomous planning system, comprising:
the article position determining module is used for obtaining a semantic position of a static article and a position relation between the static article and a dynamic article based on the family environment semantic knowledge model;
the task planning module is used for executing action planning according to the hybrid task planner based on the semantic position of the static article and the position relation between the static article and the dynamic article until the task sequence executed by the robot completes the task;
the mixed task planner firstly performs offline task planning in the process of executing action planning, determines whether the action influence of the offline task planning is definite or not to judge whether to continue executing the offline task sequence or not, and then performs online action planning when the action influence of the offline task planning is uncertain;
the hybrid mission planner includes an offline mission planner PPDDL and an online mission planner POMDP.
8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps in the method for autonomous planning of tasks of a robot according to any of claims 1-6.
9. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, carries out the steps in the method for autonomous planning of tasks of a robot according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110506117.7A CN113190012B (en) | 2021-05-10 | 2021-05-10 | Robot task autonomous planning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110506117.7A CN113190012B (en) | 2021-05-10 | 2021-05-10 | Robot task autonomous planning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113190012A CN113190012A (en) | 2021-07-30 |
| CN113190012B true CN113190012B (en) | 2022-08-12 |
Family
ID=76988645
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110506117.7A Active CN113190012B (en) | 2021-05-10 | 2021-05-10 | Robot task autonomous planning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113190012B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102430841A (en) * | 2011-08-26 | 2012-05-02 | 昆山工研院工业机器人研究所有限公司 | Arc welding robot laser vision seam tracking control method based on offline planning |
| CN106097304A (en) * | 2016-05-31 | 2016-11-09 | 西北工业大学 | A kind of unmanned plane real-time online ground drawing generating method |
| CN107031867A (en) * | 2016-10-18 | 2017-08-11 | 中国运载火箭技术研究院 | A kind of configurable Integrated Electronic System for implementing space tasks contexture by self |
| CN108983625A (en) * | 2018-07-20 | 2018-12-11 | 山东大学深圳研究院 | A kind of smart home system and service creation method |
| CN110110245A (en) * | 2019-05-06 | 2019-08-09 | 山东大学 | Dynamic article searching method and device under a kind of home environment |
| CN112013841A (en) * | 2020-08-26 | 2020-12-01 | 南京工业大学 | Semantic SLAM service robot navigation method in indoor dynamic environment |
| CN112418568A (en) * | 2021-01-25 | 2021-02-26 | 苏州澳昆智能机器人技术有限公司 | Semi-online stack type planning method and robot |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102011011542B4 (en) * | 2011-02-17 | 2016-05-25 | Convergent Information Technologies Gmbh | Method for automated programming and optimization of robotic workflows |
| JP6075343B2 (en) * | 2014-09-02 | 2017-02-08 | トヨタ自動車株式会社 | Traveling robot, operation planning method thereof, and program |
| WO2018111920A1 (en) * | 2016-12-12 | 2018-06-21 | The Charles Stark Draper Laboratory, Inc. | System and method for semantic simultaneous localization and mapping of static and dynamic objects |
| US20190184561A1 (en) * | 2017-12-15 | 2019-06-20 | The Regents Of The University Of California | Machine Learning based Fixed-Time Optimal Path Generation |
| US10606269B2 (en) * | 2017-12-19 | 2020-03-31 | X Development Llc | Semantic obstacle recognition for path planning |
| EP3734471A1 (en) * | 2019-04-30 | 2020-11-04 | Tata Consultancy Services Limited | Method and system for utilizing domain knowledge to automatically identify solution to a problem |
-
2021
- 2021-05-10 CN CN202110506117.7A patent/CN113190012B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102430841A (en) * | 2011-08-26 | 2012-05-02 | 昆山工研院工业机器人研究所有限公司 | Arc welding robot laser vision seam tracking control method based on offline planning |
| CN106097304A (en) * | 2016-05-31 | 2016-11-09 | 西北工业大学 | A kind of unmanned plane real-time online ground drawing generating method |
| CN107031867A (en) * | 2016-10-18 | 2017-08-11 | 中国运载火箭技术研究院 | A kind of configurable Integrated Electronic System for implementing space tasks contexture by self |
| CN108983625A (en) * | 2018-07-20 | 2018-12-11 | 山东大学深圳研究院 | A kind of smart home system and service creation method |
| CN110110245A (en) * | 2019-05-06 | 2019-08-09 | 山东大学 | Dynamic article searching method and device under a kind of home environment |
| CN112013841A (en) * | 2020-08-26 | 2020-12-01 | 南京工业大学 | Semantic SLAM service robot navigation method in indoor dynamic environment |
| CN112418568A (en) * | 2021-01-25 | 2021-02-26 | 苏州澳昆智能机器人技术有限公司 | Semi-online stack type planning method and robot |
Non-Patent Citations (7)
| Title |
|---|
| 《Arrangement Map for Task Planning and Localization for an Autonomous Robot in 》a Large-Scale Environment;Paulo Pinheiro 等;《IEEE》;20141231;第13-17页 * |
| 《Home service robot task planning using semantic knowledge and probabilistic inference》;Zhongli Wang 等;《Knowledge-Based Systems》;20200623;第1-12页 * |
| 《Hybrid offline and online task planning for service robot using object-level semantic map and probabilistic inference》;Zhongli Wang 等;《Information Sciences》;20220202;第78-98页 * |
| 《TOPOLOGICAL TASK SPACE MODELLING FOR AUTONOMOUS SPACE ROBOT ACTION PLANNING》;Xi Tao 等;《Automatic Control in Aerospace》;19941231;第345-356页 * |
| 《不完全可知环境下的室内移动机器人任务规划系统》;程翀 等;《工业控制计算机》;20190531;第74-79页 * |
| 《在部分观测环境下的不确定动作模型学习》;饶东宁 等;《软件学报》;20141231;第51-63页 * |
| 《基于视觉SLAM的物体实例识别与语义地图构建》;吴皓 等;《华中科技大学学报（自然科学版）》;20190930;第48-54页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113190012A (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tanaka et al. | Emd net: An encode–manipulate–decode network for cloth manipulation | |
| Moldovan et al. | Learning relational affordance models for robots in multi-object manipulation tasks | |
| Beetz et al. | Cognition-enabled autonomous robot control for the realization of home chore task intelligence | |
| Lampe et al. | Acquiring visual servoing reaching and grasping skills using neural reinforcement learning | |
| Mösenlechner et al. | Parameterizing actions to have the appropriate effects | |
| Mitash et al. | Task-driven perception and manipulation for constrained placement of unknown objects | |
| Eich et al. | Towards coordinated multirobot missions for lunar sample collection in an unknown environment | |
| Pan et al. | Real-time collision detection and distance computation on point cloud sensor data | |
| Zhang et al. | Active visual planning for mobile robot teams using hierarchical POMDPs | |
| Dearden et al. | Manipulation planning using learned symbolic state abstractions | |
| Moldovan et al. | Relational affordances for multiple-object manipulation | |
| Portugal et al. | Applying Bayesian learning to multi-robot patrol | |
| Nam et al. | A software architecture for service robots manipulating objects in human environments | |
| Li et al. | Controllable human-object interaction synthesis | |
| Safin et al. | Modelling a turtlebot3 based delivery system for a smart hospital in gazebo | |
| Schlenoff et al. | Ontology-based state representations for intention recognition in human–robot collaborative environments | |
| Finean et al. | Simultaneous scene reconstruction and whole-body motion planning for safe operation in dynamic environments | |
| Zhang et al. | Safe and efficient robot manipulation: Task-oriented environment modeling and object pose estimation | |
| Graule et al. | Gg-llm: Geometrically grounding large language models for zero-shot human activity forecasting in human-aware task planning | |
| CN113190012B (en) | Robot task autonomous planning method and system | |
| Arnold et al. | Cloth manipulation planning on basis of mesh representations with incomplete domain knowledge and voxel-to-mesh estimation | |
| Tanaka et al. | Disruption-resistant deformable object manipulation on basis of online shape estimation and prediction-driven trajectory correction | |
| Kraft et al. | Learning objects and grasp affordances through autonomous exploration | |
| Zenkri et al. | Hierarchical policy learning for mechanical search | |
| Faber et al. | Design and implementation of a cognitive simulation model for robotic assembly cells |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |