CN115796288A - Method and device for reasoning tasks in dynamic scene based on knowledge base - Google Patents

Abstract

The invention discloses a method and a device for reasoning tasks in a dynamic scene based on a knowledge base, which comprises the following steps: on the basis that a semantic network expressing a knowledge base is divided into an action relation sub-network, a state change relation sub-network, a subordination relation sub-network and a preposition relation sub-network according to relation types, reasoning of behavior tree missing judgment of tasks is conducted according to the action relation sub-network, and the reasoning acquisition of missing data such as target objects and perception data related to states is achieved through all the sub-networks, so that reasoning of a behavior tree is achieved. The method can avoid using task knowledge or a structured robot task design language, completes inference planning of tasks by using a universal semantic network, and realizes the operation problem of the robot.

Description

Method and device for reasoning tasks in dynamic scene based on knowledge base

Technical Field

The invention belongs to the technical field of combination of task behavior planning and a knowledge base of an industrial intelligent robot, and particularly relates to a method and a device for reasoning tasks in a dynamic scene based on the knowledge base.

Background

A robot is a "physical Agent that accomplishes a task by manipulating the physical world," with the ability to interact with the physical world similar to, and even exceeding in some way, a human. However, as with individual individuals, stand-alone robotic systems have been unable to meet the increasingly complex task requirements of modern society due to their own limitations in their ability to perceive processed information, decision-making, and task performance.

Robots require mission planning to sequence actions to achieve goals that are not possible with a single action. Task planning refers to a strategy of sequencing execution actions to synthesize constraint conditions and ensure efficient and satisfactory completion of tasks. Mission planning is more biased towards high-level decisions than process implementation. For simple tasks, it is generally not necessary to strictly distinguish the process of task planning. For complex tasks, the robot requires a mission planning algorithm to sequence the actions to achieve the goal that a single action may not accomplish.

Off-the-shelf mission planners can be used by intelligent robots to solve a variety of planning problems. However, there are many different planners, each with different strengths and weaknesses, and there is no general rule to decide which planner is best suited for a given problem.

Existing task planning includes state machine task decomposition, task decomposition using pre-scripted scripts, and task planning using behavior tree control. Where a state machine represents a model of a finite number of states and the behavior of transitions and actions between these states. Wherein the state stores information about the past, that is: it reflects the input changes from the beginning of the system to the present time. Transitions represent transitions of state, described by the conditions under which the transition occurs. An action is a description of an activity to be performed at a given time. The advantage of state machine based task decomposition is that it is easy to understand and can quickly implement basic functions. The method has two disadvantages, namely, lack of flexibility and incapability of realizing tasks with uncertain flows; and secondly, for complex tasks, the workload is huge when the combination conditions are many, and all processes need to be exhausted. And thirdly, the maintenance is not easy. The content to be modified may be very large if there is a change in the execution flow.

The behavior tree refers to a hierarchical node tree for controlling the decision flow of the entity. Within the scope of the tree, leaves are the actual commands that control the entity, while branches are formed of various types of utility nodes that control the direction along the tree to arrive at a command sequence that best suits the situation.

At present, the robot behavior control is mostly controlled by depending on rule constraint, and when the robot does not receive an instruction, the robot cannot spontaneously generate a behavior instruction; or executing repeated tasks according to automatic instructions preset by a robot designer; the relation between ontology nodes cannot be obtained under the condition of only an ontology knowledge graph, and the specific relation and parameters of the task planning in the unknown state cannot be obtained.

Disclosure of Invention

In view of the above, the present invention provides a method and an apparatus for reasoning tasks in a dynamic scene based on a knowledge base, so as to plan tasks and obtain missing data in the dynamic scene of a robot.

In order to achieve the above object, an embodiment provides a method for reasoning tasks in a dynamic scene based on a knowledge base, including the following steps:

acquiring a semantic network expressing a knowledge base, and dividing the semantic network into an action relation sub-network, a state change relation sub-network, a dependency relation sub-network and a preposition relation sub-network according to the relation types;

when the behavior tree corresponding to the target task cannot be searched in the task planning graph, carrying out at least one of target object reasoning and task reasoning and state and perception related reasoning under the semantic network;

reasoning aiming at the target object, determining an action set of the target task, and reasoning the target object in an action relation sub-network, a subordinate relation sub-network and a preposition relation sub-network according to actions in the action set;

aiming at task reasoning, path searching is carried out on the action relation subnetwork according to two end points of the missing segment of the behavior tree, the missing segment is determined according to a searching result, and the task reasoning is realized;

and aiming at the state and perception related reasoning, searching and determining the ontology of the state in the state change relation sub-network and the subordination relation sub-network, triggering the perception device to acquire perception data related to the ontology according to the ontology, and reasoning according to the perception data.

Preferably, after the action relation sub-network is obtained by dividing, a mapping of the task to the action set is constructed, including:

the actions associated with the single task are marked out in the action relation sub-network, the actions associated with the combined task are marked out in the action relation sub-network, then task semantic clustering is carried out on the marked actions, all the actions contained in each cluster correspond to one task semantic, and therefore mapping from each task to the action set is obtained, and further the mapping set is obtained.

Preferably, the determining the action set of the target task includes searching for a task matching the target task in the mapping set, and using the action set mapped by the task as the action set of the target task.

Preferably, inferring the target object in the action relation sub-network, the dependency sub-network from the actions in the set of actions comprises:

and determining an object of action operation in the action relation sub-network according to each action, performing multi-stage expansion of the object in the subordination relation sub-network according to the object to obtain a newly added target object, and performing screening and filtering operation on the newly added target object according to the perception condition.

Preferably, the performing of the multi-level extension of the object in the dependency sub-network includes:

and searching a sub-class object having a subordinate relationship with the object in the subordinate relationship sub-network as a new target object: searching a parent object having an affiliation with the object in the affiliation sub-network, and taking other child objects of the parent object as new target objects;

preferably, the screening and filtering operation performed on the newly added target object according to the perception condition includes: after a newly added target object is searched by a parent object of the object, the newly added target object which cannot appear in the scene is deleted according to a perception object example in the scene.

Preferably, inferring the target object in the action relation subnetwork, the dependency relation subnetwork, and the preposition relation subnetwork from the actions in the action set comprises:

when the target object set cannot be inferred in the subordination relation subnetwork, the inference of the target object is carried out in the preposition relation subnetwork according to prepositions, and the inference comprises the following steps: and on the basis of the object operated by the action, performing at least 3-hop search in the preposition relation subnetwork according to the preposition relation, wherein the object searched in each hop is used as a newly added target object.

Preferably, the performing a path search in the action relation subnetwork according to two end points of the missing segment of the behavior tree, and determining the missing segment according to the search result includes:

mapping a behavior tree containing the missing segments into a sub-semantic network, wherein the behaviors of the behavior tree are mapped into relationships in the sub-semantic network, and action targets of the behaviors are mapped into nodes of the sub-semantic network;

finding the actions of two end points of a missing segment in the sub-semantic network, finding two objects corresponding to the actions of the two end points in the action relation sub-network according to the actions of the two end points, and searching a plurality of communication paths between the two objects in the action relation sub-network, wherein each communication path represents a series of action plans between the two objects;

and judging whether each communication path can be matched with the task planning graph according to a matching rule, wherein the matching rule is as follows: all edge relationships in a communication path must be able to be classified into a subtask or action plan; the nodes in the connected path have to have instances in the scene or are generalized to the instances in the scene in a first order in the dependency sub-network;

screening the shortest communication path meeting the matching rule, converting the edges in the shortest communication path into behaviors, converting the nodes in the shortest communication path into real objects in the scene to obtain the missing segments of the behavior tree,

and supplementing the missing segments of the behavior tree into the behavior tree to finish the task reasoning of the behavior tree.

Preferably, the acquiring of sensing data related to the ontology by triggering the sensing device according to the ontology by searching the ontology for determining the state in the state change relationship sub-network and the dependency relationship sub-network includes:

on the basis of a target object and a corresponding unknown sensing parameter label, after a state related to the target object is searched in a state change relation sub-network, a state body is searched in a subordinate relation sub-network, the state body is linked to a corresponding sensing device according to the parameter description of the body, instantiation data of the sensing device on body parameters is obtained, and the label of the instantiation data is the unknown sensing parameter label, so that behavior parameters are filled by inquiring the instantiation data under the label, and the missing information is inferred.

Preferably, the method further comprises: searching in the task planning graph to obtain a behavior tree corresponding to the target task, specifically comprising: classifying a target task into one task node in a task planning graph, searching all connected subtask nodes from the task node downwards in sequence, forming a task tree by all obtained subtask nodes and edges, determining an action set of the subtask according to the mapping from the task to the action set after determining the subtask which can be actually executed by the task tree, and constructing a behavior tree according to the action set of the subtask.

Preferably, the method further comprises: and verifying the behavior tree obtained by inference, wherein the verification rule comprises the following steps: checking whether the logic of the behavior tree is correct or not according to the international specification of the behavior tree; and judging whether the task requirements can be met, namely, the object examples really exist in the scene, and the robot can execute the actions described in the behavior tree to complete the task.

Preferably, the method further comprises: and converting the successfully verified behavior tree into a data structure, transmitting the data structure to a behavior execution center of the robot, and driving the robot by using the execution logic and the behavior parameters of the behavior tree.

In order to achieve the above object, an embodiment of the present invention further provides a device for reasoning tasks in a dynamic scene based on a knowledge base, including a dividing unit, a searching unit, and a reasoning unit:

the dividing unit is used for acquiring a semantic network expressing a knowledge base and dividing the semantic network into an action relation sub-network, a state change relation sub-network, a dependency relation sub-network and a preposition relation sub-network according to the relation type;

the searching unit is used for searching a behavior tree for the target task in the task planning graph;

the reasoning unit is used for performing at least one of target object reasoning and task reasoning and state and perception related reasoning under the semantic network when the behavior tree corresponding to the target task cannot be searched in the task planning graph.

Preferably, the inference unit comprises a target object inference module, a task inference module and a state and perception related inference module;

the target object reasoning module is used for determining an action set of the target task and reasoning a target object in the action relation sub-network, the dependency relation sub-network and the preposition relation sub-network according to actions in the action set;

the task reasoning module is used for searching paths in a semantic network according to two end points of the missing segments of the behavior tree, determining the missing segments according to the searching result and realizing task reasoning;

the state and perception related reasoning module is used for searching and determining the ontology of the state in the state change relation sub-network and the subordination relation sub-network, triggering the perception device to acquire perception data related to the ontology according to the ontology, and reasoning according to the perception data.

Compared with the prior art, the invention has the advantages that at least the following steps are included;

on the basis that a semantic network expressing a knowledge base is divided into an action relation sub-network, a state change relation sub-network, a subordination relation sub-network and a preposition relation sub-network according to relation types, reasoning of behavior tree missing judgment of tasks is conducted according to the action relation sub-network, and the reasoning acquisition of missing data such as target objects and perception data related to states is achieved through all the sub-networks, so that reasoning of a behavior tree is achieved. The method can avoid using task knowledge or a structured robot task design language, completes inference planning of tasks by using a universal semantic network, and realizes the operation problem of the robot.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a flowchart of a method for reasoning tasks in a knowledge base based dynamic scenario, according to an embodiment;

FIG. 2 is a schematic structural diagram of a robot decision-making system provided by an embodiment;

FIG. 3 is an exemplary task tree provided by an embodiment;

FIG. 4 is a schematic diagram illustrating the occurrence of tasks in a behavior tree according to an embodiment;

FIG. 5 is another flowchart of a method for reasoning tasks in a knowledge base based dynamic scenario, according to an embodiment;

FIG. 6 is a schematic diagram of a device for managing tasks in a knowledge-base-based dynamic scene according to an embodiment;

fig. 7 is a schematic structural diagram of an inference unit provided in the embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.

When the existing robot uses a task planning mode controlled by a behavior tree, the following problems can be encountered: (1) The task decomposition exceeds the prefabricated decomposition range, so that the task planning fails; (2) And the target is lost or the accurate target has no problem in the task planning process. In order to solve the problem, the embodiment provides a method and a device for reasoning tasks in a dynamic scene based on a knowledge base, and the core idea is to improve the task decomposition, decision and control of the robot through a semantic network expressing the knowledge base and enhance the working capacity of the robot in an unknown environment.

The semantic network is an expression form of a knowledge base, provides a knowledge set of subject, predicate, object and subject combination, and also has a special calculation and query engine for providing knowledge. The whole semantic network presents a graph-like structure and consists of nodes and connecting edges.

The precondition of the method for realizing inference task under dynamic scene based on knowledge base includes: the system has a complete semantic network of the robot field, the family field and the environment field, converts the environment of the robot into a scene graph, and stores the scene graph in a knowledge graph mode. The behavior tree planning strategy uses a prefabricated task planning graph to perform behavior decomposition, and specific information in a scene is inquired on an execution node to complete the generation of the behavior tree, so that the behavior tree is finally realized.

Based on the precondition, in the method for reasoning tasks in a dynamic scene based on a knowledge base, when a task which cannot be decomposed is encountered in a behavior tree planning process, the method returns to a feasible reasoning scheme in a semantic network, and obtains the latest result through comprehensive sequencing; a complete behavior tree and task planning scheme is completed by combining a knowledge base, and as shown in fig. 1, the method specifically comprises the following steps:

s1, obtaining a semantic network expressing a knowledge base, and dividing the semantic network into an action relation sub-network, a state change relation sub-network, a subordination relation sub-network and a preposition relation sub-network according to relation types.

In an embodiment, the knowledge base provides attribute information in a robot scene, including ontology features of the robot, environmental motion information of the robot, motion control knowledge of the robot, behaviors of the robot, and the like. And converting the knowledge base into a semantic network Graph, and determining the description of the main meaning, the table language and the state of the semantics in the semantic network Graph.

In order to speed up the construction of arbitrary inference and behavior trees, the semantic network Graph needs to be classified into sub-networks. The semantic network Graph is point-edge relation data expressed by a language, wherein the edge relation can be described as an action relation v expressing actions, a subordinate relation o expressing subordinate relations, a preposition relation p expressing prepositions, and a state change relation s expressing states, and therefore the semantic network Graph is divided into an action relation sub-network Graph-v expressing action relations, a state change relation sub-network Graph-s expressing state change relations, a subordinate relation sub-network Graph-o expressing object subordinate relations and a preposition relation sub-network Graph-p expressing prepositions according to the relations. Wherein, the sub-network Graph-o of the dependency relationship is used as an ontology library for searching the ontology.

In an embodiment, after the action relationship sub-networks are obtained by dividing, the action relationship sub-networks are processed to construct a mapping from a task to an action set, including: the actions associated with the single task are marked out in the action relation sub-network, the actions associated with the combined task are marked out in the action relation sub-network, then task semantic clustering is carried out on the marked actions, all the actions contained in each cluster correspond to one task semantic, and therefore mapping from each task to the action set is obtained, and further the mapping set is obtained.

Any of the verbs has a probabilistic association with an action primitive of the robot, such as the verb "deliver" is associated with a robot executable action primitive "send", where an action primitive refers to an action that is not repartitionable for a robot task plan, and a single verb primitive may be considered a single task. Other verbs may also be associated with the combined tasks of the action primitives of the robot. In an embodiment, verbs v of the above two types (relationship to action primitives, relationship to task combinations) are marked in Graph-v. Because each association is not unique, a semantic clustering algorithm is used to classify the verb into an action primitive, and the verb corresponding to an action primitive is mapped to a semantic action set, such as task- > { Verbs }, wherein Verbs represent all action sets that can be mapped to task.

In the embodiment, the sub-networks are added into the robot decision system based on the sub-networks divided above, and the robot decision system with the sub-networks is used for realizing task reasoning in a dynamic scene. As shown in FIG. 2, the system comprises a plurality of perception sensors, a data backboard, a task construction algorithm, a semantic network, graph-v, graph-s, graph-p, graph-o and a task planning chart. The sensing sensor is used for collecting sensing example data in a robot scene, and the data backboard is used for converting the sensing example data into an execution parameter table of the behavior tree and recording data changes executed by the behavior tree. The task construction algorithm comprises a method for constructing a behavior tree through Graph reasoning. The task planning graph is a huge task planning network, and a subtask is searched by issuing a specific task to complete the construction of the behavior tree.

In the system, the planning of the behavior and the action of the robot in the dynamic scene is realized based on the target task T and the action object O, so that the inference planning of a simple task (a task and a target) is completed by using a semantic network as the input of inference.

And S2, searching a behavior tree corresponding to the target task in the task planning graph.

In the embodiment, when performing inference planning of a target task, searching a behavior tree corresponding to the target task in a task planning graph, including: classifying a target Task into one Task node in a Task planning graph, searching all connected subtask nodes in sequence from the Task node downwards, and forming a Task Tree Task-Tree by all obtained subtask nodes and edges, wherein if any Task node cannot be classified in the Task planning graph or any subtask node cannot be searched, the Task is considered to fail and the Task needs to be carried out in a semantic network.

In an embodiment, it is further determined whether all subtasks in the Task Tree Task-Tree can be implemented by the robot, and specifically, the determination is performed by comparing the subtasks with the tasks that can be implemented by the robot that are planned in advance. After the subtasks which can be actually executed by the task tree are determined, the action set of the subtasks is determined according to the mapping from the task to the action set, a behavior tree is constructed according to the action set of the subtasks, and the nodes in the behavior tree represent actions.

And S3, when the behavior tree corresponding to the target task cannot be searched in the task planning graph, performing at least one of target object reasoning and task reasoning and state and perception related reasoning under the semantic network.

In the embodiment, when the behavior tree corresponding to the target task cannot be searched in the task planning graph, the semantic network is adopted for calculation and reasoning, so that the problem from the task tree to the behavior tree is solved. In the specific implementation process, from the aspects of breadth and depth, jump search is carried out in four sub-networks, namely graph-s, graph-o, graph-p and graph-v, so as to find the relation between actions and actions, tasks and tasks, tasks and target objects and objects, and realize at least one of target object reasoning, task reasoning and state and perception related reasoning.

And aiming at the condition that the target object cannot be determined when the behavior tree of the target task is constructed, adopting target object reasoning to determine the target object. And reasoning the target object, determining an action set of the target task, and reasoning the target object in an action relation sub-network, a dependency relation sub-network and a preposition relation sub-network according to the actions in the action set.

In the embodiment, firstly, according to the mapping set, a task matched with the target task is searched in the mapping set, and the action set mapped by the task is taken as the action set of the target task. Then, reasoning about the target object in the action relation sub-network and the dependency sub-network according to the actions in the action set, including: and determining an object of action operation in the action relation sub-network according to each action, performing multi-stage expansion of the object in the subordination relation sub-network according to the object to obtain a newly added target object, and performing screening and filtering operation on the newly added target object according to the perception condition.

In an embodiment, the multi-stage expansion of the object in the dependency sub-network according to the object includes: and searching the sub-class object having the subordination relation with the object in the subordination relation sub-network as the new added target object. In one embodiment, starting from graph-v, the operation target of each action v is determined to be a noun object O in graph-v in parallel, and starting from O, the graph-O is entered into graph-O for generalization, namely, a set { O } formed by newly-added objects is obtained by taking all other objects with O subordination with O in graph-O generalization as the newly-added objects, and the set { O } is returned to graph-v, so that the search return of the objects based on graph-v is expanded from a (v, O) combination to (v, { O }. Every time when graph-v performs target object expansion through graph-o, the reference count is incremented by 1, after which the reference count will be used to sort to screen the target objects.

In an embodiment, the performing of the multi-stage expansion of the object in the dependency sub-network according to the object further includes: and searching the parent object having an affiliation with the object in the affiliation sub-network, and taking other child objects of the parent object as the new target object. In one embodiment, for the parallel search for determining multiple target objects, starting with a target object O, all parent objects of the target object O are searched in graph-O, and then child objects of the parent objects are used as new target objects to construct a new set { O }, which is equal to the scene object being subjected to search expansion through graph-O. The mode of determining the newly added target object based on the parent object can be repeatedly used, and the range of the set { O } is gradually enlarged.

In the embodiment, after the newly added target objects are searched for in parallel, the newly added target objects which cannot appear in the scene are deleted according to the perception object examples in the scene. Specifically, after a newly added target object is determined based on a parent object, an instance in scene perception needs to be checked for the new target object, and objects which cannot exist in a scene are removed from the set { O } in time; each time the call determines the new target object based on the parent object, a reference count plus one operation is performed.

And when the target object set { O } cannot be inferred in the graph-O, adopting the preposition state of the graph-p for inference. The preposition relation p describes the position dependence relation of the space, and only the conditions of 'including', 'up, down, left and right', 'containing', and the like are inferred when the object position is inferred. Reasoning of the target object according to prepositions in graph-p, comprising: and (4) performing at least 3-hop search in graph-p according to the preposition relation on the basis of the object operated by the action, wherein the object searched in each hop is used as a newly added target object. In one embodiment, the corresponding next hop node { O-P } in graph-P according to the preposition relation P may represent that objects are close or dependent on space, and similar objects existing in the scene are searched for in { O-P }; when a similar object is not present, the process is repeated with each element O in { O-p } as the target and the search hop count length is limited to less than 3.

Therefore, substitute objects and similar objects in the mission planning can be searched and obtained through the graph-v, the graph-o and the graph-p so as to assist the further calculation of the mission planning decision. In the embodiment, the newly added target objects determined in the graph-o are sorted from small to large according to the reference count, so that the final result is the hop count search result as few as possible, and the target is prevented from being too different from the expected result.

When the task decomposition can not be completely carried out in the task planning graph, the task reasoning is required, and the subgraph and the part planned by the composite behavior tree are searched in the semantic network to realize the completion of the missing part. As shown in FIG. 4, root is the root node, b1-b6 are the behaviors, and the question mark is the missing part.

In the embodiment, the task reasoning comprises the steps of carrying out path search in the action relation subnetwork according to two end points of the behavior tree missing segment, determining the missing segment according to a search result, and realizing the task reasoning. The method specifically comprises the following steps:

because the behavior tree is a directed acyclic graph, the behavior tree containing the missing segments is mapped into a sub-semantic network, wherein the behaviors of the behavior tree are mapped into the relationships in the sub-semantic network, and the action targets of the behaviors are mapped into nodes of the sub-semantic network; finding out actions of two end points of a missing segment in a sub-semantic network, finding out two objects O1 and O2 corresponding to the actions of the two end points in graph-v according to the actions of the two end points, and forming a path set { path } by a plurality of communication paths between the two objects in graph-v search, wherein each communication path (O1- > R-O2) represents a series of action plans R between the two objects; and judging whether each communication path can be matched with the task planning graph according to a matching rule, wherein when the specific judgment is carried out, the shortest path in the { path } is used as the starting point, and the matching rule is as follows: all edge relationships in a communication path must be able to be classified into a subtask or action plan; the nodes in the communication path must have instances in the scene or have instances in the scene through first-order generalization in graph-o; screening the shortest communication path which meets the matching rule, converting the middle edge of the shortest communication path into a behavior, converting the node in the shortest communication path into a real object in the scene, and reversely mapping the real object into a behavior tree segment which is a missing segment of the behavior tree; and supplementing the missing segments of the behavior tree to the part which cannot be decomposed in the behavior tree, and completing the task reasoning of the behavior tree.

In the task planning process, when a task action target is met and a precondition is required, the condition is an environment perception parameter, and the parameter cannot be found in current perception data, state and perception related reasoning needs to be carried out. In an embodiment, the state and perception related reasoning comprises: the ontology of the determined state is searched in the state change relation sub-network and the subordination relation sub-network, the sensing device is triggered according to the ontology to acquire sensing data related to the ontology, and reasoning is carried out according to the sensing data.

Searching content related to a fixed language in a semantic network graph-s based on a target object O and a corresponding unknown sensing parameter label, establishing a mapping relation between a fixed language table and a scene in a fixed language expression, triggering feedback if the current state cannot complete state constraint, searching a knowledge graph body, and finding out perception which has consistent attributes and can meet the current behavior planning set. In one embodiment, after searching the state s related to the target object O in the graph-s, searching an ontology of the state s in the graph-O, in the presence of the ontology, linking to the corresponding sensing device according to the parameter description of the ontology, and obtaining instantiation data of the sensing device on the ontology parameters, wherein the label of the instantiation data is the unknown sensing parameter label, so that the behavior parameters are filled by querying the instantiation data under the label, and the inference of missing information is completed.

As shown in fig. 5, on the basis of the foregoing S1-S3, the method for reasoning tasks in a dynamic scene based on a knowledge base according to an embodiment further includes the following steps:

and S4, checking the behavior tree obtained by inference.

In an embodiment, the check rule for checking the behavior tree obtained by inference includes: (1) Checking whether the logic of the behavior tree is correct or not according to the international specification of the behavior tree; (2) And judging whether the task requirements can be met, namely, the object instance really exists in the scene, and the robot can execute the action described in the behavior tree to complete the task.

And S5, driving the robot according to the behavior tree which is verified successfully.

In the embodiment, the successfully verified behavior tree is converted into a data structure such as XML or JSON, and then the data structure is transmitted to a behavior execution center of the robot, and the robot is driven by using the execution logic and the behavior parameters of the behavior tree.

Based on the same inventive concept, as shown in fig. 6, the embodiment further provides a device 600 for reasoning tasks in a dynamic scene based on a knowledge base, which includes a dividing unit 610, a searching unit 620, and a reasoning unit 630: the dividing unit 610 is configured to obtain a semantic network expressing a knowledge base, and divide the semantic network into an action relation sub-network, a state change relation sub-network, a dependency relation sub-network, and a preposition relation sub-network according to a relation type; the searching unit 620 is used for searching a behavior tree for the target task in the task planning graph; the inference unit 630 is configured to perform at least one of inference of target object inference and task inference under a semantic network and inference related to state and perception when a behavior tree corresponding to a target task cannot be searched in the task planning graph.

As shown in fig. 7, the inference unit 630 includes a target object inference module 710, a task inference module 720, and a state and perception related inference module 730; the target object reasoning module 710 is configured to determine an action set of the target task, and to reason about the target object in the action relation sub-network, the dependency relation sub-network, and the preposition relation sub-network according to the action in the action set; the task reasoning module 720 is used for searching paths in the semantic network according to two end points of the missing segment of the behavior tree, determining the missing segment according to the search result, and realizing task reasoning; the state and perception related reasoning module 730 is used for searching and determining the ontology of the state in the state change relationship sub-network and the subordination relationship sub-network, triggering the perception device to acquire perception data related to the ontology according to the ontology, and performing reasoning according to the perception data.

It should be noted that, when the apparatus for reasoning a task in a dynamic scene based on a knowledge base provided in the foregoing embodiment performs task reasoning, the division of the functional units and modules is used as an example, and the function distribution may be completed by different functional units and modules as needed, that is, the internal structure of the terminal or the server is divided into different functional units and modules, so as to complete all or part of the functions described above. In addition, the apparatus for reasoning about tasks in a knowledge base-based dynamic scene provided in the above embodiments and the method for reasoning about tasks in a knowledge base-based dynamic scene belong to the same concept, and the specific implementation process thereof is described in detail in the method for reasoning about tasks in a knowledge base-based dynamic scene, and is not described herein again.

The embodiment provides a method and a device for reasoning tasks in a dynamic scene based on a knowledge base. The robot task planning method can avoid using task knowledge or a structured robot task design language, completes the inference planning of tasks by using a universal semantic network, realizes the operation problem of the robot, does not need excessive human editing and intervention in the inference process, and can complete the autonomous control of the robot behavior by only providing a complete semantic network and real-time perception data as a technical user.

The technical solutions and advantages of the present invention have been described in detail in the foregoing detailed description, and it should be understood that the above description is only the most preferred embodiment of the present invention, and is not intended to limit the present invention, and any modifications, additions, and equivalents made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims (14)

1. A method for reasoning tasks under a dynamic scene based on a knowledge base is characterized by comprising the following steps:

acquiring a semantic network expressing a knowledge base, and dividing the semantic network into an action relation sub-network, a state change relation sub-network, a dependency relation sub-network and a preposition relation sub-network according to the relation types;

when the behavior tree corresponding to the target task cannot be searched in the task planning graph, carrying out at least one of target object reasoning and task reasoning and state and perception related reasoning under the semantic network;

determining an action set of the target task aiming at the target object reasoning, and reasoning the target object in an action relation sub-network, a dependency relation sub-network and a preposition relation sub-network according to the action in the action set;

aiming at task reasoning, path searching is carried out on the action relation sub-network according to two end points of the behavior tree missing segment, the missing segment is determined according to a searching result, and the task reasoning is realized;

and aiming at the state and perception related reasoning, searching and determining the ontology of the state in the state change relation sub-network and the subordination relation sub-network, triggering the perception device to acquire perception data related to the ontology according to the ontology, and reasoning according to the perception data.

2. The method for reasoning tasks under a dynamic scene based on a knowledge base of claim 1, wherein after the sub-network of action relationships is obtained through division, a mapping from the tasks to the action set is constructed, and the method comprises the following steps:

the actions associated with the single task are marked out in the action relation sub-network, the actions associated with the combined task are marked out in the action relation sub-network, then task semantic clustering is carried out on the marked actions, all the actions contained in each cluster correspond to one task semantic, and therefore mapping from each task to the action set is obtained, and further the mapping set is obtained.

3. The method of reasoning about tasks in a knowledge base based dynamic scenario as claimed in claim 2, wherein the determining the set of actions for the target task comprises searching for a task matching the target task in the mapping set and using the task-mapped set of actions as the target task set of actions.

4. The method for task inference in a dynamic context based on a knowledge base of claim 1, wherein the inference of the target object in the action relation sub-network, the dependency sub-network based on the actions in the set of actions comprises:

and determining an object of action operation in the action relation sub-network according to each action, performing multi-stage expansion of the object in the subordination relation sub-network according to the object to obtain a newly added target object, and performing screening and filtering operation on the newly added target object according to the perception condition.

5. The method for reasoning about tasks in a dynamic scene based on a knowledge base of claim 4, wherein the object-dependent multi-level extension of objects in the dependency sub-network comprises:

searching a sub-class object having an affiliation with the object in the affiliation sub-network as a newly added target object: and searching the parent object having an affiliation with the object in the affiliation sub-network, and taking other child objects of the parent object as the new target object.

6. The method for reasoning about tasks in a dynamic scene based on a knowledge base of claim 5, wherein the operation of filtering and filtering the newly added target objects according to the sensing situation comprises: after a newly added target object is searched through a parent object of the object, the newly added target object which cannot appear in the scene is deleted according to a perception object example in the scene.

7. The method of reasoning about tasks in a knowledge base based dynamic scenario as claimed in claim 1 or 4, wherein reasoning about the target object in the action relation sub-network, the dependency sub-network, and the preposition relation sub-network according to the actions in the set of actions comprises:

when the target object set cannot be inferred in the subordination relation subnetwork, the inference of the target object is carried out in the preposition relation subnetwork according to prepositions, and the inference comprises the following steps: and on the basis of the objects operated by the action, performing at least 3-hop searching in the preposition relation sub-network according to the preposition relation, wherein the objects searched in each hop are taken as newly-added target objects.

8. The method of claim 1, wherein the performing a path search in the sub-network of action relationships according to two endpoints of the missing segment of the behavior tree, and determining the missing segment according to the search result comprises:

mapping a behavior tree containing the missing segments into a sub-semantic network, wherein the behaviors of the behavior tree are mapped into relationships in the sub-semantic network, and action targets of the behaviors are mapped into nodes of the sub-semantic network;

finding the actions of two end points of a missing segment in the sub-semantic network, finding two objects corresponding to the actions of the two end points in the action relation sub-network according to the actions of the two end points, and searching a plurality of communication paths between the two objects in the action relation sub-network, wherein each communication path represents a series of action plans between the two objects;

and judging whether each communication path can be matched with the task planning graph according to a matching rule, wherein the matching rule is as follows: all edge relationships in a communication path must be able to be classified into a subtask or action plan; the nodes in the connected path have to have instances in the scene or are generalized to the instances in the scene in a first order in the dependency sub-network;

screening the shortest communication path which meets the matching rule, converting the edges in the shortest communication path into behaviors, converting the nodes in the shortest communication path into real objects in the scene to obtain the missing segments of the behavior tree,

and supplementing the missing segments of the behavior tree into the behavior tree to finish the task reasoning of the behavior tree.

9. The method for reasoning about tasks in a dynamic scenario based on a knowledge base of claim 1, wherein the triggering of the sensing device to collect the sensing data about the ontology by searching for the ontology for determining the status in the status change relationship subnetwork and the dependency relationship subnetwork comprises:

based on a target object and a corresponding unknown sensing parameter label, searching a state related to the target object in a state change relation sub-network, searching a state body in a subordinate relation sub-network, linking to a corresponding sensing device according to the parameter description of the body, and obtaining instantiation data of the sensing device on body parameters, wherein the label of the instantiation data is the unknown sensing parameter label, so that behavior parameters are filled by inquiring the instantiation data under the label, and the inference of missing information is completed.

10. The method for reasoning about tasks in a knowledge base based dynamic scenario as claimed in claim 2, further comprising: searching in the task planning graph to obtain a behavior tree corresponding to the target task, specifically comprising: classifying a target task into one task node in a task planning graph, searching all connected subtask nodes from the task node downwards in sequence, forming a task tree by all obtained subtask nodes and edges, determining an action set of the subtask according to the mapping from the task to the action set after determining the subtask which can be actually executed by the task tree, and constructing a behavior tree according to the action set of the subtask.

11. The method for reasoning about tasks in a knowledge base based dynamic scenario according to claim 1 or 10, further comprising: and verifying the behavior tree obtained by inference, wherein the verification rule comprises the following steps: checking whether the logic of the behavior tree is correct or not according to the international specification of the behavior tree; and judging whether the task requirements can be met, namely, the object examples really exist in the scene, and the robot can execute the actions described in the behavior tree to complete the task.

12. The method for reasoning about tasks in a knowledge base based dynamic scenario as claimed in claim 11, further comprising: and converting the successfully verified behavior tree into a data structure, transmitting the data structure to a behavior execution center of the robot, and driving the robot by using the execution logic and the behavior parameters of the behavior tree.

13. A device for reasoning tasks under a dynamic scene based on a knowledge base is characterized by comprising a dividing unit, a searching unit and a reasoning unit:

the dividing unit is used for acquiring a semantic network expressing a knowledge base and dividing the semantic network into an action relation sub-network, a state change relation sub-network, a dependency relation sub-network and a preposition relation sub-network according to the relation type;

the searching unit is used for searching a behavior tree for the target task in the task planning graph;

the reasoning unit is used for performing at least one of target object reasoning and task reasoning and state and perception related reasoning under the semantic network when the behavior tree corresponding to the target task cannot be searched in the task planning graph.

14. The knowledge base based device for task inference in dynamic scenarios as claimed in claim 13, wherein said inference unit comprises a target object inference module, a task inference module and a state and perception related inference module;

the target object reasoning module is used for determining an action set of the target task and reasoning a target object in the action relation sub-network, the dependency relation sub-network and the preposition relation sub-network according to actions in the action set;

the task reasoning module is used for searching paths in a semantic network according to two end points of the missing segments of the behavior tree, determining the missing segments according to the searching result and realizing task reasoning;

the state and perception related reasoning module is used for searching and determining the ontology of the state in the state change relation sub-network and the subordination relation sub-network, triggering the perception device to acquire perception data related to the ontology according to the ontology, and reasoning according to the perception data.

Images