CN110990594B - Robot space cognition method and system based on natural language interaction - Google Patents
Robot space cognition method and system based on natural language interaction Download PDFInfo
- Publication number
- CN110990594B CN110990594B CN201911207208.XA CN201911207208A CN110990594B CN 110990594 B CN110990594 B CN 110990594B CN 201911207208 A CN201911207208 A CN 201911207208A CN 110990594 B CN110990594 B CN 110990594B
- Authority
- CN
- China
- Prior art keywords
- distance
- target
- coordinate
- relation
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/387—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Library & Information Science (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a robot space cognition method and a system based on natural language interaction, comprising the following steps: establishing a spatial information corpus based on natural language expression, which comprises the following steps: target attribute feature description corpus and target position feature description corpus; converting the target attribute feature description corpus and the target position feature description corpus into a keyword array according to a preset grammar rule; judging the category to which the target object and the reference object belong and the spatial position calculation relation according to the object related characteristics contained in the keyword array, wherein the spatial position calculation relation comprises at least one of the following relations: a directional relationship of the target relative to the reference, a distance relationship of the target relative to the reference, and a topological relationship of the target relative to at least two references; and determining the coordinate range of the target object according to the calculation relation of the category to which the target object and the reference object belong and the space position so as to search the target object later. The invention can reduce the interaction frequency between the person and the robot and improve the man-machine interaction efficiency.
Description
Technical Field
The invention relates to the technical field of man-machine interaction, in particular to a robot space cognition method and system based on natural language interaction.
Background
Under unstructured environments such as planet detection and field rescue, a robot cannot understand all environment information, and the difficulty and efficiency of independently completing tasks still have great problems. The capability advantages of people in comprehensive perception, prediction judgment, space reasoning and the like can better make up for the defects, and man-machine cooperation is an effective method for executing tasks. The man-machine interaction mode is one of key points of the man-machine interaction process, and the natural friendly man-machine interaction mode can effectively improve the interaction level. The natural language is used as one of natural interaction modes, has 'non-restricted' and does not need to distort natural thinking and behavior modes to adapt to the requirements of the robot, and meanwhile, the requirements on the environment and equipment are not high, so that the natural language is suitable for the unstructured environment and is widely applied to the field of mobile robots.
The processing of spatial information, namely spatial cognition, is involved in the process of completing tasks by man-machine cooperation. Because of the huge difference of space cognition mechanisms between the robots, the robots are difficult to understand space information based on natural language expression, only can receive unidirectional structured control instructions, and the operation efficiency is greatly influenced by frequent low-speed interaction. To solve this problem, emphasis is placed on how the robot can understand the cognitive expression of the person for spatial information. In the prior art, the simulation of a human cognition process is realized based on a cognition theory framework, namely, the spatial reference framework selection and spatial reasoning characteristics in human language command and communication are obtained through experiments, and an attempt is made to construct a spatial cognition and reasoning module for a robot, but the task only relates to target positioning at a short distance, has a single scene, does not comprise spatial expression under a complex task, and has certain limitation. In the field of space information interaction oriented to man-machine coordination, a robot space cognition method meeting natural language interaction is needed.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to solve the technical problems that the space cognition mechanism among the robots has great difference, the robots are difficult to understand the space information based on natural language expression, only can receive the control instruction of unidirectional structuring, and the operation efficiency is greatly influenced by frequent low-speed interaction.
In order to achieve the above object, in a first aspect, the present invention provides a robot space recognition method based on natural language interaction, comprising the steps of:
establishing a spatial information corpus based on natural language expression, which comprises the following steps: target attribute feature description corpus and target position feature description corpus;
converting the target attribute feature description corpus and the target position feature description corpus into keyword arrays according to preset grammar rules; the keywords include: object name, reference object name, direction relationship, and distance relationship;
judging the category and the spatial position calculation relation of the target object and the reference object according to the object related characteristics contained in the keyword array, wherein the category comprises point objects and planar objects, and the spatial position calculation relation comprises at least one of the following relations: a directional relationship of the target relative to the reference, a distance relationship of the target relative to the reference, and a topological relationship of the target relative to at least two references;
and determining the coordinate range of the target object according to the target object, the category of the reference object and the space position calculation relation so as to search the target object later.
Optionally, the category to which the object belongs is determined by:
if the object to be judged is an independent object, abstracting the object to be judged into a punctiform object or a planar object, and considering the object as a punctiform object when the spatial position expression of the object or other objects except the object to be judged is not influenced;
if the area ratio of the object to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted into a punctiform object and the spatial position expression of the object is influenced or other objects except the object to be judged.
Optionally, according to the direction relation of the object relative to the reference object, solving the coordinate range of the object by the following steps:
when the reference object is a dot-shaped object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object to be positioned at an origin of a coordinate system, and obtaining a coordinate position set of the point-shaped object relative to the point-shaped reference object in different directions according to the constraint of a plurality of preset straight lines for any point-shaped object in a space;
when the reference object is a planar reference object, a minimum boundary rectangle model is used for determining the planar reference object and a minimum circumscribed rectangle thereof, and straight lines of four rectangle sides of the minimum circumscribed rectangle are used as dividing lines in all directions; determining a coordinate position set of the punctiform object relative to the planar reference object in different directions according to the dividing lines in all directions;
if two reference objects exist, the coordinate position range of the target object is determined according to different reference object azimuth descriptions, and then the intersection of the coordinate position range and the reference object is obtained.
Optionally, according to a distance relation between the target object and the reference object, the distance relation includes: quantitative distance, qualitative distance, or temporal distance; the coordinate range of the target object is solved by the following steps:
when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range;
when the distance relation is a qualitative distance, different distance thresholds are preset for distances of different granularity levels, and if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a qualitative distance range area;
when the distance relation is a time distance, converting the time distance into a quantitative distance, and then determining the coordinate range of the point-like reference object;
when the distance relation between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object are respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
Optionally, according to the distance relation and the direction relation of the object relative to the reference object, solving the coordinate range of the object by the following steps:
and solving the coordinate range of the target object according to the two constraint conditions of the distance relation and the direction relation of the target object relative to the reference object, and finally solving the intersection of the two coordinate ranges to determine the final coordinate range of the target object.
Optionally, according to the topological relation of the target object relative to at least two reference objects, solving the coordinate range of the target object by the following steps:
if the topological relation is that the target object is between two reference objects:
when the two reference objects are point-shaped reference objects, the target object is in a region range in which a line segment formed by connecting the two reference objects and a distance line segment are preset distances;
when the two reference objects are planar reference objects, determining the range of the target object according to the rectangular side of the minimum circumscribed rectangle of the two planar reference objects;
when the two reference objects are the dot-shaped reference object and the planar reference object, respectively, the range of the target object is determined according to the coordinates of the dot-shaped reference object and the rectangular side of the minimum circumscribed rectangle of the planar reference object.
In a second aspect, the present invention provides a robot spatial cognitive system based on natural language interaction, comprising:
the corpus establishing unit is used for establishing a spatial information corpus based on natural language expression, and comprises the following steps: target attribute feature description corpus and target position feature description corpus;
the keyword determining unit is used for converting the target attribute feature description corpus and the target position feature description corpus into a keyword array according to a preset grammar rule; the keywords include: object name, reference object name, direction relationship, and distance relationship;
the feature judging unit is used for judging the category to which the target object and the reference object belong and the spatial position calculating relation according to the object related features contained in the keyword array, wherein the category comprises point objects and planar objects, and the spatial position calculating relation comprises at least one of the following relations: a directional relationship of the target relative to the reference, a distance relationship of the target relative to the reference, and a topological relationship of the target relative to at least two references;
and the object coordinate determining unit is used for determining the coordinate range of the object according to the object, the category of the reference object and the space position calculation relation so as to search the object subsequently.
Optionally, the keyword determining unit determines the category to which the object belongs by: if the object to be judged is an independent object, abstracting the object to be judged into a punctiform object or a planar object, and considering the object as a punctiform object when the spatial position expression of the object or other objects except the object to be judged is not influenced; if the area ratio of the object to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted into a punctiform object and the spatial position expression of the object is influenced or other objects except the object to be judged.
Optionally, the object coordinate determining unit solves the coordinate range of the object by: when the reference object is a dot-shaped object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object to be positioned at an origin of a coordinate system, and obtaining a coordinate position set of the point-shaped object relative to the point-shaped reference object in different directions according to the constraint of a plurality of preset straight lines for any point-shaped object in a space; when the reference object is a planar reference object, a minimum boundary rectangle model is used for determining the planar reference object and a minimum circumscribed rectangle thereof, and straight lines of four rectangle sides of the minimum circumscribed rectangle are used as dividing lines in all directions; determining a coordinate position set of the punctiform object relative to the planar reference object in different directions according to the dividing lines in all directions; if two reference objects exist, the coordinate position range of the target object is determined according to different reference object azimuth descriptions, and then the intersection of the coordinate position range and the reference object is obtained.
Optionally, according to a distance relation between the target object and the reference object, the distance relation includes: quantitative distance, qualitative distance, or temporal distance; the target object coordinate determination unit solves the coordinate range of the target object by: when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range; when the distance relation is a qualitative distance, different distance thresholds are preset for distances of different granularity levels, and if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a qualitative distance range area; when the distance relation is a time distance, converting the time distance into a quantitative distance, and then determining the coordinate range of the point-like reference object; when the distance relation between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object are respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object are determined by intersection of the two reference object distance descriptions.
In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:
compared with the space cognition mode of the traditional robot based on receiving the unidirectional structural control instruction, the robot space cognition method and system based on the natural language interaction, provided by the invention, have the advantages that the cognition of the robot to the natural language is increased, the keywords in the natural language instruction are found out, the coordinate range of the target object is determined from the keywords, the robot can conveniently cognite the natural language, the interaction frequency between the human and the robot can be reduced, and the man-machine interaction efficiency is improved.
Compared with the traditional robot space cognition mode in which a person mainly sends a structured control instruction and needs to learn a large number of expression rules, the robot space cognition method and system based on natural language interaction provided by the invention only need to use a natural language expression mode to express space information, so that cognition load is reduced.
Drawings
FIG. 1 is a schematic flow chart of a robot space cognition method based on natural language interaction provided by the invention;
FIG. 2 is a schematic diagram of an eight-direction cone model provided by the invention;
FIG. 3 is a schematic diagram of a Minimum Bounding Rectangle (MBR) model provided by the present invention;
FIG. 4 is a schematic diagram of coordinate system transformation provided by the present invention;
FIG. 5 is a schematic diagram illustrating the distance relationship provided by the present invention;
FIG. 6 is a schematic diagram of a point-like reference description of only a topological relation description provided by the present invention;
FIG. 7 is a schematic diagram of a planar reference description of only a topological relationship description provided by the present invention;
FIG. 8 is a schematic diagram of a description of a "dot+plane" reference provided by the present invention for only topological descriptions;
fig. 9 is a schematic diagram of a robot space cognitive system based on natural language interaction provided by the invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. In addition, the technical features of the embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.
In order to solve the defects of the prior art, namely to solve the problem that the robot is difficult to recognize and understand spatial information based on natural language expression in a complex task scene, the invention provides a robot spatial recognition method meeting natural language interaction, and the robot can understand complex spatial information such as target description and positioning based on natural language expression. The specific technical route is as follows:
the robot space cognition method comprises a natural language processing module and a space cognition module, and is divided into five main steps.
(1) And (3) establishing a corresponding spatial information expression corpus which comprises target object attribute feature description and target position description corpus facing to the task scene.
(2) The natural language processing module converts the space information expressed based on the natural language into a related keyword array according to grammar rules and the corpus in the step 1, wherein the keyword array comprises a target object name, each dimension attribute of the target object, the type of the target object, a reference object name and the like.
(3) The space cognition module stores knowledge related to object features according to the keyword array, and judges the category to which the object belongs and the space position calculation type. The object belongs to the category comprising punctiform objects and planar objects, and the space position calculation type comprises a single (two) reference object+direction relation, a single (two) reference object+distance relation, a single (two) reference object+direction relation+distance relation and a two reference object+topology only relation.
(4) And the space cognition module calculates the space position according to different types of space relations.
(5) And the space cognition module completes the matching of the characteristics of the target object and the coordinates of the target object.
As shown in fig. 1, a schematic diagram of a technical route of the present robot spatial awareness method is shown. The system is divided into two main parts, namely a natural language processing module and a space cognition module. The natural language processing module comprises a spatial information corpus, processes spatial information expressed by natural language, and analyzes corresponding keywords. And the space cognition module completes the calculation and reasoning function of the space position. After the keywords are received, storing knowledge related to the object features according to the keyword array, judging the category of the reference object and the type of spatial position calculation related to the target object, carrying out spatial position reasoning calculation according to different types, and finally completing the matching of the target object features and the target object coordinates. The specific details are as follows:
1. corpus creation and natural language parsing
According to the task scene, after collecting a large amount of corpus expressed by space information through experiments, forming a space information corpus, obtaining corresponding natural language expression rules, and extracting important keywords from the corresponding natural language expression rules, wherein the method comprises the following steps: object name, reference object name, bearing relationship, distance relationship, and the like. On the basis, the analysis of natural language can be realized by writing a corpus covering the task process. For example, the expression "red flag 5 m to the left of stone" is natural language. "based on the expression rule, can be resolved into" object name: red flag, reference: stone, azimuth relationship: left, distance relationship: 5 meters ", table 1 shows part of the grammar rules.
Table 1 grammar rule example table
In table 1, "{ }" indicates a keyword, "[ ]" indicates an auxiliary word, and "|" indicates a relationship of OR.
2. And judging the object type and the space relation calculation type.
Since the position description of a person on a target object is related to the nature characteristics of the object itself, the objects in performing a spatial cognitive task are classified into two classes, namely a punctiform object and a planar object, with reference to related concepts in geography. If the object is an independent object, the area ratio belongs to an unimportant attribute, namely, the object can be regarded as a punctiform object when the punctiform object or a planar object does not influence the spatial position expression of the object or other objects; if the object area is relatively large, the object is regarded as a planar object when the spatial position expression of the object itself or other objects is greatly affected when the object is abstracted into a dot-shaped object.
Considering a space-aware scene, a target object to be positioned is generally regarded as a punctiform object, and a reference object may be a punctiform or planar object. The judgment of the category of the object mainly depends on a priori knowledge base of the robot, and the category of the object is obtained by matching the name of the target object or the reference object in the keyword array with the knowledge base.
The position description of the target object is an expression of the spatial relationship of different objects, namely the object position is commonly characterized by the target object, the reference object and the spatial relationship.
The reference object can be largely divided into a single reference object and two reference objects. The spatial relationship includes topological relationship, direction relationship and distance relationship.
The topological relation refers to the relation of connection, separation, adjacency and the like of each space element without considering the specific position. When a person describes the position of an object by using natural language, the description of the separation relationship mainly relates to the description of the separation relationship, basically without intersecting and containing equal topological relationship, and generally defaults to the separation relationship.
The directional relationship refers to the position of one object relative to another object or objects in space. In describing the directional relationship, at least three elements are generally required, namely a target object, a reference object and a reference frame used. The direction relationship may be divided into an absolute direction relationship and a relative direction relationship according to the division of the reference frame used. Absolute direction relations refer to world coordinate systems used in the description of direction relations, and relative direction relations refer to relative coordinate systems used in the description of direction relations.
The distance relation reflects the geometrical approaching degree between objects in different spaces, and quantitative distances are more commonly used in daily life, namely distances described by numerical values are adopted, and a measurement system which is set manually is generally used as a basis.
Qualitative classification can be largely divided into two categories, one category is described by the "degree adverbs + quantitative distances" such as about 5 meters; the other is described using only very close-to-equal adverbs.
The time distance is expressed as "moving mode+time".
The above results are summarized, and the calculation types of the spatial positions are classified into the following categories: (1) single (two) references + direction relationship; (2) single (two) references + distance relationship; (3) single (two) references + direction relationship + distance relationship; (4) two references + topology only.
3. Calculation of spatial position
(1) Single (two) reference object + direction relation
The direction relationship may be divided into an absolute direction relationship and a relative direction relationship. On the premise of using absolute direction relation: when the reference is a point-like reference, an eight-directional cone model is generally used, which divides the entire two-dimensional space plane into eight parts with directivity, as shown in fig. 2, with 45 degrees of separation in each two directions.
To accurately describe the directional relationship of the spatial objects, a coordinate system as shown in FIG. 2 is established. Assuming that the punctiform reference A is located at the origin O of the coordinate system, then for any punctiform object B in space, the reference is according to L 1 、L 2 、L 3 、L 4 Straight line constraintA set of coordinates thereof in different directions with respect to the point-like reference object a can be obtained as shown in table 2.
TABLE 2 schematic table of spatial relationship calculation of punctiform reference objects
When the reference is a planar reference, a Minimum Bounding Rectangle (MBR) model is used, as shown in fig. 3. In fig. 3, the hatched portion is a planar reference, which may be set as a, and the rectangle abcd is the smallest circumscribed rectangle, and then the straight line sides where the four rectangular sides ab, ac, bc, bd are located may be used as dividing lines in all directions.
A coordinate system is established as shown in fig. 3. The four vertices of the minimum bounding rectangle abcd of the planar reference object a are a (x) 1 ，y 1 )，b(x 1 ，y 2 )，c(x 2 ，y 2 )，d(x 2 ，y 1 ) For any point object B in space, according to L 1 、L 2 、L 3 、L 4 Constraint of the linear equation, a set of coordinates thereof in different directions with respect to the planar reference a can be obtained, as shown in table 3, where Dir () in table 3 represents its orientation. If a relative direction relationship is used, it is converted into an absolute direction relationship. I.e. complete the conversion from the relative coordinate system to the world coordinate system.
TABLE 3 schematic representation of the spatial calculation of the planar reference
And establishing a coordinate system by taking the starting point of the robot as the origin of the world coordinate system and taking the positive east direction as the positive x-axis direction and the positive north direction as the positive y-axis direction. Let the position of the robot at time t be (x t ，y t ) The deflection angle is θ (relative to the positive x-axis direction), as shown in fig. 4.
The position of the object P is the right front of the robot, and the coordinates thereof are in a coordinate system centered on the robotp'(x p '，y p ') now has to be converted to coordinates in the world coordinate system, assumed to be P (x) p ，y p ) The conversion formula of the two obtained by deduction according to the geometric relation is as follows:
similarly, when determining the origin coordinates (a, b) of the relative coordinate system, the deviation angle θ of the relative coordinate system with respect to the world coordinate system, the position coordinates (x p '，y p ') the coordinates P (x) of the point P in the world coordinate system can be obtained by conversion p ，y p ) The method comprises the following steps:
if two reference objects exist, the coordinate positions of the objects are respectively determined according to different reference object azimuth descriptions, and then the intersection of the two ranges is obtained.
(2) Single (two) reference + distance relationship
The distance relationships include quantitative distance, qualitative distance, and temporal distance. Since the distance relation is used more to describe the point-like reference, a simplified process is performed, considering that the descriptions related to the distance relation are all point-like references. Quantitative distance is a distance described by a numerical value, typically based on a manually set metric system. However, since the cognitive ability of the person to the space is not completely the same, the description of the partial quantitative distance has a large deviation, so that an error parameter d is required to be introduced when the position calculation of the distance relation is performed. In the natural language description, the distance between the punctiform target object A and the punctiform reference object B is D, and the actual distance D is 1 =d±d. From the relevant conclusions of the prior art, it can be approximated that:
d＝k×D (3)
k in the formula (3) is an error proportionality coefficient.
As shown in fig. 5, a is a dot-like reference object, and if the distance between the dot-like target object B and a is considered to be D, the gray area range is the actual area of B.
Qualitative distances can be broadly divided into descriptions using the term "degree adverb + quantitative distance" and descriptions using only the term degree adverb, such as "the ball is in close proximity to the cart".
For the first case, a calculation mode of the quantitative distance can be directly adopted, because when the quantitative distance is used for position calculation, the error condition is considered, and the error is not required to be calculated; for the second case, the concept of qualitative distance description framework is adopted, and distance relation division can be performed on different granularity levels according to different research tasks.
The granularity levels of four qualitative distance relationships, namely near, far and far, were introduced, and different qualitative distance relationships were quantized to set quantitative distances as shown in table 4:
table 4 qualitative distance relationship correspondence table
The time distance needs to be converted into a quantitative distance for calculation. Let the time distance be L, the speed be v, the time consumption be t, then there is a distance relationship:
L＝vt (4)
the movement speed v=2m/s of the introduced robot participates in calculation, and the time-distance relationship is quantified.
When the distance relation between two reference objects is used for describing the position of the target object, the coordinate ranges of the target object are respectively determined according to different reference object distance descriptions, and the two reference object distance descriptions are intersected.
(3) Single (two) references + direction + distance relationship
In the model, the relative direction relationship can be converted into an absolute direction relationship, and the qualitative distance relationship and the time distance relationship can be converted into quantitative distance relationships, so that the direction relationship and the distance relationship of different types are finally converted into an absolute direction relationship and a quantitative distance relationship for position calculation.
If the object M is located 5 meters in the east direction of the punctiform reference object a (0, 0), the coordinate constraint of M is:
wherein x and y respectively represent the horizontal and vertical coordinate values of the object M.
When the description is carried out by using the direction relation and the distance relation of the two reference objects, the two reference objects are respectively converted into an absolute direction relation and a quantitative distance relation, constraint conditions of the two reference objects are calculated, and the intersection of the two reference objects is obtained.
(4) Two references + topology only
Still other spatial relationships are not categorized into directional relationships and distance relationships, and only topological relationships are used in the spatial relationship representation, such as "ball between you and dolly," as discussed below.
If the two reference objects are point-like reference objects, as shown in fig. 6, the two point-like reference objects are respectively a and B, and the line segment AB is taken as the central axis, then the possible positions of the target object are around the line segment AB, and in order to quantitatively calculate the error parameter d is introduced, that is, the target object is in the region with the distance d from the off-line segment AB.
Since the slope k of the line AB is different, the expression of the formed region is also different, as shown in fig. 6. And respectively calculating the coordinate range sets of the object M under different slopes.
In FIG. 6 (a), there is A (x A ，y A )，B(x A ，y B ). The M coordinate range is:
in FIG. 6 (b), there is A (x A ，y A )，B(x B ，y A ). The M coordinate range is:
in FIG. 6 (c), there is A (x A ，y A )，B(x B ，y B ). The M coordinate range is:
wherein x is M ，y M Respectively, the horizontal and vertical coordinate values of the object M, and θ represents the deflection angle of the robot with respect to the world coordinate system.
If both the reference objects are planar reference objects, as shown in fig. 7, the target object is M, A, B is two planar reference objects, the rectangles abcd and efgh are the minimum circumscribed rectangles A, B, respectively, and the gray area in the figure is the position area of the target object M.
The location areas of M in different cases are calculated separately.
In fig. 7 (a), there is a (x) a ，y a )，b(x b ，y a )，c(x b ，y c )，d(x a ，y c )，e(x e ，y e )，f(x f ，y f )，g(x g ，y g )，h(x h ，y h ). The M coordinate range is:
in fig. 7 (b), a (x) a ，y a )，b(x b ，y a )，c(x b ，y c )，d(x a ，y c )，e(x e ，y e )，f(x f ，y f )，g(x g ，y g )，h(x h ，y h ). The M coordinate range is:
when the two reference objects are a dot-shaped reference object and a planar reference object, respectively, as shown in fig. 8, the dot-shaped reference object is a, the planar reference object is B, and the rectangle abcd is the minimum circumscribed rectangle of B. The two cases in fig. 8 can be classified according to the relative directions of a and B, and the gray area in the figure is the M range.
Has a (x) a ，y a )，b(x b ，y a )，c(x b ，y c )，d(x a ，y c )，A(x A ，y A ) The location areas of the object M in different situations are calculated separately.
In fig. 8 (a), there is an M coordinate range:
in fig. 8 (b), there is an M coordinate range:
4. object feature matching
And the space cognition module is used for matching and synthesizing the name, attribute characteristics and the object coordinate constraint range obtained through calculation of the target object. Assuming that the attribute set of object a is N, there are:
n= { name, color, size, shape, coordinates }
Considering the requirement of the robot for path planning, selecting the shortest point in the coordinate range as the end point to perform the path planning, moving and other processes, and performing related operations such as subsequent object searching and the like according to the coordinate range.
Fig. 9 is a schematic diagram of a robot space recognition system based on natural language interaction according to the present invention, as shown in fig. 9, the system includes: corpus establishing unit 910, keyword determining unit 920, feature judging unit 930, and target object coordinate determining unit 940.
A corpus establishing unit 910, configured to establish a spatial information corpus based on natural language expression, including: target attribute feature description corpus and target position feature description corpus;
the keyword determining unit 920 is configured to convert the target attribute feature description corpus and the target location feature description corpus into a keyword array according to a preset grammar rule; the keywords include: object name, reference object name, direction relationship, and distance relationship;
a feature determining unit 930, configured to determine, according to the object related features included in the keyword array, a category to which the target object and the reference object belong, and a spatial position calculating relationship, where the category includes a dot object and a planar object, and the spatial position calculating relationship includes at least one of the following relationships: a directional relationship of the target relative to the reference, a distance relationship of the target relative to the reference, and a topological relationship of the target relative to at least two references;
the target object coordinate determining unit 940 is configured to determine a coordinate range of the target object according to the target object, the category to which the reference object belongs, and the spatial position calculation relationship, so as to perform the target object search subsequently.
Alternatively, the keyword determination unit 920 determines the category to which the object belongs by: if the object to be judged is an independent object, abstracting the object to be judged into a punctiform object or a planar object, and considering the object as a punctiform object when the spatial position expression of the object or other objects except the object to be judged is not influenced; if the area ratio of the object to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted into a punctiform object and the spatial position expression of the object is influenced or other objects except the object to be judged.
Optionally, the object coordinate determining unit solves the coordinate range of the object by: when the reference object is a dot-shaped object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object to be positioned at an origin of a coordinate system, and obtaining a coordinate position set of the point-shaped object relative to the point-shaped reference object in different directions according to the constraint of a plurality of preset straight lines for any point-shaped object in a space; when the reference object is a planar reference object, a minimum boundary rectangle model is used for determining the planar reference object and a minimum circumscribed rectangle thereof, and straight lines of four rectangle sides of the minimum circumscribed rectangle are used as dividing lines in all directions; determining a coordinate position set of the punctiform object relative to the planar reference object in different directions according to the dividing lines in all directions; if two reference objects exist, the coordinate position range of the target object is determined according to different reference object azimuth descriptions, and then the intersection of the coordinate position range and the reference object is obtained.
Optionally, according to a distance relation between the target object and the reference object, the distance relation includes: quantitative distance, qualitative distance, or temporal distance; the target object coordinate determination unit solves the coordinate range of the target object by: when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range; when the distance relation is a qualitative distance, different distance thresholds are preset for distances of different granularity levels, and if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a qualitative distance range area; when the distance relation is a time distance, converting the time distance into a quantitative distance, and then determining the coordinate range of the point-like reference object; when the distance relation between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object are respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object are determined by intersection of the two reference object distance descriptions.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The robot space cognition method based on natural language interaction is characterized by comprising the following steps of:
establishing a spatial information corpus based on natural language expression, which comprises the following steps: target attribute feature description corpus and target position feature description corpus;
converting the target attribute feature description corpus and the target position feature description corpus into keyword arrays according to preset grammar rules; the keywords include: object name, reference object name, direction relationship, and distance relationship;
judging the category and the spatial position calculation relation of the target object and the reference object according to the object related characteristics contained in the keyword array; the category includes dots and facets, and the spatial position calculation relationship includes at least one of the following relationships: a directional relationship of the target relative to the reference, a distance relationship of the target relative to the reference, and a topological relationship of the target relative to at least two references;
determining a coordinate range of the target object according to the category of the target object and the reference object and the space position calculation relation so as to search the target object subsequently;
the method for determining the coordinate range of the target comprises the following steps: when the spatial position calculation relation is the direction relation of the object relative to the reference object, solving the coordinate range of the object according to the direction relation of the object relative to the reference object:
when the reference object is a dot-shaped object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object to be positioned at an origin of a coordinate system, and obtaining a coordinate position set of the point-shaped object relative to the point-shaped reference object in different directions according to the constraint of a plurality of preset straight lines for any point-shaped object in a space;
when the reference object is a planar reference object, a minimum boundary rectangle model is used for determining the planar reference object and a minimum circumscribed rectangle thereof, and straight lines of four rectangle sides of the minimum circumscribed rectangle are used as dividing lines in all directions; determining a coordinate position set of the punctiform object relative to the planar reference object in different directions according to the dividing lines in all directions;
if two reference objects exist, the coordinate position range of the target object is respectively determined according to different reference object azimuth descriptions, and then intersection of the coordinate position range and the coordinate position range is obtained.
2. The robot spatial awareness method based on natural language interactions of claim 1, wherein the method of determining the coordinate range of the object further comprises:
when the spatial position calculation relationship is a distance relationship of the object relative to the reference object, wherein the distance relationship comprises: the quantitative distance, the qualitative distance or the time distance, and according to the distance relation of the target object relative to the reference object, solving the coordinate range of the target object by the following steps:
when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range;
when the distance relation is a qualitative distance, different distance thresholds are preset for the distances of different granularity levels, and if the reference object is a point-shaped reference object, the distance of the point-shaped target object from the point-shaped reference object is a qualitative distance range area;
when the distance relation is the time distance, converting the time distance into a quantitative distance, and then determining the coordinate range of the point-like reference object;
when the distance relation between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object are respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
3. The robot spatial awareness method based on natural language interactions of claim 2, wherein the method of determining the coordinate range of the object further comprises:
when the spatial position calculation relation is the distance relation and the direction relation of the object relative to the reference object, according to the distance relation and the direction relation of the object relative to the reference object, solving the coordinate range of the object by the following steps:
and solving the coordinate range of the target object according to the two constraint conditions of the distance relation and the direction relation of the target object relative to the reference object, and finally solving the intersection of the two coordinate ranges to determine the final coordinate range of the target object.
4. The robot spatial awareness method based on natural language interactions of claim 1, wherein the method of determining the coordinate range of the object further comprises:
when the spatial position calculation relationship is the topological relationship of the target object relative to at least two reference objects, according to the topological relationship of the target object relative to at least two reference objects, solving the coordinate range of the target object by the following steps:
if the topological relation is that the target object is between two reference objects:
when the two reference objects are point-shaped reference objects, the target object is in a region range in which a line segment formed by connecting the two reference objects and a distance line segment are preset distances;
when the two reference objects are planar reference objects, determining the range of the target object according to the rectangular side of the minimum circumscribed rectangle of the two planar reference objects;
when the two reference objects are the dot-shaped reference object and the planar reference object, respectively, the range of the target object is determined according to the coordinates of the dot-shaped reference object and the rectangular side of the minimum circumscribed rectangle of the planar reference object.
5. The robot space recognition method based on natural language interaction according to any one of claims 1 to 4, wherein the category to which the object belongs is determined by:
if the object to be judged is an independent object, abstracting the object to be judged into a punctiform object or a planar object, and considering the object as a punctiform object when the spatial position expression of the object or other objects except the object to be judged is not influenced;
if the area ratio of the object to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted into a punctiform object and the spatial position expression of the object is influenced or other objects except the object to be judged.
6. A robot spatial cognitive system based on natural language interactions, comprising:
the corpus establishing unit is used for establishing a spatial information corpus based on natural language expression, and comprises the following steps: target attribute feature description corpus and target position feature description corpus;
the keyword determining unit is used for converting the target attribute feature description corpus and the target position feature description corpus into a keyword array according to a preset grammar rule; the keywords include: object name, reference object name, direction relationship, and distance relationship;
the feature judging unit is used for judging the category to which the target object and the reference object belong and the spatial position calculating relation according to the object related features contained in the keyword array, wherein the category comprises point objects and planar objects, and the spatial position calculating relation comprises at least one of the following relations: a directional relationship of the target relative to the reference, a distance relationship of the target relative to the reference, and a topological relationship of the target relative to at least two references;
the target object coordinate determining unit is used for determining the coordinate range of the target object according to the calculation relation of the category of the target object and the reference object and the space position so as to search the target object subsequently;
the method for determining the coordinate range of the target object by the target object coordinate determining unit comprises the following steps: when the spatial position calculation relation is the direction relation of the object relative to the reference object, solving the coordinate range of the object according to the direction relation of the object relative to the reference object:
when the reference object is a dot-shaped object, an eight-direction conical model is adopted to divide the whole two-dimensional space plane into eight parts with directivity, and the interval between every two directions is 45 degrees; setting a point-shaped reference object to be positioned at an origin of a coordinate system, and obtaining a coordinate position set of the point-shaped object relative to the point-shaped reference object in different directions according to the constraint of a plurality of preset straight lines for any point-shaped object in a space;
when the reference object is a planar reference object, a minimum boundary rectangle model is used for determining the planar reference object and a minimum circumscribed rectangle thereof, and straight lines of four rectangle sides of the minimum circumscribed rectangle are used as dividing lines in all directions; determining a coordinate position set of the punctiform object relative to the planar reference object in different directions according to the dividing lines in all directions;
if two reference objects exist, the coordinate position range of the target object is respectively determined according to different reference object azimuth descriptions, and then intersection of the coordinate position range and the coordinate position range is obtained.
7. The robot space-based cognitive system of claim 6, wherein the method for determining the coordinate range of the object by the object coordinate determining unit further comprises:
when the spatial position calculation relationship is a distance relationship of the object relative to the reference object, wherein the distance relationship comprises: the quantitative distance, the qualitative distance or the time distance, and according to the distance relation of the target object relative to the reference object, solving the coordinate range of the target object by the following steps:
when the distance relation is a quantitative distance, if the reference object is a point-shaped reference object, the distance between the point-shaped target object and the point-shaped reference object is a quantitative distance and an error distance range;
when the distance relation is a qualitative distance, different distance thresholds are preset for the distances of different granularity levels, and if the reference object is a point-shaped reference object, the distance of the point-shaped target object from the point-shaped reference object is a qualitative distance range area;
when the distance relation is the time distance, converting the time distance into a quantitative distance, and then determining the coordinate range of the point-like reference object;
when the distance relation between the target object and two reference objects is used for describing the position of the target object, the coordinate ranges of the target object are respectively determined according to different reference object distance descriptions, and the coordinate ranges of the target object and the two reference objects are intersected to determine the final coordinate range of the target object.
8. The robot space-based cognitive system of claim 7, wherein the method for determining the coordinate range of the object by the object coordinate determining unit further comprises:
when the spatial position calculation relation is the distance relation and the direction relation of the object relative to the reference object, according to the distance relation and the direction relation of the object relative to the reference object, solving the coordinate range of the object by the following steps:
and solving the coordinate range of the target object according to the two constraint conditions of the distance relation and the direction relation of the target object relative to the reference object, and finally solving the intersection of the two coordinate ranges to determine the final coordinate range of the target object.
9. The robot space-based cognitive system of claim 6, wherein the method for determining the coordinate range of the object by the object coordinate determining unit further comprises:
when the spatial position calculation relationship is a topological relationship of the object relative to at least two reference objects, the method for determining the coordinate range of the object comprises the following steps: according to the topological relation of the target object relative to at least two reference objects, solving the coordinate range of the target object by the following steps:
if the topological relation is that the target object is between two reference objects:
when the two reference objects are point-shaped reference objects, the target object is in a region range in which a line segment formed by connecting the two reference objects and a distance line segment are preset distances;
when the two reference objects are planar reference objects, determining the range of the target object according to the rectangular side of the minimum circumscribed rectangle of the two planar reference objects;
when the two reference objects are the dot-shaped reference object and the planar reference object, respectively, the range of the target object is determined according to the coordinates of the dot-shaped reference object and the rectangular side of the minimum circumscribed rectangle of the planar reference object.
10. The robot space recognition system based on natural language interaction according to any one of claims 6 to 9, wherein the keyword determining unit determines the category to which the object belongs by:
if the object to be judged is an independent object, abstracting the object to be judged into a punctiform object or a planar object, and considering the object as a punctiform object when the spatial position expression of the object or other objects except the object to be judged is not influenced;
if the area ratio of the object to be judged is larger than the preset value, the object is regarded as a planar object when the object is abstracted into a punctiform object and the spatial position expression of the object is influenced or other objects except the object to be judged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911207208.XA CN110990594B (en) | 2019-11-29 | 2019-11-29 | Robot space cognition method and system based on natural language interaction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911207208.XA CN110990594B (en) | 2019-11-29 | 2019-11-29 | Robot space cognition method and system based on natural language interaction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110990594A CN110990594A (en) | 2020-04-10 |
| CN110990594B true CN110990594B (en) | 2023-07-04 |
Family
ID=70088801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911207208.XA Active CN110990594B (en) | 2019-11-29 | 2019-11-29 | Robot space cognition method and system based on natural language interaction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110990594B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114139069A (en) * | 2021-11-05 | 2022-03-04 | 深圳职业技术学院 | Indoor positioning method and system based on voice interaction and electronic equipment |
| CN114064940B (en) * | 2021-11-12 | 2023-07-11 | 南京师范大学 | Fuzzy position description positioning method based on super assignment semantics |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915039A (en) * | 2012-11-09 | 2013-02-06 | 河海大学常州校区 | Multi-robot combined target searching method of animal-simulated space cognition |
| CN108377467A (en) * | 2016-11-21 | 2018-08-07 | 深圳光启合众科技有限公司 | Indoor positioning and interactive approach, the device and system of target object |
| WO2018191970A1 (en) * | 2017-04-21 | 2018-10-25 | 深圳前海达闼云端智能科技有限公司 | Robot control method, robot apparatus and robot device |
| CN109614550A (en) * | 2018-12-11 | 2019-04-12 | 平安科技（深圳）有限公司 | Public sentiment monitoring method, device, computer equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4504433B2 (en) * | 2008-01-29 | 2010-07-14 | 株式会社東芝 | Object search apparatus and method |
| AU2018282316B2 (en) * | 2017-06-19 | 2020-07-16 | Beijing Didi Infinity Technology And Development Co., Ltd. | Systems and methods for displaying movement of vehicle on map |
| CN110019863B (en) * | 2017-12-26 | 2021-09-17 | 深圳市优必选科技有限公司 | Object searching method and device, terminal equipment and storage medium |
| CN108680163B (en) * | 2018-04-25 | 2022-03-01 | 武汉理工大学 | Unmanned ship path searching system and method based on topological map |
| CN109670262B (en) * | 2018-12-28 | 2021-04-27 | 江苏艾佳家居用品有限公司 | Computer-aided home layout optimization method and system |
| CN110110823A (en) * | 2019-04-25 | 2019-08-09 | 浙江工业大学之江学院 | Object based on RFID and image recognition assists in identifying system and method |
-
2019
- 2019-11-29 CN CN201911207208.XA patent/CN110990594B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915039A (en) * | 2012-11-09 | 2013-02-06 | 河海大学常州校区 | Multi-robot combined target searching method of animal-simulated space cognition |
| CN108377467A (en) * | 2016-11-21 | 2018-08-07 | 深圳光启合众科技有限公司 | Indoor positioning and interactive approach, the device and system of target object |
| WO2018191970A1 (en) * | 2017-04-21 | 2018-10-25 | 深圳前海达闼云端智能科技有限公司 | Robot control method, robot apparatus and robot device |
| CN109614550A (en) * | 2018-12-11 | 2019-04-12 | 平安科技（深圳）有限公司 | Public sentiment monitoring method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110990594A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yao et al. | Fuzzy critical path method based on signed distance ranking of fuzzy numbers | |
| CN110990594B (en) | Robot space cognition method and system based on natural language interaction | |
| CN111665842A (en) | Indoor SLAM mapping method and system based on semantic information fusion | |
| CN111243017B (en) | Intelligent robot grabbing method based on 3D vision | |
| Yongda et al. | Research on multimodal human-robot interaction based on speech and gesture | |
| CN111098301A (en) | Control method of task type robot based on scene knowledge graph | |
| Liu et al. | Target localization in local dense mapping using RGBD SLAM and object detection | |
| CN110889221B (en) | Method for establishing ship collision avoidance model based on non-Europe-preserving transformation | |
| CN110986956A (en) | Autonomous learning global positioning method based on improved Monte Carlo algorithm | |
| CN112099627A (en) | City design virtual reality instant interaction platform based on artificial intelligence | |
| Kassab et al. | Real-time human-UAV interaction: New dataset and two novel gesture-based interacting systems | |
| CN113168729B (en) | 3D shape matching method and device based on local reference coordinate system | |
| Li et al. | Multi-joint active collision avoidance for robot based on depth visual perception | |
| Zhang et al. | Dynamic energy-efficient path planning of unmanned surface vehicle under time-varying current and wind | |
| CN112596659B (en) | Drawing method and device based on intelligent voice and image processing | |
| CN114995384A (en) | Improved HHO algorithm AGV path planning fusing neural network | |
| CN109129450A (en) | A kind of artificial intelligence execution system applied to split-type robot | |
| CN110598751B (en) | Anchor point generation method based on geometric attributes | |
| Hou et al. | Straight skeleton based automatic generation of hierarchical topological map in indoor environment | |
| Meng et al. | Route planning for unmanned aerial vehicle based on rolling RRT in unknown environment | |
| Liu et al. | Estimated path information gain-based robot exploration under perceptual uncertainty | |
| Yi et al. | SA-MLP-Mixer: A Compact All-MLP Deep Neural Net Architecture for UAV Navigation in Indoor Environments | |
| CN108765437A (en) | A kind of autonomous mobile robot SLAM builds the data correlation matching process of figure | |
| Peng et al. | Autonomous Navigation for Mobile Robot | |
| Liu et al. | 6D pose estimation of object based on fused region-level feature in cluttered scenes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |