CN115326048A - Semantic map construction method based on wall corner family as main feature - Google Patents
Semantic map construction method based on wall corner family as main feature Download PDFInfo
- Publication number
- CN115326048A CN115326048A CN202210805086.XA CN202210805086A CN115326048A CN 115326048 A CN115326048 A CN 115326048A CN 202210805086 A CN202210805086 A CN 202210805086A CN 115326048 A CN115326048 A CN 115326048A
- Authority
- CN
- China
- Prior art keywords
- corner
- wall corner
- wall
- semantic
- family
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010276 construction Methods 0.000 title claims abstract description 19
- 238000000034 method Methods 0.000 claims description 17
- 230000000007 visual effect Effects 0.000 claims description 16
- 238000005259 measurement Methods 0.000 claims description 10
- 230000010365 information processing Effects 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 3
- 241000764238 Isis Species 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 239000000470 constituent Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3804—Creation or updating of map data
- G01C21/3807—Creation or updating of map data characterised by the type of data
- G01C21/383—Indoor data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3804—Creation or updating of map data
- G01C21/3833—Creation or updating of map data characterised by the source of data
- G01C21/3841—Data obtained from two or more sources, e.g. probe vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a semantic map construction method based on wall corner family as main features, which comprises the steps of distinguishing convexity and concavity of a wall corner; judging the directionality of the wall corner to obtain a wall corner category label; establishing a wall corner family of indoor wall corners; constructing a wall corner semantic graph according to the wall corner family, obtaining a wall corner family size chain, and storing the wall corner family size chain in a wall corner family information table; constructing a non-wall corner object semantic graph according to the non-wall corner semantic objects, obtaining a non-wall corner semantic object size chain, and storing the non-wall corner semantic object size chain in a non-wall corner semantic object information table; and traversing the scene to obtain a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and obtaining the semantic map with the wall corner family as the main characteristic by overlapping the origin and aligning the pose. The invention takes the wall corner family as the main characteristic, and obtains the dimension chain of the wall corner family, the dimension chain of the non-wall corner semantic object, the information relation between the wall corner and the adjacent non-wall corner semantic object according to the position relations, thereby determining the uniqueness and the certainty of each wall corner and the non-wall corner semantic object in the wall corner family.
Description
Technical Field
The invention belongs to the field of map building and positioning navigation of indoor mobile robots, and particularly relates to a semantic map building method based on wall corner families as main features.
Background
The difficulty in the indoor working environment that the corners naturally exist and are fixed in position is how to make full use of and distinguish each corner, so that each corner can independently provide positioning reference for the mobile robot. The identity of the corners is not determined according to unique intrinsic relations among the corners in the existing research, and although related research introduces the corners into semantic maps, the corners are isolated, the intrinsic relations among the corners are not found, and the corners cannot be obviously differentiated.
Disclosure of Invention
The semantic map constructed by the invention aiming at the current relevant research does not take the wall corner family as the main characteristic and does not deeply dig the internal connection between the wall corners in the transverse direction and the longitudinal direction, so that the wall corners cannot have uniqueness and certainty.
Aiming at the problems, the invention provides a semantic map construction method based on wall corner family as main features, which comprises the following steps:
distinguishing the convexity and concavity of the indoor wall corner;
judging the directivity of the indoor wall corner, and acquiring a wall corner class label based on the directivity judgment result;
establishing a wall corner family of the indoor wall corner by combining the acquired wall corner category label and distinguishing the convexity and concavity of the indoor wall corner;
constructing a wall corner semantic graph according to the wall corner family, obtaining a wall corner family size chain according to the relative position and the azimuth relation between the wall corner in the wall corner family and the adjacent wall corner, and storing the wall corner family size chain in a wall corner family information table; constructing a non-wall corner object semantic graph according to the non-wall corner semantic objects, obtaining a non-wall corner semantic object size chain according to the relative position and orientation relation between the non-wall corner semantic objects and adjacent non-wall corner semantic objects, and storing the non-wall corner semantic object size chain in a non-wall corner semantic object information table;
and traversing the scene to obtain a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and obtaining the semantic map with the wall corner family as the main characteristic through origin point coincidence and pose alignment.
Preferably, the obtaining of the size chain of the corner family according to the relative position and the orientation relation between the corner and the adjacent corner in the corner family comprises the following steps: and (4) sequentially distributing different numbers to the identified corners, and according to the sequence of constructing the corners, pointing the front corner to the rear adjacent corner to form a corner family size chain.
Preferably, in the constituent corner family size chain, a start point of the corner family size chain stores a category, a corner number, coordinates, and an angle with respect to an origin of the map of a previous corner, an end point of the size chain stores a category, a corner number, coordinates, and an angle with respect to an origin of the map of a next adjacent corner, and distance information, direction information, and relative angle information of two adjacent corners are stored.
Preferably, the obtaining the size chain of the non-wall corner semantic object according to the relative position and the orientation relationship between the non-wall corner semantic object and the adjacent non-wall corner semantic object comprises:
and allocating different numbers to the identified non-wall corner semantic objects in sequence, and pointing a previous non-wall corner semantic object to a next adjacent non-wall corner semantic object according to the sequence of constructing the non-wall corner semantic objects to form a non-wall corner semantic object size chain.
Preferably, the traversing scene obtains a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and obtains the semantic map with the wall corner family as a main feature through origin coincidence and pose alignment, and the method further includes the following steps:
determining a relation between a camera coordinate system and a laser radar coordinate system;
and replacing the camera depth data with the laser radar depth data, converting the laser radar coordinate system into the robot coordinate system, and finally converting the robot coordinate system into the world coordinate system.
Preferably, the traversing scene obtains a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and the obtaining of the semantic map with the wall corner family as the main characteristic through the origin coincidence and the pose alignment comprises the following steps:
mapping the coordinates of the points to a grid coordinate system; fusing odometer information, inertial measurement unit information, laser information processing and identification, visual information processing and identification to obtain a wall corner family semantic graph, a non-wall corner object semantic graph and a grid map in the process of traversing a scene;
and incrementally constructing the wall corner family semantic map, the non-wall corner object semantic map and the grid map by Bayesian estimation suitable for a dynamic scene, and obtaining the semantic map based on the wall corner family as the main characteristic through origin point coincidence and pose alignment.
Preferably, the distinguishing of the concavity and convexity of the indoor wall corner includes: judging the distance between the wall corner intersected with the two straight lines and the observation point and the distance between the position far away from the wall corner and the observation point, and if the distance between the position far away from the wall corner and the observation point is greater than the distance between the wall corner intersected with the two straight lines and the observation point, determining that the wall corner is a convex wall corner; and if the distance between the position far away from the corner and the observation point is less than the distance between the corner intersected with the two straight lines and the observation point, the corner is a concave corner.
Preferably, the performing the directionality judgment of the indoor wall corner and obtaining the wall corner category label based on the directionality judgment result includes the following steps:
in an indoor environment, selecting a wall as a reference, setting a course angle to be 0 DEG when the robot is vertical to the wall, and setting the right corner of the wall as a first type of corner;
acquiring a course angle of the robot, and judging the indoor wall angle directionality according to an included angle between the wall surface and the course angle;
and sequentially determining the wall corners as a second type of wall corner, a third type of wall corner and a fourth type of wall corner according to the counterclockwise direction of the wall corners.
Preferably, the step of establishing a corner family of the indoor corner by combining the obtained wall corner category label with the convex and concave characteristics of the indoor corner comprises: after laser radar discerns the corner, distinguish the unsmooth classification of corner, combine the corner to classify the corner clan of indoor corner for the azimuth of robot, wherein, the classification of corner clan includes: the first type of convex wall corner, the second type of convex wall corner, the third type of convex wall corner and the fourth type of convex wall corner; the first type of concave wall corner, the second type of concave wall corner, the third type of concave wall corner and the fourth type of concave wall corner.
The invention has the following advantages:
according to the invention, the depth camera, the inertia measurement unit and the laser radar are used for distinguishing the direction and the concave-convex property of the corner, and finally, rich corner category information is obtained. The uniqueness and the certainty of each wall corner in the wall corner family are researched, the internal relation between the wall corners is excavated transversely and longitudinally, and the semantic map construction method based on the wall corner family as the main characteristic is constructed. In the constructed semantic map, the position relations between the wall corners and the adjacent wall corners, between the non-wall corner semantic objects and the adjacent non-wall corner semantic objects and between the wall corners and the adjacent non-wall corner semantic objects are fully expressed, the wall corner family is made to be the main characteristic, and the size chain of the wall corner family, the size chain of the non-wall corner semantic objects and the information relation between the wall corners and the adjacent non-wall corner semantic objects are obtained according to the position relations, so that the uniqueness and the certainty of each wall corner and each non-wall corner semantic object in the wall corner family are determined.
Drawings
FIG. 1 is a flow chart of a semantic map construction method based on wall corner family as main features according to the invention;
FIG. 2 is a schematic diagram of a wall corner family in an indoor typical scene according to the present invention;
FIG. 3 is a table of information of corner families in an indoor typical scene according to the present invention;
FIG. 4 is a diagram illustrating a relationship between a dimension chain of a semantic object other than a corner in an indoor typical scene according to the present invention;
FIG. 5 is a diagram showing the relationship between the wall corner categories and the information of the adjacent non-wall corner semantic objects.
Detailed Description
In order to facilitate understanding and implementation of the invention by those of ordinary skill in the art, the invention is further described in detail with reference to the drawings and the implementation examples, and it is to be understood that the implementation examples described herein are only for illustration and explanation of the invention and are not to be construed as limiting the invention.
The specific implementation example of the present invention is a semantic map construction method based on a wall corner family as a main feature, as shown in fig. 1, the method includes the following steps:
step 1: distinguishing the convexity and concavity of the indoor wall corner;
step 2: judging the directivity of the indoor wall corner, and acquiring a wall corner class label based on the directivity judgment result;
and 3, step 3: establishing a wall corner family of the indoor wall corner by combining the acquired wall corner category label and distinguishing the convexity and concavity of the indoor wall corner;
and 4, step 4: obtaining a wall corner family size chain according to the relative position and the azimuth relation between a wall corner and an adjacent wall corner in the wall corner family, and storing the wall corner family size chain in a wall corner family information table; constructing a semantic map of the non-wall corner object according to the non-wall corner semantic object, obtaining a size chain of the non-wall corner semantic object according to the relative position and the azimuth relation between the non-wall corner semantic object and an adjacent non-wall corner semantic object, and storing the size chain of the non-wall corner semantic object in a non-wall corner semantic object information table;
and 5: and traversing the scene to obtain a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and obtaining the semantic map with the wall corner family as the main characteristic through origin point coincidence and pose alignment.
The invention provides a semantic map construction method based on a corner family as a main feature, which fully expresses the position relationship between a corner and an adjacent corner, between a non-corner semantic object and an adjacent non-corner semantic object and between the corner and an adjacent non-corner semantic object in a constructed semantic map, so that the corner family becomes the main feature, and obtains a corner family size chain, a non-corner semantic object size chain and a corner and adjacent non-corner semantic object information relationship according to the position relationship, thereby determining the uniqueness and certainty of each corner and non-corner semantic object in the corner family.
In specific implementation, the hardware platform on which the method of the invention depends comprises: the system comprises a main control machine, a robot platform chassis, a depth camera, an inertia measurement unit and a laser radar; and the master control machine is respectively connected with the robot platform chassis, the depth camera and the laser radar in sequence. Wherein,
the main control machine is a core I9 mini host; the robot platform chassis is an SCOUT MINI intelligent mobile chassis; the depth camera selects KinectV2 of Microsoft; the inertial measurement unit is selected from an Riifen scientific model of AH-100B; the laser radar sensor selects SICK lms111 laser radar with stable performance.
The semantic map construction method of the present invention will be further explained in detail;
in one embodiment, step 1: distinguishing the convexity and concavity of the indoor wall corner; the indoor wall corners can be divided into convex wall corners and concave wall corners according to convex-concave categories, and the convex and concave characteristics of the wall corners are distinguished through laser and visual fusion.
The method for distinguishing the convexity and concavity of the indoor wall corner comprises the following steps: judging the distance between the wall corner intersected with the two straight lines and the observation point and the distance between the position far away from the wall corner and the observation point, and if the distance between the position far away from the wall corner and the observation point is greater than the distance between the wall corner intersected with the two straight lines and the observation point, determining that the wall corner is a convex wall corner; and if the distance between the position far away from the corner and the observation point is less than the distance between the corner intersected with the two straight lines and the observation point, the corner is a concave corner.
Specifically, the judging of the concavity and convexity of the indoor wall corner specifically includes: after the laser radar identifies the indoor wall corners, the wall corners are distinguished according to convex and concave categories, wherein one is the convex wall corner, and the other is the concave wall corner. If the wall corner where the two straight lines intersect is closest to the robot, and the laser point farther away from the wall corner is farther away from the robot, the laser point is a convex wall corner; if the wall corner where the two straight lines intersect is farthest away from the robot, the laser point which is farther away from the wall corner is closer to the robot, and the wall corner is a concave wall corner.
In one embodiment, step 2: judging the directionality of the indoor wall corner, and acquiring a wall corner category label based on the directionality judgment result;
in particular, conventional indoor corners typically have four primary orientations, each with a unique label, and therefore the orientation determination for the corner. The method comprises the specific steps that the robot obtains a robot course angle through an inertia measuring unit, the robot performs linear fitting through a laser point of a laser radar hitting the wall surface by adopting an IEPF algorithm to obtain an included angle between the wall surface and the course angle obtained by the inertia measuring unit, in an indoor environment, one wall body is selected as a reference, when the robot is perpendicular to the wall body, the course angle is set to be 0 degrees, the right wall angle of the wall body is set to be a first wall angle, and the second wall angle, the third wall angle and the fourth wall angle are sequentially set according to the anticlockwise sequence. The course angle is increased according to the clockwise direction, the range is between 0 degree and 360 degrees, the course angle of the robot is represented by theta, and the maximum visual angle range of the camera depth map is represented by alpha;
when each type of corner does not have the overlapped course angle, the identified corner is the corresponding corner type, and in the range of the left and right limit visual angles of the camera, when alpha/2 < theta < 90-alpha/2, the visual angle range of the camera is seen as the first type of corner; when 270 ° + α/2< θ <360 ° - α/2, the camera view range is seen as a second type of wall angle; when 180 ° + α/2< θ <270 ° - α/2, the camera view range is seen as a third type of wall angle; when 90 ° + α/2< θ <180 ° - α/2, the camera view range is seen as a fourth type of wall angle.
And when the overlapped course angle exists, judging whether the wall angle is positioned on the left side or the right side of the course angle. If the corner is located on the left side or the right side of the course angle and only has one corner type, the corner is the corresponding corner type.
If there are two kinds of wall angles on the left side or the right side of the course angle, calculating the included angle beta between the course angle and the side wall surface of the wall angle, wherein when beta is less than alpha/2, the included angle beta corresponds to one kind of wall angle type, and when beta is more than 90 degrees-alpha/2, the included angle beta corresponds to the other kind of wall angle type. And determining the specific wall corner type by combining the included angle beta between the course angle and the corresponding wall surface within the range set by the course angle. Under the left and right limit visual angles of the camera, when 360 degrees to alpha/2 degrees < theta <360 degrees, the first wall angle and the second wall angle are both positioned on the right side of the course angle, at the moment, an included angle beta between the course angle and the right wall surface where the wall angle is positioned is calculated, if the beta is < alpha/2, the visual angle range of the camera is seen as the first wall angle, and if the beta is greater than 90 degrees to alpha/2, the visual angle range of the camera is seen as the second wall angle; when the angle is 0 degrees < theta < alpha/2, the first wall angle and the second wall angle are both positioned on the left side of the course angle, an included angle beta between the course angle and a left wall surface where the wall angle is positioned is calculated, if the included angle beta is less than alpha/2, the visual angle range of the camera is seen as the second wall angle, and if the included angle beta is greater than 90 degrees-alpha/2, the visual angle range of the camera is seen as the first wall angle.
Specifically, the indoor wall corner is judged in the visual angle range of the camera KinectV2 and the course angle range acquired by the inertial measurement unit, and if the non-overlapping course angle range exists in each type of wall corner, the wall corner identified in the non-overlapping course angle range is the corresponding wall corner type.
Optionally, the robot is equipped with an inertial measurement unit to obtain the robot orientation, and the wall corner directionality is determined according to the robot orientation corresponding to the identified wall corner. The inertial measurement unit is corrected when the robot is used for the first time, one wall body is selected as a reference in an indoor environment, when the robot is perpendicular to the wall body, the course angle of the robot is set to be 0 degrees, the right corner of the wall body is set to be the first type corner, and the second type corner, the third type corner and the fourth type corner are sequentially set according to the anticlockwise sequence.
In one embodiment, step 3: and (4) combining the obtained wall corner category labels with the convex and concave characteristics of the indoor wall corner to distinguish, and establishing a wall corner family of the indoor wall corner.
Specifically, the robot combines the wall corner category labels obtained in the step 2 with the position relationship between the wall corners and the relationship between the wall corners and other non-wall corner semantic objects to realize that each wall corner has uniqueness. After the laser radar identifies the wall corners, concave-convex wall corners are distinguished, wherein one is a concave wall corner, and the other is a convex wall corner. The azimuth angles when the robot detects the wall corners are roughly divided into 4 types, and are distinguished by combining convex and concave wall corners, and the classification is as follows: the first type of convex wall corner, the second type of convex wall corner, the third type of convex wall corner and the fourth type of convex wall corner; the first kind of concave wall corner, the second kind of concave wall corner, the third kind of concave wall corner and the fourth kind of concave wall corner are preliminarily classified into 8 kinds for the indoor wall corner of a regular building, and the wall corner is called as a wall corner family. Specifically, fig. 2 is a schematic diagram of a wall corner group in an indoor typical scene.
In one embodiment, step 4: constructing a wall corner semantic graph according to the wall corner family, obtaining a wall corner family size chain according to the relative position and the azimuth relation between the wall corner in the wall corner family and the adjacent wall corner, and storing the wall corner family size chain in a wall corner family information table; constructing a non-wall corner object semantic graph according to the non-wall corner semantic objects, obtaining a non-wall corner semantic object size chain according to the relative position and orientation relation between the non-wall corner semantic objects and adjacent non-wall corner semantic objects, and storing the non-wall corner semantic object size chain in a non-wall corner semantic object information table;
specifically, in the construction of the wall corner semantic graph according to the wall corner family, the indoor wall corner family dimension chain, the non-wall corner semantic object dimension chain, the wall corner and the adjacent non-wall corner semantic object information relationship are combined with the odometer, the inertia measurement unit, the laser information and the visual information processing and recognition to obtain the wall corner family semantic graph and the non-wall corner object semantic graph when the semantic graph is constructed. Storing a wall corner category and a wall corner family size chain in a wall corner family semantic graph; and storing the non-wall corner semantic object category and the non-wall corner semantic object size chain in the non-wall corner object semantic map.
The relationship of the size chain of the corner family in the typical indoor scene is shown in fig. 3. In fig. 3, a portion of the corners 7, 10, 34, 47 are selected for analysis of the chain of corner types and corner family dimensions for the adjacent upstream and adjacent downstream corners shown in table 1.
TABLE 1 information Table of corner families in indoor typical scene
And storing the wall corner category and the wall corner family size chain in the wall corner family semantic graph. When the wall corner family semantic graph is constructed, different serial numbers are sequentially distributed to the identified wall corners according to the sequence, and the coordinates of the wall corners are stored in the wall corner family semantic graph. The corner family dimension chain is provided with a starting point and an end point, the direction of the corner family dimension chain is pointed to the next adjacent corner from the previous corner according to the sequence of constructing the corners, in the corner family dimension chain, the starting point of the dimension chain stores the category, the corner number, the coordinate and the angle relative to the map origin of the previous corner, and the end point of the dimension chain stores the category, the corner number, the coordinate and the angle relative to the map origin of the next adjacent corner and stores the distance information, the direction information and the relative angle information of the two adjacent corners.
In the constructed wall corner family semantic graph, the upper left corner of the map is assumed to be the original point of the map, the direction of x is the direction that the original point of the map points to the right of the map, the direction of y is the direction that the original point of the map points to the lower part of the map, and theta is the included angle between the connecting line between the wall corner coordinate and the original point of the map and the direction of the x axis. 4 wall corner family size chains are selected for separate teaching, see fig. 3. Such as a corner family dimension chain D 57 The former corner is numbered as (5), is a first concave corner and has the coordinate of (x) 5 ,y 5 ) Angle theta with respect to the origin of the map 5 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 57 The starting point of (2). The number of the next adjacent corner is (7), the corner is a fourth type concave corner, and the coordinate is (x) 7 ,y 7 ) Then the angle relative to the origin of the map is θ 7 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 57 The end point of (1). The distance between the corner 5 and the corner 7 in the x direction is d 57x ＝x 7 -x 5 The distance in the y direction is d 57y ＝y 7 -y 5 Then, then
The relative angle between two adjacent corners is delta theta 57 ＝θ 7 -θ 5 . Will be calculated
And relative angle delta theta 57 Chain D of dimensions stored in corner 57 In (1).
Wall corner family dimension chain D 910 The number of the previous corner is (9), the fourth type of convex corner is formed, and the coordinate is (x) 9 ,y 9 ) Angle theta with respect to the origin of the map 9 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 910 The starting point of (2). The next adjacent corner is numbered r, and is the first kind of convex corner, and its coordinate is (x) 10 ,y 10 ) Angle relative to the origin of the mapDegree theta 10 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 910 The end point of (1). The distance between the corner 9 and the corner 10 in the x direction is d 910x ＝x 10 -x 9 A distance d in the y direction 910y ＝y 10 -y 9 Then, then
The relative angle between two adjacent corners is delta theta 2125 ＝θ 25 -θ 21 . And will be calculated
And relative angle delta theta 2125 Chain D of dimensions stored in corner 910 In (1).
Wall corner family dimension chain D 3234 The number of the previous corner is
Is a convex wall corner of the second kind and has the coordinate of (x) 32 ,y 32 ) Angle theta with respect to the origin of the map 32 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 3234 The starting point of (2). The latter adjacent corner is numbered as
Is a concave corner of the first kind and has the coordinate of (x) 34 ,y 34 ) Then the angle relative to the origin of the map is θ 34 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 3234 The end point of (1). The corner 32 is spaced from the corner 34 by a distance d in the x-direction 3234x ＝x 34 -x 32 A distance d in the y direction 3234y ＝y 34 -y 32 Then, then
The relative angle between two adjacent corners is delta theta 3234 ＝θ 34 -θ 32 . And will be calculated
And relative angle delta theta 3234 Dimension chain D stored in corner 3234 In (1).
Wall corner family dimension chain D 4749 The number of the previous corner is
Is a concave corner of the second kind and has the coordinate of (x) 47 ,y 47 ) Angle theta with respect to the origin of the map 47 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 4749 The starting point of (2). The latter adjacent corner is numbered as
Is a concave corner of the first kind with the coordinate of (x) 49 ,y 49 ) Then the angle relative to the origin of the map is θ 49 Storing the serial number, the type, the coordinate and the angle of the corner in the dimension chain D of the corner family 4749 The end point of (1). The distance between the corner 47 and the corner 49 in the x direction is d 4749x ＝x 49 -x 47 A distance d in the y direction 4749y ＝y 49 -y 47 Then, then
The relative angle between two adjacent corners is delta theta 4749 ＝θ 49 -θ 47 . And will be calculated
And relative angle delta theta 4749 Dimension chain D stored in corner 4749 In (1). Finally, the types of the corners and the size chain of the corner families in the relative sequence are put into the information table of the corner families.
Obtaining the relation between non-wall angle semantic objects in an indoor typical scene according to non-wall angle semantic information in the environment, obtaining the relation between non-wall angle semantic objects in an indoor non-wall angle semantic object size chain according to the relative position and orientation relation between the non-wall angle semantic objects; and recording the non-wall angle semantic object types and the non-wall angle semantic object size chains of the relative sequence of the non-wall angle semantic object types in the non-wall angle semantic object information table.
The non-wall semantic object size chain relationship in an indoor typical scene is shown in fig. 4. In fig. 4, some non-corner semantic objects 6, 8, 13 are selected to analyze the non-corner semantic object types and non-corner semantic object size chains of the adjacent upstream and downstream non-corner semantic objects, as shown in table 2 below.
TABLE 2 non-wall-corner semantic object information table in indoor typical scene
In the non-wall corner object semantic graph, object categories and non-wall corner semantic object size chains are stored. When the non-wall corner object semantic map is constructed, different numbers are sequentially distributed to the identified non-wall corner semantic objects according to the sequence, and the coordinates of the non-wall corner semantic objects are stored in the map. The non-wall corner semantic object size chain is provided with a starting point and an end point, according to the sequence of constructing the non-wall corner semantic objects, the non-wall corner semantic object size chain is pointed to a next adjacent non-wall corner semantic object from a previous non-wall corner semantic object, in the non-wall corner semantic object size chain, the starting point of the non-wall corner semantic object size chain stores the coordinates of the previous non-wall corner semantic object, the classes of the non-wall corner semantic objects, the numbers of the non-wall corner semantic objects and the angles relative to the map origin, and the end point of the non-wall corner semantic object size chain stores the coordinates of the next adjacent non-wall corner semantic object, the classes of the non-wall corner semantic objects, the numbers of the non-wall corner semantic objects and the angles relative to the map origin, and stores the distance information, the direction information and the relative angle information of the two adjacent non-wall corner semantic objects.
In the non-wall corner object semantic graph, a non-wall corner semantic object size chain is selected for teaching, as shown in fig. 4. Non-wall corner semantic object size chain W 68 The number of the previous non-wall corner semantic object is (6), the previous non-wall corner semantic object is a chair, and the coordinates are (x) 6 ,y 6 ) Relative to the origin of the mapIs an angle of theta 6 Storing the number, category, coordinate and angle of the non-wall corner semantic object in the size chain W of the non-wall corner semantic object 68 The starting point of (2). The number of the latter adjacent non-wall corner semantic object is (8), the object is a cabinet, and the coordinate is (x) 8 ,y 8 ) Then the angle relative to the origin of the map is θ 8 Storing the number, category, coordinate and angle of the non-wall corner semantic object in the size chain W of the non-wall corner semantic object 68 The end point of (1). The distance between the non-wall angle semantic object 6 and the adjacent non-wall angle semantic object 8 in the x direction is d 68x ＝x 8 -x 6 A distance d in the y direction 68y ＝y 8 -y 6 Then, then
The relative angle of two semantic objects is delta theta 68 ＝θ 8 -θ 6 . And will be calculated
And relative angle delta theta 68 Storing in a non-wall-corner semantic object dimension chain W 68 In (1). And finally, putting the category of the non-wall corner semantic object and the size chain of the non-wall corner semantic object in the relative sequence into a non-wall corner semantic object information table.
In order to quickly identify the wall corner category, the wall corner category information and the surrounding semantic information are stored. A circle is drawn in the range with the maximum visual angle range of the depth map of the camera and the accurate visual distance, and all objects with the center coordinates in the circle are regarded as objects near the corner of the wall.
The relationship between the wall corner category and the semantic object information of the adjacent non-wall corners is shown in FIG. 5, a part of the wall corner points are selected for indication in the figure, searching is carried out around each wall corner point, if the non-wall corner object is near the wall corner, the wall corner category information, the object category information and the distance information are stored, and d is used for storing 1 ，d 2 ，d 3 ，d 4 ，d 5 ，d 6 ，d 7 Respectively showing the distance between the centers of the chair, the cabinet, the garbage can, the sofa, the cabinet, the bed and the air conditioner and the wall corner.
In one embodiment, step 5: traversing scenes to obtain a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and obtaining a semantic map with the wall corner family as a main characteristic through origin point coincidence and pose alignment;
specifically, the coordinates of the semantic objects of the surrounding wall corner and the non-wall corner, which are acquired by the laser radar and the depth camera, are coordinates relative to the coordinates of the semantic objects of the surrounding wall corner and the non-wall corner, and the world coordinates of the semantic objects of the wall corner and the non-wall corner are acquired by determining the conversion relation of each coordinate system.
Preferably, the determining the conversion relationship of each coordinate system and the obtaining the world coordinates of the semantic objects of the wall corner and the non-wall corner comprises determining the relationship between a camera coordinate system and a laser radar coordinate system, replacing the camera depth data with the laser radar depth data, converting the laser radar coordinate system into the robot coordinate system, converting the robot coordinate system into the world coordinate system, and mapping the coordinates of the points into the grid coordinate system. The method comprises the steps of fusing odometer information, inertial measurement unit information, laser information processing and identification, visual information processing and identification in the process of traversing a scene to obtain a wall corner family semantic map, a non-wall corner object semantic map and a grid map, wherein the three maps are incrementally constructed by Bayesian estimation suitable for a dynamic scene, and the semantic map based on the wall corner family as a main feature is obtained through origin coincidence and pose alignment.
In conclusion, the invention combines the laser SLAM algorithm and the deep learning method to establish an environment grid map and extract environment semantics, meanwhile removes the false semantics from the semantic point cloud, fuses semantic information and the environment grid map through coordinate transformation, judges whether object semantics exist in the grid by adopting an incremental method based on Bayesian estimation, obtains a semantic map construction method based on the wall corner family as the main characteristic, and obtains a wall corner family size chain according to the relative position and azimuth relation between the wall corner and the adjacent wall corner; obtaining a dimension chain of the non-wall corner semantic object according to the relative position and the orientation relation between the non-wall corner semantic object and the adjacent non-wall corner semantic object; and obtaining the information relation between the wall corner and the semantic objects adjacent to the non-wall corner according to the relative position and the orientation relation between the wall corner and the semantic objects adjacent to the non-wall corner.
The scope of the invention is not limited to the above examples, and it is apparent that those skilled in the art can make various modifications and variations to the present invention without departing from the scope and spirit of the invention. It is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.
Claims (9)
1. The semantic map construction method based on the wall corner family as the main feature is characterized by comprising the following steps: the method comprises the following steps:
distinguishing the convexity and concavity of the indoor wall corner;
judging the directivity of the indoor wall corner, and acquiring a wall corner class label based on the directivity judgment result;
the obtained wall corner category labels are distinguished by combining the convexity and the concavity of the indoor wall corner, and a wall corner family of the indoor wall corner is established;
constructing a wall corner semantic graph according to the wall corner family, obtaining a wall corner family size chain according to the relative position and the azimuth relation between the wall corner and the adjacent wall corner in the wall corner family, and storing the wall corner family size chain in a wall corner family information table; constructing a non-wall corner object semantic graph according to the non-wall corner semantic objects, obtaining a non-wall corner semantic object size chain according to the relative position and orientation relation between the non-wall corner semantic objects and adjacent non-wall corner semantic objects, and storing the non-wall corner semantic object size chain in a non-wall corner semantic object information table;
and traversing the scene to obtain a wall corner family semantic map, a non-wall corner object semantic map and a grid map, and obtaining the semantic map with the wall corner family as the main characteristic through origin point coincidence and pose alignment.
2. The method for building the semantic map based on the wall corner family as the main characteristic according to claim 1, wherein the semantic map comprises the following steps: the method for obtaining the size chain of the corner family according to the relative position and the azimuth relation between the corner in the corner family and the adjacent corner comprises the following steps: and distributing different numbers to the identified corners in sequence, and forming a corner family size chain by pointing a front corner to a rear adjacent corner according to the sequence of constructing the corners.
3. The semantic map construction method based on the wall corner family as the main feature of claim 2, characterized in that: in the size chain of the corner family, the starting point of the size chain of the corner family stores the category, the corner number, the coordinate and the angle relative to the original point of the map of the previous corner, the end point of the size chain stores the category, the corner number, the coordinate and the angle relative to the original point of the map of the next adjacent corner, and stores the distance information, the direction information and the relative angle information of the two adjacent corners.
4. The semantic map construction method based on the wall corner family as the main feature of claim 1, characterized in that: the obtaining of the size chain of the non-wall corner semantic object according to the relative position and the orientation relation between the non-wall corner semantic object and the adjacent non-wall corner semantic object comprises the following steps:
and allocating different numbers to the identified non-wall corner semantic objects in sequence, and pointing a previous non-wall corner semantic object to a next adjacent non-wall corner semantic object according to the sequence of constructing the non-wall corner semantic objects to form a non-wall corner semantic object size chain.
5. The semantic map construction method based on the wall corner family as the main feature of claim 1, characterized in that: before the traversing scene obtains a wall corner family semantic map, a non-wall corner object semantic map and a grid map and obtains the semantic map with the wall corner family as the main characteristic through origin point coincidence and pose alignment, the method further comprises the following steps:
determining a relation between a camera coordinate system and a laser radar coordinate system;
and replacing the camera depth data with the laser radar depth data, converting the laser radar coordinate system into the robot coordinate system, and finally converting the robot coordinate system into the world coordinate system.
6. The semantic map construction method based on the wall corner family as the main feature of claim 5, characterized in that: the method comprises the following steps of traversing a scene to obtain a wall corner family semantic graph, a non-wall corner object semantic graph and a grid map, and obtaining the semantic map with the wall corner family as the main characteristic through origin point coincidence and pose alignment, wherein the semantic map comprises the following steps:
mapping the coordinates of the points to a grid coordinate system; fusing odometer information, inertial measurement unit information, laser information processing and identification, visual information processing and identification to obtain a wall corner family semantic graph, a non-wall corner object semantic graph and a grid map in the process of traversing a scene; and incrementally constructing the wall corner family semantic map, the non-wall corner object semantic map and the raster map by using Bayesian estimation suitable for a dynamic scene, and obtaining the semantic map based on the wall corner family as the main characteristic by means of origin point coincidence and pose alignment.
7. The semantic map construction method based on the wall corner family as the main feature of claim 1, characterized in that: the distinguishing of the convexity and the concavity of the indoor wall corners comprises the following steps: judging the distance between the wall corner intersected with the two straight lines and the observation point and the distance between the position far away from the wall corner and the observation point, and if the distance between the position far away from the wall corner and the observation point is greater than the distance between the wall corner intersected with the two straight lines and the observation point, determining that the wall corner is a convex wall corner; and if the distance between the position far away from the corner and the observation point is less than the distance between the corner intersected with the two straight lines and the observation point, the corner is a concave corner.
8. The semantic map construction method based on the wall corner family as the main feature of claim 1, characterized in that: the step of judging the directionality of the indoor wall corner and acquiring the wall corner category label based on the directionality judgment result comprises the following steps of:
in an indoor environment, selecting a wall as a reference, setting a course angle to be 0 DEG when the robot is vertical to the wall, and setting the right corner of the wall as a first type corner;
acquiring a robot course angle, and judging the indoor wall angle directionality according to an included angle between the wall surface and the course angle;
and sequentially determining the wall corners as a second type of wall corner, a third type of wall corner and a fourth type of wall corner according to the counterclockwise direction of the wall corners.
9. The semantic map construction method based on the wall corner family as the main feature of claim 1, characterized in that: the step of establishing the wall corner family of the indoor wall corner by combining the obtained wall corner category label and distinguishing the convexity and concavity of the indoor wall corner comprises the following steps: after laser radar discerns the corner, distinguish the unsmooth classification of corner, combine the corner to classify the corner clan of indoor corner for the azimuth of robot, wherein, the classification of corner clan includes: the first type of convex wall corner, the second type of convex wall corner, the third type of convex wall corner and the fourth type of convex wall corner; the first type of concave wall corner, the second type of concave wall corner, the third type of concave wall corner and the fourth type of concave wall corner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210805086.XA CN115326048B (en) | 2022-07-08 | 2022-07-08 | Semantic map construction method based on corner family as main feature |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210805086.XA CN115326048B (en) | 2022-07-08 | 2022-07-08 | Semantic map construction method based on corner family as main feature |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115326048A true CN115326048A (en) | 2022-11-11 |
| CN115326048B CN115326048B (en) | 2024-06-07 |
Family
ID=83918526
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210805086.XA Active CN115326048B (en) | 2022-07-08 | 2022-07-08 | Semantic map construction method based on corner family as main feature |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115326048B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120143495A1 (en) * | 2010-10-14 | 2012-06-07 | The University Of North Texas | Methods and systems for indoor navigation |
| CN105841687A (en) * | 2015-01-14 | 2016-08-10 | 上海智乘网络科技有限公司 | Indoor location method and indoor location system |
| CN112785643A (en) * | 2021-02-02 | 2021-05-11 | 武汉科技大学 | Indoor wall corner two-dimensional semantic map construction method based on robot platform |
| CN113902828A (en) * | 2021-09-26 | 2022-01-07 | 武汉科技大学 | Construction method of indoor two-dimensional semantic map with corner as key feature |
-
2022
- 2022-07-08 CN CN202210805086.XA patent/CN115326048B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120143495A1 (en) * | 2010-10-14 | 2012-06-07 | The University Of North Texas | Methods and systems for indoor navigation |
| CN105841687A (en) * | 2015-01-14 | 2016-08-10 | 上海智乘网络科技有限公司 | Indoor location method and indoor location system |
| CN112785643A (en) * | 2021-02-02 | 2021-05-11 | 武汉科技大学 | Indoor wall corner two-dimensional semantic map construction method based on robot platform |
| CN113902828A (en) * | 2021-09-26 | 2022-01-07 | 武汉科技大学 | Construction method of indoor two-dimensional semantic map with corner as key feature |
Non-Patent Citations (1)
| Title |
|---|
| 张文;刘勇;张超凡;张龙;夏营威;: "基于语义建图的室内机器人实时场景分类", 传感器与微系统, no. 08, 20 August 2017 (2017-08-20) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115326048B (en) | 2024-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111486855B (en) | Indoor two-dimensional semantic grid map construction method with object navigation points | |
| CN111563442B (en) | Slam method and system for fusing point cloud and camera image data based on laser radar | |
| CN105955258B (en) | Robot global grating map construction method based on the fusion of Kinect sensor information | |
| Ulrich et al. | Appearance-based place recognition for topological localization | |
| CN109144072A (en) | A kind of intelligent robot barrier-avoiding method based on three-dimensional laser | |
| CN109557919B (en) | Variable-width grid map construction method fusing artificial road sign information | |
| CN112346463B (en) | Unmanned vehicle path planning method based on speed sampling | |
| Carmichael et al. | Large data sets and confusing scenes in 3-d surface matching and recognition | |
| CN111998846A (en) | Unmanned system rapid relocation method based on environment geometry and topological characteristics | |
| CN110110763A (en) | A kind of grating map fusion method based on maximum public subgraph | |
| Meng et al. | Efficient and reliable LiDAR-based global localization of mobile robots using multiscale/resolution maps | |
| CN109948413A (en) | Method for detecting lane lines based on the fusion of high-precision map | |
| CN111950440A (en) | Method, device and storage medium for identifying and positioning door | |
| CN112484746A (en) | Monocular vision-assisted laser radar odometer method based on ground plane | |
| CN111678516A (en) | Bounded region rapid global positioning method based on laser radar | |
| WO2022111723A1 (en) | Road edge detection method and robot | |
| Weber et al. | APR-global scan matching using anchor point relationships | |
| CN111928860A (en) | Autonomous vehicle active positioning method based on three-dimensional curved surface positioning capability | |
| WO2023131203A1 (en) | Semantic map updating method, path planning method, and related apparatuses | |
| CN113902828B (en) | Construction method of indoor two-dimensional semantic map with corner as key feature | |
| CN115457130A (en) | Electric vehicle charging port detection and positioning method based on depth key point regression | |
| CN113034504B (en) | Plane feature fusion method in SLAM mapping process | |
| CN115326048A (en) | Semantic map construction method based on wall corner family as main feature | |
| US8798957B2 (en) | Method for the computer-aided calculation of the movement of an object using sensor data | |
| Song et al. | Natural landmark extraction in cluttered forested environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |