CN115326048B - Semantic map construction method based on corner family as main feature - Google Patents
Semantic map construction method based on corner family as main feature Download PDFInfo
- Publication number
- CN115326048B CN115326048B CN202210805086.XA CN202210805086A CN115326048B CN 115326048 B CN115326048 B CN 115326048B CN 202210805086 A CN202210805086 A CN 202210805086A CN 115326048 B CN115326048 B CN 115326048B
- Authority
- CN
- China
- Prior art keywords
- corner
- semantic
- family
- map
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010276 construction Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims description 15
- 238000005259 measurement Methods 0.000 claims description 13
- 230000000007 visual effect Effects 0.000 claims description 7
- 230000010365 information processing Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000011160 research Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3804—Creation or updating of map data
- G01C21/3807—Creation or updating of map data characterised by the type of data
- G01C21/383—Indoor data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3804—Creation or updating of map data
- G01C21/3833—Creation or updating of map data characterised by the source of data
- G01C21/3841—Data obtained from two or more sources, e.g. probe vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a semantic map construction method based on corner families as main characteristics, which comprises the steps of distinguishing convexity and concavity of the corners; judging the directivity of the corner, and obtaining a corner type label; building a corner family of indoor corners; constructing a corner semantic graph according to a corner family, obtaining a corner family size chain, and storing the corner family size chain in a corner family information table; constructing a non-corner object semantic graph according to the non-corner semantic object, obtaining a non-corner semantic object size chain, and storing the non-corner semantic object size chain in a non-corner semantic object information table; traversing the scene to obtain a corner family semantic map, a non-corner object semantic map and a grid map, and obtaining a semantic map taking the corner family as a main feature through origin coincidence and pose alignment. The invention takes corner families as main characteristics, and obtains corner family size chains, non-corner semantic object size chains, and corner and adjacent non-corner semantic object information relations according to the position relations, thereby determining the uniqueness and certainty of each corner and non-corner semantic object in the corner families.
Description
Technical Field
The invention belongs to the field of indoor mobile robot map building, positioning and navigation, and particularly relates to a semantic map building method based on corner families as main features.
Background
The corners in the indoor working environment naturally exist and are fixed in position, and the difficulty in the method is how to fully utilize and distinguish each corner, so that each corner can independently provide positioning reference for the mobile robot. The prior researches do not find that the identity of the corners is determined according to the unique inherent relation among the corners, and although the related researches introduce the corners into a semantic map, the corners are isolated, the inherent relation among the corners is not discovered, and the corners cannot be obviously distinguished.
Disclosure of Invention
The semantic map constructed by aiming at the current related research does not take corner families as main characteristics, and does not deeply excavate the internal connection between corners in the transverse direction and the longitudinal direction, so that the corners cannot be unique and deterministic.
In order to solve the problems, the invention provides a semantic map construction method based on corner families as main features, which comprises the following steps:
Carrying out convex and concave distinction of indoor corners;
Performing indoor corner directivity judgment, and acquiring corner category labels based on directivity judgment results;
The obtained corner type labels are combined with the convex and concave distinction of indoor corners, and corner families of the indoor corners are established;
Constructing a corner semantic graph according to a corner family, obtaining a corner family size chain according to the relative position and azimuth relation between a corner in the corner family and an adjacent corner, and storing the corner family size chain in a corner family information table; constructing a non-corner object semantic graph according to the non-corner semantic objects, obtaining a non-corner semantic object size chain according to the relative positions and azimuth relations of the non-corner semantic objects and adjacent non-corner semantic objects, and storing the non-corner semantic object size chain in a non-corner semantic object information table;
Traversing the scene to obtain a corner family semantic map, a non-corner object semantic map and a grid map, and obtaining a semantic map taking the corner family as a main feature through origin coincidence and pose alignment.
Preferably, the obtaining the corner cluster size chain according to the relative position and orientation relation between the corners and the adjacent corners in the corner cluster comprises the following steps: and (3) sequentially assigning different numbers to the identified corners, and forming a corner family size chain by pointing the previous corner to the next adjacent corner according to the order of constructing the corners.
Preferably, in the composing corner cluster size chain, the start point of the corner cluster size chain stores the category, corner number, coordinates and angle with respect to the origin of the map of the previous corner, the end point of the size chain stores the category, corner number, coordinates and angle with respect to the origin of the map of the next adjacent corner, and distance information, direction information and relative angle information of the two adjacent corners are stored.
Preferably, the obtaining the non-corner semantic object size chain according to the relative position and orientation relation between the non-corner semantic object and the adjacent non-corner semantic object includes:
and sequentially allocating different numbers to the identified non-corner semantic objects, and pointing the previous non-corner semantic object to the next adjacent non-corner semantic object according to the order of constructing the non-corner semantic objects to form a non-corner semantic object size chain.
Preferably, the traversing scene obtains a corner family semantic map, a non-corner object semantic map and a grid map, and obtains a semantic map with a corner family as a main feature through origin coincidence and pose alignment, and the method further comprises the following steps:
Determining a relationship between a camera coordinate system and a lidar coordinate system;
and replacing the camera depth data with the laser radar depth data, converting the laser radar coordinate system into a robot coordinate system, and finally converting the robot coordinate system into a world coordinate system.
Preferably, the traversing scene obtains a corner family semantic map, a non-corner object semantic map and a grid map, and the semantic map with the corner family as a main feature is obtained through origin coincidence and pose alignment, which comprises the following steps:
Mapping coordinates of the points to a grid coordinate system; integrating odometer information, inertial measurement unit information, laser information processing and identifying, visual information processing and identifying to obtain a corner family semantic map, a non-corner object semantic map and a grid map in the process of traversing the scene;
And constructing the corner family semantic map, the non-corner object semantic map and the grid map in an incremental mode through Bayesian estimation suitable for dynamic scenes, and obtaining a semantic map based on the corner family as a main feature through origin coincidence and pose alignment.
Preferably, the distinguishing of the convexity and concavity of the indoor corner comprises: judging the distance between the corner intersected by the two straight lines and the observation point and the distance between the position far away from the corner and the observation point, and if the distance between the position far away from the corner and the observation point is greater than the distance between the corner intersected by the two straight lines and the observation point, judging the corner as a convex corner; if the distance between the corner position and the observation point is smaller than the distance between the corner where the two straight lines intersect and the observation point, the corner is a concave corner.
Preferably, the indoor corner directivity judgment is performed, and the obtaining of the corner category label based on the directivity judgment result includes the following steps:
in an indoor environment, selecting a wall body as a reference, setting a heading angle to be 0 degrees when the robot is vertical to the wall body, and setting a right wall angle of the wall body as a first type wall angle;
acquiring a course angle of the robot, and judging the directivity of an indoor wall angle according to the included angle between the wall surface and the course angle;
And according to the corner direction, the second type of corners, the third type of corners and the fourth type of corners are sequentially determined according to the anticlockwise sequence.
Preferably, the building the corner group of the indoor corner includes: after the laser radar recognizes the corner, distinguishing the concave-convex type of the corner, and classifying the corner family of the indoor corner by combining the azimuth angle of the corner relative to the robot, wherein the corner family classification comprises: the first type of protruding corner, the second type of protruding corner, the third type of protruding corner and the fourth type of protruding corner; the first type of recessed corner, the second type of recessed corner, the third type of recessed corner and the fourth type of recessed corner.
The invention has the following advantages:
The invention uses the depth camera, the inertial measurement unit and the laser radar to distinguish the directions and the concavities and convexities of the corners, and finally obtains rich corner category information. And researching the uniqueness and certainty of each corner in the corner family, transversely and longitudinally excavating the internal relation between the corners, and constructing a semantic map construction method based on the corner family as a main characteristic. In the constructed semantic map, the position relations between corners and adjacent corners, between non-corner semantic objects and adjacent non-corner semantic objects and between corners and adjacent non-corner semantic objects are fully expressed, corner families are made to be main features, and corner family size chains, non-corner semantic object size chains, corner and adjacent non-corner semantic object information relations are obtained according to the position relations, so that the uniqueness and certainty of each corner and non-corner semantic object in the corner families are determined.
Drawings
FIG. 1 is a flow chart of a semantic map construction method based on corner families as main features of the invention;
FIG. 2 is a schematic view of a corner cluster in a typical indoor scenario of the present invention;
FIG. 3 is a table of corner family information in a typical indoor scenario of the present invention;
FIG. 4 is a diagram showing a chain of non-corner semantic object size relationships in a typical indoor scenario of the present invention;
FIG. 5 is a graph showing the relationship between corner categories and adjacent non-corner semantic objects according to the present invention.
Detailed Description
In order to facilitate the understanding and practice of the invention, one of ordinary skill in the art will now recognize in view of the drawings and examples of implementation, that the examples described herein are presented for purposes of illustration and explanation only and are not intended to be limiting.
The specific implementation example of the invention is a semantic map construction method based on corner families as main features, as shown in fig. 1, the method comprises the following steps:
Step 1: carrying out convex and concave distinction of indoor corners;
Step 2: performing indoor corner directivity judgment, and acquiring corner category labels based on directivity judgment results;
step 3: the obtained corner type labels are combined with the convex and concave distinction of indoor corners, and corner families of the indoor corners are established;
Step 4: obtaining a corner group size chain according to the relative position and azimuth relation between the corner and the adjacent corner in the corner group, and storing the corner group size chain in a corner group information table; constructing a non-corner object semantic graph according to the non-corner semantic objects, obtaining a non-corner semantic object size chain according to the relative positions and azimuth relations of the non-corner semantic objects and adjacent non-corner semantic objects, and storing the non-corner semantic object size chain in a non-corner semantic object information table;
Step 5: traversing the scene to obtain a corner family semantic map, a non-corner object semantic map and a grid map, and obtaining a semantic map taking the corner family as a main feature through origin coincidence and pose alignment.
In the semantic map construction method based on the corner family as the main characteristic, the position relations between the corner and the adjacent corner, the position relations between the non-corner semantic object and the adjacent non-corner semantic object and the position relations between the corner and the adjacent non-corner semantic object are fully expressed in the constructed semantic map, the corner family is made to be the main characteristic, and the corner family size chain, the non-corner semantic object size chain, the corner and the adjacent non-corner semantic object information relations are obtained according to the position relations, so that the uniqueness and the certainty of each corner and the non-corner semantic object in the corner family are determined.
In specific implementation, the hardware platform on which the method of the invention depends comprises: the system comprises a main control computer, a robot platform chassis, a depth camera, an inertial measurement unit and a laser radar; the main control machine is respectively and sequentially connected with the robot platform chassis, the depth camera and the laser radar. Wherein,
The main control computer is selected from an Kui 9 mini-host; the robot platform chassis is a SCOUT MINI intelligent mobile chassis; the depth camera is Microsoft KinectV; the inertia measurement unit is selected from the model AH-100B; the laser radar sensor selects SICK lms111 laser radar with stable performance.
The semantic map construction method of the present invention is described in further detail below;
In one embodiment, step 1: carrying out convex and concave distinction of indoor corners; the indoor corners are distinguished according to convex-concave categories and can be divided into convex corners and concave corners, and the convex-concave distinction of the corners is realized through laser and vision fusion.
The indoor corner convexity and concavity distinguishing method comprises the following steps: judging the distance between the corner intersected by the two straight lines and the observation point and the distance between the position far away from the corner and the observation point, and if the distance between the position far away from the corner and the observation point is greater than the distance between the corner intersected by the two straight lines and the observation point, judging the corner as a convex corner; if the distance between the corner position and the observation point is smaller than the distance between the corner where the two straight lines intersect and the observation point, the corner is a concave corner.
Specifically, the judging of the convexity and concavity of the indoor corner specifically comprises the following steps: after the laser radar identifies indoor corners, the corners are distinguished according to convex-concave categories, one is convex, and the other is concave. If the corner where the two straight lines intersect is closest to the robot, the laser point which is farther from the corner is farther from the robot, and is the convex corner; if the corner where the two straight lines intersect is farthest from the robot, the laser points far away from the corner are closer to the robot, and the corner is concave.
In one embodiment, step 2: performing indoor corner directivity judgment, and acquiring corner category labels based on directivity judgment results;
in particular, conventional indoor corners generally have four main orientations, and each type of corner has a unique tag, so that the direction of the corner is judged. Judging the directivity of an indoor corner, and acquiring a corner type label, wherein the specific step comprises the steps that a robot obtains a robot course angle through an inertia measurement unit, a laser point of the robot, which hits a wall surface through a laser radar, is subjected to linear fitting through an IEPF algorithm to obtain an included angle between the wall surface and the course angle acquired by the inertia measurement unit, in an indoor environment, a wall body is selected as a reference, when the robot is perpendicular to the wall body, the course angle is set to be 0 DEG, the right corner of the wall body is set to be a first type corner, and the second type corner, the third type corner and the fourth type corner are sequentially set according to a anticlockwise sequence. The course angle increases in a clockwise direction, the range is between 0 and 360 degrees, the course angle of the robot is represented by theta, and the maximum visual angle range of the camera depth map is represented by alpha;
When each type of corner does not have an overlapping course angle, the identified corner is a corresponding corner type, and in the left and right limit view angle range of the camera, when alpha/2 < theta < 90-alpha/2, the view angle range of the camera is a first type corner; when 270 degrees+alpha/2 < theta <360 degrees-alpha/2, the view angle range of the camera is seen as a second type of corner; when 180 degrees+alpha/2 < theta <270 degrees-alpha/2, the view angle range of the camera is seen as a third type of corner; when 90 DEG+alpha/2 < theta <180 DEG alpha/2, the range of camera viewing angles is seen as the fourth type of corner.
When the overlapping course angles exist, it is determined whether the corner is located on the left or right side of the course angle. If only one corner type is located on the left or right side of the heading angle, the corner is the corresponding corner type.
If the corner is located on the left side or the right side of the course angle, the included angle beta between the course angle and the side wall surface of the corner is calculated, and the corner corresponds to one corner type when beta < alpha/2 and corresponds to the other corner type when beta > 90-alpha/2. And in the range of the interval set by the course angle, determining the specific wall corner type by combining the included angle beta between the course angle and the corresponding wall surface. Under the left and right limiting view angles of a camera, when 360 degrees-alpha/2 < theta <360 degrees, the first type of corner and the second type of corner are both positioned on the right side of a course angle, at the moment, an included angle beta between the course angle and the right side wall surface where the corner is positioned is calculated, if beta < alpha/2, the view angle range of the camera is the first type of corner, and if beta >90 degrees-alpha/2, the view angle range of the camera is the second type of corner; when 0 degree < theta < alpha/2, the first type corner and the second type corner are both positioned at the left side of the course angle, at the moment, the included angle beta between the course angle and the left side wall surface where the corner is positioned is calculated, if beta < alpha/2, the camera view angle range is seen as the second type corner, and if beta > 90-alpha/2, the camera view angle range is seen as the first type corner.
Specifically, the indoor corners are judged in the viewing angle range of the camera KinectV and the course angle range acquired by the inertial measurement unit, and if each type of corner has a non-overlapping course angle range, the corner identified in the non-overlapping course angle range is the corresponding corner category.
Alternatively, the robot may be provided with an inertial measurement unit to obtain a robot orientation, and the corner directivity determination may be performed based on the identified robot orientation corresponding to the corner. The inertial measurement unit is corrected when the inertial measurement unit is used for the first time, a wall body is selected as a reference in an indoor environment, when the robot is perpendicular to the wall body, the course angle of the robot is set to be 0 degrees, the right side wall angle of the wall body is set to be a first type wall angle, and the first type wall angle, the third type wall angle and the fourth type wall angle are sequentially set to be a second type wall angle, a third type wall angle and a fourth type wall angle according to a anticlockwise sequence.
In one embodiment, step 3: and (3) combining the obtained corner type labels with the convex and concave distinction of the indoor corners to establish corner families of the indoor corners.
Specifically, the robot combines the corner category labels obtained in the step 2 with the relationship between corners and other non-corner semantic objects to realize that each corner has uniqueness. After the laser radar recognizes the corner, the concave-convex type of the corner is distinguished, and one is a concave corner and the other is a convex corner. Azimuth angles when corners are detected by robots are roughly classified into 4 categories, which are classified as follows in combination with convex and concave corners: the first type of protruding corner, the second type of protruding corner, the third type of protruding corner and the fourth type of protruding corner; the first type of concave corners, the second type of concave corners, the third type of concave corners and the fourth type of concave corners are primarily classified into 8 types for the indoor corners of regular buildings, and are called corner families. Specifically, fig. 2 is a schematic view of a corner family in a typical indoor scene.
In one embodiment, step 4: constructing a corner semantic graph according to a corner family, obtaining a corner family size chain according to the relative position and azimuth relation between a corner in the corner family and an adjacent corner, and storing the corner family size chain in a corner family information table; constructing a non-corner object semantic graph according to the non-corner semantic objects, obtaining a non-corner semantic object size chain according to the relative positions and azimuth relations of the non-corner semantic objects and adjacent non-corner semantic objects, and storing the non-corner semantic object size chain in a non-corner semantic object information table;
Specifically, in the construction of the corner semantic graph according to the corner family, the indoor corner family size chain, the non-corner semantic object size chain, the corner and the adjacent non-corner semantic object information relationship are fused when the semantic graph is constructed, and the corner family semantic graph and the non-corner object semantic graph are obtained through processing and recognition of the odometer, the inertia measurement unit, the laser information and the visual information. In the corner family semantic graph, a corner class and corner family size chain is stored; in the non-corner object semantic graph, a non-corner semantic object class and a non-corner semantic object size chain are stored.
The relationship of the corner cluster size chain in a typical indoor scene is shown in fig. 3. In fig. 3, a selection of partial corners 7, 10, 34, 47 are analyzed for corner types and corner family size chains of adjacent upstream and adjacent downstream thereof, see table 1.
TABLE 1 corner family information table in indoor typical scene
In the corner family semantic map, a corner class and corner family size chain is stored. When the corner family semantic graph is constructed, different numbers are sequentially allocated to the identified corners through the sequence, and coordinates of the corners are stored in the corner family semantic graph. In a corner cluster size chain in which the direction of the corner cluster size chain is directed from a preceding corner to a succeeding adjacent corner in the order of constructing corners, the starting point of the size chain stores the category of the preceding corner, the corner number, the coordinates and the angle with respect to the origin of the map, and the ending point of the size chain stores the category of the succeeding adjacent corner, the corner number, the coordinates and the angle with respect to the origin of the map, and distance information, direction information and relative angle information of the two adjacent corners are stored.
In the constructed corner cluster semantic graph, the upper left corner of the map is assumed to be the origin of the map, the x direction is the direction that the origin of the map points to the right of the map, the y direction is the direction that the origin of the map points to the lower part of the map, and θ is the included angle between the connecting line between the corner coordinates and the origin of the map and the x-axis direction. The 4 corner family size chains were chosen for each description, see fig. 3. Such as corner cluster size chain D 57, the previous corner numbered ⑤, a first type of concave corner, co-ordinate (x 5,y5), angle θ 5 with respect to the origin of the map, and the corner number, corner category, corner co-ordinate and angle are stored at the start of corner cluster size chain D 57. The next adjacent corner is numbered ⑦, a fourth type of concave corner, and the coordinate is (x 7,y7), then the angle relative to the origin of the map is θ 7, and the corner number, corner category, corner coordinate and angle are stored at the end of the corner family size chain D 57. The distance between the corner 5 and the corner 7 in the x direction is d 57x＝x7-x5 and the distance in the y direction is d 57y＝y7-y5 The relative angle of two adjacent corners is Δθ 57＝θ7-θ5. Calculated/>And the relative angle Δθ 57 are stored in the corner family size chain D 57.
The corner group size chain D 910, the number of the previous corner is ⑨, the coordinates of which are (x 9,y9) convex corners, the angle relative to the origin of the map is θ 9, and the number, the corner type, the corner coordinates and the angle of the corner are stored in the starting point of the corner group size chain D 910. The next adjacent corner is numbered ⑩, a first type of protruding corner, and the coordinate is (x 10,y10), then the angle relative to the origin of the map is θ 10, and the corner number, corner type, corner coordinate and angle are stored at the end of the corner family size chain D 910. The distance between the corner 9 and the corner 10 in the x direction is d 910x＝x10-x9 and the distance in the y direction is d 910y＝y10-y9 The relative angle of two adjacent corners is Δθ 2125＝θ25-θ21. And will calculate/>And the relative angle Δθ 2125 are stored in the corner family size chain D 910.
Corner family size chain D 3234, the previous corner numberedFor the second type of convex corner, the coordinate is (x 32,y32), the angle relative to the map origin is θ 32, and the corner number, corner type, corner coordinate and angle are stored at the start point of the corner family size chain D 3234. The latter adjacent corner is numbered/>For the first type of concave corner, the coordinate is (x 34,y34), the angle relative to the map origin is θ 34, and the corner number, corner type, corner coordinate and angle are stored at the end of the corner family size chain D 3234. The distance between corner 32 and corner 34 in the x direction is d 3234x＝x34-x32 and the distance in the y direction is d 3234y＝y34-y32, then/>The relative angle of two adjacent corners is Δθ 3234＝θ34-θ32. And will calculate/>And the relative angle Δθ 3234 are stored in the corner family size chain D 3234.
Corner family size chain D 4749, the previous corner numberedFor the second type of concave corner, the coordinate is (x 47,y47), the angle relative to the map origin is θ 47, and the corner number, corner type, corner coordinate and angle are stored at the start of the corner family size chain D 4749. The latter adjacent corner is numbered/>For the first type of concave corner, the coordinate is (x 49,y49), the angle relative to the map origin is θ 49, and the corner number, corner type, corner coordinate and angle are stored at the end of the corner family size chain D 4749. The distance between corner 47 and corner 49 in the x direction is d 4749x＝x49-x47 and the distance in the y direction is d 4749y＝y49-y47, then/>The relative angle of two adjacent corners is Δθ 4749＝θ49-θ47. And will calculate/>And the relative angle Δθ 4749 are stored in the corner family size chain D 4749. Finally, the types of the corners and the corner family size chains of the relative sequence are put into a corner family information table.
Obtaining the relation between non-corner semantic objects in an indoor typical scene according to the non-corner semantic information in the environment, and obtaining a non-corner semantic object size chain according to the relative position and azimuth relation between the non-corner semantic objects in the indoor non-corner semantic object size chain; the non-corner semantic object information table records the non-corner semantic object types and the relative sequence of the non-corner semantic object size chains.
The non-corner semantic object size chain relationship in a typical indoor scenario is shown in fig. 4. In fig. 4, the non-corner semantic objects 6, 8, 13 of the sections are selected to analyze the non-corner semantic object types and the non-corner semantic object size chains of adjacent upstream and downstream thereof as shown in table 2 below.
TABLE 2 non-corner semantic object information table in indoor typical scene
In the non-corner object semantic graph, object categories, non-corner semantic object size chains are stored. When the non-corner semantic object semantic graph is constructed, different numbers are sequentially allocated to the identified non-corner semantic objects through the sequence, and the coordinates of the non-corner semantic objects are stored in the map. The non-corner semantic object size chain has a start point and an end point, and in the order of constructing the non-corner semantic objects, the non-corner semantic object size chain is directed from a previous non-corner semantic object to a next adjacent non-corner semantic object, in the non-corner semantic object size chain, the start point of the non-corner semantic object size chain stores coordinates of the previous non-corner semantic object, a non-corner semantic object class, a non-corner semantic object number and an angle relative to a map origin, and the end point of the non-corner semantic object size chain stores coordinates of the next adjacent non-corner semantic object, the non-corner semantic object class, the non-corner semantic object number and the angle relative to the map origin, and distance information, direction information and relative angle information of two adjacent non-corner semantic objects are stored.
In the non-corner object semantic graph, a non-corner semantic object size chain is selected for explanation, as shown in fig. 4. The non-corner semantic object size chain W 68, the number of the previous non-corner semantic object is ⑥, the previous non-corner semantic object is a chair, the coordinate is (x 6,y6), the angle relative to the origin of the map is θ 6, and the number, the category, the coordinate and the angle of the non-corner semantic object are stored at the starting point of the non-corner semantic object size chain W 68. The number ⑧ of the next adjacent non-corner semantic object is cabinet, the coordinate is (x 8,y8), the angle relative to the origin of the map is theta 8, and the number, the category, the coordinate and the angle of the non-corner semantic object are stored at the end point of the non-corner semantic object size chain W 68. The distance between the non-corner semantic object 6 and the adjacent non-corner semantic object 8 in the x direction is d 68x＝x8-x6, and the distance in the y direction is d 68y＝y8-y6, thenThe relative angle of the two semantic objects is Δθ 68＝θ8-θ6. And will calculate/>And the relative angle Δθ 68 are stored in the non-corner semantic object size chain W 68. And finally, placing the category of the non-corner semantic object and the relative sequence of the non-corner semantic object size chain into a non-corner semantic object information table.
In order to be able to quickly identify corner categories, corner category information and surrounding semantic information are stored. And drawing a circle in the range of the maximum visual angle of the depth map of the camera and the range of the visual distance of the depth map, wherein objects with center coordinates in the circle are all considered as objects near corners of the wall.
The relation between the corner type and the adjacent non-corner semantic object information is shown in fig. 5, part of corner points are selected for illustration in the figure, searching is carried out around each corner point, if a non-corner object is near the corner, the corner type information, the object type information and the distance information are stored, and d 1,d2,d3,d4,d5,d6,d7 is used for respectively representing the distances between the centers of a chair, a cabinet, a garbage can, a sofa, a cabinet, a bed and an air conditioner and the corner.
In one embodiment, step 5: traversing the scene to obtain a corner family semantic map, a non-corner object semantic map and a grid map, and obtaining a semantic map taking the corner family as a main feature through origin coincidence and pose alignment;
Specifically, the coordinates of the surrounding environment corner and the non-corner semantic object obtained by the laser radar and the depth camera are relative to the coordinates under the coordinate system of the laser radar and the depth camera, and the world coordinates of the corner and the non-corner semantic object are obtained by determining the conversion relation of each coordinate system.
Preferably, the determining the conversion relation of each coordinate system, obtaining the world coordinates of the corner and the non-corner semantic object includes determining the relation between the camera coordinate system and the laser radar coordinate system, replacing the camera depth data with the laser radar depth data, converting the laser radar coordinate system to the robot coordinate system, converting the robot coordinate system to the world coordinate system, and finally mapping the coordinates of the points to the grid coordinate system. In the process of traversing the scene, integrating odometer information, inertial measurement unit information, laser information processing and identification, visual information processing and identification to obtain a corner family semantic map, a non-corner object semantic map and a grid map, wherein the three maps are constructed in an incremental mode through Bayesian estimation suitable for the dynamic scene, and the semantic map based on the corner family as a main feature is obtained through origin coincidence and pose alignment.
In summary, the invention combines the laser SLAM algorithm and the deep learning method to create an environment grid map and extract environment semantics, meanwhile, performs false semantic removal on semantic point cloud, fuses semantic information with the environment grid map through coordinate transformation, adopts an incremental method based on Bayesian estimation to judge whether the semantics of objects in the grid exist or not, obtains a semantic map construction method based on the main characteristics of corner families, and obtains a corner family size chain according to the relative positions and azimuth relations of the corner and adjacent corners; obtaining a non-corner semantic object size chain according to the relative position and azimuth relation between the non-corner semantic object and the adjacent non-corner semantic object; and obtaining the information relation between the corner and the adjacent non-corner semantic object according to the relative position and azimuth relation between the corner and the adjacent non-corner semantic object.
The protective scope of the invention is not limited to the examples described above, but it will be apparent to those skilled in the art that various modifications and variations can be made to the invention without departing from the scope and spirit of the invention. It is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. The semantic map construction method based on the corner family as the main feature is characterized by comprising the following steps of: the method comprises the following steps:
Carrying out convex and concave distinction of indoor corners;
Performing indoor corner directivity judgment, and acquiring corner category labels based on directivity judgment results;
The obtained corner type labels are combined with the convex and concave distinction of indoor corners, and corner families of the indoor corners are established;
Constructing a corner semantic graph according to a corner family, obtaining a corner family size chain according to the relative position and azimuth relation between a corner in the corner family and an adjacent corner, and storing the corner family size chain in a corner family information table; constructing a non-corner object semantic graph according to the non-corner semantic objects, obtaining a non-corner semantic object size chain according to the relative positions and azimuth relations of the non-corner semantic objects and adjacent non-corner semantic objects, and storing the non-corner semantic object size chain in a non-corner semantic object information table;
Traversing the scene to obtain a corner family semantic map, a non-corner object semantic map and a grid map, and obtaining a semantic map taking the corner family as a main feature through origin coincidence and pose alignment.
2. The semantic map construction method based on corner families as a main feature according to claim 1, wherein: the method for obtaining the corner cluster size chain according to the relative position and the azimuth relation between the corner and the adjacent corner in the corner cluster comprises the following steps: and (3) sequentially assigning different numbers to the identified corners, and forming a corner family size chain by pointing the previous corner to the next adjacent corner according to the order of constructing the corners.
3. The semantic map construction method based on corner families as a main feature according to claim 2, wherein: in the composition corner cluster size chain, the start point of the corner cluster size chain stores the category, corner number, coordinates and angle relative to the origin of the map of the previous corner, the end point of the size chain stores the category, corner number, coordinates and angle relative to the origin of the map of the next adjacent corner, and stores the distance information, direction information and relative angle information of the two adjacent corners.
4. The semantic map construction method based on corner families as a main feature according to claim 1, wherein: the obtaining the non-corner semantic object size chain according to the relative position and azimuth relation of the non-corner semantic object and the adjacent non-corner semantic object comprises the following steps:
and sequentially allocating different numbers to the identified non-corner semantic objects, and pointing the previous non-corner semantic object to the next adjacent non-corner semantic object according to the order of constructing the non-corner semantic objects to form a non-corner semantic object size chain.
5. The semantic map construction method based on corner families as a main feature according to claim 1, wherein: before the traversing scene obtains the semantic map of the corner family, the semantic map of the non-corner object and the grid map, and the semantic map taking the corner family as the main characteristic is obtained through origin coincidence and pose alignment, the method further comprises the following steps:
Determining a relationship between a camera coordinate system and a lidar coordinate system;
and replacing the camera depth data with the laser radar depth data, converting the laser radar coordinate system into a robot coordinate system, and finally converting the robot coordinate system into a world coordinate system.
6. The semantic map construction method based on corner families as claimed in claim 5, wherein: the traversing scene to obtain a corner family semantic map, a non-corner object semantic map and a grid map, and obtaining the semantic map taking the corner family as a main feature through origin coincidence and pose alignment comprises the following steps:
Mapping coordinates of the points to a grid coordinate system; integrating odometer information, inertial measurement unit information, laser information processing and identifying, visual information processing and identifying to obtain a corner family semantic map, a non-corner object semantic map and a grid map in the process of traversing the scene; and constructing the corner family semantic map, the non-corner object semantic map and the grid map in an incremental mode through Bayesian estimation suitable for dynamic scenes, and obtaining a semantic map based on the corner family as a main feature through origin coincidence and pose alignment.
7. The semantic map construction method based on corner families as a main feature according to claim 1, wherein: the distinguishing of the convexity and concavity of the indoor corner comprises the following steps: judging the distance between the corner intersected by the two straight lines and the observation point and the distance between the position far away from the corner and the observation point, and if the distance between the position far away from the corner and the observation point is greater than the distance between the corner intersected by the two straight lines and the observation point, judging the corner as a convex corner; if the distance between the corner position and the observation point is smaller than the distance between the corner where the two straight lines intersect and the observation point, the corner is a concave corner.
8. The semantic map construction method based on corner families as a main feature according to claim 1, wherein: the indoor corner directivity judgment is carried out, and the corner category label is obtained based on the directivity judgment result, and the method comprises the following steps of:
in an indoor environment, selecting a wall body as a reference, setting a heading angle to be 0 degrees when the robot is vertical to the wall body, and setting a right wall angle of the wall body as a first type wall angle;
acquiring a course angle of the robot, and judging the directivity of an indoor wall angle according to the included angle between the wall surface and the course angle;
And according to the corner direction, the second type of corners, the third type of corners and the fourth type of corners are sequentially determined according to the anticlockwise sequence.
9. The semantic map construction method based on corner families as a main feature according to claim 1, wherein: the building of the corner group of the indoor corner comprises the following steps of: after the laser radar recognizes the corner, distinguishing the concave-convex type of the corner, and classifying the corner family of the indoor corner by combining the azimuth angle of the corner relative to the robot, wherein the corner family classification comprises: the first type of protruding corner, the second type of protruding corner, the third type of protruding corner and the fourth type of protruding corner; the first type of recessed corner, the second type of recessed corner, the third type of recessed corner and the fourth type of recessed corner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210805086.XA CN115326048B (en) | 2022-07-08 | 2022-07-08 | Semantic map construction method based on corner family as main feature |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210805086.XA CN115326048B (en) | 2022-07-08 | 2022-07-08 | Semantic map construction method based on corner family as main feature |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115326048A CN115326048A (en) | 2022-11-11 |
| CN115326048B true CN115326048B (en) | 2024-06-07 |
Family
ID=83918526
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210805086.XA Active CN115326048B (en) | 2022-07-08 | 2022-07-08 | Semantic map construction method based on corner family as main feature |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115326048B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105841687A (en) * | 2015-01-14 | 2016-08-10 | 上海智乘网络科技有限公司 | Indoor location method and indoor location system |
| CN112785643A (en) * | 2021-02-02 | 2021-05-11 | 武汉科技大学 | Indoor wall corner two-dimensional semantic map construction method based on robot platform |
| CN113902828A (en) * | 2021-09-26 | 2022-01-07 | 武汉科技大学 | Construction method of indoor two-dimensional semantic map with corner as key feature |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120143495A1 (en) * | 2010-10-14 | 2012-06-07 | The University Of North Texas | Methods and systems for indoor navigation |
-
2022
- 2022-07-08 CN CN202210805086.XA patent/CN115326048B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105841687A (en) * | 2015-01-14 | 2016-08-10 | 上海智乘网络科技有限公司 | Indoor location method and indoor location system |
| CN112785643A (en) * | 2021-02-02 | 2021-05-11 | 武汉科技大学 | Indoor wall corner two-dimensional semantic map construction method based on robot platform |
| CN113902828A (en) * | 2021-09-26 | 2022-01-07 | 武汉科技大学 | Construction method of indoor two-dimensional semantic map with corner as key feature |
Non-Patent Citations (1)
| Title |
|---|
| 基于语义建图的室内机器人实时场景分类;张文;刘勇;张超凡;张龙;夏营威;;传感器与微系统;20170820(08);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115326048A (en) | 2022-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ulrich et al. | Appearance-based place recognition for topological localization | |
| CN108958282B (en) | Three-dimensional space path planning method based on dynamic spherical window | |
| CN112070770B (en) | High-precision three-dimensional map and two-dimensional grid map synchronous construction method | |
| CN110936383A (en) | Obstacle avoiding method, medium, terminal and device for robot | |
| CN110110763B (en) | Grid map fusion method based on maximum public subgraph | |
| CN112789609A (en) | Map updating method and device, movable platform and storage medium | |
| CN114004869A (en) | Positioning method based on 3D point cloud registration | |
| JP2010541065A (en) | 3D beverage container positioning device | |
| CN111239763A (en) | Object positioning method and device, storage medium and processor | |
| Meng et al. | Efficient and reliable LiDAR-based global localization of mobile robots using multiscale/resolution maps | |
| CN1569558A (en) | Moving robot's vision navigation method based on image representation feature | |
| KR102450139B1 (en) | GLOBAL LOCALIZTION APPARATUS AND METHOD IN DYNAMIC ENVIRONMENTS USING 3D LiDAR SCANNER | |
| CN115326048B (en) | Semantic map construction method based on corner family as main feature | |
| CN111273316B (en) | Multi-laser radar multi-view object detection method based on profile expansion fusion | |
| CN113902828A (en) | Construction method of indoor two-dimensional semantic map with corner as key feature | |
| US12105519B2 (en) | Method for estimating positioning of moving object by using big cell grid map, recording medium in which program for implementing same is stored, and computer program stored in medium in order to implement same | |
| JP4793109B2 (en) | Object detection method and robot | |
| KR101444656B1 (en) | Method for calculating feasible search area for seeker to improve lock-on-target performance | |
| CN115453549A (en) | Method for extracting environment right-angle point coordinate angle based on two-dimensional laser radar | |
| CN113960999A (en) | Mobile robot repositioning method, system and chip | |
| CN117537803B (en) | Robot inspection semantic-topological map construction method, system, equipment and medium | |
| CN112084875A (en) | Multi-laser radar coordinate system method | |
| CN114674308B (en) | Vision-assisted laser corridor positioning method and device based on safety exit indicator | |
| Yun et al. | Multi-hypothesis outdoor localization using multiple visual features with a rough map | |
| Kotthäuser et al. | Triangulation-based plane extraction for 3D point clouds |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |