CN111739088B - Positioning method and device based on visual label - Google Patents
Positioning method and device based on visual label Download PDFInfo
- Publication number
- CN111739088B CN111739088B CN202010705312.8A CN202010705312A CN111739088B CN 111739088 B CN111739088 B CN 111739088B CN 202010705312 A CN202010705312 A CN 202010705312A CN 111739088 B CN111739088 B CN 111739088B
- Authority
- CN
- China
- Prior art keywords
- positioning
- circles
- coding
- visual
- circle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10044—Radar image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Image Analysis (AREA)
Abstract
The invention aims to provide a positioning method and equipment based on a visual tag, which can quickly and effectively solve a plurality of problems such as slow positioning, failure in repositioning and the like in the two current mainstream SLAM schemes only by arranging the visual tag in the environment and positioning the position of a robot based on all the visual tags randomly distributed in the whole space. In the robot positioning navigation, the tagSLAM scheme can more efficiently solve the problems of slow repositioning recovery of the laser SLAM scheme and mismatching caused by scene similarity and the influence of illumination and scene texture structure of the visual SLAM scheme. Meanwhile, the invention can support the scheme of fusion of the visual label SLAM and the laser SLAM, and provides a more robust mapping and positioning scheme for the robot system.
Description
Technical Field
The invention relates to the field of computers, in particular to a positioning method and device based on a visual label.
Background
With the rapid development of artificial intelligence technology, intelligent mobile robots are beginning to be widely used in industries such as industry, agriculture, national defense, and services. The SLAM problem is one of the key problems faced by intelligent mobile robots, and is continuously leading to the attention and research in the industry. The solutions to the SLAM problem are mainly the following, depending on the sensor type:
firstly, laser SLAM: the indoor robot mainly adopts a 2D laser radar to measure the environmental information around the robot, and the information is expressed as a series of dispersed point clouds with accurate angle and distance information. Through matching and comparing the point clouds, the relative motion of the laser radar can be calculated, and therefore the robot can be positioned. The laser SLAM theoretical research is relatively mature, and the products falling to the ground are abundant. However, the laser radar is expensive, the effective distance is not far enough, and the laser radar is not suitable for wide scenes.
II, visual SLAM: compared with laser radar, the camera has the characteristics of low cost, light weight, rich texture information in the image and the like. Therefore, the visual SLAM can easily distinguish similar scenes which are easy to be confused by radar, and has great advantages in relocation of the robot, closed loop detection and scene classification application. Although the application scenes of the visual SLAM are richer, the visual SLAM has high dependency on light and scenes, cannot work in dark places or some non-texture areas, and has other problems of robustness, so that the final floor product is less.
Disclosure of Invention
The invention aims to provide a positioning method and equipment based on a visual label.
According to an aspect of the present invention, there is provided a visual tag-based positioning method, including:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
and determining the position of the robot in the range of the working environment based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the map of the working environment.
Further, in the above method, setting a plurality of positioning circles at different positions on a boundary of an operation environment range of the robot, setting a plurality of coding circles at different positions on a connection line of the positioning circles or within a range surrounded by the connection line of the positioning circles, and combining the positioning circles and the coding circles into different visual labels, respectively, includes:
setting 5 positioning circles at different positions on the boundary of the working environment range of the robot, wherein the connecting lines of the 5 positioning circles form a 5-edge shape;
setting a plurality of coding circles in groups of 4 in different positions on the connecting line of the positioning circles or in a range surrounded by the connecting line of the positioning circles;
and respectively combining the 5 positioning circles and a group of 4 coding circles into different visual labels, wherein each visual label comprises 5 positioning circles and 4 coding circles in a selected group.
Further, in the above method, setting 5 positioning circles at different positions on the boundary of the working environment range of the robot includes:
taking a first positioning circle of the 5 positioning circles as an origin of a coordinate system of the visual label, and establishing an X axis of a first quadrant and a Y axis of the first quadrant by using the origin;
setting a second positioning circle of the 5 positioning circles at a position away from the origin on the X-axis of the first quadrant;
setting a fifth positioning circle of the 5 positioning circles at a position away from the origin on the Y-axis of the first quadrant;
arranging a third positioning circle of the 5 positioning circles in the first quadrant obliquely above the second positioning circle;
and arranging a fourth positioning circle of the 5 positioning circles right above the third positioning circle, wherein the X-axis coordinate of the fourth positioning circle is consistent with the X-axis coordinate of the third positioning circle, and the Y-axis coordinate of the fourth positioning circle is consistent with the Y-axis coordinate of the fifth positioning circle.
Further, in the above method, constructing a work environment map of the robot based on the respective visual tags includes:
acquiring visual label images of all positions in a working environment range through a camera carried on the robot;
detecting the positions of positioning circles and the positions of coding circles in the visual label image, determining a coordinate system of a visual label based on the positions of the positioning circles, coding the positions of the coding circles in the visual label image by using an N-system based on the coordinate system of the visual label, and taking the combination of the codes of the coding circles in each visual label as the unique ID number of the visual label;
and constructing a working environment map of the robot based on the unique ID number of each visual tag.
Further, in the above method, detecting the position of the positioning circle and the position of the coding circle in the visual tag image, determining a coordinate system of the visual tag based on the position of the positioning circle, coding the position of each coding circle in the visual tag image by using an N-ary system based on the coordinate system of the visual tag, and using a combination of the codes of each coding circle in each visual tag as the unique ID number of the visual tag, includes:
performing image thresholding on the visual label image to separate a target area and a background area;
detecting the outline of the target area, and adding ellipse verification after fitting the ellipse in the target area to filter out the detected abnormal ellipse and obtain a reserved normal ellipse;
clustering the reserved normal ellipses, and separating an area containing the positioning circle and the coding circle according to a distance mode;
decoding the separated region containing the positioning circle and the coding circle, comprising: and separating the positioning circle and the rest of the coding circles according to the side length relation of the separated region containing the positioning circle and the coding circle, determining a coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N system based on the coordinate system of the visual label, and combining the codes of each coding circle in each visual label as the unique ID number of the visual label.
Further, in the above method, constructing the work environment map of the robot based on the ID number unique to each visual tag includes:
the coordinates of each positioning circle in the coordinate system of the visual label are set as
Finding the corresponding visual tag coordinate system based on the unique ID number of the visual tag
To camera coordinate system
By rotation of
Wherein i = 1-n, n is the total number of the positioning circles;
according to the camera coordinate system
To the robot coordinate system
The current camera external parameter obtained by conversion
And the pose of the current robot in the world coordinate system
Coordinates of a positioning circle in the visual label are calculated according to the following formula
Conversion to coordinates in the world coordinate System
And completing the construction of the working environment map of the robot:
further, in the above method, determining the position of the robot in the work environment range based on a certain visual tag detected corresponding to the current position of the robot in the work environment map includes:
determining the pose of the current robot in a world coordinate system based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment map
。
According to another aspect of the present invention, there is also provided a visual tag-based positioning apparatus, wherein the apparatus comprises:
the robot comprises a first device, a second device and a third device, wherein the first device is used for arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
second means for constructing a work environment map of the robot based on the respective visual tags;
and the third device is used for determining the position of the robot in the working environment range based on that a certain visual label is correspondingly detected at the current position of the robot in the working environment map.
According to another aspect of the present invention, there is also provided a computing-based device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
and determining the position of the robot in the range of the working environment based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the map of the working environment.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
and determining the position of the robot in the range of the working environment based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the map of the working environment.
Compared with the prior art, the method has the advantages that the visual tags are arranged in the environment, and the robot is positioned based on all the visual tags distributed randomly in the whole space, so that many problems such as slow positioning, failure in repositioning and the like in the two current mainstream SLAM schemes can be quickly and effectively solved.
In the robot positioning navigation, the tagSLAM scheme can more efficiently solve the problems of slow repositioning recovery of the laser SLAM scheme and mismatching caused by scene similarity and the influence of illumination and scene texture structure of the visual SLAM scheme. Meanwhile, the invention can support the scheme of fusion of the visual label SLAM and the laser SLAM, and provides a more robust mapping and positioning scheme for the robot system.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 shows a layout of a visual label of an embodiment of the invention;
FIG. 2 illustrates a flow diagram for visual tag detection according to an embodiment of the present invention;
fig. 3 shows a SLAM flow diagram of a visual tag of an embodiment of the invention.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
The present invention provides a positioning method based on visual label as shown in fig. 1, the method includes:
step S1, arranging a plurality of positioning circles at different positions on the boundary of the operation environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and combining the positioning circles and the coding circles into different visual labels respectively, wherein the positioning circles and the coding circles are made of retro-reflective materials respectively, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
step S2, constructing a working environment map of the robot based on each visual label;
and step S3, determining the position of the robot in the working environment range based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment map.
Based on the defects of the conventional SLAM scheme, the invention provides the SLAM scheme (tagSLAM) based on the visual tag, and the problems of mapping and positioning of the robot can be solved relatively simply and efficiently.
According to the invention, the positioning of the position of the robot is carried out based on all the visual tags randomly distributed in the whole space only by arranging the visual tags in the environment, so that many problems of the two current mainstream SLAM schemes, such as slow positioning, failure in repositioning and the like, can be quickly and effectively solved.
In the robot positioning navigation, the tagSLAM scheme can more efficiently solve the problems of slow repositioning recovery of the laser SLAM scheme and mismatching caused by scene similarity and the influence of illumination and scene texture structure of the visual SLAM scheme. Meanwhile, the invention can support the scheme of fusion of the visual label SLAM and the laser SLAM, and provides a more robust mapping and positioning scheme for the robot system.
As shown in fig. 1, in an embodiment of the positioning method based on visual tags of the present invention, in step S1, a plurality of positioning circles are set at different positions on a boundary of a range of a working environment of a robot, a plurality of encoding circles are set at different positions on a connecting line of the positioning circles or in a range surrounded by the connecting line of the positioning circles, and the positioning circles and the encoding circles are respectively combined into different visual tags, which includes:
step S11, setting 5 positioning circles at different positions on the boundary of the operation environment range of the robot, wherein the connecting lines of the 5 positioning circles form a 5-sided shape;
step S12, setting a plurality of coding circles in groups of 4 at different positions on the connecting line of the positioning circles or in the range enclosed by the connecting line of the positioning circles;
and step S13, combining the 5 positioning circles and a group of 4 coding circles into different visual labels, wherein each visual label comprises 5 positioning circles and 4 coding circles in the selected group.
Specifically, a retro-reflective material such as a reflective film may be used to make a circle with a preset radius, 5 outer circles are selected as positioning circles and every 4 inner circles are selected as encoding circles, and the 5 outer circles and the 4 inner circles jointly form a visual label. The use of retro-reflective materials not only highlights visual labels for rapid detection, but also reduces the impact of illumination and scene, enhancing the robustness of the TagSLAM scheme.
As shown in fig. 1: the circles 1 to 5 are positioning circles of the system and are used for determining the coordinate system of the visual label, wherein the circle 1 is the origin of the coordinate system, and the XY axes are shown in fig. 1. The other circles are used as coding circles, and the unique identification ID of each visual label can be uniquely determined in an N-system coding mode. In addition, the present invention can modify the number of encoding circles to increase the encoding space according to the size of the scene and the number of required visual tags.
As shown in fig. 1, in an embodiment of the positioning method based on the visual tag of the present invention, in step S11, 5 positioning circles are set at different positions on the boundary of the working environment range of the robot, wherein a connection line of the 5 positioning circles forms a 5-sided polygon, including:
taking a first positioning circle of the 5 positioning circles as an origin of a coordinate system of the visual label, and establishing an X axis of a first quadrant and a Y axis of the first quadrant by using the origin;
setting a second positioning circle of the 5 positioning circles at a position away from the origin on the X-axis of the first quadrant;
setting a fifth positioning circle of the 5 positioning circles at a position away from the origin on the Y-axis of the first quadrant;
arranging a third positioning circle of the 5 positioning circles in the first quadrant obliquely above the second positioning circle;
and arranging a fourth positioning circle of the 5 positioning circles in the first quadrant right above the third positioning circle, wherein the X-axis coordinate of the fourth positioning circle is consistent with the X-axis coordinate of the third positioning circle, and the Y-axis coordinate of the fourth positioning circle is consistent with the Y-axis coordinate of the fifth positioning circle.
In an embodiment of the positioning method based on visual tags, in step S2, the method for constructing a work environment map of the robot based on the visual tags includes:
step S21, collecting visual label images of each position in the working environment range through a camera mounted on the robot;
here, each positioning circle and each encoding circle may be attached to the floor or ceiling of the working environment of the robot;
step S22, detecting the position of the positioning circle and the position of the coding circle in the visual label image, determining the coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using the N-system based on the coordinate system of the visual label, and using the combination of the codes of each coding circle in each visual label as the unique ID number of the visual label;
in step S23, a work environment map of the robot is constructed based on the ID number unique to each visual tag.
Specifically, after the visual tag is arranged in the scene, the system adopts a wide-angle camera carrying an infrared light supplement lens to detect the visual tag. Firstly, a coordinate system of the visual tag is determined by detecting a positioning circle of an image, then the position of the coding circle is coded by using an N-system, and the ID number of the visual tag is uniquely determined.
As shown in fig. 2, in an embodiment of the positioning method based on visual tags of the present invention, step S22, detecting the position of the positioning circle and the position of the coding circle in the visual tag image, determining the coordinate system of the visual tag based on the position of the positioning circle, coding the position of each coding circle in the visual tag image based on the coordinate system of the visual tag image by using the N-ary system, and using the combination of the codes of each coding circle in each visual tag as the ID number unique to the visual tag, includes:
step S221, carrying out image thresholding on the visual label image to separate out a target area and a background area;
step S222, detecting the outline of the target area, and adding ellipse verification after fitting the ellipse in the target area to filter out the detected abnormal ellipse and obtain a reserved normal ellipse;
step S223, clustering the reserved normal ellipses, and separating an area containing the positioning circle and the coding circle according to a distance mode;
step S224, decoding the separated region including the positioning circle and the encoding circle, including: and separating the positioning circle and the rest of the coding circles according to the side length relation of the separated region containing the positioning circle and the coding circle, determining a coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N system based on the coordinate system of the visual label, and combining the codes of each coding circle in each visual label as the unique ID number of the visual label.
As shown in fig. 3, in an embodiment of the positioning method based on visual tags of the present invention, in step S23, the method for constructing a work environment map of the robot based on the ID number unique to each visual tag includes:
step S231, setting the coordinates of each positioning circle in the coordinate system of the visual label as
The coordinate system of the corresponding visual tag is obtained based on the unique ID number of the visual tag
To camera coordinate system
By rotation of
Wherein i = 1-n, n is the total number of the positioning circles;
step S232, according to the camera coordinate system
To the robot coordinate system
Is rotatedThe transformed current camera external parameter
And the pose of the current robot in the world coordinate system
Coordinates of a positioning circle in the visual label are calculated according to the following formula
Conversion to coordinates in the world coordinate System
So as to complete the construction of the working environment map of the robot, namely the construction of the visual label:
here, before using the TagSLAM system, the internal and external parameters of the camera need to be calibrated so as to unify the coordinate system of the visual tag under the world coordinate system. In the moving process of the robot, images can be collected through a camera, a visual label at the position is detected, and the rotation transformation from a label coordinate system to a camera coordinate system is determined according to the position of a positioning circle of the label. Meanwhile, visual tags randomly distributed in the space are constructed into the whole map according to the external parameters of the camera and the initial camera reference frame, so that the tags with unique IDs are utilized to quickly realize the positioning of the robot.
Specifically, the TagSLAM uses a camera to collect and identify the visual tag, and the map construction and positioning in the robot traveling process are completed. As shown in fig. 3, the main flow of the whole system is as follows:
step 1: firstly, acquiring a current camera observation frame, extracting and identifying a visual label, and entering the next step if the visual label exists.
Step 2: according to the side length relation of the visual labels in actual deployment, the visual label can be setThe coordinates of 5 positioning corner points under the coordinate system of the label are
As shown in fig. 1. According to the camera internal parameters and the pixel coordinates of the current 5 positioning circular corner points in the image, the system can calculate the visual label coordinate system by utilizing a PNP (Positive-negative-Point) algorithm in pose estimation
To camera coordinate system
By rotation of
。
And step 3: according to the current camera external parameters
(i.e. camera coordinate system)
To the robot coordinate system
Transformation of) and pose of the current robot in the world coordinate system
The coordinates of the corner points in the visual label can be expressed according to the following formula
Conversion to coordinates in the world coordinate System
And completing the drawing of the visual label.
。
In an embodiment of the positioning method based on visual tags, in step S3, determining the position of the robot in the work environment range based on that a certain visual tag is detected corresponding to the current position of the robot in the work environment map includes:
determining the pose of the current robot in a world coordinate system based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment map
。
Here, the current robot pose can be completed according to the visual label constructed on the map before
Is estimated. Namely according to the operation environment map and formula
Can reversely deduce the current position and posture of the robot
。
Preferably, the constructed visual tag and the pose of the robot are uniformly added into a sliding window for local binding optimization (BA), and then the optimized visual tag and the pose of the robot are added into a map.
More preferably, besides the mapping thread, the system may also have a closed loop detection and optimization thread. Compared with the laser SLAM and visual SLAM scheme, the TagSLAM based on the visual tag can simply and quickly detect the closed loop according to the identification ID of the tag and perform global BA optimization. Wherein the cost function of the global BA
Are the actual observed corner coordinates
Angular point coordinates obtained by subtracting current pose estimation projection
The concrete formula is as follows:
according to another aspect of the present invention, there is also provided a visual tag-based positioning apparatus, wherein the apparatus comprises:
the robot comprises a first device, a second device and a third device, wherein the first device is used for arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
second means for constructing a work environment map of the robot based on the respective visual tags;
and the third device is used for determining the position of the robot in the working environment range based on that a certain visual label is correspondingly detected at the current position of the robot in the working environment map.
According to another aspect of the present invention, there is also provided a computing-based device, including:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
and determining the position of the robot in the range of the working environment based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the map of the working environment.
According to another aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
and determining the position of the robot in the range of the working environment based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the map of the working environment.
For details of embodiments of each device and storage medium of the present invention, reference may be made to corresponding parts of each method embodiment, and details are not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (8)
1. A visual tag-based positioning method, wherein the method comprises:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
determining the position of the robot in the working environment range based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment map;
constructing a work environment map of the robot based on the respective visual tags, comprising:
acquiring visual label images of all positions in a working environment range through a camera carried on the robot;
detecting the positions of positioning circles and the positions of coding circles in the visual label image, determining a coordinate system of a visual label based on the positions of the positioning circles, coding the positions of the coding circles in the visual label image by using an N-system based on the coordinate system of the visual label, and taking the combination of the codes of the coding circles in each visual label as the unique ID number of the visual label;
constructing a working environment map of the robot based on the unique ID number of each visual tag;
detecting the position of a positioning circle and the position of a coding circle in the visual label image, determining a coordinate system of a visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N-ary system based on the coordinate system of the visual label, and taking the combination of the codes of each coding circle in each visual label as a unique ID number of the visual label, wherein the combination comprises the following steps:
performing image thresholding on the visual label image to separate a target area and a background area;
detecting the outline of the target area, and adding ellipse verification after fitting the ellipse in the target area to filter out the detected abnormal ellipse and obtain a reserved normal ellipse;
clustering the reserved normal ellipses, and separating an area containing the positioning circle and the coding circle according to a distance mode;
decoding the separated region containing the positioning circle and the coding circle, comprising: and separating the positioning circle and the rest of the coding circles according to the side length relation of the separated region containing the positioning circle and the coding circle, determining a coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N system based on the coordinate system of the visual label, and combining the codes of each coding circle in each visual label as the unique ID number of the visual label.
2. The method of claim 1, wherein setting a plurality of positioning circles at different positions on the boundary of the range of the working environment of the robot, setting a plurality of coding circles at different positions on the line of the positioning circles or within the range surrounded by the line of the positioning circles, and combining the positioning circles and the coding circles into different visual labels, respectively, comprises:
setting 5 positioning circles at different positions on the boundary of the working environment range of the robot, wherein the connecting lines of the 5 positioning circles form a 5-edge shape;
setting a plurality of coding circles in groups of 4 in different positions on the connecting line of the positioning circles or in a range surrounded by the connecting line of the positioning circles;
and respectively combining the 5 positioning circles and a group of 4 coding circles into different visual labels, wherein each visual label comprises 5 positioning circles and 4 coding circles in a selected group.
3. The method of claim 2, wherein setting 5 positioning circles at different positions on the boundary of the working environment range of the robot comprises:
taking a first positioning circle of the 5 positioning circles as an origin of a coordinate system of the visual label, and establishing an X axis of a first quadrant and a Y axis of the first quadrant by using the origin;
setting a second positioning circle of the 5 positioning circles at a position away from the origin on the X-axis of the first quadrant;
setting a fifth positioning circle of the 5 positioning circles at a position away from the origin on the Y-axis of the first quadrant;
arranging a third positioning circle of the 5 positioning circles in the first quadrant obliquely above the second positioning circle;
and arranging a fourth positioning circle of the 5 positioning circles right above the third positioning circle, wherein the X-axis coordinate of the fourth positioning circle is consistent with the X-axis coordinate of the third positioning circle, and the Y-axis coordinate of the fourth positioning circle is consistent with the Y-axis coordinate of the fifth positioning circle.
4. The method of claim 1, wherein constructing a work environment map of the robot based on the ID number unique to each visual tag comprises:
the coordinates of each positioning circle in the coordinate system of the visual label are set as
Finding the corresponding rotation transformation T from the visual label coordinate system m to the camera coordinate system c based on the unique ID number of the visual labelcmWherein i is 1 to n, and n is the total number of the positioning circles;
obtaining the current camera external parameter T according to the conversion from the camera coordinate system c to the robot coordinate system bbcAnd the pose T of the current robot in a world coordinate systemwbCoordinates of a positioning circle in the visual label are calculated according to the following formula
Conversion to coordinates in the world coordinate System
To complete construction of a work environment map of the robot:
5. the method of claim 4, wherein determining the location of the robot within the work environment range based on the robot's corresponding detection of a visual tag at a current location in the work environment map comprises:
determining the pose T of the current robot in a world coordinate system based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment mapwb。
6. A visual tag-based pointing device, wherein the device comprises:
the robot comprises a first device, a second device and a third device, wherein the first device is used for arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
second means for constructing a work environment map of the robot based on the respective visual tags;
third means for determining a position of the robot in the work environment range based on a certain visual tag detected corresponding to a current position of the robot in the work environment map;
the second device is used for acquiring visual label images of all positions in a working environment range through a camera carried on the robot; detecting the positions of positioning circles and the positions of coding circles in the visual label image, determining a coordinate system of a visual label based on the positions of the positioning circles, coding the positions of the coding circles in the visual label image by using an N-system based on the coordinate system of the visual label, and taking the combination of the codes of the coding circles in each visual label as the unique ID number of the visual label; constructing a working environment map of the robot based on the unique ID number of each visual tag; the method comprises the following steps of detecting the position of a positioning circle and the position of a coding circle in a visual label image, determining a coordinate system of a visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N-system based on the coordinate system of the visual label, and taking the combination of the codes of each coding circle in each visual label as a unique ID number of the visual label, wherein the method comprises the following steps: performing image thresholding on the visual label image to separate a target area and a background area; detecting the outline of the target area, and adding ellipse verification after fitting the ellipse in the target area to filter out the detected abnormal ellipse and obtain a reserved normal ellipse; clustering the reserved normal ellipses, and separating an area containing the positioning circle and the coding circle according to a distance mode; decoding the separated region containing the positioning circle and the coding circle, comprising: and separating the positioning circle and the rest of the coding circles according to the side length relation of the separated region containing the positioning circle and the coding circle, determining a coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N system based on the coordinate system of the visual label, and combining the codes of each coding circle in each visual label as the unique ID number of the visual label.
7. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
determining the position of the robot in the working environment range based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment map;
constructing a work environment map of the robot based on the respective visual tags, comprising:
acquiring visual label images of all positions in a working environment range through a camera carried on the robot;
detecting the positions of positioning circles and the positions of coding circles in the visual label image, determining a coordinate system of a visual label based on the positions of the positioning circles, coding the positions of the coding circles in the visual label image by using an N-system based on the coordinate system of the visual label, and taking the combination of the codes of the coding circles in each visual label as the unique ID number of the visual label;
constructing a working environment map of the robot based on the unique ID number of each visual tag;
detecting the position of a positioning circle and the position of a coding circle in the visual label image, determining a coordinate system of a visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N-ary system based on the coordinate system of the visual label, and taking the combination of the codes of each coding circle in each visual label as a unique ID number of the visual label, wherein the combination comprises the following steps:
performing image thresholding on the visual label image to separate a target area and a background area;
detecting the outline of the target area, and adding ellipse verification after fitting the ellipse in the target area to filter out the detected abnormal ellipse and obtain a reserved normal ellipse;
clustering the reserved normal ellipses, and separating an area containing the positioning circle and the coding circle according to a distance mode;
decoding the separated region containing the positioning circle and the coding circle, comprising: and separating the positioning circle and the rest of the coding circles according to the side length relation of the separated region containing the positioning circle and the coding circle, determining a coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N system based on the coordinate system of the visual label, and combining the codes of each coding circle in each visual label as the unique ID number of the visual label.
8. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
arranging a plurality of positioning circles at different positions on the boundary of the working environment range of the robot, arranging a plurality of coding circles at different positions on the connecting line of the positioning circles or in the range surrounded by the connecting line of the positioning circles, and respectively combining the positioning circles and the coding circles into different visual labels, wherein the positioning circles and the coding circles are respectively made of retro-reflective materials, and each visual label comprises all the positioning circles and a plurality of selected coding circles at different positions;
constructing a working environment map of the robot based on the visual labels;
determining the position of the robot in the working environment range based on the fact that a certain visual label is detected correspondingly to the current position of the robot in the working environment map;
constructing a work environment map of the robot based on the respective visual tags, comprising:
acquiring visual label images of all positions in a working environment range through a camera carried on the robot;
detecting the positions of positioning circles and the positions of coding circles in the visual label image, determining a coordinate system of a visual label based on the positions of the positioning circles, coding the positions of the coding circles in the visual label image by using an N-system based on the coordinate system of the visual label, and taking the combination of the codes of the coding circles in each visual label as the unique ID number of the visual label;
constructing a working environment map of the robot based on the unique ID number of each visual tag;
detecting the position of a positioning circle and the position of a coding circle in the visual label image, determining a coordinate system of a visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N-ary system based on the coordinate system of the visual label, and taking the combination of the codes of each coding circle in each visual label as a unique ID number of the visual label, wherein the combination comprises the following steps:
performing image thresholding on the visual label image to separate a target area and a background area;
detecting the outline of the target area, and adding ellipse verification after fitting the ellipse in the target area to filter out the detected abnormal ellipse and obtain a reserved normal ellipse;
clustering the reserved normal ellipses, and separating an area containing the positioning circle and the coding circle according to a distance mode;
decoding the separated region containing the positioning circle and the coding circle, comprising: and separating the positioning circle and the rest of the coding circles according to the side length relation of the separated region containing the positioning circle and the coding circle, determining a coordinate system of the visual label based on the position of the positioning circle, coding the position of each coding circle in the visual label image by using an N system based on the coordinate system of the visual label, and combining the codes of each coding circle in each visual label as the unique ID number of the visual label.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010705312.8A CN111739088B (en) | 2020-07-21 | 2020-07-21 | Positioning method and device based on visual label |
| PCT/CN2021/072507 WO2022016838A1 (en) | 2020-07-21 | 2021-01-18 | Positioning method and device based on visual tag |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010705312.8A CN111739088B (en) | 2020-07-21 | 2020-07-21 | Positioning method and device based on visual label |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111739088A CN111739088A (en) | 2020-10-02 |
| CN111739088B true CN111739088B (en) | 2020-12-04 |
Family
ID=72655303
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010705312.8A Active CN111739088B (en) | 2020-07-21 | 2020-07-21 | Positioning method and device based on visual label |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111739088B (en) |
| WO (1) | WO2022016838A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111739088B (en) * | 2020-07-21 | 2020-12-04 | 上海思岚科技有限公司 | Positioning method and device based on visual label |
| CN113932814B (en) * | 2021-09-30 | 2024-04-02 | 杭州电子科技大学 | Collaborative positioning method based on multi-mode map |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106468553A (en) * | 2016-08-30 | 2017-03-01 | 上海擎朗智能科技有限公司 | A kind of localization method of the mobile object based on road sign |
| CN110243360A (en) * | 2018-03-08 | 2019-09-17 | 深圳市优必选科技有限公司 | Map structuring and localization method of the robot in moving region |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102773862B (en) * | 2012-07-31 | 2015-01-07 | 山东大学 | Quick and accurate locating system used for indoor mobile robot and working method thereof |
| CN106092090B (en) * | 2016-08-06 | 2023-04-25 | 合肥中科星翰科技有限公司 | Infrared road sign for positioning indoor mobile robot and use method thereof |
| US11125561B2 (en) * | 2016-09-30 | 2021-09-21 | Sony Interactive Entertainment Inc. | Steering assist |
| US10807236B2 (en) * | 2018-04-30 | 2020-10-20 | Beijing Jingdong Shangke Information Technology Co., Ltd. | System and method for multimodal mapping and localization |
| CN108827316B (en) * | 2018-08-20 | 2021-12-28 | 南京理工大学 | Mobile robot visual positioning method based on improved Apriltag |
| CN111739088B (en) * | 2020-07-21 | 2020-12-04 | 上海思岚科技有限公司 | Positioning method and device based on visual label |
-
2020
- 2020-07-21 CN CN202010705312.8A patent/CN111739088B/en active Active
-
2021
- 2021-01-18 WO PCT/CN2021/072507 patent/WO2022016838A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106468553A (en) * | 2016-08-30 | 2017-03-01 | 上海擎朗智能科技有限公司 | A kind of localization method of the mobile object based on road sign |
| CN110243360A (en) * | 2018-03-08 | 2019-09-17 | 深圳市优必选科技有限公司 | Map structuring and localization method of the robot in moving region |
Non-Patent Citations (2)
| Title |
|---|
| Visual Place Recognition by DTW-based sequence;A.H. Abdul Hafez;《2019 27th Signal Processing and Communications Applications Conference (SIU)》;20190401;全文 * |
| 基于多标签联合定位的自主视觉着陆技术研究;袁苏哲;《现代导航》;20200415(第2期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111739088A (en) | 2020-10-02 |
| WO2022016838A1 (en) | 2022-01-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6821762B2 (en) | Systems and methods for detecting POI changes using convolutional neural networks | |
| US7231063B2 (en) | Fiducial detection system | |
| Yang et al. | Human detection for a robot tractor using omni-directional stereo vision | |
| Gao et al. | Robust RGB-D simultaneous localization and mapping using planar point features | |
| WO2018014828A1 (en) | Method and system for recognizing location information in two-dimensional code | |
| US20150178293A1 (en) | Change invariant scene recognition by an agent | |
| Sagitov et al. | ARTag, AprilTag and CALTag Fiducial Marker Systems: Comparison in a Presence of Partial Marker Occlusion and Rotation. | |
| CN111739088B (en) | Positioning method and device based on visual label | |
| Lin et al. | A Robot Indoor Position and Orientation Method based on 2D Barcode Landmark. | |
| CN111540011A (en) | Hybrid metric-topology camera based positioning | |
| Son et al. | A multi-vision sensor-based fast localization system with image matching for challenging outdoor environments | |
| CN110428442A (en) | Target determines method, targeting system and monitoring security system | |
| Liang et al. | A novel skyline context descriptor for rapid localization of terrestrial laser scans to airborne laser scanning point clouds | |
| Park et al. | Estimating the camera direction of a geotagged image using reference images | |
| Jiang et al. | Multilayer map construction and vision-only multi-scale localization for intelligent vehicles in underground parking | |
| Gerhardt et al. | Neural network-based traffic sign recognition in 360° images for semi-automatic road maintenance inventory | |
| Shao | A Monocular SLAM System Based on the ORB Features | |
| CN114627365B (en) | Scene re-recognition method and device, electronic equipment and storage medium | |
| Tang | Development of a multiple-camera tracking system for accurate traffic performance measurements at intersections | |
| Aladem | Robust real-time visual odometry for autonomous ground vehicles | |
| Franceschini et al. | Damage estimation and localization from sparse aerial imagery | |
| Sujiwo et al. | Robust and accurate monocular vision-based localization in outdoor environments of real-world robot challenge | |
| Sallam Fatouh et al. | Image-based localization for augmented reality application: A Review | |
| CN112338910A (en) | Space map determination method, robot, storage medium and system | |
| Porzi et al. | An automatic image-to-DEM alignment approach for annotating mountains pictures on a smartphone |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |