KR20220152451A - Position tracking method or device of robot using natural objects of SLAM-based mobile robot, computer-readable recording medium and computer program - Google Patents

Abstract

The present invention relates to a method for tracking the location of a SLAM-based mobile robot by using natural objects, executable by the SLAM-based mobile robot. The method comprises the steps of: creating an environment map by using 2D LIDAR SLAM; obtaining a video image by using a 3D camera, recognizing a natural object from the obtained video image, and storing the location thereof as a landmark based on the created environment map; and recognizing the natural object when it is determined that it is required to correct the current location of the mobile robot during autonomous driving, and correcting the current location of the mobile robot based on the location of the stored landmark. Therefore, the method can easily create and use a map and can accurately correct the location of the mobile robot without the need for expensive equipment and cumbersome work.

Description

Position tracking method or device of robot using natural objects of SLAM-based mobile robot, computer-readable recording medium and computer program program}

The present invention relates to a method for tracking the position of a robot using natural objects recognized in a SLAM-based mobile robot, and more particularly, to a method for following the position of a robot using natural objects recognizable by a camera mounted thereon in a SLAM-based mobile robot. will be.

SLAM (Simultaneous Localization and Mapping) refers to a technique that is used for autonomous vehicles, in particular, to create a map of the surrounding environment and to recognize the current location of the mobile body within the created environment map. At this time, the self-driving moving object includes a vehicle, robot, electronic device, etc. that can move by itself through autonomous driving.

The most important technology in SLAM-based mobile robot navigation is the robot localization technology, which identifies the current position of the robot.

In the prior art, in order to know the current self-location of a mobile robot, an expensive GPS is installed to receive and localize the actual location information of the robot, or a specific mark, RFID sensor, ultrasonic sensor, Wi-Fi or BLE After installing an additional sensor, such as using the signal strength of , a technology that localizes the position of the robot when the corresponding mark is recognized has been used.

However, the method using GPS has a cost problem because expensive GPS equipment must be installed in the robot. In addition, in the case of GPS, there is a disadvantage in that the position error is large as 20 meters and the GPS data reception rate is low in a closed space such as inside a building or underground.

In addition, the method of installing a specific marker has a disadvantage in that a person must install the corresponding marker one by one on the moving path of the robot, and additional marker installation work must be performed when the path is changed.

Therefore, in the localization of mobile robots, there is a need for flexible positioning technology that does not require expensive equipment and cumbersome work and is advantageous for system construction.

KR 10-2021-0022935 A KR 10-2169283 B1

Accordingly, an object of the present invention is to provide a method and apparatus for tracking the position of a robot using natural objects recognizable in a SLAM-based mobile robot, which does not require expensive equipment and cumbersome work.

Another object of the present invention is to provide a method for tracking the position of a robot using recognizable natural objects in a SLAM-based mobile robot, which is advantageous for system construction.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The above object is, according to the first aspect of the present invention,

In the position tracking method of a robot using a recognizable natural object in a SLAM-based mobile robot,

Creating an environment map using 2D lidar SLAM;

Acquiring a video image using a 3D camera, recognizing a natural object from the acquired video image, and storing the location as a landmark based on the created environment map; and

Recognizing the natural object and correcting the current location of the mobile robot based on the location of the stored landmark when it is determined that the current location of the mobile robot needs to be corrected during autonomous driving of the mobile robot. doing,

It is achieved by the position tracking method of the robot.

At this time, the recognition of the natural object is preferably learned in advance using the YOLO algorithm.

Furthermore, the location of the landmark may be stored as absolute coordinate information based on (0,0) coordinates of the created environment map.

Meanwhile, it may be characterized in that it is determined that the current position of the mobile robot needs to be corrected by using a difference value between odometry coordinates of the mobile robot and coordinates on the environment map.

Furthermore, the step of correcting the current location of the mobile robot based on the location of the stored landmark may include converting the location of a natural object recognized during autonomous driving of the mobile robot to the location of the stored landmark and moving the mobile robot to the converted location. It may include moving the robot.

The above object is, according to the second aspect of the present invention,

In the position tracking device of a robot using natural objects of a SLAM-based mobile robot,

environmental maps created using 2D LiDAR SLAM;

a database in which locations of natural objects recognized from video images acquired using a 3D camera are stored as landmarks based on the environment map; and

When it is determined that the current position of the mobile robot needs to be corrected during autonomous driving of the mobile robot, a control unit recognizing the natural object and correcting the current position of the mobile robot based on the location of the stored landmark doing,

It is achieved by the position tracking device of the robot.

The above object is also according to the third aspect of the present invention,

This is achieved by a computer readable recording medium recording a program for performing the above method.

Furthermore, the above object is also according to the fourth aspect of the present invention,

It is achieved by a computer program stored in a medium for executing the above method through combination with hardware.

According to the method and apparatus for tracking the position of a robot using recognizable natural objects in a SLAM-based mobile robot of the present invention as described above, without the need for expensive equipment such as GPS and the cumbersome task of attaching a specific marker, accurate tracking of the mobile robot is possible. It has the advantage that position correction is possible.

In addition, according to the method and device for tracking the position of a robot using natural objects recognizable in a SLAM-based mobile robot of the present invention, compared to the 3D SLAM technique, there is no burden on the system when saving and loading an environment map, which is advantageous in system construction. .

1 is a flowchart of a method for following a position of a robot using recognizable natural objects in a SLAM-based mobile robot according to the present invention.
2 is a flowchart showing an embodiment of S200 of a method for following a position of a robot using natural objects executable in a SLAM-based mobile robot according to the present invention.
3 is a flowchart showing an embodiment of S300 of a method for following a position of a robot using a recognizable natural object in a SLAM-based mobile robot according to the present invention.
4 is a configuration diagram of a mobile robot including a position tracking device using a recognizable natural object in a SLAM-based mobile robot according to the present invention.
5 is a diagram showing an example of an environment map and DB according to the present invention.
6A to 6C are diagrams illustrating an embodiment of a process of correcting a current position in a SLAM-based mobile robot according to the present invention.

Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, the present invention is not limited or limited by exemplary embodiments. The same reference numerals presented in each figure indicate members performing substantially the same function.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

The terms used in the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art or precedent, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated.

1 is a flowchart of a method for following a position of a robot using recognizable natural objects in a SLAM-based mobile robot according to the present invention.

Referring to the drawings, a method for tracking the position of a robot using recognizable natural objects in a SLAM-based mobile robot according to the present invention includes the steps of creating an environment map (S100); Storing the location of the recognized natural object (S200); and correcting the current position of the mobile robot based on the position of the natural object recognized during autonomous driving (S300).

The steps of creating an environment map (S100) and storing the recognized natural object position (S200) of the present invention are steps for the SLAM-based mobile robot to create an environment map for the environment in which the robot will operate, conceptually using a 3D depth camera and YOLO A method of constructing an environment map is adopted by merging landmarks recognized using an algorithm with 2D SLAM using 2D LIDAR.

To this end, the mobile robot 1 according to the present invention, referring to FIG. 4, which is a configuration diagram of a mobile robot including a position tracking device using natural objects recognizable in the SLAM-based mobile robot according to the present invention, Basically 2D lidar (10); and an environment map 20 created using 2D lidar SLAM, which is the basis of the present invention, by having a 3D camera 30; and a database 40 in which locations of natural objects recognized from video images obtained using a 3D camera are stored as landmarks based on the environment map.

In addition, the mobile robot 1 equipped with a GPU interacts with the environment map 20, the encoder 60, the 3D camera 30, and the DB 40 to determine the current state of the mobile robot during autonomous driving of the mobile robot. and a control unit 50 that determines whether the location needs to be corrected and corrects the current location of the mobile robot to the location of the stored landmark.

Referring back to FIG. 1 , in the step of creating an environment map (S100), a 2D lidar SLAM is used. Of course, 3D SLAM can be implemented using 3D lidar and autonomous driving can be performed based on the corresponding 3D environment map. It is proposed as a method to compensate for the disadvantage that a lot of time is consumed in the process of loading a map. The 2D LiDAR SLAM environment map is created before the start of autonomous driving, but it can of course be modified at any time during autonomous driving as needed.

In the step of storing the position of the recognized natural object (S200), a video image is acquired using a 3D camera that stores the position of the recognized natural object, the natural object is recognized from the acquired video image, and based on the created environment map and storing the location as a landmark.

More specifically, it will be looked at with reference to FIG. 2, which is a flowchart showing an embodiment of S200 among the robot position tracking method using recognizable natural objects in the SLAM-based mobile robot according to the present invention.

First, a 3D video image (point cloud data) including not only RGB image information but also pixel depth information is obtained using a 3D depth camera (S210).

It is determined whether some of the natural objects included in the acquired 3D video image correspond to previously learned objects to be registered in the DB (S220). In this case, the object detection method for detecting the object in the image preferably uses a You Only Look Once (YOLO) algorithm, which has a simple processing process and a high processing speed. Here, ROS YOLO package provided by ROS (Robot Operation System) was used.

Here, the natural landmark is an object learned in advance and stored as a landmark, and any fixed object such as a fire hydrant, a table, a wall clock, or a picture frame can be used.

Coordinates corresponding to the location of the recognized object may be acquired using the jsk pcl package provided by ROS (Robot Operation System) (S230). The coordinates obtained at this time are, for example, absolute position coordinates of the robot based on the point of view of the mobile robot, and may be (x, y, z, θ) data based on camera coordinates.

Finally, the coordinates of the recognized object are the location of the landmark, which is absolute coordinate information based on (0,0) coordinates in the xy coordinate system of the created environment map or (0, 0, 0) in the xyz coordinate system. It is stored in the DB (S240). In addition to this, the recognized object may be displayed and stored as a landmark on the corresponding environment map. At this time, when registering a recognized object as a landmark, in addition to using the YOLO algorithm, a separate image reinforcement learning tool, for example, Colab, may be used for learning and addition.

For an implementation example of the environment map 20 and DB 40, refer to FIG. 5, which is a diagram showing an example of the environment map and DB according to the present invention.

In this way, using the method for generating an environment map proposed in the present invention, a landmark for a specific location can be created in a 2D SLAM configuration. In addition, since the corresponding environment map is stored in the same low capacity as the 2D map, and the location of the landmark is stored separately as a DB file, there is no great burden on the storage device and there is no system load problem in loading the environment map. .

On the other hand, in the step of correcting the current position of the mobile robot based on the position of the natural object recognized during autonomous driving (S300), when it is determined that it is necessary to correct the current position of the mobile robot during autonomous driving, the corresponding natural object Recognizing the object and correcting the current location of the mobile robot based on the location of the stored landmark.

More specifically, it will be looked at with reference to FIG. 3, which is a flowchart showing an embodiment of S300 of a method for tracking the position of a robot using natural objects recognizable in a SLAM-based mobile robot according to the present invention.

First, a 3D video image is acquired using a 3D depth camera (S310).

In autonomous driving based on the environment map, it is determined whether current position correction is necessary (S320). Steps S310 and S320 may occur periodically and in any relative order.

In general, the current position of a mobile robot follows the robot's position/posture (Odometry) using information from encoders installed on driving wheels. The location based on the information of , and the location on the environment map based on the information from the 2D rider may be different.

In this case, it is possible to determine whether the current position of the mobile robot needs to be corrected by using the difference between the odometry coordinates from the encoder and the coordinates on the environment map. Needless to say, it is possible to define whether or not correction is necessary based on an arbitrary threshold value or range.

For example, when it is determined that the current position of the mobile robot needs to be corrected, it may be when it is determined that the current position tracking error is accumulated. It may be a difference between location information obtained from SLAM.

If it is determined that the current location correction is necessary, it is determined whether some of the natural objects included in the 3D video image obtained above are registered in the DB as landmarks (S330). In this case, the object detection method for detecting the object in the image preferably uses a You Only Look Once (YOLO) algorithm, which has a simple processing process and a high processing speed. Here, ROS YOLO package provided by ROS (Robot Operation System) was used. In addition, whether or not the recognized natural object is registered in the DB as a landmark is determined based on a comparison between the recognized natural object and the landmark stored in the DB.

If the recognized natural object matches the information indicating the landmark stored in the DB, the current location of the mobile robot is corrected based on the location of the landmark (S340). At this time, correcting the current position of the mobile robot based on the position of the corresponding landmark is to convert the relative coordinates of the recognized natural object, that is, the registered landmark, to the absolute coordinates of the map based on the center coordinates of the camera mounted on the mobile robot. It can be actually achieved by converting it into coordinates and moving the mobile robot to the corresponding position.

This will be examined in more detail with reference to FIGS. 6A to 6C, which are diagrams showing an embodiment of a process of correcting a current position in a SLAM-based mobile robot according to the present invention.

First, when starting from the starting point (the start point indicated by a circle in the drawing) and moving downward in the drawing, when driving normally without slipping with the floor, 2D lidar data (indicated by a colored line in the drawing) is displayed on the wall of the map. It is marked as matching information (see Fig. 6a)

On the other hand, when starting from the starting point (the start point indicated by a circle in the drawing) and moving downward on the drawing, sliding with the floor occurs and the robot moves distorted, the 2D lidar data is the wall information of the map. is marked as inconsistent with (See FIG. 6B) At this time, it can be seen in the drawing that red arrows are accumulated in front of the mobile robot due to an error in position estimation.

As shown in FIG. 6B, if it is determined that the current position needs to be corrected because the position of the mobile robot is distorted, the YOLO algorithm is used to check whether the recognized fire extinguisher (indicated as a fire extinguisher in the drawing) is a registered landmark when creating an environmental map. Then, when the relative coordinates of the recognized fire extinguisher are converted into absolute coordinates of the registered landmark and the mobile robot is moved to the converted position, it can be confirmed that the 2D lidar data matches the wall information of the map as shown in FIG. 6c.

In this way, when performing autonomous driving based on a pre-created environment map, when the position/posture of the mobile robot is distorted due to slippage with the floor or external environmental factors, natural objects recognized by the 3D depth camera If it coincides with a landmark registered in the DB in advance, it is possible to localize the position of the robot using the coordinates stored in the corresponding landmark, so there is an advantage in that the position/posture of the mobile robot can be corrected.

Meanwhile, the methods according to the embodiments of the present invention may be at least partially implemented in the form of program instructions and recorded on a computer-readable recording medium. For example, implemented together with a program product consisting of a computer-readable medium containing program code, which may be executed by a processor to perform any or all steps, operations, or processes described.

The computer is a computing device having one or more alternative and special purpose processors, memory, storage, and networking components (whether wireless or wired). The computer may run, for example, an operating system compatible with Microsoft's Windows, for example, an operating system such as a Linux distribution.

The program instruction form may be collectively referred to as software, which may include a computer program, code, instruction, or a combination of one or more of them, and may cause a processing device to operate as desired. It can configure or command processing units independently or collectively. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or to provide instructions or data to a processing device. may be permanently or temporarily embodied in

The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

In general, the terms used herein are generally intended to be "open-ended" terms, especially in the claims (e.g., the body of a claim) (e.g., "comprising" means "including but not limited to" , "has" should be construed as "has at least more", and "includes" as "including, but not limited to") If a specific number is intended for an introduced claim statement, such intent It is understood that as expressly recited in the claims, in the absence of such recitation, such intent is not present.

While only certain features of the present invention have been shown and described herein, various modifications and changes may occur to those skilled in the art. It is understood, therefore, that the claims are intended to cover such changes and modifications as fall within the spirit of the present invention.

10: 2D LiDAR 20: Environment Map
30: 3D camera 40: DB
50: control unit 60: encoder

Claims (12)

In the position tracking method of a robot using a recognizable natural object in a SLAM-based mobile robot,
Creating an environment map using 2D lidar SLAM;
Acquiring a video image using a 3D camera, recognizing a natural object from the acquired video image, and storing the location as a landmark based on the created environment map; and
Recognizing the natural object and correcting the current location of the mobile robot based on the location of the stored landmark when it is determined that the current location of the mobile robot needs to be corrected during autonomous driving of the mobile robot. doing,
How to follow the position of the robot. According to claim 1,
Characterized in that the recognition of the natural object is learned in advance using the YOLO algorithm,
How to follow the position of the robot. According to claim 2,
The location of the landmark is stored as absolute coordinate information based on (0,0) coordinates of the created environment map,
How to follow the position of the robot. According to claim 3,
Characterized in that it is determined that the current position of the mobile robot needs to be corrected using a difference between the odometry coordinates of the mobile robot and the coordinates on the environment map.
How to follow the position of the robot. According to claim 4,
The step of correcting the current position of the mobile robot based on the position of the stored landmark may include converting the position of a natural object recognized during autonomous driving of the mobile robot into the position of the stored landmark and moving the mobile robot to the converted position. Characterized in that it comprises the step of moving,
How to follow the position of the robot. In the position tracking device of a robot using natural objects of a SLAM-based mobile robot,
environmental maps created using 2D LiDAR SLAM;
a database in which locations of natural objects recognized from video images acquired using a 3D camera are stored as landmarks based on the environment map; and
When it is determined that the current position of the mobile robot needs to be corrected during autonomous driving of the mobile robot, a control unit recognizing the natural object and correcting the current position of the mobile robot based on the location of the stored landmark doing,
Robot position tracking device. According to claim 6,
Characterized in that the recognition of the natural object is learned in advance using the YOLO algorithm,
Robot position tracking device. According to claim 7,
The location of the landmark is stored as absolute coordinate information based on (0,0) coordinates of the created environment map,
Robot position tracking device. The method of claim 8, wherein the controller
Characterized in that it is determined that the current position of the mobile robot needs to be corrected using a difference between the odometry coordinates of the mobile robot and the coordinates on the environment map.
Robot position tracking device. According to claim 9,
Correcting the current location of the mobile robot based on the location of the stored landmark converts the location of a natural object recognized during autonomous driving of the mobile robot into the location of the stored landmark and moves the mobile robot to the converted location. Characterized in that it includes,
Robot position tracking device. A computer-readable recording medium recording a program for performing the method of any one of claims 1 to 5. A computer program stored in a medium for executing the method of any one of claims 1 to 5 through combination with hardware.

Images