CN114947653A - Visual and laser fusion slam method and system based on hotel cleaning robot - Google Patents

Abstract

The present invention provides a vision and laser fusion slam method and system based on a hotel cleaning robot, including: step S1: establishing a grid map and marking the location of a preset location on the map; step S2: generating real-time generation according to the preset location Obstacle avoidance map; Step S3 : travel to the preset location according to the obstacle avoidance map to identify the target object in the preset location, and obtain the working point; Step S4 : Feedback the information of reaching the designated working point. The invention guarantees the safety of the cleaning robot during the entire task traveling process, uses vision to intelligently identify the target object, saves the time for manually marking each work point during the cleaning process, and greatly reduces the risk of manual marking errors.

Description

Vision and laser fusion SLAM method and system based on hotel cleaning robot

technical field

The invention relates to the field of robot navigation, in particular, to a vision and laser fusion slam method and system based on a hotel cleaning robot.

Background technique

The hotel cleaning robot is a special robot that works with hotel bathrooms. Because the robot carries a series of mechanical equipment (such as mechanical arms, cameras, electric grippers, electric brushes, etc.), its body is relatively large and irregular, and it may travel There will be many obstacles in space, especially the manipulator may touch obstacles or people in the retracted state, so relying solely on the two-dimensional lidar sensor cannot meet the needs of safe travel, this article is based on this A visual sensor is added to increase the obstacle avoidance capability of the space. The slam system composed of visual sensor and laser sensor will greatly increase the safety and stability of the cleaning robot.

Patent document CN109144067A (application number: CN201811083718.6) discloses an intelligent cleaning robot and its path planning method. The sensor module is used to analyze and feed back real-time cleaning environment information; the precise positioning module is used to obtain the current intelligent cleaning robot on the environment map Location; use the geometric-topological hybrid map technology to build an environmental map, and use the advanced path planning algorithm to plan the optimal cleaning path by combining the environmental map and real-time location, and upload the data to the cloud platform to realize real-time analysis, record and control; drive module It is used to drive the intelligent cleaning robot to run and clean according to the planned optimal path; the human-computer interaction module can use the temperature and humidity sensor combined with the camera to display the working status and performance of the intelligent cleaning robot, and can be completed through wifi/Bluetooth technology Remote control and reservation function of intelligent cleaning robot. However, the invention has insufficient safety guarantee for the robot.

SUMMARY OF THE INVENTION

Aiming at the defects in the prior art, the purpose of the present invention is to provide a vision and laser fusion slam method and system based on a hotel cleaning robot.

A vision and laser fusion slam method based on a hotel cleaning robot provided according to the present invention includes:

Step S1: establishing a grid map and marking the location of the preset location on the map;

Step S2: generating an obstacle avoidance map in real time according to a preset location;

Step S3: traveling to the preset location according to the obstacle avoidance map to identify the target object in the preset location, and obtain the working point;

Step S4: Feeding back the information of arriving at the designated work point.

Preferably, in the step S1:

Use 2D lidar to build a 2D grid map for the corresponding navigation.

Preferably, in the step S2:

The robot travels according to the preset location, and the planning layer generates an obstacle avoidance map with a preset range in real time during the process of traveling. The obstacle avoidance map is used for the robot's real-time dynamic obstacle avoidance. The obstacle avoidance map integrates visual and laser data information. Obstacle Avoidance Map Avoid preset obstacles in space.

Preferably, the operation process includes three parts: the task layer, the execution layer and the feedback layer; the task layer is responsible for task planning, task decomposition and the distribution of each branch task; the execution layer is responsible for re-decomposing the distributed tasks by each branch module And execute it at one time; the feedback layer gives feedback to the upper layer after each module has completed the corresponding task or issues an alarm to the upper layer when encountering a problem;

The execution layer includes the navigation module. The navigation module is responsible for letting the robot reach the designated work point or work area. The planning layer in the navigation module is responsible for the execution of the robot's walking. The planning layer plans the global path and the local path. The global path is from the starting position to the target. The complete planning route of the location is planned according to the principle of the shortest path; the local path is the local path planning within the preset range of the robot's walking process, avoiding the dynamic obstacles that are not on the map, adding vision for spatial obstacle avoidance, and visualizing when traveling. Detects moving or static space obstacles in the forward space in real time, and adds the obstacles to the local obstacle avoidance map.

Preferably, in the step S3:

When the robot arrives at the preset location, it enters the preset area, initially recognizes the preset area through vision, and uses relevant deep learning methods to find the desired target object in the photographed environment. To the target position, the point cloud matching algorithm is used to pair the object model established in advance with the target object, correct the navigation point, and obtain the working point.

A vision and laser fusion slam system based on a hotel cleaning robot provided according to the present invention includes:

Module M1: Create a grid map and mark the location of the preset location on the map;

Module M2: generate an obstacle avoidance map in real time according to a preset location;

Module M3: travel to the preset location according to the obstacle avoidance map, identify the target object in the preset location, and obtain the working point;

Module M4: Feedback the information that the designated working point is reached.

Preferably, in the module M1:

Use 2D lidar to build a 2D grid map for the corresponding navigation.

Preferably, in the module M2:

The robot travels according to the preset location, and the planning layer generates an obstacle avoidance map with a preset range in real time during the process of traveling. The obstacle avoidance map is used for the robot's real-time dynamic obstacle avoidance. The obstacle avoidance map integrates visual and laser data information. Obstacle Avoidance Map Avoid preset obstacles in space.

Preferably, the operation process includes three parts: the task layer, the execution layer and the feedback layer; the task layer is responsible for task planning, task decomposition and the distribution of each branch task; the execution layer is responsible for re-decomposing the distributed tasks by each branch module And execute it at one time; the feedback layer gives feedback to the upper layer after each module has completed the corresponding task or issues an alarm to the upper layer when encountering a problem;

The execution layer includes the navigation module. The navigation module is responsible for letting the robot reach the designated work point or work area. The planning layer in the navigation module is responsible for the execution of the robot's walking. The planning layer plans the global path and the local path. The global path is from the starting position to the target. The complete planning route of the location is planned according to the principle of the shortest path; the local path is the local path planning within the preset range of the robot's walking process, avoiding the dynamic obstacles that are not on the map, adding vision for spatial obstacle avoidance, and visualizing when traveling. Detects moving or static space obstacles in the forward space in real time, and adds the obstacles to the local obstacle avoidance map.

Preferably, in the module M3:

When the robot arrives at the preset location, it enters the preset area, initially recognizes the preset area through vision, and uses relevant deep learning methods to find the desired target object in the photographed environment. To the target position, the point cloud matching algorithm is used to pair the object model established in advance with the target object, correct the navigation point, and obtain the working point.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention ensures the safety of the cleaning robot during the entire mission process;

2. The present invention uses vision to perform intelligent identification of target objects, which saves the time for manually marking each work point in the cleaning process, and greatly reduces the risk of errors in manual marking.

Description of drawings

Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

Fig. 1 is a flow chart of the present invention.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the inventive concept. These all belong to the protection scope of the present invention.

Example 1:

A vision and laser fusion slam method based on a hotel cleaning robot provided according to the present invention, as shown in FIG. 1 , includes:

Step S1: establishing a grid map and marking the location of the preset location on the map;

Specifically, in the step S1:

Use 2D lidar to build a 2D grid map for the corresponding navigation.

Step S2: generating an obstacle avoidance map in real time according to a preset location;

Specifically, in the step S2:

The robot travels according to the preset location, and the planning layer generates an obstacle avoidance map with a preset range in real time during the process of traveling. The obstacle avoidance map is used for the robot's real-time dynamic obstacle avoidance. The obstacle avoidance map integrates visual and laser data information. Obstacle Avoidance Map Avoid preset obstacles in space.

Specifically, the job process includes three parts: the task layer, the execution layer and the feedback layer; the task layer is responsible for task planning, task decomposition, and the distribution of branch tasks; the execution layer is responsible for re-decomposing the distributed tasks by each branch module And execute it at one time; the feedback layer gives feedback to the upper layer after each module has completed the corresponding task or issues an alarm to the upper layer when encountering a problem;

The execution layer includes the navigation module. The navigation module is responsible for letting the robot reach the designated work point or work area. The planning layer in the navigation module is responsible for the execution of the robot's walking. The planning layer plans the global path and the local path. The global path is from the starting position to the target. The complete planning route of the location is planned according to the principle of the shortest path; the local path is the local path planning within the preset range of the robot's walking process, avoiding the dynamic obstacles that are not on the map, adding vision for spatial obstacle avoidance, and visualizing when traveling. Detects moving or static space obstacles in the forward space in real time, and adds the obstacles to the local obstacle avoidance map.

Step S3: traveling to the preset location according to the obstacle avoidance map to identify the target object in the preset location, and obtain the working point;

Specifically, in the step S3:

When the robot arrives at the preset location, it enters the preset area, initially recognizes the preset area through vision, and uses relevant deep learning methods to find the desired target object in the photographed environment. To the target position, the point cloud matching algorithm is used to pair the object model established in advance with the target object, correct the navigation point, and obtain the working point.

Step S4: Feeding back the information of arriving at the designated work point.

Example 2:

Embodiment 2 is a preferred example of Embodiment 1, in order to describe the present invention in more detail.

Those skilled in the art can understand a hotel cleaning robot-based vision and laser fusion slam method provided by the present invention as a specific implementation of a hotel cleaning robot-based vision and laser fusion slam system, that is, the hotel cleaning robot-based The vision and laser fusion slam system can be realized by executing the steps of the described hotel cleaning robot-based vision and laser fusion slam method.

A vision and laser fusion slam system based on a hotel cleaning robot provided according to the present invention includes:

Module M1: Create a grid map and mark the location of the preset location on the map;

Specifically, in the module M1:

Use 2D lidar to build a 2D grid map for the corresponding navigation.

Module M2: generate an obstacle avoidance map in real time according to a preset location;

Specifically, in the module M2:

The robot travels according to the preset location, and the planning layer generates an obstacle avoidance map with a preset range in real time during the process of traveling. The obstacle avoidance map is used for the robot's real-time dynamic obstacle avoidance. The obstacle avoidance map integrates visual and laser data information. Obstacle Avoidance Map Avoid preset obstacles in space.

Specifically, the job process includes three parts: the task layer, the execution layer and the feedback layer; the task layer is responsible for task planning, task decomposition, and the distribution of branch tasks; the execution layer is responsible for re-decomposing the distributed tasks by each branch module And execute it at one time; the feedback layer gives feedback to the upper layer after each module has completed the corresponding task or issues an alarm to the upper layer when encountering a problem;

The execution layer includes the navigation module. The navigation module is responsible for letting the robot reach the designated work point or work area. The planning layer in the navigation module is responsible for the execution of the robot's walking. The planning layer plans the global path and the local path. The global path is from the starting position to the target. The complete planning route of the location is planned according to the principle of the shortest path; the local path is the local path planning within the preset range of the robot's walking process, avoiding the dynamic obstacles that are not on the map, adding vision for spatial obstacle avoidance, and visualizing when traveling. Detects moving or static space obstacles in the forward space in real time, and adds the obstacles to the local obstacle avoidance map.

Module M3: travel to the preset location according to the obstacle avoidance map, identify the target object in the preset location, and obtain the working point;

Specifically, in the module M3:

When the robot arrives at the preset location, it enters the preset area, initially recognizes the preset area through vision, and uses relevant deep learning methods to find the desired target object in the photographed environment. To the target position, the point cloud matching algorithm is used to pair the object model established in advance with the target object, correct the navigation point, and obtain the working point.

Module M4: Feedback the information that the designated working point is reached.

Example 3:

Embodiment 3 is a preferred example of Embodiment 1, in order to describe the present invention in more detail.

Aiming at the problem that the traditional two-dimensional laser slam algorithm cannot be used due to the narrowness of the hotel bathroom at present, the technical problems to be solved by the present invention are embodied in the following points:

1) The slam method that combines vision and laser to ensure the safety of the robot in the process of traveling

2) Vision can also bring more spatial information to the two-dimensional lidar, so that the robot can identify the target object autonomously, and with a certain judgment logic, it can autonomously mark the target point and reach it.

The method includes the following steps:

Step 1: Use 2D LiDAR to establish a 2D grid map used for corresponding navigation;

Step 2: Mark the location of the door of each room and the location of the bathroom in the room on the map;

Step 3: With the assignment of the upper-level task, the robot starts to move towards the door of the first room, and the planning layer will generate a certain range of obstacle avoidance map in real time during the process of traveling, which is used for The real-time dynamic obstacle avoidance of the robot, the map incorporates visual and laser data information to ensure that all kinds of obstacles in space can be avoided;

Step 4: When the robot arrives at the door of the room, it can enter the bathroom of the corresponding room. Since the space in the bathroom is relatively small, the room can be visually identified at this time, and the relevant deep learning methods can be used to find all the objects in the captured environment. Targets (such as basins, mirrors, toilets, etc.) are required. When the target is found, it will drive to the target position through local path planning, and then use the point cloud matching algorithm to match the object model established in advance with the target object to correct the final navigation. point to get the final working point;

Step 5: The upper layer is notified through the feedback layer that it has reached the designated work point and waits for the next task to be issued.

Wherein, step 3 includes the following steps:

Step 3.1: The cleaning process consists of three parts: the task layer, the execution layer and the feedback layer. The task layer is mainly responsible for task planning, task decomposition and the distribution of each branch task, while the execution layer is the task issued by each branch module. It is decomposed and executed at one time. The feedback layer is that each module provides feedback to the upper layer after completing the corresponding task or sends an alarm signal to the upper layer when it encounters any problems;

Step 3.2: The navigation module is a branch module in the execution layer. This module is responsible for letting the robot reach the designated work point or work area, and the execution part responsible for making the robot walk is the navigation planning layer, and the planning layer plans the overall situation Path and local path, the global path is the complete planned route from the starting position to the target position (theoretically planned based on the principle of the shortest path), while the local path is the local path planning within a certain range of the robot’s walking process (mainly to avoid Open some dynamic obstacles that are not on the map), because the two-dimensional lidar can only detect obstacles of a certain height, and cannot guarantee the overall safety of the cleaning robot, so add vision to avoid obstacles in space, and the vision will be real-time when traveling. After detecting a moving or static space obstacle in the front space, and adding the obstacle to the local obstacle avoidance map, the local planner can plan a safe path accordingly.

Those skilled in the art know that, in addition to implementing the system, device and each module provided by the present invention in the form of pure computer readable program code, the system, device and each module provided by the present invention can be completely implemented by logically programming the method steps. The same program is implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers, among others. Therefore, the system, device and each module provided by the present invention can be regarded as a kind of hardware component, and the modules used for realizing various programs included in it can also be regarded as the structure in the hardware component; A module for realizing various functions can be regarded as either a software program for realizing a method or a structure within a hardware component.

Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essential content of the present invention. The embodiments of the present application and features in the embodiments may be combined with each other arbitrarily, provided that there is no conflict.

Claims (10)

1. a vision and laser fusion slam method based on hotel cleaning robot, is characterized in that, comprises: Step S1: establishing a grid map and marking the location of the preset location on the map; Step S2: generating an obstacle avoidance map in real time according to a preset location; Step S3: traveling to the preset location according to the obstacle avoidance map to identify the target object in the preset location, and obtain the working point; Step S4: Feeding back the information of arriving at the designated work point. 2. the vision and laser fusion slam method based on hotel cleaning robot according to claim 1, is characterized in that, in described step S1: Use 2D lidar to build a 2D grid map for the corresponding navigation. 3. the vision and laser fusion slam method based on hotel cleaning robot according to claim 1, is characterized in that, in described step S2: The robot travels according to the preset location, and the planning layer generates an obstacle avoidance map with a preset range in real time during the process of traveling. The obstacle avoidance map is used for the robot's real-time dynamic obstacle avoidance. The obstacle avoidance map integrates visual and laser data information. Obstacle Avoidance Map Avoid preset obstacles in space. 4. the vision and laser fusion slam method based on hotel cleaning robot according to claim 3, is characterized in that: The job process consists of three parts: the task layer, the execution layer and the feedback layer; the task layer is responsible for task planning, task decomposition and the distribution of branch tasks; the execution layer is responsible for re-decomposing the tasks issued by each branch module and executing them at one time ;The feedback layer gives feedback to the upper layer after each module performs the corresponding task or issues an alarm to the upper layer when encountering a problem; The execution layer includes the navigation module. The navigation module is responsible for letting the robot reach the designated work point or work area. The planning layer in the navigation module is responsible for the execution of the robot's walking. The planning layer plans the global path and the local path. The global path is from the starting position to the target. The complete planning route of the location is planned according to the principle of the shortest path; the local path is the local path planning within the preset range of the robot's walking process, avoiding the dynamic obstacles that are not on the map, adding vision for spatial obstacle avoidance, and visualizing when traveling. Detects moving or static space obstacles in the forward space in real time, and adds the obstacles to the local obstacle avoidance map. 5. the vision and laser fusion slam method based on hotel cleaning robot according to claim 1, is characterized in that, in described step S3: When the robot arrives at the preset location, it enters the preset area, initially recognizes the preset area through vision, and uses relevant deep learning methods to find the desired target object in the photographed environment. To the target position, the point cloud matching algorithm is used to pair the object model established in advance with the target object, correct the navigation point, and obtain the working point. 6. A vision and laser fusion slam system based on a hotel cleaning robot, characterized in that, comprising: Module M1: Create a grid map and mark the location of the preset location on the map; Module M2: generate an obstacle avoidance map in real time according to a preset location; Module M3: travel to the preset location according to the obstacle avoidance map, identify the target object in the preset location, and obtain the working point; Module M4: Feedback the information that the designated working point is reached. 7. the vision and laser fusion slam system based on hotel cleaning robot according to claim 6, is characterized in that, in described module M1: Use 2D lidar to build a 2D grid map for the corresponding navigation. 8. the vision and laser fusion slam system based on hotel cleaning robot according to claim 6, is characterized in that, in described module M2: The robot travels according to the preset location, and the planning layer generates an obstacle avoidance map with a preset range in real time during the process of traveling. The obstacle avoidance map is used for the robot's real-time dynamic obstacle avoidance. The obstacle avoidance map integrates visual and laser data information. Obstacle Avoidance Map Avoid preset obstacles in space. 9. the vision and laser fusion slam system based on hotel cleaning robot according to claim 8, is characterized in that: The job process consists of three parts: the task layer, the execution layer and the feedback layer; the task layer is responsible for task planning, task decomposition and the distribution of branch tasks; the execution layer is responsible for re-decomposing the tasks issued by each branch module and executing them at one time ;The feedback layer gives feedback to the upper layer after each module performs the corresponding task or issues an alarm to the upper layer when encountering a problem; The execution layer includes the navigation module. The navigation module is responsible for letting the robot reach the designated work point or work area. The planning layer in the navigation module is responsible for the execution of the robot's walking. The planning layer plans the global path and the local path. The global path is from the starting position to the target. The complete planning route of the location is planned according to the principle of the shortest path; the local path is the local path planning within the preset range of the robot's walking process, avoiding the dynamic obstacles that are not on the map, adding vision for spatial obstacle avoidance, and visualizing when traveling. Detects moving or static space obstacles in the forward space in real time, and adds the obstacles to the local obstacle avoidance map. 10. the vision and laser fusion slam system based on hotel cleaning robot according to claim 6, is characterized in that, in described module M3: When the robot arrives at the preset location, it enters the preset area, initially recognizes the preset area through vision, and uses relevant deep learning methods to find the desired target object in the photographed environment. To the target position, the point cloud matching algorithm is used to pair the object model established in advance with the target object, correct the navigation point, and obtain the working point.

Images