CN112882481A - Mobile multi-mode interactive navigation robot system based on SLAM - Google Patents

Abstract

The invention relates to a mobile multi-mode interactive navigation robot system based on SLAM, comprising: the mobile control module comprises a SLAM based on a laser radar and is used for controlling the movement of the robot and acquiring current positioning information; the mobile navigation module is electrically connected with the mobile control module and used for receiving the current positioning information and performing fixed-point cruise explanation by combining preset path planning according to the current positioning information; the information display module is electrically connected with the mobile control module and used for displaying preset information and sending the received motion control instruction to the mobile control module; and the voice interaction module adopts a C/S (client/server) framework, is respectively in communication connection with the information display module and the mobile navigation module, adopts a deep learning chat system and is used for realizing man-machine interaction, and the man-machine interaction comprises the step of receiving a motion control instruction. The invention reduces the cost of the robot.

Description

A mobile multi-modal interactive navigation robot system based on SLAM

technical field

The invention relates to the technical field of navigation robots, in particular to a mobile multi-modal interactive navigation robot system based on SLAM.

Background technique

At present, China's domestic leisure agriculture and rural tourism are booming, and the ability to help rural revitalization is continuously enhanced. Through going deep into Changshun County, Guizhou Province, in the process of poverty alleviation, I learned that the local characteristic scenic spots have the characteristics of "few tour guides, wide area and long lines", and the contradiction between the surge of tourists and fewer tour guides hinders the development of scenic spots. . Although there are currently guide robots to replace tour guides, the cost of the background used by the current guide robots is very high.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a mobile multi-modal interactive navigation robot system based on SLAM, which reduces the cost of the robot.

For achieving the above object, the present invention provides the following scheme:

A mobile multi-modal interactive guided robot system based on SLAM, including:

The movement control module, including SLAM based on lidar, is used to control the movement of the robot and obtain the current positioning information;

a mobile navigation module, electrically connected to the mobile control module, for receiving the current positioning information, and performing fixed-point cruise explanation according to the current positioning information combined with a preset path plan;

an information display module, electrically connected to the mobile control module, for displaying preset information, and sending the received motion control instruction to the mobile control module;

The voice interaction module adopts a C/S architecture, and is connected to the information display module and the mobile navigation module respectively, and adopts a deep learning chat system to realize human-computer interaction, and the human-computer interaction includes receiving motion control instructions .

Optionally, the movement control module includes:

The mapping unit is used to receive the data of the lidar in real time, and build a two-dimensional map of the surrounding environment according to the data of the lidar;

a positioning unit, which determines the pose of the robot on the two-dimensional map according to the data of the lidar and the mileage data of the robot;

A navigation unit, configured to perform global path planning according to the destination and departure place of the robot, and perform local real-time planning according to detected obstacles when the robot moves according to the global path planning.

Optionally, the local real-time planning is implemented through a dynamic window method.

Optionally, the global path planning is implemented through Dijkstra's algorithm.

Optionally, the voice interaction module includes a front-end and a server;

The front end is used to collect sound signals, convert the sound signals into text information, send the text information to the server and receive return information from the server, and the return information is converted into human voice based on the speech synthesis SDK ;

The server is used for receiving the text information, performing intention recognition and dialogue processing on the text information based on the synergy between the retrieval class and the generation class, and sending return information.

Optionally, the front end adopts iFLYTEK annular six-mic array.

Optionally, the server is a cloud server.

Optionally, an ultrasonic sensor is also included for detecting obstacle signals around the robot, and sending the obstacle signals to the movement control module.

Optionally, the mapping unit uses a GMapping algorithm to construct a two-dimensional map of the surrounding environment according to the data of the lidar.

Optionally, the positioning unit adopts an active Monte Carlo particle filter positioning algorithm to determine the pose of the robot on the two-dimensional map according to the data of the lidar and the mileage data of the robot.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The mobile control module of the present invention includes SLAM based on laser radar, which is used to control the movement of the robot, realize the autonomous navigation of the robot, and improve the navigation efficiency through the mobile navigation module. The information display module is in communication connection with the mobile navigation module, and a deep learning chatting system is used to realize human-computer interaction and reduce the background cost of the navigation robot.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative labor.

1 is a schematic structural diagram of a mobile multi-modal interactive navigation robot system based on SLAM of the present invention;

Fig. 2 is the schematic flow chart of two-dimensional map construction of the present invention;

3 is a schematic diagram of a particle swarm convergence process in the active Monte Carlo particle filter positioning algorithm of the present invention;

4 is a schematic diagram of the working principle of the voice interaction module of the present invention;

FIG. 5 is a schematic diagram of a fixed-point explanation function page of the mobile navigation module of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The purpose of the present invention is to provide a mobile multi-modal interactive navigation robot system based on SLAM, which reduces the cost of the robot.

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

FIG. 1 is a schematic structural diagram of a SLAM-based mobile multi-modal interactive navigation robot system according to the present invention. As shown in FIG. 1 , a SLAM-based mobile multi-modal interactive navigation robot system includes:

The movement control module 101 includes a SLAM based on lidar, and is used to control the movement of the robot through the mobile base 106 and acquire current positioning information at the same time.

The lidar scans the environmental distance information at a fixed frequency, and the mapping, positioning and navigation nodes in SLAM will receive the lidar data information.

The movement control module 101 includes:

The mapping unit is used to receive the data of the lidar in real time, and build a two-dimensional map of the surrounding environment according to the data of the lidar.

The positioning unit determines the pose of the robot on the two-dimensional map according to the laser radar data and the robot mileage data.

A navigation unit, configured to perform global path planning according to the destination and departure place of the robot, and perform local real-time planning according to detected obstacles when the robot moves according to the global path planning.

The local real-time planning is realized by a dynamic window method.

The global path planning is realized by Dijkstra's algorithm.

The robot system further includes an ultrasonic sensor for detecting obstacle signals around the robot and sending the obstacle signals to the movement control module 101 .

The mapping unit uses the GMapping algorithm to construct a two-dimensional map of the surrounding environment according to the data of the lidar.

The positioning unit adopts an active Monte Carlo particle filter positioning algorithm, and determines the pose of the robot on the two-dimensional map according to the data of the lidar and the mileage data of the robot.

The specific working principle involved in the movement control module 101 is as follows:

。①Mapping, the main workflow of the mapping algorithm used in the mapping unit is to continuously receive the input of lidar data during the movement of the robot, so as to construct a complete two-dimensional map of the surrounding environment. The mapping algorithm used is GMapping. GMapping is based on the RBpf particle filter algorithm, that is, the positioning and mapping process are separated, and the positioning is performed first and then the mapping is performed. The mapping process based on particle filtering will preset many particles, and each particle carries the possible pose and current map of the robot. The RBpf algorithm needs to calculate the probability of each particle separately. The formula for calculating the probability is:

.

为一个联合概率分布，目的是根据时刻1到时刻t的观测数据z_1:t，以及时刻1到时刻t-1的运动控制数据u_1:t-1，来同时预测机器人的轨迹x_1:t(从时刻1到时刻t一连串的粒子)和地图m。而联合概率可以转换为条件概率，故上式等价于

。

is a joint probability distribution, the purpose is to simultaneously predict the trajectory x 1 of the robot according to the observation data z _1:t from time 1 to time t, and the motion control data u _1:t-1 from time 1 to time t- _{1: t} (a sequence of particles from time 1 to time t) and map m. The joint probability can be converted into a conditional probability, so the above formula is equivalent to

.

，每个粒子携带的地图都依赖于机according to

, the map carried by each particle depends on the machine

The position and attitude of the robot, the RBpf algorithm can first estimate the trajectory of the robot for each particle, and then infer the correct probability of the map carried by the particle based on this trajectory. The more common particle filter algorithm is SIR (Samping Importance Resampling) filter, which is divided into the following four steps under the application of SLAM mapping:

1) Particle initialization

The main work of the first step of the algorithm is the initialization of the particle swarm, that is, according to the prediction of the state transition function, a large number of the above particles are generated, and these particles will be given initial weights, and the algorithm will update these weights later, using the weighted approximation of these particles. test probability.

2) Correction calculation

The main work of the second step is correction. The robot will collect the data output of a series of sensors during the traveling process, that is, the observation values. The weights of the particles are calculated based on these observation values. Assuming that there are n particles, the weight of the ith particle is denoted as w _i , which represents the probability that this particle obtains in the process of observation and correction. Because the process of building a robot map is a Markov process, the current state result is only related to the previous state, so the algorithm adopts the weight evaluation method based on the previous state. The state transition equation is shown in the following formula, where η is a constant.

表示第i个粒子t时刻的权值，z _t表示t时刻的观测数据，

表示第i个粒子预测机器人t时刻的轨迹。After the correction calculation, each particle gets its weight. in,

represents the weight of the i-th particle at time t, z _t represents the observation data at time t,

Represents the trajectory of the i-th particle predicting the robot at time t.

3) Particle resampling

The main work of the third step is resampling, that is, eliminating particles with lower value and replenishing particles with higher value. Because the movement process of the robot in the real environment is continuous, and the initial state of the particles is random, the weight of the particles with non-continuous distribution of states will gradually decrease. The algorithm eliminates this part of the particles with lower weights according to the weight ratio, and adds the newly sampled particles to the state transition equation. The newly sampled particles are calculated from the lidar data. After the robot moves, there are places that have not been mapped. I want to distribute the particles in these places. According to the lidar data, the state transition equation is used to calculate the distribution information of the particles to be sampled in the next step (that is, where the particles are generated). .

4) Map calculation

The algorithm finally counts the trajectories sampled by each particle and the observations of the sensors, from which it calculates the map estimate of the maximum probability. And update the newly calculated map to the original map.

During the movement, the robot explores the surrounding scene and continuously iterates four steps. In the process of map construction, you can judge whether the currently constructed map is complete according to actual needs. When the map has been constructed, you can save the current map and interrupt the execution of the algorithm. The saved map is the final map construction result. The map construction process is shown in Figure 2.

②Positioning, clarifying the current position of the robot in the space, providing a basis for subsequent movement and path planning. This function outputs the pose of the robot on the map built by the SLAM system based on the lidar data and the robot's odometer data. Using the active Monte Carlo particle filter localization algorithm (AMCL), the AMCL algorithm can obtain the pose of the robot by particle filtering according to the approximate position provided by the odometer on the premise that the map has been established. The specific process of the AMCL algorithm is similar to that of the mapping algorithm. It is based on the SIR filter. First, a series of random particles with weights are initialized, and then the particle swarm is used to approximate the posterior probability density of any state. The overall process is divided into the following five step:

1) Particle initialization

Similar to the mapping algorithm, the main work of the first step is the initialization of the particle swarm. Each particle in the particle swarm contains a set of pose information of the robot, such as position and orientation. For a single particle, the algorithm uses the weight to measure its matching degree with the real state of the robot pose. The initial state of each particle in the particle swarm is different, but the weight is the same.

2) State prediction

According to the motion state of the robot in the real scene, the pose information of each particle in the particle swarm is updated. For example, when the robot moves in the positive direction of the x-axis, all the particles in the particle swarm tend to move in the positive direction of the x-axis. This step only updates the pose, not the weights.

3) Weight update

The main work of this step is to update the weights of all particles in the particle swarm according to the sensor information. Based on the pose update in the previous step, the particles that match the real state of the robot pose more closely will be assigned higher weights.

4) Particle resampling

This step is similar to the principle of particle resampling in the mapping algorithm. The specific process is to discard some of the particles with the lowest weights, and resample according to the particles with higher weights to update the particle swarm. After a round of resampling, the particle swarm will converge to a certain extent, and after convergence, it can often better reflect the true state of the robot's pose.

5) Weighted average

In this step, the pose carried by each particle in the particle swarm is weighted and averaged, and the result obtained is the result estimated by the algorithm, that is, the pose of the robot in the map in the real scene.

The above steps are iterated continuously, and the particle swarm can be considered to be sufficiently accurate when the particle swarm converges to a given threshold. The particle swarm convergence process is shown in Figure 3.

③ Navigation, the ultimate purpose of the SLAM system is to give the robot the ability to navigate autonomously. In most cases, the navigation is based on the map built by the mapping module. The navigation algorithm firstly performs path planning, which is divided into two parts: global path planning and local real-time planning. Then use the PID control algorithm to track the path according to the planned path and positioning information.

Global path planning is to calculate the path with the least cost for the robot to get from point A to point B through the classical Dijkstra optimal path algorithm. During the movement of the robot, it will inevitably encounter obstacles that are not marked on the map. In order to flexibly avoid obstacles, it needs to plan the path locally in real time.

The local real-time planning is realized by the dynamic window method (Dynamic Window Approach, DWA). The algorithm flow is as follows:

1) Speed sampling

The algorithm first performs multiple sets of samples in the robot's velocity space, and the result of the sampling is a series of directed velocities, including magnitude and direction.

2) Trajectory simulation

For each of the above samples, the corresponding driving trajectory of the robot after driving at the corresponding directional speed for a period of time is predicted respectively.

3) Trajectory evaluation

The above trajectory is scored according to the evaluation function, and the optimal trajectory is screened out as the basis for the robot to drive. The evaluation function G(v,w) is the difference between the trajectory end and the current state azimuth heading(v,w), the shortest distance between the trajectory and the obstacle dist(v,w) and the trajectory velocity velocity(v,w). Normalized summation, the evaluation function G(v,w) is as follows:

。

.

Among them, σ, α, β, and γ are all setting parameters, and (v, w) are the linear velocity and angular velocity, respectively.

4) Loop, repeat the above steps

Based on the above steps, after the target point is given, the algorithm will determine the global path according to the robot's current pose and target pose, and update the local path in real time according to the obstacle information.

In order to detect unlabeled obstacles in the environment, in addition to lidar, the system also uses five ultrasonic sensors distributed in a pentagon to detect close-range obstacles to improve the ability to identify obstacles.

The mobile navigation module 102 is electrically connected to the mobile control module 101, and is used for receiving the current positioning information, and performing a fixed-point cruise explanation according to the current positioning information combined with a preset path plan.

The mobile navigation module 102 enables the robot to perform mobile navigation according to the path and point prefabricated by the system. Before moving, the voice interaction module 104 plays a "movement prompt tone", and the screen image is switched to a specific page. During the movement, the movement control is performed according to the movement control module 101 . When reaching a specific point, the movement is paused, and the voice interaction module 104 plays the pre-stored audio information. After the playback is completed, the screen will automatically switch to the next position, and then the "movement prompt tone" will be played to start moving. End when all the paths are moved. The function interface of the mobile navigation module 102 for fixed-point explanation is shown in FIG. 5 .

The information display module 103 is electrically connected to the movement control module 101 and is used for displaying preset information and sending the received movement control instruction to the movement control module 101 .

The information display module 103 is developed based on the Android Jetpack library, builds a single-page application based on the Navigation component, uses multiple Fragments and a single Activity component, abstracts common UI components, and reduces code coupling between projects. According to the received navigation request, trigger the corresponding action and switch to the corresponding Fragment. The pages include: the first-level page "function selection" page, and the second-level page includes specific function pages such as "scenic spot overview", "featured products" and "tourist photos". The information to be displayed on the page will be preset in the system database and can be adjusted according to different scenarios and needs. The database uses the Android Room framework as a persistent data storage framework, namely the Room database 107, which is convenient for database maintenance and version update. The iFLYTEK SDK is used to realize voice wake-up, speech synthesis, and speech recognition, and the https protocol is used to exchange data with the voice interaction module 104 to realize the human-computer interaction of the robot. Control the behavior of the robot through the upper-level Android interface provided by Ros, and provide functions such as "calibration positioning" and "mobile navigation". If no operation request is received for a long time (3 minutes), the built-in components in Android will cause interruption to switch pages and enter the "standby state". In the standby state, the screen displays the preset video resources; when there is a user operation, it will wake up again. Feature Selection page.

The voice interaction module 104 adopts the C/S architecture, and is connected to the information display module 103 and the mobile navigation module 102 respectively, and adopts the task-oriented retrieval dialogue combined with the deep learning chat system to realize human-computer interaction. , to avoid the high cost of manual background widely used in the market. The human-machine interaction includes receiving motion control instructions.

The voice interaction module 104 includes a front-end (voice front-end acquisition module 1041 ) and a server (voice interaction server 1042 ).

The front end is used to collect sound signals, convert the sound signals into text information, send the text information to the server and receive return information from the server, and the return information is converted into human voice based on the speech synthesis SDK .

The server is used for receiving the text information, performing intention recognition and dialogue processing on the text information based on the synergy between the retrieval class and the generation class, and sending return information.

The server is a cloud server.

The system also includes an interactive display output 105 for displaying the information of the information display module 103 and the mobile navigation module 102 by means of a display screen.

As a specific example, the front-end hardware adopts iFLYTEK ring six-mic array to collect sound signals, and uses its own front-end algorithm to perform echo cancellation, noise suppression and other processing; use voice wake-up technology to start dialogue, and based on The iFLYTEK speech recognition SDK converts the voice into text information, calls the server-side dialogue interface through the RESTFul API, obtains the returned information and converts it into human voice based on the speech synthesis SDK, and finally plays the voice through the power amplifier and speakers. The work of the voice interaction module 104 The principle is shown in Figure 4.

The server is developed based on Flask lightweight, deployed using Docker, and provides RESTFul API. Intent recognition and dialogue processing are performed based on the synergy between retrieval class and generation class, and relevant information is returned. At the same time, through the ring-shaped six-mic array, it uses its own front-end algorithm to locate the sound source, and control the robot to interact with the user; for other functions performed by the robot, the corresponding interactive prompt information will also be played.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other.

In this paper, specific examples are used to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; meanwhile, for those skilled in the art, according to the present invention There will be changes in the specific implementation and application scope. In conclusion, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. A SLAM-based mobile multi-modal interactive navigation robot system, comprising:

the mobile control module comprises a SLAM based on a laser radar and is used for controlling the movement of the robot and acquiring current positioning information;

the mobile navigation module is electrically connected with the mobile control module and used for receiving the current positioning information and performing fixed-point cruise explanation by combining preset path planning according to the current positioning information;

the information display module is electrically connected with the mobile control module and used for displaying preset information and sending the received motion control instruction to the mobile control module;

and the voice interaction module adopts a C/S (client/server) framework, is respectively in communication connection with the information display module and the mobile navigation module, adopts a deep learning chat system and is used for realizing man-machine interaction, and the man-machine interaction comprises the step of receiving a motion control instruction.

2. The SLAM-based mobile multi-modal interaction navigation robot system of claim 1, wherein the movement control module comprises:

the mapping unit is used for receiving data of the laser radar in real time and constructing a two-dimensional map of the surrounding environment according to the data of the laser radar;

the positioning unit is used for determining the pose of the robot on the two-dimensional map according to the data of the laser radar and the mileage data of the robot;

and the navigation unit is used for carrying out global path planning according to the destination and the departure place of the robot, and carrying out local real-time planning according to the detected obstacle when the robot moves according to the global path planning.

3. The SLAM-based mobile multi-modal interactive navigation robot system of claim 2, wherein the local real-time planning is achieved by a dynamic windowing method.

4. The SLAM-based mobile multi-modal interactive navigation robot system of claim 2, wherein the global path planning is achieved by dijkstra's algorithm.

5. The SLAM-based mobile multi-modal interaction navigation robot system of claim 1, wherein the voice interaction module comprises a front end and a server end;

the front end is used for collecting sound signals, converting the sound signals into text information, sending the text information to the server and receiving return information of the server, wherein the return information is converted into human voice based on a voice synthesis SDK;

and the server is used for receiving the character information, performing intention identification and dialogue processing on the character information in a mode of cooperation of the retrieval class and the generation class, and sending return information.

6. The SLAM-based mobile multi-modal interaction navigation robot system of claim 5, wherein the front end employs a scientific news flying ring six-microphone array.

7. The SLAM-based mobile multi-modal interaction navigation robot system of claim 5, wherein the server is a cloud server.

8. The SLAM-based mobile multi-modal interaction navigation robot system of claim 1, further comprising an ultrasonic sensor for detecting an obstacle signal around the robot and transmitting the obstacle signal to the movement control module.

9. The SLAM-based mobile multi-modal interaction navigation robot system of claim 2, wherein the mapping unit employs a GMaping algorithm to construct a two-dimensional map of the surrounding environment from lidar data.

10. The SLAM-based mobile multi-modal interaction navigation robot system of claim 2, wherein the positioning unit determines the pose of the robot on the two-dimensional map according to lidar data and robot range data using an active Monte Carlo particle filter positioning algorithm.

Images