CN113110513A - ROS-based household arrangement mobile robot - Google Patents

Abstract

The invention discloses a ROS-based household arrangement mobile robot, and relates to the technical field of robots; the method comprises the following steps: the method comprises the following steps: positioning and mapping algorithm; step two: a path planning method; step three: modeling and simulating an ROS robot: simulating in Rviz and Gazebo to realize drawing establishment, automatic navigation and mechanical arm grabbing of the target object; step four: a robot is formed; step five: visual recognition; step six: establishing a mechanical arm: the ROS Moveit is used as a software package for planning the motion of the mechanical arm; the invention realizes monocular vision shooting of an object to obtain a matched image pair, image preprocessing and stereo matching are carried out, accurate positioning of a target object is realized, and the identification of target garbage is realized by combining deep learning; and performing map construction, automatic navigation and garbage capture function test on the indoor environment by using the robot.

Description

ROS-based household arrangement mobile robot

Technical Field

The invention belongs to the technical field of robots, and particularly relates to a ROS-based household arrangement mobile robot.

Background

In recent years, with the continuous development of modern science and technology, service type robots such as intelligent food delivery robots, customer service robots, medical robots, accompanying robots, sweeping robots, and the like are gradually starting to enter the market. A service robot is a young member of a large family of robots and so far there is no strict definition. According to different purposes, the robot can be divided into a cleaning robot, an educational robot, a medical robot, a household robot, a service robot and an entertainment robot, and the application range is very wide. At present, the market share of education robots is about 16%, the share of customer service robots is about 7%, and the ratio of medical and cleaning service robots is about 4%.

The floor sweeping robot is used as a service type robot, people are liberated from heavy household cleaning work due to the appearance of the floor sweeping robot, the two hands of the people are released, the people can put energy in more important fields, and the life quality and the work efficiency of the people are improved.

The sweeping robot can not be cleaned and can only be manually cleaned due to the fact that the sweeping robot is low in chassis and small in suction opening, and large garbage such as beverage bottles, waste paper rolls, banana peels and plastic bags can not be cleaned. Dozens of sweeping robots are tested in the Guancun on line, and the largest garbage which can be processed by the robots is garbage with the size of dog food and small building block particles. The current sweeping robot can only clean small garbage such as dust and hair, and can not clean large garbage such as waste paper masses. In addition, objects encountered in the cleaning process can be directly cleaned, for example, small toys and the like, and once the objects are encountered by the sweeping robot, the objects can be directly processed, and the sweeping robot can try to clean or drag the objects, so that certain damage can be caused to the robot and personal objects.

To the limited shortcoming of rubbish kind size that current robot of sweeping the floor can clear up, hope through mobile robot, discern great rubbish, article such as toy building blocks and toy car to through snatching the object, remove garbage bin or toy case and classify and place, with quality and the work efficiency that improves the indoor health of cleaning.

Disclosure of Invention

The problem that the type and size of garbage which can be cleaned by the existing sweeping robot are limited is solved; the invention aims to provide a ROS-based household tidying mobile robot.

The invention discloses a ROS-based household arrangement mobile robot, which comprises the following steps:

the method comprises the following steps: positioning and mapping algorithm:

SLAM comprises feature extraction, data association, state estimation, state update and feature update; adopting 2D laser SLAM; input of 2D laser SLAM: IMU data, odometer data, 2D lidar data;

cartographer algorithm: generating a real-time grid map with the resolution of 5 cm through data measured by sensors such as a laser range finder; the Cartogrrapher adopts an SLAM theoretical framework based on a graph optimization method and is divided into a front end and a rear end: the front end is mainly responsible for Scan-to-Submap and loop detection, processed laser radar data is matched with a subgraph, and local loop detection is carried out when a subgraph is generated and no new data frame is inserted; after the subgraph is created, if a matching result which is the best with the current estimation pose is found, adding the matching result into a loopback constraint; the back end is mainly responsible for optimizing the pose estimation, and adopts branch-bound and pre-calculated grids to realize global closed-loop detection;

step two: the path planning method comprises the following steps:

searching a feasible collision-free path by using an A-algorithm, wherein the A-algorithm is a heuristic search algorithm, namely, the search in a state space evaluates each searched position to obtain the best position, and then the position is searched to the target; in the a-algorithm, the distance between the connection points is used as the weight of the edge points, and the distance to the target points is used as the heuristic function.

Step three: modeling and simulating an ROS robot:

simulating in Rviz and Gazebo to realize drawing establishment, automatic navigation and mechanical arm grabbing of the target object;

step four: the robot comprises:

the control system is the core for collecting and processing the posture data of the wheeled robot and sending and receiving control commands, and a Jetson Nano is adopted for the development mainboard;

step five: visual identification:

(5.1) building a visual platform according with the requirement by comparing visual system schemes, and learning the principle of monocular vision;

(5.2), determining image preprocessing,

(5.3) learning a visual technology of deep learning;

step six: establishing a mechanical arm: ROS Moveit | is used as a software package for mechanical arm motion planning.

Compared with the prior art, the invention has the beneficial effects that:

the method comprises the steps of firstly, monocular vision shooting of an object is achieved to obtain a matched image pair, image preprocessing and stereo matching are conducted, accurate positioning of a target object is achieved, and recognition of target garbage is achieved by combining deep learning;

and secondly, performing map construction, automatic navigation and garbage capture function test on the indoor environment by using the robot.

Drawings

For ease of illustration, the invention is described in detail by the following detailed description and the accompanying drawings.

FIG. 1 is a schematic structural diagram of the present invention.

FIG. 2 is a block diagram of a mobile robot motion recognition system according to the present invention;

FIG. 3 is a flow chart of image preprocessing according to the present invention;

FIG. 4 is a diagram of the overall framework of the ROS Moveit! of the present invention.

Detailed Description

In order that the objects, aspects and advantages of the invention will become more apparent, the invention will be described by way of example only, and in connection with the accompanying drawings. It is to be understood that such description is merely illustrative and not intended to limit the scope of the present invention. The structure, proportion, size and the like shown in the drawings are only used for matching with the content disclosed in the specification, so that the person skilled in the art can understand and read the description, and the description is not used for limiting the limit condition of the implementation of the invention, so the method has no technical essence, and any structural modification, proportion relation change or size adjustment still falls within the range covered by the technical content disclosed by the invention without affecting the effect and the achievable purpose of the invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.

It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.

The specific implementation mode adopts the following technical scheme: the method comprises the following steps:

the method comprises the following steps: positioning and mapping algorithm:

SLAM is an abbreviation for Simultaneous Localization And Mapping (Simultaneous Localization And Mapping) originally proposed by Hugh Durrant-Whyte And John J.Leonard. SLAM is mainly used for solving the problems of positioning navigation and map construction when a mobile robot runs in an unknown environment.

SLAM typically includes several components, feature extraction, data association, state estimation, state update, and feature update.

Localization: given the map, the pose of the robot is estimated.

SLAM: and simultaneously estimating the pose of the robot and an environment map.

Mapping: an environment map is estimated given the pose of the robot.

In summary, a 2D laser SLAM is used. Input of 2D laser SLAM: IMU data, odometry data, 2D lidar data.

A、HectorSLAM：

The method comprises the following steps: a laser scanner with high update frequency and small measurement noise. The odometer is not needed, and the possibility of using the ground trolley in the uneven area exists. The laser beam lattice is optimized using the map that has been obtained, and the representation of the laser spot on the map, and the probability of occupying the grid, are estimated. Wherein the scanning matching is solved by using a Gaussian Newton method. Finding a rigid body mapping from the laser point set to the existing map is converted to avoid the occurrence of local minimum rather than global optimum (similar to the multi-peak model, the local gradient is minimum, but not global optimum), and the map is in a multi-resolution form. The state estimation in navigation can be added with inertial measurement for EKF filtering.

B、Gmapping：

Currently, laser 2DSLAM uses the most widely method, and Gmiping uses RBPF method. The method of particle filtering (describing the result under a physical expression using statistical properties) must be understood.

Particle filtering methods generally require a large number of particles to obtain good results, but this necessarily introduces computational complexity; particles are a process of gradually updating weights and converging according to observation of the process, and this resampling process inevitably substitutes for the problem of particle dissipation, large-weight particles are significant, and small-weight particles disappear (it is possible that correct particle simulation may appear in the middle stage that the weights are small and disappear).

The self-adaptive resampling technology reduces the particle dissipation problem, only depends on the motion (mileometer) of the robot when calculating the particle distribution, and simultaneously takes the current observation into consideration, thereby reducing the uncertainty of the robot position in the particle filtering step. (idea of FAST-SLAM 2.0, particle number can be reduced appropriately)

C、KartoSLAM：

The core idea of graph optimization is mainly sparsification and least squares of the matrix. KartosLAM is a graph optimization-based method, and takes high optimization and non-iterative cholesky matrix to perform sparse system decoupling as a solution. The graph optimization method uses the average value of a graph to represent a map, each node represents a position point of a robot track and a sensor measurement data set, the connection pointed by an arrow represents the movement of the position points of the continuous robot, and each new node is added, so that the map is calculated and updated according to the constraint of the node arrow in the space.

ROS version of KartoSLAM, where sparse point adjustment is employed in connection with scan matching and closed loop detection. The more Landmark, the greater the memory requirement, however, the better the mapping in the large environment compared with other methods. In some cases KartoSLAM is more efficient because it only contains a Map of points (Robot position) and maps are found after the position is found.

D、CoreSLAM：

For simplicity and ease of understanding a SLAM algorithm that minimizes performance loss. The algorithm is simplified into two processes of distance calculation and map updating, wherein in the first step, each time scanning input is carried out, the distance is calculated based on a simple particle filter algorithm, a matcher for particle filtering is used for matching laser with a map, and each filter particle represents a possible position and a corresponding probability weight of the robot and depends on previous iteration calculation. The best hypothetical distribution is chosen, i.e. the low-weight particles disappear and new particles are generated. In the update step, the scanned line is added to the map and when an obstacle appears, a set of adjustment points is drawn around the obstacle point, instead of just one isolated point.

E、LagoSLAM：

The basic approach to map-optimized SLAM is to use a minimized nonlinear non-convex cost function. And each iteration, updating the graph configuration by solving the initial problem of local convex approximation, and iterating the process for a certain number of times until the local minimum cost function is reached. Lagosram is a linear approximation graph optimization that does not require initial assumptions.

F. Cartographer algorithm:

the algorithm can generate a real-time grid map with the resolution of 5 cm through data measured by a sensor such as a laser range finder. The Cartogrrapher adopts an SLAM theoretical framework based on a graph optimization method and is divided into a front end and a rear end: the front end is mainly responsible for Scan-to-Submap and loop detection, processed laser radar data is matched with the subgraph, and local loop detection is carried out when a subgraph is generated and no new data frame is inserted. After the sub-graph is created, if a matching result which is best with the current estimated pose is found, the matching result is added into the loopback constraint. The back end is mainly responsible for optimizing pose estimation, and adopts branch-bound and pre-calculated grids to realize global closed loop detection.

In summary, the Cartogrer algorithm is selected.

Step two: the path planning method comprises the following steps:

the probabilistic roadmapping method (PRM algorithm for short) is one of the earliest methods based on random sampling. The method randomly and uniformly places nodes on a robot moving space, then connects all or adjacent nodes to form a network graph of undirected paths, and then obtains a feasible path by using a search algorithm. The method has the advantages that environmental modeling is not needed, and the method is a good solution to the problem of path planning of high-dimensional space and multiple constraints.

The PRM algorithm has the basic idea that a moving area of a mobile robot is constructed into a network diagram of undirected paths by converting a continuous space into a discrete space (N sampling points are scattered randomly in a space servo). And finally, searching a feasible optimal path by using a graph search algorithm (such as an breadth first algorithm, an A-algorithm, a Dijkstra algorithm and the like). The graph search algorithm uses the a-x algorithm, starting from the starting unit, each neighboring unit is ranked by the lowest total cost, defined by the trip plus the heuristic cost, the trip cost being the distance from the starting unit to the next unit, the heuristic cost being the distance from any unit to the target unit. The least costly node is then expanded and explored until it reaches the target node. The probability of finding the feasible path by the algorithm is related to the number of the sampling points, along with the increase of the sampling points, the probability of finding the feasible path is increased, and the PRM algorithm is complete. But for most problems, many samples are not needed, and paths can be found by using fewer samples.

a. A learning stage: randomly sampling N points (self-defined number) in the map, then connecting each sampling point, removing a connecting line in contact with an obstacle, and constructing a undirected path network map.

b. And (3) an inquiry stage: after the undirected path network graph is constructed in the learning stage, in the query stage, a starting point and an end point of a path are required to be set, and then a feasible path is searched by using a graph search algorithm.

And searching a feasible collision-free path by using an A-algorithm, wherein the A-algorithm is a heuristic search algorithm, namely, searching in a state space evaluates each searched position to obtain the best position, and searching from the position to the target. This may omit a large number of featureless search paths, which may be useful for efficiency. In the a-algorithm, the distance between the connection points is used as the weight of the edge points, and the distance to the target points is used as the heuristic function.

Step three: modeling and simulating an ROS robot:

simulation is carried out in Rviz and Gazebo to realize drawing establishment, automatic navigation and mechanical arm grabbing of the target object

(3.1) urdf robot modeling. The description form of the whole organization is in the xml format, and a plurality of configuration files in the ROS are in the format, so that the label attributes can be used for conveniently describing each piece of related information, the organization is convenient, and the appearance is more intuitive. Problems with urdf modeling: the model is long and the repeated content is excessive; the modification of the parameters is too troublesome, and the secondary development is not convenient; there is no function of parameter calculation.

And (3.2) optimizing a robot urdf model. The evolutionary version of the urdf model, namely the xacro model file, simplifies the model code and provides a programmable interface.

And (3.3) building a Gazebo physical simulation environment. And configuring a robot model and creating a simulation environment.

And (3.4) compiling and importing programs such as drawing building and navigation into a simulation environment to complete simulation.

Step four: the robot comprises the following components and characteristics:

the control system is the core for collecting and processing the attitude data of the wheeled robot and sending and receiving control commands, and directly determines the overall performance and the running state of the robot.

The development motherboard is intended to use Jetson Nano. Jetson Nano is chosen because Jetson Nano is more suitable for a robot to be completed after comparing advantages and disadvantages of raspberry pie and Jetson Nano. The raspberry pi is a small computer and Internet of things development mainboard. The internet of things gateway device is not only a low-power-consumption internet of things device, but also a good prototype design tool, and even can be used for building internet of things gateway devices. The Jetson Nano motherboard, published by engida, as a development kit, provides all the inputs and connections required for the internet of things solution when designing a prototype, and provides all the computing power (and also integrates a GPU) required for the internet of things solution when designing a prototype. Both have an ARM processor and a series of peripheral connections. The greatest difference between the two is that Jetson Nano of england contains a higher performance and more powerful GPU (graphics processor), while raspberry pi has a low power consumption VideoCore multimedia processor. Jetson Nano by engida is expensive, but has a powerful Maxwell GPU.

The overall frame diagram of the overall hardware system of the robot to be designed is shown in FIG. 1.

Based on the ROS distributed framework, the laser radar is used for collecting environmental information of a cleaning area, the SLAM function is achieved, and path planning and traversal of the cleaning area are carried out through an optimal path algorithm. In the traversing process of the robot, the algorithm is used for carrying out target detection and target classification on the image acquired by the camera, the coordinate and the angle information of the target are acquired as the input information of the sorting control module, and the motion control module executes a garbage grabbing task. The main characteristics are:

(1) cleaning large garbage (most sweeping robots in the market are cleaning dust or small-volume garbage);

(2) automatically traversing navigation and automatically identifying and grabbing garbage;

(3) garbage and non-garbage can be accurately identified through deep learning training;

(4) the method is suitable for smoothly detecting and classifying the targets of the embedded equipment by improving the algorithm.

A block diagram of a mobile robot recognition motion system is shown in fig. 2.

Step five: visual identification:

(5.1) building a visual platform according with the requirement by comparing visual system schemes, and learning the principle of monocular vision;

(5.2) determining image preprocessing, wherein the flow is shown in figure 3.

And (5.3) learning a visual technology of deep learning.

Popular algorithms can be divided into two types, one type is R-CNN algorithm (R-CNN, Fast R-CNN, etc.) based on Region Proposal, the algorithms are two-stage algorithms, target candidate frames, namely target positions, need to be generated by the algorithms, and then classification and regression are carried out on the candidate frames. And the other is a one-stage algorithm like Yolo, SSD, which directly predicts the classes and positions of different targets using only one convolutional neural network CNN. The first category of methods is more accurate but slower, but the second category of algorithms is faster but less accurate.

The accuracy is higher by adopting the Faster R-CNN algorithm. Firstly, acquiring color images of a plurality of target objects in different poses by a Kinect2 camera, and labeling the color image of each target object in a data set by using a LabelImg tool; and then, taking the marking information and the color image information as samples used for training of the Faster R-CNN model, and finally obtaining an accurate target detection model based on the Faster R-CNN. The target detection system based on the Faster R-CNN model mainly comprises a deep full convolution neural network for generating candidate regions and a Fast R-CNN detection operator for using candidate frames. And the Fast R-CNN detection module extracts features according to the region of interest proposed by the region suggestion network RPN module so as to complete the search task of the target object.

The Faster R-CNN replaces the selective search, the Region to be detected is directly generated through a Region pro-social Network (RPN), and the time is reduced from 2s to 10ms when the RoI Region is generated.

a. Firstly, extracting feature maps for the whole graph by using a shared convolution layer;

b. sending the obtained feature maps into an RPN (resilient packet network), generating a frame to be detected (specifying the position of the RoI) by the RPN, and correcting the bounding frame of the RoI for the first time;

selecting features corresponding to each RoI on a feature map by a RoI Pooling Layer according to the output of the RPN, and setting the dimension as a fixed value;

d. the box is classified using a full connectivity Layer (FC Layer) and a second correction of the target bounding box is made.

Step six: mechanical arm:

the ROS Moveit is a software package specially used for mechanical arm motion planning, integrates the latest results in the field of motion control, and provides a software platform for developing advanced robot applications. ROS Moveit! overall framework is shown in FIG. 4.

The working principle of the specific embodiment is as follows: firstly, a garbage sorting robot is put in a working environment to be cleaned, the robot is controlled through a handle or a keyboard to establish an indoor 2D map, then the robot automatically navigates indoors, images are collected through a camera, algorithm processing is carried out on the images to judge whether garbage exists, and if not, navigation is continued and whether garbage exists is scanned. If so, a signal is sent to allow the mobile robot to navigate around the trash. The robot reaches the periphery of the garbage, is used for adjusting the relative position of the garbage sorting robot according to the recognized position of the garbage, and then controls the mechanical arm to grab the garbage, and continues navigation scanning after the garbage is grabbed.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.

Claims (1)

1. The utility model provides a house arrangement mobile robot based on ROS which characterized in that: the method comprises the following steps:

the method comprises the following steps: positioning and mapping algorithm:

SLAM comprises feature extraction, data association, state estimation, state update and feature update; adopting 2D laser SLAM; input of 2D laser SLAM: IMU data, odometer data, 2D lidar data;

cartographer algorithm: generating a real-time grid map with the resolution of 5 cm through data measured by sensors such as a laser range finder; the Cartogrrapher adopts an SLAM theoretical framework based on a graph optimization method and is divided into a front end and a rear end: the front end is mainly responsible for Scan-to-Submap and loop detection, processed laser radar data is matched with a subgraph, and local loop detection is carried out when a subgraph is generated and no new data frame is inserted; after the subgraph is created, if a matching result which is the best with the current estimation pose is found, adding the matching result into a loopback constraint; the back end is mainly responsible for optimizing the pose estimation, and adopts branch-bound and pre-calculated grids to realize global closed-loop detection;

step two: the path planning method comprises the following steps:

searching a feasible collision-free path by using an A-algorithm, wherein the A-algorithm is a heuristic search algorithm, namely, the search in a state space evaluates each searched position to obtain the best position, and then the position is searched to the target; in the A-algorithm, the distance between the connecting points is used as the weight of the edge points, and the distance between the target points is used as a heuristic function;

step three: modeling and simulating an ROS robot:

simulating in Rviz and Gazebo to realize drawing establishment, automatic navigation and mechanical arm grabbing of the target object;

step four: the robot comprises:

the control system is the core for collecting and processing the posture data of the wheeled robot and sending and receiving control commands, and a Jetson Nano is adopted for the development mainboard;

step five: visual identification:

(5.1) building a visual platform according with the requirement by comparing visual system schemes, and learning the principle of monocular vision;

(5.2), determining image preprocessing,

(5.3) learning a visual technology of deep learning;

step six: establishing a mechanical arm: ROS Moveit | is used as a software package for mechanical arm motion planning.

Images