CN113515131B - Mobile robot obstacle avoidance method and system based on condition variation automatic encoder - Google Patents
Mobile robot obstacle avoidance method and system based on condition variation automatic encoder Download PDFInfo
- Publication number
- CN113515131B CN113515131B CN202110997644.2A CN202110997644A CN113515131B CN 113515131 B CN113515131 B CN 113515131B CN 202110997644 A CN202110997644 A CN 202110997644A CN 113515131 B CN113515131 B CN 113515131B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- obstacle avoidance
- robot
- point
- temporary obstacle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012216 screening Methods 0.000 claims abstract description 16
- 239000011159 matrix material Substances 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 9
- 238000001514 detection method Methods 0.000 claims description 9
- 230000036651 mood Effects 0.000 claims description 6
- 230000004888 barrier function Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 2
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000008451 emotion Effects 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 241000209140 Triticum Species 0.000 description 1
- 235000021307 Triticum Nutrition 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0217—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with energy consumption, time reduction or distance reduction criteria
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
Abstract
The invention relates to a mobile robot obstacle avoidance method and system based on a condition variation automatic encoder. Initializing a pedestrian obstacle avoidance area of a robot; the robot receives an original target point according to the grid matrix and navigates according to the navigation information; detecting pedestrians and acquiring pedestrian position information and pedestrian attribute information through the robot; judging whether the pedestrian is in the obstacle avoidance range or not according to the position of the pedestrian; a training condition variation automatic encoder; screening the optimal temporary obstacle avoidance points with the lowest cost according to the temporary obstacle avoidance points; the robot receives the optimal temporary obstacle avoidance point and navigates to the optimal temporary obstacle avoidance point; and judging whether the robot reaches the optimal temporary obstacle avoidance point. The invention combines deep learning and navigation dynamic obstacle avoidance to generate temporary obstacle avoidance points, guides the robot to plan a path meeting the comfort level of pedestrians, and reduces the problems of robot loss, collision and the like caused by environment uncertainty.
Description
Technical Field
The invention relates to the technical field of intelligent robots, in particular to a mobile robot obstacle avoidance method and system based on a condition variation automatic encoder.
Background
The breakthrough of artificial intelligence technology and the rapid upgrade of computer software and hardware technology promote the research and progress of mobile service robots. At present, the industrial scale of the intelligent service robot is continuously enlarged, the application scenes are richer, and the related mature products are applied to various environments such as families, hotels, hospitals, airports and the like. The dynamic obstacle avoidance technology of the mobile robot refers to a process that the robot automatically adopts corresponding motion planning to prevent the robot from colliding according to map information and pedestrian information moving in the environment under the condition of no human intervention, and the motion planning can be that the robot selects to detour or waits for pedestrians to pass in situ. Compared with an industrial robot, the service type mobile robot has the advantages that the complexity of a man-machine co-fusion environment is higher, the uncertainty is larger, and the requirement on the real-time performance of an obstacle avoidance strategy is higher. The current dynamic obstacle avoidance module in the navigation framework generally regards pedestrians as dynamic obstacles, such as a dynamic window method, or concentrates on the next moment movement of an object according to a specific interaction rule, such as a relative velocity obstacle algorithm and an optimal relative collision avoidance algorithm. The methods enable the robot to avoid pedestrians through passive reaction, and the generated path is unsafe and unnatural and cannot meet the requirements of the pedestrians. It is therefore desirable that the robot detect pedestrians in advance during navigation and use a deep learning method to generate a natural continuous and comfortable path for the pedestrians.
Taking a Dynamic Window Algorithm (DWA) as an example, a general step of a Dynamic obstacle avoidance algorithm based on robot speed sampling is briefly described:
the method comprises the following steps: initializing, configuring the maximum and minimum speed and acceleration of the robot movement, evaluating function weight and the like;
step two: it is determined whether the destination is reached. Firstly, judging whether the current position of the robot is a target point, if not, executing a third step, and if so, terminating;
step three: velocity sampling, calculating the currently sampled velocity range (dynamic window) based on the motion model of the mobile robot, as shown in fig. 1. The motion models of mobile robots are roughly divided into two types:
model one: the chassis of the robot is not moved in all directions, and only can move forwards (x-axis speed v) and rotate (z-axis angular speed w), for example, the chassis of the robot has short movement distance in adjacent moments (millisecond grade), and the movement track between two adjacent points can be regarded as a straight line. The position of the next state of the robot can be expressed as
x＝x+vΔt cos(θ t )
y＝y+vΔt sin(θ t )
θ t ＝θ t +ωΔt
Model two: with omni-directional motion, having x-axis velocity Vx, y-axis velocity Vy and angular velocity w, as in a wheat wheel vehicle, the next state position in this model can be represented as
x＝x+v x Δt cos(θ t )-v y Δt sin(θ t )
y＝y+v x Δt sin(θ t )+v y Δt cos(θ t )
θ t ＝θ t +wΔt
After the chassis motion model is available, information of the current chassis speed (linear speed v and angular speed w) can be represented, and a speed range is set on the basis of the current speed;
step four: traversing, namely traversing the linear velocity and the angular velocity in all ranges, and simulating a path for a period of time according to the model, as shown in fig. 2;
step five: screening paths, and scoring the paths in (4) according to an evaluation function, wherein the evaluation function G (v, w) is composed of three parts, an azimuth evaluation function heading (v, w), namely an angle difference between a trolley and a target, a distance dist (v, w) between a chassis and a nearest obstacle, and a track speed magnitude velocity (v, w).
G(v,ω)＝σ(α·heading(v,ω)+β·dist(v,ω)+γ·velocity(v,ω))
The physical meaning of the evaluation function is that the robot avoids the obstacle, faces the target and runs at a higher speed;
step six: the robot moves, selects the optimal solution (linear velocity and angular velocity), sends the optimal solution to the motion chassis, and drives the robot to move continuously; and repeating the second step to the sixth step until the robot reaches the end point.
The dynamic obstacle avoidance algorithm based on robot speed sampling guarantees feasibility in an environment with a small number of pedestrians and a low moving speed of the pedestrians, however, in a crowded environment, the current method can cause the robot to take unnatural, short-sight and unsafe movement. For example, in a narrow corridor, when the robot encounters a pedestrian coming on the head, it may not be able to pass because the pedestrian occupies a large half of the passage, but due to the attractiveness of the navigation destination, the robot cannot stop the path planning, thus outputting a very discontinuous trajectory, causing the robot to stall, rotate or sway, thereby affecting the normal passage of the pedestrian.
As shown in fig. 3, the human-machine co-fusion working environment is guided by a navigation target point, and a current dynamic obstacle avoidance strategy forms a path planning result shown in a path 1 according to speed sampling, however, the path 1 needs to pass through a walking crowd, the moving speed of a pedestrian in a real navigation environment may be very fast, and a collision with the movement of the pedestrian is generated with a high probability when the path is taken, and even a failure of a navigation task is caused. Although the distance of the path 2 is long, the robot is far away from pedestrians in the environment, the motion of the robot is safer and is more easily accepted by the pedestrians, and the current dynamic obstacle avoidance algorithm cannot distinguish the pedestrians from common obstacles, so that a path meeting the requirements of the pedestrians cannot be generated.
As described above, the current dynamic obstacle avoidance algorithm only regards pedestrians as dynamic obstacles, the generated path is unnatural, and the safety of navigation and the psychological feeling of pedestrians are ignored.
Disclosure of Invention
Therefore, the technical problem to be solved by the invention is to overcome the problems that in the prior art, the dynamic obstacle avoidance algorithm only takes the pedestrian as a dynamic obstacle, the generated path is unnatural, and the navigation safety and the psychological feeling of the pedestrian are ignored.
In order to solve the technical problem, the invention provides a mobile robot obstacle avoidance method based on a condition variation automatic encoder, which comprises the following steps:
step S1: initializing a pedestrian obstacle avoidance area of the robot, constructing a grid matrix by taking the robot as a center and acquiring the position of an obstacle point;
step S2: the robot navigates and moves from the starting point to the target point according to the grid matrix;
and step S3: the robot detects the pedestrian and obtains the pedestrian position information and the pedestrian attribute information;
and step S4: judging whether the pedestrian is in the obstacle avoidance range or not according to the position of the pedestrian, if not, returning to the step S2, and if so, executing the step S5;
step S5: according to the pedestrian position information, expressing the position of a pedestrian by using coordinates and storing the position of the pedestrian into a pedestrian position array, expressing the attribute of the pedestrian by using a variable quantity and storing the attribute into a pedestrian attribute array according to the attribute of the pedestrian, training a condition variation automatic encoder by taking the position of an obstacle point, the position of a starting point and the position of a target point of a robot, the pedestrian position array and the pedestrian attribute array as input, and generating a temporary obstacle avoidance point in a coordinate form;
step S6: screening the optimal temporary obstacle avoidance points with the lowest cost according to the temporary obstacle avoidance points;
step S7: the robot carries out navigation again according to the optimal temporary obstacle avoidance points;
step S8: and judging whether the robot reaches the optimal temporary obstacle avoidance point, if not, continuing to move until the robot reaches the optimal temporary obstacle avoidance point, if so, continuing to judge whether the robot reaches the target point, otherwise, returning to the step S3, and if so, considering that the robot reaches the target point.
In one embodiment of the present invention, in step S3, the method for performing pedestrian detection and acquiring pedestrian position information and pedestrian attribute information includes:
step S31: capturing surrounding images in real time through a depth camera carried by a robot;
step S32: detecting pedestrians in the surrounding image by applying a pedestrian detection algorithm to obtain positions of the pedestrians under a pixel coordinate system, and identifying the pedestrians in the surrounding image by applying a pedestrian attribute identification algorithm to obtain pedestrian attribute information;
step S33: transforming the positions of the pedestrians in the pixel coordinate system through coordinates, and outputting the positions of the pedestrians in the global coordinate system;
step S34: and calculating the moving speed and the moving direction of the pedestrian according to the position of the pedestrian in the global coordinate system and the moving distance and the passing time of the pedestrian, and predicting the position of the pedestrian at the next moment according to the constant-speed model.
In an embodiment of the present invention, in step S4, the method for determining whether a pedestrian is in an obstacle avoidance range according to a position of the pedestrian includes:
step S41: according to the position of the pedestrian in the global coordinate system, calculating the distance d from the pedestrian to the position of the robot in the abscissa and the ordinate x And d y ；
Step S42: judgment of d x And d y And whether the side length of the pedestrian obstacle avoidance area is less than half of the side length of the pedestrian obstacle avoidance area, wherein the pedestrian obstacle avoidance area is a square area with a robot as a center in the grid map, if so, the pedestrian is in the obstacle avoidance area, and if not, the pedestrian is not in the obstacle avoidance area.
In one embodiment of the present invention, in step S5, the method for representing the position of the pedestrian by coordinates and storing the position of the pedestrian into the pedestrian position array according to the pedestrian position information comprises: the current position coordinates of the pedestrian and the position coordinates of the next two moments are respectively expressed through global to local coordinate conversion and stored in an array, the former two numbers respectively express the x coordinate and the y coordinate of the pedestrian at the moment, and the latter four numbers express the coordinates of the pedestrian at the next two moments.
In an embodiment of the present invention, in step S5, the method for representing the attribute of the pedestrian by using a variable quantity and storing the attribute into the pedestrian attribute array according to the attribute of the pedestrian includes: representing the attribute of the pedestrian by four 0/1 variables and storing the attribute into an array, wherein in the four 0/1 variables, the first numerical value represents the sex of the pedestrian, 0 represents the male, and 1 represents the female; the second number represents body width, 0 represents narrow, and 1 represents wide; the third value represents age, 0 represents middle age, and 1 represents the elderly or children; the fourth value represents mood, 0 represents happy or normal, and 1 represents negative mood.
In an embodiment of the present invention, in step S6, the method for screening the optimal temporary obstacle avoidance points with the lowest cost according to the temporary obstacle avoidance points includes: designing a cost function according to the distances from the temporary obstacle avoidance points to the pedestrians, the nearest obstacle and the original target point, calculating a cost value of each temporary obstacle avoidance point according to the cost function, and selecting the temporary obstacle avoidance point with the lowest cost as the optimal temporary obstacle avoidance point.
In one embodiment of the present invention, the cost function is:
wherein x temp Indicating temporary obstacle avoidance points, x pi Coordinates, x, representing the ith pedestrian obs Coordinates, x, representing nearest obstacles goal Coordinates representing an original target point; omega 1 、ω 2 、ω 3 Are weights.
In an embodiment of the present invention, in step S7, the method for the robot to receive the optimal temporary obstacle avoidance point for navigation includes: the optimal temporary obstacle avoidance point is issued to the robot in the form of topics in the ROS, the original navigation is interrupted by the robot, and the robot preferentially navigates the optimal temporary obstacle avoidance point.
In an embodiment of the present invention, in step S8, the method for determining whether the robot reaches the optimal temporary obstacle avoidance point includes: and acquiring the real-time position coordinate of the robot through the ROS, calculating the distance between the real-time position coordinate of the robot and the coordinate of the optimal temporary obstacle avoidance point, if the distance is less than or equal to a preset threshold value, indicating that the robot arrives, and if the distance is greater than the preset threshold value, indicating that the robot does not arrive.
The invention also provides a mobile robot obstacle avoidance system based on the condition variation automatic encoder, which comprises:
the initialization module is used for initializing a pedestrian obstacle avoidance area of the robot;
the navigation module is used for receiving the original target point and navigating by the robot;
the pedestrian detection module is used for detecting pedestrians and outputting the positions and the attributes of the pedestrians;
the obstacle avoidance range judging module is used for judging whether the pedestrian is in the obstacle avoidance range according to the position of the pedestrian;
the condition variation automatic encoder training module is used for generating temporary obstacle avoidance points in a coordinate form;
the temporary obstacle avoidance point screening module is used for screening the optimal temporary obstacle avoidance point with the lowest cost according to the temporary obstacle avoidance points;
the module for receiving the optimal temporary obstacle avoidance point is used for the robot to receive the optimal temporary obstacle avoidance point for navigation;
and the optimal temporary obstacle avoidance point arrival judging module is used for judging whether the robot arrives at the optimal temporary obstacle avoidance point.
Compared with the prior art, the technical scheme of the invention has the following advantages:
the invention takes pedestrian information into full consideration, encodes pedestrian position information (position, speed and direction), pedestrian attribute information (sex, body width, age and emotion), static environment information (barrier position) and robot navigation information (starting point and end point) into a deep learning model (condition variation automatic encoder) for training, ensures that temporary obstacle avoidance points can be generated according to a new navigation environment, guides a path planner to generate a path meeting the comfort level of pedestrians, and adopts different obstacle avoidance strategies when the robot faces pedestrians with different sexes, body widths, ages and emotions;
the invention combines deep learning with navigation dynamic obstacle avoidance, trains a model with pedestrian attention, comprehensively considers the position and attribute of the pedestrian after the robot identifies the pedestrian which will cause navigation conflict, generates temporary obstacle avoidance points, guides the robot to plan a path meeting the comfort level of the pedestrian, reduces the problems of robot loss, collision and the like caused by environment uncertainty, and enhances the social consciousness in the navigation process of the mobile robot.
Drawings
In order that the present disclosure may be more readily and clearly understood, reference is now made to the following detailed description of the embodiments of the present disclosure taken in conjunction with the accompanying drawings, in which
Fig. 1 is a diagram of a chassis motion model.
Fig. 2 is a simulation path diagram.
Fig. 3 is a graph of the impact of environmental uncertainty on path planning.
FIG. 4 is a flow chart of the method of the present invention.
Fig. 5 is a schematic diagram of model generation of temporary obstacle avoidance points.
Fig. 6 is a schematic diagram of screening temporary obstacle avoidance points.
Fig. 7 is a schematic diagram of a scene-a path guided by temporary obstacle avoidance points and an original path.
FIG. 8 is a diagram illustrating the overall process of completing one task according to the present invention.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
Example one
Referring to fig. 1, in one embodiment of the present invention, a mobile robot obstacle avoidance method based on a conditional variation automatic encoder is provided, including the following steps:
step S1: initializing a pedestrian obstacle avoidance area of the robot, constructing a grid matrix by taking the robot as a center and acquiring obstacle positions;
step S2: the robot navigates and moves from the starting point to the target point according to the grid matrix;
and step S3: the robot detects the pedestrians and obtains the position information and the attribute information of the pedestrians;
and step S4: judging whether the pedestrian is in the obstacle avoidance range or not according to the position of the pedestrian, if not, returning to the step S2, and if so, executing the step S5;
step S5: inputting external information (position, speed and direction) of the pedestrian in the step S3, internal information (sex, body width, age and emotion) of the pedestrian, static environment information (barrier position) in a two-dimensional grid map form and robot navigation task information (starting point and end point) in the step S2 into the deep learning model, and outputting a group of temporary obstacle avoidance points; as shown in fig. 2, the number of temporary obstacle avoidance points output in this embodiment is 5;
step S6: screening the optimal temporary obstacle avoidance points with the lowest cost according to the temporary obstacle avoidance points; as shown in fig. 3, a cost value of each temporary obstacle avoidance point is calculated through a designed cost function, and the lowest cost is selected as a final temporary obstacle avoidance point.
Step S7: the robot carries out navigation again according to the optimal temporary obstacle avoidance points; path 1 in fig. 4 is a path that we guide the robot through the temporary obstacle avoidance point, and path 2 is a path generated by the original method;
step S8: and judging whether the robot reaches the optimal temporary obstacle avoidance point, if not, continuing to move until the optimal temporary obstacle avoidance point, and if so, returning to the step S3 until the robot reaches a target point.
Fig. 5 is the whole process of completing one task by the whole method, wherein path 1 is the path that we guide the robot through the temporary obstacle avoidance point, and path 2 is the path generated by the original method;
specifically, the obstacle avoidance area in step S1 is a square area centered on the robot in the grid map, and the size of the obstacle avoidance area may be adjusted according to the resolution of the map and the sensitivity of the robot to pedestrians that a user desires. The obstacle avoidance area is mainly used for preventing the pedestrian from operating in a mode when the pedestrian is far away from the robot, so that the obstacle avoidance is not real-time and the effect is not good.
Specifically, the algorithm used in the navigation in step S2 may be most path planning algorithms, such as an a-star algorithm and a fast search random tree algorithm (RRT), and the path planning algorithm adopted in this embodiment is a fast search random tree algorithm.
Specifically, the capturing of the pedestrian information in step S3 includes the steps of:
step S31: capturing surrounding images in real time through a depth camera carried by the robot;
step S32: detecting pedestrians in the surrounding image by applying a pedestrian detection algorithm, and outputting the positions of the pedestrians in a pixel coordinate system;
step S33: according to the camera calibration result, the position of the output pedestrian under the pixel coordinate system is subjected to coordinate transformation, and the position of the pedestrian under the global coordinate system is output;
step S34: calculating the position change of the pedestrian in the global coordinate system at the time t in the step S33, calculating the moving speed and the moving direction of the pedestrian according to the moving distance and the passing time of the pedestrian, and predicting the position of the pedestrian at the next time according to the constant speed model, wherein the size of t is set to 1S in the embodiment;
step S35: capturing pedestrian images through a camera, applying a pedestrian attribute recognition algorithm, and outputting attributes of pedestrians.
Specifically, the step S4 of determining the position of the pedestrian includes the steps of:
step S41: according to the position of the pedestrian in the global coordinate system output in the step S3, the distance d from the pedestrian to the position of the robot under the abscissa and the ordinate is calculated x And d y ；
Step S42: judgment of d x And d y Whether the side lengths of the obstacle avoidance areas are smaller than half of the side lengths L of the obstacle avoidance areas, if yes, the pedestrians are in the obstacle avoidance areas, otherwise, the pedestrians are not in the obstacle avoidance areas, and the size of L is set to be 11 in the embodiment;
specifically, the condition variation automatic encoder in the step S5 is a new generation model, and mainly includes two parts, namely encoding and decoding, wherein a navigation task and information of robots and pedestrians are used as a condition y, a temporary obstacle avoidance point is used as a label x, and the encoding part compresses the label x with the condition y into a latent variable z in a low-dimensional space; the decoding section reconstructs the latent variable z into a new label z' according to the condition y. The function of the conditional variation automatic encoder is to form a mapping relation from the condition y to the label x.
Specifically, the network structure of the coding part of the conditional variation auto-encoder in step S5 is that the coding part is composed of two full-connection layer networks and one dropout layer network, in this embodiment, the number of neurons in the first full-connection layer is 512, and 50% of nodes are reserved after the dropout layer is connected to the first full-connection layer. The second fully connected layer also has 512 neurons. The input of the coder is a sample x and a condition variable y, the output is a latent variable z, and the parameter of the coding network is
The process of encoding can be expressed as
The decoding network has the same network structure as the encoding network, two fully connected layers and one dropout layer. The dropout layer is set to prevent overfitting of the model, the input of the decoder is latent variable z and condition y in the encoder, the output is new reconstructed sample z', the parameter of the decoding network is theta, and the decoding process can be expressed as p θ (x | z, y); the training goal of the entire network is to maximize the Lower Bound Eviance Lower Bound (ELBO) of the likelihood function
Wherein the first term KL represents the relative entropy Kullback-Leibler divergence and the second term is the reconstruction loss, θ and
is a parameter vector.
Specifically, the running of the trained conditional variational automatic encoder in the step S5 includes the following steps:
step S51: initializing a pedestrian obstacle avoidance area with the robot as the center, and constructing a grid matrix with the robot as the center, wherein 0 represents no obstacle, and 1 represents an obstacle; the size of the grid matrix in this embodiment is 11 × 11;
step S52: respectively representing the current position coordinates of the pedestrian and the position coordinates of the next two moments t1 and t2 through global-to-local coordinate conversion (converting the position of the pedestrian in the whole map to the position in the pedestrian obstacle avoidance area), and storing the coordinates into a pedestrian position array; the first two numbers of the array in the embodiment respectively represent the x coordinate and the y coordinate of the pedestrian at that time, the last four numbers represent the coordinates of the pedestrian at the next two times, wherein t1 is 1s, and t2 is 2s;
step S53: representing the attribute of the pedestrian by four 0/1 variables and storing the attribute into a pedestrian attribute array, wherein the first numerical value represents the gender of the pedestrian, 0 represents a male, and 1 represents a female; the second number represents body width, 0 represents narrow, and 1 represents wide; the third value represents the age, 0 represents the middle age, and 1 represents the elderly or children; the fourth value represents mood, 0 represents happy or normal, 1 represents negative mood;
according to the psychological research of pedestrians, women have higher requirements on the space of the pedestrians than men, the pedestrians with wider body widths have higher requirements on the space than narrow pedestrians, the old and children have higher requirements on the space than the users of middle ages, and the pedestrians with negative emotions have higher requirements on the space than the pedestrians with normal happiness; therefore, when facing pedestrians with high space requirements, the model generates temporary obstacle avoidance points which are farther away from the pedestrians;
step S54: inputting the grid matrix, the pedestrian position array and the pedestrian attribute array representing the environment information and the starting point and the end point of the navigation task into a trained model; generating temporary obstacle avoidance points x in coordinate form temp 。
Specifically, in the cost function in step S6, a weight ω is given to each of the distances from the temporary obstacle avoidance point to the pedestrian, the nearest obstacle, and the original target point 1 、ω 2 、ω 3 Calculated, the cost function is:
wherein x pi Coordinates, x, representing the ith pedestrian obs Coordinates representing nearest obstacle, x goal Coordinates representing an original target point; comparing the costs of all temporary obstacle avoidance points, and taking the point with the minimum cost as the optimal temporary obstacle avoidance point x k ；ω 1 、ω 2 、ω 3 As a weight, ω in this embodiment 1 Is set to 4, omega 2 Is set to-2, omega 3 Set to 2.5.
Specifically, the optimal temporary obstacle avoidance point in the step S7 is issued to the robot in the form of a topic in the ROS, the original navigation is interrupted by the robot, and the robot preferentially navigates to the optimal temporary obstacle avoidance point;
specifically, in the step S8, it is determined whether the robot reaches the temporary obstacle avoidance point, the real-time position coordinates of the robot are subscribed through the ROS and compared with the optimal temporary obstacle avoidance point coordinates, and if the distance between the two coordinates is less than or equal to a threshold, the robot reaches the optimal temporary obstacle avoidance point; the coordinate difference threshold value set in this embodiment is 0.3.
Example two
Based on the same inventive concept, the embodiment provides the mobile robot obstacle avoidance system based on the condition variation automatic encoder, the principle of solving the problem is similar to the mobile robot obstacle avoidance method based on the condition variation automatic encoder, and repeated parts are not repeated.
The embodiment provides a mobile robot keeps away barrier system based on condition variational automatic encoder, includes:
the initialization module is used for initializing a pedestrian obstacle avoidance area of the robot;
the navigation module is used for receiving the original target point and navigating by the robot;
the pedestrian detection module is used for detecting pedestrians and outputting the positions and attributes of the pedestrians;
the obstacle avoidance range judging module is used for judging whether the pedestrian is in the obstacle avoidance range according to the position of the pedestrian;
the condition variation automatic encoder training module is used for generating temporary obstacle avoidance points in a coordinate form;
the temporary obstacle avoidance point screening module is used for screening the optimal temporary obstacle avoidance point with the lowest cost according to the temporary obstacle avoidance points;
the module for receiving the optimal temporary obstacle avoidance point is used for the robot to receive the optimal temporary obstacle avoidance point for navigation;
and the optimal temporary obstacle avoidance point arrival judging module is used for judging whether the robot arrives at the optimal temporary obstacle avoidance point.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should it be exhaustive of all embodiments. And obvious variations or modifications of the invention may be made without departing from the spirit or scope of the invention.
Claims (8)
1. The mobile robot obstacle avoidance method based on the condition variation automatic encoder is characterized by comprising the following steps:
step S1: initializing a pedestrian obstacle avoidance area of the robot, constructing a grid matrix by taking the robot as a center and acquiring obstacle positions;
step S2: the robot navigates and moves from the starting point to the target point according to the grid matrix;
and step S3: the robot detects the pedestrian and obtains the pedestrian position information and the pedestrian attribute information;
and step S4: judging whether the pedestrian is in the obstacle avoidance range or not according to the position of the pedestrian, if not, returning to the step S2, and if so, executing the step S5;
step S5: according to the pedestrian position information, expressing the position of a pedestrian by using coordinates and storing the position of the pedestrian into a pedestrian position array, expressing the attribute of the pedestrian by using a variable quantity and storing the attribute into a pedestrian attribute array according to the attribute of the pedestrian, training a condition variation automatic encoder by taking an obstacle point position, the starting point position and the target point position of the robot, the pedestrian position array and the pedestrian attribute array as input, and generating a temporary obstacle avoidance point in a coordinate form;
step S6: screening the optimal temporary obstacle avoidance points with the lowest cost according to the temporary obstacle avoidance points;
step S7: the robot carries out navigation again according to the optimal temporary obstacle avoidance points;
step S8: judging whether the robot reaches the optimal temporary obstacle avoidance point, if not, continuing to move until the robot reaches the optimal temporary obstacle avoidance point, if so, continuing to judge whether the robot reaches a target point, otherwise, returning to the step S3, and if so, determining that the robot reaches the target point;
in step S6, the method for screening an optimal temporary obstacle avoidance point with the lowest cost according to the temporary obstacle avoidance points includes: designing a cost function according to the distances from the temporary obstacle avoidance points to the pedestrians, the nearest obstacle and the original target point, calculating a cost value of each temporary obstacle avoidance point according to the cost function, and selecting the temporary obstacle avoidance point with the lowest cost as the optimal temporary obstacle avoidance point;
the cost function is:
wherein x is temp Indicating temporary obstacle avoidance points, x pi Coordinates, x, representing the ith pedestrian obs Coordinates representing nearest obstacle, x goal Coordinates representing an original target point; omega 1 、ω 2 、ω 3 Are weights.
2. The obstacle avoidance method of the mobile robot based on the condition variation automatic encoder as claimed in claim 1, wherein in step S3, the method for the robot to detect the pedestrian and obtain the pedestrian position information and the pedestrian attribute information comprises:
step S31: capturing surrounding images in real time through a depth camera carried by the robot;
step S32: detecting pedestrians in the surrounding image by applying a pedestrian detection algorithm to obtain positions of the pedestrians under a pixel coordinate system, and identifying the pedestrians in the surrounding image by applying a pedestrian attribute identification algorithm to obtain pedestrian attribute information;
step S33: transforming the positions of the pedestrians in the pixel coordinate system through coordinates, and outputting the positions of the pedestrians in the global coordinate system;
step S34: and according to the position of the pedestrian in the global coordinate system, calculating the moving speed and the moving direction of the pedestrian according to the moving distance and the passing time of the pedestrian, and predicting the position of the pedestrian at the next moment according to the constant speed model.
3. The obstacle avoidance method for the mobile robot based on the condition variation automatic encoder as claimed in claim 1, wherein in step S4, the method for determining whether the pedestrian is in the obstacle avoidance range according to the position of the pedestrian comprises:
step S41: according to the position of the pedestrian in the global coordinate system, calculating the distance d from the pedestrian to the position of the robot in the abscissa and the ordinate x And d y ；
Step S42: judgment of d x And d y And whether the side length of the pedestrian obstacle avoidance area is less than half of the side length of the pedestrian obstacle avoidance area, wherein the pedestrian obstacle avoidance area is a square area with a robot as a center in the grid map, if so, the pedestrian is in the obstacle avoidance area, and if not, the pedestrian is not in the obstacle avoidance area.
4. The obstacle avoidance method for the mobile robot based on the condition variation automatic encoder as claimed in claim 2, wherein in step S5, the method for representing the position of the pedestrian by coordinates and storing the position into the pedestrian position array according to the pedestrian position information comprises: the current position coordinates of the pedestrian and the position coordinates of the next two moments are respectively expressed by global-to-local coordinate conversion, in the pedestrian position array, the first two numbers respectively express the x coordinate and the y coordinate of the pedestrian at the current time, and the last four numbers express the coordinates of the pedestrian at the next two moments.
5. The obstacle avoidance method for the mobile robot based on the conditional variation automatic encoder according to claim 1, wherein in step S5, the method for expressing the attribute of the pedestrian with variable quantity and storing the attribute into the attribute array of the pedestrian according to the attribute of the pedestrian comprises: representing the attribute of the pedestrian by four 0/1 variables and storing the attribute into a pedestrian attribute array, wherein in the four 0/1 variables, the first numerical value represents the gender of the pedestrian, 0 represents a male, and 1 represents a female; the second number represents body width, 0 represents narrow, and 1 represents wide; the third value represents the age, 0 represents the middle age, and 1 represents the elderly or children; the fourth value represents mood, 0 represents happy or normal, and 1 represents negative mood.
6. The obstacle avoidance method for the mobile robot based on the conditional variation automatic encoder as claimed in claim 1, wherein in step S7, the method for the robot to navigate again according to the optimal temporary obstacle avoidance point includes: the optimal temporary obstacle avoidance point is issued to the robot in the form of a topic in the ROS, the original navigation is interrupted by the robot, and the robot preferentially navigates the optimal temporary obstacle avoidance point.
7. The obstacle avoidance method for the mobile robot based on the condition variation automatic encoder as claimed in claim 1, wherein in step S8, the method for determining whether the robot reaches the optimal temporary obstacle avoidance point includes: and acquiring the real-time position coordinate of the robot through the ROS, calculating the distance between the real-time position coordinate of the robot and the coordinate of the optimal temporary obstacle avoidance point, if the distance is less than or equal to a preset threshold value, indicating that the robot has arrived, and if the distance is greater than the preset threshold value, indicating that the robot has not arrived.
8. Mobile robot keeps away barrier system based on condition variational automatic encoder, its characterized in that includes:
the system comprises an initialization module, a fault detection module and a fault detection module, wherein the initialization module is used for initializing a pedestrian obstacle avoidance area of a robot, constructing a grid matrix by taking the robot as a center and acquiring the position of a fault point;
the navigation module is used for the robot to navigate and move from the starting point to the target point according to the grid matrix;
the pedestrian detection module is used for detecting pedestrians and outputting the positions and attributes of the pedestrians;
the obstacle avoidance range judging module is used for judging whether the pedestrian is in the obstacle avoidance range according to the position of the pedestrian;
the condition variation automatic encoder training module is used for representing the position of a pedestrian by coordinates and storing the position of the pedestrian into a pedestrian position array according to pedestrian position information, representing the attribute of the pedestrian by variables and storing the attribute into a pedestrian attribute array according to the attribute of the pedestrian, training the condition variation automatic encoder by taking the position of an obstacle point, the position of a starting point and a target point of a robot, the pedestrian position array and the pedestrian attribute array as input, and generating a temporary obstacle avoidance point in a coordinate form;
the temporary obstacle avoidance point screening module is used for screening the optimal temporary obstacle avoidance point with the lowest cost according to the temporary obstacle avoidance points;
the module for receiving the optimal temporary obstacle avoidance point is used for the robot to perform navigation again according to the optimal temporary obstacle avoidance point;
the optimal temporary obstacle avoidance point arrival judging module is used for judging whether the robot arrives at the optimal temporary obstacle avoidance point or not until the robot arrives at a target point;
in the temporary obstacle avoidance point screening module, the method for screening the optimal temporary obstacle avoidance point with the lowest cost according to the temporary obstacle avoidance point includes: designing a cost function according to the distances from the temporary obstacle avoidance points to the pedestrians, the nearest obstacle and the original target point, calculating a cost value of each temporary obstacle avoidance point according to the cost function, and selecting the temporary obstacle avoidance point with the lowest cost as the optimal temporary obstacle avoidance point;
the cost function is:
wherein x temp Indicating temporary obstacle avoidance points, x pi Coordinates, x, representing the ith pedestrian obs Coordinates, x, representing nearest obstacles goal Coordinates representing an original target point; omega 1 、ω 2 、ω 3 Are weights.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110997644.2A CN113515131B (en) | 2021-08-27 | 2021-08-27 | Mobile robot obstacle avoidance method and system based on condition variation automatic encoder |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110997644.2A CN113515131B (en) | 2021-08-27 | 2021-08-27 | Mobile robot obstacle avoidance method and system based on condition variation automatic encoder |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113515131A CN113515131A (en) | 2021-10-19 |
| CN113515131B true CN113515131B (en) | 2022-12-27 |
Family
ID=78062951
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110997644.2A Active CN113515131B (en) | 2021-08-27 | 2021-08-27 | Mobile robot obstacle avoidance method and system based on condition variation automatic encoder |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113515131B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114199266A (en) * | 2021-11-25 | 2022-03-18 | 江苏集萃智能制造技术研究所有限公司 | Path planning method for occupied target based on diagnosis guide service robot |
| CN115309164B (en) * | 2022-08-26 | 2023-06-27 | 苏州大学 | Man-machine co-fusion mobile robot path planning method based on generation of countermeasure network |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4464912B2 (en) * | 2004-12-03 | 2010-05-19 | パナソニック株式会社 | Robot control apparatus and autonomous mobile robot |
| JP4576445B2 (en) * | 2007-04-12 | 2010-11-10 | パナソニック株式会社 | Autonomous mobile device and program for autonomous mobile device |
| CN103558856A (en) * | 2013-11-21 | 2014-02-05 | 东南大学 | Service mobile robot navigation method in dynamic environment |
| CN106708054B (en) * | 2017-01-24 | 2019-12-13 | 贵州电网有限责任公司电力科学研究院 | Routing planning method for inspection robot by combining map grids and potential field method obstacle avoidance |
| CN108255182B (en) * | 2018-01-30 | 2021-05-11 | 上海交通大学 | Service robot pedestrian perception obstacle avoidance method based on deep reinforcement learning |
| CN108762264B (en) * | 2018-05-22 | 2021-05-04 | 重庆邮电大学 | Dynamic obstacle avoidance method of robot based on artificial potential field and rolling window |
| CN110561432B (en) * | 2019-08-30 | 2021-04-23 | 广东省智能制造研究所 | Safety cooperation method and device based on man-machine co-fusion |
| CN111612078A (en) * | 2020-05-25 | 2020-09-01 | 中国人民解放军军事科学院国防工程研究院 | Transformer fault sample enhancement method based on condition variation automatic encoder |
| CN112015186A (en) * | 2020-09-09 | 2020-12-01 | 上海有个机器人有限公司 | Robot path planning method and device with social attributes and robot |
| CN112378408B (en) * | 2020-11-26 | 2023-07-25 | 重庆大学 | Path planning method for realizing real-time obstacle avoidance of wheeled mobile robot |
| CN112631294B (en) * | 2020-12-16 | 2022-06-28 | 上海应用技术大学 | Intelligent path planning method for mobile robot |
| CN112904842B (en) * | 2021-01-13 | 2022-07-15 | 中南大学 | Mobile robot path planning and optimizing method based on cost potential field |
| CN112882469B (en) * | 2021-01-14 | 2022-04-08 | 浙江大学 | Deep reinforcement learning obstacle avoidance navigation method integrating global training |
-
2021
- 2021-08-27 CN CN202110997644.2A patent/CN113515131B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113515131A (en) | 2021-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Song et al. | Pedestrian trajectory prediction based on deep convolutional LSTM network | |
| CN110285813B (en) | Man-machine co-fusion navigation device and method for indoor mobile robot | |
| JP7471397B2 (en) | Simulation of various long-term future trajectories in road scenes | |
| KR101986002B1 (en) | Artificial agents and method for human intention understanding based on perception-action connected learning, recording medium for performing the method | |
| Lyu et al. | Robot path planning by leveraging the graph-encoded Floyd algorithm | |
| JP7339029B2 (en) | Self-motion estimation device and method using motion recognition model and motion recognition model training device and method | |
| CN107272680A (en) | A kind of automatic follower method of robot based on ROS robot operating systems | |
| CN113515131B (en) | Mobile robot obstacle avoidance method and system based on condition variation automatic encoder | |
| CN112771542A (en) | Learning-enhanced neural network based on learned visual entities | |
| Yang et al. | A novel graph-based trajectory predictor with pseudo-oracle | |
| Mirsky et al. | Prevention and resolution of conflicts in social navigation–a survey | |
| CN110737968A (en) | Crowd trajectory prediction method and system based on deep convolutional long and short memory network | |
| Cheng et al. | Mixed traffic trajectory prediction using lstm–based models in shared space | |
| CN113759901A (en) | Mobile robot autonomous obstacle avoidance method based on deep reinforcement learning | |
| Xu et al. | Avoidance of manual labeling in robotic autonomous navigation through multi-sensory semi-supervised learning | |
| Singh et al. | Understanding socially aware robot navigation | |
| CN116202526A (en) | Crowd navigation method combining double convolution network and cyclic neural network under limited view field | |
| Katuwandeniya et al. | Multi-modal scene-compliant user intention estimation in navigation | |
| KR102331803B1 (en) | Vision and language navigation system | |
| Luo et al. | Recursive neural network based semantic navigation of an autonomous mobile robot through understanding human verbal instructions | |
| Chen et al. | Q‐EANet: Implicit social modeling for trajectory prediction via experience‐anchored queries | |
| Botzheim et al. | Growing neural gas for information extraction in gesture recognition and reproduction of robot partners | |
| Katyal et al. | Prediction-based uncertainty estimation for adaptive crowd navigation | |
| Wang et al. | Deep reinforcement learning-aided autonomous navigation with landmark generators | |
| CN115309164B (en) | Man-machine co-fusion mobile robot path planning method based on generation of countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |