CN117572773A - Foot robot motion trail planning method, system, equipment and terminal - Google Patents
Foot robot motion trail planning method, system, equipment and terminal Download PDFInfo
- Publication number
- CN117572773A CN117572773A CN202311582605.1A CN202311582605A CN117572773A CN 117572773 A CN117572773 A CN 117572773A CN 202311582605 A CN202311582605 A CN 202311582605A CN 117572773 A CN117572773 A CN 117572773A
- Authority
- CN
- China
- Prior art keywords
- track
- robot
- state
- motion
- foot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000005457 optimization Methods 0.000 claims abstract description 63
- 238000005312 nonlinear dynamic Methods 0.000 claims abstract description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 24
- 238000005070 sampling Methods 0.000 claims description 15
- 230000001133 acceleration Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 4
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 claims description 3
- 239000010931 gold Substances 0.000 claims description 3
- 229910052737 gold Inorganic materials 0.000 claims description 3
- 230000005484 gravity Effects 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 2
- 230000008859 change Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000007689 inspection Methods 0.000 description 10
- 238000012360 testing method Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 241001465754 Metazoa Species 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 206010048669 Terminal state Diseases 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 239000003245 coal Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000011664 nicotinic acid Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 241000282324 Felis Species 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 241000124008 Mammalia Species 0.000 description 1
- 206010034701 Peroneal nerve palsy Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/042—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention belongs to the technical field of robots, and discloses a method, a system, equipment and a terminal for planning a motion trail of a foot robot, which comprise the following steps: generating an upper global collision-free track, optimizing a middle nonlinear dynamic track, and predicting and controlling a desired state track by a bottom model; the upper module rapidly generates a rough polynomial track of robot mass heart movement on a global obstacle map, wherein the rough polynomial track comprises plane positions [ x, y ], direction angles theta and track time T of the robot movement; the middle layer module constructs a nonlinear optimization problem according to the initial track and the robot dynamics, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit, foot robot omnidirectional motion constraint and track time cost, wherein the optimization variables are track polynomial coefficients and time expressed in a segmented manner; and the bottom layer module takes the optimized track as a desired centroid space-time state track of the nonlinear model predictive controller to carry out specific motion control of the robot.
Description
Technical Field
The invention belongs to the technical field of robots, and particularly relates to a foot-type robot motion trail planning method, a system, equipment and a terminal.
Background
The foot robot is a bionic robot which simulates the motion of foot mammals in nature, such as human beings, felines and the like, and has omnidirectional motion capability and complex terrain passing capability. At present, the motion control of the body of the foot robot has a higher level, has a certain actual animal motion capability, such as translational motion, steering motion, jumping and rugged ground walking, but most motion control involves a single motion mode, the flexible motion of the animal in the real world is a combination of various motion modes, and the current various motion controllers are at a distance from realizing the agile motion of the animal.
The model predictive controller is widely applied to motion control of a foot-type robot with no collision constraint condition to realize navigation planning of the robot. However, the control effect is greatly influenced by the prediction period of the model prediction controller, the long prediction period causes long time spent on solving, and the short prediction period easily causes the optimization solving to sink into local optimum. Learning-based methods enable real-time navigation of robots, but require pre-training or use of data sets. These datasets are mainly acquired in a simulated environment, and differences occur when used in an actual environment, and the manufacturing cost of the real world dataset is very high. The track generated by using the traditional path planning method based on sampling or searching is difficult to conform to the complex dynamic model of the foot robot, so that larger errors or slow tracking speed are easy to occur in the track tracking process.
Through the above analysis, the problems and defects existing in the prior art are as follows:
(1) The actual movement of the robot is slow: the potential of the motion speed of the foot-type robot is not exerted, and the motion is slow in order to ensure the track tracking effect;
(2) The motion trail of the robot is single: most of the robot is a geometric track in space, only shows obstacle avoidance capability, and does not show the flexibility of the motion of the foot-type robot;
(3) The real-time performance of the planning algorithm is difficult to ensure: the dynamics model of the foot robot is very complex, the dimension of the planning problem is large, the planning solving time is long, and the real-time performance is difficult to ensure.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a motion trail planning method, a motion trail planning system, motion trail planning equipment and motion trail planning terminal for a foot-type robot.
The invention is realized in such a way that the motion trail planning method of the foot-type robot comprises three layers, namely: the system comprises an upper global collision-free track generation module, a middle nonlinear dynamic track optimization module and a bottom model prediction controller state track tracking module. The upper module rapidly generates a rough polynomial track of robot mass heart movement on a global 2D obstacle map, wherein the rough polynomial track comprises a plane position [ x, y ], a direction angle theta and a track duration time T of the robot movement; the middle layer module constructs a nonlinear optimization problem according to the initial track obtained by searching and the robot dynamics, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit (speed, acceleration), foot robot omnidirectional motion constraint and track time cost, wherein optimization variables are coefficients of polynomial tracks expressed in a segmented manner and duration of each track; the bottom layer module predicts the expected centroid space-time state track of the controller by using the optimized track as a nonlinear model, and tracks the motion track of the centroid of the robot by using the controller to control the motion of the robot body.
Further, in the generation of the upper global collision-free trajectory, the planned robot state is the plane position and direction angle [ x, y, θ ]]Dispersing a given uniform space into g×g grids, each grid being associated with a corresponding state P 2D (idx，idy)＝[x，y，θ]In association, the sampling strategy is:
x＝idx·grid+rand(-1，1)·bias
y＝idy·grid+rand(-1，1)·bias.
where (idx, idy) is the index of the state point, grid is the grid size, g is the number of discrete grids, P 2D Is a state point stored in the state point set RoadMap, and the total number of state points is g×g=n.
Further, foot-type machineThe robot can move omnidirectionally, so the state quantity [ x, y, theta ]]Separately considering the problem of constructing the connection of two state points as an optimal boundary value, the initial state is given as s i ＝[s pi ，s vi ]Is the state of the father node, and the termination state is s f ＝[s pf ，s vf ]The termination position is the position of the child node, the termination speed is obtained by solving, and the energy J (T) = [ pi ] of the whole state track is optimized T s a (t) 2 dt is minimum (i.e., acceleration integral is minimum), and using the poincare gold maximum principle, the display solution for the state estimation is:
wherein a numerical solution of the problem requires a trajectory time T by giving a reference line speed v ref And angular velocity omega ref Setting a reference time t=t ref ：＝max(||[Δx，Δy]|| 2 /v ref ，Δθ/ω ref )。
Further, in the nonlinear dynamic trajectory optimization, the optimization index includes: the smoothness cost of the state track is lambda s Cost of state track distance obstacle, weight is lambda c And the time cost of the track segment, the weight is lambda t Limiting robot dynamics such as maximum speed, maximum acceleration toThe limitation of the omnidirectional movement of the robot is thatThe continuity of the front and rear states at the state point is limited to +.>As a constraint on the optimization problem. The solving variables of the optimization problem are polynomial coefficients c and time T of each segment of state track:
where s (T) is the N-th order polynomial trajectory of the state variables x, y, θ, N is the polynomial segment number, j represents the j-th segment, R is the positive weighting matrix of the state smoothness cost, the cost specific gravity between the three state variables is measured, T is the time vector of the trajectory segment, T j Is the duration of the j-th polynomial, c ji An nth order coefficient vector representing a j-th stage polynomial.
Further, in the state track tracking of the model predictive controller, a nonlinear model predictive control problem is constructed:
g(x，u，t)＝0
h(x，u，t)＜0
where x (t) and u (t) are state variables and state inputs, Φ (·) is the terminal state constraint cost function, L (·) is the quadratic cost function of trajectory tracking,is the current observation state. f (f) c (. Cndot.), g (. Cndot.) and h (. Cndot.) are system dynamic equations, equality constraints and inequality constraints, respectively.
A speed safety constraint is added in the model predictive controller to ensure the stability during the rapid movement,
wherein lambda is 1 And lambda (lambda) θ The weights of the translation speed and the angular speed are measured,is a safety threshold.
And the model prediction controller solves and calculates a control instruction of a robot joint motor according to the expected state track and a dynamic model of the robot, so as to realize autonomous movement of the robot.
Another object of the present invention is to provide a motion trajectory planning system for a foot robot applying the motion trajectory planning method for a foot robot, including:
the upper layer module is used for quickly generating a rough polynomial track of robot mass heart movement on the global obstacle map, and comprises plane positions [ x, y ], direction angles theta and track time T of the robot movement;
the middle layer module is used for constructing a nonlinear optimization problem according to the initial track and the robot dynamics, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit, foot robot omnidirectional motion constraint and track time cost, wherein the optimization variables are track polynomial coefficients and time expressed in a segmented manner;
and the bottom layer module is used for taking the optimized track as a desired centroid space-time state track of the nonlinear model predictive controller and carrying out specific motion control on the robot.
Another object of the present invention is to provide a computer device, where the computer device includes a memory and a processor, and the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the motion trail planning method of the foot-type robot.
Another object of the present invention is to provide a computer readable storage medium storing a computer program, which when executed by a processor, causes the processor to execute the steps of the motion trajectory planning method of the foot robot.
The invention further aims to provide an information data processing terminal which is used for realizing the motion trail planning system of the foot-type robot.
In combination with the technical scheme and the technical problems to be solved, the technical scheme to be protected has the following advantages and positive effects:
first, the foot robot is used as a novel bionic robot, a dynamic model is complex, and problems related to planning and control are less in research and difficult to solve. The motion problems with sharp characteristics and key of the foot-type robot are extracted from the complex dynamics model, and modeling, expressing and solving the motion problems are very difficult. In addition, in order to ensure the real-time performance of planning, the model complexity, the problem complexity and the solving efficiency must be balanced, so as to realize online planning. The research and development of the foot robot are for practical application, the real-time navigation task in a real unknown environment is the basis of the application of the foot robot, and the autonomy of the robot is a key problem of wide application. The foot-type robot is applied to the real scenes of inspection, transportation, security, search and rescue, pursuit and the like, autonomous movement is the basis for various tasks to be developed, and the application of the foot-type robot with great assistance is landed by flexible and quick real-time planning.
1) Real-time motion trail and tracking of the mobile robot are limited by an on-board computing unit, so that on-line planning time is long, and efficiency is difficult to improve. The invention adopts a hierarchical planning method to balance the planning problem dimension and the planning length in different planning layers, efficiently distributes the computing resources and realizes stable real-time planning.
2) Searching a global track by using a hierarchical planning method and combining track searching to avoid sinking into a local optimal solution; the quality of the track is improved by using track optimization, so that the track is more in line with the actual motion characteristic of the foot-type robot; the model predictive controller tracks the track of a preset period and ensures the tracking effect of the motion track.
3) The current foot robot planning method mostly considers the motion stability, and the robot moves slowly. The method considers the anisotropy of the omnidirectional movement of the foot-type robot, plans the space-time track (comprising the time sequence of position, speed and acceleration, not only the space geometric track, but also the track related to time with multiple dimensions) of the movement of the robot, optimizes the track by using multiple indexes, and plays the advantages of the rapid movement and the flexibility of the foot-type robot.
Secondly, the technical scheme is regarded as a whole or from the perspective of products, and the technical scheme to be protected has the following technical effects and advantages:
1) The invention divides the planning task into three layers, the complexity of the problem from the top layer to the bottom layer is increased, the planning distance is shortened, and compared with a single planning layer, the method and the system can efficiently allocate the computing resources and realize the online planning.
2) Compared with random sampling, the uniform sampling with random deviation greatly quickens the searching efficiency on the premise of ensuring that the optimality of the generated track is almost unchanged.
3) The track cost calculation strategy used in the optimal track searching process considers the motion characteristics of the foot-type robot, additionally considers the planning of the direction angle of the robot on the basis of the plane 2D position, calculates the geometrical distance of the track space, and calculates the steering cost, so that the initial track is more attached to the dynamics of the robot, and has pertinence.
4) The invention builds the local track optimization problem on the basis of the polynomial track in the search stage, considers obstacle avoidance, dynamic limit and time optimization, and can further improve the overall track quality. In particular, anisotropic constraints of the omnidirectional movement of the foot robot, namely constraints of the omnidirectional movement speed and the movement direction, are introduced.
5) According to the nonlinear optimization problem solving method, a local optimization method with gradient descent is used, and the searched result is used as an initial solution, so that the quality of the optimization problem solving can be guaranteed, the probability of abnormal results is reduced, and the solving speed of nonlinear optimization is improved.
6) The optimized track is directly used as the expected state track of the robot body nonlinear model predictive controller, and compared with an instruction tracking mode using acceleration, speed or position, the optimized track can realize more complex motion behaviors, and the tracking effect is greatly improved.
Thirdly, as inventive supplementary evidence of the claims of the present invention, the following important aspects are also presented:
(1) The expected benefits and commercial values after the technical scheme of the invention is converted are as follows:
the invention relates to a motion planning method of a foot robot, which is a basis of autonomous motion of the foot robot and has wide application scenes. At present, the motion trail planning algorithm of the mobile robot is mostly an unmanned plane and a wheeled robot, and the configuration, the dynamic model and the motion mode of the mobile robot are obviously different from those of a foot-type robot. The related methods have difficulty in functioning well on foot robots. The method is suitable for the foot robot to execute the inspection task or provide an autonomous navigation basis for the complex task. The floor application of the foot robot has wide space, the market value is extremely high, and the method for planning the motion trail of the foot robot is a very key technology and is also a key about whether the foot robot can play a role in various potential application scenes. The innovation of the foot robot planning method has extremely high application value in the commercial field, the research and development cost is reflected in the development and test stage, the cost for deploying the algorithm on the robot is extremely low, the profit space is huge, and the method has considerable profit space in the form of technical authorization or algorithm core single-machine encapsulation.
(2) The technical scheme of the invention fills the technical blank in the domestic and foreign industries:
at present, motion planning of a foot robot is mainly focused on a stable motion level of a robot body, or foot drop planning of the foot robot on rough terrain, and the control target of the foot robot is that the foot robot can stably pass through various terrains. In these methods, stability is the most critical consideration, so most robots move slowly, and for practical application scenes, the use requirements in most scenes are not satisfied. The robot motion planning method provided by the invention focuses more on the motion potential of the robot body, so that the robot can move in space quickly and flexibly, and the autonomous motion of the robot body is realized in various general scenes.
Fourth, the motion trail planning method of the foot robot provided by the invention is a great innovation of the robot path planning technology, and the remarkable technical progress brought by the method is mainly realized in the following aspects:
1) Three-layer planning framework: according to the method, the motion track return planning of the robot is divided into three layers, namely, upper global collision-free track generation, middle nonlinear dynamic track optimization and bottom model prediction control expected state track tracking, so that accurate and efficient motion planning is realized.
2) Omnidirectional exercise planning: the method fully considers the omnidirectional movement characteristic of the foot-type robot, can optimize the track for generating the omnidirectional movement, and improves the action efficiency of the robot in a complex environment.
3) Optimization index diversification: in nonlinear dynamic track optimization, a plurality of optimization indexes such as obstacle avoidance cost, state track smoothness, robot dynamic limit, omnidirectional motion constraint, track time cost and the like are considered, so that the requirements of various aspects such as obstacle avoidance, smooth motion, quick response and the like are balanced, and the motion performance of the robot is improved.
4) Model predictive control expected state trajectory tracking: by taking the optimized track as the expected centroid state track of the model predictive controller, the precise control of the specific movement of the robot is realized, and the stability and the accuracy of the movement of the robot are improved.
5) Safety enhancement: the bottom model predictive controller adds speed safety constraint, so that the robot can meet the requirement of track optimization and meanwhile, the motion safety can be guaranteed.
The foot-type robot motion trail planning method provided by the invention can realize the efficient, safe and accurate motion of the robot in a complex environment, and greatly improves the application value and practicability of the robot.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of track distance of a hierarchical planner provided by an embodiment of the present invention;
FIG. 2 is a schematic diagram of a sample search test provided by an embodiment of the present invention; wherein, (a) random sampling+dijkstra's algorithm, (b) uniform sampling with random bias+dijkstra's algorithm, (c) uniform sampling with random bias+lazyprm algorithm;
FIG. 3 is a schematic diagram of results of different track calculation costs in a search stage according to an embodiment of the present invention;
FIG. 4 is a flow chart of track generation provided by an embodiment of the present invention;
fig. 5 is a schematic diagram of a robot body coordinate system according to an embodiment of the present invention;
FIG. 6 is a flow chart of a navigation framework provided by an embodiment of the present invention;
FIG. 7 is a schematic diagram of a programming algorithm test provided by an embodiment of the present invention;
FIG. 8 is a graph of a change track of a horizontal position x state quantity provided by an embodiment of the present invention;
FIG. 9 is a diagram of a change track of a state quantity of a horizontal position y provided by an embodiment of the present invention;
FIG. 10 is a diagram of a change track of a state quantity of a direction angle θ provided by an embodiment of the present invention;
FIG. 11 is a block diagram of a motion trajectory planning system for a foot-type robot provided by an embodiment of the present invention;
fig. 12 is a schematic diagram of a robot motion trail plan test tracking effect provided by an embodiment of the present invention;
FIG. 13 is a night real world test of a robotic motion planning system provided by an embodiment of the present invention;
FIG. 14 is a daytime real world test of a robotic motion planning system provided by an embodiment of the present invention;
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Aiming at the problems in the prior art, the invention provides a motion trail planning method, a motion trail planning system, motion trail planning equipment and motion trail planning terminal for a foot-type robot, and the invention is described in detail below with reference to the accompanying drawings.
Example 1: service robot in indoor environment
In indoor environments, foot robots such as service robots are required to avoid various obstacles such as furniture, decorations, etc., while also reaching a target location as quickly as possible. In this case, the trajectory planning method described above may be used.
1. The upper module generates a rough polynomial trajectory on a global indoor environment map (e.g., a 2D map generated by a lidar) that indicates the preliminary path of travel of the robot.
2. The middle layer module solves the nonlinear optimization problem through an optimization algorithm (such as a gradient descent method or a genetic algorithm) according to the initial path and a dynamic model of the robot, and generates an optimization track which is smooth, obstacle-avoiding and accords with the omnidirectional motion constraint.
3. The bottom layer module uses a model predictive controller to generate a motion control instruction of the robot, such as a rotating speed and a steering instruction of a motor, according to the optimized track, so as to realize autonomous motion of the robot.
Example 2: search and rescue robot in outdoor environment
In an outdoor environment, such as a mountain area, a forest, etc., a search and rescue robot needs to search for a target rapidly while ensuring safety. In this case, the trajectory planning method described above may also be used.
1. The upper layer module generates a rough polynomial trajectory on a global outdoor environment map (e.g., a 3D map generated by the drone) that indicates the preliminary path of travel of the robot.
2. The middle layer module solves the nonlinear optimization problem through an optimization algorithm (such as a particle swarm optimization algorithm or a simulated annealing algorithm) according to the initial path and a dynamic model of the robot, and generates an optimization track which is smooth, obstacle-avoiding and accords with the omnidirectional motion constraint.
3. The bottom layer module uses a model predictive controller to generate a motion control instruction of the robot, such as a rotating speed and a steering instruction of a motor, according to the optimized track, so as to realize autonomous motion of the robot.
The hierarchical planning method of the invention has three layers, namely: generating an upper global collision-free track, optimizing a middle nonlinear dynamic track, and predicting and controlling the track tracking of an expected state by a bottom model. The upper layer module rapidly generates a rough polynomial track of robot mass heart movement on the global obstacle map, wherein the rough polynomial track comprises plane positions [ x, y ], direction angles [ theta ] and track time T of the robot movement. The middle layer module constructs a nonlinear optimization problem according to the initial track obtained by searching and the robot dynamics model, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit, foot robot omnidirectional motion constraint and track time cost, and optimization variables are track polynomial coefficients and time expressed in a segmented mode. And the bottom layer module takes the optimized track as a desired centroid space-time state track of the nonlinear model predictive controller to carry out specific motion control of the robot. The planned trajectory distance of the hierarchical planner is shown in fig. 1.
Global collision-free trajectory generation: foot robots have six degrees of freedom, but only horizontal position and direction angle are of major concern during movement, as body height, pitch angle and roll angle will vary depending on the terrain. The planned robot state is the planar position and direction angle [ x, y, θ ]]Dispersing a given uniform space into g×g grids, each grid being associated with a corresponding state P 2D (idx，idy)＝[x，y，θ]Associated, sampling strategy is
x＝idx·grid+rand(-1，1)·bias
y＝idy·grid+rand(-1，1)·bias.
Where (idx, idy) is the index of the state point, grid is the grid size, g is the number of discrete grids, P 2D Is a state point stored in the state point set RoadMap, and the total number of state points is g×g=n. The invention modifies LazyPRM algorithm, only consider neighbor nodes around father node in expansion searching stage, and index neighbor state point in state point set, time complexity is O (n), if complete random sampling is carried out to whole space, dijkstra method is used to search to obtain optimal result, time complexity is O (n) 2 ) The improved method of the invention greatly improves the searching efficiency on the basis of hardly influencing the optimality of the final result, and the comparison result is shown in figure 2.
The direction angle of the robot is initialized to 0, and is determined by the connection between the father node and the child node in the searching process and is expressed as theta curr ＝arctan((y curr -y pare )/(x curr -x pare )). Wherein x is curr ，y curr For the current node coordinates, x pare ，y pare Parent node coordinates.
As shown in fig. 2, in a space of 8m x 8m, 400 points are randomly sampled, a grid size is set to 0.4m, 100 sampling search tests are performed, fig. 2 (a) is a random sampling+dijkstra's algorithm, average time consumption 39.8697s, average path length 9.6137m, fig. 2 (b) is a uniform sampling+dijkstra's algorithm with random bias, average time consumption 40.4524s, average path length 9.6254m, fig. 2 (c) is a uniform sampling+lazyprm algorithm with random bias, average time consumption 0.6953s, average path length 9.6216m,
the foot robot can move omnidirectionally, so the state quantity [ x, y, theta ]]It can be considered separately that the connection of two state points is constructed as an optimal boundary value problem, and the initial state is given as s i ＝[s pi ，s vi ]Is the state of the father node, and the termination state is s f ＝[s pf ，s vf ]The termination position is the position of the child node, the termination speed is obtained by solving, and the energy J (T) =f of the whole state track is optimized T s a (t) 2 dt is minimum (i.e., acceleration integral is minimum), and using the poincare gold maximum principle, the display solution for the state estimation is:
wherein a numerical solution of the problem requires a trajectory time T by giving a reference line speed v ref And angular velocity omega ref Setting a reference time t=t ref ：＝max(||[Δx，Δy]||2/v ref ，Δθ/ω ref )。
The direction angle and the linear velocity direction have a significant influence on the stability of the robot movement. Thus, the present invention proposes the following trajectory cost
Wherein the first term is the direction angle change cost, and the weight coefficient is lambda yaw The second term is the arc length of the track. The use of quadratic form to calculate the cost of yaw angle can make the change of direction angle between state points smoother, see FIG. 3 (schematic diagram of different results of track calculation cost in search stage)
From P 1 To P 5 The angle difference of the points is consistent, but the secondary cost left graph of the angle change is larger than that of the right graph, the track change of the right graph is obviously smoother, and the secondary cost of the angle change provided by the invention can obtain a smoother track.
Through sampling and searching, the invention obtains a rough robot mass center motion track and a series of state nodes on the track, the track state quantity is [ x, y, theta ], the state nodes are expressed by polynomial coefficients and time, and the positions and the speeds at the state nodes are continuous. The track generation flow chart is shown in fig. 4.
Nonlinear dynamic trajectory optimization
The rough track is an unconstrained optimal problem, a plurality of track segments with unreachable dynamics exist in the whole state track, and the potential of the movement speed of the robot is further fully utilized on the basis of considering the dynamics characteristics of the foot-type robot, so that a nonlinear track optimization problem based on a piecewise polynomial is constructed.
As shown in fig. 5, the forward and backward motion and translational motion capabilities of the foot robot are different, and the present invention proposes an assumption to represent the anisotropy of the omni-directional motion of the foot robot: the maximum speed of the forward and backward movement of the robot is v mx Maximum speed of translational movement v my And v my ＜v mx The maximum linear velocity of the robot movement is related to the robot movement direction, both of which constitute an elliptical constraint, which can be expressed as in fig. 5:
where θ is the direction angle of the robot and R (θ) is the rotation matrix from the world inertial coordinate system I to the robot body coordinate system B.
The invention constructs nonlinear optimization problem based on piecewise polynomials, and the optimization indexes comprise: the smoothness cost of the state track is lambda s Cost of state track distance obstacle, weight is lambda c And the time cost of the track segment, the weight is lambda t Limiting robot dynamics such as maximum speed, maximum acceleration toThe restriction of the omnidirectional movement of the robot is +.>The continuity of the front and rear states at the state point is limited to +.>As a constraint on the optimization problem. The solving variables of the optimization problem are polynomial coefficients c and time T of each segment of state track.
Where s (T) is the N-th order polynomial trajectory of the state variables x, y, θ, N is the polynomial segment number, j represents the j-th segment, R is the positive weighting matrix of the state smoothness cost, the cost specific gravity between the three state variables is measured, T is the time vector of the trajectory segment, T j Is the duration of the j-th polynomial, c ji An nth order coefficient vector representing a j-th stage polynomial.
The nonlinear optimization problem is solved using a numerical optimization method of continuous state discretization and gradient descent. And the state track coefficient and time of the search stage are used as an initial solution of the non-believing optimizing solver, so that the solving quality is ensured.
The invention can obtain the state track conforming to the dynamics characteristic of the foot-type robot.
Model predictive control expected state trajectory tracking
And taking the optimized state track as an expected centroid track of the model predictive controller, and performing motion control on the robot body. In the state track tracking of the model predictive controller, constructing a nonlinear model predictive control problem:
g(x，u，t)＝0
h(x，u，t)＜0
where x (t) and u (t) are state variables and state inputs, Φ (·) is the terminal state constraint cost function, L (·) is two of the traceA secondary cost function of the model is provided,is the current observation state. f (f) c (. Cndot.), g (. Cndot.) and h (. Cndot.) are system dynamic equations, equality constraints and inequality constraints, respectively.
In the course of high-speed movement, if both the angular velocity and the linear velocity are large, the movement of the robot becomes very unstable and is liable to fall. The present invention adds a speed safety constraint in the model predictive controller,
wherein lambda is 1 And lambda (lambda) θ The weights of the translation speed and the angular speed are measured,is a safety threshold.
And the model prediction controller solves and calculates a control instruction of a robot joint motor according to the expected state track and a dynamic model of the robot, so as to realize autonomous movement of the robot.
(3) The working principle part is as follows:
1. the invention integrates the planning algorithm into a real-time navigation frame, combines front-end environment sensing and back-end motion control, tests the planning algorithm of the invention, and a navigation frame flow chart is shown in figure 6.
2. The method comprises the steps of setting a target pose through a computer end, sensing the surrounding environment by using a radar, positioning a robot, generating a motion track to an end point by a planner according to map information and the current pose, transmitting a state track with a certain prediction time in the future to a motion controller, and calculating an expected instruction of a joint motor by the motion controller.
3. The algorithm of the invention tests the planning algorithm in different scenes, such as the scene shown in fig. 7, generates a continuous track with smooth state, five-pointed star is an optimized track end point, and triangle is a search track end point. NO-T is an optimized track, KD-T is a search track, bound is a dynamic limit, horizontal obstacle avoidance track, black is an obstacle, gray scale is a distance field gradient, a horizontal position [ x ] state quantity change track is shown in FIG. 8, a horizontal position [ y ] state quantity change track is shown in FIG. 9, and a direction angle [ theta ] state quantity change track is shown in FIG. 10.
As shown in fig. 11, a motion trajectory planning system for a foot robot according to an embodiment of the present invention includes:
the upper layer module is used for quickly generating a rough polynomial track of robot mass heart movement on the global obstacle map, and comprises plane positions [ x, y ], direction angles theta and track time T of the robot movement;
the middle layer module is used for constructing a nonlinear optimization problem according to the initial track and the robot dynamics, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit, foot robot omnidirectional motion constraint and track time cost, wherein the optimization variables are track polynomial coefficients and time expressed in a segmented manner;
and the bottom layer module is used for taking the optimized track as a desired centroid space-time state track of the nonlinear model predictive controller and carrying out specific motion control on the robot.
An application embodiment of the present invention provides a computer device, where the computer device includes a memory and a processor, and the memory stores a computer program, and when the computer program is executed by the processor, the processor executes steps of a motion trail planning method of a foot robot.
An application embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to execute steps of a foot robot motion trajectory planning method.
The embodiment of the application of the invention provides an information data processing terminal which is used for realizing a foot-type robot motion trail planning system.
The invention is an innovation guided by practical application, and aims to improve the autonomy of the foot-type robot and promote the application of the foot-type robot in a real scene to land. The application field is the basic field of autonomous movement of foot robots.
1 the existing coal mineral collection mostly uses large-scale automatic machinery, and the autonomous mining has realized wide application and forms a standard operation flow. The number of constructors in the mine tunnel is greatly reduced, the safety of coal mine control is increased, but the need of inspection personnel to check whether various devices in an autonomous operation channel are complete, normally operate or have potential safety hazards is still unavoidable. The space in the mine tunnel is relatively closed, but the environment is complex, the working condition is bad, and the foot-type robot has excellent terrain adaptability and can adapt to complex working conditions. The planning algorithm provided by the invention can be used in such an environment and is applied to the movement track planning of the mine tunnel inspection foot-type robot.
2, the expressway needs maintenance personnel to periodically patrol and examine the road surface condition. The tunnel section environment of expressway is sealed, and inspection channel environment is complicated, and wheeled robot is difficult to normally pass, and the unavoidable manual inspection of needs. Moreover, as the tunnel security problem is of great importance, the inspection frequency is relatively high, the expressway scale of China has the first world, and the inspection investment is huge. The foot robot is very suitable for the inspection work of complex road conditions of tunnel environments, the robot self-suffices to identify the tunnel environments, a feasible safety interval and a working interval are dynamically constructed according to tasks and environments, and an inspection path of the robot is planned. The path planning method of the present invention is expected to find application in this field.
General task scenes of the 3-legged robot, such as tasks of picking, carrying and stacking of the humanoid robot, transporting and accompanying of the four-legged robot, and the like. The motion trail planning is needed to be used as a mobile planning module, a foundation is provided for the completion of the upper layer task, and the method is one of basic conditions for the realization of the complex task.
The invention verifies on a foot robot physical platform, the experimental platform is a small quadruped robot, and the sensor equipment and the computer unit are carried.
Firstly, the track planning result of a given scene in an example is tested in real time, the track rationality obtained by the planning method provided by the invention is verified, and the experimental result is shown in figure 12. The actual motion planning of the foot robot swings at the planned expected track, the motion effect is good, and the algorithm provided by the invention is reasonable and effective.
And sensing the surrounding environment by using a laser radar, updating an obstacle map, and planning and controlling the robot motion planning on the basis. Setting the reference speed, the dynamic limit and the radius of the local map to be 3m, and carrying out real-time online planning. Real world night scenes, as in fig. 13, and real world day scenes, as in fig. 14, were tested. The planner searches the global collision-free reachable track rapidly in real time, and optimizes the track continuously in real time in the moving process, so that the track can meet the robot dynamics better. The robot can flexibly pass through between obstacles, the planning framework can adapt to uneven grasslands and rugged terrains, the influence of moving obstacles on the planning system can be avoided to a certain extent by the probability obstacle map updated in real time, and the robot can flexibly and rapidly reach the target position and gesture. The motion parameters of the robot in both scenes are shown in table 1.
It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.
Claims (10)
1. The motion track planning method of the foot robot is characterized by comprising three layers, namely:
generating an upper global collision-free track, optimizing a middle nonlinear dynamic track, and predicting and controlling a desired state track by a bottom model; the upper module rapidly generates a rough polynomial track of robot mass heart movement on a global obstacle map, wherein the rough polynomial track comprises plane positions [ x, y ], direction angles theta and track time T of the robot movement; the middle layer module constructs a nonlinear optimization problem according to the initial track and the robot dynamics, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit, foot robot omnidirectional motion constraint and track time cost, wherein the optimization variables are track polynomial coefficients and time expressed in a segmented manner; and the bottom layer module takes the optimized track as a desired centroid space-time state track of the nonlinear model predictive controller to carry out specific motion control of the robot.
2. The method for planning motion trajectory of foot robot according to claim 1, wherein in the generation of upper global collision-free trajectory, the planned robot state is plane position and direction angle [ x, y, θ]Dispersing a given uniform space into g×g grids, each grid being associated with a corresponding state P 2D (idx，idy)＝[x，y，θ]In association, the sampling strategy is:
x＝idx·grid+rand(-1，1)·bias
y＝idy·grid+rand(-1，1)·bias.
where (idx, idy) is the index of the state point, grid is the grid size, g is the number of discrete grids, P 2D Is a state point stored in the state point set RoadMap, and the total number of state points is g×g=n.
3. The method for planning motion trajectory of foot robot according to claim 1, wherein the foot robot can move omnidirectionally so that the state quantity [ x, y, θ ] is calculated]Separately considering the problem of constructing the connection of two state points as an optimal boundary value, the initial state is given as s i ＝[s pi ，s vi ]Is the state of the father node, and the termination state is s f ＝[s pf ，s vf ]The termination position is the position of the child node, the termination speed is obtained by solving, and the energy J (T) = [ pi ] of the whole state track is optimized 0 T s a (t) 2 dt is minimum (i.e., acceleration integral is minimum), and using the poincare gold maximum principle, the display solution for the state estimation is:
wherein a numerical solution of the problem requires a trajectory time T by giving a reference line speed v ref And angular velocity omega ref Setting a reference time t=t ref ：＝max(||[Δx，Δy]|| 2 /v ref ，Δθ/ω ref )。
4. The motion trajectory planning method of a foot robot according to claim 1, which is characterized in thatThe method is characterized in that in nonlinear dynamics track optimization, optimization indexes comprise: the smoothness cost of the state track is lambda s The cost weight of the state track from the obstacle is lambda c And the time cost weight of the track segment is lambda t Limiting robot dynamics such as maximum speed, maximum accelerationRestriction of robot omnidirectional movement ∈>Continuity of front and back states at a state pointAs a constraint on the optimization problem. The solving variables of the optimization problem are polynomial coefficients c and time T of each segment of state track:
n-th order polynomial rails where s (t) is the state variable x, y, θTrace, N is the polynomial segment number, j represents the j-th segment, R is a positive weighting matrix of state smoothness costs, measure the cost specific gravity between three state variables, T is the time vector of the trace segment, c ji An nth order coefficient vector representing a j-th stage polynomial.
5. The method for planning a motion trajectory of a legged robot according to claim 1, wherein a speed safety constraint is added to the model predictive controller in tracking a trajectory of a desired state of the model predictive control,
wherein lambda is 1 And lambda (lambda) θ The weights of the translation speed and the angular speed are measured,is a safety threshold.
And the model prediction controller calculates a control instruction of a robot joint motor according to the expected state track to realize autonomous movement of the robot.
6. A foot robot motion trajectory planning system applying the foot robot motion trajectory planning method according to any one of claims 1 to 5, comprising:
the upper layer module is used for quickly generating a rough polynomial track of robot mass heart movement on the global obstacle map, and comprises plane positions [ x, y ], direction angles theta and track time T of the robot movement;
the middle layer module is used for constructing a nonlinear optimization problem according to the initial track and the robot dynamics, and the optimization indexes comprise: obstacle avoidance cost, state track smoothness, robot dynamics limit, foot robot omnidirectional motion constraint and track time cost, wherein the optimization variables are track polynomial coefficients and time expressed in a segmented manner;
and the bottom layer module is used for taking the optimized track as a desired centroid space-time state track of the nonlinear model predictive controller and carrying out specific motion control on the robot.
7. A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the foot robot motion trajectory planning method of any one of claims 1 to 5.
8. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the foot robot motion trajectory planning method of any one of claims 1 to 5.
9. A motion trajectory planning system for a foot robot, the system comprising:
a global collision-free trajectory generation module configured to generate a rough polynomial trajectory on a global environment map, the trajectory indicating a preliminary path of travel of the robot;
the nonlinear dynamic track optimization module is used for solving nonlinear optimization problems through an optimization algorithm according to the initial path and a dynamic model of the robot to generate an optimized track which is smooth, obstacle-avoiding and accords with the omnidirectional motion constraint;
and a model predictive control expected state track tracking module generates a motion control instruction of the robot according to the optimized track to realize autonomous motion of the robot.
10. The motion trajectory planning system of a foot robot of claim 9, wherein the system comprises:
the nonlinear dynamic track optimization module considers the obstacle avoidance cost, the state track smoothness, the robot dynamic limit, the omnidirectional movement constraint and the track time cost optimization index, balances the obstacle avoidance, smooth movement and quick response requirements, and improves the movement performance of the robot;
and, the model predictive control desired state trajectory tracking module adds a speed safety constraint.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311582605.1A CN117572773A (en) | 2023-11-24 | 2023-11-24 | Foot robot motion trail planning method, system, equipment and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311582605.1A CN117572773A (en) | 2023-11-24 | 2023-11-24 | Foot robot motion trail planning method, system, equipment and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117572773A true CN117572773A (en) | 2024-02-20 |
Family
ID=89863919
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311582605.1A Pending CN117572773A (en) | 2023-11-24 | 2023-11-24 | Foot robot motion trail planning method, system, equipment and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117572773A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108089578A (en) * | 2017-12-07 | 2018-05-29 | 东莞深圳清华大学研究院创新中心 | A kind of walking movement planing method for bipod walking robot |
| CN114022824A (en) * | 2021-12-03 | 2022-02-08 | 浙江大学 | Narrow environment-oriented quadruped robot motion planning method |
| CN114442621A (en) * | 2022-01-17 | 2022-05-06 | 浙江大学 | Autonomous exploration and mapping system based on quadruped robot |
| US20230089978A1 (en) * | 2020-01-28 | 2023-03-23 | Five AI Limited | Planning in mobile robots |
| CN116185015A (en) * | 2023-01-18 | 2023-05-30 | 燕山大学 | Motion trail generation method combining long time domain and reactivity of foot robot |
-
2023
- 2023-11-24 CN CN202311582605.1A patent/CN117572773A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108089578A (en) * | 2017-12-07 | 2018-05-29 | 东莞深圳清华大学研究院创新中心 | A kind of walking movement planing method for bipod walking robot |
| US20230089978A1 (en) * | 2020-01-28 | 2023-03-23 | Five AI Limited | Planning in mobile robots |
| CN114022824A (en) * | 2021-12-03 | 2022-02-08 | 浙江大学 | Narrow environment-oriented quadruped robot motion planning method |
| CN114442621A (en) * | 2022-01-17 | 2022-05-06 | 浙江大学 | Autonomous exploration and mapping system based on quadruped robot |
| CN116185015A (en) * | 2023-01-18 | 2023-05-30 | 燕山大学 | Motion trail generation method combining long time domain and reactivity of foot robot |
Non-Patent Citations (1)
| Title |
|---|
| 陈佳: "仿生四足机器人三关节单腿轨迹研究", 中国优秀硕士学位论文全文数据库 基础科学辑, no. 03, 15 March 2023 (2023-03-15), pages 006 - 147 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111338346B (en) | Automatic driving control method and device, vehicle and storage medium | |
| Jiang et al. | Autonomous navigation system of greenhouse mobile robot based on 3D Lidar and 2D Lidar SLAM | |
| Zghair et al. | A one decade survey of autonomous mobile robot systems | |
| CN110471426B (en) | Automatic collision avoidance method for unmanned intelligent vehicle based on quantum wolf cluster algorithm | |
| CN112684807A (en) | Unmanned aerial vehicle cluster three-dimensional formation method | |
| CN114022824A (en) | Narrow environment-oriented quadruped robot motion planning method | |
| Liu et al. | Reinforcement-tracking: An effective trajectory tracking and navigation method for autonomous urban driving | |
| CN113848974A (en) | Aircraft trajectory planning method and system based on deep reinforcement learning | |
| Luo et al. | UAV path planning based on the average TD3 algorithm with prioritized experience replay | |
| Chen et al. | Deep reinforcement learning of map-based obstacle avoidance for mobile robot navigation | |
| Haddeler et al. | Explore bravely: Wheeled-legged robots traverse in unknown rough environment | |
| Liu et al. | Intelligent robot motion trajectory planning based on machine vision | |
| Quang et al. | Design a nonlinear MPC controller for autonomous mobile robot navigation system based on ROS | |
| Lu et al. | An optimal frontier enhanced “next best view” planner for autonomous exploration | |
| Fu et al. | Collision-free and kinematically feasible path planning along a reference path for autonomous vehicle | |
| Sebastian et al. | Neural network based heterogeneous sensor fusion for robot motion planning | |
| CN113296519A (en) | Mecanum wheel-based mobile robot motion planning method and system | |
| Ugur et al. | Fast and efficient terrain-aware motion planning for exploration rovers | |
| CN117572773A (en) | Foot robot motion trail planning method, system, equipment and terminal | |
| He et al. | Intelligent navigation of indoor robot based on improved DDPG algorithm | |
| Chen et al. | From topological map to local cognitive map: a new opportunity of local path planning | |
| CN113959446B (en) | Autonomous logistics transportation navigation method for robot based on neural network | |
| Lebedeva et al. | Method for distributed mapping of terrain by a heterogeneous group of robots based on google cartographer | |
| Hyla et al. | Automated guided vehicles: the survey | |
| Wang et al. | Obstacle detection and obstacle-surmounting planning for a wheel-legged robot based on Lidar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |