CN104063541A - Hierarchical decision making mechanism-based multirobot cooperation method - Google Patents

Abstract

The present invention provides a multi-robot collaboration method based on a hierarchical decision-making mechanism. Players judge the position of the ball to make formation selections to deal with the game; then all players vote for the best ball-handler forward ball-handler they think at this time, Then assign other roles; judge whether it is a forward ball holder, if it is a forward ball holder, walk to the ball, walk with the ball, use the ideal behavior prediction model to mathematically model the opponent's speed for the forward ball holder to walk The decision-making module for kicking the ball; if it is not the forward ball holder, after other role assignments, walk to the position point and select the formation. The present invention sequentially realizes the selection of the striker with the ball and the distribution of the roles of all other players. At the same time, the DOBMP model is established for the decision-making module of the striker with the ball. Finally, the dynamic programming algorithm is used to optimize the high-dimensional calculation amount brought by the role function. The problem is to ensure the smoothness of the role rotation based on the constantly changing football position.

Description

Multi-robot Collaboration Method Based on Hierarchical Decision-Making Mechanism

technical field

The invention relates to a multi-robot cooperation method based on a hierarchical decision-making mechanism.

Background technique

Today, the most influential FIRA (Federation of International Robot-soccer Association, International Robot-soccer Association) and RoboCup are two robot-soccer football tournaments. The biggest difference between the two is that FIRA allows a team to adopt traditional centralized The control method is equivalent to that all teammates in a team are controlled by the same brain. However, RoboCup must adopt a distributed control method, which means that each team member has its own brain and thus is an independent "subject". This requires in-depth research on MAS, allowing multiple agents to plan to complete certain target tasks in a cooperative and competitive manner, and using evolutionary algorithms and group intelligence to achieve an overall breakthrough behavioral goal.

In the RoboCup3D simulation game, in order to win a football match, it is impossible to rely on individual ability alone, and all players must cooperate and cooperate with each other, and the RoboCup3D simulation game mainly reflects how multi-agents can play in a complex and dynamic environment. Achieve efficient collaboration and tenacious confrontation. The number of players in the RoboCup3D simulation environment has changed from 6 agents in 2010 to 9 in 2011 to 11 agents so far, which puts forward higher requirements for the cooperation of multiple agents.

Regarding the cooperation mechanism of multi-robots, in recent years, both at home and abroad have begun to explore to varying degrees. For example, FC Portugal in Portugal adopts the Iterated Optimal Assignment (IOA, Iterated Optimal Assignment) method for the problem of player role assignment, which is based on the famous greedy algorithm to seek the restricted optimal value, combined with the role exchange mechanism; In football, some people proposed to integrate complex human behaviors and robot actions by establishing an imitation learning mechanism. However, due to the unknown basic framework of imitation learning, the interactive interface is also difficult to obtain; the US UT Austin Villa team applied the subtask set optimization method Complete the design of the target framework, and use the dynamic role allocation algorithm to coordinate the occupancy of the entire team; the British BoldHearts team uses the alliance algorithm to build a strong alliance team to meet the requirements of the external environment, and can optimize its action parameters according to the algorithm. Gradient-free Infotaxis strategy search algorithm, which locally maximizes the rate value of information gain; the Robocanes team in the United States uses a space-time model-based matching method to establish the relevant motion model and its internal state, while referring to the walking engine mechanism of the German B-Human team , and use the genetic algorithm and the SARSA learning algorithm to optimize the parameter configuration of different behaviors.

The above methods all require a certain optimization mechanism and learning method. For the problem of role assignment, the calculation load is large and the update speed is slow. The above problems should be considered and solved in the process of multi-robot collaboration.

Contents of the invention

The purpose of the present invention is to provide a multi-robot collaboration method based on a hierarchical decision-making mechanism, to realize the effective collaboration of the entire multi-robot team, to realize the selection of the forward ball-handler and the distribution of all other player roles in turn, and at the same time for the forward-ball-handler The dribbling decision-making module establishes the DOBMP model, and finally uses the dynamic programming algorithm to optimize the high-dimensional calculation problem caused by the role function, so as to ensure the smoothness of role rotation based on the constantly changing football position.

Technical solution of the present invention is:

A multi-robot collaboration method based on hierarchical decision-making mechanism,

Players make formation choices based on the position of the ball to deal with the game;

Then all players vote for the forward ball holder who they think is the best ball holder at this time, and then assign other roles;

Judging whether it is a striker with the ball, if it is a striker with the ball, walk to the ball, walk with the ball, use the ideal behavior prediction model to mathematically model the opponent's speed, and use it in the striker's walking and kicking decision module, that is Whether to kick the ball to the target point or walk with the ball to the target point;

If it is not the striker with the ball, after assigning other roles, walk to the position point and select the formation.

Further, using the ideal behavior prediction model to mathematically model the opponent's speed is used in the striker's ball-handler's walking and kicking decision-making module, specifically:

From the average speed of the opponent and its current position, calculate the time T it takes for the opponent to reach the ball position; at the same time, know the time it takes for our players to perform kicking actions, and set a threshold to predict whether our robot can succeed Kick the ball to the target point;

Assuming that the opponent can prevent us from kicking the ball within t time, when the T-t value is smaller, the possibility of our team successfully completing the kicking task is greater;

When the value of T-t is less than the set threshold, it is considered that the kicking task can be successfully completed, and at this time kick the ball to the target point.

Furthermore, the opponent can still prevent us from kicking the ball after making a decision, and change the established opponent's instantaneous speed table, that is, if we fail to complete the kicking task, we must set a penalty value p for the speed table:

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; err</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; rea</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; ave</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, V _err is the difference between the real speed and the average speed of the opponent, and n is the number of instantaneous speeds sampled.

Further, use the dynamic programming function optimization algorithm to reduce the amount of calculation:

First calculate the distance value of each agent to the first role position, and then use the role assignment function yr to calculate the distance value of all possible combinations of each agent to reach the first and second positions respectively, and save each pair of agents The lowest positioning cost combination to reach these two locations;

The establishment of a new location for the kth agent is based on k-1 agents arriving at {p ₁ …p _k-1 } position, that is, using the role assignment function yr to calculate each agent’s arrival at {p ₁ …p _{k -1} } the distance values of all possible combinations of positions, and save the combination of the lowest positioning cost for each pair of agents to reach {p ₁ ...p _k-1 } position;

Then assign the distance value of each agent to reach the p _kth position and calculate the lowest positioning cost combination for all agents to reach these three different positions.

Furthermore, when calculating the lowest positioning cost combination: if there is a lower positioning cost in any subset, the cost of the entire positioning method including this positioning must be lower.

Further, a voting system with different weights is used for voting.

Further, in the voting system, the allocation of communication information bytes is:

Further, for the dynamic allocation of player roles, use the role allocation function yr to achieve the best occupancy:

According to the selection method of dictionary sorting, the sum of the positions of all agents is the shortest path among all possible occupancy methods of each agent;

In the shortest path, when two players have an intersection on the path, there will be a collision, and the role assignment function yr obtains a lower cost by exchanging the target positions of the two players according to the triangle inequality.

The beneficial effects of the present invention are: the method sequentially realizes the selection of the forward ball holder CF and the distribution of other player roles under the support of the voting communication system, and synchronously updates all player roles; the DOBMP model analysis and decision-making is adopted for the CF kick judging mechanism ;Aiming at the computational complexity of character update, the use of dynamic programming function greatly reduces the computational complexity, which is of great help to the speed of character update and ensures the smoothness of character rotation based on the change of football position.

Description of drawings

Figure 1 is a schematic diagram of the optimization process of the hierarchical decision-making mechanism.

Figure 2 is a schematic diagram of formation selection.

Figure 3 is a diagram of the overall occupancy formation.

Fig. 4 is a schematic diagram illustrating the minimum cost occupancy.

Fig. 5 is a decision-making flow chart of using DOBPM to mathematically model the opponent's speed for the CF walking and kicking decision-making module.

Figure 6 shows the formation occupancy in different game modes.

Fig. 7 is a schematic diagram of a role rotation attack.

Detailed ways

Preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

Based on the RoboCup3D simulation platform, the embodiment designs a multi-robot cooperation strategy based on hierarchical decision-making to realize effective cooperation of the entire multi-robot team. The strategy mainly includes three aspects based on the role assignment function, the voting communication system and the hierarchical decision-making mechanism under the Desired of Behavior Prediction Model (DOBMP). The distribution of all player roles, and the establishment of a DOBMP model for the CF dribbling decision-making module. Finally, the dynamic programming algorithm is used to optimize the high-dimensional calculation problem caused by the role function, so as to ensure the smoothness of role rotation based on the changing football position. .

Example

In Apollo3D's strategy, Hierarchical Decision Making (HDM) is adopted, as shown in Figure 1. The so-called hierarchical decision-making is that the players first judge what formation should be used to deal with the game based on the position of the ball, and then all players Vote for the CF that you think is the best ball holder at this time, because the key to a football game is CF, whether it should pass the ball or carry the ball forward, this is the key to the overall team strategy choice. The relationship between roles and players is not always fixed throughout the decision-making process. At the moment, robot A is the most convenient to receive the ball. It may be the striker CF. At the next moment, due to the interception of the opponent, A cannot achieve his own breakthrough with the ball and pass the ball. To a teammate, after passing the ball, A will change roles according to the occupancy at the current moment. At this time, the roles and positions of other players are determined according to their own current positions. Finally, a coordination mechanism is adopted to realize the communication between all players and the synchronous update of roles, that is, to send the player's own position and ball information through the communication system. In this way, each player can know the best position of his teammates, so that the players can reach a consensus, which is more conducive to the cooperation between players. Among them, the choice of CF has to be based on: whether the player fell, whether he can see the ball, whether the ball is in front of him or behind him, the distance from the football, whether he is a goalkeeper, and whether he was CF in the last decision cycle. The weight of each case is different.

formation selection

Like a human soccer game, RoboCup3D simulation games also set corresponding game modes for different situations, such as Kick-Off, Goal_Kick, Throw_In, Corner_Kick and so on. From the perspective of a football game, the team's overall strategy can be divided into two systems: offense and defense. The player's action selection is actually based on whether the ball is controlled by us or the opponent. We enter the offensive state when we control the ball, and the opponent controls the ball. Get into a defensive state. The different formations are shown in Figure 2.

Generally speaking, the formation of the team is based on the position of the ball. As shown in Figure 3, when the ball is in the center of the court, the overall position of the team, the entire formation can be divided into two parts: attacking and guarding. Part of the character position is obtained by adding a certain offset based on the requested coordinate position, including CF, WFL, WFR, SFL, SFR, CAM and FF. The only special thing is the role of CF, which is always the player closest to the ball, and the position of the ball is set as its coordinate position. A line is formed by the center of the goal and the position of the ball. The positions of the guard players CDM, CBL and CBR are all on this line, and a certain offset can be added according to the bottom line of the court. The position of the goalkeeper GK is basically not affected by his teammates. At the center of the goal.

role assignment function y _r

After the overall formation is determined, the key is the dynamic allocation of player roles. The role allocation function y _r is used to achieve the best occupancy. When the external state information is input, the function will calculate the best match between players and roles at the current moment. Three prerequisites must be met before discussing this function:

(1) Select the closest position: each agent takes out the nearest (nearer) position from all possible occupying ways to ensure that the sum of the positions of all agents is the shortest, which is The selection needs to be sorted lexicographically.

(2) Obstacle Avoidance: Players should try to avoid collisions with other players when moving to their predetermined positions.

(3) Dynamic consistency: As long as a series of target positions are given, if y _r outputs occupancy mode m at time T, then f will output m when the player moves to the target position.

If there are n players, there will be n! placeholder. Given the state information of the outside world, especially the positions of n players and n target positions. Use n-tuples to represent the cost of a placeholder method and arrange them in descending order. In this way, n can be obtained according to the cost! A feasible placeholder, and compare these costs in lexicographic order, as shown in Figure 4 and Table 1.

The cost of various placeholders is sorted according to the dictionary:

Table 1 Placeholder cost sorting

This sorting lexicographically easily yields the minimum value required by attribute 1. If there is a collision between two players on the path, the function y _r can obtain a lower cost by exchanging their target positions according to the triangle inequality.

voting communication system

In order for all players on the team to reach their respective target positions accurately, it is necessary for all players to be able to coordinate and have no objection to the role occupation being performed. If players know exactly where the ball and their teammates are on the pitch, then there is no need for coordination between players, as each player can independently calculate the best position to use. But the problem is that the players themselves have a 120° viewing angle limitation, and the received perceptual information is mixed with noise interference, so the seen objects have errors in distance and angle, so accurate position information cannot be obtained. Fortunately, Simspark allows agents to communicate internally, that is, players can communicate with each other every other simulation cycle (40ms), but the bandwidth of this communication channel is limited, and only one player can send information at a time And the content of the message is limited to 20 bytes.

The 3D simulation environment provides a so-called sound system, so that each robot can broadcast the information it wants to 'speak' every two cycles (40ms), and other robots can receive the information as 'listening' in the next simulation cycle, but they cannot know it The received message is from that agent, so it is necessary to add the player number to the sent message. All messages sent and received by players are limited to 20 bytes of ASCII code, and some ASCII codes are not allowed to be used. In order to compress the amount of data, Apollo3D divides the stadium into 5000*5000 squares, and uses 83 characters between '*' and '~' to encode, so 8320 bits of information can be transmitted.

The specific allocation of information bytes is shown in Table 2 below, among which the noteworthy 14th-18th bytes serve as the basis for the hierarchical decision-making system used by Apollo3D. In addition, Apollo3D uses an encryption and decryption strategy for the 'talking' and receiving 'listening' information sent by each player to ensure the security of our information communication and increase certain anti-interference capabilities.

Table 2 Allocation of communication information bytes

It must be emphasized that it is unwise to only use the occupancy information received by the communication at the end, because there is noise interference during the game, when the player falls or the positioning error accumulates, or it is sent from the server Information is occasionally lost or delayed, and the information received by players is even more inaccurate. So using a voting system with different weights allows the whole team to use a unified placeholder even if there is an occasional loss of information or wrong data sent by a player.

Use a voting system with different weights, specifically, in the game, the ball carrier has the heaviest task, which can be called CF. Among them, CF is selected based on: whether the player fell, whether he can see the ball, whether the ball is in front of him or behind him, the distance from the football, whether he is a goalkeeper, and whether the player was CF in the previous decision cycle. The weight of each of the above cases is different, but represented by a probability between (0, 1).

Ideal Behavioral Prediction Model

In MAS, behavior prediction for other agents is a challenging research. In theory, a single agent can directly observe the behavior of other agents to establish a fixed behavior model, but only when there are many repetitive information interactions between agents can the model be established. In the RoboCup3D simulation game, it is impossible to predict the behavior of the opponent through simple observation, and it is difficult to have enough interactive behaviors to build a useful model during the real-time changes of the game.

Embodiments An ideal behavior prediction model DOBPM is designed to predict the optimal behavior of a single agent under given conditions. Instead of assuming what other agents will do based on theoretical analysis, DOBPM analyzes their best behavior errors to describe their expected behavior. The DOBPM model can be used to decide when to shoot, pass and the best moment to grab the ball, etc. The embodiment uses DOBPM to mathematically model the opponent's speed for the CF walking and kicking decision-making module, that is, whether to kick the ball to the target point or to walk and take the ball to the target point. The flow chart of the overall decision-making is shown in Figure 5.

During the game, first sample the walking speed value of the opponent in several cycles, and calculate its instantaneous speed V _i :

${\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\frac{\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}{\mathrm{<span\; class="notranslate">\; \&Delta;t</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Wherein (x _b , y _b ) is the position value of the opponent at the previous sampling moment, and (x _c , y _c ) is the position value of the opponent at the current moment. To get the average speed of the opponent, the harmonic mean method can be used:

${\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; ave</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; no</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; no</span>}}{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

From the opponent's average speed and its current location, the time T it takes the opponent to reach the ball position can be calculated. At the same time, knowing the time it takes our players to kick the ball, we can set a threshold to predict whether our robot can successfully kick the ball to the target point. Assuming that the opponent can prevent us from kicking the ball within t time, when the value of T-t is smaller, the possibility of our team successfully completing the kicking task is greater. When the value of T-t is less than the set threshold, it is considered that the kicking task can be successfully completed, and at this time kick the ball to the target point. If the opponent can still prevent us from kicking the ball after making a decision, it means that the prediction of the opponent's average speed is inaccurate, and the established opponent's instantaneous speed table should be changed at this time. In other words, if we fail to complete the kicking task, we must set a penalty value p for the speedometer:

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; err</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; rea</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; ave</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, V _err is the difference between the real speed and the average speed of the opponent, and n is the number of instantaneous speeds sampled. And its real speed is calculated from the time spent by the opponent walking from the initial position to the end position (the position of the ball) and the distance between the two positions.

Dynamic Programming Optimization Method Based on Hierarchical Decision

The four modules of the hierarchical decision-making mechanism are described above, based on the establishment of the voting communication mechanism and the ideal behavior model, so as to obtain the process of player role allocation and the distribution of 11 different roles of a team to 11 robots. However, the goalkeeper always plays the role of guarding the goal, CF is always the player closest to the ball, and the other nine role positions are all obtained by the dynamic programming function _yr . If the goalkeeper happens to be the player closest to the ball, that is, when GK is CF, then y _r needs to be 10! = 3,628,800 different positioning schemes, and then calculate their costs separately and select the optimal cost value according to the dictionary sorting, all these calculations must be completed within the simulation cycle of 0.02s, which requires the use of dynamic planning function (Dynamic PlanningFunction ) optimization algorithm to reduce the amount of computation.

Among them, A and P respectively represent the collection of n agents and their positions, and the positioning method m:=y _r (A, P), if there is a lower positioning cost in any subset, then the entire positioning method including the positioning The price must be lower. Establishing a new location for the kth agent is based on k-1 agents reaching {p ₁ ...p _k-1 } positions. For example, the dynamic programming process of three robots, as shown in Table 3, first calculates the distance value of the three agents to the first role position, and then uses the role assignment function yr to calculate the three agents to reach the first and second different positions respectively The distance values of all possible combinations of , and save the combination of the lowest positioning cost for each pair of agents to reach these two positions. Then assign the distance value for each agent to reach the third location and calculate the lowest combination of positioning costs for all agents to reach these three different locations.

Table 3 Occupancy allocation scheme of three robots

N agents undergo n dynamic programming iterative calculations, and each time is equivalent to the calculation of the binomial with the highest degree n-1:

$\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; k</span>}\end{array}\right){\mathrm{<span\; class="notranslate">\; 2</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Then, when 11 agents participate in the game, except for the goalkeeper, a total of 10 agents participate in the role assignment. After using the dynamic programming optimization algorithm, the calculation amount is n2 ^n-1 = 10×2 ⁹ = 5120, but the calculation of the unoptimized role assignment algorithm is Quantity is 10! = 3,628,800, which significantly reduces the amount of calculation, and also reduces the time spent on role switching.

Experimental verification

All experiments use the DrawAnnotation function in Roboviz to display the character names of our players at the current moment above their heads. The meaning of each role is explained in Figure 5. Our Apollo3D is a blue robot, and the red robot is opponent.

Experiment 1: Formation occupancy in different game modes

This experiment is mainly aimed at the formation occupancy in different game modes in the RoboCup3D simulation game, as shown in Figure 6. Figure 6 is the formation occupancy in different game modes. Bit map; (b) our left corner corner map; (c) our goal ball map; (d) our penalty area corner map. CF, WFL, WFR, SFL, SFR, CAM, and FF are the offensive roles in the entire team. Among them, WFL, WFR, SFL, SFR, and CAM follow CF to form a rectangle, standing at the four corners of the rectangle and At the center, and face the ball as far as possible, so that each player is closest to the ball position while ensuring the formation. FF always stands in front of the opponent's goal penalty area. After many experiments, it shows that: CF's shot may be intercepted by the opponent or the shooting angle is deviated. This is because FF can occupy a favorable position as soon as possible. Switching to the next moment of CF's remedial shot is very effective. Good; the role of CDM, CBL, CBR and GK is to undertake the defensive task of their own half. If our team is in an offensive state, CDM will occupy the midfield position. This is to prevent the opponent from counterattacking or responding to our players to form our counterattack.

Experiment 2: Role rotation and DOBMP model validation

Group picture 7 describes the position and role switching of the offensive part. Player No. 2 in picture a is the forward CF. Because he was blocked by the opponent and fell down, the role of No. 2 quickly switched to CAM. At this time, player No. 7 faced the ball And the distance from the ball is the closest, and its role is quickly switched to CF, as shown in Figures b and c; when player No. 7 is also intercepted by the opponent, the DOBMP model is used to judge that the ball is kicked to the target point, which is the position of player No. 3, and No. 3 rotates at the same time As the CF role, No. 2 and No. 7 are rotated into SFL and CAM at the same time, as shown in Figure d; due to the Simspark game platform regulations: when there are more than 2 players from the same side within a radius of 1 meter with the ball as the center, all players will be automatically ejected. The player who is far away from the ball, so when the player No. 2 is close to the ball, he is automatically bounced off by the platform. At the same time, No. 5 is rotated to CF, and No. 3 is rotated to SFL, as shown in Figure e.

The layered decision-making mechanism based on role distribution is to realize the selection of striker CF and the distribution of other player roles in sequence with the support of the voting communication system, and simultaneously update all player roles; the DOBMP model is used to analyze the CF kicking judgment mechanism Decision-making: Aiming at the computational complexity of character update, the dynamic programming function is used to greatly reduce the computational complexity, which is of great help to the speed of character update and ensures the smoothness of character rotation based on the change of football position.

Claims (8)

1. A multi-robot collaboration method based on hierarchical decision-making mechanism, characterized in that: Players make formation choices based on the position of the ball to deal with the game; Then all players vote for the forward ball holder who they think is the best ball holder at this time, and then assign other roles; Judging whether it is a striker with the ball, if it is a striker with the ball, walk to the ball, walk with the ball, use the ideal behavior prediction model to mathematically model the opponent's speed, and use it in the striker's walking and kicking decision module, that is Whether to kick the ball to the target point or walk with the ball to the target point; If it is not the striker with the ball, after assigning other roles, walk to the position point and select the formation. 2. the multi-robot collaboration method based on hierarchical decision-making mechanism as claimed in claim 1, is characterized in that, uses ideal behavior prediction model to carry out mathematical modeling to opponent's speed and is used for forward ball holder walking and kicking decision-making module, specifically : From the average speed of the opponent and its current position, calculate the time T it takes for the opponent to reach the ball position; at the same time, know the time it takes for our players to perform kicking actions, and set a threshold to predict whether our robot can succeed Kick the ball to the target point; Assuming that the opponent can prevent us from kicking the ball within t time, when the T-t value is smaller, the possibility of our team successfully completing the kicking task is greater; When the value of T-t is less than the set threshold, it is considered that the kicking task can be successfully completed, and at this time kick the ball to the target point. 3. The multi-robot collaboration method based on hierarchical decision-making mechanism as claimed in claim 2, characterized in that: after making a decision, the opponent can still stop us from kicking the ball, and change the opponent's instantaneous speed table that is established, that is, If we fail to complete the kicking task, we must set a penalty value p for the speedometer: $\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; err</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; rea</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; ave</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$ Among them, V _err is the difference between the real speed and the average speed of the opponent, and n is the number of instantaneous speeds sampled. 4. The multi-robot collaboration method based on hierarchical decision-making mechanism according to any one of claims 1-3, wherein the dynamic programming function optimization algorithm is used to reduce the amount of calculation: First calculate the distance value of each agent to reach the first role position, and then use the role assignment function yr to calculate the distance value of all possible combinations of each agent to reach the first and second positions respectively, and save each pair of agents The lowest positioning cost combination to reach these two locations; The establishment of a new location for the kth agent is based on k-1 agents arriving at {p ₁ …p _k-1 } position, that is, using the role assignment function yr to calculate each agent’s arrival at {p ₁ …p _{k -1} } the distance values of all possible combinations of positions, and save the combination of the lowest positioning cost for each pair of agents to reach {p ₁ ...p _k-1 } position; Then assign the distance value of each agent to reach the p _kth position and calculate the lowest positioning cost combination for all agents to reach these three different positions. 5. The multi-robot collaboration method based on hierarchical decision-making mechanism as claimed in claim 4, wherein when calculating the minimum positioning cost combination: if there is a lower positioning cost in any subset, then the entire positioning method including the positioning The cost is necessarily lower. 6. The multi-robot collaboration method based on a hierarchical decision-making mechanism as claimed in claim 5, wherein a voting system with different weights is used to vote. 7. the multi-robot collaboration method based on hierarchical decision-making mechanism as claimed in claim 6, is characterized in that, in the voting system, the allocation situation of communication information byte is: 8. the multi-robot collaboration method based on hierarchical decision-making mechanism as claimed in claim 7, is characterized in that, the dynamic allocation of player's role, the role assignment function yr of use is to realize optimal occupancy: According to the selection method of dictionary sorting, the sum of the positions of all agents is the shortest path among all possible occupancy methods of each agent; In the shortest path, when two players have an intersection on the path, there will be a collision, and the role assignment function yr obtains a lower cost by exchanging the target positions of the two players according to the triangle inequality.