JP4964255B2 - Autonomous mobile robot operation planning apparatus, method, program and recording medium, and autonomous mobile robot operation control apparatus and method - Google Patents

Description

  The present invention relates to a technique for performing an operation plan of an autonomous mobile robot such as an airship or a submarine. In particular, the present invention relates to a technique for making an operation plan for an autonomous mobile robot to avoid an obstacle.

  As a technique for performing an operation plan of an under-actuated autonomous mobile robot having high inertia (inertia), a technique using an operation planning method in a Markov decision process is known (for example, see Patent Document 1 and Non-Patent Document 1). ).

In this technique, under the flow rate is assumed, each state _{s∈ {s 1, ..., s} N} autonomous mobile robot each behavior in _{a∈ {a 1, ..., a} M} each when taking the state _{s'∈ {s 1, ..., s} N} ' and reward ^R _{a ss} obtained at that _time' transition to the state transition probability ^P _{a ss} to determine first and. For the reward R ^a _{ss ′} , for example, the reward R ^a _{ss ′} given when transitioning to the state s ′ including the reaching point is 1, and the reward R ^a _{ss ′} given when transitioning to the state s ′ including the obstacle is performed. -1, and 0 'reward R ^a _ss given when transitioning to _the' state s free of obstructions.

Then, using a state transition probability ^P _{a ss 'and} reward ^R _{a ss',} based on dynamic programming in Markov decision processes, obtains the state value function V ^[pi a (s). Then, the action a that maximizes the state value function V ^π (s) is selected in consideration of the difference between the assumed flow speed and the actual flow speed, and the autonomous mobile robot is controlled according to the selected action a.

JP 2007-317165 A

H. Kawano, “Three Dimensional Obstacle Avoidance of Autonomous Blimp Flying in Unknown Disturbance”, Proceeding of 2006 IEEE / RSJ International Conference on Intelligent Robots and Systems, pp.123-130, October, 2006

  However, in the techniques described in Non-Patent Document 1 and Patent Document 1, a reward is not determined in consideration of an unexpected temporary flow that may occur in an environment in which an autonomous mobile robot acts.

  Therefore, when an unexpected temporary flow occurs in an environment in which an autonomous mobile robot acts, there is a problem that the motion plan is likely to fail.

  In order to solve the above-mentioned problem, when a plurality of different temporary flows occur, it is determined whether or not the autonomous mobile robot in each state collides with an obstacle when moving based on each action Then, a collision probability which is a sum of the probabilities of occurrence of temporary flows determined to collide is calculated, a reward is determined in consideration of the collision probability, and dynamic planning is performed based on the reward.

  Since the present invention determines a reward in consideration of an unexpected temporary flow, there is an effect that the motion plan becomes more difficult to break down.

The functional block diagram of the example of the operation | movement plan apparatus of an autonomous mobile robot. The block diagram of the example of a state transition probability calculation part. The functional block diagram of the example of the operation control apparatus of an autonomous mobile robot. It is a schematic diagram of an autonomous mobile robot, (a) is a front view, (b) is a side view. The figure for demonstrating the displacement amount of the position of a horizontal direction. The figure for demonstrating calculation of a state transition probability. The conceptual diagram for demonstrating the determination of a collision, and its omission. Continuous conceptual view of a set S _g of temporary flow collision occurs. Discrete conceptual view of a set S _g of temporary flow collision occurs. The flowchart which shows the example of the operation control method of an autonomous mobile robot. The figure for demonstrating the process of a collision determination part. The flowchart which shows the example of the operation control method of an autonomous mobile robot.

[Markov decision process]
First, an outline of a Markov decision process in reinforcement learning (Reinforcement Learning), which is basic knowledge for grasping the present invention, will be described.

_S = the set of discrete states which constitute the environmental _{{s 1, s 2, ...} , s N}, action _A = the set of entities can take action _{{a 1, a 2, ...} , a M} and To express. In a state sεS in the environment, when an action aεA is executed, the environment probabilistically changes to the state s′εS. The transition probability (also referred to as state transition probability) is P ^a _{ss ′} = Pr {s _{t + 1} = s ′ | s _t = s, a _t = a}
Is represented by At this time, the reward r is probabilistically given from the environment to the action subject, and the expected value is expressed as R ^a _{ss ′} = E {r _t | s _t = s, a _t = a, s _{t + 1} = s ′}
And

  The symbol 'attached to the state s' is a symbol for identifying the state s. The symbol 'may be used as a symbol representing time differentiation, but the meaning of the symbol' can be easily identified by whether or not the object with the symbol 'is a state of the Markov state transition model. Suppose you follow this notation.

In order to evaluate how much the action performed at a certain time step t contributed to the subsequent reward acquisition, a time series of rewards obtained thereafter is considered. The time series evaluation of reward is called value. The goal of the action subject is to maximize the value or to find a policy π (s, a) that maximizes the value. Policy π (s, a) means taking action a in state s, and is defined for each of a plurality of sets of state s and action a. The value is totaled by discounting the reward with a discount rate γ (0 ≦ γ <1) over time. That is, the state value function V ^π (s) that is the value of the state s under a certain policy π is defined as follows. E _π is a function for _obtaining an expected value.

Here, the state value function V ^π (s) that is the value of the state s under the policy π is adopted as the value function, but the behavior value function that is the value of taking the action a in the state s under the policy π. Q ^π (s, a) can also be adopted.

The goal of the action subject is to obtain an optimal policy π, that is, the value function (in the above example, the state value function V ^π (s)) for an arbitrary state s than when another policy π is adopted. It is to find a policy π that is not inferior. Quest This measure π is represented by Bellman equations long as definite values of and reward R ^a _ss' 'state transition probability P ^a _ss for each combination of _the' state s and action a destination state s For example, the optimal state value function V ^π (s), action value function Q ^π (s, a), and policy π can be calculated by dynamic programming (dynamic programming method) (for example, Sadayoshi Mikami) , Masaaki Minagawa co-translation, RSSutton, AGBarto Original work "Reinforcement Learning" Morikita Publishing, 1998, pp.94-118.) Since the dynamic programming process is a well-known technique, the description thereof is omitted.

[Operation planning apparatus and method for autonomous mobile robot]
An embodiment of an operation planning apparatus and method for an autonomous mobile robot will be described.

  An example of an autonomous mobile robot that is an action subject will be described with reference to FIGS. The autonomous mobile robot includes a main thruster 101, a vertical thruster 102, a rudder 103, a gondola 104, a flow velocity difference acquisition unit 21, and a position measurement unit 25. This autonomous mobile robot cannot move directly in the lateral direction. Since the autonomous mobile robot has a higher degree of freedom of motion than the degree of freedom of motion that can be controlled by the main thruster 101, the vertical direction thruster 102, and the rudder 103 which are the mounted actuators, this autonomous mobile robot is an underactuated robot. In this embodiment, an airship type robot is used as the autonomous mobile robot, but any autonomous mobile robot such as an underwater robot such as an underwater unmanned searcher may be used. In the autonomous mobile robot, actions that can be taken every action unit time T are determined according to the mounted actuator.

  An autonomous mobile robot navigates a space filled with a fluid having an indefinite flow velocity. The space is discretely modeled by a Markov transition state model, and is composed of four dimensions of a two-dimensional coordinate (x, y), an azimuth angle ψ, and a turning speed ψ ′ of an autonomous mobile robot. Each dimension is discretized according to the resolution of the sensor that measures the physical quantity of that dimension. In this space, the departure position and arrival position of the autonomous mobile robot are determined in advance, and obstacles are arranged.

  The autonomous mobile robot located in the state s including the departure position selects one action a from a predetermined set of actions. Then, the robot moves according to the action a for a predetermined action unit time T and moves to the transition destination state s ′. In this transition destination state s ′, again, one action a is selected from a predetermined set of actions, moved according to the action for the action unit time T, and changed to the transition destination state s ″. Moving. By repeating this action selection and state transition, the autonomous mobile robot initially in a state including the departure point tries to move to a state including a predetermined arrival point arranged in space. The operation planning apparatus for an autonomous mobile robot performs an operation plan for that purpose.

<Step S1 (FIG. 10)>
The state transition probability calculation unit 1 (FIG. 1) performs a state transition probability P ^a _ss that transitions to each state s when the autonomous mobile robot in each state s takes each action a under a predetermined flow. _' Is calculated (step S1). That is, the state transition probability P ^a _{ss ′} for each combination of the state s, the action a, and the transition destination state s _′ is calculated. The calculated state transition probability P ^a _{ss ′} is sent to the dynamic planning unit 2.

It will be described an example of how to calculate the state transition probability P ^a _ss'. In this example, the state transition probability calculation unit 1 includes a target speed calculation unit 11, a displacement amount calculation unit 12, and a probability calculation unit 13, as illustrated in FIG.

<< Step S11 >>
The target speed calculation unit 11 (FIG. 2) of the state transition probability calculation unit 1 determines a target speed when the autonomous mobile robot takes each action a in each state s (step S11). The target speed is sent to the displacement amount calculation unit 12. For example, for each action a, the turning speed ψ ^′ _τ (t) of the autonomous mobile robot and the speed v _xwτ (t) in the front-rear direction are determined as the target speed of the autonomous mobile robot according to the following formula. (B ₁ , b ₂ ) is a two-dimensional vector corresponding to the action a in each state s of the Markov state transition model, α is a predetermined turning acceleration, β is a predetermined longitudinal acceleration, τ is the elapsed time from the start of each action a, ψ ′ _τ0 is the turning speed of the autonomous mobile robot at the start of action a, and v _x0 is the longitudinal speed of the autonomous mobile robot at the start of action a.

Here, the turning acceleration α and the longitudinal acceleration β are set so as not to exceed the performance limit of the autonomous mobile robot. Further, the longitudinal velocity v _x0 (t) and the longitudinal acceleration β are respectively described as the fluid velocity and the fluid acceleration.

<< Step S12 >>
The displacement amount calculation unit 12 includes an X coordinate of a position in a horizontal plane in the world coordinate system of the autonomous mobile robot when the autonomous mobile robot in each state s moves according to each action a under a predetermined flow, How much the Y coordinate, the azimuth angle ψ, and the turning speed ψ ′ are displaced is calculated (step S12). The calculated displacement amount is sent to the probability calculation unit 13.

  The predetermined flow is, for example, an assumed flow or a flow faster than the assumed flow. As will be described later in the section [Operation control device and method for autonomous mobile robot], when the flow determined in advance is different from the actual measured flow velocity, the flow is approximated as appropriate. The predetermined flow may be set to zero. However, the closer the expected flow is to the actual measurement value of the flow velocity, the greater the accuracy of this operation plan and operation control based on it. Therefore, the accuracy of the operation plan and the operation control based on the operation plan is increased by setting the predetermined flow to an assumed flow. In addition, when calculation is performed under a flow that is faster than the flow assumed in the same direction as the direction from the departure position to the arrival position of the autonomous mobile robot, the robot that receives the flow from behind has a larger turning radius. Since the operation plan is performed under severe conditions, a safer operation plan can be performed.

The X-coordinate displacement amount of the position in the horizontal plane of the autonomous mobile robot is D _X (ψ ₀ , a), the Y-coordinate displacement amount is _DY (ψ ₀ , a), and the azimuth angle ψ displacement amount is D _ψ (ψ ₀ , A), where the displacement amount of the turning speed ψ ′ is D _{ψ ′} (ψ ₀ , a), the respective displacement amounts are given by the following equations (see FIG. 5).

Here, ψ ₀ is an azimuth angle at the start of each state s, T is an action unit time until the transition from the state s to the next state s ′, f _mx is a component of the X coordinate of a predetermined flow, f _my is a predetermined Y-coordinate component of the flow. Since the displacement amount D _ψ (ψ ₀ , a) of the azimuth angle ψ and the displacement amount D _{ψ ′} (ψ ₀ , a) of the turning speed ψ ′, the turning speed ψ ′ is controlled. No correction is made due to wind effects. The action unit time can be set to 15 seconds, for example.

<< Step S13 >>
The probability calculation unit 13 includes an X coordinate displacement amount D _X (ψ ₀ , a), a Y coordinate displacement amount D _Y (ψ ₀ , a), and an azimuth angle displacement amount D _{ψ of the} position in the horizontal plane of the autonomous mobile robot. Based on (ψ ₀ , a) and the displacement amount D _{ψ ′} (ψ ₀ , a) of the turning speed ψ ′, the state transition probability P ^a _{ss ′} is calculated (step S13).

First, it is assumed that the state s is represented by a grid composed of four dimensions of an X coordinate, a Y coordinate, an azimuth angle ψ, and a turning speed ψ ′ of the position in the horizontal plane of the autonomous mobile robot. Define (see FIG. 6). Then, the lattice R (s) is converted into displacement vector (D _X (ψ ₀ , a), D _Y (ψ ₀ , a), D _ψ (ψ ₀ , a), D) composed of the respective displacements. _{ψ ′} (ψ ₀ , a)) is defined as R _t (s).

Here, when the autonomous mobile robot is in the state s, it is assumed that the autonomous mobile robot exists with an equal probability at any point of the four-dimensional lattice R (s) representing the state s. Under this assumption, the state transition probability P ^a _{ss ′} can be obtained in proportion to the volume of the overlapping portion of R _t (s) and each R (s ′). Here, R (s ′) is a lattice overlapping with R _t (s). That is, R (s ′) is a four-dimensional lattice corresponding to the transition destination candidate state s ′ when the action a in the state s is taken. R _t (s) may overlap with up to 8 R (s ′).

The state transition probability P ^a _{ss ′} is the volume of the overlapping portion of R _t (s) and a certain R (s ′) as V ₀ (s, s ′, a), R _t (s) and all R (s If the volume of the portion overlapping with ') is Σ _s' V ₀ (s, s', a), it can be obtained by the following equation.

By appropriately repeating the processing from step S11 to step S13, the state transition probability P ^a _{ss ′} for each combination of the state s, the action a, and the transition destination state s ′ is obtained (the description of step S1 has been described above).

Next, data stored in the flow occurrence probability storage unit 3 will be described. The flow occurrence probability storage unit 3 (FIG. 1) stores a plurality of different temporary flows (f _g , ψ _g ) and the probabilities that each temporary flow (f _g , ψ _g ) occurs. . The temporary flow may be described in any of the polar coordinate system, the orthogonal coordinate system, and the oblique coordinate system, but in this example, the temporary flow is described in the polar coordinate system. f _g means the speed of the temporary flow, and ψ _g means the direction of the temporary flow. Temporary streams are identified by a set of speed f _g and direction [psi _g. A temporary flow rate _{f g} and direction _{ψ g (f g, ψ g} ) also referred to. The range of possible temporary flows is defined by the set S of f _g −ψ _g planes.

For example, the set S is defined as follows, where f _gmax is the maximum flow velocity that has occurred in the past in the behavioral environment of the autonomous mobile robot.

S = {(f _g , ψ _g ) | 0 ≦ f _g ≦ f _gmax , 0 ≦ ψ _g ≦ 360 °}
p _g (f _g , ψ _g ) is a temporary flow probability distribution, which is a discrete probability when handled by a computer, but p _g (f _g , ψ _g ) is used for conceptual explanation. Sometimes called a probability density function. When a temporary flow is considered continuously, the probability distribution of the temporary flow can be expressed by a probability density function p _g (f _g , ψ _g ). By definition, the probability density function p _g (f _g , ψ _g ) is integrated with the set S to be 1.

1 = ∫p _g (f _g , ψ _g ) df _g dψ _g
Since the temporary flow direction ψ _g that can actually be generated cannot be predicted, the probability density function _pg (f _g , ψ _g ) is assumed to follow a normal distribution only with respect to the speed f _g , for example, Is defined as σ is the variance, and μ is the average value.

It is also possible to define μ _g = 0 and define the range of the speed f _g as f _g > 0. In this _{case, 1 = ∫p g (f g} , ψ g) in order to _df g d [phi] _g, defining the probability density function _{p g (f g, ψ g} ) and as follows.

The probability density function p _g (f _g , ψ _g ) may be defined using statistical data regarding the flow velocity measured in the past in the behavior environment of the autonomous mobile robot.

The flow generation probability storage unit 3 stores data for expressing these probability density functions p _g (f _g , ψ _g ). For example, the f _g -ψ _g plane is discretely expressed by dividing it by a grid having a width of Δf _g and Δψ _g , and the value f _g and ψ _g of the representative point (for example, the center point of the grid) of each grid. Define the value of the temporary flow probability p _g (f _g , ψ _g ). At this time, the total sum of p _g (f _g , ψ _g ) is 1. That is, Σ _{S pg} (f _g , ψ _g ) = 1.

<Step S2>
The collision determination unit 4 allows the autonomous mobile robot in each state s to move to each action a under a predetermined flow (f _x , f _y ) and each temporary flow (f _g , ψ _g ). It is determined whether or not the vehicle collides with an obstacle when moving based on this (step S1). Information about the temporary flow (f _g , ψ _g ) determined to collide with the obstacle is stored in the collision flow storage unit 5. For example, as will be described later, information on the lattice set S _g of the temporary flow (f _g , ψ _g ) determined to collide with the obstacle is stored in the collision flow storage unit 5.

A specific example of the processing in step S1 will be described with reference to FIG. First, the collision determination unit 4 selects, from the set S in the f _g −ψ _g space, a lattice having a minimum value for both the speed f _g and the direction ψ _g (step S21). τ = 0 is set (step S22). The values of D _Xg (s, a, f _g , ψ _g ) and D _Yg (s, a, f _g , ψ _g ) defined by the above formula are calculated, and the values of X and Y in the state s are respectively calculated. Add (step S23). That is, the values of D _Xg (s, a, f _g , ψ _g ) + X, _{DY g} (s, a, f _g , ψ _g ) + Y are calculated. It is determined whether or not the position indicated by D _Xg (s, a, f _g , ψ _g ) + X, D _Yg (s, a, f _g , ψ _g ) + Y is an obstacle (step S24). If it is determined that the object is not an obstacle, it is determined whether τ <T (step S25). If τ <T, τ is incremented by a predetermined value (step S26), and then the process proceeds to step S23. If not tau <T indicates whether the speed _{f g} is the maximum value _{f gmax} faster _{f g,} i.e. determines whether the _f g _{<f gmax} (step S27). If f _g <f _gmax , the speed f _g is incremented (step S28), and the process proceeds to step S22. If f _g <f _gmax is not satisfied, the process proceeds to step S210.

If it is determined in step S24 that the vehicle is an obstacle, the flow (f _x , f _y ) assumed in the behavior environment of the autonomous mobile robot and the temporary flow (f of the currently selected lattice) _g , ψ _g ), when the autonomous mobile robot in the currently selected state s moves based on the currently selected action a, it is determined that it collides with an obstacle and is currently selected. The temporary flow (f _g , ψ _g ) grid is added to the temporary flow set Sg where the collision occurs (step S29). If necessary, not only the currently selected temporary flow (f _g , ψ _g ) lattice but also the currently selected temporary flow (f _g , ψ _g ) and the direction ψ _g a grid of fast transient flow rate than the same, may be added to the set S _g of temporary flow collision occurs (step S29 ').

For example, if it is determined that a collision occurs in the temporary flow at the speed f _g2 in FIG. 7, the speed is temporarily the same as the speed f _g2 and the speed f _g4 is faster than the speed f _g2. It can be seen that a collision occurs even in a simple flow without calculation. Therefore, the calculation of whether or not a collision with respect to the temporary flow of f _g4 occurs can be omitted. In this way, it is possible to reduce the amount of calculation by omitting the determination of whether or not the temporary flow collides. In the context of FIG. 7, FIG. 8 continuous conceptual view of a set S _g of temporary flow collision occurs, shows a discrete conceptual view of a set S _g of temporary flow collision occurs in Figure 9 .

In step S210, it is determined whether the direction ψ _g > 360 ° (step S210). When it is determined that the direction ψ _g > 360 ° is not satisfied, the direction ψ _g is incremented by a predetermined value, the speed f _{g is set} to 0 (step S211), and the process proceeds to step S22. If it is determined that the direction ψ _g > 360 degrees, the process ends. This process is performed for each set of state s and action a.

<Step S3>
The collision probability calculation unit 6 reads from the flow generation probability storage unit 3 and adds the probability that a temporary flow determined to collide with an obstacle is generated, so that the autonomous mobile robot in each state is based on each action. The collision probability P _c (s, a) that collides with the obstacle when moving is calculated (step S3). For example, by calculating the sum of the probabilities p _g (f _g , ψ _g ) of each of the temporary flows (f _g , ψ _g ) in which the collisions included in the set S _g occur, the collision probability P _c (s , A).

Information about temporary flow that is determined to collide with the obstacle (for example, information about the set S _g) reads from the collision flow storage unit 5. The calculated collision probability P _c (s, a) is sent to the reward determination unit 7.

<Step S4>
The reward determination unit 7 has the highest reward when the transition destination state s ′ includes the arrival position, and the collision probability P _c (s, a) is small when the transition destination state s ′ does not include the arrival position. The reward r (s, a) obtained when the autonomous mobile robot in each state s takes each action a and transitions to each state s ′ is determined so that the reward becomes higher (step S4). The determined reward r (s, a) is sent to the dynamic planning unit 2.

Let function F be a monotonically decreasing function. The monotone decreasing function means a function f that satisfies f (x ₁ ) ≧ f (x ₂ ) with respect to arbitrary x ₁ , x ₂ (where x ₁ <x ₂ ). _{P c} (s, a) P by the definition of c (s, a) from the minimum value is 0 the maximum value of is 1, the maximum value of the function F is F (0) minimum is F (1) is there. Further, an arbitrary constant R _max is set to R _max > F (0). For example, F (x) = − x and R _max = 1. At this time, to determine the compensation R ^a _ss', for example, as follows.

When transition destination state s ′ includes an arrival position => R ^a _{ss ′} = R _max
Otherwise ⇒ R ^a _{ss ′} = F (P _c (s, a))
<Step S5>
The dynamic planning unit 2 obtains, for each state, an index that represents the ease of reaching the reaching position of the autonomous mobile robot based on the dynamic programming method in the Markov decision process using the state transition probability and the reward (step S5). For example, any one of the state value function V ^π (s), the behavior value function Q ^π (s, a), and the policy π can be used as an index representing the ease of reaching. The calculated index is stored in the index storage unit 17.

As explained in the column of [Markov decision processes, state s, if it is and calculated reward R ^a _ss' 'state transition probability P ^a _ss for each combination _of' action a and destination state s Based on dynamic programming, the state value function V ^π (s), the action value function Q ^π (s, a) and the policy π can be calculated.

Thus unexpected temporary flow (f _g, ψ _{g) 'is} determined and the reward R ^a _ss' reward R ^a _ss considering the by performing dynamic programming based on, the generated Dynamic planning is less likely to fail.

  The above is the description of the embodiment of the operation planning apparatus and method for the autonomous mobile robot.

[Operation control apparatus and method for autonomous mobile robot]
Hereinafter, with reference to FIG. 3 and FIG. 12, an embodiment of an operation control apparatus and method for an autonomous mobile robot that controls the operation of the autonomous mobile robot using the generated dynamic plan (= index) will be described.

<Step A1 (FIG. 12)>
The flow velocity difference acquisition unit 21 (FIG. 3) obtains a flow velocity difference that is a difference between a predetermined flow predicted at the time of the operation plan and an actual measurement value of the flow velocity (step A1). The obtained flow velocity difference is sent to the transition destination prediction unit 22. The X component of the predetermined flow and _f x, the Y component and _{f y,} the actual flow rate of the X component _{f xa,} when the Y component is _{f ya,} flow rate difference _{d fx, d fy} are the following respectively as It is expressed in

d _fx = _f x _{-f xa}
d _fy = _f y _{-f ya}
When the predetermined flow predicted at the time of the operation plan is set to 0, the flow velocity difference acquisition unit 21 can obtain the actual measured value of the current flow velocity as the flow velocity difference.

<Step A2>
The transition destination prediction unit 22 obtains the transition destination state s ′ when the autonomous mobile robot takes each action by moving the position of the autonomous mobile robot by the flow velocity differences d _fx and d _fy (step A2). ). The obtained transition destination state s ′ is sent to the action determining unit 23.

An example of how to obtain the transition destination state s ′ will be described.
The displacement D _Xa (ψ ₀ , a) of the position of the autonomous mobile robot in the X-axis direction when considering the flow velocity difference d _fx and the position of the autonomous mobile robot in the Y-axis direction when considering the flow velocity difference d _fy The displacement amount D _Ya (ψ ₀ , a) is expressed as follows.

First, the transition destination prediction unit 22 moves the position of the autonomous mobile robot by the flow velocity differences d _fx and d _fy by the above-described equation, so that the actual displacement amount X _Xa (ψ ₀ , A) and the actual displacement amount D _Ya (ψ ₀ , a) in the Y-axis direction.

Next, the transition destination predicting unit 22 obtains the transition destination state s ′ when the action a is taken according to the following formula. Specifically, at the start of the action a, the position X (s) in the X-axis direction, the position Y (s) in the Y-axis direction, the azimuth angle ψ ₀ (s), and the turning speed ψ ′ ₀ (s) , Displacement amounts D _Xa (ψ ₀ , a) of actual positions in the X-axis direction, displacement amounts D _Ya (ψ ₀ , a) of actual positions in the Y-axis direction, and displacement amounts D _ψ (ψ _{0 of} azimuth angles) , A) and the displacement D _{ψ ′} (ψ ₀ , a) of the turning speed ψ ′ are added to obtain the transition destination state s ′.

As the position X (s) in the X-axis direction, the position Y (s), the azimuth angle ψ ₀ (s), and the turning speed ψ ′ ₀ (s) in the Y-axis direction are those measured by the position measurement unit 25. . The D _X (ψ _0, a) and _{D Y (ψ 0, a)} , D X calculated at the operation plan (ψ _0, a) and _{D Y} (ψ _0, a) may be reused. In this case, D _X (ψ ₀ , a) and D _Y (ψ ₀ , a) are stored in a storage unit (not shown), and the transition destination prediction unit 22 reads them appropriately. Of course, the transition destination prediction unit 22 may calculate these again.

<Step A3>
The action determination unit 23 compares the indicators for the transition destination state s ′ obtained by the transition destination prediction unit 22 according to each action a that can be taken in the state s, and the action that is most likely to reach the arrival point. a is determined (step A3). The determined action a is sent to the control unit 24.

In the case of using the state value function V π ^(s) as an index, since the reward as reward obtained when a transition to a state containing an arrival point is the highest is determined, the state value function V ^[pi ( The action a that maximizes the value of s) is the action that most easily reaches the reaching point.

Therefore, the action determination unit 23 refers to the index storage unit 17 and obtains the state value function V ^π (s ′) in the transition destination state s ′ when moving according to each action a that can be taken in the state s. By comparison, the action a that ^{maximizes the} value of the state value function V ^π (s ′) is determined.

<Step A4>
The control unit 24 controls the autonomous mobile robot to move according to the determined action a (step A4). Specifically, the main propulsion device 101 and the rudder 103 of the autonomous mobile robot are controlled so that the target speed corresponding to the action a can be maintained.

[Modifications, etc.]
As an index, the action value function Q ^π (s, a) or the policy π may be used as described above. For example, when the behavior value function Q ^π (s, a) is used as an index, the dynamic planning unit 2 uses the state transition probability P ^a _{ss ′} for each combination of the state s, the behavior a, and the transition destination state s _′. And the behavior value function Q ^π (s, a) is calculated based on the dynamic programming using the first reward R ^a _{ss ′} . Then, the behavior determining unit 23 (FIG. 3) compares the behavior value function Q ^π (s, a) that is an index to determine the behavior that most easily reaches the reaching point. Specifically, when the action value function Q ^π (s, a) is determined so as to represent that the reaching point is easily reached as the value increases, actions that can be taken in the state s before the transition are determined. The action a and the action that can be taken in the transition destination state s ′ are set as action a ′, and max _{a ′} Q ^π (s ′, a ′) is compared, and max _{a ′} Q ^π (s ′, a ′) is maximized. The action a to be selected is selected.

  The processing functions of the autonomous mobile robot motion planning device and the autonomous mobile robot motion control device can be realized by a computer. In this case, the processing contents of the functions that each of these apparatuses should have are described by a program. Then, by executing this program on a computer, each processing function in these devices is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. In this embodiment, these apparatuses are configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

  The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 State transition probability calculation part 11 Target speed calculation part 12 Displacement amount calculation part 13 Probability calculation part 2 Dynamic planning part 3 Occurrence probability memory | storage part 4 Collision determination part 5 Storage part 6 Collision probability calculation part 7 Compensation determination part 17 Index storage part 21 Flow velocity difference acquisition unit 22 Transition destination prediction unit 23 Action determination unit 24 Control unit 25 Position measurement unit

Claims (8)

An autonomous mobile robot that repeats the movement that moves only for the above behavior unit time based on the behavior determined in the behavior unit time in a space filled with a fluid with an indefinite flow and filled with an obstacle In the motion planning device of an autonomous mobile robot that performs motion planning to reach the arrival position from the departure position,
The space is discretely modeled by a Markov transition state model composed of four dimensions of two-dimensional coordinates (x, y), azimuth angle ψ, and turning speed ψ ′ of an autonomous mobile robot.
State transition probability calculation unit that calculates the state transition probability that is the probability of transitioning to each state when the autonomous mobile robot in each state takes each action and moves for the above behavior unit time in a predetermined flow When,
A flow occurrence probability storage unit that stores a probability of occurrence of a plurality of different temporary flows;
A collision determination that determines whether or not the autonomous mobile robot in each state collides with an obstacle when the autonomous mobile robot in each state moves under the predetermined flow and the temporary flow. And
When the autonomous mobile robot in each state moves based on each action by reading from the flow occurrence probability storage unit and adding the probability that a temporary flow determined to collide with an obstacle is generated A collision probability calculation unit for calculating a collision probability of colliding with an obstacle;
The autonomous system in each state is such that the reward when the transition destination state includes the arrival position is the highest, and the reward is higher as the collision probability is lower when the transition destination state does not include the arrival position. A reward determination unit that determines a reward obtained when the mobile robot takes each action and transitions to each state;
Based on the dynamic programming method in the Markov decision process using the state transition probability and the reward, a dynamic planning unit that obtains an index representing the reachability of the autonomous mobile robot to the reaching position for each state,
An autonomous mobile robot motion planning device. The operation planning device for an autonomous mobile robot according to claim 1,
When it is determined that the collision determination unit collides with an obstacle for a certain temporary flow, the collision determination unit temporarily collides with the obstacle even for a temporary flow having the same direction and the same speed as the temporary flow. An autonomous mobile robot motion planning device, characterized in that it is determined that the flow is slow. Based on the motion plan determined by the motion planning device of the autonomous mobile robot according to claim 1 or 2,
An autonomous mobile robot that repeats the movement that moves only for the above behavior unit time based on the behavior determined in the behavior unit time in a space filled with a fluid with an indefinite flow and filled with an obstacle In the autonomous mobile robot motion control device that controls to reach the arrival position from the departure position,
A flow velocity difference acquisition unit for obtaining a flow velocity difference which is a difference between the predetermined flow and an actual measurement value of the flow velocity;
A transition destination prediction unit for obtaining a state of a transition destination when the autonomous mobile robot takes each action by moving the position of the autonomous mobile robot by the amount of the flow velocity difference; and
An action determination unit that compares the indicators for the state of the transition destination obtained by the transition destination prediction unit with each other and determines an action that is most likely to reach the arrival position;
A control unit that controls the autonomous mobile robot so that the autonomous mobile robot moves according to the determined behavior;
Control device for autonomous mobile robot including An autonomous mobile robot that repeats the movement that moves only for the above behavior unit time based on the behavior determined in the behavior unit time in a space filled with a fluid with an indefinite flow and filled with an obstacle In the operation planning method of an autonomous mobile robot that performs an operation plan to reach the arrival position from the departure position,
The above space is discretely modeled by a Markov transition state model composed of four dimensions of the two-dimensional coordinates (x, y), azimuth angle ψ and turning speed ψ ′ of the autonomous mobile robot.
The flow occurrence probability storage unit stores the probability of occurrence of a plurality of different temporary flows from each other,
A state transition probability calculation unit calculates a state transition probability that is a probability of transitioning to each state when the autonomous mobile robot in each state takes each action and moves for the action unit time in a predetermined flow. A state transition probability calculation step to be calculated;
The collision determination unit collides with an obstacle when the autonomous mobile robot in each state moves based on each action under the predetermined flow and each temporary flow. A collision determination step for determining whether or not
The collision probability calculation unit reads from the flow generation probability storage unit and adds the probability that a temporary flow determined to collide with an obstacle is added. A collision probability calculation step for calculating the collision probability of colliding with an obstacle when moving based on
Each reward determination unit has the highest reward when the transition destination state includes the arrival position, and when the transition destination state does not include the arrival position, the reward increases as the collision probability decreases. A reward determination step for determining a reward obtained when the autonomous mobile robot in a state takes each action and transitions to each state;
The dynamic planning unit obtains, for each state, an index indicating the ease of reaching the reaching position of the autonomous mobile robot based on the dynamic programming method in the Markov decision process using the state transition probability and the reward. A dynamic planning step;
Planning method for autonomous mobile robots. The operation planning method for an autonomous mobile robot according to claim 4,
In the collision determination step, when it is determined that a certain temporary flow collides with an obstacle, a temporary flow that has the same direction and a high speed as the temporary flow also collides with the obstacle. An autonomous mobile robot motion planning method, characterized in that it is determined that the flow is slow. Based on the motion plan determined by the motion planning method of the autonomous mobile robot according to claim 4 or 5, the behavior is performed for each behavior unit time in a space filled with a fluid having an indefinite flow velocity and in which an obstacle is arranged. In an autonomous mobile robot operation control method for controlling an autonomous mobile robot that repeats the movement of the above behavior unit time based on the behavior to reach the arrival position from the departure position,
A flow rate difference acquisition unit for obtaining a flow rate difference that is a difference between the predetermined flow and an actual measurement value of the flow rate;
A transition destination prediction step in which the transition destination prediction unit obtains the state of the transition destination when the autonomous mobile robot takes each action by moving the position of the autonomous mobile robot by the amount of the flow velocity difference;
An action determination unit that compares the indicators for the state of the transition destination obtained by the transition destination prediction unit with each other and determines an action that is most likely to reach the arrival position;
A control unit that controls the autonomous mobile robot such that the autonomous mobile robot moves according to the determined behavior;
Control method of autonomous mobile robot including   An autonomous mobile robot motion planning program for causing a computer to execute the autonomous mobile robot motion planning device according to claim 1.   The computer-readable recording medium which recorded the operation plan program of the autonomous mobile robot of Claim 7.

Images