JP4269055B2 - Obstacle avoidance device and moving body - Google Patents

Description

  The present invention relates to an obstacle avoidance device and a mobile object, and more particularly to an obstacle avoidance device provided in a mobile object such as an electric wheelchair and a mobile object provided with the obstacle avoidance device.

  As an aging society progresses, there are diverse needs for mobility aids, from those who feel a little inconvenient to those who are bedridden.

  Many of them ask for resistance to the image of a conventional wheelchair and demand for a small turn in a small place. For example, Patent Document 1 discloses a wheelchair that is easy to turn in a small place. Non-patent documents 1 to 3 disclose a wheelchair including an omnidirectional movement mechanism and an omnidirectional movement mechanism.

  In addition, in a moving vehicle such as a wheelchair, a configuration in which an obstacle avoidance assist device (obstacle avoidance device) for enabling safe driving and facilitating operation is known.

  For example, Patent Document 2 discloses a mobile robot that avoids obstacles, and Patent Document 3 discloses a wheelchair that avoids obstacles.

Moreover, when avoiding an obstacle, the structure which controls operation | movement supposing a repulsive force is disclosed by patent document 4. FIG. The repulsive force is described in Non-Patent Document 4.
JP 2000-116720 A (publication date: April 25, 2000) Japanese Patent Laid-Open No. 7-110711 (Publication date: April 25, 1995) JP 2000-210340 A (publication date: August 2, 2000) JP 2002-225741 A (publication date: August 14, 2002) H. Asama, M. Sato, L. Bogoni et al .: Development of an omnidirectional Mobile robot with 3 DOF decoupling drive mechanism, Proc. Of IEEE 1995 Int. Conf. On Robotics and Automation, pp.1925-1930 (1995) S. Hirose and S. Asano; The VUTON: High Payload High efficiency holonomic omni-directional vehicle, 6th Int. Symp. On Robotics Research, October (1993) M. Wada and HHAsada: A holonomic omnidirectional wheelchair with a variable footprint mechanism, Proc.of the 3rd Int. Conf.on Advanced Mechatronics Okayama, Japan, pp.268-274, (1998) Jun Ohta, Daisuke Kurabayashi, Tamio Arai: Introduction to Intelligent Robots-Solving Motion Planning Problems, Corona (2001)

  However, the obstacle avoidance device having the conventional configuration does not respond according to each user and each surrounding situation, so that there is a problem that a sufficiently fine response cannot be performed.

For example, the configuration described in Patent Document 4 controls the operation assuming a repulsive force when avoiding an obstacle. However, since the operation state differs for each user and changes depending on the surrounding situation, when a predetermined repulsive force is fixed and used as in the configuration of Patent Document 4, depending on the user, the repulsive force However, there are cases where it can get in the way. For this reason, there is a case where a sufficiently fine response cannot be made and the obstacle avoidance device is obstructed.

  The present invention has been made in view of the above-described problems, and an object of the present invention is an obstacle avoidance device that corrects an instruction speed from a user in order to avoid an obstacle. It is an object to provide an obstacle avoidance device and a moving body that can perform the above.

  In order to solve the above-described problem, the obstacle avoidance device according to the present invention provides an object from an obstacle sensor that detects an obstacle around the moving body relative to the moving body that moves based on an instruction speed from the user. An obstacle avoidance device that avoids a collision of the moving object with an obstacle by correcting the indicated speed according to information, and adds / subtracts to / from the indicated speed after multiplying a set weight coefficient. A repulsive force generation unit for calculating a repulsive force for avoiding a collision with the obstacle, which is used to correct the indicated speed according to the information from the obstacle sensor, and The magnitude of the weight coefficient in the repulsive force generation unit is learned by reinforcement learning according to the coincidence or mismatch of the direction of the command speed corrected by the repulsive force and the directed speed, and the repulsion It is characterized by having a learning section that sets the generated section.

  In the obstacle avoidance device configured as described above, the repulsive force generation unit calculates the repulsive force so as to avoid the obstacle detected by the obstacle sensor. For example, obstacle avoidance is realized by adding this repulsive force to the speed. That is, if the vehicle drive device and the power unit are adjusted so as to obtain the obtained speed, the force in the direction away from the obstacle becomes stronger when the vehicle approaches the obstacle, and the obstacle can be avoided.

  Here, the obstacle avoidance device having the above configuration corrects the speed by learning the operation of the operation unit and adjusting the magnitude and direction of the repulsive force. This is because the strength of operation on the operation unit differs for each user, but if the repulsive force generated by the repulsive force generating unit is uniform, the repulsive force is disturbed by some users, and a desired operation cannot be realized. This is because there is a fear.

  For this reason, the obstacle avoidance device adjusts the strength of the repulsive force by learning the user's operation. More specifically, the learning unit learns the magnitude of the weighting factor by reinforcement learning according to the coincidence or mismatch of the direction of the instruction speed from the user with the repulsive force and the direction of the instruction speed, and the repulsive force Set for the generator. Here, since the direction is not determined when the magnitude of the speed is 0, it is assumed that the directions coincide when the magnitudes of the corrected speed and the indicated speed are both 0, for example, one of the other speeds. If the size of is not 0 but the other is not 0, the directions are assumed to be inconsistent.

  Therefore, when the user operates to move the moving body, the obstacle avoidance device learns that the magnitude of the repulsive force is appropriate. Here, since the learning unit uses reinforcement learning which is unsupervised learning, it can learn even when there is no correct answer, such as obstacle avoidance for each user.

  Therefore, it is possible to respond sufficiently finely by responding to each user and each surrounding situation.

In the obstacle avoidance device according to the present invention, in the configuration described above, the learning unit determines a positive reward when the direction of the corrected speed matches the direction of the indicated speed, and the corrected speed and the indicated speed are If the directions are inconsistent with each other, reinforcement learning is performed by using a negative reward.

  As described above, since the positive reward is obtained when the corrected speed matches the instruction speed, learning can be performed so that the corrected speed matches the instruction speed. Therefore, it is possible to make a desired progress according to the user's wish as much as possible while performing obstacle avoidance.

  In the obstacle avoidance device according to the present invention, in the above configuration, the operation unit for detecting the instruction speed is a joystick type operation unit, and the learning unit has the corrected speed of 0 and the When the moving body is stopped, if the joystick is held at the center position, a positive reward is given, and the corrected speed is not 0 and the moving body is moving and the joystick is tilted. If the direction is not consistent, or if the corrected speed is 0 and the joystick is tilted when the moving body is stopped, a negative reward is provided. Yes.

  When the corrected speed is 0 and the moving body is stopped, if the joystick is held at the center position, the reward is a positive reward, so that learning can be performed immediately at a desired timing. This enables safe operation of the moving body.

  Also, if the corrected speed is 0 and the joystick is tilted when the moving body is stopped, it will be a negative reward, so it will learn to start immediately without making an undesired stop be able to.

  In order to solve the above-described problem, a mobile object according to the present invention includes the obstacle avoidance device described above.

  Since the obstacle avoidance device described above is provided, the above-described effects can be obtained in the moving body.

  In order to solve the above-described problem, the obstacle avoidance device according to the present invention provides a weighting coefficient in the repulsive force generation unit according to the coincidence or inconsistency between the direction in which the indicated speed is corrected by the repulsive force and the indicated speed. Is learned by reinforcement learning, and has a learning unit that is set for the repulsive force generation unit.

  In this way, “the speed at which the indicated speed is corrected by the repulsive force”, which is the actual speed of the moving body, is compared with the indicated speed, and reinforcement learning is performed according to the coincidence or mismatch of directions. Therefore, for example, learning can be performed so that the direction of the speed in which the user instruction is corrected matches the direction of the user instruction speed.

  For this reason, the magnitude | size of a repulsive force can be made appropriate by learning the magnitude | size of a weighting coefficient according to operation of the moving body by a user. Therefore, a fine response can be made for each user. Also, the obstacle avoidance device will not get in the way.

  An embodiment of the present invention will be described below with reference to the drawings.

An electric wheelchair (moving body) 1 according to an embodiment of the present invention is configured as shown in a schematic perspective view of FIG. The electric wheelchair 1 is a compact omnidirectional mobile wheelchair that looks like an office chair. The electric wheelchair 1 can control the moving speed (traveling speed) using the operation unit 2. Moreover, as shown in FIG. 3, the electric wheelchair 1 can be moved in all directions by the rotation of the right wheel 7a, the left wheel 7b, and the rotating shaft 7c of the power unit. For this reason, the electric wheelchair 1 has a configuration that is very easy to turn.

  As shown in FIG. 1, the electric wheelchair 1 includes an operation unit 2, an avoidance device (obstacle avoidance assist device, obstacle avoidance device) 3, an objective sensor (obstacle sensor) 4, a speed sensor 5, a drive device 6, and power. Part 7 is provided.

  The operation unit 2 of this embodiment is a joystick type user interface. The user of the electric wheelchair 1 uses the operation unit 2 to specify the moving direction and moving speed (instruction speed) W of the electric wheelchair 1.

  More specifically, the operation unit 2 is provided with a knob (not shown) in addition to the joystick. The movement speed in the XY directions on the plane can be specified using the joystick, and the rotational movement direction and the rotational movement speed can be specified using the knob.

  The avoidance device 3 changes the speed U of the electric wheelchair 1 so as to avoid obstacles around the electric wheelchair 1. More specifically, an obstacle is detected by the objective sensor 4, and a virtual repulsive force F corresponding to the distance from the obstacle is calculated by the repulsive force generator 11 of the avoidance device 3. By adding this virtual repulsive force F to the moving speed, the speed U is obtained and obstacle avoidance is realized.

  The avoidance device 3 of the present embodiment includes a learning unit 12 that learns user operations. Using the learning unit 12 of the avoidance device 3, the user-specified moving speed W is corrected to an appropriate value and output. More specifically, the learning unit 12 of the present embodiment corrects the virtual repulsive force F to an appropriate value. This is because, for example, the virtual repulsive force F used in the avoidance device 3 may be disturbed by some users.

  The learning unit 12 of the present embodiment learns the user's operation by reinforcement learning. Since reinforcement learning is unsupervised learning, it can be applied even when there is no correct answer, such as user operation. For reinforcement learning, see Richard S. Sutton, Andrew G. Barto: Reinforcement Learning, MIT-Press (1999).

  The objective sensor 4 detects surrounding obstacles. The objective sensor 4 can be used to measure the distance and direction from surrounding obstacles. The electric wheelchair 1 is provided with a plurality of objective sensors, which are collectively shown as an objective sensor 4 in FIG. The speed sensor 5 measures the moving speed of the electric wheelchair 1. The objective sensor 4 and the speed sensor 5 can be used to detect the distance and relative speed between the electric wheelchair 1 as the moving body and the obstacle.

  The driving device 6 generates a driving force for adjusting to a user-specified speed. The power unit 7 moves the electric wheelchair 1 according to the driving force. The power unit 7 of the present embodiment includes a right wheel 7a, a left wheel 7b, and the like.

  The electric wheelchair 1 having the above-described configuration operates as follows according to the user's operation.

  The controller 8 of the avoidance device 3 receives the user-specified moving speed W from the operation unit 2 and the current speed Y of the electric wheelchair 1 from the speed sensor 5. The speed sensor 5 determines the speed of the electric wheelchair 1 based on the operation of the power unit 7 of the electric wheelchair 1. Based on these inputs, the generator 9 of the controller 8 outputs a target speed (target speed difference) Um using the gain coefficient Kp.

On the other hand, the learning unit 12 of the repulsive force generator 11 includes a storage unit 13 and an extraction unit 14.
The storage unit 13 is a storage area for storing the previous movement speed W specified by the user and the output from the objective sensor 4. The extraction unit 14 obtains a restitution coefficient η for correction output by reinforcement learning. The repulsive force generation unit 15 calculates and outputs the virtual repulsive force F using the repulsion coefficient η from the extraction unit 14, the speeds W and Y, and the output from the objective sensor 4.

  Here, operation | movement of the extraction part 14 which performs reinforcement learning is demonstrated. The extraction unit 14 compares the previous user-specified moving speed W stored in the storage unit 13 with the current speed Y, and counts a positive reward when the directions of W and Y are the same, When the directions of W and Y are opposite, a negative reward is counted. At this time, learning is performed in consideration of the input from the objective sensor 4. An appropriate joystick operation corresponding to the virtual repulsive force F is performed, and a positive reward is given when moving in a desired direction. Due to the virtual repulsive force F, a negative reward is given when moving in a direction different from the user's intention. For this reason, only by the user operating the joystick of the operation unit 2, an appropriate restitution coefficient η is extracted by the extraction unit 14.

  As a more specific example of the reward, for example, when the user holds the joystick in the center position (W = 0) and the electric wheelchair 1 stops (Y = 0), 1 is given as a positive reward, and the user When the electric wheelchair 1 moves in the opposite direction to the joystick when the joystick is pushed down or stops, -1 may be given as a negative reward.

  As described above, when an appropriate repulsion coefficient η is extracted, the repulsive force generation unit 15 calculates the virtual repulsive force F based on this. When the virtual repulsive force F is obtained, the adding unit 10 of the avoidance device 3 adds the target speed Um and the virtual repulsive force F to obtain the speed U.

  Here, the electric wheelchair 1 can change independently the direction of the right wheel 7a of the power part 7, the left wheel 7b, and a seat. The obtained speed U is converted into the driving speed ω of the right wheel 7a, the left wheel 7b, etc. of the electric wheelchair 1 by the converter 16 of the drive device 6. The driving force for realizing the speed ω is calculated by the calculation unit 17 and an instruction is output to the power unit 7.

  Further details of the electric wheelchair 1 will be described in the following examples.

  As described above, the electric wheelchair 1 according to the present embodiment estimates the distance from the obstacle necessary for avoiding the collision with the obstacle from the operation state of the user's joystick, and maintains the distance. An avoidance device 3 for assisting in avoiding an obstacle is provided.

  The electric wheelchair 1 is an electric wheelchair equipped with an objective sensor 4 that measures the distance from the user and an obstacle, and includes a (virtual) repulsive force generator 11 and a learning unit 12 as a repulsive force adjuster. . As described above, the repulsive force generation unit 15 calculates a virtual repulsive force vector from the obstacle according to the distance between the obstacle detected by the objective sensor 4 and the electric wheelchair 1, and is based on the joystick operation of the wheelchair user. The electric wheelchair 1 is caused to travel based on the combination with the vector. Then, whether or not the magnitude of the repulsive force calculated by the repulsive force generation unit 15 is appropriate is determined by comparison with the user's maneuver, and is adjusted by the learning device 12 using reinforcement learning.

  The learning device 12 compares the way of operating the joystick of the operation unit 2 (determined as the intention of the operator) with the actual running state. If the direction of operation is the same as the direction of movement, a positive reward is given. If there is any other difference, a negative reward is given. This takes into account the case where the virtual repulsive force is in the direction opposite to the intention and is disturbed, for example, when the joystick is moved in the direction opposite to the direction in which the joystick is tilted. Reinforcement learning is used to determine the gain of virtual repulsion. As a result, the gain is optimized simply by moving the joystick and performing trial and error. For this reason, if the vehicle is operated normally, the virtual repulsive force is learned so as not to disturb the operation, and a comfortable operation is possible.

  The electric wheelchair 1 can also be operated with the magnitude of the coefficient of the virtual repulsive force being kept constant by stopping learning of the virtual repulsive force by selecting a switch (not shown). Furthermore, the operation can be performed only by the user's operation without using the virtual repulsive force itself by using another switch.

  Here, in order to run the electric wheelchair 1 while avoiding the obstacle, it is necessary to keep the distance from the obstacle at least a certain level. This distance is a distance that is less likely to collide with an obstacle and can travel without significantly deviating from the travel route. For this reason, this distance differs depending on the driving environment and the psychological and ability state of the user. Therefore, as described above, the electric wheelchair 1 that can be in an optimum state according to the traveling environment and the user is provided. According to the electric wheelchair 1, the distance from the obstacle can be kept at least a certain distance.

  Note that in each part and each processing step of the avoidance device 3 of the above-described embodiment, an arithmetic unit such as a CPU executes a program stored in a storage unit such as a ROM (Read Only Memory) or a RAM, and an input unit such as a keyboard. It can be realized by controlling output means such as a display or communication means such as an interface circuit. Therefore, various functions and various processes of the avoidance device of the present embodiment can be realized simply by a computer having these means reading the recording medium storing the program and executing the program. In addition, by recording the program on a removable recording medium, the various functions and various processes described above can be realized on an arbitrary computer.

  Further, the above-described electric wheelchair 1 can be realized by connecting the operation unit 2, the objective sensor 4, the speed sensor 5, the driving device 6, the power unit 7, and the like shown in FIG.

  As a recording medium on which the program is recorded, a program medium such as a memory (not shown) such as a ROM may be used for processing by the microcomputer, and an external storage device is not shown. It may be a program medium provided with a program reading device and readable by inserting a recording medium therein.

The program medium is a recording medium configured to be separable from the main body, such as a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a flexible disk or a hard disk, or a disk such as a CD / MO / MD / DVD. Fixed disk, IC card (including memory card), etc., or semiconductor ROM such as mask ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash ROM, etc. In particular, there are recording media that carry programs.

  Moreover, in the above-mentioned embodiment, the structure which adjusts a repulsive force and avoids an obstacle was demonstrated. However, the configuration for avoiding the obstacle is not limited to this. For example, in the configuration of the electric wheelchair 1 described above, the avoidance device 3 may be configured to adjust the speed Um obtained from the user's instruction speed W input to the operation unit 2 without adjusting the repulsive force. In this case, the learning unit 12 including the storage unit 13 and the extraction unit 14 is provided not on the repulsive force generator 11 side but on the controller 8 side. Even in this configuration, the speed U = Um + F is adjusted via the adder 10, and obstacle avoidance can be realized. In addition, appropriate adjustment by learning is possible.

[Example 1]
A detailed embodiment of the above-described electric wheelchair 1 will be described. The electric wheelchair 1 of this embodiment is
As described above, it has a non-holonomic mechanism that rotates a drive unit having two drive wheels and a seat surface portion using a motor. For this reason, movement in all directions is possible.

As shown in FIG. 3, the electric wheelchair 1 has an angular velocity of the seat ω _s , an angular velocity of the right wheel ω _r , and a left wheel rotational velocity (Angular velocity of the seat). left wheel) ω _l , Radius of the right wheel R _r , Radius of the left wheel R _l , Tread of the vehicle T, Angular velocity of It includes parameters such as the gravity center) ω and translational velocity (Velocity of the gravity center) ν. In addition, the angular velocity of the left and right wheels and the seating surface is represented as ω _{1 / r / s} , and the angular velocity of the motors of the left and right wheels and the seating surface is represented as ω _{Ml / r / s} .

Moreover, the example of the specific value in the electric wheelchair 1 of a present Example is as follows. The mass of the wheelchair M is 81.2 [kg]. The inertia of the wheelchair drive part (Inertia of the drive part of the wheelchair) J is 1.98 [kg m ² ]. Inertia of the left / right wheel axis J _{l / r} is 4.38 × 10 ⁻⁴ [kg m ² ]. Inertia of bearing surface axis
of the seat axis) J _s is 11.0 [kg m ² ]. Inertia of the left / right / seat motor axis J _{Ml / r / s} is 2.16 × 10 ⁻⁵ [kg m ² ]. Friction coefficient of the left and right wheels and the seat (Friction constant of the left / right wheel and seat) δ l / r / s is ^{0.02 [kg m 2 / sec]} . Gear ratio of the left / right wheel and seat γl _{/ r / s} is 70. The left and right wheel distance T is 0.6 [m]. The radius R _{l / r} of the left and right wheels is 0.05 [m].

FIG. 4 shows a modification of the configuration of the drive unit 6 in the present embodiment. When a trajectory (desired trajectory) is input to the trajectory tracker 20, the trajectory tracker (trajectory tracker) 20 calculates a target velocity and inputs it to an inverse transformer (Inverse PWS kinematics) 21. The reverse converter 21 converts the speed into an operation speed for the electric wheelchair 1. A motion control unit (PWS controller) 22 calculates an external force necessary to reach the speed, and a dynamics of electric wheelchair 23 calculates a speed obtained according to the external force. The obtained speed is output to the motion control unit 22 for adjustment. A converter (PWS kinematics) 24 converts the speed for operation into the speed of the electric wheelchair 1 and outputs it to the power instruction unit (Wheelchair kinematics) 25.

  FIG. 5 shows another outline of a modification. A desired speed is input by a human controller 26. The repulsive force calculated by the adjustment unit (RL Tuner) 29 and calculated by the avoidance unit (Object Avoider) 28 is added to the speed and input to the drive unit (System) 27. In addition, the user inputs a desired speed using the operation switch 26 according to the current speed.

More specifically, the joystick input from the user causes the x-direction axial velocity ν _x ^ref (t),
A y-direction axial velocity ν _y ^ref (t) and a rotational angular velocity ω _u ^ref (t) are given.

The trajectory tracker 20 calculates the target translation speed ν ^ref (t) and the target vehicle body rotation speed ω ^ref (t) by the following equations.

However,

And a is a positive constant. The conversion in the inverse converter 21 and the converter 24 shown in FIG. 4 is as follows.

  As described above, the control target value of each drive shaft is determined based on the inverse transformation of the above equation from the joystick input. Further, based on the conversion of the above equation, the actual translation speed ν (t) and the vehicle body rotation speed ω (t) are obtained from the control output of each drive shaft, and input to the obstacle avoidance device together with the sensor information.

  An example of the arrangement of the objective sensors in this embodiment is shown in FIG. Thus, with respect to the electric wheelchair (moving body) 30 as an embodiment provided with the joystick 31, the objective sensor (obstacle sensor) 32 is mounted around the vehicle body cover. The objective sensor 32 is an ultrasonic sensor or an infrared sensor and functions as an obstacle distance measuring sensor.

  In this embodiment, the avoidance assist operation is realized by an algorithm mounted on a built-in microcomputer.

  The equation of motion of the electric wheelchair 1 in the present embodiment can be described as follows using the parameters shown in FIG.

However, A to L are the following coefficients, respectively.

  From the above equation of motion, it can be seen that the left and right wheels and the seat surface rotation interfere with each other. This effect is removed by using a PI controller using a feedforward compensator of the following equation. For PI controllers, see S. Iida and S. Yuta: Control of vehicle with power wheeled steerings using feedforward dynamics compensation, Proc. Of 1991 IEEE Int. Conf. On Industrial Electronics, pp. 2264-2269 (1991) .

However, K _PFF and K _IFF are feedback gains.

FIGS. 6A to 6C below show simulation results when only the PI controller is used. Moreover, the following FIGS. 7A to 7C show simulation results when a feedforward compensator is used. FIGS. 6A and 7A are for the translational velocity ν, and FIGS. 6B and 7B are for the angular velocity ω, and FIGS. c) is for the angular velocity ω _u . In each figure, a broken line indicates an actual simulation result, and a solid line indicates a time change that is a simulation target for reference. As can be seen from the figure, interference can be removed by the feedforward compensator, and control performance can be improved.

  Next, a specific method for calculating the virtual repulsive force and a method for calculating the virtual repulsive force in the present embodiment will be described below. First, the repulsive potential from the obstacle is calculated based on the following equation.

Where x is a vector representing the current position of the wheelchair, and η is a positive weighting factor. ρ (x) represents the closest distance from x to the obstacle, and ρ ₀ is a positive constant. The repulsive potential value U (x) takes a value of 0 or more. The value of U (x) increases as the wheelchair approaches the obstacle area and approaches infinity. When the distance from the wheelchair to the obstacle is ρ ₀ or more, the value of U (x) is 0. At this time, the virtual repulsive force F _OAS the potential function U acting on the wheelchair (x), F _OAS
= -ΔU

In this modification, a vector U _man based on the input from the joystick of the user synthesizes the virtual repulsive force vector F _OAS, the manipulated variable u = U _man + F _OAS of the electric wheelchair 1.

The adjusting unit 29 as a repulsive force adjuster adjusts the weighting coefficient η in the equation (28), so that the avoiding unit 28 as a virtual repulsive force generator has a repulsion having a size suitable for the environment and the state of the user. Let the force be calculated. Reinforcement learning is used for parameter adjustment. Sarsa (λ), which will be described later, was used for the processing procedure in the reinforcement learning device, and tile coding by CMAC was introduced for the generation of the state space. For tile coding by CMAC, JSAlbus: Data storage in the control cerebellar model articulation control 1 (CMAC), Trans.on ASME, J.of Dynamic Systems, Measurement, and Control, 97-9, 228/233 (1975) Please refer to.

  The state of the reinforcement learner includes the distance between the obstacle detected by the sensor and the wheelchair, the approach speed, and the position of the joystick. Reinforcement learner rewards are given as follows.

  First, when giving a positive reward, the user holds the joystick in the center position and the wheelchair stops. In this case, 1 reward is given.

  On the other hand, when a negative reward is given, it is assumed that when the user defeats the joystick, the electric wheelchair is moving in the direction opposite to the direction of defeating the joystick or is stopped. In this case, -1 reward is given.

  In other words, the user's intention is estimated from the operation of the joystick, a positive reward is given when it can be determined that the target distance is secured, and the virtual repulsive force is too weak or too strong, the target distance If the wheelchair is moving in a direction contrary to the user's intention, a negative reward is given.

  Next, the Sarsa (λ) processing procedure using tile coding by CMAC, which is used for generating the state space, will be briefly described.

First, the CMAC load table is expressed in vector by the vector θ. Vector e represents the eligibility trace vector. Q _a is a value function. S is a set of states s, and the distance between the obstacle detected by the sensor and the wheelchair, the approach speed, and the position of the joystick are given as numerical values. a is an action and gives a weighting coefficient η. The reward r is a real value based on the above criteria. Γ is a discount rate, and λ is a trace-decay parameter.

  First, the vector θ is arbitrarily initialized, and a zero vector is substituted for the vector e. The following procedure is repeated for each episode.

First, let state S be the initial state of the episode. Next, for an action a that is an element of the set A (s), a feature set at s and a is F _a , a sum of vectors θ (i) is Q _a for an element i of F _a , and arg max _a Q _Let a be a.

  Next, an action is selected from the action a of the set A (s) by the softmax policy. However, in the softmax policy, the selection probability is determined based on the following equation, and the action is selected from the action set.

  Next, the following processing procedure is repeated for each step of the episode.

  First, let γλ × vector e be a vector e.

Then, for all elements i of the set F _a, the vector e (i) +1, and the vector e (i). Take action a and observe reward r and next state s ′. Let r-Q _a be δ. Set A (s'
For all elements a contained), s ', a feature set at a and F _a, the sum of the vector theta (i) the elements i of F _a and Q _a, the arg max _a Q _a a' a To do. Then, an action is selected from the action a ′ of the set A (s) by the softmax policy. Then, δ + γQ _{a ′} is δ, and vector θ + αδ × vector e is vector θ. If s ′ is in the terminal state, the repetition ends.

  Using the above procedure, it was confirmed whether a certain distance desired by the user can be maintained between the obstacle and the obstacle according to the input of the joystick.

In practice, the system should be based on the above-described equations (7)-(21), but for simplification, only one-way experiment was performed. That is, for convenience, the wheelchair system is expressed as y (t) = 1.905 y (t-1) −0.905 y (t−2) +0.005 u (t−1) +0.005 u (t−2). Further, the user's operation input is expressed as u _m (t) = K _p {w (t) −y (t)} as a p controller. Here, _Kp is the proportional gain of the p controller, and w (t) is the target value. Further, it is assumed that K _p , w (t), and ν (t) are unknown to the obstacle avoidance assistance system. The parameters of the reinforcement learner used for the repulsive force adjuster are as follows. First, a step-size parameter α is 0.2, a discount rate γ is 1.0, a trace-decay parameter λ is 0.9, and a temperature coefficient T is 1.0. The number of CMAC weight tables (Number of weight tables of CMAC) was 5.

First, the result of performing a one-dimensional simulation with the target value w (t) = 45 and changing K _p and ρ ₀ will be described. As a simulation, 100 trials were performed. FIGS. 8A and 8B show simulation results when K _p = 0.5 and ρ ₀ = 60. Also, FIG.
The (a) (b), it shows the simulation results for _{K p = 1.0, ρ 0 =} 65.

A thick solid line with y = 60 on the vertical axis in FIG. 8A and y = 65 on the vertical axis in FIG. 9A represents a wall surface. In each figure, the alternate long and short dash line at the position of y = 45 on the vertical axis represents the distance to be avoided. Moreover, in each figure, the locus | trajectory according to the time change of the wheelchair when a black continuous line gives virtual repulsive force is shown. Moreover, a broken line shows the locus | trajectory when not giving virtual repulsive force.

FIGS. 8B and 9B show changes over time in the parameter η to be adjusted. FIG. 8B shows a state where the parameter η is about 4. By the repulsive force calculated by the virtual repulsive force generator, the steady distance by the P controller is suppressed, and the target distance is maintained. In FIG. 9B, the parameter η is about 8, and the target distance can be maintained as in FIG. 8B.

Next, a description will be given of the result of a two-dimensional simulation with the target value w (t) = 300, K _p = 5, and changing ρ ₀ . As a simulation, 20 trials were performed. FIGS. 10A and 10B show simulation results in the case of ρ ₀ = 380. FIGS. 11A and 11B show simulation results when ρ ₀ = 350.

A thick solid line with x = 380 on the horizontal axis in FIG. 10A and x = 350 on the horizontal axis in FIG. 11A represents a wall surface. Moreover, in each figure, the broken line of the position of x = 300 in a horizontal axis is
Represents the distance to avoid. Moreover, in each figure, the locus | trajectory according to the time change of a wheelchair when the white circle (18a, 19a) by a black continuous line gave virtual repulsion force is shown. Further, gray areas (18b, 19b) are shown in a shaded position on the right side in the figure with respect to the white circle area, and the trajectories when virtual repulsive force is not applied are superimposed.

FIGS. 10B and 11B show changes over time in the parameter η to be adjusted. FIG. 10B shows a state where the parameter η is about 1. By the repulsive force calculated by the virtual repulsive force generator, the steady distance by the P controller is suppressed, and the target distance is maintained. In FIG. 11B, the parameter η is about 0.5, and the target distance can be maintained as in FIG.

  From the above results, it is understood that the repulsive force adjuster using reinforcement learning can maintain the target distance by the same system even if the distance from the wall is different, based on the judgment from the user's input.

[Modification 1]
The modification about the wheelchair which can move to all directions in the above-mentioned Example 1 is demonstrated. In this modification, an implementation of an obstacle avoidance device for a conventional electric wheelchair having a left and right drive wheel mechanism will be described.

  FIG. 14 is a diagram showing an outline of the electric wheelchair (moving body) 35 of the present modification. The electric wheelchair 35 has a schematic configuration substantially the same as that of the electric wheelchair 1 shown in FIG. 1, and the description of the same points is omitted. FIG. 14 shows a joystick 36 corresponding to the operation unit 2 shown in FIG. 1, a avoidance device 3 shown in FIG. 1, a microcomputer 37 equivalent to the driving device 6, and an obstacle distance measuring sensor 38 equivalent to the objective sensor 4. Indicates.

Since the electric wheelchair 35 has left and right drive wheels mechanism, parameters, as shown in FIG. 13, the rotational speed of the left wheel [nu _L, the rotation speed [nu _R of the right wheel, these translational velocity [nu _x The relationship with the angular velocity ω is given by ν _R = ν _x + rω and ν _L = ν _x −rω. However, r represents the distance from the vehicle body center to the wheel ground contact point.

Thus, the configuration of the present invention can be applied not only to a wheelchair that can move in all directions but also to a conventional electric wheelchair 35 having a left and right drive wheel mechanism. However, since the electric wheelchair 1 has a smaller turn than the electric wheelchair 35, it can be said that the steering is difficult, and more obstacle avoidance effects using the obstacle avoidance assisting device can be obtained.

[Modification 2]
The mobile body according to the present invention can be applied to other than wheelchairs. That is, as the moving body, a form in which an obstacle avoidance device is used for a vehicle or the like that is operated and operated by a person is assumed. The vehicle to be used is not limited as long as the moving speed and the moving direction can be controlled. As a form of operation, when riding a vehicle with a joystick, steering wheel, accelerator, brake, etc., assisting with power by pushing a cart, remote control with radio control etc. Can be considered. Therefore, the present invention may be applied to omnidirectional vehicles, vehicles using steered wheels (automobiles), vehicles using left and right drive wheels (electric wheelchairs), walking vehicles, ships, and the like. The moving speed is assumed to be the same speed as when a person walks or runs, but is not particularly limited.

  In this modified example, a cart-type vehicle (moving body) 40 that loads and conveys luggage will be described with reference to FIG. 15 as an example of mounting on a light vehicle. The vehicle 40 is a type in which a person moves by pushing a handle 41, and an obstacle avoidance assisting device (avoidance device) (not shown) assists the operation. The vehicle 40 has a schematic configuration substantially similar to that of the electric wheelchair 1 shown in FIG. 1, and the description of the same points is omitted.

  The handle 41 of the vehicle 40 has a built-in force sensor that detects the force applied by the user. The handle 41 detects the magnitude and direction of the force applied to the handle 41. Further, the vehicle 40 includes an obstacle distance measuring sensor 42 around the carriage. The avoidance device of the vehicle 40 adjusts the driving force of the left and right wheels of the carriage, and assists the force pushed by the person. This implements an obstacle avoidance assist operation.

  According to the vehicle 40 having the above configuration, for example, even when the load to be transported is large and the road surface condition cannot be sufficiently grasped, it is possible to travel while avoiding the obstacle safely.

[Modification 3]
In this modification, an example of a mobile body that is remotely controlled will be described with reference to FIGS. The system shown in FIGS. 16A and 16B includes a schematic configuration similar to that of the electric wheelchair 1 shown in FIG. 1 when integrated. Below, description is abbreviate | omitted about the point similar to the electric wheelchair 1. FIG.

  A vehicle (moving body) 45 shown in FIG. 16B includes a CCD camera 48, an obstacle distance measuring sensor 49, and an obstacle avoidance device (avoidance device) (not shown). On the other hand, FIG. 16A shows an operation unit 46 and a display unit 47 which are a part of a remote control device for operating the vehicle 45. The operation unit 46 has a handle shape. The display unit 47 is a flat display. Image data captured by the CCD camera 48 of the vehicle 45 is transmitted to the remote control device and displayed on the display unit 47.

  The user uses the operation unit 46 to instruct the vehicle 45 about a desired moving speed or the like. A surrounding obstacle is detected by an obstacle distance measuring sensor 49. The avoidance device adjusts the driving force of the vehicle 45 to realize the obstacle avoidance assist operation.

As described above, when a moving body is controlled by a remote operation such as radio control, a CCD camera mounted on the moving body may not be able to sufficiently grasp an obstacle in the traveling direction. Even in such a case, if the proposed obstacle avoidance assisting device is mounted, steering can be supported.

  In addition, in this modification, although the structure provided with the obstacle avoidance assistance apparatus in the vehicle 45 was demonstrated, it is not restricted to this, You may provide an obstacle avoidance assistance apparatus in the remote control apparatus side. The corrected speed may be calculated by using the obstacle avoidance assisting device on the remote operation device side and transmitted to the moving body. In this way, the moving body can be made compact.

  Further, the moving body is not limited to the illustrated configuration, and may be a vehicle, a flying body, a ship, or the like.

  As explained above, we proposed an omnidirectional mobile wheelchair using a simple mechanism and verified its dynamic characteristics by simulation. Moreover, the interference of each axis was removed by using a feedforward compensator. In addition, as an obstacle avoidance assistance system for electric wheelchairs, we proposed a system that adjusts the virtual repulsive force from obstacles by reinforcement learning, and verified its effectiveness by simulation. As a result, the repulsive force adjuster using reinforcement learning maintains the target distance by the same system even if the user's operability and the target distance (distance to the wall) differ according to the judgment from the user's input. Was shown to be possible.

  Further, as described above, the embodiment of the present invention can also be expressed as relating to development of an omnidirectional mobile wheelchair using a simple mechanism and implementation of an obstacle avoidance assisting device by reinforcement learning.

  Here, with the progress of an aging society, there are various needs for mobility aids ranging from those who feel a little inconvenient to those who are bedridden. Among them, there are common demands for a small turn in a narrow place and a wheelchair with a different image from the conventional one. Therefore, as a device for dealing with these problems, the present invention proposes a small and inexpensive omnidirectional mobile wheelchair that uses this to realize omnidirectional movement by greatly simplifying the mechanism and controlling it with software to facilitate operation. In addition, by implementing an obstacle avoidance assistance system using reinforcement learning, the obstacle avoidance is appropriately assisted according to the user's steering ability. In the proposed method, obstacle avoidance assistance is performed by synthesizing the virtual repulsive force from the obstacle into the user's operation input. The virtual repulsive force is adjusted using reinforcement learning to individual users and environmental conditions. The proposed method was evaluated by numerical simulation.

  Conventionally, an omnidirectional movement mechanism and a wheelchair equipped with an omnidirectional movement mechanism are disclosed in Non-Patent Documents 1 to 3, but the mechanism is complicated and expensive so that it can be widely used in general. Not reached.

  Thus, as described above, a simple mechanism is used to facilitate omnidirectional movement by software control, and a small and inexpensive omnidirectional wheelchair using the same is proposed.

  Moreover, although the conventional patent document 4 has described about the structure which carries out operation control supposing a repulsive force, the detail of the algorithm like this case is not disclosed. Further, how to calculate the virtual repulsive force and how to use it are different from the configuration of the above-described embodiment.

Moreover, it is difficult for a person with an obstacle to operate an electric wheelchair while avoiding an obstacle. Therefore, as described above, a system that performs obstacle avoidance assistance suitable for the user's condition and use environment is constructed. Specifically, avoiding an obstacle is assisted by applying a virtual repulsive force from the obstacle to the user's joystick operation input. I think that the magnitude of this virtual repulsive force needs to be changed according to environmental conditions and users. For example, the appropriate magnitude of the virtual repulsive force differs between a wide area with few obstacles and a narrow area with many obstacles, and the appropriate virtual repulsive force changes depending on the environment. Also, the appropriate magnitude of the virtual repulsive force varies depending on the degree of obstacle of the user and the psychological characteristics (personal space size, etc.) of the individual. Therefore, we propose a method for judging whether the virtual repulsive force is good or bad in the environment from the operation input of the user's joystick, and generating an appropriate output using reinforcement learning.

  As a method of adjusting the virtual repulsive force using a method other than reinforcement learning, a method based on prior learning (offline learning) can be considered. In this case, using a supervised learning method such as a neural network, traveling in an assumed environment is performed in advance, and an appropriate virtual repulsive force is given as a teacher signal. In order to perform such pre-learning, an appropriate distance and speed from the obstacle are required in advance, and the virtual repulsive force needs to be calculated backward from there. However, this method is not practical because the appropriate distance and speed with respect to the obstacle vary depending on the person who uses it, is not obtained explicitly, and varies depending on the driving environment. That is, this conventional method is not adapted to the user. In addition, it does not correspond to the dynamic response of human beings, and for this reason, it can also be said that it does not adjust by interaction with humans.

  It is also conceivable that this method is learned by always inputting the distance and speed that the user wants to maintain against the obstacle in some way while manipulating the method. However, this is a burden on the user. Also, in reality, the user doesn't want to maintain this distance, this speed exactly, but sometimes feels far better, closer, simply better or worse It seems that you just want to improve it.

  Therefore, as described above, the virtual repulsive force is adjusted using reinforcement learning, which is an unsupervised learning method that does not require a teacher signal and gives positive or negative rewards depending on whether the current state is good or bad.

  The specific embodiments or examples described above are merely to clarify the technical contents of the present invention, and the present invention is not limited to such specific examples and should not be interpreted in a narrow sense. Various modifications can be made within the scope of the claims, and the modified embodiment and the embodiment obtained by appropriately combining the technical means respectively disclosed in the embodiment are also the technical features of the present invention. Included in the range.

  Since the obstacle avoidance device according to the present invention can realize an optimal movement state according to the user's operation, it can be mounted on, for example, an electric wheelchair.

  Moreover, since the obstacle avoidance device according to the present invention can cope not only with different users but also when the environment changes greatly, it can also be mounted on a mobile object that is remotely operated, for example.

It is a block diagram which shows the outline of one Embodiment about the electric wheelchair as a moving body provided with the obstacle avoidance apparatus which concerns on this invention. It is a perspective view of the electric wheelchair. It is a figure which shows the one part outline of the said electric wheelchair. It is a block diagram which shows a part of one Example of the said electric wheelchair. It is a block diagram which shows the other part in one Example of the said electric wheelchair. (A) is a figure which shows an example of the simulation result about the said electric wheelchair, (b) is a figure which shows another example of the said simulation result, (c) shows another example of the said simulation result. FIG. (A) is a figure which shows an example of the other simulation result about the said electric wheelchair, (b) is a figure which shows another example of the said simulation result, (c) is another example of the said simulation result. FIG. (A) is a figure which shows an example of the other simulation result about the said electric wheelchair, (b) is a figure which shows another example of the said simulation result. (A) is a figure which shows an example of the other simulation result about the said electric wheelchair, (b) is a figure which shows another example of the said simulation result. (A) is a figure which shows an example of the other simulation result about the said electric wheelchair, (b) is a figure which shows another example of the said simulation result. (A) is a figure which shows an example of the other simulation result about the said electric wheelchair, (b) is a figure which shows another example of the said simulation result. It is a perspective view which shows an example of the sensor mounting state in the said electric wheelchair. It is a figure for demonstrating operation | movement of the modification of the said electric wheelchair. It is a perspective view of the modification of the said electric wheelchair. It is a perspective view which shows the outline of the light vehicle as an example of the said mobile body. (A) is a perspective view which shows the outline of the remote control apparatus of an example of the said mobile body, (b) is a perspective view which shows an example of the said mobile body.

Explanation of symbols

1, 30, 35 Electric wheelchair (moving body)
2, 31, 36, 41, 46 Operation unit 3 Avoidance device (obstacle avoidance device)
4, 32 Objective sensor (obstacle sensor)
DESCRIPTION OF SYMBOLS 5 Speed sensor 6 Drive apparatus 7 Power part 11 Repulsive force generator 12 Learning part 15 Repulsive force production | generation part 38, 42, 49 Anti-obstacle distance measurement sensor (obstacle sensor)
40, 45 Vehicle (moving body)

Claims (4)

By specifying the speed obtained by correcting the target speed according to the instruction speed from the user for the moving body that is going to move at the specified speed, the collision of the moving body with the obstacle is avoided. An obstacle avoidance device,
It used to modify the target speed by subtracting to the target speed after having multiplied the selected weighting factor, the repulsive force for avoiding a collision between the obstacle, obstacle around the movable body It has a repulsive force generator that calculates according to information from obstacle sensors that detect objects,
Further, the weighting factor is selected with reference to the distance between the moving body and the obstacle, the approach speed between the moving body and the obstacle, and the indicated speed, and the direction of the moving speed of the moving body and the An obstacle avoidance device comprising: a learning unit that performs reinforcement learning for determining a policy for selecting the weighting factor based on a reward according to a match or mismatch with a direction of an instruction speed . The learning section, if the direction of the direction and the instruction speed of the moving speed of the moving object matches a positive reward, and the direction of the direction and the instruction speed of the moving speed of the moving body in the case of disagreement The obstacle avoidance apparatus according to claim 1, wherein reinforcement learning is performed by setting a negative reward. The operation unit for detecting the indicated speed is a joystick type operation unit,
The learning section, when the upper Symbol moving body is stopped, and a positive compensation when the joystick is held in a central position,
If the direction is tilted directional and the joystick of the moving speed of the moving object does not match, or, if the joystick is tilted when the upper Symbol moving body is stopped, the negative The obstacle avoidance device according to claim 2, wherein the obstacle avoidance device is a reward.   A moving body comprising the obstacle avoidance device according to any one of claims 1 to 3.

Images