KR20230085609A - Segway Robot Based on Reinforcement Learning and Its Method for Motion Control - Google Patents

Abstract

One embodiment of the present invention provides a robot driving state control system based on reinforcement learning, the robot driving state control system comprising: a driving data collection unit which collects driving data including inertial data measured by an IMU sensor of a robot and rotation data of a motor measured by an encoder of the robot; a control logic operation unit which calculates a robot state value using the driving data obtained from the driving information collection unit and performs reinforcement learning to compare with a target robot state value; and a robot driving control unit which generates a robot control signal for controlling the driving conditions of the robot based on the robot state value obtained from the control logic operation unit.

Description

Segway Robot Based on Reinforcement Learning and Its Method for Motion Control

This embodiment relates to a reinforcement learning-based Segway-type robot and its motion control method, and more specifically, to a motion control method for self-balancing and autonomous driving of a Segway-type robot to which deep learning reinforcement learning is applied.

Segway (Segway) is a means of transportation that transports users using electric motors, and is attracting attention as a means of transportation to replace internal combustion engines as next-generation mobility.

Segway uses one or two wheels unlike conventional four-wheel drive vehicles, and driver safety is greatly affected by changes in the driving environment of mobility. For example, when driving conditions change due to a change in the center of gravity or spin of a wheel, there is a need to search for optimal driving conditions by self-learning.

In particular, since there is a difference between driving conditions in the case of Segway wheel slip (spin) and general driving conditions, it is necessary to appropriately reflect these driving characteristics.

In the driving process of conventional Segway-type robots, changes in physical characteristics of the robots (eg, changes in mass, change in speed, change in acceleration, etc.) cannot be sensed in real time. There is a problem that cannot be collected and updated.

In addition, conventional Segway-type robots such as KR 10-1816136 B1 have limitations in that they cannot design sophisticated learning models without comprehensively considering control parameters such as mass (m) and rotational inertia (I).

Against this background, the purpose of this embodiment is, in one aspect, a Segway-type robot capable of applying a reinforcement learning algorithm that reflects parameters related to the mass and rotation of a Segway-type robot and controlling the robot's operation by reflecting real-time driving conditions. It is to provide a robot and its motion control method.

In another aspect, an object of the present embodiment is to provide a Segway-type robot and an operation control method thereof capable of performing learning and a running state of the robot by reflecting whether or not wheels of the Segway-type robot have slipped (spin).

In another aspect, an object of the present embodiment is to provide a segway-type robot capable of self-balancing and autonomous driving by applying a deep learning learning algorithm for predicting a driving state of the robot and a method for controlling its operation.

In order to achieve the above object, an embodiment of the present invention, in a reinforcement learning-based robot driving state control system, inertia data measured by an IMU sensor of the robot and rotation data of a motor measured by an encoder of the robot A travel data collection unit that collects travel data including; a control logic operation unit that calculates a robot state value with the driving data obtained from the driving information collection unit and performs reinforcement learning to compare the value with a target robot state value; and a robot driving control unit generating a robot control signal for controlling a driving condition of the robot based on the robot state value acquired by the control logic operation unit.

In the robot driving state control system, the robot is a Segway, the inertia data includes angular velocity values and acceleration values for the x-axis, y-axis, and z-axis of the robot, and the rotation data is a rotation angle value for each wheel of the robot Including, it is possible to recognize the posture of the robot based on the inertia data, and recognize the movement of the wheel based on the rotation data.

In the robot driving state control system, the control logic calculation unit obtains the robot state value by calculating the mass value (m) and the rotational inertia value (I) of the robot by setting the linear motion acceleration value (a) of the robot as an input value. can do.

In the robot driving state control system, the reinforcement learning calculates the difference between the acquired robot state value and the preset target robot state value, and ends the learning when it is recognized that the difference value is 0 or within a standard range, and the difference If the value is out of the reference range, the robot state value may be updated.

In the robot driving state control system, the reinforcement learning may be performed by a simulation program or by receiving a robot state estimation value delivered by a user terminal.

In the robot driving state control system, the robot state value may include one or more of the robot's mass, rotational inertia, inclination, tilt variation, speed, acceleration, wheel torque, and wheel angular velocity.

In the robot driving state control system, the control logic calculation unit obtains linear motion speed data through the sum of the speeds of the left and right wheels of the robot and obtains rotational motion speed data through the speed difference between the left and right wheels of the robot to estimate the robot state value. can do.

The robot driving state control system may further include a robot driving control unit that controls the operation of the motor by transmitting a robot control signal corresponding to the robot state value calculated by the control logic operation unit to the motor driver.

In order to achieve the above object, another embodiment of the present invention is a method for determining and controlling a driving state of a Segway, comprising: collecting driving data of the Segway; determining whether a wheel spins based on the driving data; performing control logic for calculating a Segway state value and comparing it with a target Segway state value when the spin of the wheel does not occur; and controlling driving of a motor to correspond to the Segway state value.

In the Segway driving state control method, the Segway includes sensors that collect driving data, and the driving data includes inertia data including angular velocity values and acceleration values of the Segway and rotation data including rotation angle values for each wheel of the Segway. can include

In the Segway driving state control method, the Segway includes an encoder for measuring rotational angular displacement of the wheel; and a processor determining the rotation of the wheel by comparing the number of rotations of the wheel measured by the encoder with the number of rotations of the target wheel.

In the Segway driving state control method, whether or not the wheel spins is determined by comparing the linear movement distance and the rotational distance of the wheel, and if the rotational distance of the wheel is greater than the linear movement distance, it can be determined that spin exists.

In the Segway driving state control method, the Segway includes a hub motor providing driving force to the wheels; and a processor obtaining acceleration data by applying output data of the hub motor to a predetermined algorithm.

The method for controlling the driving state of the Segway may further include performing a control algorithm for determining whether a disturbance occurs based on a change in mass (m) or rotational inertia (I) generated during the driving process of the Segway.

In the Segway driving state control method, when it is determined that there is a difference between the Segway state value and the target Segway state value, reinforcement learning may be performed to converge to the target Segway state value by iterating over the Segway state value.

In the Segway driving state control method, driving of the wheels may be individually controlled by transmitting a motor control signal corresponding to the Segway state value to a motor driver.

As described above, according to one embodiment of the present invention, the operation of the robot can be controlled by reflecting the real-time driving situation by applying a reinforcement learning algorithm that reflects parameters related to the mass and rotation of the Segway-type robot.

According to an embodiment of the present invention, the driving state of the robot can be controlled by reflecting the slip (spin) of wheels of the Segway-type robot and changes in the driving state.

According to an embodiment of the present invention, it is possible to reduce the amount of computation of a processor for self-balancing and autonomous driving by extracting and learning some parameters based on modeling parameters of a Segway-type robot, and reinforcement learning for self-balancing and autonomous driving. speed can be improved.

1 is a diagram illustrating a data communication method between a Segway and a processor according to an embodiment of the present invention.
2 is a diagram illustrating an operation sequence of a processor according to an embodiment of the present invention.
3 is a diagram for explaining an operation control sequence of a Segway according to an embodiment of the present invention.
4 is a configuration diagram of a Segway according to an embodiment of the present invention.
5 is a diagram illustrating modeling parameters of a Segway according to an embodiment of the present invention.
6 is a diagram explaining a method of calculating Segway modeling parameters according to an embodiment of the present invention.
7 is a diagram illustrating a reinforcement learning method according to an embodiment of the present invention.
8 is a flowchart illustrating a method of generating a control signal for a Segway according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, a detailed description of a related known configuration or function, which is determined to obscure the gist of the present invention, will be omitted.

In addition, in describing the components of the present invention, terms such as first, second, a, and b may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is directly connected or connectable to the other element, but there is another element between the elements. It will be understood that elements may be “connected”, “coupled” or “connected”.

In addition, the terms 'system', 'server', etc. used in this specification may be defined as a computer program or device that transmits and receives information to and from a client through a network for data storage and calculation, but is not limited thereto.

In addition, the terms 'Segway', 'Segway-type robot', and 'robot' used in this specification may be defined as a device that provides a driving force to transport a user, and each term may be used interchangeably as needed. .

1 is a diagram illustrating a data communication method between a Segway and a processor according to an embodiment of the present invention.

Referring to FIG. 1 , the Segway 10 and the processor 20 according to an embodiment may control the operation of the Segway through data communication.

The Segway 10 may be a robot for user transportation that includes one or more wheels. For example, the Segway 10 includes two wheels and a motor, may include a motor driver for controlling the motor of each wheel, and transmits and receives a control signal of the motor driver according to an operation result of the processor 20. can do.

The Segway 10 may further include an IMU sensor (not shown) capable of measuring inertial data and an encoder (not shown) capable of measuring rotational data of a motor.

The IMU sensor (not shown) is an inertial measurement unit (IMU), which measures inertial data such as angular velocity values and acceleration values for the three axes of the robot—for example, the x-axis, y-axis, and z-axis. can be measured In addition, the IMU sensor can measure the inclination and attitude of the robot in each direction by combining the 3-axis acceleration and 3-axis gyro sensors. For example, the IMU sensor may obtain a measurement value by applying a Kalman filter, but is not limited thereto.

An encoder (not shown) may measure rotational data such as the number of rotations, rotational speed, rotational angle, and rotational direction of the motor. In addition, the encoder may measure the motion of the wheel of the robot based on the rotation data. For example, the encoder may adopt various measurement methods such as an optical or magnetic encoder, but is not limited thereto.

The processor 20 may be a device for calculation such as a server separate from the Segway 10, but includes a program included in the Segway 10 and performing calculations for controlling the operation of the Segway 10, and the same. It may be a device that

The Segway 10 may transmit information about the driving state to the processor 20 (S101), and the processor 20 receives and learns information about the driving state, etc., and controls the driving state of the Segway 10. The control signal may be transferred to the Segway 10 (S102).

The processor 20 may be implemented as a physical computing device, but is not limited thereto and may be implemented to receive and calculate information about the driving state of the Segway 10 through a cloud server or the like.

2 is a diagram illustrating an operation sequence of a processor according to an embodiment of the present invention.

Referring to FIG. 2 , the processor 20 may include a driving data collection unit 21, a control logic operation unit 22, a robot driving control unit 23, and the like. Blocks implementing each operation of the processor 20 may be physically or conceptually separated blocks.

The driving data collection unit 21 may receive driving data of the Segway from each sensor and generate a robot control signal for controlling the robot.

The driving data of the Segway may include inertia data measured by an IMU sensor, rotation data of a motor measured by an encoder, and the like.

The control logic operation unit 22 may calculate a robot state value using the driving data obtained from the driving information collection unit and perform reinforcement learning to compare the robot state value with the target robot state value.

The robot state value may be a parameter for representing a driving state of the robot, such as mass, rotational inertia, inclination, tilt variation, speed, acceleration, wheel torque, and wheel angular velocity.

For example, the control logic calculation unit 22 sets the linear motion acceleration value ( ) of the robot as an input value and calculates the mass value (m) and rotational inertia value (I) of the robot to obtain a robot state value of the robot. can

The control logic operation unit 22 may obtain a robot state value through a preset algorithm, and may perform machine learning - for example, reinforcement learning - to obtain a target robot state value as needed. .

Reinforcement learning performed by the control logic operation unit 22 calculates the difference between the acquired robot state value and the preset target robot state value, and ends the learning when the difference value is recognized as 0 or within the standard range. , If the difference value is out of the standard range, the robot status value can be repeatedly updated.

Reinforcement learning performed by the control logic operation unit 22 may be performed by a simulation program or by receiving a robot state estimation value transmitted by a user terminal. When reinforcement learning does not reach the convergence value, or by performing learning based on the input data of the driving state value transmitted from the user terminal to reflect the driving condition desired by the user, the offset of the driving state of the robot is adjusted or the user Convenience can be improved.

The control logic operation unit 22 obtains linear motion speed data through the sum of the speeds of the left and right wheels of the robot to estimate a robot state value, or acquires rotational motion speed data through the speed difference between the left and right wheels of the robot to estimate the robot state value. can be estimated In this case, parameters related to the driving state—for example, the robot's linear motion speed and the robot's rotational motion angular speed of FIG. 6—can be calculated by a predetermined calculation formula.

The robot driving control unit 24 may control the driving condition of the robot based on the robot state value obtained from the control logic operation unit. The control logic calculation unit 22 updates the robot state value acquired through repetitive learning, and based on this, a robot control signal for controlling components such as a motor of the robot can be changed in real time.

The robot driving control unit 24 may transmit a robot control signal for controlling the robot only when it is determined that the current robot state value and the target robot state value match or come within a standard error range.

The robot driving control unit 24 may control the operation of the motor by transmitting a robot control signal corresponding to the robot state value calculated by the control logic operation unit 22 to the motor driver.

3 is a diagram for explaining an operation control sequence of a Segway according to an embodiment of the present invention.

Referring to FIG. 3 , the operation of the Segway 10 may be controlled by receiving an operation result of the processor 20 as an input.

The Segway 10 may include an IMU sensor 11, a motor encoder 12, a motor driver 13, a left motor 14, a right motor 15, and the like.

The IMU sensor 11 may be a device for measuring rotational angular velocity values for three axes of the robot and acceleration values in each axis direction, and the motor encoder 12 may be a device for measuring rotational angles, etc. It may be a device for measuring

The Segway 10 may include a left motor 14 corresponding to left and right wheels, a right motor 15, and a motor driver 13 controlling each motor.

The motor driver 13 may receive a robot control signal from the processor 20 and control driving conditions such as the number of rotations, rotational speed, rotational acceleration, and rotational angle of the left and right motors 14 and 15 .

The processor 20 includes an inertia data collection unit 21-1, a rotation data collection unit 21-2, a control logic operation unit 22, a robot state value estimation unit 23, a robot travel control unit 24, and the like. can do.

The inertial data collection unit 21-1 may collect inertial data measured by the IMU sensor 11 to determine the posture of the robot, such as inclination.

The rotation data collection unit 21-2 may collect rotation data measured by the encoder 12 of the motor and determine the movement of the wheel.

The control logic calculation unit 22 may collect driving data such as inertia data and rotation data to calculate a driving state value related to the driving state of the robot. In addition, the control logic operation unit 22 may update the driving state value through time-sequential data learning to generate the driving state value for determining the operation of the robot by the robot driving control unit 24 .

The control logic operation unit 22 may learn the input driving data according to a preset learning algorithm, and further determine whether or not to continue the learning by determining the accuracy of the learning.

The control logic operation unit 22 may compare a target driving state value, which is data related to the driving state of the robot desired by the user, and a driving state value, which is data related to the current driving state of the robot, to determine whether to end the learning.

The robot state value estimator 23 may reduce the amount of learning by determining the learning order by considering the priority of the parameters and the correlation between the parameters without performing calculations on all the parameters. For example, as shown in FIG. 7 , the acceleration obtained by the robot's IMU sensor and the target acceleration are first compared, and the angular acceleration obtained by the motor's encoder and the target angular acceleration are then compared.

If the parameter estimation by the robot state value estimator 23 is not completed or it is determined that the estimated parameter is different from the target value, the resulting value may be transferred to the control logic calculator 22 to perform learning again.

The robot state value estimation unit 23 may be included in a lower block of the control logic operation unit 22, but may be defined as a separate block to sequentially perform data calculation and update as needed.

The robot driving control unit 24 may generate a driving control signal based on the robot state value determined and transmitted by the control logic operation unit 22 and transmit it to the motor driver 13 . Since the robot driving control unit 24 generates a driving control signal based on the calculation result of the control logic calculating unit 22, more sophisticated driving control is possible.

Here, the configuration of the Segway 10 and the processor 20 is divided to explain the process of collecting and calculating data and transmitting control signals, and is all or part of the configuration included in one robot, and the technical idea of the present invention is It is not limited to FIG. 3 .

4 is a configuration diagram of a Segway according to an embodiment of the present invention.

Referring to FIG. 4 , the Segway 100 may include a left wheel 101, a right wheel 102, a body part 103, a handle part 104, and the like.

According to the rotation state of the wheels 101 and 102 of the Segway 100, the linear motion and the rotational motion of the Segway 100 may be determined by dynamic modeling.

For example, when the center of mass of the Segway 100 is located adjacent to the main body 103, the movement of the Segway 100 is replaced with the movement of the main body 103, or the location of the center of gravity is defined and used for calculation. can

For example, when the rotation distance of the wheels 101 and 102 is the same as the linear movement distance of the Segway 100, it is determined that the wheel slip (spin) phenomenon has not occurred, and the rotation data of the wheels and the linear movement of the Segway are determined. Correlation of distance data can be defined.

In addition, if the current revolutions per minute (RPM) of the wheels 101 and 102 are the same as the target revolutions per minute (RPM), it is determined that the slip (spin) phenomenon of the wheels has not occurred, and the rotation data of the wheels and the Segway It is possible to define the correlation of straight-line travel distance data.

The speed in the linear motion direction or the rotational motion direction of the Segway 100 may be defined by utilizing the sum of left and right speeds or the speed difference of the wheels 101 and 102 .

The handle part 104 or the body part 103 may be utilized to determine the inclination of the robot. For example, the inclination of the pole of the handle part 104 may be defined as the inclination of the robot, and the inclination of one plane of the main body 103 may be defined as the inclination of the robot.

4 is for explaining a simplification method and a modeling method of each configuration of the Segway 100, and the technical spirit of the present invention is not limited thereto.

5 is a diagram illustrating modeling parameters of a Segway according to an embodiment of the present invention.

Referring to FIG. 5 , the configuration of the Segway 100 may be simplified and modeled.

), 가속도(

)를 정의할 수 있고, 회전운동의 회전각도(

), 회전각속도(

)를 운동데이터로 정의할 수 있다.The speed of linear motion of the Segway 100 (

), acceleration (

) can be defined, and the rotation angle of rotation (

), rotational angular velocity (

) can be defined as exercise data.

), 회전각속도(

), 회전관성(

), 각가속도(

) 등을 회전데이터로 정의할 수 있고, 우측바퀴(102)의 회전각도(

), 회전각속도(

), 회전관성(

), 각가속도(

) 등을 회전데이터로 정의할 수 있다.Rotation angle of the left wheel 101 of the Segway 100 (

), rotational angular velocity (

), rotational inertia (

), angular acceleration (

) can be defined as rotation data, and the rotation angle of the right wheel 102 (

), rotational angular velocity (

), rotational inertia (

), angular acceleration (

) can be defined as rotation data.

), 각속도(

), 토크(

), 각가속도(

) 등을 기울기데이터로 정의할 수 있다.The rotation angle of the tilt direction of the Segway 100 (

), angular velocity (

), talk(

), angular acceleration (

) can be defined as slope data.

), 전체 회전관성(

), 전체 각운동량(

), 본체부(103)의 일 평면에서 질량중심까지의 거리(미도시) 등을 정의하여 각 파라미터들의 계산에 활용할 수 있다.In addition, the total mass of the Segway 100 (

), total rotational inertia (

), total angular momentum (

), a distance from one plane of the main body 103 to the center of mass (not shown), etc. may be defined and used for calculating each parameter.

In order to obtain the state driving value according to an embodiment, the inclination parameter of the Segway 100 may be a solution obtained by substituting an equation of motion of a rotary inverted pendulum, and the solution of the nonlinear equation may be obtained through learning such as repetitive calculation. can be obtained

6 is a diagram explaining a method of calculating Segway modeling parameters according to an embodiment of the present invention.

Referring to FIG. 6 , it is possible to calculate estimated values for the driving state of the robot using the Segway modeling parameters of FIG. 5 .

The linear motion speed of the Segway can be derived using the sum of the velocities of the left and right wheels.

)는 바퀴의 반지름(

)과 각 바퀴의 회전속도 평균값(

)의 곱으로 정의할 수 있으며, 로봇의 직선운동 가속도(

)는 로봇의 직선운동 속도를 시간으로 미분하여 계산할 수 있다.For example, the linear motion speed of the robot (

) is the radius of the wheel (

) and the average value of the rotational speed of each wheel (

), and the linear motion acceleration of the robot (

) can be calculated by differentiating the linear motion speed of the robot with time.

The rotational angular velocity of the Segway can be derived using the speed difference between the left and right wheels.

)는 각 바퀴의 회전속도의 편차(

)에 바퀴의 반지름(

)을 곱한 뒤, 바퀴 사이의 거리(

)를 나눈 값으로 정의할 수 있으며, 로봇의 회전각도는 로봇의 회전운동 각속도를 시간으로 적분하여 계산할 수 있다.For example, the rotational angular velocity of the robot (

) is the deviation of the rotational speed of each wheel (

) to the radius of the wheel (

), then the distance between the wheels (

), and the rotation angle of the robot can be calculated by integrating the rotational angular velocity of the robot with time.

6 is for explaining a method of calculating parameters related to the driving state of the Segway 100, and the technical spirit of the present invention is not limited thereto.

7 is a diagram illustrating a reinforcement learning method according to an embodiment of the present invention.

Referring to FIG. 7 , the method 200 of reinforcement learning and robot motion control according to an embodiment includes collecting sensor data (S201), calculating robot acceleration (S202), current acceleration and target acceleration. Comparing (S203), calculating the angular velocity of the wheel (S204), comparing the current angular velocity and the target angular velocity (S205), calculating the mass of the robot (S206), calculating the angular acceleration of the robot (S207), comparing current angular acceleration and target angular acceleration (S208), calculating angular momentum and rotational inertia (S209), calculating control parameters (S210), controlling the operation of the robot (S211), etc. can include

The step of collecting sensor data ( S201 ) may be a step of measuring and collecting various types of sensing values in the form of data by sensors of the robot.

The step of calculating the acceleration of the robot (S202) may be a step of determining the measured value of the IMU sensor as the acceleration.

The step of comparing the current acceleration and the target acceleration (S203) may be a step of comparing the calculated acceleration of the robot in the current state with the target acceleration. If the current acceleration and the target acceleration are the same or within a preset reference range, the angular acceleration calculation step (S207) may be performed immediately. If the current acceleration and the target acceleration are not the same or out of a preset reference range, the step of calculating the angular velocity (S204) may be performed.

The step of calculating the angular velocity of the wheel (S204) may be a step of calculating the angular velocity of the left and right wheels/motors, and the measured value of the motor encoder may be used for calculating the angular velocity.

Comparing the current angular velocity and the target angular velocity (S205) may be a step of comparing the calculated angular velocity of the robot in the current state with the target angular velocity. If the current angular velocity and the target angular velocity are the same or within a predetermined reference range, the total mass calculation step (S206) of the robot may be performed immediately. If the current angular velocity and the target angular velocity are not the same or out of a preset reference range, a step of calculating a control parameter (S210) may be performed.

The step of calculating the mass of the robot ( S206 ) may be a step of calculating the total mass of the robot based on an equation related to classical mechanics such as Newton's law of motion.

The step of calculating the angular acceleration of the robot (S207) may be a step of obtaining and calculating the angular acceleration measurement value on the IMU sensor.

Comparing the current angular acceleration and the target angular acceleration (S208) may be a step of comparing the calculated angular acceleration of the robot in the current state with the target angular acceleration. If the current angular acceleration and the target angular acceleration are the same or within a preset reference range, the robot control parameter calculation (S210) may be performed immediately. If the current angular acceleration and the target angular acceleration are not the same or out of a predetermined reference range, a step of calculating the angular momentum and rotational inertia (S209) may be performed.

)을 계산할 수 있다. 각운동량(

)은 회전관성 공식 및 평행축 정리 등을 이용하여 계산할 수 있다.The step of calculating the angular momentum and rotational inertia (S209) may be a step of calculating the angular momentum and rotational inertia of the robot based on equations related to classical mechanics such as Newton's law of motion. The torque value is calculated from the equation of motion of the rotary inverted pendulum, and the rotational inertia (

) can be calculated. Angular momentum (

) can be calculated using the rotational inertia formula and the parallel axis theorem.

Calculating the control parameters (S210) may be a step of determining the parameters obtained through the above-described logic and determining whether to end learning.

Controlling the operation of the robot (S211) may be a step of controlling the operation of the robot by generating and transmitting a robot control signal corresponding to the obtained robot state value based on the determined parameters.

8 is a flowchart illustrating a method of generating a control signal for a Segway according to an embodiment of the present invention.

Referring to FIG. 8, the method 300 for determining and controlling the driving state of the Segway includes collecting driving data of the Segway (S301), determining whether a wheel spins based on the driving data (S302), and Determining whether a disturbance occurs during the driving process (S303), performing the Segway control logic (S304), generating a motor control signal of the Segway to control the driving of the motor (S305), etc. can

The step of collecting driving data of the Segway (S301) may be a step of collecting data generated during the driving process of the Segway. Data collected through the sensors can be calculated in the form of parameters to define and control the driving state of the Segway.

The driving data may include various data sets, such as inertia data including angular velocity values and acceleration values of the Segway, and rotation data including rotation angle values for each wheel of the Segway. For example, if the motor providing driving force to the wheels of the Segway is a hub motor, data such as rotational speed and rotational acceleration of the motor may be obtained based on the axis of the hub. In addition, separate converted data may be obtained by applying the output data of the motor to a preset algorithm.

Determining whether the wheel spins based on the driving data (S302) may be a step of determining whether the wheels of the Segway spin based on the acquired driving data. If the Segway wheels spin or not, the consistency or correlation of the acquired driving data becomes inaccurate. Therefore, to simplify driving modeling and improve accuracy, it is possible to first determine whether or not the Segway wheels spin.

Only when it is determined that the spin of the wheels of the Segway does not exist, a step for calculating parameters and calculating Segway driving state values may be performed.

The Segway according to an embodiment may further include an encoder for measuring rotational angular displacement of the wheel, and a processor for determining the rotation of the wheel by comparing the number of rotations of the wheel measured by the encoder with the number of rotations of the target wheel.

Whether or not the Segway spins according to another embodiment is determined by comparing the straight-line travel distance of the Segway and the rotational distance of the wheel, and when the rotational distance of the wheel is greater than the straight-line travel distance, it can be determined that spin exists.

Determining whether a disturbance occurs during the driving process of the Segway (S303) is a step of determining whether a disturbance occurs by monitoring data change values such as mass (m) or rotational inertia (I) generated during the driving process of the Segway. can be In this process, a preset control algorithm may be performed, or a change value exceeding a reference value may be determined to be the occurrence of a disturbance.

In addition, in the process of determining whether a disturbance has occurred, the disturbance may be determined based on the weight of the Segway itself, excluding the weight of the occupant.

The step of performing the Segway control logic (S304) may be a step of performing data learning by applying a control logic that calculates the Segway state value and compares it with the target Segway state value when the wheel does not spin.

If necessary, the steps of performing dynamic modeling of the Segway and extracting parameters necessary for modeling may be previously performed before the step of performing the Segway control logic. Since the degree of freedom of operation increases according to the number of parameters used in the learning algorithm, the number of parameters may be limited to a number within a preset range.

The control logic may be a step of comparing a target Segway state value corresponding to a user's input target value with a current Segway state value, and calculating parameters according to a set order.

The control logic may repeatedly update parameters to improve data accuracy, and may repeat learning until a predetermined target deviation or target ratio is reached.

In addition, when it is determined that there is a difference between the Segway state value and the target Segway state value, the control logic may perform reinforcement learning to converge to the target Segway state value by repeatedly calculating the Segway state value in an iteration method. For example, a circular iteration may be performed in which all or part of the acquired output values are used as input data.

The step of controlling driving of the motor by generating a motor control signal of the Segway (S305) may be a step of simultaneously or simultaneously controlling the driving of the motor according to the Segway motor control signal. Depending on the motion state of the Segway (for example, linear motion or rotational motion), driving of the motor may be independently controlled.

Through the above method, it is possible to perform a self-balancing function that keeps parameters such as direction and speed constant during the operation of the Segway, and an autonomous driving function that controls the operation of the Segway by autonomously updating the parameters without continuous manipulation by the rider. can be implemented.

The Segway driving state control method 300 according to the technical concept of the present invention is not limited to the order and steps of FIG. 8, and some of the order of each step may be omitted or the order of each step may be changed.

Claims (16)

In the reinforcement learning-based Segway-type robot driving state control system,
a travel data collection unit that collects travel data including inertia data measured by an IMU sensor of the robot and rotation data of a motor measured by an encoder of the robot;
a control logic operation unit that calculates a robot state value with the driving data obtained from the driving information collection unit and performs reinforcement learning to compare the value with a target robot state value; and
and a robot driving control unit generating a robot control signal for controlling a driving condition of the robot based on the robot state value obtained from the control logic operation unit. According to claim 1,
The robot is a Segway, the inertial data includes angular velocity values and acceleration values with respect to the x-axis, y-axis, and z-axis of the robot, and the rotation data includes rotation angle values for each wheel of the robot;
A robot driving state control system for recognizing a posture of the robot based on the inertia data and recognizing a movement of a wheel based on the rotation data. According to claim 2,
The control logic calculator calculates the mass value (m) and rotational inertia value (I) of the robot by taking the linear motion acceleration value (a) of the robot as an input value to obtain the robot state value. system. According to claim 1,
The reinforcement learning calculates the difference between the acquired robot state value and the preset target robot state value, and ends the learning when the difference value is recognized as 0 or within a reference range, and the difference value is outside the reference range. In this case, the robot driving state control system updates the robot state value. According to claim 1,
The reinforcement learning is performed by a simulation program or by receiving a robot state estimation value transmitted by a user terminal, the robot driving state control system. According to claim 1,
The robot state value includes at least one of mass, rotational inertia, inclination, tilt variation, speed, acceleration, torque of wheels, and angular velocity of wheels of the robot. According to claim 1,
The control logic calculation unit obtains linear motion speed data through the sum of the speeds of the left and right wheels of the robot, obtains rotational motion speed data through the speed difference between the left and right wheels of the robot, and estimates the robot state value. system. According to claim 1,
The robot driving state control system further comprising a robot driving control unit for controlling the operation of the motor by transmitting a robot control signal corresponding to the robot state value calculated by the control logic operation unit to the motor driver. In the method for determining and controlling the driving state of the Segway,
Collecting driving data of the Segway;
determining whether a wheel spins based on the driving data;
performing control logic for calculating a Segway state value and comparing it with a target Segway state value when the spin of the wheel does not occur; and
and controlling driving of a motor to correspond to the Segway state value. According to claim 9,
The Segway includes sensors that collect driving data,
The driving data includes inertia data including angular velocity values and acceleration values of the Segway and rotation data including rotation angle values for each wheel of the Segway. According to claim 9,
The Segway includes an encoder for measuring rotational angular displacement of the wheel; and
The Segway driving state control method further comprises a processor for determining the rotation of the wheel by comparing the number of wheel rotations measured by the encoder with the number of target wheel rotations. According to claim 9,
Whether or not the wheel spins is determined by comparing the linear movement distance and the rotational distance of the wheel, and determining that spin exists when the rotational distance of the wheel is greater than the linear movement distance. According to claim 9,
The Segway includes a hub motor providing driving force to the wheels; and
The Segway driving state control method further comprises a processor for obtaining acceleration data by applying output data of the hub motor to a preset algorithm. According to claim 9,
The Segway driving state control method further comprising the step of performing a control algorithm for determining whether a disturbance occurs based on a change in mass (m) or rotational inertia (I) generated during the driving process of the Segway. According to claim 9,
When it is determined that there is a difference between the Segway state value and the target Segway state value, reinforcement learning is performed to converge to the target Segway state value by iterating over the Segway state value. According to claim 9,
The Segway driving state control method of individually controlling driving of the wheels by transmitting a motor control signal corresponding to the Segway state value to a motor driver.

Images