JPH04211801A - Learning controller - Google Patents

Abstract

PURPOSE:To reduce the necessary number of times of repetition in a learning controller to detect output (y) obtained by changing input U to a controlled object by a very small amount, and try repeatedly this very small change until the output (y) coincides with a target value yd. CONSTITUTION:In a qualitative model arithmetic circuit 303 and an input change vector selection circuit 309, since only an input change vector U1 capable of bringing the output (y) close to the target value yd is selected, and a trial is executed in respect of this vector, it is unnecessary to execute the trial for all input change vectors like in the conventional one, and the number of times of repetition can be greatly reduced. Further, in the case that a state varies, and the output (y) tends to go away from the target value yd, since a qualitative model is corrected by a qualitative model correction circuit 312 so that the output (y) approaches to the target value yd an effect to reduce the number of times of the repetition can be kept in all the states.

Description

[Detailed description of the invention]

0001

[Field of Industrial Application] The present invention relates to a learning control device capable of controlling control objects for which it is difficult to accurately grasp the relationship between input and output in advance, such as walking robots and chemical plants. It is something.

0002

[Prior Art] Conventional learning control devices include, for example,
Paper ``Machine that acts'' (Science of living organisms, Vol. 37,
No. 1, pp. 41-48, 1986), proposed by Nakano. This paper discusses the control of the walking robot shown in Figure 2.

[0003] In FIG. 2, a walking robot 105 is composed of front legs 102A and hind legs 102B, which are connected by a body 100. Furthermore, the front legs 102A and the rear legs 102B are provided with motors 103A and 103B, respectively.
The rotation of each motor is driven by driver circuit 1.
Commanded by 04. Further, the distance traveled by the walking robot is detected by the output detector 106.

Walking robot 10 configured as described above
The operation of No. 5 can be expressed as in equation (2).

[0005]

[Math 2]

Here, y is the walking distance which is the output of the walking robot, U=(u1A, u1B, u2A, u2B) is the motor rotation angle vector which is the input vector to the front leg 102A and the rear leg 102B of the walking robot, and g is a function that is difficult to understand accurately.

[0007] In order to obtain U such that y in equation (2) is as large as possible, conventional learning control devices generally use a "hill climbing method" consisting of the following steps.

Step 1: For example, (△u1A,0,0,0), (0
, -△u1B, △u2A, △u2B), the input change vector △Ui has minute values in each element, such as △U1,...,
Create △U81 and 81 pieces. In this example, the number of input change vectors is 34=81, and "3" corresponds to the number of types of signs for each element, that is, "+", "-", or "0", and the power number " 4” is the input change vector △Ui
corresponds to the order of

Step 2: Add the above input change vectors one by one to the current input vector U, that is, Ui←U+△Ui
input to the walking robot as , and the output change at that time △y
1,. ．． ．． , Δy81 are detected.

Step 3: Select the input change vector △Uj that maximizes the above output change, and change the current input vector U by U←U
Update +ΔUj and repeat steps 2 and 3. however,
If all of the above output changes are negative or zero, the current input vector is the desired vector, and the above iteration ends.

[0011]

[Problem to be solved by the invention] This learning control device
It has the advantage that it can be applied not only to walking robots but also to any controlled object whose characteristics are unknown, using exactly the same configuration. However, 81 trials are required in step 2, and if we assume that the number of repetitions of steps 2 and 3 required for the output y to reach its maximum value is 10, a total of 810 trials will be repeated. There was a practical problem that it had to be done.

0012

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a learning control device that requires a significantly smaller number of repetitions than conventional learning control devices.

[0013] In order to achieve this object, the present invention has the following configuration. That is, means for generating a plurality of input change vectors ΔUi for changing the control input U applied to the controlled object;
Predicted code data is obtained by performing calculations on i based on a predetermined qualitative model.

[0014]

[Math 3]

Qualitative model calculating means for outputting y, detecting means for detecting the output y of the controlled object, and error sign detecting means for detecting the sign of the difference between the detected value y of the detecting means and the target value yd. an input change vector selection circuit that selects the input change vector ΔUi based on the output [e] of the error code detection means and the predicted code data (Equation 3); output sign detection means for detecting a predetermined sign expressed by the control object; input vector updating means for adding the input change vector selected by the input vector selection circuit to the input of the controlled object; and detection of the input and output sign of the controlled object. The present invention provides a learning control device characterized by comprising: qualitative model modification means for modifying the qualitative model based on a detection output of the means.

[0016]

[Operation] According to the present invention, in the qualitative model calculation means and the input change vector selection means, the input change vector △Uj that can bring the output y closer to the desired target value yd
Since only one of the input change vectors is selected and tested, there is no need to try all the input change vectors as in the conventional method, and the number of repetitions until the output y matches the target value yd can be extremely reduced. If the state further changes and the output y tends to deviate from the target value yd, the qualitative model correction means changes the output y to the target value yd.
Since the qualitative model is modified to approach , the effect of reducing the number of repetitions can be maintained in all conditions.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a learning control device in a first embodiment of the present invention. In FIG. 1, the controlled object is a walking robot 105 shown in FIGS. 3(a) and 3(b). In FIG. 3(a) and FIG. 3(b),
The walking machine 105 has front legs 102A and hind legs 102B attached to a body 100, each of which is powered by a motor 1.
It is configured to be able to rotate at 03A and 103B. The respective friction coefficients of the front foot tip 102C and the rear foot tip 102D that are in contact with the floor 101 are different from each other. Also, the distance traveled by the walking robot is measured by the output detector 106.
Detected in

The operation of the walking robot 105 described above will be explained below. The input vector U given to the walking robot is expressed by equation (4).

[0019]

[Math 4]

In equation (4), u1A is the angle of the front foot before the movement, u1B is the angle of the front foot after the movement, u2A is the angle of the hind foot before the movement, and u2B is the angle of the hind foot after the movement.

The control input U is a vector quantity, and its elements u1A, u1B, u2A and u2B are all defined as real numbers.

The front legs 102A and the rear legs 102B are rotated by respective motors 103A and 103B as shown in FIGS. 3(a) and 3(b). As a result, the tip of the front foot is 10
If the frictional forces of the foot 2C and the rear foot 102D against the floor 101 are not the same, the walking robot 105 moves in a certain direction.

The walking robot moves from the state shown in FIG. 3(a) to the state shown in FIG. 3(b), then returns to the state shown in FIG. 3(a) again, and completes one cycle of walking motion. Therefore, equation (4) represents the motion of the walking robot in a half cycle.

The walking robot 105 is shown in FIGS. 3(a) and 3(a).
The distance traveled by one cycle of walking motion shown in b) is y
Then, the relationship between the control input U and the distance y is expressed by equation (2). The function g in this equation (2) is based on the weight distribution of the walking robot 105 between the front legs 102A and the hind legs 102B, the ratio of the length L1 of the front legs 102A to the length L2 of the hind legs 102B, and the floor 101 and the tip of each foot 102C. , 102D
It changes depending on the friction coefficient between the two.

In FIG. 1, the learning control device of the first embodiment determines the input vector to be input to the walking robot based on the output of the input change vector determination circuit 310 and the input change vector determination circuit 310, which determines the input change vector. An input vector update circuit 311 that updates the distance detector 106
It has an output sign detection circuit 313 that detects the sign of the moving direction (a fixed direction is defined as positive or negative) from the output of the output, a qualitative model correction circuit 312, and an error sign detection circuit 308.

The input change vector determining circuit 310 has the following circuit. (1) Input change vector memory 301: 81 predetermined input change vectors ΔU1,...,ΔU8
1 is stored in memory. The number of input change vectors ΔUi is determined by the method described in the "Prior Art" section. The input change vector ΔUi consists of four data (Δu1A, Δu
1B, Δu2A, Δu2B), and each data is either a positive value, a negative value, or zero. For example, (Δu1
A, 0, 0, 0), (0, -Δu1B, Δu2A, Δu
2B). Positive values represent an increase in the predetermined direction, and negative values represent a decrease. Zero represents no change. Each data (Δu1A, Δu1B,
Δu2A, Δu2B) are front legs 102A and hind legs 102B
This is a minute angle added to the rotation angle of , and a minute value such as 2 degrees is set, for example. It is not necessary for each data to be the same angle, and different values may be set (eg, 2, -3°, 0°, 2°). (2) Switch 305A: Closed when inputting data from input change vector memory 301 to code vector detector 302. (3) Sign vector detector 302: Based on the input change vector ΔUi input from the input vector memory 301, the sign (+, -,
A code vector [ΔUi] representing 0) is output. (Hereafter, characters placed in brackets [ ] indicate the sign "+", "-", or "0" of the data represented by that character.) For example, input change vector ΔUi = (0, -Δu1B, Δu2A,
When Δu2B) is input, the code vector [ΔUi]=
(0, -, +, +) is output. (4) Qualitative model calculation circuit 303: Based on the code vector [ΔUi] output from the code vector detector 302, the sign of the output y representing the moving distance and moving direction of the walking robot 105 (corresponding to the moving direction)
It has an arithmetic circuit that predicts. The calculation is performed according to a preset qualitative model, and the resulting predicted sign data is

[0027]

[Math 5]

##EQU1## is output. From now on, the hat above the letters “＾
” represents the predicted data of the data represented by that character. The predicted code data (Equation 5) represents the sign indicating the direction of change in the output y, with “+” for predicted increase, “-” for predicted decrease, and no change. is “0” and unpredictable is “?” (5) Switch 305B: Transfers the output data of the qualitative model calculation circuit 303 to the memory 304.
Closed when typing. (6) Memory 304: The predicted code data (Equation 5) output from the qualitative model calculation circuit 303 is stored in the memory 304 via the switch 305B. Normally 81 predicted code data

[0029]

[Math 6]

##EQU1## is stored in memory. (7) Input change vector selection circuit 309: memory 30
Predicted code data from 4 (Equation 5) and input change vector Δ
Ui is input, and all its predicted code data (Equation 6)
One predicted code data whose code matches the sign [e] of the error value input from the error code detection circuit 308 described later.

[0031]

[Math 7]

##EQU1## is selected and applied to the qualitative model modification circuit 311. This learning control device further includes the following circuit. The error sign detection circuit 308 includes an error calculation circuit 306 that calculates the difference between the value y detected by the distance detector 106 and the target value yd, and inputs the calculation result error e to the sign detection circuit 307. In the code detection circuit 307,
The sign [e] of the value of the error e is detected and input to the input change vector selection circuit 309. Code [e] is “+”, “-”
, "0". That is, the code [e] has information as to whether the output y should be increased or decreased in order to bring it closer to the target output yd, or whether the current value should be maintained.

The input vector update circuit 311 performs an addition operation on the input change vector ΔUj output from the input change vector selection circuit 309 and the current input U, and outputs an updated new input U. Switch 316 is open during the above addition operation.

The input U and predicted code data (Equation 7) are input to the qualitative model correction circuit 312. Furthermore, when the output sign detection circuit 313 detects a sign change vector [Δy] representing the direction of change in the moving distance, the switch 314 is closed (steps 1 and 2 in the flowchart of FIG. 4).
The sign change vector [Δy] is the qualitative model correction circuit 312
(Step 3).

In the qualitative model correction circuit 312, the sign change vector [Δy] and the predicted sign data (Equation 7) are compared (step 4), and if the two are not equal, the switch 315 is closed and the correction outputs QA and QB are The data is input to the qualitative model calculation circuit 303 (steps 5 and 6).

The qualitative model will be explained below. Figure 3 shows the walking robot opening its front legs 102A and hind legs 102B (
Both legs 102A, 102 shown in FIG. 3(b) from the posture of a)
When moving B to the closed position, if the friction force at the front foot tip 102C is greater than the friction force at the rear foot tip 102D, the front foot tip 10
2C does not slide on the floor 101, only the tip of the rear foot 102D slides on the floor 101, and the walking robot moves by a distance yAB as shown in FIG. In this case, the larger the amount of change in the angle of the front foot 102A (u1A-u1B), the larger the distance yAB of movement. Therefore, the amount of rotation of the hind leg 102B does not contribute to the distance traveled. As a result, the moving distance yAB due to the change in posture is expressed by equation (8).

[0037]

[Math. 8]

Here, F1A is the frictional force of the front foot tip 102C, and F2A is the frictional force of the rear foot tip 102D.

[0039] g1 and g2 are increasing functions, and g1(0)
=g2(0)=0. In equation (8), it is necessary to determine the sign of the value of equation (F1A-F2A), but it is extremely difficult to detect these frictional forces. Therefore, an expression equivalent to this expression (F1A-F2A) is expressed using input vectors (u1A, u1B, u2A, u2B) that are detectable angle data.

In the equation (8), the equation (F1A-F2A=
0) indicates that the friction force between the front foot tip 102C and the rear foot tip 102D is equal. Length L1 of front leg 102A and hind leg 1
Assuming that the length L2 of 02B is equal, the friction coefficient μ1 between the front foot 102A and the floor 101, and the friction coefficient μ2 between the rear foot 102B and the floor 101 are equal, the formula (F1A-F2A=0)
is equivalent to the formula (u1A-u2A=0).

The above relationship is generally expressed by equation (9).

[0042]

[Math. 9]

Here, QA is a boundary parameter that varies depending on the relationship between L1, L2, μ1, μ2, and therefore u
2A-u1A-QA is a boundary function consisting of an input and a boundary parameter, and has the same dimension as the input. However, L1=
When L2 and μ1=μ2, QA=0.

When formula (4) and formula (8) are combined, formula (10) is obtained.

[0045]

[Math. 10]

Considering the same way, FIGS. 3(b) to 3(a)
) The walking distance yBA when changing to ) is expressed by equation (11).

[0047]

[Math. 11]

[0048] Furthermore, the walking robot moves from Fig. 3(a) to Fig. 3(
When changing from b) to FIG. 3(a), the walking distance y is expressed by equation (12).

[0049]

[Math. 12]

To summarize equations (9) to (11), we get
(Table 1).

[0051]

[Table 1]

In (Table 1), the area numbers (1 to 9) are the inputs U=(u1A, u1B, u2) given to the walking robot.
A, u2B) and the boundary parameters QA, QB). In equation (10), the area is the input value (u1A-u2A)
It can be divided into three types based on the sign of the difference value between and the boundary parameter QA. Also, in (Equation 11), the input value (u2B-u
1A) and the boundary parameter QB can be divided into three regions based on the sign of the difference value. Therefore, it is divided into 9 (3×3=9) regions, and the function for determining the walking distance y is different in each region.

The sign of the value of the boundary function is obtained as follows. For example, in area number (1), for boundary function code [u2A-u1A-QA], u2A-u1A-
Since QA>0, the sign of the value is "+". Similarly, in region number (2), boundary function code [u2
B-u1B-QB], u2B-u1B-QB=
Since it is 0, its value is "0".

The output value y for each area number is determined as follows. That is, in area number (1), from equation (10), yAB=g1(u1A-u1B), ( equation 1
From formula 1), yBA=-g1(u1A-u1B), so the walking distance y is

[0055]

[Math. 13]

[0056] Also, in area number (2), from equation (10), yAB=g1(u1A-u1B), (formula 1
From formula 1), yBA=0, so the walking distance y is

00
57]

[Math. 14]

[0058] Since the functions g1 and g2 are increasing functions, it is possible to predict the sign of the output relative to the sign of the input vector value. This “sign prediction” is performed based on the “qualitative model” set in the qualitative model calculation circuit 303. (Table 2) represents this "qualitative model", with boundary function codes [u2A-u1A-QA] and [
Predicted code data (Equation 3) corresponding to the code combination [u2B-u1B-QB] is shown.

[0059]

[Table 2]

In (Table 2), the predicted code data (Equation 3
) can be obtained as follows. For example, area number (1)
In the case, code vector [△Ui] = (+, 0, −,
+), the predicted code data (Equation 5) is “0”. (No matter what value the code vector [△Ui] takes, the predicted code data

[0061]

[Math. 15]

[0062] ) In the case of area number (2), for example, the predicted code data (Equation 5) becomes "+" for code vector [ΔUi]=(+, -, -, +).

[0063]

[Math. 16]

For example, code vector [△Ui]=
For (+, +, -, +), predicted code data (Equation 5
) has no fixed value.

[0065]

[Math. 17]

The output of the qualitative model correction circuit 312 is the friction coefficient μ1 between the front foot tip 102C and the floor 101, and the rear foot tip 10
It includes a coefficient of friction μ2 between 2D and the floor 101, and boundary parameters QA and QB determined by the respective lengths of the front foot 102A and the rear foot 102B. The friction coefficients μ1 and μ2 are data that are difficult to measure and cannot be predicted. Therefore, the boundary parameters QA and QB that include them cannot be accurately predicted, and the predictions in (Table 2) are not necessarily correct. . If this prediction is incorrect, the code data of the actual output value detected by the output code detection circuit 313 [
Δy] and the predicted code data (Equation 3) output from the input vector selection circuit 309 do not match. In such a case, the qualitative model used by the qualitative model calculation circuit 303 is considered to be inappropriate, so the boundary parameters QA and QB of the qualitative model are changed.

An example of a correction operation applying actual numerical values is shown below. The input of the walking robot is

[0068]

[Math. 18]

[0069] If QA=20° and QB=10°, then from equation (10),

[0070]

[Math. 19]

[0071] Also, from equation (11),

[0072]

[Math. 20]

Region number (2) in Table 2 is selected from the calculation results of equations (19) and (20).

At this time, it is assumed that the following data, for example, is input as the input change vector.

[0075]

[Math. 21]

In this case, the predicted code data (Equation 3) is calculated from (Table 2) as follows.

[0077]

[Math. 22]

[0078] Next, if the code data [Δy] of the walking robot given the above input change vector after the completion of the walking motion becomes "-", it is predicted that the selection of the area number is incorrect. . Therefore, in (Table 2), a region number where the predicted code data (Equation 3) becomes "-" is searched. As a result, it is found that the matching area number is (4) ((
From the calculation of equation 20).

Therefore, in the data of equations (18) and (21), boundary parameters QA and QB that match the boundary function of area number (4) are determined.

From equation (10) and equation (11),

008
1]

[Math. 23]

In order for the above two equations to hold true, QA', QB
The value of ' can be set as follows.

[0083]

[Math. 24]

[0084] Here, "ε" is a positive real number. On the other hand, if the sign data [Δy] is “+”

[0085]

[Math. 25]

Therefore, the predicted code data and the code data match. Therefore, the boundary parameters QA and QB are not modified.

[0087] The friction coefficients of both feet are equal (μ1=μ2),
And when the lengths of the front and hind legs are equal (L1=L2), QA=QB=0. Therefore, we do not modify the qualitative model. As a result, a qualitative model correction circuit 312, an output change sign detection circuit 313, and switches 314 and 315
It is possible to use the circuit of FIG. 6 without.

Furthermore, although this embodiment applies learning control to a walking robot, the learning control of the present invention can also be applied to chemical plants, air conditioning systems, and the like.

[0089]

As described above, according to the present invention, the qualitative model calculation circuit 303 and the input change vector selection circuit 309 select only the input change vector △Uj that can bring the walking distance y closer to the desired target walking distance yd. However, since the walking motion is performed only for this, there is no need to try all the input change vectors as in the conventional case, and the number of repetitions of the walking motion until the target walking distance yd is reached can be extremely reduced. Furthermore, the friction coefficients μ1 and μ2, the length L1 of the front foot 102A, and the rear foot 1
When the length L2 of 02B changes and the walking distance y tends to move away from the target walking distance yd, the qualitative model is corrected in the qualitative model correction circuit so that the walking distance y approaches the target walking distance yd. The effect of reducing the number of repetitions can be maintained. In actual experiments, in order to reach the same target walking distance yd, the conventional example required approximately 810 trials as mentioned above, whereas the present invention can achieve this in approximately 10 trials, which is a significant I was able to confirm the effect.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a learning control device in a first embodiment of the present invention.

FIG. 2 is a perspective view of a walking robot, which is an example of the object to be controlled by the learning control device of the present invention.

FIG. 3(a) is a front view showing an example of the operation of a walking robot, which is an example of the object to be controlled by the learning control device of the present invention. (b) is a front view showing an example of the operation of a walking robot, which is an example of the object to be controlled by the learning control device of the present invention.

FIG. 4 is a flowchart showing the operations of a qualitative model correction circuit and an output sign detection circuit in the learning control device according to the first embodiment of the present invention.

FIG. 5 is a front view showing a walking robot in operation, which is an example of the object to be controlled by the learning control device of the present invention.

FIG. 6 is a block diagram of a learning control device in a second embodiment of the present invention.

[Explanation of symbols]

100 Torso 101 Floor 102A front leg 102B Hind leg 102C Front foot tip 102D Hind foot tip 103A motor 103B Motor 104 Driver circuit 105 Walking robot 106 Output detector 305A, 305B switch 306 Error calculation circuit 308 Error sign detection circuit 310 Input change vector determination circuit 311 Input vector update circuit 314 Switch 315 Switch 316 Switch

Claims (5)

[Claims] 1. Means for generating a plurality of input change vectors ΔUi for changing a control input U applied to a controlled object;
qualitative model calculating means for performing calculations on the input change vector ΔUi based on a predetermined qualitative model and outputting predicted code data [Equation 1]; a detecting means for detecting the output y of the controlled object; and a detecting means for detecting the output y of the controlled object. an error sign detection means for detecting the sign of the difference between the value y and the target value yd, and the input change vector Δ based on the output [e] of the error sign detection means and the predicted sign data (Equation 1)
an input change vector selection circuit for selecting Ui; an output sign detection means for detecting a predetermined sign representing a change in the value of the output of the controlled object; and a qualitative model modification means for modifying the qualitative model based on the input of the controlled object and the detection output of the output sign detection means, and by repeating the above series of operations. A learning control device that makes the output y of the controlled object match a target value Yd. 2. The qualitative model calculation means uses an input vector U, a boundary function having at least one boundary parameter, and at least one qualitative formula corresponding to the sign of a value obtained by substituting the input vector into the boundary function. The learning control device according to claim 1, further comprising a qualitative model represented. 3. The learning control device according to claim 2, wherein the qualitative model modification means includes means for changing boundary parameters. 4. Means for generating a plurality of input change vectors ΔUi for changing the control input U applied to the controlled object;
a qualitative model calculating means for performing calculations on the input change vector ΔUi based on a predetermined qualitative model and outputting predicted code data (Equation 1); a detecting means for detecting the output y of the controlled object; and a detecting means for detecting the output y of the controlled object. an error code detection means for detecting the sign of the difference between the value y and the target value yd, and an output [e] of the error code detection means and the predicted code data (
An input change vector selection circuit that selects the input change vector ΔUi based on Equation 1), and input vector update means that adds the input change vector selected by the input vector selection circuit to the input of the controlled object. , a learning control device that makes the output y of the controlled object match the target value Yd by repeating the above series of operations. 5. The qualitative model calculation means comprises an input vector U, a boundary function having at least one boundary parameter, and at least one qualitative formula corresponding to the sign of a value obtained by substituting the input vector into the boundary function. 5. The learning control device according to claim 4, further comprising a qualitative model represented by .