JP7005872B2 - Convergence calculation support device, convergence calculation support method, and convergence calculation support program - Google Patents

Description

The present invention relates to a convergence calculation support device, a convergence calculation support method, and a convergence calculation support program.

Conventionally, a reinforcement learning system for improving the learning speed of an agent in the initial stage of learning has been proposed (for example, Patent Document 1).

Further, a technique has been proposed in which a set consisting of a training action sequence representing a series of states transitioned by an action and an action in each state, and a set of training order evaluation values representing the appropriateness of each training action sequence are stored. (For example, Patent Document 2). In this technique, the reward function is based on the difference between the set of ordinal evaluation values corresponding to the set of reward values of a series of states represented by the training action sequence obtained by using the reward function and the set of stored training order evaluation values. Update.

Japanese Unexamined Patent Publication No. 2006-309519 Japanese Patent No. 5815458

In recent years, with the development of analysis technology, large models such as the number of nodes and the number of elements have come to be used. However, when performing nonlinear analysis that requires convergence calculation, a large amount of time is spent on the analysis. There is a point.

However, in the techniques described in Patent Document 1 and Patent Document 2, reduction of analysis time in nonlinear analysis is not considered.

In view of the above facts, the present invention aims to reduce the analysis time spent on nonlinear analysis.

In order to achieve the above object, in the convergence calculation support device of the present invention, each of the plurality of states in the convergence calculation when performing the nonlinear analysis is associated with the action value of each of the plurality of solution methods to be selected. Using the obtained information, the selection unit that selects the solution method according to the current state, the selection of the solution method and the derivation of the predicted solution by the selected solution method are repeated until the selected solution method derives the convergent solution. It includes an update unit that updates the action value corresponding to the selected or unselected solution method by using the reward value according to the convergence period which is the period.

According to the convergence calculation support device of the present invention, the action value corresponding to the selected or unselected solution method is updated by selecting the solution method according to the action value and using the reward value according to the convergence period. ing. Therefore, by performing the nonlinear analysis using the updated behavioral value, the analysis time spent on the nonlinear analysis can be shortened.

In the convergence calculation support device of the present invention, the selection unit may select a solution method according to the weighting probability of each action value of the plurality of solution methods. As a result, it becomes easier to select a solution method having a high behavioral value, and as a result, the analysis time spent on the nonlinear analysis can be further shortened.

Further, in order to achieve the above object, in the convergence calculation support method of the present invention, each of the plurality of states in the convergence calculation when performing the nonlinear analysis and the action value of each of the plurality of solution methods to be selected are set. Using the associated information, the solution method according to the current state is selected, and the selection of the solution method and the derivation of the predicted solution by the selected solution method are repeated until the selected solution method derives the convergent solution. The computer executes a process of updating the action value corresponding to the selected or unselected solution method by using the reward value corresponding to a certain convergence period. Therefore, as with the convergence calculation support device, the analysis time spent on the nonlinear analysis can be shortened.

Further, in order to achieve the above object, in the convergence calculation support program of the present invention, each of the plurality of states in the convergence calculation when performing the nonlinear analysis and the action value of each of the plurality of solution methods to be selected are set. Using the associated information, the solution method according to the current state is selected, and the selection of the solution method and the derivation of the predicted solution by the selected solution method are repeated until the selected solution method derives the convergent solution. The purpose is to cause a computer to perform a process of updating the action value corresponding to a selected or unselected solution method by using a reward value corresponding to a certain convergence period. Therefore, as with the convergence calculation support device, the analysis time spent on the nonlinear analysis can be shortened.

According to the present invention, the analysis time spent on the nonlinear analysis can be shortened.

It is a block diagram which shows an example of the hardware composition of the convergence calculation support apparatus which concerns on embodiment. It is a figure which shows an example of the action value table which concerns on embodiment. It is a figure which shows an example of the reward table which concerns on embodiment. It is a block diagram which shows an example of the functional structure of the convergence calculation support apparatus which concerns on embodiment. It is a figure for demonstrating the state transition which concerns on embodiment. It is a figure for demonstrating the update process of the action value which concerns on embodiment. It is a flowchart which shows an example of the analysis process which concerns on embodiment. It is a perspective view which shows an example of the model of the analysis target which concerns on embodiment. It is a figure which shows an example of the action value table which concerns on a modification.

Hereinafter, examples of embodiments for carrying out the present invention will be described in detail with reference to the drawings. In this embodiment, as an example, a configuration example in which convergence calculation in non-linear analysis for analyzing displacement of a material when an external force (load) is applied due to an earthquake or the like is performed using reinforcement learning will be described. ..

First, with reference to FIG. 1, the hardware configuration of the convergence calculation support device 10 according to the present embodiment will be described. The convergence calculation support device 10 is realized by the computer shown in FIG. As shown in FIG. 1, the convergence calculation support device 10 includes a CPU (Central Processing Unit) 11, a memory 12 as a temporary storage area, and a non-volatile storage unit 13. Further, the convergence calculation support device 10 includes a display device 14 such as a liquid crystal display, an input device 15 such as a keyboard and a mouse, and a communication I / F (InterFace) 16 used when communicating with an external device. The CPU 11, the memory 12, the storage unit 13, the display device 14, the input device 15, and the communication I / F 16 are connected to the bus 17. Examples of the convergence calculation support device 10 include an information processing device such as a personal computer or a server computer.

The action value table 20, the reward table 22, and the analysis program 24 are stored in the storage unit 13. FIG. 2 shows an example of the action value table 20. As shown in FIG. 2, the action value table 20 stores the action value for each combination of a plurality of states in the convergence calculation when performing the nonlinear analysis and a plurality of solution methods to be selected. The action value table 20 is an example of information in which each of the plurality of states in the convergence calculation when performing the nonlinear analysis is associated with the action value of each of the plurality of solution methods to be selected.

In this embodiment, a case where the initial rigidity method is applied as the method A in the example of FIG. 2, the tangential rigidity method is applied as the method B, and the BFGS (Broyden, Fletcher, Goldfarb, Shanno) method is applied as the method C will be described. do. Although FIG. 2 illustrates the case where the number of states is three, the number of states is not limited to three, and may be two or four or more. Further, although FIG. 2 illustrates a case where the number of solution methods is three, the number of solution methods is not limited to three, and may be two or four or more. Further, as the solution method, a method using the nearest tangential rigidity, that is, the tangential gradient at the time of the previous update may be applied.

FIG. 3 shows an example of the reward table 22. As shown in FIG. 3, the reward table 22 stores reward values according to the convergence period. The reward value of the reward table 22 according to the present embodiment is set to a value in which the shorter the convergence period, the higher the reward. The details of the convergence period will be described later.

Next, with reference to FIG. 4, a functional configuration of the convergence calculation support device 10 according to the present embodiment will be described. As shown in FIG. 4, the convergence calculation support device 10 includes a selection unit 30, a derivation unit 32, and an update unit 34. By executing the analysis program 24 stored in the storage unit 13, the CPU 11 of the convergence calculation support device 10 functions as the selection unit 30, the derivation unit 32, and the update unit 34 shown in FIG.

The selection unit 30 refers to the action value table 20 and selects a solution method according to the current state. In the present embodiment, the selection unit 30 selects a solution method according to the current state according to the weighting probability of the action value of the action value table 20. Specifically, for example, when the current state is "first iteration", the selection unit 30 selects "method C" with a probability of 50% (= 4 ÷ (2 + 2 + 4) × 100). Further, in this case, the selection unit 30 selects "method A" or "method B" with a probability of 25% (= 2 ÷ (2 + 2 + 4) × 100).

The derivation unit 32 derives a predicted solution by using the solution method selected by the selection unit 30. The update unit 34 sets the convergence period, which is a period in which the selection unit 30 repeats the selection of the solution method by the selection unit 30 and the derivation unit 32 using the selected solution method repeats the derivation of the predicted solution until the derivation unit 32 derives the convergence solution. measure. The convergent solution here means a predicted solution in which the error from the true solution is within a predetermined range. Further, in the present embodiment, the selection of the solution method by the selection unit 30 once according to the current state and the derivation of the predicted solution by the derivation unit 32 using the selected solution method are referred to as “iteration”.

FIG. 5 shows an example of the state transition when the derivation unit 32 derives the convergent solution by five iterations. The "state A" in FIG. 5 corresponds to the "first iteration" in FIG. 2, the "state B" in FIG. 5 corresponds to the "error less than the threshold" in FIG. 2, and the "state C" in FIG. 5 corresponds to. Corresponds to "the error is equal to or greater than the threshold value" in FIG. Further, “the error is less than (or more) than the threshold value” in FIG. 2 means that the absolute value of the difference between the predicted solution derived by the derivation unit 32 and the true solution is less than (or more than) the predetermined threshold value TH. It means that.

In the example of FIG. 5, first, the method C is selected by the selection unit 30 in the first iteration (state A), and the prediction solution is derived by the derivation unit 32 using the method C. Then, since the absolute value of the difference between the derived predicted solution and the true solution is equal to or greater than the threshold value TH, the state transitions to the state C, and the current state becomes the state C.

Next, in the second iteration, the method A is selected by the selection unit 30, and the prediction solution is derived by the derivation unit 32 using the method A. Then, since the absolute value of the difference between the derived predicted solution and the true solution is equal to or greater than the threshold value TH, the state transitions to the state C, and the current state becomes the state C. Next, in the third iteration, the method B is selected by the selection unit 30, and the prediction solution is derived by the derivation unit 32 using the method B. Then, since the absolute value of the difference between the derived predicted solution and the true solution is equal to or greater than the threshold value TH, the state transitions to the state C, and the current state becomes the state C.

Next, in the fourth iteration, the method B is selected by the selection unit 30, and the prediction solution is derived by the derivation unit 32 using the method B. Then, since the absolute value of the difference between the derived predicted solution and the true solution is less than the threshold value TH and the predicted solution is not a convergent solution, the state transitions to the state B and the current state becomes the state B. Next, in the fifth iteration, the selection unit 30 selects the method B, and the derivation unit 32 derives the predicted solution using the method B. Then, since the absolute value of the difference between the derived predicted solution and the true solution is less than the threshold value TH and the predicted solution is a convergent solution, the process ends. The above-mentioned convergence period corresponds to the time required for these five iterations in the example shown in FIG.

The update unit 34 updates the action value corresponding to the selected solution method by using the reward value according to the convergence period. In the present embodiment, the update unit 34 first copies (copies) the action value table 20 stored in the storage unit 13 to the memory 12. The update unit 34 may duplicate the action value table 20 stored in the storage unit 13 in an area different from the area in which the action value table 20 of the storage unit 13 is stored. Further, in the following, the action value table 20 stored in the memory 12 by this duplication will be referred to as a “duplicate action value table”.

Further, the update unit 34 refers to the reward table 22 and acquires the reward value corresponding to the measured convergence period. Further, the update unit 34 updates the action value of the duplicate action value table by distributing the acquired reward value at a ratio according to the number of selections of the selected solution method according to the following equation (1). ..
Value to be distributed (added) to the action value = acquired reward value x (number of selections ÷ total number of iterations)

In the example of FIG. 5, method C is selected once in the state A, method B is selected once in the state B, method A is selected once in the state C, and method C is selected in the state C. Method B has been selected twice. Further, it is assumed that the convergence period in the example of FIG. 5 is 5 seconds or more and less than 20 seconds. That is, in the example of FIG. 5, as shown in FIG. 6 as an example, “2” (= 10 × (1/5)) is added to the action value corresponding to the combination of the state A and the method C. Similarly, in the example of FIG. 5, "2" is added to the action value corresponding to the combination of the state B and the method B, and "2" is added to the action value corresponding to the combination of the state C and the method A. , "4" is added to the action value corresponding to the combination of the state C and the method B.

In other words, in the present embodiment, the action value is updated so that the solution method selected when the convergence period is short has a higher action value.

In addition, the update unit 34 may reduce the action value of the unselected solution method instead of distributing the reward value to the selected solution method, or the update unit 34 may reduce the action value to the selected solution method. And reduce the action value of unselected solution methods.

Further, the update unit 34 repeats the above processing by the selection unit 30, the derivation unit 32, and the update unit 34 only for a predetermined number of analysis steps, and analyzes the finally obtained replication action value table in the actual production. It is stored in the storage unit 13 as the action value table 20 used for the above. The analysis step referred to here means, for example, a processing unit executed every predetermined period (for example, 1/100 second).

Next, with reference to FIG. 7, the operation of the convergence calculation support device 10 according to the present embodiment will be described. The convergence calculation support device 10 executes the analysis program 24 to execute the analysis process shown in FIG. 7. The analysis process shown in FIG. 7 is executed, for example, when an execution instruction of the analysis program 24 is input by the user via the input device 15.

In step S10 of FIG. 7, the update unit 34 duplicates the action value table 20 stored in the storage unit 13 in the memory 12. In step S12, the selection unit 30 selects a test analysis model similar to the production analysis model. The analysis model for this test may be stored in advance in the storage unit 13, or may be acquired from an external device via a network, for example. If there are a plurality of test analysis models similar to the actual analysis model, the selection unit 30 selects any one of the test analysis models. In this case, when step S12 is executed from the second time onward, the selection unit 30 selects an analysis model for testing that has not been selected so far. The following processes from step S14 to step S30 are performed on the analysis model selected by the process of step S12.

In step S14, the renewal unit 34 starts measuring the convergence period. In step S16, as described above, the selection unit 30 refers to the action value table 20 and selects a solution method according to the current state. In step S18, the derivation unit 32 derives a predicted solution using the solution method selected by the selection unit 30. In step S20, the derivation unit 32 shifts the state to the next state according to the predicted solution derived by the process of step S18. For example, when the absolute value of the difference between the predicted solution derived by the process of step S18 and the true solution is less than the threshold value TH, the next state is “state B”. The state of the transition destination in this step S20 becomes the current state when the next step S16 is executed.

In step S22, the derivation unit 32 determines whether or not the predicted solution derived by the process of step S18 is a convergent solution. If this determination is a negative determination, the process returns to step S16, and if the determination is affirmative, the process proceeds to step S24. In step S24, the renewal unit 34 ends the measurement of the convergence period. Further, the update unit 34 acquires the period from the start of the measurement by the process of step S14 to the end of the measurement by the process of this step S24 as the convergence period.

In step S26, the renewal unit 34 refers to the reward table 22 and acquires the reward value corresponding to the measured convergence period. In step S28, as described above, the update unit 34 distributes the reward value acquired by the process of step S26 at a ratio according to the number of selections of the selected solution method, thereby performing the action of the duplicate action value table. Update the value.

In step S30, the update unit 34 determines whether or not the processes from step S14 to step S28 have been completed for all the analysis steps in the predetermined number of analysis steps. If this determination is a negative determination, the process returns to step S14, and if the determination is affirmative, the process proceeds to step S32.

In step S32, the update unit 34 determines whether or not the processing from step S12 to step S30 has been completed for all the analysis models for testing similar to the actual analysis model. If this determination is a negative determination, the process returns to step S12, and if the determination is affirmative, the process proceeds to step S34.

In step S34, the update unit 34 outputs (stores) the duplicate action value table stored in the memory 12 to the storage unit 13 as the action value table 20 used for the actual analysis. When the process of step S34 is completed, the analysis process is completed.

For the actual analysis model, the same process is executed using the actual action value table 20 stored in the storage unit 13 by the above analysis process. In this case, the analysis processing steps S16, S18, S20, S22, and S30 shown in FIG. 7 are executed for the actual analysis model. Further, in this case, the action value may be updated by further executing steps S10, S14, S24, S26, S28, and S34 of the analysis process shown in FIG. 7. Further, in this case, instead of updating the duplicate action value table, the action value of the action value table 20 for production may be updated. In this case, it is not necessary to execute steps S10 and S34 of the analysis process shown in FIG. 7, and the action value of the action value table 20 for production may be updated in step S28.

As an example, the analysis time when the analysis model shown in FIG. 8 is analyzed under the following analysis conditions (A) to (C) will be described.
(A) The material conditions are as shown below.
G = 18000 [kPa], ν = 0.3, ρ = 1.8 [t / m ³ ], nonlinear Von Mises σy = 50 [kPa], Hardening = 2000.0 [kPa]
In addition, G indicates shear rigidity, ν indicates Poisson's ratio, and ρ indicates specific gravity.
(B) The boundary conditions are fixed on the bottom surface and repeated boundary on the side surface.
(C) As for the load, −2000.0 [kN] is assumed to be loaded on the shaded portion in the center of the upper surface shown in FIG. 8, and the number of analysis steps is 10.

Under the above analysis conditions, the analysis time when only the initial rigidity method was used as the solution method was about 35 seconds, and the analysis time when only the tangential rigidity method was used as the solution method was about 210 seconds.

On the other hand, under the same analysis conditions, when either the initial rigidity method or the tangential rigidity method is selectively used as the solution method as in the present embodiment, the tangential rigidity is obtained only by the 13th iteration of each analysis step. When the method was used and the initial stiffness method was used for other iterations, the analysis time was about 27 seconds. That is, the analysis time could be shortened by selectively using a plurality of solution methods.

As described above, according to the present embodiment, in the nonlinear analysis, the action value corresponding to the selected solution method is updated by using the reward value according to the convergence period. Therefore, by performing the nonlinear analysis using the updated behavioral value, the analysis time spent on the nonlinear analysis can be shortened.

Further, according to the present embodiment, the solution method is selected according to the weighting probability of the action value. Therefore, as a result of making it easier to select a solution method having a high behavioral value, it is possible to further shorten the analysis time spent on the nonlinear analysis.

In the above embodiment, the case of updating the action value of the duplicate action value table has been described, but the present invention is not limited to this. It may be a form of updating the action value of the action value table 20. In this case, it is not necessary to execute steps S10 and S34 of the analysis process shown in FIG. 7, and the action value of the action value table 20 may be updated in step S28. In this embodiment, in the next step S16, the solution method is selected with reference to the action value table 20 in which the action value is updated.

Further, the state of the action value table 20 in the above embodiment may be defined differently. For example, the state of the action value table 20 may be defined by information indicating the number of iterations. FIG. 9 shows an example of the action value table 20 in this case.

Further, in the above embodiment, the disc I / O amount may be used as an evaluation condition for analysis in addition to the convergence period. In this case, for example, the shorter the convergence period and the smaller the amount of disk I / O, the higher the reward.

In the above embodiment, the case where the solution method is selected according to the weighting probability of the action value has been described, but the present invention is not limited to this. For example, one of the solution methods having an action value of a predetermined value or more may be randomly selected.

Further, although the processing performed by the CPU 11 in the above embodiment has been described as software processing performed by executing a program, it may be processing performed by hardware. Further, the process performed by the CPU 11 may be a process performed by combining both software and hardware.

Further, in each of the above embodiments, the embodiment in which the analysis program 24 is stored (installed) in the storage unit 13 in advance has been described, but the present invention is not limited to this. Even if the analysis program 24 is provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. good. Further, the analysis program 24 may be downloaded from an external device via a network.

10 Convergence calculation support device 11 CPU
12 Memory 13 Storage unit 20 Action value table 22 Reward table 24 Analysis program 30 Selection unit 32 Derivation unit 34 Update unit

Claims (4)

Select the solution method according to the current state by using the information associated with each of the multiple states in the convergence calculation when performing the nonlinear analysis and the action value of each of the multiple solution methods to be selected. Selection part and
Selected or unselected using the reward value according to the convergence period, which is the period during which the selection of the solution method and the derivation of the predicted solution by the selected solution method are repeated until the selected solution method derives the convergent solution. The update unit that updates the action value corresponding to the solution method of
Convergence calculation support device including. The convergence calculation support device according to claim 1, wherein the selection unit selects a solution method according to a weighted probability of the action value of each of the plurality of solution methods. Select the solution method according to the current state by using the information associated with each of the multiple states in the convergence calculation when performing the nonlinear analysis and the action value of each of the multiple solution methods to be selected. ,
Selected or unselected solution method using the reward value according to the convergence period, which is the period in which the selection of the solution method and the derivation of the predicted solution by the selected solution method are repeated until the selected solution method derives the convergent solution. Convergence calculation support method in which a computer executes a process of updating the action value corresponding to the above. Select the solution method according to the current state by using the information associated with each of the multiple states in the convergence calculation when performing the nonlinear analysis and the action value of each of the multiple solution methods to be selected. ,
Selected or unselected solution method using the reward value according to the convergence period, which is the period in which the selection of the solution method and the derivation of the predicted solution by the selected solution method are repeated until the selected solution method derives the convergent solution. A convergence calculation support program that causes a computer to execute a process that updates the action value corresponding to the above.

Images