JP7376450B2 - A method for constructing a trained model and a design support device using the trained model - Google Patents

Description

The present disclosure relates to a learned model construction method, and particularly to a learned model construction method related to a design support device for providing design support, and a design support device using the learned model.

A design support system for supporting vehicle design, which automatically changes the specification values of other related elements based on the input specification values input when the design of one element is changed. A device has been developed (for example, Patent Document 1).

A design support device for supporting the design of buildings that accepts input of image information and linguistic information from the architect, performs calculations using a trained model, and outputs the design conditions of the building being designed. known (for example, Patent Document 2).

Japanese Patent Application Publication No. 2009-37509 JP2019-200721A

In order to design a vehicle, it is necessary to go through many activities such as setting goals and considering load input. In the design support device of Patent Document 1, although changing one element changes the specification values of other elements, it is difficult for an inexperienced user to know what activities to perform at each stage of vehicle design. Therefore, it is difficult to provide sufficient design support to inexperienced users.

Therefore, it is conceivable to configure the design support apparatus so as to be able to support design using a learned model as described in Patent Document 2.

It is conceivable to construct a trained model using reinforcement learning, which maximizes the value obtained by sequentially performing the actions in a state that represents the design situation, by assigning rewards to the actions that should be taken in that state. At this time, it is required to set the value so that the action selected to maximize the value matches the action of a user (expert) with sufficient experience.

However, as in vehicle design, when the number of selectable activities in each state increases, the number of rewards to be set increases, and there is a problem that the burden associated with setting rewards increases.

In view of the above background, it is an object of the present invention to provide a method for constructing a trained model using reinforcement learning and a design support device using the trained model to facilitate setting of rewards.

In order to solve the above problems, an aspect of the present invention provides a state (S) that is determined depending on whether or not a design activity is executed, and an action (A) that is the activity that can be selected under the state. A learned model construction method that constructs a trained model (50) by giving a reward (R) to a combination of and maximizing the value, the method comprising: The rule includes a step of providing the reward (ST4) and a step of performing reinforcement learning based on the reward that has been provided, and the rule is based on a route source activity and a route destination activity that is performed after the route source activity. and a thickness indicating the importance of the route destination activity being performed after the route source activity, and in the step of giving the reward, the behavior immediately before leading to the state is the route source activity. and when the action and the route destination activity match, the reward is set based on the thickness.

According to this aspect, by inputting rules based on the expert's experience and knowledge, it is possible to easily and appropriately set a reward that matches the expert's behavior.

In the above aspect, the state may be defined by a combination including whether or not a plurality of the activities are executed (T, F) and the activity that was executed immediately before.

According to this aspect, it is possible to easily determine whether a combination of a state and an action matches a rule according to an expert's design procedure using the activity and action immediately before the state is reached.

In the above aspect, the state is defined by a combination including whether or not a plurality of the activities are executed, the activity executed immediately before, and a number (a, b, u) corresponding to a design condition. good.

According to this aspect, design support can be provided according to the design conditions.

In the above aspect, the number corresponding to the design condition may include the number (u) indicating that the design condition is undetermined.

According to this aspect, even if design conditions are undetermined, design support can be provided.

In the above aspect, a knowledge graph (21) including a plurality of nodes (22) corresponding to the activities related to the design and edges (23) connecting the corresponding nodes based on the relationship between the activities is used. , the step of extracting the activity for defining the state by matching with the rule from among the activities corresponding to the node, and generating a state space (P) that is a set of the states. good.

According to this aspect, a state can be generated using a knowledge graph in which information related to design is stored, and design support can be performed.

In the above aspect, the knowledge graph preferably includes design information and data related to a design procedure.

According to this aspect, since the knowledge graph includes data related to design information and data related to design procedures, the usefulness of the knowledge graph is enhanced.

In the above aspect, the nodes are recorded in a hierarchical manner in the knowledge graph, and the rule is such that the node corresponding to the route source activity is located above the node corresponding to the route destination activity. In some cases, the thickness may be a predetermined value.

According to this aspect, since the activity instructions are given so as to descend from the activities corresponding to the nodes located in the upper layer of the knowledge graph toward the activities corresponding to the nodes located in the lower layer, the user does not understand the contents of the instructions. It becomes easier.

In the above aspect, the rule may be such that the reward is not given when a difference in hierarchy between the node corresponding to the route source activity and the node corresponding to the route destination activity is a predetermined value or more.

According to this aspect, it is possible to prevent the difference in hierarchy between the node corresponding to the route source activity and the node corresponding to the route destination activity from becoming large and making it difficult for the user to understand the flow of the activity.

In the above aspect, the rule is input by text, and in the step of generating the state space, the activity corresponding to the node is determined by matching words included in the text with the activity corresponding to the node. It is preferable to extract the activity for defining the state from among the activities.

According to this aspect, it is possible to easily match the activity corresponding to the node with the rule.

In the above aspect, the design support apparatus (1, 101) performs design support based on the trained model constructed by the above method, and the step (1) of setting an initial state based on information input by the user ST24) and a step (ST25) of outputting an instruction to the user to sequentially perform the actions that maximize the value from the initial state.

According to this aspect, the design support device can be configured to perform design support based on a learning model that matches the expert's behavior by inputting rules based on the experience and knowledge of the expert.

In the above aspect, the design support device (101) performs design support based on the learned model constructed by the above method, and a step (ST24) of setting an initial state based on information input by the user. and a step (ST25) of outputting instructions to the user to sequentially perform the actions that maximize value from the initial state, and in the step of outputting, When an input is received, the state is changed to one that matches the input condition, and then the output is performed to instruct the user to sequentially perform the actions that maximize value.

According to this aspect, the design support device can be configured to be able to provide design support based on a learning model that matches the behavior of an expert. Moreover, even if the design conditions are changed, the design support apparatus can be configured to be able to support the design using a learning model that matches the changed conditions.

According to the above configuration, it is possible to easily set a reward in a method for constructing a trained model using reinforcement learning and a design support device using the trained model.

A block diagram showing the hardware configuration of a design support device that implements the learned model construction method according to the first embodiment. Functional block diagram of design support equipment Diagram showing an example of a knowledge graph Diagram showing an example of a superordinate concept model Flowchart of trained model construction process (A) Extraction graph and (B) Explanatory diagram for explaining state space Diagram showing the reward table Flowchart of reward setting process Diagram showing Q table Support processing flowchart Explanatory diagram for explaining learned state transitions A diagram showing an example of a superordinate concept model according to the second embodiment Explanatory diagram for explaining states and state transitions according to the second embodiment

The learned model construction method according to the present invention is implemented by a design support device that provides design support using a learned model. DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which a design support apparatus according to the present invention is applied to vehicle design support will be described below with reference to the drawings.

Design support here refers to receiving input from users involved in design and instructing the users on the steps (activities) to be performed in order to design.

<<First embodiment>>
As shown in FIG. 1, the design support device 1 includes a computer having a known hardware configuration, and includes a processor 2, a RAM 3, a ROM 4, a storage 5, an input device 6, and an output device 7. The storage 5 may be a known storage device for storing information, such as an HDD or an SSD. Further, the input device 6 may be a keyboard, a mouse, a microphone, etc., and the output device 7 may be a monitor, a speaker, etc.

As shown in FIG. 2, the design support apparatus 1 includes a storage section 10, an input section 11, an output section 12, a model construction section 13, and a support processing section 14 as functional sections.

The storage unit 10 includes a storage 5 and stores information related to design support. The storage unit 10 stores (holds) at least a knowledge database 15 and a superordinate concept model 16.

The knowledge database 15 is a database in which knowledge related to design is recorded in a hierarchical manner, and includes a knowledge graph 21. As shown in FIG. 3, the knowledge graph 21 has a tree structure with a predetermined height. In this embodiment, the knowledge graph 21 is constructed by combining a database related to design information (SysML) and a database related to design procedures (GSN). The knowledge graph 21 classifies and holds specifications of parts and parts of the vehicle to be designed, their target values, design constraints, development strategies (procedures), evidence, and the like. In this way, since the knowledge graph 21 includes the design information and the information related to the design procedure, its usefulness is increased compared to the case where only one of them is included.

The knowledge graph 21 is composed of a plurality of nodes 22 and a plurality of edges 23, each of which connects two nodes 22. Each node 22 included in the knowledge graph 21 corresponds to one activity required for design. The activities here correspond to the steps that must be performed in order to perform a design. As shown in FIG. 3, the node 22 corresponds to, for example, activities such as "setting a management target for the upper part of the door" and "considering the load input for the part α."

In the knowledge graph 21, each node 22 is recorded in a plurality of hierarchies depending on the abstraction level (concept) of the corresponding activity. That is, the higher the concept related to the activity, the higher the corresponding node 22 is located. For example, a node 22 corresponding to an upstream activity in the design process (e.g., goal setting) is located at a higher level than a node 22 corresponding to a downstream activity (e.g., consideration required to reach the goal). do. More specifically, as shown in FIG. 3, the node 22 related to vehicle design is located at a higher level than the node 22 corresponding to consideration of input, strength, and plate thickness.

Regarding the same activities related to members in an inclusive relationship, the node 22 in which the activity related to the included member is described is located at a lower level than the node 22 in which the activity related to the included member is described. Furthermore, the node 22 that describes the design of a member (for example, a vehicle body) is located at a higher level than the node 22 that describes the study of a part (for example, the part α) included in the member.

Each edge 23 represents a relationship between nodes 22 and connects nodes 22 in which mutually related (that is, correlated) activities are described. In this embodiment, the edge 23 mainly connects two nodes 22 of different hierarchies. For example, the edge 23 connects two activities described in the nodes 22 when they have a parent-child relationship, an inclusion relationship, a dependent relationship, a causal relationship, and the like.

The superordinate concept model 16 is a model of experts' judgment criteria (tacit knowledge) related to the design order, and describes dependencies based on design conditions (conditions of the entire project and conditions based on design results). . The superordinate concept model 16 is constructed by extracting the know-how possessed by experts. The superordinate concept model 16 is preferably constructed by being extracted from actual business scenarios, past trouble analysis, hearings, and causal relationships between dynamically occurring events.

In this embodiment, as shown in FIG. 4, the superordinate concept model 16 includes a plurality of superordinate concept units 30. Each superordinate concept unit 30 includes a text acquired from an expert, one or more pairs of activities related to each of the texts (hereinafter, two paired activities will be referred to as an activity pair), and a thickness assigned to each activity pair. including. Each activity pair includes an activity (source activity) and an activity that occurs immediately after the source activity (destination activity). The superordinate concept model 16 indicates the rule that an expert would perform a post-route activity after performing a route-source activity, and the thickness indicates the importance of performing a route-destination activity after a route-source activity. Show degree. In this embodiment, the degree of importance increases as the thickness value increases. The superordinate concept model 16 is stored in the storage unit 10 as a table including a plurality of superordinate conceptual models 16. However, the superordinate concept model 16 is not limited to this aspect, and may further include preconditions (for example, a development model, etc.) for performing the activity pair.

The superordinate concept model 16 is stored in advance in the storage unit 10 before constructing the learning model. The superordinate concept model 16 may be constructed by inputting an expert into the design support device 1 and stored in the storage unit 10. The design support device 1 also includes a concept construction unit that constructs a superordinate concept model 16 from the text acquired from the expert, and the concept construction unit constructs a plurality of superordinate concept units 30 and stores them in the storage unit 10 as the superordinate concept model 16. May be stored.

The input unit 11 includes the input device 6, and acquires input information input by the user. In this embodiment, the input unit 11 can acquire linguistic information related to the design from the user.

The output unit 12 includes an output device 7 and outputs output information to the user. In this embodiment, the output unit 12 outputs the content of the action to be performed by the user in the form of audio, image, text, etc., to convey the action to be performed to the user.

The model construction unit 13 is configured by software executed by the processor 2. The model construction unit 13 implements a learned model construction method by performing a model construction process. In this embodiment, in the model construction process, the model construction unit 13 determines the combination of a state S that is determined depending on whether or not a design activity is executed, and an action A that is an activity that can be selected under the state S. A trained model is constructed by assigning a reward R (see FIG. 7) to the model and performing so-called reinforcement learning to maximize the value.

The support processing unit 14 is configured by software executed by the processor 2. The support processing unit 14 performs design support using the trained model constructed by the model construction unit 13.

Next, details of the model construction process executed by the model construction unit 13 will be described with reference to FIG. 5.

In the first step ST1 of the model construction process, the model construction unit 13 executes a selection process for selecting a high-level concept unit 30 required for designing from the high-level concept model 16. More specifically, the model construction unit 13 outputs a list of the superordinate concept units 30 included in the superordinate concept model 16 to the output unit 12 in the selection process. The expert checks the output of the output section 12 and inputs it to the input section 11. The model construction unit 13 receives from the input unit 11 the selection of a general concept unit 30 suitable for the vehicle to be designed. When the acceptance of the selection of the superordinate concept unit 30 is completed, the model construction unit 13 executes step ST2.

The model construction unit 13 executes an extraction process in step ST2. In the extraction process, the model construction unit 13 extracts from the knowledge graph 21 a region that matches the superordinate concept unit 30 selected in step ST1, and stores it in the storage unit 10 as an extraction graph 32.

In this embodiment, the model construction unit 13 first determines, from among the nodes 22 of the knowledge graph 21, that the activity corresponding to the node 22 is compatible with the route source activity or route destination activity included in the extracted superordinate concept unit 30. Extract what you want. At this time, when the word written in the activity and the word written in the route source activity or the word written in the route destination activity are respectively the same or similar, the model construction unit 13 It is preferable to determine that the activity corresponding to the route source activity or the route destination activity is compatible with the route source activity or the route destination activity. In order to perform such similarity determination, the storage unit 10 preferably stores a predetermined database including a synonym dictionary, etc., and the model construction unit 13 preferably determines the similarity of words by referring to this database. . Thereafter, the model construction unit 13 extracts a region (see the region surrounded by the broken line in FIG. 3) containing the matching node 22 from the knowledge graph 21 as a region matching the superordinate concept unit 30, and creates an extracted graph. 32 (see FIG. 6(A)) in the storage unit 10. When the storage of the extracted graph 32 is completed, the model construction unit 13 executes step ST3.

In step ST3, the model construction unit 13 extracts nodes 22 that match the superordinate concept unit 30 from the extraction graph 32, and performs state space generation processing based on the extracted nodes 22. The state S here is determined depending on at least a combination indicating whether or not each activity included in the extraction graph 32 is executed, and may be based on the history of the activity in addition to the combination. In this embodiment, as shown in FIG. 6, the state S is defined by the execution or non-execution of each activity and the combination with the activity executed immediately before that combination, and the state space P includes all the states S. is defined as a set containing

In this embodiment, the model construction unit 13 extracts nodes 22 that match the superordinate concept unit 30 from the extraction graph 32, and executes (T) or does not execute (T) each activity of the activities corresponding to each of the extracted nodes 22. A combination of logical expressions indicating F) is calculated. However, the model construction unit 13 always executes (T) the activity corresponding to the most upstream node 22 of the extraction graph 32 (activity 1 in FIG. 4). After that, the model construction unit 13 selects and combines all the activities that are executed (T) one by one for each combination of executed and unexecuted activities as the activities that were executed immediately before that combination. , and configure a state space P by combining them.

As shown in FIG. 6(B), when activities 1 to 3 are extracted, the combination of logical expressions indicating execution or non-execution of activities 2 and 3 is (activity 2, activity 3) = (F, F), (T,F), (F,T) and (T,T). However, (activity 2, activity 3) = (F, F), both activities are not executed, (T, F), activity 2 is executed, activity 3 is not executed, (F, T), activity 2 is not executed. , activity 3 has been executed, (T, T) means both activities have been executed. All states S are expressed by combining activities that can be executed immediately before each combination of logical expressions indicating execution or non-execution of activities 2 and 3. More specifically, as shown in FIG. 6(B), the state S is (activity 2, activity 3, previous activity) = (F, F, 1), (T, F, 2), ( There are five types: F, T, 3), (T, T, 2), and (T, T, 3). Here, the last element in parentheses represents the number of the immediately previous activity.

As shown in Figure 6(B), in order to simplify the notations, (activity 2, activity 3, previous activity) = (F, F, 1), (T, F, 2), (F, S _00,1 , S _10,2 , S _01,3 , S _11,2 , S _11, It is written as ₃ . Here, in S _ij,k , i indicates execution (1) or non-execution (0) of activity 2, j indicates execution (1) or non-execution (0) of activity 3, and k indicates the immediately preceding activity. It shows the number.

When the configuration of the state space P is completed, the model construction unit 13 stores the state space P in the storage unit 10, and then executes step ST4.

The model construction unit 13 performs remuneration setting processing in step ST4. The reward setting process is performed based on the extraction graph 32 and the rules (superordinate concept unit 30) input in advance in step ST1, for each state S included in the state space P and the actions that can be selected under the state S. This is a process of giving reward R (more specifically, immediate reward) to the combination with A. Specifically, the model construction unit 13 performs reward setting processing for each combination of all the states S in the state space P and all the actions A that move from the state S to the states S included in the state space P. A table 40 (see FIG. 7) is constructed and stored in the storage unit 10. As shown in FIG. 7, the reward table 40 describes the reward R obtained by each action A for each state S.

Details of the remuneration setting process will be described with reference to FIG. 8. Hereinafter, for convenience of explanation, in the state S and action A in which the reward R is set, the state S will be described as a transition source, and the state S to which the state S to which the action A is transitioned by performing action A in the transition source will be described as the transition destination. That is, as shown in FIG. 9, a transition source (transition destination) is one of the states S included in the state space P, and all states S included in the state space P can be a transition source (transition destination). . Action A is defined as an activity required to transition from a transition source to a transition destination, regardless of whether the transition is possible or not, or whether there is an activity during the transition, and includes not performing any activity.

In the first step ST11 of the reward setting process, the model construction unit 13 determines whether the transition source and transition destination are the same. If they are the same, the model construction unit 13 executes step ST12, and if they are different, executes step ST13.

In step ST12, the model construction unit 13 determines that if all activities have been executed in the transition source (in FIG. 7, both the transition source and the transition destination correspond to S _11,2 or both correspond to S _11,3 ), The reward R is set to zero, and in other cases (when at least one activity has not been executed in the transition source), the reward R is set to a negative value (-1 in this embodiment) (Fig. 7 (See the lightest shaded part). When the settings are completed, the model construction unit 13 ends the reward setting process (ie, ST4).

In step ST13, the model construction unit 13 determines whether the transition source and the transition destination can be transitioned by executing one activity. If the source and destination are not transitionable, if more than one activity needs to be performed to transition from the source to the destination (for example, if the source is S _00,1 and the transition When the activity immediately before the transition source is equal to the activity immediately before the transition ( _for example, when the transition source is S _10,2 and the transition destination is S _11,2 ) ) is included. If the model construction unit 13 determines that the transition source and the transition destination can be transitioned by executing one activity, the model construction unit 13 performs step ST14, and if it determines that the transition is not possible, the model construction unit 13 performs step ST15. Execute.

In step ST14, the model construction unit 13 compares the transition source and the transition destination with each of the extracted superordinate concept units 30, and determines whether the transition source and the transition destination meet the conditions indicated by the superordinate concept unit 30. do. More specifically, the model construction unit 13 selects the transition source when the route source activity of the superordinate concept unit 30 matches the activity immediately before the transition source, and the route destination activity matches the activity immediately before the transition destination. It is determined that the transition destination matches the conditions indicated by the superordinate concept unit 30. The model construction unit 13 determines the similarity between the word written in the route source activity (route destination activity) and the word included in the activity immediately before the transition source (transition destination), and if the two are similar, , it may be determined that the route source activity and the activity immediately before the transition source match.

For example, in the state space P shown in FIG. 6B, the transition source that satisfies the condition indicated by the superordinate concept unit 30 shown in FIG. The transition destination that satisfies this condition is the state S in which the immediately preceding activity becomes activity 2 (upper row) or activity 3 (lower row) (see the darkest shaded area in FIG. 7). Therefore, the combinations where the transition source and the transition destination meet the conditions indicated by the superordinate concept unit 30 are the case where the transition source is S _00,1 and the transition destination is S _10,2 (thickness 5), and the case where the transition source is S 00,1 and the transition destination is S 10,2 (thickness 5) is S _00,1 and the transition destination is S _10,3 (thickness 4).

Thereafter, the model construction unit 13 sets the reward R based on the thickness included in the superordinate concept unit 30 to which both the transition source and transition destination fit. More specifically, the model construction unit 13 acquires the thickness values of the superordinate concept unit 30 that both the transition source and the transition destination match, and adds a predetermined value (1 in this embodiment) to each of the thickness values. The remuneration R is calculated by calculating the sum. The model construction unit 13 sets the reward R to zero when there is no superordinate concept unit 30 that matches both the transition source and the transition destination (see the medium-dark shaded area in FIG. 7). When the setting of the reward R is completed, the model construction unit 13 finishes the reward setting process (ST4).

In step ST15, the model construction unit 13 sets the reward R to a negative value (-1 in this embodiment). When the settings are completed, the model construction unit 13 ends the reward setting process (ST4).

When the reward setting process (ST4) is completed, the model construction unit 13 executes a learning process in which reinforcement learning is performed based on the given reward R in step ST5, as shown in FIG. More specifically, in the learning process, the model construction unit 13 uses the reward table 40 to perform reinforcement learning using a known algorithm to generate a Q table 50 (see FIG. 9) that represents the learning model. The Q table 50 shows the value of performing action A (transition) when in state S, that is, action value function Q(S, A) (S is state, A is action). However, if a transition between states S is virtually impossible (for example, when the transition source is S _00,1 and the transition destination is S _11,2 ), the reward R is set to negative. Therefore, reinforcement learning reduces its value to 0, and that action A (transition) is not performed. Any algorithm may be used to generate the Q-table 50, for example, it may be based on dynamic programming, Monte Carlo method, or TD method (more specifically, Q-learning, SARS, etc.). It's good. When the generation of the Q table 50 is completed, the model construction unit 13 ends the model construction process.

Next, details of the support processing executed by the support processing unit 14 will be described with reference to FIG. In the first step ST21 of the support process, the support processing unit 14 outputs information to the output unit 12 in order to allow the user to input information such as the design process to be supported, and outputs information regarding the process to be supported from the input unit 11 to text. Obtained by When the acquisition is completed, the support processing unit 14 executes step ST22.

In step ST22, the support processing unit 14 extracts, from the activities corresponding to the nodes 22 of the extraction graph 32, those that most match the acquired information. When the extraction is completed, the support processing unit 14 executes step ST23.

In step ST23, the support processing unit 14 outputs to the output unit 12 an instruction to the user to perform the activity extracted in ST22. When the output is completed, the support processing section 14 executes step ST24.

In step ST24, the support processing unit 14 extracts one state from the state space P based on the activity extracted in step ST22, and sets it as an initial state for outputting based on the learning model. In this embodiment, if the node 22 corresponding to the activity extracted in ST22 is the node 22 located most upstream in the extraction graph 32, the support processing unit 14 is in a state where only that activity has been executed. Set S to the initial state. If the node 22 corresponding to the extracted activity in ST22 is not the node 22 located most upstream in the extraction graph 32, the support processing unit 14 selects the activity corresponding to the most upstream node 22 and step ST22. The state S in which the extracted activity has been executed is set as the initial state. When the initial state setting is completed, the support processing unit 14 executes step ST25.

In step ST25, the support processing unit 14 repeatedly performs the process of selecting the most valuable action A corresponding to the current state S and transitioning the state S based on the learning model (Q table 50) from the initial state. At this time, the support processing unit 14 outputs an output to the output unit 12 instructing the user to perform the activity corresponding to the action A for each state transition. The support processing unit 14 ends the support processing when all activities are completed except for the activities that have already been executed in the initial state.

Next, the operation of the design support device 1 will be explained. When the vehicle to be designed is determined, the expert selects the general concept unit 30 suitable for the vehicle to be designed from the general concept model 16, thereby inputting rules for designing (ST1). FIG. 4 shows an example of the selected general concept unit 30. Thereafter, the model construction unit 13 collates the route source activity and route destination activity included in the superordinate concept unit 30 with the activity corresponding to the node 22 of the knowledge graph 21 (see "Match" in FIG. 3). , a part of the area of the knowledge graph 21 (see the part surrounded by the broken line in FIG. 3 and FIG. 6(A)) is extracted as the extraction graph 32 (ST2).

Next, the model construction unit 13 generates a state space P (see FIG. 6(B)) using the extracted graph 32 (ST3). Thereafter, as shown in FIG. 7, the model construction unit 13 sets a reward R for each combination of each state S and an action A that causes a transition of the state S, and A table 40 (see FIG. 7) is created (ST4). When the setting of the reward table 40 is completed, the model construction unit 13 performs reinforcement learning and generates a Q table 50 (see FIG. 9), which is a learning model (ST5). This completes the construction of the learning model.

Thereafter, when the user inputs information related to the process to be supported into the input unit 11 according to the output of the output unit 12, the support processing unit 14 acquires the information (ST21) and performs the activity corresponding to the node 22 of the extraction graph 32. From these, the one that best matches the acquired information is extracted (ST22).

More specifically, when the user inputs "I want to design the upper part of the door," the support processing unit 14 extracts the word "upper part of the door" and extracts the word "upper part of the door" and extracts it from the extraction graph 32 (see FIG. 6(A)). From among them, activity 1 "Setting management goals for the upper part of the door" is extracted as a matching activity.

Thereafter, the support processing unit 14 outputs to the output unit 12 an instruction to execute activity 1, that is, to set a management target for the upper part of the door (ST23). Next, the support processing unit 14 sets a state _S00,1 in which only activity 1 is being executed as an initial state (ST24), and based on the learning model (Q table 50), the high value action A (transition ) and instruct the user to perform the action A (ST25).

FIG. 11 shows a state transition diagram corresponding to the Q table 50 obtained by reinforcement learning. The support processing unit 14 sets the state S _00,1 as the initial state, refers to the Q table 50, and instructs the user to perform activity 2, which is of high value, that is, consider inputting the load on the part α. As a result, the state S transitions from the state S _00,1 to the state S _10,2 .

Next, in state _S10,2 , the support processing unit 14 instructs the user to perform a high-value activity 3, that is, to examine the resistance of the region β. As a result, the state S transitions from the state S _10,2 to the state S _11,3 . Thereafter, the support processing unit 14 determines that all activities have been performed, and ends the support processing.

Next, the effects of the learned model construction method executed by the design support device 1 and the model construction unit 13 will be described.

The design support device 1 sets a reward R based on a rule (superordinate concept model 16) input based on the experience and knowledge of an expert, and constructs a learning model. The design support device 1 issues activity instructions based on the learning model. In this way, the design support device 1 functions as an inference agent that autonomously constructs a learning model based on rules input from an expert and instructs the user to perform activities based on the learning results.

Activity instructions by the design support device 1 are performed using a learning model based on rules (superordinate concept model 16, superordinate concept unit 30) input from an expert. Therefore, the design support device 1 can give activity instructions in line with the expert's actions. Furthermore, even design beginners with little experience can select and execute high-value, optimal activities from a large number of options, reducing the chances of design beginners asking experts for instructions. can. This can reduce the burden on experts (for example, individual training for beginners in design, on-the-job training, etc.).

In the design support device 1, when the expert selects a suitable superordinate concept unit 30 from the superordinate concept model 16, a reward R is set for each combination of a state S and an action A that transitions between states S. Therefore, it is not necessary for the expert to set the reward R for each combination of the state S and the action A, and the reward R can be easily set.

A state S is also defined by the activities immediately preceding that state S. Therefore, it is possible to easily determine whether the combination of the activity immediately before reaching the state S and the action A that can be performed thereafter matches the rules according to the expert's design procedure. In addition, whether state S and action A match the rules is determined by the activity immediately before reaching state S, the word indicating the activity corresponding to action A that can be performed after that, the route source activity of the superordinate concept model 16, and the route destination. This is done by comparing the words related to the activity. In this way, by comparing the words, it is possible to determine whether the state S and the action A match the rules, so the determination can be made mechanically and is easy.

<<Second embodiment>>
The method for constructing a learned model according to the second embodiment and the design support device 101 that implements the method are different from those in the first embodiment in terms of the configuration of the superordinate concept model 16, the definition of the state S, and the construction of the learned model. Processing ST3 to ST5 is different from support processing ST23 and ST24. The rest of the configuration of the second embodiment is the same as that of the first embodiment, so a description thereof will be omitted.

As shown in FIG. 12, the high-level concept model 16 according to the second embodiment includes design conditions in addition to the high-level concept model 16 according to the first embodiment. The design conditions may be extracted from text input by the expert, or may be input directly from the expert. Each design condition is given a condition number (a, b in FIG. 12) or a number indicating that the condition is not determined (u in FIG. 12). In the superordinate concept model 16, an activity pair (a route source activity and a route destination activity) and the corresponding thickness are recorded for each condition number.

In the state space creation process (ST3 in FIG. 5), the model construction unit 13 extracts nodes 22 that match the superordinate concept unit 30 from the extraction graph 32, and creates a state based on the extracted nodes 22, as in the first embodiment. Performs space generation processing. However, as shown in FIG. 12, the state S according to the second embodiment includes the presence or absence of execution of each activity corresponding to the node 22, the number of the activity performed immediately before the combination, and the number of the design condition. Defined in combination with

As shown in FIG. 6(A), the extraction graph 32 includes three activities (activities 1 to 3), and as shown in FIG. 12, the superordinate concept unit 30 has two design conditions (a, b, u). If so, the state space P includes 15 states S. In FIG. 13, each state S is S _ij,k,l (i indicates execution (1) or non-execution (0) of activity 2, and j indicates execution (1) or non-execution (0) of activity 3. where k indicates the previous activity and l indicates the design condition).

When the generation of the state space P is completed, the model construction unit 13 performs reward setting processing. In the second embodiment, the model construction unit 13 performs a reward setting process (ST4) for each design condition, performs a learning model generation process (ST5), and generates a learning model for each design condition.

In the first step ST21 of the support process, the support processing unit 14 acquires information regarding the process to be supported, similarly to the first embodiment. Thereafter, in step ST22, the support processing unit 14 extracts the activity that most matches the acquired information from the activities corresponding to the node 22 of the extraction graph 32, as in the first embodiment. Thereafter, in step ST23, similarly to the first embodiment, the support processing unit 14 outputs to the output unit 12 an instruction to the user to perform the activity extracted in ST22.

Next, in step ST23, the support processing unit 14 extracts one state S from the state space P based on the activity extracted in step ST22, and sets it as an initial state for outputting based on the learning model. . In this embodiment, the support processing unit 14 first obtains the design conditions of the process to be supported based on the information obtained in ST21, and sets the number of the design conditions. However, when the design condition could not be acquired or when it is determined that the condition is undetermined, the support processing unit 14 adds a number (u in this embodiment) indicating that the condition is undetermined to the design condition number. Set.

Next, in step ST24, the support processing unit 14 determines that if the node 22 corresponding to the activity extracted in ST22 is the node 22 located most upstream in the extraction graph 32, it corresponds to the number of the set design condition. Then, a state S in which only that activity has been executed is set as the initial state. If the node 22 corresponding to the activity extracted in ST22 is not the node 22 located at the most upstream position in the extraction graph 32, the support processing unit 14 selects a node 22 that corresponds to the set design condition number and is located at the most upstream position. The state S in which the activity corresponding to the node 22 and the activity extracted in step ST22 have been executed is set to the initial state. However, if the information acquired in ST21 includes information related to design conditions, the support processing unit 14 sets a number indicating that the condition is undetermined (in this embodiment, as the number of the corresponding design condition). Use u). When the initial state setting is completed, the support processing unit 14 executes step ST25.

In step ST25, the support processing unit 14 selects the most valuable action A corresponding to the current state S based on the learning model (Q table 50) from the initial state and changes it to the state S, as in the first embodiment. The process of transitioning is repeated. At this time, the support processing unit 14 outputs to the output unit 12 an output instructing the user to perform an activity corresponding to action A for each state transition. However, when the input unit 11 receives an input indicating a change in design conditions, the support processing unit 14 transitions the state S to a state S in which only the design conditions have been changed. After that, the support processing unit 14 selects the most valuable action A corresponding to the current state S, performs a state transition, and outputs an output instructing the user to perform the activity corresponding to action A in sequence. It will be held on the 12th. When all activities have been executed, the support processing unit 14 ends the support processing.

The operation and effects of the design support apparatus 101 configured in this way will be explained.

At the beginning of design, the design conditions required for the design object may not be determined. In that case, a number indicating that the condition is undetermined (u in this embodiment) is set as the design condition number. Thereafter, if the extracted activity matches activity 1 in ST22, the state S becomes S _00,1,u , and activity 1 is executed.

Immediately after executing activity 1, if there is an input from the user indicating that the design condition has become a, the state S transitions from S _{00, 1, u} to S _{00, 1, a} ( (See dashed arrow in FIG. 13). After that, the support processing unit 14 selects the most valuable action A corresponding to the current state S based on the learning model (Q table 50), performs a state transition, and sequentially performs the activities corresponding to the action A. The output unit 12 outputs instructions to the user as follows.

In this way, the state S is defined by a combination including whether or not a plurality of activities related to the design are executed, the number of the activity performed immediately before the combination, and the design condition. Thereby, when the design conditions are changed and an input related to the design conditions is performed, the state S can be changed to one that matches the input design conditions. Thereafter, the design support device 1 instructs the user to sequentially perform actions that maximize value using a learning model that matches the conditions of the accepted design. Therefore, the design support device 1 enables design support according to design conditions.

In particular, in this embodiment, the numbers indicating design conditions include numbers indicating that they are undetermined. Therefore, even if design conditions are not determined in detail, design support can be provided. Furthermore, once the design conditions are determined, the system transitions to a state S that meets the conditions, and instructions for activities in accordance with the design conditions are given, making it possible to provide design support that better matches the progress of design and development.

Although the description of the specific embodiments has been completed above, the present invention is not limited to the above-mentioned embodiments and can be widely modified and implemented.

In the embodiment described above, the general concept unit 30 is related to the order of activities, but it is not limited to this aspect. For example, the superordinate unit 30 may relate to the position in the knowledge graph 21 of the corresponding node 22 of the source activity and the destination activity. More specifically, the superordinate concept model 16, as the superordinate concept unit 30, uses a predetermined method when a node 22 corresponding to a route source activity is located in a layer above a node 22 corresponding to a route destination activity in the knowledge graph 21. It is good to include something to set the thickness of the value. When the expert selects this superordinate concept unit 30, after finishing the activity corresponding to the node 22 located in the upper layer, and executing the activity corresponding to the node 22 located in the lower layer and the state transitions, a positive reward R will be given. Granted. This makes it easier to give activity instructions in order from the activity located at the node 22 located in the upper layer to the activity corresponding to the node 22 located in the lower layer. This makes it easier for the user to understand the content of the activity instruction because the activity instructed later becomes a subordinate concept (at a lower abstraction level) of the activity instructed earlier.

Further, in the superordinate concept model 16, as the superordinate concept unit 30, the difference in hierarchy between the node 22 corresponding to the route source activity and the node 22 corresponding to the route destination activity is a predetermined value or more (for example, 3 or more hierarchies). In some cases, it is good to include a reward R that is not set (set to zero). When the difference in the hierarchy between the node 22 corresponding to the route source activity and the node 22 corresponding to the route destination activity becomes larger, the reward R becomes smaller for a state transition in which the route destination activity is executed immediately after the route source activity. It will no longer be set. As a result, the reward R will not be set for cases where the difference in abstraction level between the route source activity and the route destination activity is large and the flow of activities between the two is difficult to understand, so the flow of activities will not be difficult for the user to understand. It can be prevented.

In the above embodiment, the support processing unit 14 repeatedly performs state transitions based on the learning model in step ST25, and the support processing unit 14 instructs the user to perform an activity for each state transition. It is not limited to the aspect. For example, in step ST25, the support processing unit 14 outputs selectable actions A and their corresponding values to the output unit 12 in the current state S based on the learning model (Q table 50), and inputs The unit 11 may be configured to receive the desired action A from the user and repeat the state transition according to the input. However, when the support processing section 14 outputs the selectable actions A and their corresponding values to the output section 12, it is preferable to arrange the actions A in descending order of value and output them to the output section 12.

In the embodiments described above, the design support apparatuses 1 and 101 perform design support by constructing a learned model based on the extracted graph extracted from the knowledge graph 21, but the design support apparatuses 1 and 101 are not limited to this aspect. For example, the design support device 1, 101 maintains a list of activities required to perform a design, and uses the list to generate the state space P by comparing it with the high-level concept unit 30 of the high-level concept model 16, After setting the reward R, the learned model may be generated.

In the embodiments described above, the design support apparatuses 1 and 101 are configured by one computer, but the design support apparatuses 1 and 101 are not limited to this aspect. The design support apparatus 1 may be configured by a plurality of computers connected through a network such as the Internet working together. The design support device 1 may be configured by a so-called cloud computing system including a plurality of servers on a network.

In the embodiment described above, the design support device 1 is used for vehicle design support, but the present invention is not limited to this aspect. The design support device 1 may be applied, for example, to support the design of transportation equipment and machines such as ships and aircraft.

1: Design support device according to the first embodiment 16: Higher level conceptual model 21: Knowledge graph 22: Node 23: Edge 50: Q table (learned model)
101: Design support device A according to the second embodiment: Action P: State space R: Reward S: State

Claims (11)

The learned model maximizes the value by giving rewards to the combination of a state that is determined depending on whether or not a design activity is executed and a behavior that is the activity that can be selected under the state. A method for constructing a trained model, the method comprising:
granting the reward based on rules input in advance;
performing reinforcement learning based on the given reward,
The rule includes information regarding a route source activity, a route destination activity to be performed after the route source activity, and a thickness indicating the importance of the route destination activity being performed after the route source activity,
In the step of giving the reward, when the action immediately before reaching the state matches the route source activity, and the action and the route destination activity match, the reward is given based on the thickness. How to build the trained model to be set. 2. The learned model construction method according to claim 1, wherein the state is defined by a combination including whether or not a plurality of the activities are executed and the activity that was executed immediately before. The learned model construction method according to claim 1, wherein the state is defined by a combination including whether or not a plurality of the activities are executed, the activity that was executed immediately before, and a number corresponding to a design condition. . 4. The learned model construction method according to claim 3, wherein the number corresponding to the design condition includes the number indicating that the design condition is undetermined. Using a knowledge graph that includes a plurality of nodes corresponding to the activity related to the design and edges connecting the corresponding nodes based on the relationship between the activities, the activity corresponding to the node is selected from among the activities corresponding to the node. The learning according to any one of claims 1 to 4, comprising the step of extracting the activity for defining the state by matching it with a rule, and generating a state space that is a set of the states. How to build an existing model. 6. The learned model construction method according to claim 5, wherein the knowledge graph includes design information and data related to a design procedure. In the knowledge graph, the nodes are recorded in layers,
7. The learned model construction method according to claim 5, wherein the rule includes that the node corresponding to the route source activity is located above the node corresponding to the route destination activity. 8. The trained model according to claim 7, wherein the rule is such that when a difference in hierarchy between the node corresponding to the route source activity and the node corresponding to the route destination activity is a predetermined value or more, the reward is not given. How to build. said rule is entered by text,
In the step of generating the state space, the state space is selected from among the activities corresponding to the nodes by matching words included in the text with the activities corresponding to the nodes. The learned model construction method according to any one of claims 5 to 8, which extracts activities. A design support device that performs design support based on a trained model constructed by the method according to any one of claims 1 to 9,
setting an initial state based on information input by the user;
A design support apparatus executes a step of outputting an instruction to the user to sequentially perform the actions that maximize value from the initial state. A design support device that performs design support based on a learned model constructed by the method according to claim 3 or 4,
setting an initial state based on information input by the user;
outputting instructions to the user to sequentially perform the actions that maximize value from the initial state;
In the step of performing the output, when input related to the conditions of the design is received, after transitioning the state to one that matches the input conditions, in order to sequentially perform the actions that maximize the value, A design support device that performs the output to instruct the user.

Images