WO2022215233A1

WO2022215233A1 - Automatic sequence generation device, automatic sequence generation method, and program

Info

Publication number: WO2022215233A1
Application number: PCT/JP2021/014929
Authority: WO
Inventors: 克希小林; 宏治田中; 淳平羽藤
Original assignee: 三菱電機株式会社
Priority date: 2021-04-08
Filing date: 2021-04-08
Publication date: 2022-10-13
Also published as: JPWO2022215233A1; JP7387061B2

Abstract

The purpose of the present invention is to automatically generate an HMI sequence that carries out an operation having good usability by, when automatically generating the sequence, evaluating the sequence using an order of processing of a machine, which is structure information of the sequence, and a sensitivity indicator of a person.　The present invention comprises: a sequence generation unit (2) that generates a plurality of HMI sequences using specification information relating to a function possessed by a machine that is a control target and environment information relating to the conditions in which the machine is used; an evaluation unit (4) that references HMI evaluation criteria data (3) having one or more evaluation criteria defined using structure information representing an order of processing of the sequence of the machine and a sensitivity indicator of a person, and that evaluates and gives evaluation values to the plurality of HMI sequences using the one or more evaluation criteria; and an optimization unit (5) that uses the evaluation values to select an HMI sequence from among the plurality of HMI sequences that have been given evaluation values.

Description

AUTOMATIC SEQUENCE GENERATOR, AUTOMATIC SEQUENCE GENERATION METHOD AND PROGRAM

The present disclosure relates to an automatic sequence generation device, an automatic sequence generation method, and a program for HMI (Human Machine Interface) of equipment.

In order for a device to automatically execute a series of actions, it needs information that defines the control structure of the action, that is, the sequence. To describe the sequence of device operations, the operation content, operating conditions, control method, etc. are combined, and the behavior tree expressed in a tree structure (Behavior Tree), the processing state of the device (for example, waiting for input, searching, etc.) State transition diagrams (state charts) expressed by the transitions of are widely used.

Conventionally, such sequences were manually written by designers and created by programming. On the other hand, with the spread of AI (Artificial Intelligence) technology and IoT (Internet of Things) technology, complex control design that integrates various information is required, and the cost of manually writing down sequences is increasing. Therefore, there is an increasing need for an automatic sequence generation technique.

To address these issues, we have disclosed a technology that automatically generates sequences based on evolutionary computation, even for problems that are large and complex, or for which it is difficult to describe the processing details of conventional automatic sequence generators. (See, for example, Patent Document 1).

Japanese Patent No. 6663873

In devices such as home appliances, in-vehicle devices, mobile devices, etc., which are operated directly by the user and whose operation is directly fed back to the user, that is, in devices having an HMI, the sequence of the HMI determines the usability of the device. . Therefore, it is important to consider usability when designing HMI sequences.

However, the conventional automatic sequence generation device described above does not evaluate the processing order of the device, which is the structure information of the sequence, in evaluating the sequence that is the individual of evolutionary computation. Therefore, when a conventional sequence automatic generation device is applied to HMI sequence generation for a device, there is a problem that a sequence may be generated that performs an operation that is not comfortable to use, although the target operation can be performed. .

The present disclosure has been made to solve the above-mentioned problems, and when automatically generating a sequence, the sequence is evaluated using the processing order of the device, which is the structural information of the sequence, and the human sensitivity index. By doing so, the object is to automatically generate an HMI sequence that performs operations that are comfortable to use.

The sequence automatic generation device according to the present disclosure is
a sequence generator that generates a plurality of HMI sequences using specification information about the functions of a device to be controlled and environment information about the situation in which the device is used;
HMI evaluation criteria data having one or more evaluation criteria defined by using structural information indicating the sequence processing order of the equipment and human sensitivity indices, and using the evaluation criteria to generate the plurality of HMIs. an evaluation unit that evaluates the sequence of and assigns an evaluation value;
an optimization unit that selects an HMI sequence using the evaluation value from the plurality of HMI sequences to which the evaluation value is assigned.

Further, the sequence automatic generation method according to the present disclosure includes:
a sequence generation step of generating a sequence of a plurality of HMIs using specification information about functions of a device to be controlled and environment information about the situation in which the device is used;
HMI evaluation criteria data having one or more evaluation criteria defined by using structural information indicating the sequence processing order of the equipment and human sensitivity indices, and using the evaluation criteria to generate the plurality of HMIs. an evaluation step for evaluating the sequence of and assigning an evaluation value;
and an optimization step of selecting an HMI sequence using the evaluation value from the plurality of HMI sequences to which the evaluation value is assigned.

According to the present disclosure, there is an effect that it is possible to automatically generate an HMI sequence that performs operations that are comfortable to use.

1 is a block configuration diagram of an automatic sequence generation device according to Embodiment 1; FIG. 2 is a hardware configuration diagram of an automatic sequence generation device according to Embodiment 1. FIG. 4 is a flow chart showing the operation of the sequence automatic generation device according to Embodiment 1. FIG. 4 is an example of a behavior tree, which is an example of a sequence description format according to Embodiment 1. FIG. It is an example of specification information and environment information in Embodiment 1. FIG. 4 is an example of HMI evaluation criteria data in Embodiment 1. FIG. 4 is a flow chart of a sequence evaluation procedure in Embodiment 1. FIG. It is an example of evaluation based on the manipulated variable of the sequence in Embodiment 1. FIG. It is an example of evaluation based on the operation timing of the sequence in Embodiment 1. FIG. It is an example of evaluation based on the permutation of the modal information of the sequence in Embodiment 1. FIG. FIG. 10 is a block configuration diagram of an automatic sequence generation device according to Embodiment 2; 10 is a flow chart showing the operation of the automatic sequence generation device according to Embodiment 2. FIG. It is an example of HMI evaluation criteria data in Embodiment 2. FIG. FIG. 11 is a block configuration diagram of an automatic sequence generation device according to Embodiment 3;

Embodiment 1.
<<1-1>> Configuration The automatic sequence generation device according to the first embodiment will be described with reference to FIGS. 1 to 10. FIG. FIG. 1 is a block configuration diagram of an automatic sequence generation device showing the first embodiment.

In FIG. 1, the sequence automatic generation device 100 is composed of an input unit 1, a sequence generation unit 2, an HMI evaluation criterion 3, an evaluation unit 4, an optimization unit 5, and an output unit 6.

The input unit 1 receives specification information and environment information input from the outside of the automatic sequence generation device 100 . Here, the specification information is, for example, specification information relating to the HMI of the device, such as functions possessed by the device to be controlled and possible parameters. Environmental information is, for example, the situation in which the equipment is used.

The sequence generation unit 2 generates a plurality of HMI sequences based on the specification information and environment information received from the input unit 1.

HMI evaluation criteria 3 stores a plurality of HMI evaluation criteria data defined using sequence structure information and human sensitivity indices. Here, the structural information of the sequence is information defined by, for example, the processing order of devices, the flow of processing, the sequence of operations, and the like. In addition, the human sensitivity index is an index that expresses the subjective and intuitive reaction of human senses, such as the degree of feeling natural (or unnatural) to the person, the degree of comfort (or discomfort) to the person, etc. . The evaluation criteria in the HMI evaluation criteria data are used to evaluate the multiple HMI sequences generated by the sequence generator 2 . This evaluation criterion is defined using a human sensibility index, and is set, for example, to conditions that make the HMI comfortable to use (feeling natural to people, easy to use, and comfortable).

The evaluation unit 4 receives the sequences generated by the sequence generation unit 2, evaluates each sequence while referring to the HMI evaluation criteria data output by the HMI evaluation criteria 3, and assigns an evaluation value to each sequence. Here, the evaluation value is, for example, a positive value normalized to 1.0 or less. In other words, it is an uncomfortable HMI sequence. Note that the evaluation value does not have to be normalized as long as sequence evaluation is possible. Also, the evaluation value is not limited to a numerical value as long as it has a format that allows sequence evaluation. For example, the rating values may be ranked alphabetically (eg, "A" being the highest rating, "Z" being the lowest rating, etc.).

The optimization unit 5 receives a plurality of HMI sequences to which evaluation values are assigned from the evaluation unit 4, and performs processing for selecting sequences with high evaluation values. In addition, after the sequence selection process, the optimization unit 5 changes the internal state of the sequence (eg, parameter conditions related to each of the device specification information, the environment information, and the device structural information, etc.) using, for example, evolutionary calculation. Alternatively, the evaluation value of the entire sequence is increased by repeating the process of correcting, passing the sequence after the internal state change to the evaluation unit 4 again, and performing the evaluation again a certain number of times or until a predetermined convergence condition is reached. Also process.

The output unit 6 selects a sequence finally obtained as a result of the alternation of generations from among a plurality of sequences with increased evaluation values in the optimization unit 5, and outputs the selected sequence to the outside.

<<1-2>> Hardware Configuration Each configuration of the sequence automatic generation device 100 shown in FIG. 1 can be realized by a computer, which is an information processing device with a built-in CPU (Central Processing Unit). A computer with a built-in CPU is, for example, a stationary computer such as a personal computer or a server computer, a portable computer such as a smartphone or a tablet computer, or an embedded vehicle system such as an autonomous vehicle driving system or a car navigation system. and SoC (System on Chip).

Further, each configuration of the sequence automatic generation device 100 shown in FIG. 1 is an LSI (Large scale integrated circuit). Further, each configuration of the sequence automatic generation device 100 shown in FIG. 1 may be a combination of a computer and an LSI.

FIG. 2 is a block diagram showing an example of the hardware configuration of the sequence automatic generation device 100 configured using an information processing device such as a computer.

In the example of FIG. 2, the sequence automatic generation device 100 includes a memory 11, a processor 12 containing a CPU (not shown), a recording medium 13, an input device 14, an output device 15, and a signal path 16 such as a bus. there is

The memory 11 serves as a program memory for storing various programs, a work memory used when the processor 12 performs data processing, a memory for developing signal data, and the like, for realizing the sequence automatic generation processing of the first embodiment. Storage means such as ROM (Read Only Memory) and RAM (Random Access Memory) to be used.

More specifically, the memory 11 can store programs and data for the sequence generation unit 2, HMI evaluation criteria 3, evaluation unit 4, and optimization unit 5. In addition, the memory 11 can store intermediate data generated by the processing of each unit.

The processor 12 uses the CPU and the RAM in the memory 11 as working memory, and operates according to a computer program (that is, an automatic sequence generation program) read from the ROM in the memory 11 via the signal path 16. do.

More specifically, the processor 12 reads programs and data corresponding to each process of the sequence generation unit 2, HMI evaluation criteria 3, evaluation unit 4, and optimization unit 5 from the memory 11, and processes them by the CPU. Thus, the program for automatic sequence generation processing shown in the first embodiment can be executed.

The recording medium 13 is used to store various data such as various setting data and signal data for the processor 12 . As the recording medium 13, for example, it is possible to use a volatile memory such as a RAM, or a non-volatile memory such as a removable HDD (Hard Disk Drive) or SSD (Solid State Drive). The recording medium 13 contains, for example, a boot program including an OS (Operating System), a program for automatic sequence generation processing, initial state and various setting data, constant data for control, a database for HMI evaluation criteria 3, and equipment specification information. In addition, various data such as environment information and error information logs can be accumulated. Various data in the memory 11 can also be stored in the recording medium 13 .

The input device 14 corresponds to the input unit 1 and is input means for inputting specification information and environment information, which are external data, into the sequence generation device 100 . When inputting external data via a network connected to the sequence generation device 100, the input device 14 is composed of a communication interface for wired or wireless communication. Alternatively, when external data is input manually, the input device 14 is composed of an input interface such as a keyboard, touch panel, or mouse. It should be noted that the input device 14 can be omitted when the external data is input using the recording medium 13 described above.

The output device 15 corresponds to the output unit 6 and is output means for presenting the obtained sequence, which is the result of the processing of the sequence automatic generation device 100 . When outputting via a network connected to the sequence generator 100 , the output device 15 is configured with a communication interface for wired or wireless communication, but this communication interface can also be shared with the input device 14 . Alternatively, when converting the obtained sequence into human-readable information, for example, converting it into text and presenting it, the output device 15 is configured with a display interface such as a display. It should be noted that the output device 15 can be omitted when the sequence obtained by using the recording medium 13 described above is output.

In addition, the output device 15 sequentially receives sequences in the middle of optimization in the optimization unit 5 and displays them on the display interface. By displaying the sequence in the middle of optimization, it is possible to visualize whether the optimization process is progressing correctly (that is, whether it has converged), which has the effect of improving the efficiency of automatic sequence generation.

As described above, the functions of the input unit 1, the sequence generation unit 2, the HMI evaluation criteria 3, the evaluation unit 4, the optimization unit 5, and the output unit 6 shown in FIG. 13 , an input device 14 and an output device 15 .

The program for executing the automatic sequence generation device 100 may be stored in a storage means inside the computer that executes the software program, or may be stored in a computer-readable external storage medium such as a disk medium or a flash memory. It may be held in a form distributed by the computer, and may be read and operated when the computer is started. It is also possible to acquire programs from other computers through a wired or wireless network such as a LAN (Local Aera Network).

In addition, the program that executes the sequence automatic generation device 100 can be combined on software with a program that is executed externally, for example, a program that controls mobile equipment (for example, an autonomous operation control program), and run on the same computer. is also possible. Alternatively, distributed processing on multiple computers is also possible.

<<1-3>> Processing Operation Next, the detailed operation of the present invention in the first embodiment will be described. In the first embodiment, a mobile device such as a vehicle will be described as an example of a device to be controlled. Here, the vehicle or the like is, for example, a vehicle such as an automobile, an electric wheelchair, a PMV (Personal Mobility Vehicle), or a two-wheeled vehicle. FIG. 3 is a flow chart showing the operation of the first embodiment. It should be noted that "unit" in each step below may be read as "step", "processing", or "process".

First, as a preparation for explaining the operation of the first embodiment, an example of a sequence format (format) used in the first embodiment and an example of a framework for automatic sequence generation will be explained.

In the first embodiment, for example, the behavior tree described in Non-Patent Document 1 (abbreviated as BT in the following description) can be used as the format of the sequence of operations of the device or system. BT is a format that expresses sequence structural information in a tree structure that combines various nodes representing operation details, operating conditions, and control methods. Run with

An example of BT notation is shown in FIG. In FIG. 4, four types of nodes are connected in a tree structure: an Action node representing operation details, a Condition node representing operation conditions, a Selector (or Fallback) node representing control methods, and a Sequence node. Any node returns two types of success or failure to the parent node. It is also possible to define Running in an asynchronous system.

Next, each node will be explained. The Action node represents the execution of the device or system action itself, such as displaying on the screen, sending out a guidance sound, or tilting the traveling direction of the vehicle body to the right. Basically, after execution, it returns Success to the parent node.

The Condition node returns success to the parent node if the predetermined conditions are met, and failure if not. As the predetermined condition, for example, the time is before 12:00, the number of passengers is 1, and the like.

The Selector node and Sequence node define the control method. The Selector node evaluates the child nodes in order from the left, and if there is even one successful child node, returns success to the parent node and terminates the process. If all child nodes fail, return failure to the parent node. The Sequence node evaluates the child nodes in order from the left, and if there is even one failed child node, returns failure to the parent node and terminates the process. If all child nodes succeed, return success to the parent node.

The operation of BT in FIG. 4 will be specifically described. First, processing is started from a Selector node (ND1), which is the root, and a Sequence node (ND2), which is a left child node, is visited. The Sequence node (ND2) evaluates the Condition node (ND3), which is the left child node. If the Condition node (ND3) satisfies Condition 1, it returns success to its parent node, the Sequence node (ND2).

Since the Sequence node (ND2) succeeds in the Condition node (ND3), it evaluates the Action node (ND4) on the right and executes Action 1. After execution, the Action node (ND4) returns success to its parent node, the Sequence node (ND2).

The Sequence node (ND2) returns success to the parent node, the Selector node (ND1), because all child nodes are successful. Since the Sequence node (ND1) succeeds in one child node, the subsequent processing is not performed and the processing of one sequence ends.

If the Condition node (ND3) does not satisfy Condition 1, failure is returned to the Sequence node (ND2). Furthermore, the Sequence node (ND2) returns failure to the Selector node (ND1) because one child node fails. At this time, the Selector node (ND1) visits the right child node, the Sequence node (ND5).

In the Sequence node (ND5), the child nodes Action node (ND6) to Action node (ND8) are executed in order from the left, and success is returned to the parent node Selector node (ND1). That is, the processing from action 2 to action 4 is executed. As described above, all the processing steps for one sequence from ND1 to ND8 are completed.

Next, a framework for automatically generating sequences in the first embodiment will be described. In the first embodiment, as an algorithm for automatically generating BT tree structure data, Grammatical Evolution (abbreviated as GE in the following description), which is one of the evolutionary calculation methods described in Non-Patent Document 2, is used. can be done. The evolutionary computation method is not limited to GE, and a known evolutionary computation method such as genetic programming may be used.

　In GE, individuals are first generated that have a sequence of 0 or 1 as their chromosomes. After that, by applying a context-free grammar algorithm called Backus-Naur Form (abbreviated as BNF in the following description) to each individual, the individual is converted from the sequence to a tree structure. Fitness calculations are mainly performed using a tree structure, and operations such as crossover and mutation are performed on individual chromosomes or genes as in other evolutionary calculations.

Returning to the original, the details of the operation of the first embodiment will be described using the flowchart shown in FIG. First, in step ST1, the input unit 1 acquires HMI specification information and environmental information of the device to be controlled. FIG. 5A shows an example of specification information, and FIG. 5B shows an example of environment information.

In FIG. 5(a), the specification information indicates, for example, functions possessed by the device, parameters that the functions can take, and the like. In BT, these are mainly the information that is the base of the Action node. For example, the functions include control of the main body of the mobile device such as acceleration, deceleration, right and left turns of the vehicle body, and transmission of information to passengers such as telop display and guidance sound transmission. Also, for example, the parameter is a value that each function can take, such as acceleration, deceleration speed, and angle to be changed when turning right or left. The ON/OFF of the function itself, such as the display/non-display of a telop or the presence/absence of transmission of a guidance sound, can also be used as a parameter.

In FIG. 5(b), for example, environment information refers to the surrounding environment in which the mobile device is used, information about the user operating the mobile device, parameters that these information can take, and the like. In BT, these are mainly the information that forms the basis of the Condition node. For example, environmental information includes observation data such as weather, time, and temperature, peripheral sensing data such as target positions and distances to obstacles, and user sensing data such as anxiety levels. Note that even if the parameter is a discrete value such as time (for example, morning/noon/evening/night in the table), it can be a continuous physical quantity such as temperature (for example, -10°C to 40°C in the table). or sensing data such as coordinates or distance.

Moreover, the specification information and the environment information are not limited to the above. Data that can be obtained using the user as an information source may be included. Also, the specification information may be changed in chronological order or according to environmental information. That is, the available functions or possible parameter values may change from moment to moment, or may change according to the environment such as a busy place, a place with few people, a wide road, a narrow road, and the like.

Next, in step ST2, the sequence generator 2 automatically generates a plurality of sequences using the input specification information and environment information (step ST2). This corresponds to the initial population generation process in evolutionary computation. In the case of BT automatic generation using GE, by embedding the specification information and environment information acquired in step ST1 in BNF, the specification information and environment information can be reflected in the generated BT. An example of BNF is shown in Formula (1).

In expression (1), the <action> tag is converted to an Action node, and the <condition> tag is converted to a Condition node. The specific contents of each node are the <act_contents> tag and the <cond_contents> tag, respectively. By embedding the processing contents based on the specification information and environment information in step ST1 above, the actual BT is generated. can be done.

Here, for the processing contents embedded from the specification information and environment information, for example, a known test case enumeration method such as an orthogonal table can be used. Also, in order to reduce the number of parameters, parameters that are continuous values can be discretized, quantized, or abstracted. For discretization of the parameters, for example, equivalence partitioning or predefined threshold processing can be used. Furthermore, if a significant increase in the number of combinations of processing patterns (for example, the number of sequences (BT) to be created becomes enormous, the number of branches of BT becomes enormous, etc.), constraints are added to the BNF. You may suppress the increase of a pattern.

In the subsequent step ST3, the evaluation unit 4 calculates the evaluation value of each sequence generated in step ST2 while referring to the HMI evaluation criteria data stored in the HMI evaluation criteria 3 (step ST3). FIG. 6 shows an example of HMI evaluation criteria data. The evaluation criterion data shown in FIG. 6 includes an ID representing a sequence pattern number, a sequence pattern for obtaining a partial sequence to which the criterion is applied from the sequence to be evaluated, and a partial sequence that feels natural to humans ( and an evaluation criterion for determining whether the conditions for an HMI that is easy to use, pleasant to use, and comfortable are satisfied. It should be noted that the HMI evaluation criteria data only needs to include a sequence pattern and evaluation criteria for determining whether the conditions for HMI that feels natural to humans are satisfied. You may

FIG. 7 shows a flow chart of the sequence evaluation, which is the internal processing of step ST3. First, in step ST11, it is determined whether or not there is an unreferenced HMI evaluation criterion (step ST11). If unreferenced HMI evaluation criteria data exists (Yes in step ST11), the sequence pattern of the unreferenced evaluation criteria is compared with the sequence to be evaluated (step ST12). If there is no unreferenced HMI evaluation criteria data (No in step ST11), the process ends.

After the matching process in step ST12, it is determined in step ST13 whether or not the sequence to be evaluated includes a partial sequence that matches the sequence pattern (step ST13). If the sequence to be evaluated includes a partial sequence, it is acquired (Yes in step ST13). If the partial sequence is not included, the process returns to step ST11 (No in step ST13).

As an example, the sequence pattern of ID1 in the HMI evaluation criteria data shown in FIG. 6 is a child node of the Sequence node (acquired as a wildcard "*"), and the child node acquires an Action related to mobile device control. , It should be noted that, as described above, the definition of the actual sequence pattern and the acquisition of the partial sequence may use pattern matching of the graph or search for the same type of partial graph, in addition to the regular expression of the ID1 sequence pattern.

Next, in step ST14, it is determined whether or not the partial sequence obtained in the process of step ST13 satisfies the evaluation criteria (step ST14). If the evaluation criteria are satisfied (Yes in step ST14), a high evaluation value (for example, 0.8) is set for the sequence (step ST15). If the evaluation criteria are not satisfied (No in step ST14), a low evaluation value (for example, 0.2) is set for the sequence (step ST16). After the processing of steps ST15 and ST16, the process returns to step ST11.

When applied to evolutionary calculation, the process of step ST14 gives a high fitness to a sequence that satisfies the evaluation criteria, and a low fitness to a sequence that does not satisfy the evaluation criteria. Equivalent to feedback. The actual evaluation criteria may be defined based on universal human characteristics such as design principles, known knowledge such as universal design, or indices of comfort and discomfort.

Next, an example of sequence evaluation using IDs 1 to 3 of the HMI evaluation criteria data in FIG. 6 is shown. FIG. 8 is an example of evaluation based on the operation amount of the mobile device when moving to the left front target according to the evaluation criterion ID1. For example, the evaluation criterion ID1 is defined based on the property that a person "feels naturally or comfortably" that a mobile device (vehicle in which he or she rides) moves smoothly. In other words, it is defined based on the property that a person "feels uncomfortable or uneasy about using" a mobile device that operates suddenly.

BT shown in FIG. 8(a) is a sequence of decelerating at 10 km/h after a 90-degree left turn. When this is illustrated, the mobile device operates as shown in FIG. 8(b). On the other hand, BT shown in FIG. 8(c) is a sequence in which the vehicle first turns left at 45 degrees, then decelerates at 5 km/h, then turns left at 45 degrees, and decelerates at 5 km/h. When this is illustrated, the mobile device operates as shown in FIG. 8(d).

In both the BT in FIG. 8(a) and the BT in FIG. 8(c), the mobile equipment decelerates to the same speed while moving to the target. BT in FIG. 8(a) has a large amount of change in traveling direction and speed by one action, while BT in FIG. 8(c) changes step by step. That is, the BT in FIG. 8(a) controls the mobile device with a sharp motion, and the BT in FIG. 8(c) controls it with a smooth motion. According to the evaluation criterion ID1, a low evaluation value (for example, 0.2) is set for the BT of FIG. (eg, 0.8) is set.

FIG. 9 is an example of evaluation based on HMI timing using evaluation criteria ID2. For example, the evaluation criterion ID2 is defined based on the timing of presentation of information notifying that the mobile device (vehicle in which the user rides) moves. Specifically, it is defined based on the property that, unless information is presented before the vehicle in which the person has boarded starts to move, the effect of information presentation is low and the person feels unnatural.

BT in FIG. 9(a) sends a guidance sound after turning left. When this is illustrated, the mobile device operates as shown in FIG. 9(b). On the other hand, the BT in FIG. 9(c) makes a left turn after sending out the guidance sound. When this is illustrated, the mobile device operates as shown in FIG. 9(d). Both BTs perform two operations, turning left and transmitting guidance sound, but since the BT in FIG. , 0.2) are set. On the other hand, BT in FIG. 9(c) is set to a high evaluation value (for example, 0.8) because it starts to operate after transmitting the guidance sound.

FIG. 10 is an example of evaluation based on the order in which modal information is presented, using evaluation criteria ID3. For example, the evaluation criterion ID3 is defined based on the human cognitive load when information is presented using a plurality of modals, such as visual information and auditory information. Specifically, it was defined based on the property that the way of presentation that goes back and forth between modals has a high cognitive load and makes people uncomfortable to use.

In BT in FIG. 10(a), actions related to visual modal and auditory modal appear alternately, so a low evaluation value (eg, 0.2) is set according to evaluation criterion ID3. On the other hand, BT in FIG. 10(b) performs an action related to the visual modal after the action related to the auditory modal. The actions themselves are the same as those in FIG. 10A, but a high evaluation value (for example, 0.8) is set because the modals are in a unified processing order.

In step ST4, the optimization unit 5 performs an optimization process of selecting (searching for) a sequence with a high evaluation value using each sequence assigned an evaluation value in step ST3. This corresponds to the processing of performing various operations such as selection, crossover, and mutation in evolutionary computation. Here, the object of operation, ie, an individual, is a sequence, ie, BT. Further, various calculations are performed using the genetic information of the individual used to generate the BT in step ST2 described above. It should be noted that the selection algorithm is such that, for example, the BT of an individual with a high evaluation value is likely to be selected, while the BT of an individual with a low evaluation value is less likely to be selected. For crossover calculation, for example, the two-point crossover method, for mutation calculation, for example, any known algorithm in evolutionary calculation such as replacement method may be used. Also, when generating BT individuals from genetic information, the BNF of step ST2 described above is used, but the BNF may be modified to generate BTs with properties different from those of the initial population. If the above optimization processing is applied to the process of evolutionary calculation, some excellent BTs with high evaluation values are left in the current generation, and the information of these excellent BTs is used to create next-generation BTs. On the other hand, by eliminating (selecting) BTs with low evaluation values in that generation, BTs are brought closer to ideal BTs each time the generation is updated. This corresponds to processing for increasing the evaluation value of the entire sequence.

In step ST5, it is determined whether or not the optimization process in step ST4 has converged, that is, whether or not the convergence condition is satisfied (step ST5). If the convergence condition is satisfied (Yes in step ST5), the process proceeds to step ST6. If the convergence condition is not satisfied (No in step ST5), each sequence obtained by the optimization process is transferred again to the evaluation unit 4, and the process of step ST3 is repeated.

Here, the convergence condition in step ST5 can be, for example, the case where the average of fitness (that is, evaluation value) of all individuals exceeds a predetermined threshold. In addition, a method generally used in evolutionary computation, such as that the maximum fitness among all individuals does not change for a certain period of time, or that the process is simply repeated a certain number of times, can be applied.

Finally, in step ST6, the output unit 6 acquires one sequence that is the optimum individual (that is, the sequence finally obtained as a result of generational change) and outputs it to the outside or presents it to a person.

As described above, according to the first embodiment, when a sequence is automatically generated, based on the sequence structure information, such as the processing order of the device, the flow of processing, the arrangement of actions, etc., the human sensitivity index , is configured to use HMI evaluation criteria data defined by whether it is natural and comfortable for humans. Therefore, the sequence automatic generation device according to the first embodiment can automatically generate an HMI sequence that performs actions that are natural and comfortable for humans.

Embodiment 2.
A sequence automatic generation device according to Embodiment 2 will be described with reference to FIG. FIG. 11 is a block configuration diagram of an automatic sequence generation device showing the second embodiment. In FIG. 11, a configuration different from that in FIG. In FIG. 11, the same reference numerals as in FIG. 1 denote the same or corresponding parts.

Based on the operating conditions (eg, peripheral information) of the device (i.e., the mobile device with the HMI), the evaluation criterion selection unit 7 instructs the evaluation unit 4 to select which evaluation criterion from among the HMI evaluation criterion data. to apply or select.

FIG. 12 is a flowchart representing the operation of the second embodiment. Except that the process of step ST7 is added to the flowchart of the first embodiment shown in FIG. 4, the process is the same as that of the first embodiment, so the explanation of the operation unrelated to step ST7 will be omitted.

In step ST7, the evaluation criterion selection unit 7 selects HMI evaluation criterion data to be used for evaluation of the target HMI based on the input information received from the input unit 1 (for example, peripheral information that is a type of environmental information). do. FIG. 13 is an example of HMI evaluation criteria data according to the second embodiment. In FIG. 13, a peripheral information column is added as compared with the HMI evaluation criteria data shown in FIG. 6, and the right evaluation criteria are selected according to the peripheral information.

The peripheral information shown in FIG. 13 is the presence or absence of obstacles around the environment in which the mobile device is operating. If there are no obstacles around the mobile device, a sequence in which the device moves smoothly to the target in terms of speed and direction will have a high evaluation value. On the other hand, when there is an obstacle around the mobile device, the closer the mobile device in which the person rides (that is, the vehicle) is to the obstacle, the more the person feels uneasy. At this time, priority should be given to the evaluation of the sequence whether or not the mobile device moves to avoid obstacles. At step ST7, the HMI evaluation criteria data is selected or changed according to the operation criteria (for example, obstacle avoidance behavior) based on the peripheral information. In this way, by switching the sequence evaluation method according to the peripheral information, it is possible to perform an evaluation that is more suitable for the actual environment.

In the second embodiment, peripheral information is given as an example of the situation in which the device operates, and presence or absence of obstacles around the environment in which the mobile device operates is given as an example of the peripheral information. is not limited to For example, the peripheral information may include road surface conditions in the traveling direction of the mobile device (eg, unevenness of the road surface, paved road surface, frozen road surface, inclination angle of the road surface, etc.). The operating conditions of the device can be appropriately set according to the specification information and environmental information of the mobile device.

As described above, in the second embodiment, the evaluation criterion selection unit is configured to switch the evaluation criterion of the sequence according to the peripheral information. Therefore, the sequence automatic generation apparatus according to the second embodiment can perform evaluation more suitable for the actual environment, and can automatically generate an HMI sequence that performs operations that are natural and comfortable for humans. It becomes possible.

Embodiment 3.
As a modification of the second embodiment, the evaluation criterion selection unit 7 may learn or sequentially update the HMI evaluation criterion data corresponding to the peripheral information based on the sequence evaluation result in the evaluation unit 4 .

A sequence automatic generation device according to Embodiment 3 will be described with reference to FIG. FIG. 14 is a block configuration diagram of an automatic sequence generation device showing the third embodiment. In FIG. 14, the configuration different from that in FIG. 11 is that the evaluation unit 4 passes output information to the evaluation criteria selection unit 7, and the evaluation criteria selection unit 7 can access the HMI evaluation criteria 3. . In FIG. 14, the same reference numerals as in FIG. 11 denote the same or corresponding parts.

After calculating the evaluation value of the sequence, the evaluation unit 4 outputs the evaluation value of each sequence to the evaluation criterion selection unit 7 .

The evaluation criterion selection unit 7 refers to the evaluation values of each sequence, and, for example, if the variance of the evaluation values according to a certain evaluation criterion is smaller than a predetermined threshold value, it determines that individual differences have not been sufficiently evaluated, The HMI evaluation criteria data in the HMI evaluation criteria 3 is learned (eg, switching to another evaluation criteria, modifying the evaluation criteria, etc.).

As described above, in the third embodiment, the sequence evaluation results are used to learn (update) the peripheral information and the corresponding HMI evaluation criteria data. Therefore, the automatic sequence generation device according to the third embodiment can improve the accuracy of automatic generation of HMI sequences that perform actions that are natural and comfortable for humans.

In each of the above-described embodiments, BT is used as an example of a sequence format, and evolutionary computation, especially GE, is used as an example of a framework for automatic sequence generation, but the present invention is not limited to these. That is, as long as the same functions and effects can be obtained, it may be used as a form using it, or another known format such as a state chart, or a known framework such as a genetic algorithm may be used. In addition, mobile equipment has been described as an example of equipment to be controlled, but for example, home appliances (televisions, air conditioners, etc.), transportation equipment (elevators, etc.), manufacturing equipment (industrial robots, factory production equipment), etc., HMI It can also be applied to other devices with

In addition to the above, within the scope of the disclosure, any component of the embodiment can be modified, or any component of the embodiment can be omitted.

1 input unit, 2 sequence generation unit, 3 HMI evaluation criteria, 4 evaluation unit, 5 optimization unit, 6 output unit, 7 evaluation criteria selection unit,
11 memory, 12 processor, 13 storage medium, 14 input device, 15 output device, 16 signal path,
100 sequence automatic generator

Claims

a sequence generation unit that generates sequences of a plurality of Human Machine Interfaces (HMIs) using specification information about functions of a device to be controlled and environment information about the situation in which the device is used;
HMI evaluation criteria data having one or more evaluation criteria defined by using structural information indicating the sequence processing order of the equipment and human sensitivity indices, and using the evaluation criteria to generate the plurality of HMIs. an evaluation unit that evaluates the sequence of and assigns an evaluation value;
An automatic sequence generation device comprising an optimization unit that selects an HMI sequence using the evaluation value from the plurality of HMI sequences to which the evaluation value is assigned.
The optimization unit changes the internal state of the HMI sequence selected from the plurality of HMI sequences using evolutionary calculation, re-inputs the selected HMI sequence to the evaluation unit, and converges to a predetermined level. 2. The sequence automatic generation device according to claim 1, wherein the evaluation value of the entire sequence is increased until a condition is satisfied.
The evaluation unit assigns a high evaluation value to a sequence that satisfies the evaluation based on the HMI evaluation criteria data, and gives a low evaluation value to a sequence that does not satisfy the evaluation based on the HMI evaluation criteria data. 3. The sequence automatic generation device according to claim 1 or claim 2.
4. The method according to any one of claims 1 to 3, further comprising an evaluation criterion selection unit that instructs the evaluation unit to select an evaluation criterion from among the HMI evaluation criterion data according to a situation in which the device operates. Automatic sequence generator as described.
5. The evaluation criterion selection unit learns the HMI evaluation criterion data according to the result of the sequence evaluation by the evaluation unit so as to correspond to the situation in which the device operates. The sequence automatic generation device according to any one of 1.
a sequence generation step of generating a sequence of a plurality of HMIs using specification information about functions of a device to be controlled and environment information about the situation in which the device is used;
HMI evaluation criteria data having one or more evaluation criteria defined by using structural information indicating the sequence processing order of the equipment and human sensitivity indices, and using the evaluation criteria to generate the plurality of HMIs. an evaluation step for evaluating the sequence of and assigning an evaluation value;
An automatic sequence generation method comprising an optimization step of selecting an HMI sequence using the evaluation value from the plurality of HMI sequences to which the evaluation value is assigned.
The optimization step changes the internal state of the HMI sequence selected from the plurality of HMI sequences using evolutionary calculation, re-inputs the selected HMI sequence to the evaluation step, and converges to a predetermined level. 7. The method of automatically generating a sequence according to claim 6, wherein the evaluation value of the entire sequence is increased until the condition is satisfied.
wherein the evaluation step assigns a high evaluation value to a sequence that satisfies the evaluation based on the HMI evaluation criteria data, and gives a low evaluation value to a sequence that does not satisfy the evaluation based on the HMI evaluation criteria data. The sequence automatic generation method according to claim 6 or claim 7.
9. The method according to any one of claims 6 to 8, further comprising an evaluation criterion selection step of instructing selection of an evaluation criterion from among the HMI evaluation criterion data in accordance with a situation in which the device operates, for the evaluation step. Automatic sequence generation method as described.
10. The evaluation criterion selection step learns the HMI evaluation criterion data so as to correspond to the situation in which the device operates according to the result of the sequence evaluation in the evaluation step. The sequence automatic generation method according to any one of 1.
a sequence generation step of generating a sequence of a plurality of HMIs using specification information about functions of a device to be controlled and environment information about the situation in which the device is used;
HMI evaluation criteria data having one or more evaluation criteria defined by using structural information indicating the sequence processing order of the equipment and human sensitivity indices, and using the evaluation criteria to generate the plurality of HMIs. an evaluation step for evaluating the sequence of and assigning an evaluation value;
A program causing a computer to execute an optimization step of selecting an HMI sequence using the evaluation value from the plurality of HMI sequences to which the evaluation value is assigned.
The optimization step changes the internal state of the HMI sequence selected from the plurality of HMI sequences using evolutionary calculation, re-inputs the selected HMI sequence to the evaluation step, and converges to a predetermined level. 12. A program according to claim 11, characterized in that the evaluation value of the entire sequence is increased until the condition is met.
wherein the evaluation step assigns a high evaluation value to a sequence that satisfies the evaluation based on the HMI evaluation criteria data, and gives a low evaluation value to a sequence that does not satisfy the evaluation based on the HMI evaluation criteria data. 13. The program according to claim 11 or 12.
14. The method according to any one of claims 11 to 13, further comprising an evaluation criterion selection step of instructing selection of an evaluation criterion according to a situation in which the device operates from the HMI evaluation criterion data in the evaluation step. program as described.
14. The HMI evaluation criterion data is learned in the evaluation criterion selection step so as to correspond to the situation in which the device operates according to the result of the sequence evaluation in the evaluation step. The program according to any one of Claims 1 to 3.