JP2022158400A - System for controlling execution system including multiple subsystems - Google Patents

Abstract

To improve overall control of a system including multiple subsystems.SOLUTION: A system is disclosed for controlling an execution system that includes multiple subsystems. A storage device stores data for constructing a first model for extracting feature quantities of each input graph. A processor acquires measurement data from each of the multiple subsystems and generates graphs representing the measurement data of each of the multiple subsystems based on the measurement data from each of the multiple subsystems. The processor uses the first model to extract feature quantities from respective graphs and controls the execution system based on the feature quantities.SELECTED DRAWING: Figure 5

Description

The present invention relates to a system for controlling an execution system containing multiple subsystems.

Many systems today consist of multiple heterogeneous subsystems, increasing system complexity. On the other hand, there is a demand for more efficient operation and control of the entire system consisting of these subsystems. Examples of such subsystems are power grids, telecommunication networks, rail and road networks, logistics networks, supply chains, financial transactions, the flow of people in offices and shopping malls, and the like.

Also, it has been proposed to analyze or automatically control a system by an AI (Artificial Intelligence) model using machine learning. Various machine learning methods have been proposed. For example, Non-Patent Document 1 discloses a classification method using a graph convolution network.

Thomas N. Kipf, Max Welling, "Semi-Supervised Classification with Graph Convolutional Networks", ICLR 2017.

Data areas such as types of data targeted by the subsystems, data structures, systems for acquiring data, and KPIs (Key Performance Indicators) differ among the different types of subsystems that make up the system. Therefore, it is difficult to describe the relationships between heterogeneous subsystems and optimize the control of the entire system.

A representative example of the present disclosure is a system for controlling an execution system including a plurality of subsystems, comprising: one or more processors; one or more storage devices that store programs executed by the one or more processors; ,including. The one or more storage devices store data for constructing a first model for extracting the feature amount of each input graph. The one or more processors acquire measurement data from each of the plurality of subsystems and generate graphs representing the measurement data of each of the plurality of subsystems based on the measurement data from each of the plurality of subsystems. and extracting features from each of the graphs using the first model and controlling the execution system based on the features.

Exemplary embodiments of the present invention provide improved control over systems that include multiple subsystems. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

1 schematically shows a functional configuration example of an execution system including a plurality of subsystems and a control system that controls the execution target system; 4 schematically shows a hardware configuration example of a system control unit; An example of measurement of purchasing behavior by the measurement unit of the subsystem is schematically shown. 4 schematically shows an example of congestion degree measurement by a measuring unit of a subsystem in a shopping mall. 4 shows a flowchart of an example of processing executed by a system control unit. 4 shows an example of a graph representing a user's purchasing behavior over a predetermined period of time, generated from measurement data of a subsystem; 6 shows an example of a graph representing the degree of congestion during a predetermined period, which is generated based on the measured data of the subsystem; 4 schematically shows an example of a logical configuration of a graph feature extraction unit and an example of a partial logical configuration of a state-action-value update unit;

The following description is divided into multiple sections or examples when necessary for convenience, but they are not independent of each other and one may be a part of the other or All modifications, details, supplementary explanations, etc. are related. In addition, hereinafter, when referring to the number of elements (including number, numerical value, amount, range, etc.), unless otherwise specified or clearly limited to a specific number in principle, is not limited to the number of , and may be greater than or less than a specific number.

A system according to one embodiment herein includes an execution system that performs actions on an environment and a control system that controls the execution system. The execution system includes a plurality of different types of subsystems, each having a different KPI (Key Performance Indicator).

The control system graphically represents each of the multiple subsystems. A graph is a data type that contains nodes and edges that connect the nodes. Graphs allow abstraction of heterogeneous subsystems with different data domains, such as data types, data structures, data acquisition systems, and KPIs, into a common representation.

The control system uses graphs to optimize overall system control in systems where different data domain subsystems are combined. The control system can balance the KPIs of multiple subsystems and optimize the control of the entire execution system in an execution system in which the relationships between multiple subsystems are difficult to describe.

A system according to one embodiment of the present specification is described below. The system includes an execution system that acts and affects the environment and a control system that controls the execution system. In the following, an example of controlling the flow of customers in a shopping mall will be described. The execution system controls the behavior of patrons by executing specific actions in the environment, the shopping mall.

The control disclosed herein can be applied to any system where the data obtained from the subsystems can be represented graphically. The control system disclosed herein is useful for controlling execution systems that include heterogeneous subsystems in environments different from shopping malls, such as power networks, communication networks, railway/road networks, distribution networks, supply chains, and financial transactions. Applicable.

As mentioned above, the execution system includes heterogeneous subsystems. Each subsystem has different KPIs and performs different actions. In the example explained below, in a shopping mall with multiple stores, one subsystem measures the purchasing behavior of shoppers and distributes coupons to smartphone apps in order to improve the overall sales of the shopping mall. to encourage purchasing behavior.

Another example of a subsystem is to reduce congestion in a shopping mall by measuring the congestion situation on the aisles of users and presenting the congestion situation on signage to reduce congestion. Encourage users to move.

The first KPI above is the sales of the entire shopping mall, and the other KPI is the degree of congestion in the aisles. Thus, the two subsystems have different KPIs. A control system according to one embodiment herein performs the task of balancing and maximizing different KPIs, such as increasing sales and reducing congestion, among subsystems. . The control system learns optimal control for this task.

FIG. 1 schematically shows a functional configuration example of an execution system including a plurality of subsystems and a control system that controls the execution target system. The execution system to be controlled includes a plurality of subsystems, and FIG. 1 specifies two subsystems 0 and 1 as an example. Any number of subsystems may be included in the execution system.

Subsystem 0 includes a measurement unit 101 and a control unit 104 . Subsystem 1 includes a measurement unit 102 and a control unit 105 . The environment 106 is what each subsystem works on. In the example described below, environment 106 corresponds to a shopping mall. The system control unit 103 cooperates with each of the plurality of subsystems and controls the entire execution system.

The measurement units 101 and 102 collect data necessary for improving the KPIs of the corresponding subsystems 0 and 1 in the environment 106 . Control units 104 and 105 execute actions in environment 106 that are determined based on the data collected by measurement units 101 and 102 as necessary for improving the KPI of the corresponding subsystem, respectively.

The KPI for subsystem 0 is the sales for the entire shopping mall. The higher the sales, the more favorable the situation is evaluated. The measurement unit 101 of the subsystem 0 includes a POS (Point of Sale) system using point cards. The measurement unit 101 collects measurement data such as the date and time of purchase behavior, the store used, the purchase amount, the user's gender, the user's age, etc., in association with the unique ID assigned to the point card. The measurement unit 101 also collects information on stores in the shopping mall, such as store type, area, total sales, and the like.

The control unit 104 of the subsystem 0 includes smartphones used by shopping mall users and applications (application programs) executed by them. The smartphone can display information transmitted from the system control unit 103, such as discount coupons and store locations.

The KPI of subsystem 1 is the degree of congestion in the shopping mall. The smaller the degree of congestion, the more preferable the state is evaluated. The measurement unit 102 of the subsystem 1 includes multiple TOF (Time of Flight) sensors installed in the shopping mall.

The TOF sensor can collect the position of each shopping mall user within the measurement range as measurement data. The position of the user obtained by the TOF sensor must be grasped as the position of the entire shopping mall. The measurement unit 102 holds the position and orientation (orientation) of each TOF sensor and a map of the shopping mall (data describing geometric shapes and attributes of stores and aisles).

The control unit 105 includes a digital signage and a user's smartphone that executes a smartphone application. Digital signage and smart phones can display information transmitted from the system control unit 103 . For example, information that encourages movement of the user, such as congestion status for each location in the shopping mall, discount coupons with specific attachments, event information at event venues, etc., can be distributed.

The system control unit 103 includes a graph processing unit 107, a measurement data reception unit 108, a state action value update unit 111, an action selection unit 112, an action/control data conversion unit 113, a control data transmission unit 114, a learning management unit 115, and an error generation unit. It includes a unit 116 and a measurement data/reward conversion unit 117 . The graph processing unit 107 includes a measurement data/graph conversion unit 109 and a graph feature extraction unit 110 .

A measurement data reception unit 108 receives measurement data obtained by the measurement unit of each subsystem. The measurement data/reward conversion unit 117 converts the measurement data obtained by the measurement data reception unit 108 into a numerical value called a reward, which corresponds to the KPI of each subsystem.

The measurement data/graph conversion unit 109 converts the measurement data obtained by the measurement data reception unit 108 into graph data including nodes, edges, and attribute data for nodes. If the measurement data contains graphs, conversion processing is unnecessary.

The graph feature extraction unit 110 extracts feature amounts from the graph generated by the measurement data/graph conversion unit 109 . According to one embodiment of the present specification, each node reflects the value of an adjacent node by performing a convolution process on the attribute data of each node within a range of a predetermined number of steps or less. It is responsible for the processing of calculating the amount (feature amount) obtained by The number of steps is the shortest distance from one node to another node via an edge, and the distance to an adjacent node is one step.

The state action value updating unit 111 estimates the value of actions that can be taken in a certain state (a combination of control data that can be commanded to a subsystem) along the framework of reinforcement learning, and Based on the reward, it updates (learns) the value of actions taken in a certain state. The action selection unit 112 stochastically selects an action that is expected to give a high reward, based on the value of actions that can be taken in a certain state, in accordance with the framework of reinforcement learning.

The action/control data conversion unit 113 converts the action selected by the action selection unit 112 into a combination of control data that can be commanded to the subsystem. The control data transmission unit 114 transmits control data generated by the action/control data conversion unit 113 to each subsystem.

The learning management unit 115 manages the overall processing, evaluates the progress of learning in particular, and determines whether to apply an error to the measurement data for learning. The error generator 116 applies an error to the measurement data used for learning.

FIG. 2 schematically shows a hardware configuration example of the system control unit 103. As shown in FIG. The system control unit 103 can have a computer configuration. It includes a processor 151 having arithmetic performance and a DRAM 152, which is a main memory providing a volatile temporary storage area for storing programs and data executed by the processor 151. FIG. Furthermore, the system control unit 103 includes an auxiliary storage device 154 that provides a permanent information storage area using a HDD (Hard Disk Drive), flash memory, or the like. DRAM 152, secondary memory 154, and combinations thereof are each memory devices.

The system control unit 103 further includes a communication device 153 that performs data communication with other devices including other devices in this system, an input device 155 that receives operations from the user, and presents the output results of each process to the user. and a monitor 156 (an example of an output device). These components can communicate via a bus. The number of each component of the system control unit 103 is arbitrary, and some components such as the input device 155 and the monitor 156 may be omitted.

The functional units of the system control unit 103 described with reference to FIG. 1 can be implemented, for example, by the processor 151 that executes programs including instruction codes. Programs for realizing the functional units are stored in the auxiliary storage device 154, for example. Programs to be executed by the processor 151 and data to be processed are loaded from the auxiliary storage device 154 to the DRAMU 152 . Functions within the system may be implemented by function-specific circuitry in place of a processor operating according to a program.

The system control unit 103 may be a physical computer system (one or more physical computers) as shown in FIG. It can be system. A computer system or a group of computing resources includes one or more interface devices (for example, including communication devices and input/output devices), one or more storage devices (for example, including memory (main storage) and auxiliary storage devices), and one or more processor.

When a function is realized by executing a program by a processor, the defined processing is performed while appropriately using a storage device and/or an interface device, etc., so the function may be at least part of the processor. good. A process described with a function as the subject may be a process performed by a processor or a system including the processor.

Programs may be installed from program sources. The program source may be, for example, a program distribution computer or a computer-readable storage medium (eg, a computer-readable non-transitory storage medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

Other components included in the system shown in FIG. 1, specifically, the measurement units 101 and 102 and the control units 104 and 105 can each include a computer system that performs necessary processing. At least part of the functions of the measurement units 101 and 102 and the control units 104 and 105 may be implemented in a computer in which the functions of the system control unit 103 are implemented.

Next, the flow of processing in the entire system will be described. First, each subsystem measures a given target in environment 106 . As described above, the measuring unit 101 of the subsystem 0 measures the purchasing behavior of shopping mall users. The measurement unit 102 of the subsystem 1 measures the degree of congestion in the shopping mall 106 .

FIG. 3 schematically shows an example of purchase behavior measurement by the measurement unit 101 of the subsystem 0 . FIG. 3 shows a plurality of stores 301 in shopping mall 200 and aisles 302 connecting stores 301 . In FIG. 3, store 301 is indicated by a rectangle. In FIG. 3, one store and aisle are indicated by numerals 301 and 302 for ease of illustration. Stores 301 are lined up along aisles 302 .

FIG. 3 shows movement 303 accompanied by the user's purchasing behavior with an arrow. A user's purchases can be measured by referring to the use of the point card by the user through the POS system. In FIG. 3, one arrow is indicated by reference numeral 303 for ease of illustration. Each arrow 303 indicates two stores 301 where the user made purchases in succession. Each arrow 303 indicates that the user made a purchase at the store 301 at the start point of the arrow 303 and then made the next purchase at the store 301 at the end point of the arrow 303 .

It is assumed that the user uses a point card at a certain store, moves to the next store 205, and takes a purchasing action of using the point card. The arrow 303 can indicate the movement of the user accompanied by the order of use of the two shops by indicating the first shop and the next shop with starting and ending points, respectively. Note that the arrow 303 merely expresses the user's purchase at the store and the accompanying movement for convenience, and does not represent the actual physical movement route between the stores.

FIG. 4 schematically shows an example of congestion degree measurement by the measurement unit 102 of the subsystem 1 in the shopping mall 200. As shown in FIG. In FIG. 4, sector 401 indicates the measurement range of the TOF sensor. A vertex 402 of the sector 401 indicates the viewpoint position of the TOF sensor. In FIG. 4, for ease of illustration, reference numeral 401 denotes the measurement range of a single TOF sensor as an example.

A sector 401 indicates the measuring range of the TOF sensor. Each TOF sensor is installed so that its measurement range includes a portion of passageway 302 . Each TOF sensor can continuously measure users within the measurement range 401, specifically, the number of users at each measurement time, the movement route of each user, and each user at each measurement time Velocity can be measured. Note that the TOF sensor may be installed so as to include the store in its measurement range.

As previously mentioned, only Subsystem 0 and Subsystem 1 are illustrated for simplicity, but more subsystems may be included. The other subsystems, like subsystems 0 and 1, respectively, perform measurement processing. Also, measurement data obtained in each subsystem is transmitted to the system control unit 103 .

Next, processing in the system control unit 103 based on data measured in each subsystem will be described. FIG. 5 shows a flowchart of a processing example executed by the system control unit 103 . The system control unit 103 executes feature extraction by graph convolution, determination of actions to be executed for each subsystem, and learning of action values.

The subsystem executes an action on the environment 106 corresponding to the action of the system control unit 103 . Therefore, the system control unit 103 determines the behavior of the subsystems with respect to the environment 106 based on the measurement data.

Referring to FIG. 5, learning management unit 115 executes initialization processing (501). The initialization process, for example, sets initial values of parameters of the machine learning model for determining actions. Next, the measurement data receiving unit 108 receives the measurement data transmitted from the measurement unit of each subsystem (502). The measurement data/remuneration conversion unit 117 calculates the value of the remuneration of the entire execution system from the measurement data received from the measurement unit of each subsystem (503). Rewards are calculated based on KPIs (rewards) for each subsystem.

More specific description will be given. Assume that the reward R0 of subsystem 0 is expressed as the total sales of all stores in the shopping mall over a certain period of time. The measurement data/reward conversion unit 117 calculates a reward R0 based on the measurement data indicated by the POS system of the subsystem 0. FIG.

Similarly, it is assumed that the reward R1 of the subsystem 1 is represented by the average distance between users within the measurement range of the TOF sensor in the shopping mall. For example, in the measurement range of each TOF sensor, the average distance between users over a certain period of time is calculated. Users present in the measurement range and distances between users change with time.

The calculated average distance is, for example, the average value of the average distances at each time within a certain period. The measurement data/reward conversion unit 117 calculates the average value of the average distances of all TOF sensors and determines it as a reward. A weight may be given to each measurement value of the TOF sensor. In this example, the period for calculating rewards for each of subsystems 0 and 1 is common.

This R1 is calculated based on the measurement data. Similarly, for other subsystems, rewards corresponding to the KPIs of the subsystems are calculated from their respective measurement data. Note that multiple rewards may be defined for one subsystem.

Assuming that the number of subsystems is (N+1), the total reward R of all subsystems is calculated by the following formula.
R=α0・K0・R0+α1・K1・R1+...+αN・KN・RN

α(i) is a weighting factor for each subsystem and represents how much importance is given to each subsystem. α(i) is a real number equal to or greater than 0 and equal to or less than 1, and α0+α1+ . . . +αN=1. K(i) is a coefficient for normalizing the reward of each subsystem. α(i) and K(i) are preset constants.

Next, the measurement data/graph conversion unit 109 generates graph data for each subsystem from the measurement data received from each subsystem for a certain period of time (504). In this example, the period of measurement data for graph generation is common to all subsystems and is the same as the period for calculating rewards. This allows for more appropriate action. Depending on the configuration of the execution system, the period for graph generation may differ between subsystems.

The measurement data measured by the POS system of subsystem 0 as described above may include, for example, date and time of purchase, user age, store used, purchase amount, type of store, area of store, total sales of store, and the like. The measurement data/graph conversion unit 109 uses the values of the measurement data to generate a graph having the stores as nodes and the customer's purchase-related movements between stores as edges. The node is given an attribute vector composed of one or more values representing store attributes generated from the measurement data. Edges are defined, for example, for movements by users exceeding a specified number.

FIG. 6 shows an example of a graph 600 representing a user's purchasing behavior over a period of time, generated from sub-system 0 measurement data. In FIG. 6, circles 601 and 603 are nodes representing stores. Line 604 is an edge that represents a user's movement between stores. In FIG. 6, some nodes and edges are indicated by symbols as an example. A dashed range 602 indicates the node 601 and adjacent nodes of the node. As described above, attribute data (attribute vector) based on the user's measurement data is assigned to each node.

The measurement data includes data unique to each store and is reflected in the attribute data of each node. Attribute data can be represented by a vector consisting of one or more values obtained from measurement data. The attribute data of the store node can indicate, for example, the store's user average purchase amount, total sales, user age average, store area, store type, and the like.

The measurement data measured by each TOF sensor of the subsystem 1 as described above includes temporal changes in the position of each user within the measurement range within a predetermined period. In addition to the measurement data of each TOF sensor, the subsystem 1 transmits map information of the shopping mall and attribute information of each TOF sensor, such as position and orientation information of the TOF sensor, to the system control unit 103. can be done. Map information indicates the geometry and attributes of stores and aisles. The map information of the shopping mall and the attribute information of the TOF sensor may be preset in the system control unit 103 by the system administrator.

The graph generated by the measurement data/graph conversion unit 109 can be composed of nodes representing a plurality of preset representative points in the passage of the shopping mall and edges representing passages between the representative points. The measurement data/graph conversion unit 109 can define the nodes and edges of the graph from the map data of the shopping mall.

The measurement data/graph conversion unit 109 specifies the measurement range of each TOF sensor from the position and orientation of each TOF sensor and map information, and further calculates the degree of congestion at each representative point from the measurement data of each TOF sensor. . The degree of congestion can be represented, for example, by the average distance between users as described above.

The measurement data/graph conversion unit 109 includes the attribute data of each node corresponding to the representative point. Attribute data can also include other values indicative of congestion, such as the average number of users, and location attributes. The attribute of a point can indicate, for example, a point in front of a store having a specific attribute, an intersection, a resting plaza, an event site, and the like.

FIG. 7 shows an example of a graph 700 representing the degree of congestion during a predetermined period, which is generated based on the measurement data of subsystem 1 . In FIG. 7, circles 701 and 703 are nodes representing representative points such as points in front of the store and intersections. A line 704 is an edge representing a path connecting representative points. In FIG. 7, some circles and edges are indicated by symbols as an example. The range indicated by dashed line 702 indicates node 701 and its neighboring nodes. As described above, attribute data (attribute vector) based on the user's measurement data is assigned to each node.

Returning to FIG. 5, next, the graph feature extraction unit 110 extracts features of the graph data of each subsystem (505). The graph feature extractor 110 according to one embodiment of this specification uses a graph convolutional network. In graph convolution, a convolution operation is performed for each node of a graph. Specifically, attribute data (attribute vectors) of adjacent nodes connected to each node are weighted and added. Note that attribute data of nodes (nodes of two or more steps) farther than adjacent nodes (nodes of one step) may be included in the convolution operation.

Focusing on the node 601 in FIG. 6 among the graph data by the subsystem 0, the attribute data of the adjacent nodes (nodes within the range 602) connected to the node 601 are folded into the attribute data of the node 601. FIG. As a result, the attribute data of the adjacent nodes connected to the node 601 are reflected in the attribute data of the node 601, and the tendency of the attribute data around the node 601, that is, the characteristics are obtained.

The graph data by the subsystem 1 is also the same. Focusing on the node 701 in FIG. 7, the attribute data of adjacent nodes (nodes within the range 702) connected to the node 701 are convoluted with the attribute data of the node 701. FIG. Similar convolution processing is performed on graph data obtained by other subsystems.

FIG. 8 schematically shows a logical configuration example of the graph feature extraction unit 110 and a partial logical configuration example of the state-action-value update unit 111 . The graph feature extraction unit 110 receives graph data based on the measurement data of each subsystem from the measurement data/graph conversion unit 109 . FIG. 8, by way of example, indicates graph data for subsystems 0 and 1 respectively at 801 and 802 . The graph data indicates the graph structure and the attribute data of each node.

The graph feature extractor 110 inputs the graph data of each subsystem to the corresponding graph convolutional neural network 807 . A graph convolutional neural network 807 is an example of a model for extracting feature amounts of an input graph, and data for constructing the model is stored in a storage device.

FIG. 8 designates one graph convolutional neural network as an example at 807 . In the configuration example of FIG. 8 , the graph convolutional neural network is composed of multiple convolution layers 805 . FIG. 8 designates one convolutional layer at 805 as an example.

Convolution processing (one convolution processing) by one convolution layer 805 convolves the attribute data of directly connected adjacent nodes into the attribute data of each node. A plurality of convolution layers 805 are connected, and multiple convolution processes are performed on each attribute data. As a result, the attribute data of a distant node that is not directly connected is convoluted with the attribute data of each node, and a feature quantity reflecting the attribute data of a node distant from each node is extracted.

Returning to FIG. 5, next, the state-action-value update unit 111 executes state-action-value update processing (506). This process corresponds to updating state-action values in the framework of reinforcement learning. A state indicates a feature amount obtained by convolution processing of node attribute data based on the graph structure.

Action refers to the selection from a candidate set of control data for each subsystem. One control data candidate can include, for example, distribution of discount coupons and store locations using a smartphone application, distribution of congestion conditions for each location in a shopping mall using digital signage and a smartphone application, and the like. Different control data candidates may indicate different delivery information and/or different delivery destinations, for example.

The update of the state action value is based on the reward obtained as a result of the action selected in the previous state, and the estimated value of choosing that action in the previous state is Update the parameters of the estimated value or function used for estimation to match the reward obtained.

An example of updating the state action value will be described with reference to FIG. The state-action-value updating unit 111 receives the feature amount (vector) of the graph data of each subsystem from the graph feature extracting unit 110 . FIG. 8 clearly shows a purchasing behavior feature quantity 803 and a congestion degree feature quantity 804 as an example. A set of features indicates the current state si.

The state-action-value update unit 111 includes a fully connected layer 806 and receives feature quantities from the graph feature extraction unit 110 . Fully connected layer 806 outputs the Q-values of actions a0 through aM in state si. M is a natural number. FIG. 8 demonstrates, as an example, the Q-value (si, a0) 808 of action a0 and the Q-value (si, aM) 809 of action aM. The Q value indicates action value. The state action value updating unit 111 updates the parameters of the graph convolution neural network 807 and the neural network of the fully connected layer 806 so that the difference (error) between the calculated Q value and the actually obtained reward becomes small. .

The fully connected layer 806 is an example of a model that determines index values for determining control data for controlling the execution system based on feature amounts from the graph feature extraction unit 110 . Data for constructing the model is stored in a storage device. Different models than fully connected layers may be used.

Returning to FIG. 5, next, the action selection unit 112 executes action selection processing (507). The action selection unit 112 selects a high-value action with a high probability from actions that can be taken in the current state. Next, the action/control data conversion unit 113 executes action/control data conversion processing (508). As mentioned above, actions correspond to the distribution of discount coupons and store locations using smartphone applications, but in order to realize this, they are converted into data corresponding to each device.

Next, the control data transmission unit 114 executes control data transmission processing (509). Thereby, the control data is transmitted to the control unit of each subsystem, and each control unit executes processing based on the control data. For example, distribution of coupons to the above-mentioned smartphone application, presentation of congestion status on signage, etc. are performed.

With the series of processes described above as one episode, the learning management unit 115 advances learning by executing a plurality of episodes. After completing each episode, the learning management unit 115 performs learning management determination processing (510). For example, when the number of episodes reaches a predetermined number, or when the change rate of the sum of episode rewards becomes smaller than a threshold, learning is determined to be completed. If learning is not completed (510: not completed), the flow returns to step 502;

When learning is completed (510: completed), the learning management unit 115 executes robust learning completion determination processing (511). After the normal learning, the learning management unit 115 determines whether or not learning has already been performed with an error applied to the measurement data. If learning is not performed by applying an error (511: incomplete), the error generator 116 sets an error (512). Thereafter, the flow returns to step 502, and the series of processes described above are performed with the error applied to the measurement data.

An error is added to numerical data among measurement data. For example, an error can be added to the number of users measured by the TOF sensor. Learning is performed with errors in the attribute data of the graph. Based on the learning result obtained in the initial learning, learning is performed in a situation in which an error is further added, making it possible to learn more robust control against errors.

Learning is performed with the error applied as described above, and the learning management unit 115 executes robust learning completion determination processing (511). If it is determined that the learning is completed (511: completed), this process ends. For example, when a predetermined number of episodes is reached or when the rate of change in the sum of rewards for each episode becomes smaller than a threshold, it is determined that the robust learning process is completed. If the robust learning is not completed (511: incomplete), the error is changed and the processing of the learning/behavior selecting unit is executed again.

The processing shown in FIG. 5 performs machine learning of the learning model while operating the execution system in the real environment 106 . After completing the machine learning shown in FIG. 5, the system control unit 103 can use the trained model to control the execution system based on the measurement data, or the system control unit 103 can perform the flow shown in FIG. may continue without ending. The learning process by the system control unit 103 may be performed on the actual environment 106 or may be performed on the simulated environment 106 .

As described above, the system acquires the control law of each subsystem that increases the sum of the KPIs of the subsystems in the form of a reward while maintaining a balance.

In the above embodiment, attribute data is added only to nodes, out of nodes and edges. Another example is to add attribute data to edges and convolve them. Also, the graph neural network for feature extraction is not limited to the above configuration. For example, by combining a pooling layer or a dropout layer with a convolution layer, it is possible to emphasize features. Also, a model different from convolution or neural network may be used for extracting feature values of graph data.

The above example determines data for the execution system based on behavioral values and updates learning model parameters based on behavioral values and rewards in a reinforcement learning framework. The system may control the execution system based on features extracted from graphs of sub-models according to other techniques.

A system according to one embodiment of the present disclosure can be applied to control an execution system that includes different metrology data subsystems. The measurement data of some subsystems may be of the same type, and the system of one embodiment of the present specification may be applied to an execution system comprising subsystems of the same type of measurement data.

In addition, the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations. In addition, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

Further, each of the configurations, functions, processing units, etc. described above may be realized by hardware, for example, by designing a part or all of them using an integrated circuit. Moreover, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, and files that implement each function can be stored in recording devices such as memories, hard disks, SSDs (Solid State Drives), or recording media such as IC cards and SD cards.

In addition, the control lines and information lines indicate those considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In fact, it may be considered that almost all configurations are interconnected.

101, 102 measurement unit 103 system control unit 104, 105 control unit 106 environment 107 graph processing unit 108 measurement data reception unit 109 measurement data/graph conversion unit 110 graph feature extraction unit 111 state action value update Unit 112 Action Selection Unit 113 Action/Control Data Conversion Unit 114 Control Data Transmission Unit 115 Learning Management Unit 116 Error Generation Unit 117 Measurement Data/Reward Conversion Unit 151 Processor 152 DRAM 153 Communication Device 153 Monitor, 154 Auxiliary storage device, 155 Input device, 200 Shopping mall, 205, 301 Store, 302 Passage, 303 Movement of user, 401 Measurement range, 402 Sensor viewpoint, 600, 700 Graph data, 601, 603, 701, 703 node, 602, 702 adjacent node, 604, 704 edge, 803, 804 feature quantity, 805 convolution layer, 806 fully connected layer, 807 graph convolution neural network, 808, 809 Q value

Claims (10)

A system for controlling an execution system comprising multiple subsystems, comprising:
one or more processors;
and one or more storage devices that store programs executed by the one or more processors,
The one or more storage devices are
storing data for constructing a first model for extracting feature values of each input graph;
The one or more processors
obtaining measurement data from each of the plurality of subsystems;
generating a graph representing the metrology data for each of the plurality of subsystems based on the metrology data from each of the plurality of subsystems;
extracting features from each of the graphs using the first model;
A system that controls the execution system based on the feature quantity. 2. The system of claim 1, wherein
The one or more storage devices are
data for configuring a second model for determining an index value for determining control data for controlling the execution system based on the feature value;
The one or more processors
using the second model to obtain an index value based on the feature amount of the graph;
A system that determines control data for the executing system based on the index value. 2. The system of claim 1, wherein
the first model includes a sub-model for each of the plurality of subsystems;
The system, wherein each of the sub-models includes multiple convolutional layers. 2. The system of claim 1, wherein
The system, wherein the metrology data of the plurality of subsystems includes different types of metrology data. 2. The system of claim 1, wherein
The system, wherein the measurement data of the plurality of subsystems are measured in the same period. 3. The system of claim 2, wherein
The system, wherein the one or more processors update parameters of the first model and the second model based on the obtained index value. 7. The system of claim 6, wherein
index values from the second model represent behavioral values for the plurality of subsystems;
The one or more processors
selecting an action for the plurality of subsystems based on the value of the action;
determining a reward for the selected behavior based on measured data from the plurality of subsystems;
A system that updates parameters of the first model and the second model based on the value and the reward. 7. The system of claim 6, wherein
The system, wherein the one or more processors use the graph plus error data to update the parameters of the first model and the second model. 2. The system of claim 1, wherein
the execution system controls the flow of people in the shopping mall;
measurement data of a first subsystem in the plurality of subsystems includes data related to sales at stores in the shopping mall;
The system, wherein the metrology data of a second subsystem in the plurality of subsystems includes data regarding the location of the person in the shopping mall. A method for a system to control an execution system comprising multiple subsystems, comprising:
The system stores data for constructing a first model that extracts features of each input graph,
The method includes
the system obtaining metrology data from each of the plurality of subsystems;
the system generating graphs representing metrology data for each of the plurality of subsystems based on metrology data from each of the plurality of subsystems;
the system extracting features from each of the graphs using the first model;
A method comprising: said system controlling said execution system based on said feature quantity.

Images