JP2018147087A - Control scheme analyzing device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To estimate relation between a control scheme and output of a simulator so as to search for an optimum scheme.SOLUTION: A simulator execution part 31 executes, based upon an input parameter consisting of control contents for each user, simulation to obtain output data. A feature vector conversion part 32 converts, in every time zone, the input parameter into a feature vector representing the number of users of each control content. A relation estimation part 33 estimates, based upon a plurality of data points as a combination of a result of conversion into the feature vector and the output data, relation between the feature vector and the output data.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a control strategy analysis apparatus, method, and program, and more particularly, to a control strategy analysis apparatus, method, and program for searching for an optimal control strategy.

  In recent years, there are many sites that need to be controlled, such as crowds on the way back to the station after an event such as a fireworks display or so-called congestion such as long waiting times for attractions in theme parks. On the way home to the station after the event, there is a risk of causing traffic jams and hustle and bustle accidents. In large-scale theme parks, it has been pointed out that long waiting times for attractions not only reduce satisfaction, but also reduce profits.

  In response to the above problems, countries and companies are taking various control measures. For example, there are control measures to make some roads pedestrian heaven only for a certain period of time after the event ends, and control measures such as guiding waiting times to vacant attractions by presenting waiting time information to the user through the application at the theme park. It has been implemented. In addition, as a control measure for leveling congestion, a method of recommending an attraction with the shortest waiting time to the user has been proposed. Actually, the effect of leveling the congestion has been confirmed using a simulator simulating a theme park (hereinafter, theme park simulator) (see Non-Patent Document 1).

However, in the above method, since the control strategy is considered manually, an optimal control strategy cannot be selected from a large number of control strategies. For example, in a theme park, there are (30 ⁴ ) ⁵⁰⁰⁰⁰ combinations of attraction tour routes when 4 out of 30 attractions with 50000 users ride.

  On the other hand, a technique called Bayesian optimization has been devised as a method for searching for parameters that improve learning accuracy from a large number of machine learning parameters (see Non-Patent Document 2). Bayesian optimization efficiently searches for parameters that improve the learning rate without performing learning on all parameters under the assumption that if the parameters are similar, the learning accuracy is also similar. Technology.

  By applying this Bayesian optimization to a simulator, it is considered that an optimal control strategy can be found from a large number of control strategies.

Shimizu Jin, Matsubayashi Tatsushi, Naya Ta. Simulation evaluation of waiting time reduction method in crowded saturated amusement park. SIG-DOCMAS Study Group. J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems (NIPS), 2012. Nello Cristianini, John Shawa-Taylor, Takeshi Ohkita: Pattern analysis by kernel method (2010)

  In the technology for adapting Bayesian optimization to a simulator, it is considered that the simulator is regarded as a function in which an input value serves as a control measure and an output value serves as an evaluation index, and Bayesian optimization is applied. The control measures / evaluation indicators mentioned here may be different for each problem. For example, in the alleviation of congestion on the way back from the event venue, it is possible to recommend a route for the user to return from the event venue to the station as a control measure, and population density on each road as an evaluation index. Further, in reducing congestion at a theme park, it is conceivable to recommend an attraction tour route for each user as a control measure, and an attraction average waiting time as an evaluation index. An image of the theme park simulator is shown in FIG.

  Here, since Bayesian optimization is a function optimization method, it is necessary to convert input / output values into vector values. In particular, it is necessary to convert the input value into a vector value suitable for the control problem. In order to convert to a vector value, the following problems exist.

[Problem 1] When converted into a high-dimensional vector, the prediction accuracy of the function deteriorates.

[Problem 2] If necessary information is lacking, it is difficult to convert the vector back into a control strategy.

  Each problem will be described.

  Problem 1 is a problem that if a high-dimensional vector is used as an input, the number of data necessary to maintain the accuracy of function approximation increases exponentially due to a dimensional curse. When the number of data increases, a large-scale simulator takes a long time to execute once, so it is impossible to obtain a large number of simulation samples, and as a result, the accuracy of function approximation deteriorates. For example, when the control policy of the user's cyclic route is expressed as shown in FIG. 8 in which 1 is assigned to the cyclic route selected by the user and 0 is assigned to other cyclic routes, the dimension number of the feature vector is cyclic. Since it is expressed by the product of the combination of routes and the number of users, it becomes very high-dimensional and the function prediction accuracy deteriorates.

  Problem 2 needs to be converted into a low-dimensional vector in order to solve Problem 1, but if the information necessary for the control strategy is dropped, the inverse conversion to the control strategy becomes difficult. It is a problem. For example, as shown in FIG. 9, when the cyclic route is converted into information indicating the number of users who have selected the same route, the dimension can be reduced. This is equivalent to the operation of crushing the matrix represented in FIG. 8 in the user direction. However, with such a conversion method, difficulty occurs when the feature vector is inversely converted into a control strategy. For example, as shown in FIG. 10, it is assumed that the feature vector in which 80 people select a cyclic route such as 1 → 2 → 3 is optimal. At this time, as a control measure, candidates can be considered as many as combinations of which 80 users are assigned to which visitors. Since the result of the attraction average waiting time varies greatly depending on how the 80 persons are allocated, it is difficult to reversely convert the feature vector into a control strategy.

  The present invention was made to solve the above problems, and even when a large amount of control measures exist, a control measure analysis device that can efficiently search for an optimal control measure, It is an object to provide a method and a program.

  In order to achieve the above object, a control strategy analyzing apparatus according to a first aspect of the present invention includes a simulator execution unit that executes a simulation based on input parameters including control contents for each user and obtains output data, and the input parameters Is converted into a feature vector that represents the number of users of each control content for each time period, and a plurality of data points that are a set of the result converted to the feature vector and the output data, A relationship estimation unit for estimating a relationship between the feature vector and the output data.

  Further, in the control strategy analyzing apparatus according to the first aspect of the present invention, a set of the result converted to the feature vector by the feature vector conversion unit and the output data determined according to the relationship estimated by the relationship estimation unit An acquisition function that takes the feature vector as an argument and represents the possibility of obtaining better output data than optimal output data of all data points, and the possibility increases as the variance value obtained from the data points increases. A next input parameter determining unit that determines a feature vector that maximizes the possibility based on an acquisition function that increases the likelihood, and determines an input parameter corresponding to the determined feature vector as a next input parameter; Until the predetermined repetition condition is satisfied, the execution by the simulator execution unit, the estimation by the relationship estimation unit, and the next input parameter Was repeated a determination by chromatography data determining section, a repetition judgment unit for determining the input parameters for the best output data may further include a.

  Further, in the control strategy analyzing apparatus according to the first aspect of the present invention, the relationship estimation unit is based on all data points that are a set of the result data converted by the feature vector conversion unit and the output data. A relational expression between the feature vector and the average value of the output data and a relational expression between the feature vector and the variance of the output data are estimated, and the next input parameter determination unit is configured to determine the estimated feature vector and the output Based on the relational expression with the average value of data and the relational expression between the feature vector and the variance of the output data, the feature vector having the maximum possibility is determined. May be.

  In the control strategy analyzing apparatus according to the first aspect of the present invention, the input parameter is a circulation route of an attraction for each user in the theme park, and the simulator for performing the simulation is a waiting time for each attraction user in the theme park The output data may be an average value or a maximum value of waiting times of users of the plurality of tour routes.

  According to a second aspect of the present invention, there is provided a control strategy analysis method comprising: an input unit; a simulator executing unit executing simulation based on input parameters comprising control content for each user; obtaining output data; and a feature vector converting unit A step of converting the input parameter into a feature vector representing the number of users of each control content for each time zone, and a plurality of sets of results obtained by the relationship estimation unit converting into the feature vector and the output data And executing a step of estimating a relationship between the feature vector and the output data based on the data points.

  A program according to the third invention is a program for causing a computer to function as each part of the control strategy analyzing apparatus according to the first invention.

  According to the control strategy analysis apparatus, method, and program of the present invention, simulation is executed based on input parameters including control contents for each user, output data is obtained, and input parameters are set for each time zone. Control is performed by converting to feature vectors representing the number of users in the control content, and estimating the relationship between the feature vectors and output data based on multiple data points that are pairs of the results of the conversion into feature vectors and output data. Even when there are a large number of measures, it is possible to efficiently search for an optimal control measure.

It is a figure which shows the example which converts the cyclic route which is a control policy into a feature vector in the same time slot | zone in congestion reduction of a theme park. It is a block diagram which shows the structure of the control strategy analysis apparatus which concerns on embodiment of this invention. It is a flowchart which shows the control strategy analysis processing routine in the control strategy analysis apparatus which concerns on embodiment of this invention. 5 is a schematic diagram of processing of a Bayesian optimization execution unit 30. FIG. It is a figure which shows an example of the setting of the simulator in an experiment example. It is a figure which shows an example of the experimental result in an experimental example. It is a figure which shows the image of the conventional theme park simulator. In the conventional theme park simulator, it is a figure which shows the example of the feature vector used as a high dimension according to a user's traveling route. It is a figure which shows the example at the time of converting into the information of the number of users who selected the same route in the conventional theme park simulator. In the conventional theme park simulator, it is a figure which shows the example which becomes difficult to carry out reverse conversion to a control measure, when converting into the information of the number of users who selected the same route.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline according to Embodiment of the Present Invention>

  First, an outline of the embodiment of the present invention will be described.

  In the present embodiment, a control policy analysis device that searches for an optimal control policy by converting the control policy into a vector and performing Bayesian optimization is proposed.

  As described in the above requirements, it is necessary to convert the control strategy into a vector so as not to drop the necessary information. On the way back from the event venue and the theme park, a large number of users can act almost simultaneously. At this time, even if the route selected by the user as a return route or the order of attractions circulation is slightly changed, the influence of evaluation indexes such as the population density of the road and the average waiting time for attractions is very small. Therefore, there is no need to strictly distinguish individual users. The control strategy is determined by the user selecting “when” and “which” return route or patrol route. Therefore, it is necessary to consider time information for action start.

  Considering the above two things, the control policy representing the cyclic route of each user is converted into a vector.

  Specifically, the information is converted into information indicating the number of users who have selected the same return route or the same route among the users who have behaved in the same time zone. FIG. 1 shows an example of converting the traveling route of each user, which is a control measure, into a feature vector in reducing congestion at a theme park. By this conversion method, it is possible to convert into a vector while solving the above-mentioned problem in the control strategy.

  Hereinafter, a case of searching for a control strategy that minimizes the evaluation index using a simulator will be described. As mentioned earlier, control measures and evaluation indices may differ from problem to problem.

<Configuration of Control Measure Analysis Device According to Embodiment of Present Invention>

  Next, the configuration of the control strategy analysis apparatus according to the embodiment of the present invention will be described. As shown in FIG. 2, the control strategy analysis apparatus 1 includes a simulator setting processing unit 10, a data storage unit 20, a Bayes optimization execution unit 30, and an optimal parameter processing unit 40. The simulator setting processing unit 10 and the optimum parameter processing unit 40 are each connected to an external device 2 such as an input device.

  The simulator setting processing unit 10 receives a plurality of control measures including a cyclic route for each user as input parameters. Further, the simulator setting processing unit 10 receives at least one of the maximum number of simulations and an output value threshold value for determining whether or not to end the processing of the Bayesian optimization executing unit 30, and receives data Each field is set in the storage unit 20.

  The data storage unit 20 includes a simulator setting table 21 and an optimum parameter table 22.

  The simulator setting table 21 has a maximum simulation execution frequency field and a threshold field. In the maximum simulation execution number field, the maximum number of times input by the simulator setting processing unit 10 is set, and in the threshold value field, the threshold value of the output value is set.

  In the optimum parameter table 22, optimum parameters input by the Bayesian optimization execution unit 30 are set.

  The Bayesian optimization execution unit 30 includes a simulator execution unit 31, a feature vector conversion unit 32, a relationship estimation unit 33, a next input parameter determination unit 34, a Bayes optimization result storage unit 35, and an iterative determination unit 36. .

  The simulator execution unit 31 executes a simulation based on input parameters representing a control strategy including a cyclic route for each user, and obtains output data.

  The feature vector conversion unit 32 converts the control strategy X into a feature vector x representing the number of users of each cyclic route for each time zone.

  The relationship estimation unit 33 converts the control strategy X into a feature vector x representing the number of users of each cyclic route and a plurality of data points, which are sets of output data, for each time zone, based on a plurality of data points. And the output data are estimated.

  The next input parameter determination unit 34 determines the control strategy X that is the next input parameter based on the relationship between the estimated feature vector x and the output data. Specifically, the optimum output of all the data points that are a set of the result converted to the feature vector x by the feature vector converting unit 32 and the output data, which is determined according to the relationship estimated by the relationship estimating unit 33 An acquisition function that takes a feature vector x as an argument and represents the possibility of obtaining better output data than data, and the possibility is greatest based on an acquisition function that has a higher probability as the variance value obtained from the data point increases. And the control strategy X corresponding to the determined feature vector x is determined as the next input parameter.

  The Bayesian optimization result storage unit 35 stores the simulation output data together with the input parameters.

  The iterative determination unit 36 repeats the execution by the simulator execution unit 31, the estimation by the relationship estimation unit 33, and the determination by the next input parameter determination unit 34 until the predetermined repetition condition is satisfied, thereby obtaining optimum output data. Find the input parameters for.

  The optimum parameter processing unit 40 sets, in the optimum parameter table 22 of the data storage unit 20, an input parameter that gives the minimum value of the output data among the input parameters input to the Bayes optimization result storage unit 35, and stores the data. The optimum parameters set in the optimum parameter table 22 of the unit 20 are output to the external device 2.

  Note that the control strategy analysis apparatus 1 according to the present embodiment includes, for example, a computer device including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) that stores various programs. . Moreover, the computer which comprises the control strategy analysis apparatus 1 may be provided with memory | storage parts, such as a hard disk drive and a non-volatile memory. In the present embodiment, the CPU reads and executes a program stored in a storage unit such as a ROM or a hard disk, whereby the hardware resources and the program cooperate to realize the above-described function.

<Operation of Control Measure Analysis Device According to Embodiment of Present Invention>

  Next, the operation of the control strategy analysis apparatus 1 according to the embodiment of the present invention will be described. Control strategy analysis processing executed by the control strategy analysis apparatus 1 according to the present embodiment will be described with reference to FIG. The timing when the control strategy analysis device 1 executes the control strategy analysis processing is, for example, the timing when the user instructs the execution of the control strategy analysis processing from the external device 2.

  In step S <b> 110, the simulator setting processing unit 10 sets the maximum simulation execution count M and the termination condition threshold value ε input by the external device 2 in the simulator setting table 21 of the data storage unit 20.

  Next, in the following steps S210 to S270, the Bayesian optimization executing unit 30 determines the optimal parameters to be input to the optimal parameter table 22 of the data storage unit 20 by the following method. FIG. 4 shows a schematic diagram of the processing of the Bayesian optimization execution unit 30.

In step S210, the simulator execution unit 31 executes a simulation with the control measure X randomly selected from the set of control measures or the control measure X _next determined in step S280 described later, and obtains an evaluation index value y of the control result. .

  In step S220, the feature vector conversion unit 32 converts the control strategy X into a feature vector x representing the number of users of each cyclic route for each time period. Here, as an example, a method for converting a control policy called a user's patrol route in a theme park into a feature vector will be described.

First arrival time AT _n users n, based on the information that the patrol route ROUTE _n, the following (1) into a matrix as shown in equation.

... (1)

Here, # represents the number of elements, and Δ _j represents the simulation time.

Then, x = (x ₁₁ , x ₁₂ ,..., X _1n , x ₂₁ ...) _Obtained by _{converting the} matrix x _tmp into a vector is set as a feature vector corresponding to the control strategy X that is a cyclic route.

  Thereafter, the entire set (x, y) of the characteristic vector x of the control strategy and the evaluation index value y obtained by performing the simulation is obtained.

... (2)

Write. Here, a set of all feature vectors corresponding to the control strategy is written as C _para . The feature vector conversion unit 32 stores the D in the Bayes optimization result storage unit 35.

  In step S230, the simulator execution unit 31 updates the number of simulation executions.

  In step S240, the simulator execution unit 31 determines whether or not the number of simulation executions exceeds the maximum number of repetitions M set in step S110. If it is determined in step S240 that the number of simulation executions has exceeded the maximum number of repetitions M (S240, Y), the process proceeds to step S310. If it is determined in step S240 that the number of simulation executions does not exceed the maximum number of repetitions M (S240, N), the process proceeds to step S250.

  In step S250, the simulator execution unit 31 determines whether or not the simulation output value is smaller than the threshold value ε set in step S110. If it is determined in step S250 that the simulation output value is smaller than the threshold ε (S250, Y), the process proceeds to step S310. If it is determined in step S250 that the simulation output value is greater than or equal to the threshold ε (S250, N), the process proceeds to step S260.

  In step S260, the relationship estimation unit 33, based on all data points of the Bayes optimization result storage unit 35, the relational expression between the feature vector and the average value of the output data and the feature vector and the output regarding the input / output of the simulator. Estimate the relation with the variance of data by Gaussian process. The estimation result for a certain feature vector x follows a Gaussian distribution having an average value μ (x) expressed by the following equation (3) and a variance value σ (x) expressed by the following equation (4).

... (3)

... (4)

(3), (4)

Can be written as equations (5) and (6) using a function that defines the similarity k (x ₁ , x ₂ ) between the input parameters x ₁ and x ₂ called a kernel function.

... (5)

... (6)

However, k _ij = k (x _i , x _j ), the superscript symbol T represents the transpose of the matrix, and the superscript symbol −1 represents the inverse matrix. This kernel function may be changed depending on the problem. Typical kernels include a linear kernel and a Gaussian kernel (see Non-Patent Document 3).

In step S270, the next input parameter determination unit 34 calculates a feature vector x _{next in} order to determine the next control strategy according to the following equation (7).

... (7)

Here, α (x) is called an acquisition function and is an index for quantitatively evaluating the possibility that the feature vector x gives the minimum value. In the above equation (7), x _next maximizing the acquisition function value is obtained. As typical acquisition functions, probability improvement (PI) and expected value improvement (EI) are used (see Non-Patent Document 2). In the present embodiment, the acquisition function α (x) is defined as the following equation (8). Here, σ (x) is the variance given by the above equation (4), Φ (x) represents the cumulative distribution function in the standard normal distribution, and N (x) is the probability density in the standard normal distribution. Represents a function.

... (8)

Γ (x) is a function given by the following equation (9) using the average μ and the variance σ given by the above equations (3) and (4). Note that f _best in the following equation (9) is the minimum value of the output result obtained by the simulator.

... (9)

As shown in FIG. 4, x _next of the _next search point to be input _next is a point having a large variance value, in other words, a point in an uncertain region where there are few observation data points and no search is performed. The acquisition function makes it easy to select “an uncertain region with a large variance value” and “near the parameter that gives the current minimum value” as the next search points.

In step S280, the next input parameter determination unit 34 does not always have the feature vector corresponding to the actual control strategy as x _next obtained in step S270. among the feature vector ^{x int} obtained by converting the control measures _X, to determine the closest _x ^{next int} the _{x next.}

... (10)

Then, the control strategy X corresponding to x _next ^int is set as the control strategy X _next .

As described above, the next input parameter determination unit 34 determines the control strategy X _next based on x _next derived based on the acquisition function and has the highest possibility.

Although it is conceivable that there are a plurality of control measures corresponding to x _next ^int determined by the equation (10), any one control measure is determined at random. Then, returning to step S210, X _next is input to the simulator execution unit 31, and simulation is performed using X _next as an input control measure.

  In step S310, the Bayesian optimization execution unit 30 inputs, to the data storage unit 20, a control strategy that gives the minimum value among the data included in the Bayesian optimization result storage unit 35 when the Bayesian optimization processing is completed. The data storage unit 20 sets the parameters in the optimum parameter table 22.

  In step S410, the optimum parameter processing unit 40 outputs the optimum parameters input from the data storage unit 20 to the external device 2. The output process may be executed when an output request is input from the external device 2, for example.

[Example of experimental results]

  The theme park simulator was used to search for a patrol route that shortens the waiting time for attractions by the method of the present embodiment.

  FIG. 5 shows the simulator settings, and FIG. 6 shows the experimental results. As can be seen from the results of FIG. 6, the user succeeded in finding a cyclic route with a shorter average waiting time for the attraction than the result of a random search in which the user randomly selects a cyclic route and performs a cyclic search. ing.

  In this experimental example, we considered a patrol route that shortens the average waiting time for attractions in the theme park. It is possible to search for an optimal control strategy for alleviating congestion on general roads on the way back from the event venue to the station. Further, it is possible to search for control measures not only for congestion but also for various purposes of control.

  As described above, according to the control strategy analyzing apparatus according to the embodiment of the present invention, based on the input parameters composed of the control content for each user, the simulation is performed, the output data is obtained, and the input parameters are For each time zone, convert to a feature vector representing the number of users of each control content, and based on a plurality of data points that are a combination of the result of the conversion to the feature vector and the output data, the relationship between the feature vector and the output data is By estimating, even when there are a large number of control strategies, an optimal control strategy can be searched efficiently.

  In addition, by recommending to the user the patrol route discovered by the method of the embodiment of the present invention, it becomes possible to discover an optimal control measure for alleviating the waiting time of attractions and congestion of general roads.

  In addition, since it is not necessary to think about the control measure manually, the cost for formulating the control measure can be reduced.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, it is possible to construct the operation of each component of the control strategy analysis apparatus as a program, install it on a computer used as the control strategy analysis apparatus, execute it, or distribute it via a network.

DESCRIPTION OF SYMBOLS 1 Control measure analysis apparatus 2 External device 10 Simulator setting process part 20 Data storage part 21 Simulator setting table 22 Optimal parameter table 30 Bayes optimization execution part 31 Simulator execution part 32 Feature vector conversion part 33 Relation estimation part 34 Next input parameter determination part 35 Bayes optimization result storage unit 36 Iterative determination unit 40 Optimal parameter processing unit

Claims (6)

Based on the input parameters consisting of the control content for each user, a simulation is executed, and a simulator execution unit for obtaining output data;
A feature vector conversion unit that converts the input parameter into a feature vector representing the number of users of each control content for each time period;
A relationship estimation unit that estimates a relationship between the feature vector and the output data based on a plurality of data points that are a set of the result converted into the feature vector and the output data;
Control measure analyzer including Better than optimal output data among all data points that are a set of the result data converted by the feature vector conversion unit and the output data, which are determined according to the relationship estimated by the relationship estimation unit An acquisition function that takes the feature vector as an argument and represents the possibility that output data can be obtained, and the possibility is maximum based on the acquisition function that increases as the variance value obtained from the data point increases. A next input parameter determining unit that determines a feature vector as follows, and determines an input parameter corresponding to the determined feature vector as a next input parameter;
Until the predetermined repetition condition is satisfied, the execution by the simulator execution unit, the estimation by the relationship estimation unit, and the determination by the next input parameter determination unit are repeated to obtain an input parameter for obtaining optimum output data An iterative determination unit;
The control policy analysis apparatus according to claim 1, further comprising: The relation estimation unit is a relational expression between the feature vector and the average value of the output data based on all data points that are a set of the result data converted by the feature vector conversion unit and the output data. And estimating a relational expression between the feature vector and the variance of the output data,
The next input parameter determination unit is configured to calculate the acquisition function expressed using a relational expression between the estimated feature vector and an average value of the output data and a relational expression between the feature vector and the variance of the output data. The control policy analysis apparatus according to claim 2, wherein a feature vector that maximizes the possibility is determined based on the control vector. The input parameter is a circulation route of attractions for each user in the theme park, the simulator that performs the simulation is a theme park simulator that reproduces the waiting time of the user of each attraction in the theme park, and the output data is the The control policy analysis apparatus according to any one of claims 1 to 3, wherein the waiting time of users of a plurality of traveling routes is an average value or a maximum value. A simulator execution unit executes a simulation based on input parameters including control contents for each user to obtain output data;
A feature vector conversion unit converting the input parameter into a feature vector representing the number of users of each control content for each time zone;
A step of estimating a relationship between the feature vector and the output data based on a plurality of data points that are a set of the output data and the result of the relationship estimation unit converting the feature vector;
Control strategy analysis method including   The program for functioning a computer as each part of the control strategy analyzer of any one of Claims 1-4.