JP2021047625A - Evacuation guidance device, and evacuation guidance model learning device - Google Patents

Abstract

To lead evacuees to evacuate in a manner to minimize risks by taking into account of an evacuation simulation result when a disaster occurs.SOLUTION: An obtainment section 36 of an evacuation guidance device 30 obtains observation information expressing positions or motions of people when a disaster occurs. Then, an evacuation information generation section 38 inputs, on the basis of an evacuation simulation result when the disaster occurs, the observation information obtained by the obtainment section 36, into a trained model to which reinforcement learning has been previously performed by using a count of deaths and a count of injured persons in the disaster of the evacuation simulation as well as a remuneration set in accordance with a time until the evacuation completes, and generates evacuation information which is information relating to an evacuation route when the disaster occurs. And further, a control section 40 controls a display device 42 in accordance with the evacuation information generated by the evacuation information generation section 38.SELECTED DRAWING: Figure 1

Description

The present invention relates to an evacuation guidance device and an evacuation guidance model learning device.

Conventionally, an evacuation simulation system is known (for example, Patent Document 1). This evacuation simulation system uses multi-agent simulation technology to simulate disaster evacuation methods in high-rise buildings. This evacuation simulation system tracks the evacuation situation at any time during evacuation by taking the approach of modeling the individual evacuees as one action unit and sequentially reproducing the state of each individual during the evacuation action. The purpose is to easily identify bottlenecks that hinder safe evacuation and consider improvement measures.

Further, an evacuation route output device that outputs an evacuation route avoiding a damaged area is known (for example, Patent Document 2). This evacuation route output device generates a route that can safely reach the evacuation site in the event of a disaster.

Further, an evacuation simulation device capable of quickly and appropriately formulating an evacuation plan according to a disaster situation is known (for example, Patent Document 3). This evacuation simulation device calculates the flow of routes based on the density of evacuees, and derives a plurality of optimal evacuation route candidates with the shortest evacuation completion time. Then, the evacuation simulation device calculates the behavior of the evacuees by the multi-agent method, and selects the optimum evacuation route with the shortest evacuation completion time from a plurality of optimum evacuation route candidates.

Japanese Patent No. 5342421 Japanese Patent No. 5686479 Japanese Patent No. 5996689

When evacuation guidance is given to people in the building in the event of a disaster, it is necessary to properly present the evacuation route. In addition, promptness is required to present the evacuation route.

However, the technique of Patent Document 1 is for easily identifying a bottleneck that hinders safe evacuation and examining improvement measures, and is a technique used when evaluating a building to be planned.

Further, the technique of Patent Document 2 outputs an evacuation route avoiding the damaged part when a disaster occurs. However, when a disaster actually occurs, it is necessary to consider various situations other than the affected area. For example, it is necessary to consider the movement of people who evacuate.

Further, the technique described in Patent Document 3 performs a simulation according to an actual disaster situation, but it takes time to execute the simulation and is not appropriate from the viewpoint of speed.

In view of the above facts, an object of the present invention is to evacuate an evacuee so as to minimize the risk in consideration of the evacuation simulation result when a disaster occurs.

In order to achieve the above object, the evacuation guidance device of the present invention obtains an acquisition unit that acquires observation information indicating the position or movement of a person when a disaster occurs, and the observation information acquired by the acquisition unit. Based on the results of the evacuation simulation when a disaster occurs, learning that has been strengthened and learned in advance using rewards set according to the number of deaths, injuries, and the time until evacuation is completed in the disaster of the evacuation simulation. Evacuation information generation unit that inputs to the completed model and generates evacuation information that is information about the evacuation route when the disaster occurs, and evacuation information output according to the evacuation information generated by the evacuation information generation unit. It is an evacuation guidance device including a control unit that controls the device. According to the evacuation guidance device of the present invention, the evacuees can be evacuated so as to minimize the risk in consideration of the evacuation simulation result when a disaster occurs.

The trained model of the present invention can be made to be the trained model that has been reinforcement-learned in advance according to the results of a plurality of types of the evacuation simulations. As a result, the evacuees can be evacuated so as to minimize the risk in consideration of the results of a plurality of types of evacuation simulations.

The evacuation guidance model learning device of the present invention executes an evacuation simulation when a disaster occurs, and based on the evacuation simulation result, the number of deaths, the number of injured, and the evacuation in the disaster of the evacuation simulation until the evacuation is completed. Using the rewards set according to the time, strengthen learning the model for outputting information on the evacuation route when the disaster occurs from the observation information showing the position or movement of the person when the disaster occurs. , An evacuation guidance model learning device including a learning unit that obtains a learned model that outputs information about the evacuation route from the observation information. According to the evacuation guidance model learning device of the present invention, it is possible to acquire a learned model for evacuating an evacuee so as to minimize the risk in the event of a disaster.

The learning unit of the present invention can execute a plurality of types of the evacuation simulations and obtain the learned model based on the evacuation simulation results. This makes it possible to take into account multiple types of evacuation simulations and obtain a trained model for evacuating evacuees to minimize risk in the event of a disaster.

According to the present invention, it is possible to evacuate an evacuee so as to minimize the risk in consideration of the evacuation simulation result when a disaster occurs.

It is a block diagram which shows the schematic structure of the evacuation guidance model learning apparatus which concerns on this embodiment. It is explanatory drawing for demonstrating the relationship between the simulation result of the evacuation simulation and the reward. It is a block diagram which shows the schematic structure of the evacuation guidance device which concerns on this embodiment. It is a figure which shows an example of the installation image in a building of a display device. It is a figure which shows an example of the learning processing routine of 1st Embodiment. It is a figure which shows an example of the evacuation guidance processing routine of this embodiment. It is explanatory drawing for demonstrating the evacuation in a building and the evacuation of a block which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating the relationship between the variable of 2nd Embodiment. It is a figure which shows an example of the learning processing routine of 2nd Embodiment. It is a figure which shows an example of the learning processing routine of 2nd Embodiment.

<Outline of this embodiment>

If a disaster occurs when there are people in the building, when evacuating from the building, the victims themselves will compare the evacuation routes and select the most suitable route. Here, since the quality of the disaster victim's judgment is influenced by the knowledge about the disaster and the available information, a dangerous evacuation route may be selected depending on the situation.

Against this background, a method for presenting evacuation instructions using evacuation simulation (see, for example, Japanese Patent No. 5996689) has been proposed, but the conventional technique optimizes the result (output) of evacuation simulation. As described above, it is a method of identifying an input parameter called an evacuation order. Therefore, optimizing evacuation instructions by considering the simulation results of a plurality of methods and a plurality of models in parallel becomes difficult to identify as the number of methods and models increases.

On the other hand, in the trained model obtained by machine learning, the optimum input can be selected after learning the relationship between the input and output of the simulation. Therefore, even if the evacuation simulations are of different types, it is easy to use them together as long as the evacuation simulations have the same input and output.

Therefore, in the present embodiment, we propose a method that enables a plurality of evacuation simulations to be used together by utilizing a learned model obtained by machine learning in determining an evacuation order. According to this embodiment, it is possible to consider the simulation results of a plurality of evacuation simulations and evacuate the evacuees so as to minimize the risk.

Hereinafter, embodiments of the present invention will be described in detail.

<First Embodiment>

<System configuration of evacuation guidance model learning device>

FIG. 1 is a block diagram showing an example of the configuration of the evacuation guidance model learning device 10 according to the first embodiment of the present invention. Functionally, the evacuation guidance model learning device 10 can be represented by a configuration including a reception unit 12 and a computer 20 as shown in FIG.

The reception unit 12 receives the information input from the user. The reception unit 12 is realized by, for example, a keyboard, a mouse, or the like. The reception unit 12 receives specification information representing the specifications of the virtual learning building for which the evacuation simulation is executed. The learning building is a virtual building on a computer used for simulation in the learning unit 24, which will be described later. The specification information includes, for example, information on the arrangement of rooms in the learning building, information on the structural type of the building for learning, information on the materials of the building for learning, information on the number of floors of the building for learning, and information on the number of floors of the building for learning. Contains information about the facilities of the building for learning.

Further, the reception unit 12 receives condition information which is information on various conditions when executing the evacuation simulation. The condition information includes disaster occurrence condition information, which is information on virtual disaster occurrence conditions in the evacuation simulation, and evacuation, which is information on evacuation instruction conditions for virtual people in the building when a virtual disaster occurs. It contains instruction condition information. In addition, the condition information includes information on the placement status of virtual evacuees.

The setting information storage unit 22 stores the specification information received by the reception unit 12 and the condition information. The evacuation simulation is executed in the learning unit 24, which will be described later, according to the specification information and the condition information stored in the setting information storage unit 22.

The learning unit 24 executes an evacuation simulation when a disaster occurs in the building for learning based on the specification information and the condition information stored in the setting information storage unit 22. The position and movement of the virtual person at each time when the evacuation simulation is being executed are sequentially recorded in a predetermined storage area (not shown). The evacuation simulation used in this embodiment is the same as the existing evacuation simulation, and uses a conventional technique (for example, the technique described in Japanese Patent No. 5342421). In addition, the number of evacuation simulations can be set in the same manner as in conventional reinforcement learning.

Then, based on the simulation result, the learning unit 24 outputs evacuation information which is information on an evacuation route when a disaster occurs from observation information representing the position or movement of a person in the building when a disaster occurs. Reinforcement learning of the model for. Through reinforcement learning of the model by the learning unit 24, what kind of disaster situation and what kind of evacuation instruction should be issued at the position of the evacuees are learned.

Hereinafter, reinforcement learning will be described. Reinforcement learning is a method of learning optimal behavior through trial and error in the environment. In reinforcement learning, rewards are a substitute for teacher data. The cumulative reward R _t is defined as shown in the following equation (1), where the discount rate of the reward is γ and the reward in each phase is _{rt + k + 1.} In addition, t represents a time.

（１）

(1)

The value of selecting the action a in the state s under the policy π is expressed by the action value function Q ^π (s, a) shown in the following equation (2). In addition, E _π {・} represents an expected value.

（２）

(2)

^{Using the action value function Q π} (s, a) shown in the above equation (2), the action a having the highest value is selected. The optimal action value function Q ^* is expressed by the following equation (3).

（３）

(3)

Examples of the method for learning the behavioral value function Q ^* (s, a) include Q-Learning (for example, publicly known literature (Watkins, CJCH, "Learning from Delayed Rewards", 1989). Q-Learning includes the following. As shown in the equation (4), learning is performed while sequentially updating the Q value. α is a preset constant. In the present embodiment, the Q-Learning shown in the following equation (4) The action value function is strengthened and learned by.

（４）

(4)

In the present embodiment, the disaster situation, the position of the victim, etc. are set as the state s, the evacuation instruction according to the state s and the policy π is set as the action a, and the victim who sees the display device on which the evacuation order a is displayed is set. Shall evacuate. In the trained model trained by Q-Learning, the evacuation instruction a according to the policy π can be selected so that the action value function Q is optimized.

Based on the risk evaluation results based on the number of casualties and the time of the evacuation route, the learning unit 24 of the present embodiment responds to the number of casualties D, the number of injured I, and the time T until the evacuation is completed in the disaster of the evacuation simulation. using a set reward r _t, to reinforcement learning model for outputting information about the evacuation route from the observation information Te. Specifically, in the present embodiment, to set a reward r _t represented by the following formula (5).

（５）

(5)

In the above formula (5), D represents the number of dead and I represents the number of injured. In addition, T is the time until the evacuation is completed. C _d is a coefficient representing the loss for per dead, C _i is a coefficient representing the loss for per injured, the C _t is the coefficient that relates the loss evacuation time. C _{d, C i,} and _{C t} is set in advance.

FIG. 2 shows an explanatory diagram for explaining the relationship between the simulation result of the evacuation simulation and the reward. As shown in FIG. 2, when a plurality of evacuees U exist in the building, it is assumed that an evacuation simulation is executed when a fire F, which is an example of a disaster, occurs. In this case, when the evacuation order A is issued, it is shown that the evacuation time is X1 minutes, the dead is Y1, the injured is Z1, and the reward is high. Further, when the evacuation order B is issued, it is shown that the evacuation time is X2 minutes, the dead are Y2, the injured are Z2, and the reward is medium. Further, when the evacuation order C is issued, it is shown that the evacuation time is X3 minutes, the dead are Y3, the injured are Z3, and the reward is low. In this way, since the simulation result and the reward are linked, in the present embodiment, the model for outputting the evacuation information from the observation information is strengthened and learned based on the reward according to the simulation result.

Specifically, the learning unit 24, as reward r _t represented by the above formula (5) is increased, to enhance learning model for outputting evacuation information from observation information, which is an example of the learned model Obtain the action value function Q ^* (s, a). When the observation information representing the state s is ^{input to the action value function Q *} (s, a), the evacuation instruction representing the action a corresponding to the observation information is output as an example of the evacuation information.

Any function may be used as a model of the action value function for outputting evacuation information from the observation information. For example, a neural network model can be used as a model of the action value function. Alternatively, a table (also referred to as a Q table) in which the observation information representing the state s and the evacuation instruction representing the action a are associated with each other may be used.

^{The trained model storage unit 26 stores an action value function Q *} (s, a), which is an example of the trained model learned by the learning unit 24. The action value function Q ^* (s, a) is used in the evacuation guidance device described later, and when the observation information is ^{input to the action value function Q *} (s, a) at each time, it is an example of evacuation information. A certain evacuation order is displayed on the display device.

Optimization of evacuation instructions in the event of a disaster by conventional simulation is a method of identifying an input parameter called evacuation instructions so that the result is optimal. Therefore, when optimizing evacuation instructions by considering the simulation results of a plurality of methods and a plurality of models in parallel, identification becomes difficult as the number of methods and models increases.

On the other hand, in the present embodiment, the simulation result of the evacuation simulation is converted into the variables of the state s and the action a, and then the optimum measure π is selected. Therefore, even if the simulation results of different methods and different models can be converted into a common state s and a common action a, it is possible to select the optimum policy π without changing the framework. This makes it possible to evaluate the optimum evacuation order by combining completely different simulation results in parallel, such as an evacuation simulation for individual buildings and an evacuation simulation from river flooding for an area. become.

<System configuration of evacuation guidance device>

FIG. 3 is a block diagram showing an example of the configuration of the evacuation guidance device 30 according to the embodiment of the present invention. Functionally, as shown in FIG. 3, the evacuation guidance device 30 can be represented by a configuration including an observation device 32, a computer 34, and a plurality of display devices 42. The display device 42 is an example of the evacuation information output device of the present invention.

The observation device 32 of the evacuation guidance device 30 and the plurality of display devices 42 are installed in a target building equivalent to the building for learning. For example, the target building is a building constructed based on the blueprint of a virtual learning building. Then, when a disaster occurs, the evacuation guidance device 30 controls the display of the plurality of display devices 42 installed in the building based on the information sequentially observed by the observation device 32 installed in the building. , Guide the evacuation of people in the building. Hereinafter, a specific description will be given.

The observation device 32 is installed in the building and sequentially acquires observation information indicating the position or movement of a person in the building when a disaster occurs. Further, as the observation device 32, for example, a terminal having a global positioning system (GPS) function such as a mobile terminal carried by a person can be used. In the case of evacuation guidance inside a building, a system that judges the movement of people from image data from a camera installed inside the building (for example, Vitracom Site View by the Structural Planning Institute and Crowd by the National Institute of Advanced Industrial Science and Technology). Walk etc.) can be used. In addition, the observation device 32 may also observe the disaster situation (for example, the degree of spread of the fire and the degree of collapse of the building due to the earthquake).

The computer 34 includes a CPU (Central Processing Unit), a ROM (Read Only Memory) that stores programs for realizing each processing routine, a RAM (Random Access Memory) that temporarily stores data, and a memory as a storage means. , Network interface, etc. are included. Functionally, as shown in FIG. 3, the computer 34 includes an acquisition unit 36, a learned model storage unit 37, an evacuation information generation unit 38, and a control unit 40.

The acquisition unit 36 acquires the observation information sequentially acquired by the observation device 32. For example, when the observation information is an image captured by a camera or the like, the acquisition unit 36 detects the position and movement of a person appearing in the image by a predetermined image process.

The trained model storage unit 37 stores the same trained model as the trained model stored in the trained model storage unit 26 of the evacuation guidance model learning device 10. The trained model of this embodiment is the behavioral value function Q ^* (s, a).

^{The evacuation information generation unit 38 reads out the action value function Q *} (s, a) as a learned model stored in the learned model storage unit 37. Then, the evacuation information generation unit 38 inputs the observation information s acquired by the acquisition unit 36 into the action value function Q ^* (s, a) to generate an evacuation instruction a when a disaster occurs. The evacuees shall take actions according to the evacuation order a.

The control unit 40 controls a plurality of display devices 42 installed in the building where the disaster has occurred according to the evacuation information generated by the evacuation information generation unit 38.

Each of the plurality of display devices 42 is installed at various points in the building as shown in FIG. Then, each of the plurality of display devices 42 individually changes the display at each location according to the control by the control unit 40. The content displayed on the display device 42 is, for example, an arrow indicating the evacuation direction according to the evacuation order or a sentence corresponding to the evacuation order (for example, "The right hand direction is impassable. Please evacuate from the left hand direction.") Is displayed. This makes it possible to appropriately guide people to evacuate in the event of a disaster.

<Operation of evacuation guidance model learning device>

Next, the operation of the evacuation guidance model learning device 10 will be described. The evacuation guidance model learning device 10 executes the learning processing routine shown in FIG.

<Learning processing routine>

When the specification information and the condition information are input to the evacuation guidance model learning device 10 and the reception unit 12 receives the specification information and the condition information, the setting information storage unit 22 stores the specification information and the condition information. Then, when the evacuation guidance model learning device 10 receives the instruction signal for the learning process, the evacuation guidance model learning device 10 executes the learning process routine shown in FIG.

In step S100, the learning unit 24 reads the specification information and the condition information stored in the setting information storage unit 22.

In step S102, the learning unit 24 sets the disaster occurrence condition in the evacuation simulation based on the disaster occurrence condition information in the condition information read in step S100. For example, the learning unit 24 sets the location and scale of the fire in the building as the disaster occurrence condition based on the disaster occurrence condition information.

In step S104, the learning unit 24 sets the evacuation instruction condition in the evacuation simulation based on the evacuation instruction condition information in the condition information read in step S100. For example, when a fire breaks out in a certain place, evacuation instruction conditions are set so that the victim is instructed to leave the place. In the evacuation simulation, various evacuation orders are issued according to the evacuation instruction conditions, and the action value function Q ^* (s, a) described later is learned based on the actions and results of the victims according to the evacuation instructions.

In step S106, the learning unit 24 learns based on the building specification information read in step S100, the disaster occurrence condition set in step S102, and the evacuation instruction condition set in step S104. Execute an evacuation simulation when a disaster occurs in a building for use.

In step S108, the learning unit 24 stores the simulation result of the evacuation simulation executed in step S106 in a storage unit (not shown).

In step S109, the learning unit 24, based on the results simulation stored in the step S108, as reward r _t becomes large, to enhance learning model for outputting information on evacuation route from the observed information, learning ^{We obtain the action value function Q *} (s, a), which is an example of the completed model.

In step S110, the learning unit 24 determines whether or not the evacuation simulation has been executed a predetermined number of times. When the evacuation simulation is executed a predetermined number of times, the process proceeds to step S112. On the other hand, if the evacuation simulation has not been executed a predetermined number of times, the process returns to step S104. As a result, the evacuation simulation in which only the evacuation instruction according to the evacuation instruction condition is changed is executed as many times as necessary.

In step S112, the learning unit 24 determines whether or not the evacuation simulation of all the disaster occurrence conditions has been executed. When the evacuation simulation of all the disaster occurrence conditions is executed, the process proceeds to step S114. On the other hand, if there is a disaster occurrence condition for which the evacuation simulation has not been executed, the process returns to step S102. As a result, the evacuation simulation in which only the disaster occurrence conditions of the disaster are changed is executed, and the evacuation simulation for the assumed disaster is executed.

In step S114, the learning unit 24 stores the learned action value function Q ^* (s, a) learned in step S109 in the learned model storage unit 26, and ends the learning processing routine.

<Action of evacuation guidance device>

Next, the operation of the evacuation guidance device 30 will be described. The evacuation guidance device 30 executes the evacuation guidance processing routine shown in FIG.

<Evacuation guidance processing routine>

^{When the learned behavior value function Q *} (s, a) learned by the evacuation guidance model learning device 10 is input to the evacuation guidance device 30, the learned behavior value function Q ^* (s, a) has been learned. It is stored in the model storage unit 37.

Then, when it is detected that a disaster has occurred in the building where the evacuation guidance device 30 is installed, the evacuation guidance device 30 executes the evacuation guidance processing routine shown in FIG. The evacuation guidance device 30 executes the evacuation guidance processing routine shown in FIG. 6 every time observation information is obtained by the observation device 32.

In step S200, the acquisition unit 36 acquires the observation information acquired by the observation device 32. The observation information is the position and movement of the victim in the damaged building.

^{In step S202, the evacuation information generation unit 38 reads out the action value function Q *} (s, a) as a learned model stored in the learned model storage unit 37.

In step S204, the evacuation information generation unit 38 inputs the observation information acquired in step S200 into the action value function Q ^* (s, a) read in step S202, and uses information on the evacuation route. Generate some evacuation information. Specifically, the observation information action value function Q ^* (s, a) is input to, action value function Q ^* (s, a) evacuation order, which is an example of evacuation information is outputted from.

In step S206, the control unit 40 controls a plurality of display devices 42 installed in the building where the disaster has occurred in response to the evacuation instruction generated by the evacuation information generation unit 38, and ends the evacuation guidance processing routine. To do.

Each of the plurality of display devices 42 changes the display according to the control by the control unit 40. The evacuees in the building evacuate according to the evacuation instructions displayed on each of the plurality of display devices 42.

As described in detail above, the evacuation guidance device of the present embodiment uses observation information indicating the position or movement of a person when a disaster occurs, and the number of dead, injured, and evacuation in the disaster of the evacuation simulation are completed. Evacuation information, which is information on evacuation routes in the event of a disaster, is generated by inputting into a pre-enhanced trained model using rewards set according to the time until the evacuation. Controls the display device installed in the building. As a result, the evacuees can be evacuated so as to minimize the risk in consideration of the evacuation simulation result when a disaster occurs.

Further, the evacuation guidance model learning device of the present embodiment executes an evacuation simulation when a disaster occurs, and based on the simulation result, the number of deaths, the number of injured, and the evacuation in the disaster of the evacuation simulation are completed. A model for outputting evacuation information, which is information on evacuation routes when a disaster occurs, from observation information showing the position or movement of a person when a disaster occurs, using the reward set according to the time of Get a trained model by training. This makes it possible to obtain a trained model for evacuating evacuees to minimize risk in the event of a disaster.

Further, in the present embodiment, after converting the simulation result of the evacuation simulation into the variables of the state s, the action a, and the policy π, the evacuation instruction is optimized by learning the relationship between these three variables in the model. Can be done. That is, in determining the optimum evacuation instruction, by using the trained model obtained by reinforcement learning, it is possible to consider different methods and simulation results by different models in parallel.

<Second Embodiment>

Next, the second embodiment will be described. The second embodiment is different from the first embodiment in that a plurality of types of evacuation simulations are executed, a trained model is obtained based on the simulation results, and evacuation information is acquired using the trained model. Since the configuration of each device according to the second embodiment has the same configuration as that of the first embodiment, the same reference numerals are given and the description thereof will be omitted.

In the second embodiment, different types of evacuation simulations (for example, an evacuation simulation targeting the inside of a building and an evacuation simulation targeting a block outside the building) are executed, and the simulation results are reflected in the trained model.

In the second embodiment, the first evacuation simulation, which is an evacuation simulation for the building alone (evacuation from the inside of the building to the outside), and the evacuation simulation at the district level outside the building (inside the district outside the building). Evacuation) is assumed as the second evacuation simulation. For example, in the second embodiment, as shown in FIG. 7, it is assumed that the evacuee U goes out from the building Ax and also evacuates in the block. In the present embodiment, the case where two different types of evacuation simulations are executed will be described as an example, but it is also possible to apply a plurality of types of evacuation simulations, which is more than two, to the present embodiment.

The learning unit 24 of the second embodiment is an example of a learned model in which a plurality of types of evacuation simulations are executed, and a model for outputting evacuation information from observation information is strengthened and learned based on the simulation results. Obtain the action value function Q ^* (s, a).

Specifically, in the second embodiment, the internal state of the building when a disaster occurs is set to s ₀ , the measure for evacuation from the building to the outside is set to π _1, and the evacuation instruction from the building to the outside is a. _Let it be 1. It should be noted that the evacuees shall take action in accordance with the evacuation order a _1.

In addition, the state when evacuation from the building is completed is s ₁ , the measure for evacuation in the block outside the building is π ₂ , the action for evacuation in the block outside the building is a _2, and the outside of the building. _{Let s 2} be the situation when the evacuation in the block is completed.

In this case, the relationship between the variables, evacuation order a ₁ by applying measures Pai ₁ relative state s ₀ is issued, take evacuation of evacuees in accordance with the evacuation order a _1, so that state at the time of evacuation of the building has been completed is s ₁ as. Then, by applying the measure π ₂ to this state s ₁ , an evacuation order a ₂ is issued, the evacuees _{take evacuation actions according to the evacuation order a 2} , and as a result, evacuation to an evacuation center or the like is performed. The state when completed is s ₂ .

In Q-Learning, for learning and reward r ₁ corresponding to the state s _1, it is necessary to calculate the reward r ₂ in accordance with a state s _2. The relationship between these variables is as shown in FIG.

In the present embodiment, reflecting the reward r ₂ at the time of the final state s ₂ methods evacuation order to maximize, to the reinforcement learning action-value function, which is an example of the learned model by Q ^* (s, a) The purpose is to make it. The reward r ₁ and the reward r ₂ are set as functions based on the number of dead D, the number of injured I, the evacuation time T, and the like, as shown in the above equation (5).

In the second evacuation simulation, the block-level evacuation simulation, the evacuation state s ₂ is calculated from the simulation results. Therefore, it is possible on the basis of the second evacuation simulation simulation results, so as to maximize the reward r _2, to obtain the action value function Q outputs of evacuation instruction ^* (s, a).

On the other hand, only a simulation result of the evacuation simulation in a building, which is the first evacuation simulation, only the state s ₁ evacuation not calculated. Therefore, although it is possible to calculate the reward r ₁ by the first evacuation simulations, it is impossible to calculate the reward r ₂ directly.

Thus, for example, performs a reinforcement learning using reward r ₁ based on the first evacuation simulation simulation results, when the the building was learned evacuation instruction outside the model, evacuation out of the building There action value, such as to optimize until the completion function Q ^* (s, a) it is obtained, this action-value function Q ^* (s, a) is, the evacuation order, such as to maximize the reward r ₂ It does not always output.

For example, assume a building having exits on the west side and the east side, and the exit on the west side is wider than the exit on the east side. It is also assumed that there is a park as a shelter on the east side of this building. In this case, it is preferable to use the exit on the west side only to evacuate from the building to the outside. However, considering evacuating from the building to the shelter, it is considered more preferable to use the exit at the east exit of the building. In such a case, it is inappropriate to perform reinforcement learning based only on the simulation result of the first evacuation simulation ^{and obtain the action value function Q *} (s, a).

Therefore, based on the reward r _2, approach to enhance learning evacuation instruction in the first evacuation simulation is required. It reinforced Therefore, in this embodiment, by associating a reward r ₁ of the first evacuation simulations and reward r ₂ of the second evacuation simulated by likelihood function, a model for outputting evacuation information from observation information Let them learn.

Specifically, first, the learning unit 24 sets various evacuation states s ₁ as initial conditions, and performs a second evacuation simulation based on each set initial condition. Then, the learning unit 24 obtains the state s ₂ evacuation obtained with the second evacuation simulation based on the initial conditions set.

Then, the learning unit 24 performs reinforcement learning based on reward r ₂ in accordance with the state s ₂ evacuation obtained with the second evacuation simulation, obtaining a behavior value function Q ^* (s, a).

Then, the learning unit 24, based on the state s ₂ states s ₁ and evacuation Evacuation obtained with the second evacuation simulation calculates the reward r ₁ and reward r _2. Then, the learning unit 24 associates the remuneration _{r 2} reward _{r 1} calculated.

Next, the learning unit 24 calculates the likelihood L (r ₁ | r ₂ ) of the reward r ₁ with respect to the reward r _{2 by using the associated reward r 1} and the reward r _2. The likelihood L (r ₁ | r ₂ ) is an index showing the likelihood of the reward r _{1 when the reward r 2 is observed.} Due to this likelihood L (r ₁ | r ₂ ), the reward r ₁ and the reward r ₂ have a relationship as shown in the following equation (6).

P (r ₂ ) = P (r ₁ ) x L (r ₁ | r ₂ )
(6)

In the above equation (6), P (r ₂ ) represents the probability that the reward will be r _{2, and P (r 1} ) represents the probability that the reward will be r ₁ .

Next, the learning unit 24 _{calculates the likelihood L (r 1} | r ₂ ), executes _{the first evacuation simulation with the evacuation state s 0} as the initial condition, and obtains the simulation result.

Then, the learning unit 24, based on the first evacuation simulation simulation results, calculates the reward r _1. Then, the learning unit 24 calculates the probability distribution of the reward r _{2 by multiplying the reward r 1} calculated from the simulation result of the first evacuation simulation by L (r ₁ | r ₂ ), and calculates the reward r 2. to maximize the _second expected value, already reinforcement learning behavioral value function Q ^* (s, a) by the results of the second evacuation simulations further enhance learning, action value function Q ^* (s, a) To get.

The evacuation information generation unit 38 of the second embodiment inputs the observation information s acquired by the acquisition unit 36 into the action value function Q ^* (s, a) learned by the learning unit 24 of the second embodiment. , Generate an evacuation order a when a disaster occurs.

<Operation of evacuation guidance model learning device>

Next, the operation of the evacuation guidance model learning device 10 of the second embodiment will be described. The evacuation guidance model learning device 10 of the second embodiment executes the learning processing routines of FIGS. 9 and 10.

<Learning processing routine>

When the specification information and the condition information are input to the evacuation guidance model learning device 10 and the reception unit 12 receives the specification information and the condition information, the setting information storage unit 22 stores the specification information and the condition information. Then, when the evacuation guidance model learning device 10 of the second embodiment receives the instruction signal of the learning process, it executes the learning process routine shown in FIGS. 9 and 10.

Steps S100 to S104 are executed in the same manner as in the first embodiment.

In step S306, the learning unit 24 blocks the block based on the building specification information read in step S100, the disaster occurrence condition set in step S102, and the evacuation instruction condition set in step S104. Execute a second evacuation simulation, which is a level evacuation simulation.

In step S308, the learning unit 24 stores the simulation result of the second evacuation simulation executed in step S306 in the storage unit (not shown).

In step S309, the learning unit 24, based on the reward r ₂ in accordance with the state s ₂ evacuation is the result obtained by the second evacuation simulations in step S308, to reinforcement learning model, action value function Q ^* (S, a) is obtained.

Steps S110 to S112 are executed in the same manner as in the first embodiment.

Next, in step S316 shown in FIG. 10, the learning unit 24 rewards r based on the _{evacuation state s 1} and the evacuation state s _{2 stored in the second evacuation simulation. 1} and reward r ₂ are calculated. Then, the learning unit 24 associates the respective each and reward _{r 2} reward _{r 1} calculated in step S316.

In step S318, the learning unit 24 uses the respective each and reward _{r 2} reward _{r 1} associated in step S316, the likelihood L _(r 1 of the reward _{r 1} for each reward _{r 2} | Calculate r ₂ ).

Step S320 is executed in the same manner as in step S102.

Step S322 is executed in the same manner as in step S104.

In step S324, the learning unit 24 _{executes the first evacuation simulation with the evacuation state s 0} as the initial condition, and obtains the simulation result.

In step S326, the learning unit 24 stores the simulation result of the first evacuation simulation in the storage unit (not shown).

In step S327, the learning unit 24 L (r ₁ | r ₂ ) with _{respect to the reward r 1} calculated from the simulation result of the first evacuation simulation stored in the storage unit (not shown) in step S326. Multiply. Then, the learning unit 24 in accordance with the above equation (6), to calculate the probability distribution of the reward r _2, to maximize the expected value of the reward r _2, already reinforcement learning by the results of the second evacuation simulations The behavioral value function Q ^* (s, a) is further strengthened and learned to obtain the behavioral value function Q ^* (s, a).

In step S328, the same execution as in step S110 is executed.

In step S330, it is executed in the same manner as in step S112.

In step S332, the learning unit 24 stores the action value function Q ^* (s, a) obtained in step S327 in the learned model storage unit 26, and ends the learning processing routine.

As described in detail above, the evacuation guidance model learning device of the second embodiment executes a plurality of types of evacuation simulations and obtains a learned model based on the simulation results. This makes it possible to take into account multiple evacuation simulations and obtain a trained model for evacuating to minimize the risk of disaster victims in the event of a disaster.

The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

For example, in the above embodiment, the case where the evacuation information output device is the display device 42 has been described as an example, but the present invention is not limited to this. For example, the evacuation information output device may be a voice output device, in which case the evacuation instruction is output by voice. Further, the evacuation information output device may be a terminal such as a smartphone owned by each evacuees.

In addition, although the mode in which the program according to the present invention is stored (installed) in advance in the storage unit (not shown) has been described above, the program according to the present invention includes a CD-ROM, a DVD-ROM, a micro SD card, and the like. It is also possible to provide the form recorded on the recording medium of.

10 Evacuation guidance model learning device 12 Reception unit 20 Computer 22 Setting information storage unit 24 Learning unit 26 Learned model storage unit 30 Evacuation guidance device 32 Observation device 34 Computer 36 Acquisition unit 37 Learned model storage unit 38 Evacuation information generation unit 40 Control Part 42 Display device

Claims (4)

An acquisition unit that acquires observation information that represents the position or movement of a person when a disaster occurs,
The observation information acquired by the acquisition unit is set according to the number of deaths, injuries, and time until evacuation is completed in the disaster of the evacuation simulation based on the result of the evacuation simulation when a disaster occurs. An evacuation information generation unit that generates evacuation information, which is information on the evacuation route when the disaster occurs, by inputting into a trained model that has been strengthened and learned in advance using the rewards.
A control unit that controls an evacuation information output device according to the evacuation information generated by the evacuation information generation unit.
Evacuation guidance device including. The trained model is a trained model that has been reinforcement-learned in advance according to the results of a plurality of types of evacuation simulations.
The evacuation guidance device according to claim 1. An evacuation simulation is executed when a disaster occurs, and based on the evacuation simulation results, the number of deaths and injuries in the disaster of the evacuation simulation, and the reward set according to the time until the evacuation is completed are used. Then, the model for outputting the information on the evacuation route when the disaster occurs is strengthened and learned from the observation information representing the position or movement of the person when the disaster occurs, and the information on the evacuation route is obtained from the observation information. A learning unit that obtains a trained model that outputs
Evacuation guidance model learning device including. The learning unit executes a plurality of types of the evacuation simulations, and obtains the learned model based on the evacuation simulation results.
The evacuation guidance model learning device according to claim 3.

Images