KR102614856B1 - System and method for predicting risk of crowd turbulence - Google Patents

Abstract

The present invention relates to a system and method for predicting crowd turbulence risk based on images, and specifically, to a system and method for predicting crowd turbulence risk based on images, and specifically, to include at least one camera installed in a surveillance target area, taking images of the surveillance target area and transmitting the captured image data; Receives video data transmitted from the camera, identifies moving target objects from the received video data, analyzes the movement direction and relative speed of the identified target objects, and determines the movement status of the crowd in the surveillance target area and the level of risk according to the movement status. Operation server that makes judgments; and a monitoring terminal that displays processing results of the operation server.

Description

Crowd turbulence risk prediction system and method {System and method for predicting risk of crowd turbulence}

The present invention relates to a system and method for predicting crowd turbulence risk based on images.

Generally, in public places such as airports, bus stops, subway stations, etc., the excessive influx of floating population causes crowd turbulence, and if proper floating population control is not implemented, it can lead to pedestrian safety accidents.

For example, the stampede accident that occurred in Itaewon in 2022 occurred when many people crowded into a narrow road in a short period of time, and is recorded as a preventable disaster if sufficient control had been implemented.

In addition, crowd congestion accidents in emergency situations such as pedestrian safety accidents and fires due to floating population continue to occur, and various research and technological developments are being conducted to solve these problems.

The conventional way to prevent crowd congestion accidents is to control crowds through real-time monitoring using CCTV and control centers and linkage with related agencies. However, surveillance systems using human resources lead to high costs and excessive work for management authorities.

To solve this problem, methods have been developed to automate crowd congestion monitoring using AI techniques. In the case of crowd congestion monitoring technology using AI techniques, it is based on movement (quantity) or movement patterns in the video, and a method of calculating crowd flow or net crowd inflow speed using movement data is proposed.

However, in the above-described prior art, crowd congestion after a short period of time is predicted in an environment where various variables such as the intention of each floating population, external environment (congestion situation), structures that affect population flow, and traffic signal systems exist, and There is a problem in that there are limitations in predicting danger signs by evaluating the risk of crowd turbulence based on this.

Korean Patent Application No. 10-2012-0024666 (Announcement Date: 2013.09.23.)

Therefore, the present invention was developed to solve the above-mentioned problems. Based on images, it is possible to analyze and monitor the crowding of people in public places, the direction and amount of movement, sudden changes, etc., and according to the analysis results, it is possible to analyze and monitor crowd turbulence. The purpose is to provide a crowd turbulence risk prediction system and method that can predict danger signs in advance.

As a means to solve the above-described problem, the crowd turbulence risk prediction system of the present invention includes one or more cameras installed in a surveillance target area, taking images of the surveillance target area and transmitting the captured image data; Receives video data transmitted from the camera, identifies moving target objects from the received video data, analyzes the movement direction and relative speed of the identified target objects, and determines the movement status of the crowd in the surveillance target area and the level of risk according to the movement status. Operation server that makes judgments; and a monitoring terminal that displays processing results of the operation server.

As one example, the operation server includes a data acquisition unit that receives image data transmitted from the camera and acquires a plurality of consecutive image frames from the received image data; an object detection unit that detects one or more target objects estimated to be floating populations from the image frame acquired by the data acquisition unit; a moving point tracking unit that derives a moving point vector by analyzing the change in position of the target object between the current image frame and the previous image frame in a plurality of consecutive image frames; And the moving point vector for each target object derived from the moving point tracking unit is mapped to a two-dimensional rectangular coordinate system to derive a moving point coordinate system, and the movement direction of the crowd within the surveillance target area is determined through the moving point distribution of the derived moving point coordinate system. It is characterized by including a movement state analysis unit that identifies the movement state including density and relative speed, and determines the risk level based on the movement state of the identified crowd.

As an example, the operation server calculates the amount of change in the moving point for each target object after the first time set based on a moving point matrix that stores the moving point vector of each target object as a time series, and adds the calculated change amount to It is characterized by further including a moving point prediction unit that predicts the direction and relative speed of the moving point.

As an example, the operation server maps the moving point vector for each target object predicted by the moving point prediction unit to a two-dimensional rectangular coordinate system to derive a predicted moving point coordinate system, and the moving point of the derived predicted moving point coordinate system It further includes a crowd turbulence prediction unit that predicts the movement state including the movement direction, density, and relative speed of the crowd within the monitoring target area after the first time set through distribution, and determines the degree of risk based on the predicted movement state of the crowd. The characteristic is that

As an example, the moving point prediction unit sets the predicted moving point as an eliminated moving point when it leaves the monitoring target area after the set first time and reflects the effect to exclude it from the effective moving point to be used in the prediction analysis of the crowd turbulence prediction unit. It is characteristic.

Meanwhile, as a means to solve the above-described problem, the crowd turbulence risk prediction method of the present invention includes the step of taking an image of the surveillance target area through one or more cameras installed in the surveillance target area and transmitting the captured image data (S100). ; The operation server receives video data transmitted from the camera, identifies moving target objects from the received video data, and analyzes the movement direction and relative speed of the identified target objects to determine the movement status and movement status of the crowd within the surveillance target area. Step of determining the degree of risk (S200); And it is characterized by including a step (S300) of receiving and displaying the processing results of the operation server and monitoring them at the monitoring terminal.

As an example, step S200 includes receiving image data transmitted from the camera and obtaining a plurality of consecutive image frames from the received image data (S201); Detecting one or more target objects estimated to be floating populations from the acquired image frame (S202); Deriving a moving point vector by analyzing the change in position of the target object between the current image frame and the previous image frame in a plurality of consecutive image frames (S203); And the moving point vector for each target object is mapped to a two-dimensional rectangular coordinate system to derive a moving point coordinate system, and the moving point distribution of the derived moving point coordinate system determines the movement direction, density, and relative speed of the crowd within the surveillance target area. It is characterized by including a step (S204) of identifying the movement status and determining the risk based on the movement status of the identified crowd.

As an example, step S200 calculates the amount of change in the moving point for each target object after the first time set based on a moving point matrix that stores the moving point vector of each target object in time series, and adds the calculated change amount to It is characterized by further including a step (S205) of predicting the direction and relative speed of the moving point.

As an example, in step S200, a predicted moving point coordinate system is derived by mapping the moving point vector for each target object predicted in step S205 to a two-dimensional rectangular coordinate system, and the moving point distribution of the derived predicted moving point coordinate system is calculated. A step (S206) of predicting the movement state, including the movement direction, density, and relative speed of the crowd within the monitoring target area after the first time set through, and determining the degree of risk based on the predicted movement state of the crowd (S206); It is a characteristic.

As an example, in step S205, if the predicted moving point leaves the monitoring target area after the set first time, it is set as an eliminated moving point and reflected to be excluded from the effective moving point to be used in the predictive analysis in step S206. am.

In this way, according to the system and method of the present invention, the concentration of crowds, direction of movement, amount of movement, and sudden changes in public places are analyzed based on images and displayed in a display method that managers can intuitively understand, thereby reducing the concentration of crowds. It is effective in determining the occurrence of risks and responding quickly accordingly.

In particular, it predicts future floating population flows based on analysis results such as moving point metrics, and through this, predicts signs of danger caused by crowd turbulence in advance and establishes an effective response system to prevent human accidents that may occur from crowd turbulence. It works.

1 is a diagram schematically showing a crowd turbulence risk prediction system of the present invention.
Figure 2 is a block diagram showing the configuration of an operation server according to an embodiment of the present invention.
Figure 3 is a schematic diagram showing the process of deriving moving points of the moving point tracking unit according to an embodiment of the present invention.
Figure 4 is a coordinate system of movement points of a crowd derived from a movement state analysis unit according to an embodiment of the present invention.
Figure 5 is a moving point coordinate system showing the distribution of moving points of the crowd for each surveillance target area according to an embodiment of the present invention.
Figure 6 is a schematic diagram showing the process of setting a moving point to be dropped by the moving point prediction unit according to an embodiment of the present invention.
Figure 7 is a flowchart showing the crowd turbulence risk prediction method of the present invention.

In describing the present invention, the terms and words used in the specification and claims are based on the principle that the inventor can appropriately define the concept of the term in order to explain the invention in the best way. It must be interpreted with meaning and concepts consistent with the technical ideas of.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

1 is a diagram schematically showing the crowd turbulence risk prediction system of the present invention.

Referring to Figure 1, the system of the present invention may be configured to include a camera 10, an operation server 20, and a monitoring terminal 30.

One or more cameras 10 may be installed in the area to be monitored (1), and can capture images of the area to be monitored (1) and transmit the captured image data. For example, the camera 10 may include a general CCTV used for crime prevention as well as a web camera.

The camera 10 can communicate data with the operation server 20, which will be described below, through a known wired or wireless communication network, and can return image data in response to an information request from the operation server 20.

In addition, the monitoring target area 1 may be a place where crowd crowding frequently occurs or a place where crowd crowding is estimated, and the range can be selectively set by the administrator.

The operation server 20 can receive and store video data transmitted from the camera 10 installed in the surveillance target area 1.

The operation server 20 can receive and store video data from cameras 10 in a plurality of surveillance target areas 1, and monitor and manage crowd turbulence, etc. for each surveillance target area 1.

In this case, the camera 10 can generate unique identification information and transmit it to the operation server 20 along with the image data, allowing the operation server 20 to identify the corresponding surveillance target area 1. .

The operation server 20 can identify a moving target object from the received video data, and analyzes the movement direction and relative speed of the identified target object to determine the movement state of the crowd in the surveillance target area and the risk level according to the movement state. You can judge.

In addition, if the operation server 20 determines that the risk of crowd disturbance, etc. is high in the processing results, it can prevent the occurrence of safety accidents, etc. by transmitting the risk to one or more selected responsible organizations so that prompt action can be taken.

The responsible agency is an agency that manages the surveillance target area (1), and may be, for example, one of a district unit adjacent to the relevant area, a fire center, a community center, or an autonomous crime prevention unit.

Figure 2 is a block diagram showing the configuration of an operation server according to an embodiment of the present invention.

As shown in Figure 2, the operation server 20 according to an embodiment of the present invention includes a data acquisition unit 200, an object detection unit 210, a moving point tracking unit 220, a movement state analysis unit 230, and It may include a database unit 240.

The data acquisition unit 200 may receive image data transmitted from the camera 10 and obtain a plurality of image frames sequentially in chronological order from the received image data.

The image frame acquisition method by the data acquisition unit 200 can be implemented in various ways through known techniques, and detailed description thereof will be omitted.

The object detection unit 210 may detect one or more target objects estimated to be floating populations from the image frame acquired by the data acquisition unit 200.

As an example, the object detection unit 210 is implemented using an image object extraction method that applies known deep learning techniques such as R-CNN (Region-based Convolutional Neural Network), YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector). It can be.

The target object refers to a person who enters the surveillance target area (1) and is specified as an analysis target.

The moving point tracking unit 220 can receive the image frame acquired by the data acquisition unit 200, and can derive and track the moving point for the target object extracted from the object detection unit 210.

As an example, in the case of the moving point derivation method, the previously mentioned image object extraction algorithm is repeatedly executed for each consecutive frame, or an image object extraction algorithm is performed by taking into account the frame processing speed or optical flow vector estimation of all pixels in the image, such as Dense Optical flow. It can be derived by mixing and optical flow tracking algorithms.

And, as shown in FIG. 3, the moving point tracking unit 220 determines the moving point of the current image frame T1 and the previous image frame ( Changes in location over time can be analyzed through the differences between the moving points of T2).

The moving point tracking unit 220 may derive a moving point vector based on the continuous change in position of the analyzed target object.

Here, the moving point vector is a vector having information on the direction of movement and the amount (velocity) of movement of the moving target object, and can be calculated as the difference between the location of the current target object and the location of the previous target object.

'이라 하고, 유동하는 대상 객체가 Y축 방향으로 이동한 정보를 '

'이라 하며, 유동하는 대상 객체의 이동량을 '

'라고 정의될 경우, 이동점 벡터는

일 수 있다.In other words, the information that the floating target object has moved in the

', and the information that the floating target object has moved in the Y-axis direction is '

', and the movement amount of the floating target object is '

If defined as ', the moving point vector is

It can be.

The movement state analysis unit 230 receives the moving point vector for each target object derived from the moving point tracking unit 220, and maps the moving point vector for each target object to a two-dimensional rectangular coordinate system to derive a moving point coordinate system. can do.

That is, as shown in FIG. 3, the movement state of the target object can be identified by indicating the movement amount (relative speed) as the distance between the movement point and the origin as well as the movement direction of the target object in the direction of the X and Y axes of the movement point coordinate system. there is.

In this case, the faster the target object moves, the farther away the moving point in the coordinate system is from the origin. Conversely, if the target object's movement becomes stagnant, it gets closer to the origin.

The moving point coordinate system derived by the movement state analysis unit 230 can be visually displayed through the monitoring terminal 30, which will be described below.

In addition, the movement state analysis unit 230 can identify the movement state including the movement direction, density, and relative speed of the crowd within the surveillance target area (1) through the movement point distribution of the movement point coordinate system derived for all target objects. The risk level can be determined based on the movement status of the identified crowd.

Figure 4 is a diagram showing the coordinate system of the moving points of the crowd by the movement state analysis unit 230. In the coordinate system, the distribution of moving points of target objects is concentrated at the origin, so as the density increases, crowd congestion can be determined, and the target object It can be judged that the more the distribution of movement points is less concentrated and the farther away from the origin the crowd flows, the smoother it is.

Figure 5(a) is a moving point coordinate system that analyzes the movement state of the crowd using the place where pedestrians cross the crosswalk as the monitoring target area (1). The moving point detected in each direction is shown in the coordinate system, and the moving point is shown in the coordinate system. You can check the direction of movement and relative speed of the crowd using a point coordinate system.

Also, in this case, abnormal behavior or dangerous situations of pedestrians can be identified through the moving point coordinate system. For example, pedestrians leaving the crosswalk or moving too slowly can be identified based on the coordinate system.

Figure 5(b) shows an example of analysis of a place where crowds move in one direction. The movement direction, density, and amount of movement of the crowd can be identified through the moving point coordinate system, and the risk due to crowd crowding is determined based on this. An algorithm can be derived.

And Figure 5(c) shows an example of a crowd waiting at a subway station. Through the moving point coordinate system, it is possible to check whether the crowd is stationary, that is, without movement, and its density.

The database unit 240 may store image data received from the camera 10 for each surveillance target area 1 and image frames obtained therefrom.

And the database unit 240 provides the moving point vector information for each target object processed by the moving point tracking unit 220 as well as the monitoring target area (1) derived based on the moving point vector in the moving state analysis unit 230. ), rectangular coordinate system information, etc. can be stored, and the corresponding information can be searched and provided in response to a request from the monitoring terminal 30.

The monitoring terminal 30 is connected to the operation server 20 through a known wired or wireless communication network and can display processing results from the operation server 20 or information previously stored in the database 240 of the operation server 20. there is.

The operation server 20 can grant access rights to the monitoring terminal 30, and process the processing results or previously stored information into monitoring information suitable for the operating system of the connected monitoring terminal 30 and provide it. .

That is, the monitoring terminal 30 monitors the crowding, direction of movement, and amount of movement in the area in real time by displaying the movement status of the crowd in the area to be monitored (1) in a two-dimensional rectangular coordinate system that can be intuitively understood by the manager. Not only can this be done, but it also makes it easier to identify risks, including sudden changes in flow.

The monitoring terminal 30 may be either a personal computer of the manager or a mobile terminal carried by the manager, and is connected to the operation server 20 in the form of a web page-based, web application, or mobile application to provide monitoring information. You can receive and display, select the surveillance target area (1) you want to monitor, or set the UI (user interface) environment, etc.

Meanwhile, the operation server 20 further includes a moving point prediction unit 250 and a crowd turbulence prediction unit 260, as shown in FIG. 2, to detect signs of danger due to crowd turbulence in the monitoring target area 1 in advance. It can be sensed.

First, the moving point prediction unit 250 calculates the amount of change in the moving point for each target object after the first time set based on the moving point matrix that stores the moving point vector of each target object as a time series, and calculates the amount of change in the moving point for each target object according to the calculated amount of change. The direction and relative speed of the moving point can be predicted.

At this time, the first time may be in seconds or minutes, and it is natural that it can be selectively set by the administrator.

The moving point prediction algorithm by the moving point prediction unit 250 may apply a known mathematical algorithm. For example, the moving point can be predicted through [Equation 1] below, which applies a stochastic differential equation.

는 시간에 따라 변화하는 이동점 정보로 향후 이동점의 다음 움직임 값을 표현하는 것이고,

는 drift 항으로 현재까지의 이동점 변화를 설명하는 것이며,

는 diffusion 항으로 이동점의 변동성을 표현하는 것이고,

는 시간에 따라 변화하는 확률변수를 의미한다. In Equation 1 above,

is moving point information that changes over time and expresses the next movement value of the moving point in the future.

is a drift term that explains the change in moving point to date,

is a diffusion term that expresses the volatility of the moving point,

refers to a random variable that changes over time.

That is, based on previously acquired moving point information, the amount of change in the moving point can be calculated and the direction and movement amount of the moving point can be predicted.

At this time, if the predicted moving point leaves the monitoring target area after the set first time, the moving point prediction unit 250 sets it as an eliminated moving point and excludes it from the effective moving point to be used in the predictive analysis of the crowd turbulence prediction unit 260. It can be reflected as well.

Predicted moving points can be divided into effective moving points and eliminated moving points. A valid moving point means that the target object pointed to by the predicted moving point exists in the surveillance area after the prediction time, that is, the set first time, and an dropped moving point means a moving point where the predicted moving point leaves the monitoring area at the predicted time.

As shown in FIG. 6, the surveillance target area 1 may be composed of a virtual N The outside of can be divided into a virtual area.

When a target object is detected and identified by the object detection unit 210, its location can be mapped on a map, and its movement trajectory can be tracked using the direction of the movement point and the movement amount value.

Additionally, the movement point prediction unit 250 can predict the movement trajectory of the target object up to the prediction time. Based on this, it can be confirmed whether the target object is included in the surveillance area. If the moving point of the target object is not within the surveillance area, more reliable analysis results can be derived by excluding it from the density analysis.

Like the movement state analysis unit 230, the crowd turbulence prediction unit 260 maps the movement point vector for each target object predicted by the movement point prediction unit 250 to a two-dimensional rectangular coordinate system to derive a predicted movement point coordinate system. can do.

In addition, the crowd turbulence prediction unit 260 can predict the movement state, including the movement direction, density, and relative speed of the crowd in the surveillance target area after the first time set through the movement point distribution of the derived predicted movement point coordinate system, and predicts The level of risk can be judged based on the movement status of the crowd.

At this time, the crowd density through the moving point coordinate system can be analyzed on a rule basis by applying prior information on the area to be monitored (1), such as the area of the area, whether there is a structure, the maximum number of people, etc.

For example, the crowd density can be calculated by analyzing the total number of moving points or the distance between moving points in the moving point coordinate system, and as mentioned above, moving points that are eliminated as effective moving points can be used to predict future density after the first time. The method of excluding and the inflow speed of the target object can be used as basic data.

In addition, the crowd turbulence prediction unit 260 can predict and determine the future risk after the first hour based on the predicted movement state of the crowd.

The degree of risk due to crowd turbulence can be determined by the density of the crowd and whether the crowd is stagnant, and the distribution of moving points in the moving point coordinate system can be derived using a known statistical analysis method.

For example, through cluster analysis of the distribution of moving points, the results of the number of clusters, size of clusters, distance between clusters, etc. can be interpreted to derive crowd flow and density, and whether crowd turbulence is formed.

Additionally, the risk level according to the situation can be analyzed as a result of the analysis. Even if the density of target objects is maintained high, if a certain amount of movement (speed) is predicted to be secured, the risk level can be considered relatively low.

However, if the density of target objects remains high and the amount of movement (speed) also shows a decreasing trend, congestion is expected to occur in the near future and the risk can be judged high.

In this way, the crowd turbulence prediction unit 260 can assume various crowd turbulence situations based on the moving point coordinate system and use them to determine risk. At this time, the risk level is judged step by step according to the predicted results, such as alert level and risk level, and the judgment result is displayed as a notification signal through the monitoring terminal 30, allowing the manager to take prompt on-site action according to the situation. It is desirable to do so.

Figure 7 is a flowchart showing the crowd turbulence risk prediction method of the present invention.

The method of FIG. 7 may be performed by the crowd turbulence risk prediction system shown in FIG. 1.

The crowd turbulence risk prediction method of the present invention includes the step of capturing an image of the surveillance target area (1) through one or more cameras (10) installed in the surveillance target area (1) and transmitting the captured image data (S100); The operation server 20 receives video data transmitted from the camera 10, identifies a moving target object from the received video data, analyzes the movement direction and relative speed of the identified target object, and moves the crowd within the surveillance target area. It may include a step of determining the degree of risk according to the state and movement state (S200) and a step of receiving and displaying the processing results of the operation server 20 at the monitoring terminal 30 and monitoring them (S300).

The S200 step performed by the operation server 20,

A step (S201) in which the data acquisition unit 200 receives image data transmitted from one or more cameras 10 installed in the surveillance target area 1 and acquires a plurality of consecutive image frames from the received image data. Perform.

Then, the object detection unit 210 receives the image frame acquired by the data acquisition unit 200 and performs a step (S202) of detecting one or more target objects estimated to be floating population from the image frame.

Object extraction by the object detection unit 210 is performed by applying known deep learning techniques including R-CNN (Region-based Convolutional Neural Network), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), etc. Since this may be the case, detailed description of this will be omitted.

Then, the moving point prediction unit 250 performs a step (S203) of deriving a moving point vector by analyzing the change in position of the target object between the current image frame and the previous image frame in a plurality of consecutive image frames.

Afterwards, the moving point vector for each target object derived from the moving point prediction unit 250 is mapped to a two-dimensional rectangular coordinate system to derive a moving point coordinate system, and the moving point distribution of the derived moving point coordinate system is used to determine the number of crowds in the surveillance target area. Movement status, including movement direction, density, and relative speed, is identified, and the level of risk is judged based on the movement status of the identified crowd.

At this time, the processing results of the operation server 20, including the moving point coordinate system derived from the moving point prediction unit 250, are provided and displayed on the monitoring terminal 30, allowing the manager to determine the density and movement of the monitoring target area 1. Not only will it be possible to monitor the direction and amount of movement in real time, but it will also be possible to easily identify risks including sudden changes in flow.

Meanwhile, the step S200 by the operation server 20 calculates the amount of change in the moving point for each target object after the first time set based on the moving point matrix storing the moving point vector of each target object in time series, and calculates A step (S205) of predicting the direction and relative speed of the moving point according to the amount of change can be further performed.

At this time, the first time may be in seconds or minutes, and it is natural that it can be selectively set by the administrator.

Additionally, the moving point prediction algorithm by the moving point prediction unit 250 may be implemented by applying known mathematical algorithms, including stochastic differential equations.

Afterwards, in step S200, the crowd turbulence prediction unit 260 maps the moving point vector for each target object predicted in step S205 to a two-dimensional rectangular coordinate system to derive a predicted moving point coordinate system.

In addition, the movement state including the movement direction, density, and relative speed of the crowd within the surveillance area is predicted after the first time set through the movement point distribution of the derived predicted movement point coordinate system, and the risk level is determined based on the predicted movement state of the crowd. A judgment step (S206) may be further performed.

By determining the future risk, it is possible to detect in advance signs of danger due to crowd turbulence in the monitored area (1).

Here, in step S205, if the predicted moving point leaves the monitoring target area after the set first time, it can be set as an eliminated moving point and excluded from the effective moving point to be used in the predictive analysis in step S206.

In this process (step S205), the surveillance target area (1) is configured as a virtual N After predicting the movement trajectory of the target object, it is possible to set an eliminated movement point by checking whether the movement point according to the predicted movement trajectory of the target object is included in the surveillance area or deviates from the surveillance area.

In this way, in the method of the present invention, the crowd turbulence prediction unit 260 can predict in advance the signs of danger caused by future crowd turbulence, and in this process, performs a correction process to exclude dropped moving points that may act as noise, thereby increasing reliability. analysis results can be derived.

Although the preferred embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also possible. falls within the scope of rights.

10: Camera 20: Operation server
30: monitoring terminal 200: data acquisition unit
210: object detection unit 220: moving point tracking unit
230: Movement status analysis unit 240: Database unit
250: moving point prediction unit 260: crowd turbulence prediction unit

Claims (10)

One or more cameras installed in the area to be monitored and taking images of the area to be monitored and transmitting the captured image data;
Receives video data transmitted from the camera, identifies moving target objects from the received video data, analyzes the movement direction and relative speed of the identified target objects, and determines the movement status of the crowd in the surveillance target area and the level of risk according to the movement status. Operation server that makes judgments; and
It includes a monitoring terminal that displays the processing results of the operation server,
The operating server is,
a data acquisition unit that receives image data transmitted from the camera and acquires a plurality of consecutive image frames from the received image data;
an object detection unit that detects one or more target objects estimated to be floating populations from the image frame acquired by the data acquisition unit;
a moving point tracking unit that derives a moving point vector by analyzing the change in position of the target object between the current image frame and the previous image frame in a plurality of consecutive image frames;
A moving point coordinate system is derived by mapping the moving point vector for each target object derived from the moving point tracking unit to a two-dimensional rectangular coordinate system, and in the derived moving point coordinate system, when the moving point moves away from the origin, the target object moves quickly. When the moving point is close to the origin, the target object is judged to be stagnant; if the moving point distribution is concentrated at the origin, it is judged to be crowded; the farther the moving point distribution is from the origin, the smoother the crowd flow is judged to be. A movement status analysis unit that identifies the movement state, including the movement direction, density, and relative speed of the crowd within the surveillance target area, and determines the degree of risk based on the movement status of the identified crowd; and
Calculate the amount of change in the moving point of each target object after the first time set based on the moving point matrix that stores the moving point vector of each target object as a time series, and predict the direction and relative speed of the moving point according to the calculated change amount. It includes a moving point prediction unit that does,
The moving point prediction unit,
A crowd turbulence risk prediction system characterized by predicting moving points through the equation below applying a stochastic differential equation.
[Equation]

here, is moving point information that changes over time and expresses the next movement value of the moving point in the future. is a drift term that explains the change in moving point to date, is a diffusion term that expresses the volatility of the moving point, is a random variable that changes over time
delete delete According to clause 1,
The operating server is,
A predicted moving point coordinate system is derived by mapping the moving point vector for each target object predicted by the moving point prediction unit to a two-dimensional rectangular coordinate system, and the monitoring target after the first time set through the moving point distribution of the derived predicted moving point coordinate system A crowd turbulence risk prediction system further comprising a crowd turbulence prediction unit that predicts the movement state, including the movement direction, density, and relative speed of the crowd in the area, and determines the risk based on the predicted movement state of the crowd. .
According to clause 4,
The moving point prediction unit,
A crowd turbulence risk prediction system, wherein if the predicted moving point leaves the monitoring target area after the set first time, it is set as an eliminated moving point and reflected to be excluded from the effective moving point to be used in the predictive analysis of the crowd turbulence prediction unit.
Taking an image of the surveillance target area through one or more cameras installed in the surveillance target area and transmitting the captured image data (S100);
The operation server receives video data transmitted from the camera, identifies moving target objects from the received video data, and analyzes the movement direction and relative speed of the identified target objects to determine the movement status and movement status of the crowd within the surveillance target area. Step of determining the level of risk (S200); and
It includes a step (S300) of receiving and displaying the processing results of the operation server and monitoring them at the monitoring terminal,
In step S200,
Receiving image data transmitted from the camera and obtaining a plurality of consecutive image frames from the received image data (S201);
Detecting one or more target objects estimated to be floating populations from the acquired image frame (S202);
Deriving a moving point vector by analyzing the change in position of the target object between the current image frame and the previous image frame in a plurality of consecutive image frames (S203);
A moving point coordinate system is derived by mapping the moving point vector for each target object to a two-dimensional rectangular coordinate system. In the derived moving point coordinate system, if the moving point moves away from the origin, the target object is judged to be moving quickly, and the moving point is located at the origin. If it is close, the target object is judged to be stagnant. If the distribution of moving points is concentrated at the origin, it is judged to be crowded. The farther the distribution of moving points is from the origin, the smoother the flow of the crowd is judged to be, and the more the crowd moves within the surveillance area. Identifying the movement state including the direction of movement, density, and relative speed, and determining the risk level based on the movement state of the identified crowd (S204); and
Calculate the amount of change in the moving point of each target object after the first time set based on the moving point matrix that stores the moving point vector of each target object as a time series, and predict the direction and relative speed of the moving point according to the calculated change amount. Including a step (S205);
In step S205,
A crowd turbulence risk prediction method characterized by predicting moving points through the equation below applying a stochastic differential equation.
[Equation]

here, is moving point information that changes over time and expresses the next movement value of the moving point in the future. is a drift term that explains the change in moving point to date, is a diffusion term that expresses the volatility of the moving point, is a random variable that changes over time
delete delete According to clause 6,
In step S200,
In step S205, the predicted moving point vector for each target object is mapped to a two-dimensional rectangular coordinate system to derive a predicted moving point coordinate system, and within the monitoring target area after the first time set through the moving point distribution of the derived predicted moving point coordinate system. A crowd turbulence risk prediction method further comprising: predicting the movement state of the crowd, including the movement direction, density, and relative speed, and determining the risk level based on the predicted movement state of the crowd (S206).
According to clause 9,
In step S205,
A crowd turbulence risk prediction method, characterized in that, if the predicted moving point leaves the monitoring target area after the set first time, it is set as an eliminated moving point and reflected to be excluded from the effective moving point to be used in the predictive analysis in step S206.

Images