JP2022187237A - Crowd flow analysis device, crowd flow analysis method, and program - Google Patents

Abstract

To appropriately grasp an average flow of a crowd from a time series static images while reducing a calculation load.SOLUTION: A crowd flow analysis device includes: an image acquisition unit which acquires time series static images obtained by photographing a plurality of objects; an image dividing unit which divides each of the time series static images acquired by the image acquisition unit into a plurality of areas; an object number calculation unit which detects objects included in each of the plurality of areas divided by the image dividing unit, and calculates the number of objects included in each of the plurality of areas; a moving velocity and direction calculation unit which calculates an average moving velocity and an average moving direction of objects included in each of the plurality of areas divided by the image dividing unit; and a crowd flow calculation unit which calculates an average flow of the crowd in the entire static images on the basis of the number of objects included in each of the plurality of areas calculated by the object number calculation unit and the average moving velocity and the average moving direction of objects included in each of the plurality of area calculated by the moving velocity and direction calculation unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a crowd flow analysis device, a crowd flow analysis method, and a program.

For smart city projects in Japan and overseas, images taken by cameras installed indoors and outdoors are used to digitize the positions and movements of pedestrians, vehicles, bicycles, etc. using image analysis technology using artificial intelligence (AI).・There are increasing cases of visualization and use for various administrative services and commercial purposes.

FIG. 5 is a diagram schematically showing a conventional method of converting the flow of pedestrians and the like into data from images captured by a camera.
In the conventional method as shown in FIG. 5, the following (1) and (2) are mainly used. are the same).
(1) Pedestrians are detected by performing image analysis using deep learning, etc. in each still image that constitutes a video (= time-series still image), and the position in the screen is compared to the real space. Find coordinates.
(2) By identifying the pedestrians detected in (1) in each time-series still image (=identifying as the same pedestrian), the coordinates of each pedestrian at each time are obtained, and from the time change The flow line of each pedestrian is converted into data. The following (2-1), (2-2), etc. are available for identifying pedestrians between images, and (2-1), (2-2), etc. may be used in combination.
(2-1) Comparing the positions of pedestrians between temporally consecutive images to identify adjacent pedestrians as identical.
(2-2) Identify the same pedestrian in images at different times based on feature values such as clothes of each pedestrian.

In the above method, each pedestrian is recognized and their flow lines are converted into data, so the flow of individual pedestrians can be analyzed in detail. On the other hand, when used for flow analysis of a so-called "crowd" in which dozens of pedestrians exist in the same image, the calculation load increases for the following reasons <1> to <3>.
<1> As the number of pedestrians in the image increases, the computational load for detecting objects by deep learning or the like increases.
<2> When the above method (2-1) is used to identify pedestrians between images, in order to accurately identify the same pedestrian by position in a crowd of pedestrians, the time interval between still images to be analyzed needs to be shortened, increasing the amount of computation per time of interest (5-10 frames/s).
<3> When the above method (2-2) is used to identify pedestrians between images, in addition to pedestrian detection, a calculation process is required to identify the same pedestrian between images from the feature values of each pedestrian. , and the amount of calculation increases significantly. In addition, when the pedestrians in the screen overlap and are densely packed, and the whole body image of each pedestrian cannot be obtained, identification by the feature amount may be difficult.

For this reason, for the purpose of analyzing and using crowd flow in real time (evacuation guidance in the event of a disaster, etc.), a high-performance computer that can handle a large computational load is required. In addition, cloud servers are often used to perform such high-speed processing, but it is necessary to send images of pedestrians and others via the Internet, etc., which poses a risk in terms of personal information protection. Become.

Conventionally, there is known a video surveillance system that directly captures the macroscopic behavior of a group such as traffic flow without capturing the microscopic behavior of individual vehicles (see Patent Document 1). In the technique described in Patent Document 1, for a traffic surveillance image captured by a surveillance camera, an image signal for each pixel of a frame image is sequentially averaged in the most recent preset time (frame). Averaging processing is performed to obtain an inter-frame average image. Further, in the technique described in Patent Literature 1, a movement field (optical flow) in the image is calculated for the inter-frame average image to obtain the movement of the trajectory.
By the way, in the technique described in Patent Literature 1, no consideration is given to how many vehicles collectively represent the movement of one optical flow (movement vector). Therefore, with the technique described in Patent Document 1, there is a possibility that the flow of an average crowd cannot be properly grasped.

JP 2010-140371 A

In view of the above points, the present invention provides a crowd flow analysis device, a crowd flow analysis method, and a program capable of appropriately grasping the average crowd flow from time-series still images while reducing the computational load. for the purpose.

According to one aspect of the present invention, an image acquisition unit acquires time-series still images obtained by photographing a plurality of objects, and each of the time-series still images acquired by the image acquisition unit is divided into a plurality of regions. an image dividing unit that detects objects included in each of the plurality of regions divided by the image dividing unit, and a target object number calculating unit that calculates the number of target objects included in each of the plurality of regions; a moving speed direction calculating unit for calculating an average moving speed and an average moving direction of objects included in each of the plurality of regions divided by the image dividing unit; Based on the number of objects included in each region and the average moving speed and average moving direction of the objects included in each of the plurality of regions calculated by the moving speed direction calculation unit, the average and a crowd flow calculator for calculating the flow of a typical crowd.

An aspect of the present invention includes an image acquisition step of acquiring time-series still images of a plurality of objects photographed, and dividing each of the time-series still images acquired in the image acquisition step into a plurality of regions. an image dividing step for detecting objects included in each of the plurality of regions divided in the image dividing step, and a target number calculating step for calculating the number of target objects included in each of the plurality of regions; a moving speed direction calculating step of calculating an average moving speed and an average moving direction of objects included in each of the plurality of regions divided in the image dividing step; Based on the number of objects included in each of the regions and the average moving speed and average moving direction of the objects included in each of the plurality of regions calculated in the moving speed direction calculating step, the average of the overall still image and a crowd flow calculation step of calculating a typical crowd flow.

In one aspect of the present invention, a computer is provided with an image acquisition step of acquiring time-series still images of a plurality of objects photographed, and a plurality of time-series still images acquired in the image acquisition step. an image dividing step of dividing the image into regions; and a target object count calculation of detecting a target object included in each of the plurality of regions divided in the image dividing step and calculating the number of target objects included in each of the plurality of regions. a moving speed direction calculating step of calculating an average moving speed and an average moving direction of objects included in each of the plurality of regions divided in the image dividing step; Based on the number of objects included in each of the plurality of regions and the average moving speed and average moving direction of the objects included in each of the plurality of regions calculated in the moving speed direction calculating step, the still image and a crowd flow calculation step of calculating the overall average crowd flow.

According to the present invention, it is possible to provide a crowd flow analysis device, a crowd flow analysis method, and a program capable of appropriately grasping the average crowd flow from time-series still images while reducing the computational load.

It is a figure which shows an example of the crowd flow analysis apparatus of 1st Embodiment. FIG. 4 is a diagram for explaining an example of a plurality of areas divided by an image division unit; 4 is a flowchart for explaining an example of processing executed in the crowd flow analysis device of the first embodiment; FIG. 5 is a diagram for explaining an application example of average crowd flow in the entire still image calculated by the crowd flow analysis device of the first embodiment; FIG. 2 is a diagram schematically showing a conventional method of converting the flow of pedestrians and the like into data from images captured by a camera;

Embodiments of the crowd flow analysis apparatus, crowd flow analysis method, and program according to the present invention will be described below with reference to the drawings.

[First embodiment]
FIG. 1 is a diagram showing an example of the crowd flow analysis device 1 of the first embodiment.
In the example shown in FIG. 1, the crowd flow analysis device 1 includes an image acquisition unit 11, an image division unit 12, an object number calculation unit 13, a moving speed direction calculation unit 14, and a crowd flow calculation unit 15. there is
The image acquisition unit 11 acquires time-series still images in which a plurality of objects are captured. The image acquisition unit 11 acquires a plurality of still images taken every few seconds, for example. A plurality of objects are included in each of the plurality of still images.
The image dividing unit 12 divides each of the time-series still images acquired by the image acquiring unit 11 into a plurality of regions.

FIG. 2 is a diagram for explaining an example of a plurality of areas divided by the image division unit 12. As shown in FIG. Specifically, FIG. 2A shows an example of a plurality of areas divided by the image division unit 12. As shown in FIG.
In the example shown in FIG. 2A, a still image acquired by the image acquiring unit 11 (more specifically, a still image captured by a camera (not shown) at time t1) is divided by the image dividing unit 12 into 96 ( = 8 x 12) regions.
In another example, the still image acquired by the image acquiring section 11 may be divided into areas other than 96 by the image dividing section 12 .

In the example shown in FIG. 2A, the still image acquired by the image acquisition unit 11 includes a plurality of persons (pedestrians in detail) as a plurality of objects.
In another example, the still image acquired by the image acquisition unit 11 may include multiple runners or the like as multiple objects.

In the example shown in FIG. 1, the object number calculation unit 13 detects objects included in each of the plurality of regions divided by the image dividing unit 12, and detects each of the plurality of regions divided by the image dividing unit 12. Calculate the number of objects contained in .
In the example shown in FIG. 2A, the object number calculation unit 13 calculates “1” as the number of objects included in the first area from the left and the first area from the top, for example. For example, the object number calculation unit 13 calculates “0” as the number of objects included in the first area from the left and the sixth area from the top.

As shown in FIG. 2A, the number of objects included in each of the 96 regions divided by the image dividing unit 12 is not limited to "1" or "0". In the example shown in FIG. 2A, for example, the number of objects included in the first area from the right and the second area from the top is "2".
Therefore, in the example shown in FIG. 1 , the moving speed direction calculator 14 calculates the average moving speed and average moving direction of the object included in each of the plurality of regions divided by the image dividing unit 12 . Specifically, the moving speed direction calculating unit 14 calculates the average moving speed and the average moving speed of the object included in each of the plurality of regions based on the change in the feature amount in each of the plurality of regions divided by the image dividing unit 12 . Calculate the direction of movement.

In the example shown in FIGS. 2A and 2B, the moving speed direction calculator 14 calculates the average moving speed and the average moving direction of the objects included in each of the 96 regions (for example, the first object from the right). Also, the average moving speed and average moving direction of two pedestrians included in the second area from the top, etc.) are calculated.
FIG. 2B shows the average moving speed and the average moving direction of the object (pedestrian) included in each of the 96 regions calculated by the moving speed direction calculator 14 by vectors (arrows). Specifically, FIG. 2B shows the average moving speed and the average moving direction of objects included in each of the 96 regions that constitute the still image captured at time t1, and the time after time t1. The average moving speed and average moving direction of the object included in each of the 96 areas that make up the still image captured at t2, and the 96 areas that make up the still image captured at time t3, which is later than time t2. The average moving speed and the average moving direction of the object included in each of the regions are indicated by vectors.

In the example shown in FIG. 1, the crowd flow calculation unit 15 calculates the number of objects included in each of the plurality of areas calculated by the object number calculation unit 13 and the number of the plurality of areas calculated by the movement speed direction calculation unit 14. Based on the average moving speed and the average moving direction of the objects included in each, the average crowd flow in the entire still image is calculated.

In the example shown in FIGS. 2A and 2B, the crowd flow calculation unit 15 calculates the number of objects included in each of 96 regions that constitute the still image captured at time t1, and Based on the average movement speed and average movement direction of objects included in each of the 96 regions that make up the captured still image, the average crowd flow in the entire still image captured at time t1 is calculated. do.
In addition, the crowd flow calculator 15 calculates the number of objects included in each of the 96 regions forming the still image captured at time t2, and the number of objects included in each of the 96 regions forming the still image captured at time t2. Based on the average moving speed and the average moving direction of the objects included in 96 At time t3, based on the number of objects included in each of the 96 regions and the average moving speed and average moving direction of the objects included in each of the 96 regions that make up the still image captured at time t3 Calculate the average crowd flow over the entire captured still image.

For example, when calculating the average crowd flow in the entire still image captured at time t3, the crowd flow calculation unit 15 selects an area where the number of objects is “1” (for example, FIG. 2(A) and FIG. 2(B) ), and the weight of the vector in the region where the number of objects is “2” (for example, the first from the right and the 2nd from the top in FIGS. 2A and 2B). th region) are made different from the vector weights. Specifically, the crowd flow calculation unit 15 makes the weight of the vector of the area where the number of objects is "2" larger than the weight of the vector of the area where the number of objects is "1". Therefore, the average crowd flow can be calculated more appropriately than when the number of objects in the area is not considered.
Furthermore, the crowd flow calculation unit 15 calculates the average crowd flow in the entire still image shot at time t1, the average crowd flow in the entire still image shot at time t2, and the average crowd flow in the entire still image shot at time t3. It is also possible to calculate a change in the average crowd flow over the entire still image during the period from time t1 to time t3 based on the average crowd flow over the entire still image.

As described above, in the crowd flow analysis device 1 of the first embodiment, the image dividing unit 12 divides the target image (still image in which a plurality of targets are captured) into a plurality of regions such as grids.
The object number calculation unit 13 detects and counts objects (for example, pedestrians) using still images sampled at regular time intervals.
A moving speed direction calculating unit 14 detects a moving vector of a feature point in a still image by optical flow or the like.
The crowd flow calculation unit 15 combines the result of processing by the object number calculation unit 13 and the result of processing by the movement speed direction calculation unit 14 to calculate the average density/flow (velocity/flow) of objects for each region. direction). Furthermore, the crowd flow calculator 15 models the crowd flow of the entire still image by integrating the average density/flow (speed/direction) of the object for each region.

In the example shown in FIG. 2A, the image dividing unit 12 divides the still image into, for example, 96 areas in a grid pattern.
The object number calculation unit 13 detects objects (for example, pedestrians) as indicated by rectangles in each region in FIG. count the number of objects. Depending on the speed and density of the target crowd flow, the object number calculation unit 13 executes this process about once every several seconds. Therefore, the frequency of detection can be reduced compared to the conventional method (5 to 10 frames/second) shown in FIG. 5, and the computational load can be reduced. In addition, in the processing by the target object number calculation unit 13, only the number of target objects needs to be known. Therefore, it is not necessary to identify the target object using a feature amount or the like. may be detected. When only the pedestrian's face is detected, the computational load can be reduced compared to when the pedestrian's whole body is detected. It is possible to accurately detect a person (accurately calculate the number of pedestrians).

Moving object detection technology using optical flow, etc., as shown in Fig. 2(B), is a technique for detecting "moving objects" from changes in images in moving images (= still images in time series), and is also implemented in OpenCV. . Since the movement speed direction calculation unit 14 captures the movement of the object from the change in the feature amount of the image, it is possible to reduce the calculation load compared to the method of detecting the object by deep learning or the like. The movement speed direction calculation unit 14 calculates an average flow direction/speed from the feature amount in the screen in each of the plurality of regions divided by the image division unit 12 . Depending on the target crowd flow speed, the movement speed direction calculator 14 executes this process about once per second.

As described above, the object number calculation unit 13 calculates the number of objects (for example, pedestrians) included in each of the plurality of regions, and the moving speed direction calculation unit 14 calculates the number of objects included in each of the plurality of regions. Calculate the average moving speed and the average moving direction of an object (for example, a pedestrian).
The crowd flow calculation unit 15 integrates the result of processing by the object number calculation unit 13 and the result of processing by the moving speed direction calculation unit 14, thereby calculating an average number of objects included in each of the plurality of areas. Calculate flow. Furthermore, the crowd flow calculator 15 calculates the average crowd flow of the entire still image by integrating the average flow of the objects included in each of the plurality of regions.

The crowd flow analysis device 1 of the first embodiment can significantly reduce the computational load compared to the existing method of detecting individual flow lines of objects. Therefore, the crowd flow analysis apparatus 1 of the first embodiment can process real-time images in real time using a simpler computing environment than existing methods.
Since the data obtained by the crowd flow analysis apparatus 1 of the first embodiment is the average crowd flow (density/speed) in the image, it is not suitable for the purpose of tracking the movement of individual pedestrians. It is useful for real-time analysis of crowd flow and guidance based on the results.
In fact, in the estimation of the number of people attending the New Year's visit, etc., a method of calculating the total number of worshipers from the density and flow velocity of pedestrians visually observed at measurement points is used. It can be said that image processing and AI are applied to this method.

FIG. 3 is a flow chart for explaining an example of processing executed in the crowd flow analysis device 1 of the first embodiment.
In the example shown in FIG. 3, in step S11, the image acquiring unit 11 acquires time-series still images in which a plurality of objects are captured.
Next, in step S12, the image dividing unit 12 divides each of the time-series still images acquired in step S11 into a plurality of regions.
Next, in step S13, the object number calculation unit 13 detects objects included in each of the plurality of regions divided in step S12, and detects the number of objects included in each of the plurality of regions divided in step S12. Calculate
Further, in step S14, the average moving speed and average moving direction of the object included in each of the plurality of regions divided in step S12 are calculated.
Next, in step S15, the crowd flow calculator 15 calculates the number of objects included in each of the plurality of regions calculated in step S13 and the average movement of the objects included in each of the plurality of regions calculated in step S14. Based on the velocity and the average direction of movement, the average crowd flow over the still image is calculated.

FIG. 4 is a diagram for explaining an application example of the average crowd flow in the entire still image calculated by the crowd flow analysis apparatus 1 of the first embodiment. Specifically, it shows an example in which the flow of people calculated by the crowd flow analyzer 1 of the first embodiment is visualized.
In the example shown in FIG. 4, the crowd flow analysis device 1 of the first embodiment calculates the people flow of "small movement/stagnation" based on the time-series still images (camera images) shown in the upper left of FIG. The direction and density of the flow of people are visualized with a histogram. In addition, the crowd flow analysis apparatus 1 of the first embodiment calculates the flow of people with "large movement (multi-direction)" based on the time-series still images (camera images) shown in the lower left of FIG. • Density is visualized in a histogram. Furthermore, the crowd flow analysis apparatus 1 of the first embodiment calculates the flow of people with "large movement (one direction)" based on the time-series still images (camera images) shown in the lower right of FIG. Directions and densities are visualized by histograms.

In the crowd flow analysis apparatus 1 of the first embodiment and its application example, regarding the analysis of the flow of a crowd by camera image analysis, the average crowd flow (density, flow velocity, flow direction, etc.) is analyzed in a relatively simple computing environment. can be converted into data and visualized.
In addition, the crowd flow analysis device 1 of the first embodiment can perform real-time analysis of crowd flow on-premises, and real-time guidance and other countermeasures are possible. Also, by processing images in real time and maintaining only the data, it is possible to perform processing that is more considerate of personal information than the method of transferring images via the Internet or recording them.

[Second embodiment]
A second embodiment of the crowd flow analysis apparatus, crowd flow analysis method, and program according to the present invention will be described below.
The crowd flow analyzer 1 of the second embodiment is configured in the same manner as the crowd flow analyzer 1 of the first embodiment described above, except for the points described later. Therefore, according to the crowd flow analyzer 1 of the second embodiment, it is possible to achieve the same effects as the crowd flow analyzer 1 of the first embodiment described above, except for the points described later.

As described above, the time-series still images acquired by the image acquisition unit 11 of the crowd flow analysis apparatus 1 of the first embodiment include a plurality of persons as a plurality of objects. On the other hand, in the time-series still images acquired by the image acquisition unit 11 of the crowd flow analysis device 1 of the second embodiment, objects other than a plurality of people (for example, automobiles, bicycles, animals, etc.) are included as a plurality of objects. It is included.
Similarly to the crowd flow analysis device 1 of the first embodiment, the crowd flow analysis device 1 of the second embodiment also appropriately grasps the average crowd flow from time-series still images while reducing the computational load. be able to.

Although the embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and examples, and can be changed as appropriate without departing from the scope of the present invention. can be added. You may combine the structure as described in each embodiment and each example which were mentioned above.

It should be noted that all or part of the function of each part provided in the crowd flow analysis apparatus 1 in the above-described embodiment is obtained by recording a program for realizing these functions in a computer-readable recording medium and recording it in this recording medium. It may be realized by loading the program into a computer system and executing it. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices.
The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage units such as hard discs incorporated in computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically retains a program for a short period of time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a device that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system.

1 crowd flow analysis device 11 image acquisition unit 12 image division unit 13 object number calculation unit 14 movement speed direction calculation unit 15 crowd flow calculation unit

Claims (5)

an image acquisition unit that acquires time-series still images in which a plurality of objects are photographed;
an image dividing unit that divides each of the time-series still images acquired by the image acquiring unit into a plurality of regions;
an object number calculation unit that detects objects included in each of the plurality of regions divided by the image division unit and calculates the number of objects included in each of the plurality of regions;
a moving speed direction calculating unit that calculates an average moving speed and an average moving direction of an object included in each of the plurality of regions divided by the image dividing unit;
The number of objects included in each of the plurality of areas calculated by the number-of-objects calculation unit, and the average moving speed and average of the objects included in each of the plurality of areas calculated by the moving speed direction calculation unit and a crowd flow analysis device for calculating an average flow of the crowd in the entire still image based on the direction of movement. The plurality of objects are a plurality of pedestrians,
The number-of-objects calculation unit calculates the number of pedestrians included in each of the plurality of regions by detecting only the faces of pedestrians included in each of the plurality of regions divided by the image dividing unit. calculate,
A crowd flow analyzer according to claim 1. The moving speed direction calculating unit calculates an average moving speed and an average moving direction of an object included in each of the plurality of regions based on changes in feature amounts in each of the plurality of regions.
A crowd flow analyzer according to claim 1. an image acquisition step of acquiring time-series still images in which a plurality of objects are photographed;
an image dividing step of dividing each of the time-series still images acquired in the image acquiring step into a plurality of regions;
an object number calculation step of detecting objects included in each of the plurality of regions divided in the image dividing step, and calculating the number of objects included in each of the plurality of regions;
a moving speed direction calculating step of calculating an average moving speed and an average moving direction of an object included in each of the plurality of regions divided in the image dividing step;
The number of objects included in each of the plurality of areas calculated in the number-of-objects calculating step, and the average moving speed and average of the objects included in each of the plurality of areas calculated in the moving speed direction calculating step and a crowd flow analysis method for calculating an average flow of the crowd in the entire still image based on the direction of movement. to the computer,
an image acquisition step of acquiring time-series still images in which a plurality of objects are photographed;
an image dividing step of dividing each of the time-series still images acquired in the image acquiring step into a plurality of regions;
an object number calculation step of detecting objects included in each of the plurality of regions divided in the image dividing step, and calculating the number of objects included in each of the plurality of regions;
a moving speed direction calculating step of calculating an average moving speed and an average moving direction of an object included in each of the plurality of regions divided in the image dividing step;
The number of objects included in each of the plurality of areas calculated in the number-of-objects calculating step, and the average moving speed and average of the objects included in each of the plurality of areas calculated in the moving speed direction calculating step and a crowd flow calculation step of calculating an average crowd flow in the entire still image based on the direction of movement.

Images