WO2017088092A1

WO2017088092A1 - Pedestrian travel time estimation

Info

Publication number: WO2017088092A1
Application number: PCT/CN2015/095274
Authority: WO
Inventors: Xiaogang Wang; Shuai Yi; Hongsheng LI
Original assignee: Chinese University of Hong Kong CUHK
Current assignee: Chinese University of Hong Kong CUHK
Priority date: 2015-11-23
Filing date: 2015-11-23
Publication date: 2017-06-01
Anticipated expiration: 2018-05-23

Abstract

Disclosed are methods and devices for estimating pedestrian travel time from an entrance to a destination. A method comprises generating an active region from the entrance to the destination based on a surveillance video including the entrance and the destination (101); extracting a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region (102); calculating an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in a second order polynomial regression model (103).

Description

Pedestrian Travel Time Estimation

Technical Field

This disclosure relates to pedestrian travel time estimation, in particular, to method and apparatus for estimating pedestrian travel time from an entrance to a destination.

Background

Crowd scene understanding and pedestrian behavior analysis are important for video surveillance. People would like to discover scene properties and understand what is happening in the scene. In the meanwhile, they are also interested in revealing the rules governing individual behaviors. Scene information and pedestrian behaviors are correlated. Pedestrian travel time from an entrance to an exit is such a measurement that reflects information from both sides.

For surveillance applications, people show great interest in estimating pedestrian travel time during transportation between entrances and exits, since it indicates traffic efficiency and travel cost of a scene. Based on the travel time information, security administrators can take prompt actions in response to the quick increase of travel time due to traffic congestion. For example, they may block some entrances until the congested crowds disperse, or open extra exists. Such information is also useful for travelers to make plans. Travel time itself is also an important feature to describe each individual's behavior. It can be determined whether one pedestrian is behaving normally or not based on the information of travel time.

However, estimating travel time is challenging, especially for scenes with crowds. Firstly, pedestrian travel time shows large inter-person variation. Even under the same situation and for the same source-destination pair, the walking paths and speed of individuals might be quite different, which leads to large variance of travel times. Secondly, pedestrian decision making is complex and the travel time of individuals might be influenced by a variety of factors, such as the interactions with moving persons, stationary persons, and the scene layout. Lastly, the problem becomes much more challenging in crowded scenes, where existing computer vision techniques, such as pedestrian detection, tracking, and re-identification, cannot provide accurate results.

Content of the Invention

The present disclosure aims to provide a method and a device for estimating pedestrian travel time accurately.

According to one aspect, a method for estimating pedestrian travel time from an entrance to a destination is disclosed, which comprises: generating an active region from the entrance to the destination based on a surveillance video including the entrance and the destination； extracting a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and calculating an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in a second order polynomial regression model.

According to another aspect, a device for estimating pedestrian travel time from an entrance to a destination is disclosed, which comprises a processor and a memory storing computer executable instructions and a second order polynomial regression model, wherein, when the computer executable instructions are executed by the processor, the processor is operable to generate an active region from the entrance to the destination based on a surveillance video including the entrance and the destination； extract a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and calculate an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in the second order polynomial regression model.

According to a further aspect, a computer program product storing computer executable instructions and a second order polynomial regression model is disclosed, wherein, when the computer executable instructions are executed by a processor, the processor is operable to generate an active region from the entrance to the destination based on a surveillance video including the entrance and the destination； extract a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and calculate an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in the second order polynomial regression model.

Drawings

Figure 1 illustrates a flowchart of a method for estimating pedestrian travel time according to an embodiment of the present application；

Figure 2 illustrates a schematic of features extracted from moving pedestrians；

Figure 3 illustrates a schematic of features extracted from stationary pedestrians； and

Figure 4 illustrates a block diagram of a device for estimating pedestrian travel time according to an embodiment of the present application.

Detailed Description

The following detailed description is provided with reference to the accompanying drawings and embodiments. It should be appreciated that the embodiments described herein are provided to illustrate the present invention, but not to limit the present invention. In addition, it should be noted that only the related parts of the present invention are shown in the accompanying drawings for ease of description.

It should be noted that the embodiments and features of the embodiments in the present application, on a non-conflicting basis, may be combined. The present application will be discussed in details below with reference to the accompanying drawings.

Figure 1 illustrates a flowchart of a method 100 for estimating pedestrian travel time according to an embodiment of the present application. As shown, at step 101, an active region from the entrance to the destination is generated based on a surveillance video including the entrance and the destination. At step 102, a first set of features are extracted from moving pedestrians in the active region and a second set of features are extracted from stationary crowd groups in the active region, respectively. At step 103, an estimated pedestrian travel time from the entrance to the destination is calculated by using the first set of features and the second set of features in a second order polynomial regression model.

In an embodiment, the active region is a weighted map including an active weight corresponding to each location in the active region.

In an embodiment, the first set of features comprise a first location feature based on the active weight corresponding to the location of each moving pedestrian and a first influence feature based on a walking direction of each moving pedestrian.

In an embodiment, the second set of features comprise a second location feature based on the active weight corresponding to the location of each stationary pedestrian in the stationary crowd groups, a second influence feature based on a size of each stationary crowd group, and a third influence feature based on a density of each stationary crowd group.

In an embodiment, the second order polynomial regression model is previously obtained by learning.

Hereinafter, embodiments of the present application will be described in detail.

According to the present application, a surveillance camera video is used as input to estimate the pedestrian travel time. For each time point t, an average travel time

within a short period [t-τ， t+τ] ， is estimated for pedestrians coming from a source S and going to a destination D.

can be considered as a property of the scene. From the travel times between sources and destinations, one can better understand the current status of the scene. For clarity, the solution of the present application is described by estimating the travel time between one source and one destination, and T is used to denotein the descriptions below.

Since complete walking routes of pedestrians are not observed when estimating travel time, the active region R is estimated at step 101to cover the areas occupied by traffic flows of most pedestrians traveling between the source and the destination. The activities happening inside R are considered to extract features from these activities to estimate the travel time.

Different source-destination traffic flows have different active regions, which may have overlap. The active regions can be used to discover the underlying relationships between scene locations and source-destination traffic flows. From the abnormal increase of travel time of some source-destination flows, probable blocking areas can be located in the active region. On the other hand, if certain activities happen at some locations inside the active region, the probable increase of travel time of some source-destination traffic can be predicted.

According to an embodiment of the present application, the active region of one source-destination traffic flow is a weighted map, instead of a binary one. This is because different locations inside the active region should have different importance. For example, activities happening at some important locations on the main roads should have greater influence on the travel time, and higher influence weights should be assigned to these locations. Let

be the collection of locations covered by walking routes of all pedestrians coming from the source S and going to the destination D. The influence weight of the active region map at location l is calculated as

where k is the Gaussian kernel and the kernel bandwidth is defined as the size of one pedestrian. #

counts the number of elements in

and is used as the normalization term.

can be obtained by clustering fragmented tracklets between source and destination regions with a dynamic agent-based model, for example, the dynamic agent-based model disclosed by B. Zhou, X. Wang, and X. Tang in “Understanding collective crowd behaviors: Learning a mixture model of dynamic pedestrian-agents” in Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2012.

It is observed that the active region of a source-destination pair may contain multiple potential walking routes. Moreover, narrow areas (e.g. entrance regions and exit regions) tend to have higher influence weights, which indicates more attention should be paid to these areas in traffic management. Note that the active region is not symmetric i.e.

which means traffic flows in different directions might occupy different scene regions. This is common in transportation systems where mixing of traffic flows in opposite directions is avoided in order to increase traffic efficiency and ensure safety.

According to statistical study on which factors have large influence on the travel time between sources and destinations, it is learned that, in addition to locations of the pedestrians in the active region, walking directions of the moving pedestrians and the group size and the group density of stationary crowd groups have strong correlation with the travel time.

For the traffic flow from source S to destination D at the current time point t, features are extracted from all the moving pedestrians inside the active region. The locations of these moving pedestrians are denoted as

and

For each of these moving pedestrians, two features (one location feature and one influence feature) can be computed, as shown in Figure 2. As shown, the first set of features 200comprise a first location feature 201based on the active weight corresponding to the location of each moving pedestrian and a first influence feature 202based on a walking direction of each moving pedestrian

The active region weights at the locations of the N^mp moving persons are used as the first feature. For the i-th moving pedestrian, the location feature is calculated

For example, given an active region map and two locations (A and B) , if the active region weight at Location A is larger than that at Location B, Location A is in the main traffic flow. A moving pedestrian appearing at A plays a more important role than that appearing at B.

The influence to travel time delay is also related to pedestrian's walking direction θ_i， i＝1， ...， N^mp. For example, if a moving pedestrian appears at one location and walks along the direction being opposite to the source-destination traffic flow, the influence of this pedestrian should be significant. In contrast, if he/she walks along the direction similar to the source-destination ow, the influence should be small.

For all the locations inside the active region, the distributions of walking directions of the source-destination ow are first computed. The second feature is disclosed to describe the deviation of the moving pedestrian's walking direction from the speed directional distribution of the source-destination traffic ow. For the i-th moving pedestrian, the influence feature is calculated as

where

is the directional distribution of the traffic ow at the location

Features are extracted from all the stationary pedestrians sp_j， j＝1， ...， N^sp inside the active region, whose locations are denoted by

For each stationary pedestrian, three features (one location feature and two influence features) are computed, as shown in Figure 3. As shown, the second set of features 300comprise a second location feature 301 based on the active weight corresponding to the location of each stationary pedestrian in the stationary crowd groups, a second influence feature 302based on a size of each stationary crowd group, and a third influence feature 303based on a density of each stationary crowd group. Stationary pedestrians at different locations have different importance, which can be described by the map weights of

For example, there is an active region map and two stationary groups A and B. Stationary pedestrians of group B block the main source-destination traffic flow, which leads to larger influence on the travel time than the stationary pedestrians of group A. For the j-th stationary pedestrian, the location feature is calculated as

There are two other influence features, S² _j and S³ _j. S² _j is related to the size of the stationary crowd group. It is more likely for a pedestrian to change the route and detour a longer way when facing a larger stationary group. The second stationary pedestrian feature describes the size of a stationary group,

where G (sp_j) is the collection of all the stationary persons that form the same stationary crowd group with sp_j, and#is the element counting operation.

The influence feature S³ _j is related to the density of the stationary crowd group. Stationary pedestrians of denser groups should have larger blocking effect. If a stationary group is small or sparse, some aggressive pedestrians may choose to go through it instead of changing their routes. The third stationary pedestrian feature therefore describes the density of a stationary group,

where p is a stationary pedestrian from the same stationary group with sp_i, l_p is the location of stationary pedestrian p, and

measures the distance between p and sp_j.

Features extracted from moving and stationary pedestrians are concatenated into feature vectors and a regression pipeline is adopted to map these feature vectors to travel times. For a source-destination traffic flow at time point t, moving features

are extracted from N^mp moving pedestrians inside the active region, and stationary features

are extracted from N^sp s tationary pedestrians inside the active region, i.e.

where tr denotes matrix transpose. In order to keep a fixed number of features at each time point, only N^m* moving pedestrians with top

values and N^s* stationary pedestrians with top

values are selected. These selected pedestrians are expected to have the most influences on travel time as they have the largest active region weights, i.e. they are at locations that significantly influences travel time. Features of the selected persons are then used for regression. Zeros will be padded if N^m*＞N^mp or N^s*＞N^sp.

It is assumed that the mapping function from moving features to travel time should be the same for all the moving pedestrians (denoted as f_m) . Similarly, the mapping function from stationary features to travel time is denoted as fs. The travel time can then be estimated as

where B_mp is the index set of moving pedestrians with top

values, and B_sp is the index set of stationary pedestrians with top

values.

According to an embodiment, the second-order polynomial regression is used, and f_m (M_i) and f_s (S_j) can be written as,

where W_m and W_s are systemic matrices to be learned. That is, a previous learning work has been performed so that systemic matrices W_m and W_s are for the second order polynomial regression model is obtained and thus the second order polynomial regression model is established for later using.

Figure 4 illustrates a block diagram of a device for estimating pedestrian travel time according to an embodiment of the present application. As shown, the device 400 includes a processor 401, for example a central processing unit (CPU) ； a communication interface 402； a user interface 403； a memory 404； and a communication bus 405. The communication bus 405 is configured for the connections and communications among the above-mentioned components. The device 400 may optionally include a user interface 403, for example a display component, a keyboard or a pointing device (for example, a mouse, a track ball, a touchpad, or a touch screen) , and the like. The memory 404 may comprise a high-speed random access memory (RAM) , or a non-volatile memory, such as a magnetic disc memory. The memory 404 may optionally include a storage device located remotely from the above-mentioned processor 401.

In some embodiments, the memory 404 stores executable modules or data structures, or the subset or extended set thereof, such as: an operation system 414 comprising various system programs for implementing various basic services and handling hardware interfacing tasks； and an application program 424 comprising various application programs for implementing various application services.

In this embodiment, the memory 404 is configured to store computer executable instructions and a second order polynomial regression model which may be obtained by previous learning. When the computer executable instructions are executed by the processor 401, the processor 401 is operable to generate an active region from the entrance to the destination based on a surveillance video including the entrance and the destination； extract a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and calculate an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in the second order polynomial regression model.

In addition, there is provided a computer readable storage media, which may be a computer readable storage media included in the apparatus as disclosed in the above embodiments, or a computer readable storage media existing independently rather than embedded in any terminal. The computer readable storage media stores one or more computer programs, the computer programs executed by one or more processors to perform the method for estimating pedestrian travel time from an entrance to a destination.

The estimated travel time can provide rich information for video surveillance, and various applications can be implemented based on the disclosed travel time estimation pipeline, including crowd scene understanding and pedestrian behavior analysis.

The estimated travel time is an important indicator of scene status and can be used to monitor whether the scene is unobstructed or not. For example, at some time point A, the scene is unobstructed and the estimated travel time is short. However, at time point B, the source-destination ow might be intersected by multiple moving pedestrians, which leads to the increase of travel time. For time point C, some region can be blocked by the large dense stationary crowds, thus the estimated travel time increases significantly. When the stationary crowds disperse several minutes later, the estimation of travel time returns to normal value. In this way, the estimated pedestrian travel time can be used to indicate whether the scene is in good condition or not.

In this disclosure, the travel time is estimated by summing all the influences of moving and stationary pedestrians inside the active region together. On the other way around, the total travel time can be factorized into different influence factors, and the time delay caused by each individual inside the active region can be inferred. In this way, the regions that blocking traffics can be located by identifying the moving/stationary persons that contribute most to the estimation result of the travel time.

When focus is on each individual, it can be determined whether the pedestrian is walking in a normal way, i.e. the travel time is close to the estimation result. For example, a pedestrian walks too slowly, so that the actual travel time is significantly greater than the estimation. Some pedestrian might be running fast, which leads to a much shorter travel time. Some other pedestrians are walking along tortuous routes, so the travel times are greater than expected. These pedestrians'travel time might be quite different from the estimation result according to the present disclosure and these special activities can be detected based on the travel time information.

The above description only refers to the preferred embodiments of the present application and explains the applicable technical principles. It will be appreciated by those skilled in the art that the scope of the claimed solutions as disclosed in the present application are not limited to those consisted of particular combinations of features described above, but should cover other solutions formed by any combination of features from the foregoing or an equivalent thereof without departing from the inventive concepts, for example, a solution formed by replacing one or more features as discussed in the above with one or more features with similar functions disclosed (but not limited to) in the present application.

Claims

A method for estimating pedestrian travel time from an entrance to a destination, comprising:

generating an active region from the entrance to the destination based on a surveillance video including the entrance and the destination；

extracting a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and

calculating an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in a second order polynomial regression model.
The method of claim 1, wherein the active region is a weighted map including an active weight corresponding to each location in the active region.
The method of claim 2, wherein the first set of features comprise a first location feature based on the active weight corresponding to the location of each moving pedestrian and a first influence feature based on a walking direction of each moving pedestrian.
The method of claim 3, wherein the second set of features comprise a second location feature based on the active weight corresponding to the location of each stationary pedestrian in the stationary crowd groups, a second influence feature based on a size of each stationary crowd group, and a third influence feature based on a density of each stationary crowd group.
The method of claim 1, wherein the second order polynomial regression model is previously obtained by learning.
A device for estimating pedestrian travel time from an entrance to a destination, comprising:

a processor； and

a memory storing computer executable instructions and a second order polynomial regression model,

wherein, when the computer executable instructions are executed by the processor, the processor is operable to:

generate an active region from the entrance to the destination based on a surveillance video including the entrance and the destination；

extract a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and

calculate an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in the second order polynomial regression model.
The device of claim 6, wherein the active region is a weighted map including an active weight corresponding to each location in the active region.
The device of claim 7, wherein the first set of features comprise a first location feature based on the active weight corresponding to the location of each moving pedestrian and a first influence feature based on a walking direction of each moving pedestrian.
The device of claim 8, wherein the second set of features comprise a second location feature based on the active weight corresponding to the location of each stationary pedestrian in the stationary crowd groups, a second influence feature based on a size of each stationary crowd group, and a third influence feature based on a density of each stationary crowd group.
The device of claim 6, wherein the second order polynomial regression model is previously obtained by learning.
A computer program product, including storing computer executable instructions and a second order polynomial regression model, wherein, when the computer executable instructions are executed by a processor, the processor is operable to:

generate an active region from the entrance to the destination based on a surveillance video including the entrance and the destination；

extract a first set of features from moving pedestrians in the active region and a second set of features from stationary crowd groups in the active region； and

calculate an estimated pedestrian travel time from the entrance to the destination by using the first set of features and the second set of features in the second order polynomial regression model.
The computer program product of claim 11, wherein the active region is a weighted map including an active weight corresponding to each location in the active region.
The computer program product of claim 12, wherein the first set of features comprise a first location feature based on the active weight corresponding to the location of each moving pedestrian and a first influence feature based on a walking direction of each moving pedestrian.
The computer program product of claim 13, wherein the second set of features comprise a second location feature based on the active weight corresponding to the location of each stationary pedestrian in the stationary crowd groups, a second influence feature based on a size of each stationary crowd group, and a third influence feature based on a density of each stationary crowd group.
The computer program product of claim 11, wherein the second order polynomial regression model is previously obtained by learning.