CN114287023A

CN114287023A - Multi-sensor learning system for traffic prediction

Info

Publication number: CN114287023A
Application number: CN201980099757.1A
Authority: CN
Inventors: 克里斯蒂安·阿克塞尼; 拉杜·都铎兰; 斯蒂法诺·波托利; 穆罕默德·啊·哈吉·哈桑; 戈茨·布拉舍
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2019-09-25
Filing date: 2019-09-25
Publication date: 2022-04-05
Anticipated expiration: 2039-09-25
Also published as: WO2021058100A1; CN114287023B

Abstract

A road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and second and third sensors for acquiring data in a second data domain, the first and second sensors being located at a first location and the third sensor being located at a second location spatially remote from the first location, wherein the system is for: receiving data from the first sensor, the second sensor, and the third sensor; processing data received from the third sensor in accordance with data received from the first and second sensors to estimate a flow of traffic at the second location.

Description

Multi-sensor learning system for traffic prediction

Technical Field

The present invention relates to traffic control systems, and more particularly to a system for optimizing traffic predictions by utilizing a multi-sensor correlation learning and fusion engine that applies statistics and machine learning to big data processing.

Background

Traffic congestion presents a serious challenge to urban infrastructure and also affects people's socioeconomic life because time is wasted while waiting for traffic.

Although there are many features that describe traffic flow models, we can follow a general assumption that traffic flow data is a time series, which is a time indexed sequence of values (i.e., tuples containing various types of data: number of vehicles, speed of vehicles, etc.). Traffic data may be collected chronologically from various sources such as cameras or street induction loop sensors, pollution sensors, noise sensors, and weather sensors. These data are continuously generated over time by sensors that measure traffic indicators. These sensors are typically spatially distributed in the urban infrastructure.

Various traffic modeling and control methods have been developed, such as macroscopic or microscopic models, filtering models, and other combinatorial models. Such studies lack practical deployment for reasons including low prediction accuracy, inability to deal with correlations, strict requirements on data, lack of expressiveness or lack of understanding of internal mechanisms, and intolerable prediction time costs.

A key reason for the failure of existing traffic prediction models in practical applications is that they fail to take full advantage of the available multi-sensor sources. All in oneCameras and inductive loops that directly measure the number of vehicles at a particular location are expensive, intrusive, difficult to maintain, and require algorithms to process the data, which increases the cost of such a solution, making it impractical to implement on a large scale. Existing models also fail to take full advantage of the unique information provided by the traffic network, including dynamic spatial factors, i.e., the topology and inherent time flow of the vehicle and its correlation, which can be sensed by a plurality of inexpensive sensors available in the urban mass, such as pollution sensors (e.g., NO)₂NO, PM10), noise sensors, weather sensors (e.g., humidity, precipitation duration), online data, and cellular line data. These are all available and alternative data sources that indirectly characterize traffic flow information.

Typical statistical and machine learning models for time series prediction, such as autoregressive moving average series (AR, MA, ARMA, ARIMA), bayesian inference, and regression trees, can only model and predict one-dimensional time series. Such predictive models assume that the correlation in the data can be adequately described by a global time-fixed parameter. Furthermore, they cannot be extended to multivariate spatial correlation predictions per se, which makes them unsuitable for cases where the correlation between data is dynamic and heterogeneous, such as road traffic data.

In the field of machine learning-based time series modeling and prediction, there are various approaches that attempt to remedy the limitations of typical statistical methods and supplement new techniques to improve performance and flexibility. The prior art has generally focused on off-line procedures that employ bayesian integration, support vector regression, non-linear least squares, combinatorial methods, and expert systems.

The methods described in the following documents assume that the diversity and accuracy of the involved models are the most important factors to consider when selecting models, and a new method of time series prediction based on neural networks and meta-features was studied: fonseca et al, "automatic model selection in time series prediction combinatorial approach", IEEE Collection, Vol.14, No. 8, pages 3811 to 3819, 2016.8 months. The method automatically adjusts the desired balance between diversity and accuracy in selecting predictors and provides good results over highly non-linear time series. However, offline unsupervised training and complex model updates make this approach difficult to implement in real-time scenarios.

Another approach described in the following document assumes that knowledge of the complexity of the time series enables the design of an adaptive predictive decision support system to actively support the accuracy of the predicted behavior and outcome: adya et al, "develop and validate rule-based time series complexity scoring techniques to support the design of adaptive predictive Decision Support Systems (DSS)", volume 83, 2016. The system is based on a rule-based complexity scoring technique that generates a time series of complexity scores using twelve rules that depend on fourteen features of the sequence. Although embedding expert system rules in modeling is an interesting approach, the decision-making system requires a large number of rules and features in selecting a model, making it difficult to meet the low resource and low time budget requirements required for real-time processing. Furthermore, the predictive Decision Support System (FDSS) considers complex features such as discontinuity, cardinal trend direction, horizontal discontinuity and domain knowledge, which may indeed increase the adaptability but also increase the complexity and computation time. These indices are not calculated incrementally, but are calculated off-line, so the calculation time depends on the length of the time series, but there is no real-time prediction requirement.

In the field of traffic prediction, in order to meet the requirement of traffic jam early warning, a real-time traffic flow state identification and prediction method based on a big data driving theory is developed through a plurality of researches. The traffic big data has the characteristics of time correlation, space correlation, historical correlation, polymorphism and the like.

The method described in the following documents quantifies traffic flow conditions by using a traffic cluster model based on fuzzy c-means (SAGA-FCM) of simulated annealing genetic algorithm, which is the basis of traffic flow condition identification: hua-pu Lu, Zhi-yuan Sun and Wen-conn Qu, "real-time traffic flow state identification and prediction based on big data drive", dynamics of nature and social discreteness, volume 2015, article number 284906, page 11, 2015. In consideration of simple calculation and high prediction accuracy, a regional traffic flow correlation analysis double-layer optimization model is established to predict traffic flow parameters based on time-space-historical correlation. Although this model has flexibility, it makes many assumptions about the spatio-temporal parameterization of the model, and only achieves a 10% gain in traffic congestion mitigation.

Another approach described in the following documents uses a Stochastic Cell Transmission Model (SCTM) for probabilistic traffic state estimation, which incorporates covariance structures calibrated from spatial correlation analysis: t.l.pan, a.sumalee, r.x.zhong and n.indra-payoong, "short-term traffic status prediction based on spatio-temporal correlation", IEEE intelligent traffic systems, 14 rd 3 rd, pages 1242 to 1254, 9 months in 2013. Despite the large computational load behind the model, the overall Maximum Absolute Percentage Error (MAPE) for all predictions for this system is about 16.2%, which number, unfortunately, is only obtained in static empirical studies (not in real scenes).

The following studies in the literature have employed a more real-time oriented approach to investigate the potential of Probabilistic Hypothesis Density (PHD) filters in real-time traffic state estimation: m. canaud, l.michaylova, j.sau et al (2013), "probabilistic hypothesis density filtering for real-time traffic state estimation and prediction", (NHM, 8 (3)), pages 825 to 842. The method uses a Cell Transmission Model (CTM) coupled with a PHD filter, taking into account the uncertainty of the measurement source and showing that this can provide high accuracy in terms of traffic setup and real-time computation costs. Although this model is very attractive, it is only used in highway environments without a large number of lanes and intersections, and the complexity is often increased in real traffic environments.

US 2002/0067292 a1 describes the use of a sensor system for environmental sensing for intelligent scene interpretation. To determine the position of the motor vehicle relative to the roadway, data from a digital road map coupled with a navigation system is fused with data provided by a distance-resolving sensor. In this context, the characteristics of the received signals of the distance-resolving sensors and the distance-related changes are evaluated to determine the distance to the road edge.

There is a need to develop a traffic prediction system and method that addresses these problems.

Disclosure of Invention

According to a first aspect, there is provided a road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and second and third sensors for acquiring data in a second data domain, the first and second sensors being located at a first location and the third sensor being located at a second location spatially remote from the first location, wherein the system is for: receiving data from the first sensor, the second sensor, and the third sensor; processing data received from the third sensor in accordance with data received from the first and second sensors to estimate a flow of traffic at the second location.

The data received from each of the sensors may include a time series of values. This may allow the sensor to continuously generate data over time to measure traffic metrics.

The first sensor may be a camera and the first data field may be one or more images of a vehicle at the location. This enables learning of relationships between traffic flow and other types of sensor data.

The second and third sensors may be for acquiring data relating to the level of the environmental attribute at the first and second locations, respectively. This enables learning of relationships between traffic flow and data that may be collected from ubiquitous and/or less expensive sensors.

Each of the second sensor and the third sensor may include one of a weather sensor, a pollution sensor, a noise sensor, a CO sensor, and an inductive loop. These are convenient sensor implementations that can learn the relationship between sensor data and traffic flow.

The third sensor may be used to acquire data relating to the same environmental attribute as the second sensor. This enables the relationship learned at one location to be communicated to a second location so that data from the third sensor can be used to infer the flow of traffic at the second location.

The system may be configured to perform the processing by implementing the learned artificial intelligence model. The artificial intelligence model may be a neural network. This may be a convenient implementation.

The system may be used to learn a mapping from the second data domain to the first data domain. The learned mapping may be applied to other locations to predict traffic flow for locations where the number of vehicles cannot be directly measured.

The system may be configured to iteratively update parameters of the model over time based on data received from the first sensor and the second sensor. This can improve the accuracy of the learned relationship.

The system may also be configured to process data received from the third sensor based on data received from at least one other sensor to estimate the flow of traffic at the second location. This may allow more types of sensors to be used to improve the robustness of the prediction.

The system may also be configured to: receiving data from a fourth sensor at the first location and a fifth sensor at the second location, the fourth sensor and the fifth sensor for acquiring data in a third data domain; processing data received from the third sensor and the fifth sensor in accordance with data received from the first sensor, the second sensor and the fourth sensor to estimate a flow of traffic at the second location.

The first location and the second location may be a first traffic intersection and a second traffic intersection, respectively. The system may also be configured to generate time plans for respective sets of traffic lights at the first intersection and the second intersection. Thus, the system may be implemented to manage traffic in an urban environment.

According to a second aspect, there is provided a method for implementation at a road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and second and third sensors for acquiring data in a second data domain, the first and second sensors being located at a first location and the third sensor being located at a second location spatially remote from the first location, the method comprising: receiving data from the first sensor, the second sensor, and the third sensor; processing data received from the third sensor in accordance with data received from the first and second sensors to estimate a flow of traffic at the second location.

The processing may include implementing the learned artificial intelligence model. The artificial intelligence model may be a neural network. This may be a convenient implementation.

A computer program may also be provided which, when executed by a computer, causes the computer to perform the above-described method. The computer program may be stored on a non-transitory computer readable storage medium.

Drawings

The invention will now be described by way of example with reference to the accompanying drawings. Wherein:

FIG. 1 shows a graphical depiction of a traffic control system and typical sensor data for different locations;

FIG. 2 shows an overview of a general multi-sensor processing arrangement that may be used for traffic prediction;

FIG. 3 shows an example of the use of sensors located at two spatially distant traffic intersections;

FIG. 4 illustrates an overview of a multi-sensor learning traffic prediction architecture;

FIG. 5 illustrates modules of a multi-sensor learning system;

FIGS. 6(a) and 6(b) illustrate time series correlation learning in a system;

7(a) to 7(d) show another example of relevance learning in a system;

FIGS. 8(a) and 8(b) illustrate the scalability features of the system;

fig. 9(a) to 9(c) show reasoning and prediction in the system. FIG. 9(a) shows the input, FIG. 9(b) shows the learned relationship, and FIG. 9(c) shows the decoded relationship;

FIG. 10 illustrates the runtime functionality of the system;

FIG. 11 illustrates traffic predictions using models learned at one location to predict traffic flow at other locations that are spatially distant;

fig. 12(a) and 12(b) show the learned sensor relationship. FIG. 12(a) shows the NO learned₂The relationship between level and humidity; FIG. 12(b) shows the NO learned₂The relationship between level and number of vehicles;

FIG. 13 illustrates inferring data from missing sensors;

fig. 14 illustrates an example of a method for implementation at a road traffic flow prediction system.

Detailed Description

The present invention relates to a processing system for collecting traffic data from different types of sensors located at intersections, modeling the data by extracting correlations between available sensors, and making predictions of future traffic flow. Such a processing system may be used to control a sequence of traffic lights to improve the flow of road traffic. Traffic flow may be defined as the number of vehicles passing a particular point during a particular time period (e.g., number of vehicles per minute), or the number of vehicles passing through an intersection per traffic light cycle (i.e., the number of vehicles passing during a time period when the traffic light at the intersection is green). Thus, the time resolution for evaluating the number of vehicles may be a unit time period, as well as the duration of a traffic light cycle.

As shown in fig. 1, there may be different types of sensors at different road traffic locations. For example, at site 1 (generally shown as 101), the NO level, NO, is included₂Variables including level, particle count and humidity are measured by respective sensors or from sensors used to collect data that can be processed to determine these quantities. At site 2 (generally indicated as 102), including NO level, CO level, NO₂Variables including level, particle count and humidity are measured by respective sensors or from sensors used to collect data that can be processed to determine these quantities. Neither site 1 nor site 2 has a sensor, such as a camera or inductive loop, that can directly measure the flow of traffic (i.e., the number of vehicles) at that site. There is a camera at site 3 (shown generally at 103) through which the number of vehicles can be measured. In addition, other types of sensors are provided at site 3 to measure variables, including NO₂、O₃、NO_xNO, humidity level and precipitation duration. Thus, the traffic flow at location 3 can be measured directly by the camera. However, at the site 1 and the site 2, there is no camera or induction loop capable of measuring the number of vehicles, and thus the traffic flow cannot be directly measured.

Fig. 2 shows another typical scenario illustrating an urban traffic environment (i.e., an intersection of two or more roads). At the intersection 201 of the roads S1-S4, a camera 202, a noise sensor 203, a humidity sensor 204, and CO are provided₂A sensor 205. Given the measured traffic flow from the cameras (i.e., the number of vehicles passing through each lane, as generally indicated at 206), the system predicts the future traffic flow (as indicated at 207) and the control times of the traffic lights (as indicated at 208) to maximize the traffic flow.

As described above, traffic flow prediction using only camera data at each location or intersection is not only costly, but also requires additional infrastructure, such as dedicated image processing software. Thus, it is possible to provideLarge scale installations are not cost effective. In addition, privacy concerns are also a barrier. However, many other sensors are available (e.g., noise sensor 203, CO)₂ Sensor 204 and humidity sensor 205) are also ubiquitous.

The present invention utilizes data from these other types of sensors and utilizes all available data sources describing traffic conditions to improve traffic flow predictions.

As shown in fig. 3, the urban area 300 includes: a series of roads along which vehicles may travel in at least one direction; and intersections or crossroads between intersecting roads. The first sensor 301 is located at a first location at a traffic intersection 302. The first sensor is used to acquire data in a first data field. The first data field provides a measure of the number of vehicles at the intersection from which the traffic flow is determined. Preferably, the first sensor 301 is a camera for collecting images of the number of vehicles at the intersection, from which the traffic flow can be determined. A sensor 303 is also located at the first location of the intersection 302 for collecting data related to environmental attributes. In this example, the sensor 303 is a CO sensor. The second CO sensor 305 is located at a second location spatially distant from the intersection 304 of the first intersection 302.

As will be described in more detail below, the system learns the relationship between the data collected by the camera 301 and the CO sensor 303. The system then applies the learned relationship to the data collected from the CO sensor 305 at the second location to infer the number of vehicles at that road segment location and, thus, the traffic flow.

The system may also learn a relationship between the camera and at least one other sensor at the first location and use the learned relationship to determine a traffic flow at the second location. For example, in fig. 3,

noise sensors

306 and 307 are also located at the first and second locations, respectively. The system may receive data from the noise sensor 307 and predict traffic flow at the second location (i.e., at the intersection 304) based on the learned relationship between the camera 301 and the noise sensor 306. The system may also receive data from a plurality of sensors at the second location and use the plurality of relationships learned from the first location to form an output of the traffic flow at the second location.

The determined traffic flow may be used to predict future traffic flows (e.g., several hours ahead of the current time) and generate time plans for corresponding sets of traffic lights at the first intersection and the second intersection, as shown at 308 and 309, respectively.

Thus, the system aggregates multiple sensor data sources available at different locations to describe and predict traffic scenarios in urban environments. Such systems are components that interact with the traffic control system at each intersection. Thus, the system is located at the intersection between traffic estimation and modeling and the traffic light sequence control component.

Fig. 4 presents the overall architecture and data flow of the proposed system. The architecture shown at 400 in fig. 4 includes several sections that cooperate to process incoming sensor time series data 401.

The first component shown at 402 is a road traffic flow time series representation and modeling module. As an inherently complex process, road traffic flow can be modeled using a temporal model that assumes that the data describes local variations of global phenomena in the form of a spatially distributed time series. This module uses a 1D Self-Organizing Map (SOM), which is a neural network that is capable of encoding sensor time series in a distributed activity pattern on a lattice of processing units (i.e., neurons). Each processing element has a preference value for which it will issue an output when fed to the network. This mechanism enables the system to represent adjacent values of the SOM that are close to each other in input sensor space. Using this mechanism, the system can extract the data distribution from the time series by modulating the gaussian function to encode the data distribution (i.e., narrow gaussian represents low distribution and wide gaussian represents high distribution). The representation and modeling system converges quickly, thereby restoring the distribution of sensor data for subsequent correlation learning. Therefore, the system belongs to a lightweight system and has high running speed.

The second component shown at 403 is a multi-sensor relevance learning system for road traffic prediction. The system uses distributed encoding of the sensor time series determined by module 402 to extract temporal co-activation between different types of sensors. This is achieved by a relevance learning mechanism, such as hebrew learning. This allows the processing unit or neuron in each input sensor SOM to strengthen the connection between the sensors according to the co-activation (i.e. neurons are active at the same time for a particular sample from the sensor time series). For each new sensor data sample, learning quickly converges to a representation representing the relationship between the input sensors. Conveniently, the representation is a connection matrix, similar to an adjacency matrix. The intrinsic mechanism is fast and resource efficient in terms of convergence time and memory allocation.

The third component shown at 404 is a fault tolerant inference and prediction system for road traffic. The learned relationships between the sensors are then extracted from the matrix representation, while taking into account the neuron positions in the input coding system (i.e., SOM). The process includes a simple optimization method for finding the value closest to the correct value from the mathematical representation of the relationship in the learned matrix representation. This is achieved by simple mathematical calculations (e.g. sum, product, square root) that support overall simple and fast operations. The decoded functional mathematical relationship is used to infer missing sensor data (e.g., in the absence of a camera at an intersection) or to predict the correct value in the event of a sensor failure, as shown at 405.

The overall architecture is modular and provides an adaptive system suitable for modeling and predicting highly heterogeneous time series and non-stationary processes. The system can exploit correlations between different non-stationary, deterministic time series that describe multi-sensor large-scale (network) phenomena. The system can achieve improved time series prediction performance in highly non-stationary problems (such as traffic flow prediction) using a distributed representation mechanism capable of capturing sensor time series shape and time distribution, as well as a time-dependent learning mechanism for fusing data and a powerful inference mechanism based on simple operations.

Thus, the system provides a pipelined processing mechanism that converts time series data acquired from sensors into a representation that can extract potential correlations between sensors in the representation and modeling module, learn and fuse correlations between sensors into a multi-sensor correlation learning module, and infer traffic flow (e.g., for locations without cameras, or locations where cameras are malfunctioning) or improve traffic flow estimation (when cameras are present) in a fault tolerant inference and prediction module.

The operation of these elements will now be described in further detail with reference to fig. 5.

As shown at 501, the presentation and modeling module processes the time series data from all sensors in a distributed encoding process, as shown at 500, to extract statistics and data distributions from the time series.

Modeling of non-stationary and deterministic time series requires the use of specialized mechanisms to interpret their characteristics. Furthermore, processing multiple such correlated time series requires an appropriate layer to describe its covariance. Module 501 encodes sensor time series 500 using a distributed network of processing units. The sensor time series is encoded in a distributed representation using Self-Organizing Maps (SOM). The machine learning algorithm is responsible for extracting statistics of incoming data and encoding sensor samples in a distributed activity pattern, as shown in fig. 6(a) and 6 (b). This pattern of activity is generated such that the neuron closest to the input sample (in terms of its preferred value) will be strongly activated. The activation decays as a function of the distance between the input and the preferred value. Using the SOM distributed representation, the model learns the boundaries of the input data such that after relaxation, the SOM provides a topologically preserved representation of the input space.

The multi-sensor correlation learning module 502 extracts sensor correlations from sensor data using machine learning. In a preferred implementation, the machine learning algorithm used is hebrew learning. The algorithm learns the correlations between the data collected by the sensors and stores them in an efficient, cost-effective and interpretable representation. The method is based on a lightweight neural network-based learning system, and does not need back propagation training. It can represent and extract underlying statistics of sensor data in the same learning layer. This allows the system to learn the data representation from the extracted statistics and allocate resources accordingly.

The system uses a time co-activation mode to encode the correlation between different sensors, as shown by the two sensors in fig. 6(a) and 6 (b). In fig. 6(a), SOM of time-series data from

sensors

1 and 2 is extracted, and a machine learning algorithm is used to learn the relationship between the two data sets. In fig. 6(b), a cross-modal weight matrix is generated to represent the correlation between the two sensor data sets.

This approach has several advantages. For example, it may provide an interpretable structure ("black box-less design") and an easily interpretable output. The method can handle bi-modal, tri-modal or multi-modal extensions using the same representation and learning layers. Furthermore, in contrast to many existing approaches, in the system described herein, there is no need to explicitly encode sensor association and sensor fusion rules.

To illustrate the potential of the system and its underlying mechanisms, fig. 7(a) to 7(d) show the learning of a non-linear relationship (in this example, a third-order power-law relationship) between two sensor data sets. Fig. 7(a) shows input data and input data distribution similar to the third-order power law relationship. Fig. 7(b) shows an internal model architecture for determining cross-modal weight matrices. Fig. 7(c) shows the calculation stage. Upon receipt of sensor data, the system detects whether the data is from a new sensor that has not previously learned relationships. If the sensor is not a new sensor, the mechanism enters an inference phase to predict traffic flow using sensor data. If the data is from a new sensor, the sensor data is used in model training to determine a relationship between the new sensor data and the received data from the known sensor type. Fig. 7(d) shows the correlation between the learned data from

sensors

1 and 2.

The system is capable of handling situations where there is any non-linear relationship between data from different sensor types and has a high degree of accuracyDegree scalability, ability to handle multiple sensors. FIGS. 8(a) and 8(b) illustrate a sensor s including a structure having tree-shaped correlations₁、s₂And s₃Of the three-dimensional system of (1). FIG. 8(a) shows a comparison between the models m₁、m₂And m₃And decoded learned representations, while figure 8(b) shows learned sensor relationships encoded in neural network weights.

After learning the underlying sensor correlations, the system is able to decode an efficient and interpretable learning representation for prediction or inference in a fault tolerant interference and prediction module, shown at 503 in FIG. 5. More specifically, after the learning process, the network stores a stable representation of the relationship between the two sensor inputs considered during training. By considering only one input sensor source (although multiple may be used), the network can infer a corresponding number of missing sources using the learned co-activation pattern. This module uses a decoder that computes only one condition to fine tune the preferred value of the most active (winning) neuron to a more accurate estimate. These calculations are simple, such as summation and multiplication operations.

Fig. 9(a) to 9(c) show examples of the reasoning capabilities of the system. Fig. 9(a) shows the input signals and the relations, fig. 9(b) shows the learned relations, and fig. 9(c) shows the decoded relations for different input data sets.

The above-described system components cooperate in the operation of the system and correspond to the functional blocks shown in fig. 10. The system runtime sequence and operations performed by the functional modules (blocks) 1001 to 1005 are as follows. In block 1001, a distributed representation is computed and input sensor time series statistics describing traffic are extracted, as shown generally at 1000. In module 1002, data statistics of sensor data 1000 are combined and the interpretable representation is used to learn underlying sensor relationships. In block 1003, the sensor correlations from the interpretable code are decoded. Module 1004 performs predictions for the sensors based on the learned correlations and other available sensors. Module 1005 performs inference of missing or malfunctioning sensors based on the learned relationships and data from other available sensors. The output of module 1005, shown at 1006, is a time series prediction (dotted line) compared to the true value (broken line).

The system can efficiently learn sensor correlations in various traffic scenarios. In the exemplary traffic scenarios shown in fig. 11-13, environmental parameters (e.g., NO, O) are learned at site 3₃、NO₂、NO_x) Correlation between weather (e.g., humidity, precipitation) and traffic flow (number of vehicles determined by camera), generally shown as 1101 in fig. 11.

After learning the correlations, the learned relationships can be communicated to different locations to infer traffic flow in areas where traffic sensors that can directly measure vehicle quantities are not installed, but where other types of sensors exist. For example, at

sites

1 and 2, generally 1102 and 1103 respectively, there are NO direct traffic flow sensors, but there are NO's at these locations₂And a humidity sensor. The system learns the sensor relationships between different sensors at site 3 and can use only NO collected from site 2₂And humidity sensor data to predict vehicle counts at site 2, as shown in fig. 13. Thus, the system learns NO at site 3₂And the pair-wise correlation between vehicle count (via camera) and humidity. Learned NO₂And humidity and NO₂And the vehicle count are shown in fig. 12(a) and 12(b), respectively. After learning at site 3, the correlation between the two different sensor types can be used to infer traffic flow at locations such as site 2 that are not equipped with traffic sensors such as cameras or inductive loops that can directly measure the number of vehicles at the intersection. Since there are other types of sensors at this location, the number of vehicles can be inferred using the correlations learned from site 3.

The output of the system may be applied to a traffic control unit to update the traffic lights at the intersections in the area to maximize traffic flow. The unit may operate using any available traffic sensor data. The system is supported by flexible instrumentation, ensuring updates with low latency, high incoming event rates, and fixed resource budgets. Further, the system can be deployed at any type of location or intersection without prior training and regardless of road or intersection layout, size, and available sensors. This provides a significant advantage in reducing deployment costs, particularly since learned underlying dependencies can be passed on to new infrastructure layouts equipped with different sensors.

Optimization of road traffic may be performed continuously and in conjunction with historical data and incoming streams of current traffic data (e.g., number of vehicles, vehicle speed, occupancy at traffic lights, noise for particular road segments, pollution values recorded for road segments). Thus, the system is able to continuously learn traffic sensor correlations, fuse sensor data describing road traffic and adapt to changes to improve road traffic predictions. In addition, in traffic flow prediction and control, the control unit can estimate and adapt to changes in data distribution and provide accurate predictions and intelligent control actions, such as controlling traffic signal green time. Thus, the system can predict traffic flow with available sensor data with a fixed resource budget.

In addition to predicting traffic flow at locations where no cameras or inductive loops are present, the system may alternatively or additionally be used to detect when sensors fail. For example, when a camera or inductive loop fails, the system can use data collected from other sensors to predict the correct parameter data based on the learned relationships. Assuming that correlations between other sensor types and the faulty sensor type have been previously learned, the system can provide data to replace data from the faulty sensor from data from another sensor. Thus, the system is able to infer the amount of missing sensors. This method can be used if the amount of faulty sensors needs to be corrected. For example, when the contamination sensor drifts due to humidity.

The system can be implemented using a large number of available sensors. Examples of sensors that may be used include, but are not limited to: camera, pollution sensor, noise sensor, CO sensor, NO₂Sensor, NO sensor, O₃Sensor, NO₂Sensor, NO_xSensors, weather sensor (e.g., humidity, precipitation) sensing loops, mobility data, GSM user cell switches covering geospatial motion parameters, high intensity sound range installed on streets, and particle count sensors for a large range of exhaust gases.

FIG. 14 summarizes a method for implementation in a road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and second and third sensors for acquiring data in a second data domain, the first and second sensors being located at a first location and the third sensor being located at a second location spatially remote from the first location. In step 1401, the method includes receiving data from the first sensor, the second sensor, and the third sensor. In step 1402, the method includes processing data received from the third sensor in accordance with data received from the first sensor and the second sensor to estimate a flow of traffic at the second location.

The systems and methods described herein may help minimize the overall cost of a traffic control system by inferring traffic flow from multiple sources. This alleviates the need for expensive sensors and data sources (e.g., cameras and inductive loops) per location or intersection by inferring traffic flow from other, less expensive and readily available, related sensors, thus enabling the system to scale efficiently for large urban scenarios.

The system is particularly suited for traffic prediction, which requires modeling, prediction, and rapid adaptation to the sensor time series. As shown in the modularized structure, the system adopts an efficient time series representation and modeling method, can extract data distribution, and learns the correlation of multiple sensors in a unified computing unit by utilizing time co-activation to carry out fault-tolerant traffic prediction.

Regardless of the deployment scenario, the data processing units are able to model the time series distributively according to their temporal structure using their automatically extracted representation of the data distribution. The computing unit can accommodate any type of sensor time series and any number of available sensors describing road traffic.

The system also supports encoding various sensor time series into a distributed representation that can capture underlying statistics and data distribution models, and provides a universal approach to time series modeling that can find a best-fit model describing the underlying structure of the sensor data.

The system includes a generic architecture that can be run as part of a traffic control system to obtain a predictive model for each location. The system can be deployed for a variety of scenarios that require flexibility and scalability, independent of road geometry, size and configuration, and available sensors.

The system may include a processor and a non-volatile memory. The system may include a plurality of processors and a plurality of memories. The memory may store data executable by the processor. The processor may be configured to operate in accordance with a computer program stored in a non-transitory form on a machine-readable storage medium. The computer program may store instructions for causing the processor to perform its methods in the manner described herein. These components may be implemented in physical hardware, or may be deployed on various edge or cloud devices.

When data enters the system, the time span of calculation processing is limited, and only simple operation can be executed due to the limitation of resource allocation and execution time. The proposed system proposes a road traffic prediction calculation unit that utilizes the time correlation between different traffic sensor data describing traffic conditions, thereby overcoming the prior art methods of large resource consumption, high calculation cost and complexity, such as complex analytic flow models based on differential equations, numerical methods and empirical methods.

The road traffic flow time series representation and modeling and calculation unit is capable of modeling the sensor time series using an efficient distributed model that supports data representation and statistical modeling of lightweight learning algorithms. The system provides a multi-sensor correlation learning system for road traffic prediction that learns the temporal correlation pattern between pairs of sensor data describing traffic conditions, encoding underlying functional mathematical relationships between them, given the pairs, without any a priori information about underlying statistics and correlations. The decoding mechanism decodes the learned correlations between sensors and encodes them in an efficient data representation as interpretable mathematical functional relationships.

Thus, the proposed system provides a flexible infrastructure to analyze arbitrary sensor correlations and extract understandable sensor data correlations for traffic prediction. The lightweight learning system represents and extracts underlying statistics of sensor data in an efficient data representation. In addition, the system has an interpretable structure, no "black box design," and the interpretable output is applicable to dual-sensor, tri-sensor, multi-sensor scenarios. The proposed system combines the learning capabilities of neural networks and SOMs in terms of efficient representation of time series, as well as efficient relevance learning and multi-sensor fusion mechanisms. This combination allows for real-time learning and relearning. Because the system has learning capabilities, it does not require explicit coding of sensor associations and sensor models. The system can transmit the learned correlation information to other positions or intersections, and can predict the road traffic flow through other available sensor data without reconfiguration, thereby maximally reducing the related cost of equipping all intersections with expensive sensors.

The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.

Claims

1. A road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and second and third sensors for acquiring data in a second data domain, the first and second sensors being located at a first location and the third sensor being located at a second location spatially remote from the first location, the system being adapted to:

receiving data from the first sensor, the second sensor, and the third sensor;

processing data received from the third sensor in accordance with data received from the first and second sensors to estimate a flow of traffic at the second location.

2. The system of claim 1, wherein the data received from each of the sensors comprises a time series of values.

3. The system of claim 1 or claim 2, wherein the first sensor is a camera and the first data field is one or more images of a vehicle at the location.

4. The system of any preceding claim, wherein the second and third sensors are for acquiring data relating to the level of an environmental attribute at the first and second locations respectively.

5. The system of claim 4, wherein each of the second sensor and the third sensor comprises one of a weather sensor, a pollution sensor, a noise sensor, a CO sensor, and an inductive loop.

6. A system according to claim 4 or claim 5, wherein the third sensor is arranged to acquire data relating to the same environmental attribute as the second sensor.

7. The system of any preceding claim, wherein the system is configured to perform the processing by implementing a learned artificial intelligence model.

8. The system of claim 7, wherein the artificial intelligence model is a neural network.

9. The system of claim 7 or claim 8, wherein the system is configured to learn a mapping from the second data domain to the first data domain.

10. The system of any one of claims 7 to 9, wherein the system is configured to iteratively update parameters of the model over time based on data received from the first sensor and the second sensor.

11. The system of any preceding claim, wherein the system is further configured to process data received from the third sensor in dependence on data received from at least one other sensor to estimate the flow of traffic at the second location.

12. The system of any of the preceding claims, further configured to:

receiving data from a fourth sensor at the first location and a fifth sensor at the second location, the fourth sensor and the fifth sensor for acquiring data in a third data domain;

processing data received from the third sensor and the fifth sensor in accordance with data received from the first sensor, the second sensor and the fourth sensor to estimate a flow of traffic at the second location.

13. The system of any of the preceding claims, wherein the first location and the second location are a first traffic intersection and a second traffic intersection, respectively.

14. The system of claim 13, further configured to generate time plans for respective sets of traffic lights at the first intersection and the second intersection.

15. A method for implementation at a roadway traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and a second sensor and a third sensor for acquiring data in a second data domain, the first sensor and the second sensor being located at a first location and the third sensor being located at a second location spatially remote from the first location, the method comprising:

receiving data from the first sensor, the second sensor, and the third sensor;

16. The method of claim 15, wherein the processing comprises implementing the learned artificial intelligence model.