CN114287023B - Multi-sensor learning system for traffic prediction - Google Patents

Abstract

A road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and a second sensor and a third sensor for acquiring data in a second data domain, the first sensor and the second sensor being located at a first location and the third sensor being located at a second location spatially distant from the first location, wherein the system is for: receiving data from the first sensor, the second sensor, and the third sensor; data received from the third sensor is processed in accordance with data received from the first sensor and the second sensor to estimate traffic flow at the second location.

Description

Multi-sensor learning system for traffic prediction

Technical Field

The present invention relates to traffic control systems, and more particularly to a system for optimizing traffic prediction by utilizing a multi-sensor correlation learning and fusion engine that applies statistics and machine learning to big data processing.

Background

Traffic congestion presents a serious challenge to urban infrastructure and also affects people's socioeconomic life because time is wasted waiting for traffic.

Although there are a variety of features describing traffic flow models, we can follow a general assumption that traffic flow data is a time series, which is a time-indexed series of values (i.e., tuples containing various types of data: number of vehicles, speed of vehicles, etc.). Traffic data may be collected from various sources (e.g., cameras or street induction loop sensors, pollution sensors, noise sensors, and weather sensors) in a time-series manner. These data are continuously generated by sensors measuring traffic metrics over time. These sensors are typically spatially distributed in the urban infrastructure.

Various traffic modeling and control methods have been developed, such as macroscopic or microscopic models, filtering models, and other combination models. Such studies lack practical deployment due to low prediction accuracy, inability to handle dependencies, stringent data requirements, lack of expressivity or lack of understanding of internal mechanisms, and intolerable prediction time costs.

The key reason that existing traffic predictive models fail in practical applications is that they fail to adequately addressUtilizing available multi-sensor sources. At the same time, cameras and inductive loops that directly measure the number of vehicles in a particular location are expensive, invasive, difficult to maintain, and require algorithms to process the data, which increases the cost of such solutions, making them impractical to implement on a large scale. Existing models also fail to take full advantage of unique information provided by traffic networks, including dynamic space factors, i.e., topology and inherent time flows of vehicles and their correlation, which can be sensed by multiple inexpensive sensors available in urban clusters, such as pollution sensors (e.g., NO ₂ NO, PM 10), noise sensors, weather sensors (e.g., humidity, duration of precipitation), on-line data, and cellular line data. These are available and alternative data sources that indirectly characterize traffic flow information.

Typical statistical and machine learning models for time series prediction, such as auto-regressive moving average series (AR, MA, ARMA, ARIMA), bayesian reasoning and regression trees, can only model and predict one-dimensional time series. Such predictive models assume that the correlation in the data can be fully described by a global time-fixed parameter. Furthermore, they cannot be extended by nature to multivariate spatially correlated predictions, which makes them unsuitable for situations where correlations between data are dynamic and heterogeneous, such as road traffic data.

In the field of machine learning based time series modeling and prediction, there are various approaches that attempt to remedy the limitations of typical statistical approaches and supplement new techniques to improve performance and flexibility. The prior art is generally focused on offline procedures employing bayesian integration, support vector regression, nonlinear least squares, combinatorial methods, and expert systems.

The method described in the following document assumes that the diversity and accuracy of the models involved are the most important factors to consider when selecting models, and explores a new approach to neural network and meta-feature based time series prediction: fonseca et al, "automatic model selection in time series predictive combination method", IEEE Magazine, volume 14, phase 8, pages 3811 to 3819, month 8 of 2016. This method automatically adjusts the desired balance between diversity and accuracy in selecting predictors and provides good results over highly non-linear time sequences. However, offline unsupervised training and complex model updates make this approach difficult to implement in real-time scenarios.

Another approach described in the following documents assumes that knowledge of time series complexity can enable the design of an adaptive predictive decision support system to actively support the accuracy of the predicted behavior and results: adya et al, "develop and validate rule-based time series complexity scoring techniques to support the design of adaptive predictive Decision Support Systems (DSS)", volume 83, 2016. The system is based on a rule-based complexity scoring technique that uses twelve rules that depend on fourteen features of the sequence to generate a complexity score for the time series. While embedding expert system rules in modeling is an interesting approach, the decision system requires a large number of rules and features when selecting the model, making it difficult to meet the low resource and low time budget requirements required for real-time processing. Furthermore, the predictive decision support system (Forecasting Decision Support System, FDSS) considers complex features such as discontinuities, base trend directions, horizontal discontinuities and domain knowledge, which does increase adaptability, but also increases complexity and computation time. These indices are not calculated incrementally, but off-line, so the calculation time depends on the length of the time series, but without any real-time prediction requirements.

In the field of traffic prediction, in order to meet the demand of traffic jam early warning, many researches develop a real-time traffic flow state recognition and prediction method based on big data driving theory. The traffic big data has the characteristics of time correlation, space correlation, historical correlation, polymorphism and the like.

The method described in the following document quantifies traffic flow states by using a traffic cluster model based on fuzzy c-means (simulated annealing genetic algorithm based fuzzy c-means, SAGA-FCM) of simulated annealing genetic algorithm, which is the basis of traffic flow state recognition: hua-pu Lu, zhi-yuan Sun and Wen-cong Qu, "real-time traffic flow status recognition and prediction based on big data driving", nature and social discrete dynamics, volume 2015, article No. 284906, page 11, 2015. In consideration of simple calculation and high prediction accuracy, a regional traffic flow correlation analysis double-layer optimization model is established to predict traffic flow parameters based on time-space-history correlation. Although the model has flexibility, it makes many assumptions on the spatio-temporal parameterization of the model, achieving only 10% gain in terms of traffic congestion mitigation.

Another approach described in the following documents uses a random cell transmission model (stochastic cell transmission model, SCTM) for probabilistic traffic state assessment, which incorporates covariance structures calibrated from spatial correlation analysis: T.L.Pan, A.Sumalee, R.X.Zhong and N.Intra-payong, "short-term traffic state prediction based on spatiotemporal correlation", "IEEE Intelligent transportation systems journal", volume 14, phase 3, pages 1242 to 1254, month 9 in 2013. Although the calculation behind the model is very computationally intensive, the overall maximum absolute percentage error (Maximum Absolute Percentage Error, MAPE) of all predictions for the system is about 16.2%, unfortunately this number is only obtained in static demonstration studies (rather than in real scenes).

The study in the following literature explores the potential of probability hypothesis density (Probability Hypothesis Density, PHD) filters in real-time traffic state estimation using a more real-time oriented approach: canaud, L.Mihaylova, J.Sau et al (2013), "probability hypothesis Density Filter for real-time traffic state estimation and prediction" (network and non-uniform Medium (NHM), 8 (3), pages 825 through 842). The method uses a cell transmission model (Cell Transmission Model, CTM) coupled with PHD filters, taking into account measurement source uncertainty and indicating that this can provide high accuracy in terms of traffic setup and real-time computational costs. Although this model is attractive, it is only used in highway environments where there are no large number of lanes and intersections, and the complexity is generally increased in real traffic environments.

US 2002/0067292 A1 describes the use of a sensor system for environmental sensing for intelligent scene interpretation. To determine the position of the motor vehicle relative to the roadway, data from a digital road map coupled to the navigation system is fused with data provided by a range-resolving sensor. In this context, the characteristics and distance-related variations of the received signal of the distance-resolving sensor are evaluated to determine the distance from the road edge.

There is a need to develop a traffic prediction system and method that addresses these issues.

Disclosure of Invention

According to a first aspect, there is provided a road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and a second sensor and a third sensor for acquiring data in a second data domain, the first sensor and the second sensor being located at a first location and the third sensor being located at a second location spatially distant from the first location, wherein the system is for: receiving data from the first sensor, the second sensor, and the third sensor; data received from the third sensor is processed in accordance with data received from the first sensor and the second sensor to estimate traffic flow at the second location.

The data received from each of the sensors may comprise a time series of values. This may allow the sensor to continuously generate data over time to measure traffic metrics.

The first sensor may be a camera and the first data field may be one or more images of the vehicle at the location. This enables learning of the relationship between traffic flow and other types of sensor data.

The second sensor and the third sensor may be used to acquire data relating to the level of an environmental attribute at the first location and the second location, respectively. This enables learning of the relationship between traffic flow and data that can be collected from ubiquitous and/or less expensive sensors.

Each of the second sensor and the third sensor may include one of a weather sensor, a pollution sensor, a noise sensor, a CO sensor, and an induction loop. These are convenient sensor implementations that can learn the relationship between sensor data and traffic flow.

The third sensor may be used to acquire data relating to the same environmental attribute as the second sensor. This enables the transfer of relationships learned at one location to a second location so that data from the third sensor can be used to infer traffic flow at the second location.

The system may be used to perform the process by implementing a learned artificial intelligence model. The artificial intelligence model may be a neural network. This may be a convenient implementation.

The system may be configured to learn a mapping from the second data domain to the first data domain. The learned map may be applied to other locations to predict traffic flow at locations where the number of vehicles cannot be measured directly.

The system may be configured to iteratively update parameters of the model over time based on data received from the first sensor and the second sensor. This can improve the accuracy of the learned relationship.

The system may also be configured to process data received from the third sensor based on data received from at least one other sensor to estimate traffic flow at the second location. This may allow more types of sensors to be used to improve the robustness of the prediction.

The system may also be used to: receiving data from a fourth sensor located at the first location and a fifth sensor located at the second location, the fourth sensor and the fifth sensor for acquiring data in a third data domain; data received from the third sensor and the fifth sensor is processed in accordance with data received from the first sensor, the second sensor and the fourth sensor to estimate traffic flow at the second location.

The first location and the second location may be a first traffic intersection and a second traffic intersection, respectively. The system may also be used to generate a time plan for respective sets of traffic signals at the first intersection and the second intersection. Thus, the system may be implemented to manage traffic in an urban environment.

According to a second aspect, there is provided a method for implementation at a road traffic flow prediction system for receiving data from a first sensor for acquiring traffic flow data in a first data domain and a second sensor and a third sensor for acquiring data in a second data domain, the first sensor and the second sensor being located at a first location and the third sensor being located at a second location spatially distant from the first location, the method comprising: receiving data from the first sensor, the second sensor, and the third sensor; data received from the third sensor is processed in accordance with data received from the first sensor and the second sensor to estimate traffic flow at the second location.

The processing may include implementing a learned artificial intelligence model. The artificial intelligence model may be a neural network. This may be a convenient implementation.

A computer program may also be provided which, when executed by a computer, causes the computer to perform the above method. The computer program may be stored on a non-transitory computer readable storage medium.

Drawings

The invention will now be described by way of example with reference to the accompanying drawings. Wherein:

FIG. 1 shows a graphical depiction of a traffic control system and typical sensor data for different locations;

FIG. 2 illustrates an overview of a generic multi-sensor processing arrangement that may be used for traffic prediction;

FIG. 3 illustrates an example of using sensors located at two spatially distant traffic intersections;

FIG. 4 illustrates an overview of a multi-sensor learning traffic prediction architecture;

FIG. 5 illustrates modules of a multi-sensor learning system;

FIGS. 6 (a) and 6 (b) illustrate time series correlation learning in a system;

fig. 7 (a) to 7 (d) show another example of correlation learning in a system;

FIGS. 8 (a) and 8 (b) illustrate the scalability features of the system;

fig. 9 (a) to 9 (c) show reasoning and prediction in the system. Fig. 9 (a) shows an input, fig. 9 (b) shows a learned relationship, and fig. 9 (c) shows a decoded relationship;

FIG. 10 illustrates runtime functionality of the system;

FIG. 11 illustrates traffic prediction using a model learned at one location to predict traffic flow at other locations spatially distant;

fig. 12 (a) and 12 (b) show the learned sensor relationship. FIG. 12 (a) shows the learned NO ₂ A relationship between level and humidity; FIG. 12 (b) shows the learned NO ₂ A relationship between level and number of vehicles;

FIG. 13 illustrates inferring data from missing sensors;

fig. 14 shows an example of a method for implementation at a road traffic flow prediction system.

Detailed Description

The present invention relates to a processing system for collecting traffic data from different types of sensors located at intersections, modeling the data by extracting correlations between available sensors, and predicting future traffic flows. Such a processing system may be used to control traffic signal sequences to improve road traffic flow. Traffic flow may be defined as the number of vehicles passing a particular point in a particular time period (e.g., the number of vehicles per minute), or the number of vehicles passing an intersection per cycle of traffic lights (i.e., the number of vehicles passing in a time period where the traffic lights at the intersection are green). Thus, the time resolution for evaluating the number of vehicles may be a unit time period or may be the duration of a traffic light cycle.

As shown in fig. 1, there may be different types of sensors at different road traffic locations. For example, at site 1 (shown generally as 101), including NO levels, NO ₂ The variables including level, particle count and humidity are measured by the respective sensors or from the sensors used to collect the data which can be processed to determine these quantities. At site 2 (shown generally at 102), including NO level, CO level, NO ₂ The variables including level, particle count and humidity are measured by the respective sensors or from the sensors used to collect the data which can be processed to determine these quantities. Neither spot 1 nor spot 2 has a sensor, such as a camera or an inductive loop, that can directly measure the traffic flow (i.e. the number of vehicles) at that spot. At location 3 (shown generally at 103) there is a camera by which the number of vehicles can be measured. In addition, other types of sensors are provided at site 3 to measure variables, including NO ₂ 、O ₃ 、NO _x NO, humidity level and duration of precipitation. Thus, the traffic flow at location 3 can be measured directly by the camera. However, at the sites 1 and 2, there is no camera or induction loop capable of measuring the number of vehicles, and thus traffic flow cannot be directly measured.

Fig. 2 shows another exemplary scenario (i.e., an intersection of two or more roads) illustrating an urban traffic environment. At the intersection 201 of the roads S1-S4, there are provided a camera 202, a noise sensor 203, a humidity sensor 204 and CO ₂ A sensor 205. Given the measured traffic flow from the cameras (i.e., the number of vehicles passing through each lane, generally indicated at 206), the system predicts future traffic flow (indicated at 207) and control time of the traffic lights (indicated at 208) to maximize traffic flow.

As described above, using camera data only at each location or intersection for traffic flow prediction is not only costly, but also requires additional infrastructure, for exampleSuch as dedicated image processing software. Thus, large-scale installation is not cost-effective. In addition, privacy concerns are also an obstacle. However, many other sensors are available (e.g., noise sensor 203, CO ₂ Sensor 204 and humidity sensor 205) are also common.

The present invention utilizes data from these other types of sensors and utilizes all available data sources describing traffic conditions to improve traffic flow predictions.

As shown in fig. 3, urban area 300 includes: a series of roads along which a vehicle can travel in at least one direction; and intersections or crossroads between intersecting roads. The first sensor 301 is located at a first location of the traffic intersection 302. The first sensor is for acquiring data in a first data domain. The first data field provides a measure of the number of vehicles at the intersection from which traffic flow is determined. Preferably, the first sensor 301 is a camera for collecting images of the number of vehicles at the intersection, from which the traffic flow can be determined. A sensor 303 is also located at the first location of the intersection 302 for collecting data related to environmental attributes. In this example, sensor 303 is a CO sensor. The second CO sensor 305 is located at a second location of the intersection 304 spatially distant from the first intersection 302.

As will be described in more detail below, the system learns the relationship between the data collected by the camera 301 and the CO sensor 303. The system then applies the learned relationship to the data collected from the CO sensor 305 at the second location to infer the number of vehicles at the road segment location and thus the traffic flow.

The system may also learn a relationship between the camera and at least one other sensor at the first location and use the learned relationship to determine traffic flow at the second location. For example, in fig. 3, the noise sensors 306 and 307 are also located at the first position and the second position, respectively. The system may receive data from the noise sensor 307 and predict traffic flow at the second location (i.e., at the intersection 304) based on the learned relationship between the camera 301 and the noise sensor 306. The system may also receive data from a plurality of sensors at the second location and use a plurality of relationships learned from the first location to form an output of the traffic flow at the second location.

The determined traffic flow may be used to predict future traffic flows (e.g., several hours ahead of the current time) and generate time plans for respective sets of traffic lights at the first intersection and the second intersection, as shown at 308 and 309, respectively.

Thus, the system aggregates a plurality of sensor data sources available at different locations to describe and predict traffic scenarios in urban environments. Such a system is a component that interacts with the traffic control system of each intersection. Thus, the system is located at the junction between the traffic estimation and modeling and the traffic light sequence control components.

Fig. 4 presents the overall architecture and data flow of the proposed system. The architecture shown at 400 in fig. 4 includes several parts that cooperatively process the incoming sensor time series data 401.

The first component shown at 402 is a road traffic flow time series representation and modeling module. As an essentially complex process, road traffic flows can be modeled using a temporal model that assumes that the data describes local changes in global phenomena in the form of spatially distributed time series. The module uses 1D Self-Organizing Map (SOM), which is a neural network that is capable of encoding a sensor time series in a distributed active mode over a lattice of processing units (i.e., neurons). Each processing unit has a preference value for which it will issue an output when fed to the network. This mechanism enables the system to represent adjacent values in the SOM's input sensor space that are close to each other. Using this mechanism, the system is able to extract the data distribution from the time series by encoding the data distribution by a modulated gaussian function (i.e., a narrow gaussian representing a low distribution and a wide gaussian representing a high distribution). The representation and modeling system converges rapidly, restoring the distribution of sensor data for subsequent correlation learning. Therefore, the system belongs to a lightweight system and has high running speed.

The second component shown at 403 is a multi-sensor relevance learning system for road traffic prediction. The system uses distributed encoding of the sensor time series determined by module 402 to extract time co-activations between different types of sensors. This is achieved by a correlation learning mechanism, such as e.g. a herd learning. This allows the processing unit or neuron in each input sensor SOM to strengthen the association between sensors according to co-activation (i.e., the neurons are active at the same time for a particular sample from the sensor time series). For each new sensor data sample, the learning converges rapidly to a representation representing the relationship between the input sensors. Conveniently, the representation is a connection matrix, similar to an adjacency matrix. The inherent mechanism is fast and resource efficient in terms of convergence time and memory allocation.

The third component shown at 404 is a fault-tolerant reasoning and prediction system for road traffic. The learned relationships between the sensors are then extracted from the matrix representation while taking into account the neuron locations in the input encoding system (i.e., SOM). The procedure comprises a simple optimization method for finding the value closest to the correct value from the mathematical representation of the relationship in the learned matrix representation. This is achieved by simple mathematical calculations (e.g. summation, product, square root), supporting overall simple and fast operations. The decoded functional mathematical relationship is used to infer missing sensor data (e.g., in the absence of a camera at the intersection), or to predict the correct value when a sensor fails, as shown at 405.

The overall architecture is modular and provides an adaptive system suitable for modeling and predicting highly heterogeneous time series and non-stationary processes. The system can take advantage of the correlation between different non-stationary, deterministic time sequences describing multi-sensor massive (network) phenomena. The system can achieve improved time series prediction performance in highly non-stationary problems (e.g., traffic flow predictions) using a distributed representation mechanism capable of capturing sensor time series shape and time distribution, as well as a time-dependent learning mechanism for fusing data and a powerful inference mechanism based on simple operations.

The system thus provides a pipeline processing mechanism that converts time series data acquired from sensors into a representation that can extract potential correlations between the sensors in a representation and modeling module, learn the correlations between the sensors and fuse them into a multi-sensor correlation learning module, and infer traffic flow (e.g., for locations without cameras, or where cameras fail) or improve traffic flow estimation (when cameras are present) in a fault tolerant reasoning and prediction module.

The operation of these elements will now be described in further detail with reference to fig. 5.

The representation and modeling module processes the time series data from all sensors in a distributed encoding process, as indicated at 501, to extract statistics and data distribution from the time series, as indicated at 500.

Modeling of non-stationary and deterministic time series requires the use of specialized mechanisms to interpret their characteristics. Furthermore, processing a plurality of such correlated time sequences requires an appropriate layer to describe its covariance. The module 501 encodes the sensor time series 500 using a distributed processing unit network. The sensor time series is encoded in a distributed representation using Self-Organizing Maps (SOM). The machine learning algorithm is responsible for extracting statistics of the incoming data and encoding the sensor samples in a distributed active mode, as shown in fig. 6 (a) and 6 (b). This generation of the activity pattern is such that the neuron closest to the input sample (in terms of its preferred value) will be strongly activated. The activation decays as a function of the distance between the input and the preferred value. Using the SOM distributed representation, the model learns the boundaries of the input data such that after relaxation, the SOM provides a topology preserving representation of the input space.

The multi-sensor correlation learning module 502 uses machine learning to extract sensor correlations from sensor data. In a preferred implementation, the machine learning algorithm used is a heuristics learning. The algorithm learns the correlation between the data collected by the sensors and stores it in an efficient, cost-effective and interpretable representation. The method is based on a lightweight neural network-based learning system, and does not need back propagation training. It can represent and extract the underlying statistics of the sensor data in the same learning layer. This allows the system to learn the data representation from the extracted statistics and allocate resources accordingly.

The system uses a time co-activation mode to encode the correlation between different sensors, as shown by the two sensors in fig. 6 (a) and 6 (b). In fig. 6 (a), SOM of time series data from sensors 1 and 2 is extracted, and a machine learning algorithm is used to learn the relationship between the two data sets. In fig. 6 (b), a cross-modal weight matrix is generated to represent the correlation between the two sensor data sets.

This approach has several advantages. For example, it may provide an interpretable structure ("black box free design") and an easily interpretable output. The method can use the same presentation and learning layers to handle bimodal, trimodal or multimodal extensions. Furthermore, in contrast to many existing approaches, in the systems described herein, explicit coding of sensor associations and sensor fusion rules is not required.

To illustrate the potential of the system and its underlying mechanisms, fig. 7 (a) to 7 (d) show the learning of a nonlinear relationship (in this case a third order power law relationship) between two sensor datasets. Fig. 7 (a) shows input data and input data distribution similar to the third-order power law relation. Fig. 7 (b) shows an internal model architecture for determining a cross-modal weight matrix. Fig. 7 (c) shows a calculation phase. Upon receipt of the sensor data, the system detects whether the data is from a new sensor that has not previously learned a relationship. If the sensor is not a new sensor, the mechanism enters an inference phase to predict traffic flow using the sensor data. If the data is from a new sensor, the sensor data is used in model training to determine a relationship between the new sensor data and the received data from the known sensor types. Fig. 7 (d) shows the correlation between the learned data from sensors 1 and 2.

The system is capable of handling the existence of any non-wires between data from different sensor typesThe situation of sexual relation, and has high extensibility, can handle multiple sensors. Fig. 8 (a) and 8 (b) show a sensor s comprising a tree-like correlation structure ₁ 、s ₂ Sum s ₃ Is a scene in a three-dimensional system. FIG. 8 (a) compares the model m ₁ 、m ₂ And m ₃ A learning representation of the input data encoded and decoded, while fig. 8 (b) shows the learned sensor relationships encoded in neural network weights.

After learning the underlying sensor correlations, the system is able to decode an efficient and interpretable learning representation for prediction or reasoning in the fault tolerant disturbance and prediction module, as shown at 503 in FIG. 5. More precisely, after the learning process, the network stores a stable representation of the relationship between the two sensor inputs considered during training. By considering only one input sensor source (although multiple may be used), the network may infer the corresponding number of missing sources using the learned co-activation pattern. The module uses a decoder that only needs to calculate a condition to fine tune the preferred value of the most active (winning) neuron to approximate a more accurate estimate. These calculations are simple, such as summation and product operations.

Fig. 9 (a) to 9 (c) show examples of the reasoning capability of the system. For different input data sets, fig. 9 (a) shows the input signals and relationships, fig. 9 (b) shows the learned relationships, and fig. 9 (c) shows the decoded relationships.

The above-described system components cooperate when the system is running and correspond to the functional modules shown in fig. 10. The system runtime sequence and the operations performed by the functional modules (blocks) 1001 to 1005 are as follows. In block 1001, a distributed representation is calculated and input sensor time series statistics describing traffic are extracted, generally as shown at 1000. In block 1002, data statistics of sensor data 1000 are combined and an interpretable representation is used to learn the underlying sensor relationships. In block 1003, sensor correlations from the interpretable code are decoded. The module 1004 performs predictions for the sensors based on the learned correlations and other available sensors. Module 1005 performs reasoning about missing or faulty sensors based on the learned relationships and data from other available sensors. The output of block 1005 shown at 1006 is a time series prediction (dotted line) compared to the true value (broken line).

The system can efficiently learn the sensor correlation in various traffic scenes. In the exemplary traffic scenario shown in fig. 11-13, environmental parameters (e.g., NO, O ₃ 、NO ₂ 、NO _x ) The correlation between weather (e.g., humidity, precipitation) and traffic flow (number of vehicles determined by the camera) is generally shown as 1101 in fig. 11.

After learning the correlation, the learned relationship can be transferred to a different location to infer traffic flow in areas where traffic sensors that can directly measure the number of vehicles are not installed, but other types of sensors are present. For example, at sites 1 and 2, there are NO direct traffic flow sensors, as shown generally at 1102 and 1103, respectively, but NO at these sites ₂ And a humidity sensor. After learning the sensor relationships between the different sensors at site 3, the system can use only the NO collected from site 2 ₂ And humidity sensor data to predict vehicle counts at location 2, as shown in fig. 13. Thus, the system learns NO at site 3 ₂ And the pairwise correlation between vehicle count (via camera) and humidity. Learned NO ₂ And humidity and NO ₂ The relationship between the vehicle count and the vehicle count is shown in fig. 12 (a) and 12 (b), respectively. After learning at location 3, the correlation between the two different sensor types can be used to infer traffic flow at locations 2, which are not equipped with traffic sensors such as cameras or inductive loops that can directly measure the number of vehicles at the intersection. Since the location has other types of sensors, the correlation learned from location 3 can be used to infer the number of vehicles.

The output of the system may be applied to a traffic control unit to update traffic signals at intersections in the area to maximize traffic flow. The unit may operate using any available traffic sensor data. The system is supported by flexible instrumentation, ensuring updates with low latency, high incoming event rates, and a fixed resource budget. Furthermore, the system can be deployed at any type of location or intersection without pre-training and independent of road or intersection layout, size, and available sensors. This provides a significant advantage in terms of reduced deployment costs, particularly because the learned underlying dependencies can be passed into new infrastructure layouts equipped with different sensors.

Optimization of road traffic may be performed continuously, in combination with incoming flows of historical data and current traffic data (e.g., number of vehicles, vehicle speed, occupancy at traffic lights, noise for a particular road segment, pollution values recorded for the road segment). Thus, the system is able to continually learn traffic sensor correlations, fuse sensor data describing road traffic, and adapt to changes to improve road traffic predictions. In addition, in traffic flow prediction and control, the control unit may estimate and adapt to changes in data distribution and provide accurate predictions and intelligent control actions, such as controlling traffic light green time. Thus, the system can predict traffic flow with available sensor data having a fixed resource budget.

In addition to predicting traffic flow at locations where no cameras or inductive loops are present, the system may alternatively or additionally be used to detect when a sensor is malfunctioning. For example, when a camera or inductive loop fails, the system may use data collected from other sensors to predict the correct parameter data based on the learned relationship. Assuming that correlations between other sensor types and the faulty sensor type have been previously learned, the system may provide data to replace data from the faulty sensor based on data from another sensor. Thus, the system can infer the missing sensor amount. This method may be used if correction of the faulty sensor quantity is required. For example, when the pollution sensor drifts due to humidity.

The system can be implemented using a large number of available sensors. Examples of sensors that may be used include, but are not limited to: camera, pollution sensor, noise sensor, CO sensor, NO ₂ Sensor, NO sensor, O ₃ Sensor, NO ₂ Sensor, NO _x Sensors, weather sensor (e.g., humidity, precipitation) sensing loops, mobility data, GSM subscriber cell switches covering geospatial motion parameters, high intensity sound ranges installed on streets, and particle count sensors for large range exhaust.

Fig. 14 summarizes a method for implementation in a road traffic flow prediction system to receive data from a first sensor to obtain traffic flow data in a first data domain and a second sensor and a third sensor to obtain data in a second data domain, the first sensor and the second sensor being located at a first location and the third sensor being located at a second location spatially distant from the first location. In step 1401, the method includes receiving data from the first sensor, the second sensor, and the third sensor. In step 1402, the method includes processing data received from the third sensor based on data received from the first sensor and the second sensor to estimate traffic flow at the second location.

The systems and methods described herein may help minimize the overall cost of a traffic control system by inferring traffic flows from multiple sources. This alleviates the need for expensive sensors and data sources (e.g., cameras and inductive loops) per location or intersection by inferring traffic flow from other, less expensive and readily available, related sensors, thereby enabling the system to be efficiently extended for metropolitan scenarios.

The system is particularly suited for traffic prediction, which requires modeling, prediction and fast adaptation of sensor time series. As shown in the modularized structure, the system adopts an efficient time sequence representation and modeling method, can extract data distribution, and learns multi-sensor correlation in a unified computing unit by utilizing time coactivation to conduct fault-tolerant traffic prediction.

Regardless of the deployment scenario, the data processing units are able to use their automatically extracted representations of the data distribution to distributively model the time series according to its temporal structure. The computing unit can accommodate any type of sensor time series and any number of available sensors describing road traffic.

The system also supports encoding various sensor time series into a distributed representation that captures underlying statistics and data distribution models, and provides a general approach for time series modeling that can find a best fit model that describes the underlying structure of the sensor data.

The system includes a generic architecture that can be run as part of a traffic control system to obtain a predictive model for each location. The system can be deployed for a variety of scenarios, independent of road geometry, size and configuration, and available sensors, which require flexibility and scalability.

The system may include a processor and a non-volatile memory. The system may include a plurality of processors and a plurality of memories. The memory may store data executable by the processor. The processor may be configured to operate in accordance with a computer program stored on a machine-readable storage medium in a non-transitory form. The computer program may store instructions for causing the processor to perform its methods in the manner described herein. These components may be implemented in physical hardware, as well as deployed on various edge or cloud devices.

When data enters the system, the time span of the calculation process is limited, and only simple operation can be performed due to the limitation of resource allocation and execution time. The proposed system proposes a road traffic prediction calculation unit that uses the time correlation between different traffic sensor data describing traffic conditions, thereby overcoming the prior art methods of large resource consumption, high calculation cost and complexity, such as complex analytic flow models based on differential equations, numerical methods and empirical methods.

The road traffic flow time series representation and modeling and computation unit is able to model the sensor time series using an efficient distributed model supporting data representation and statistical modeling of lightweight learning algorithms. The system provides a multi-sensor correlation learning system for road traffic prediction that learns the temporal correlation pattern between sensor data pairs describing traffic conditions, encoding the underlying functional mathematical relationship between them without any prior information about underlying statistics and correlations. The decoding mechanism decodes the learned correlations between sensors and encodes them in an efficient data representation as an interpretable mathematical functional relationship.

Thus, the proposed system provides a flexible infrastructure to analyze arbitrary sensor correlations and extract understandable sensor data correlations for traffic prediction. The lightweight learning system represents and extracts underlying statistics of sensor data in an efficient data representation. In addition, the system has an interpretable structure without a 'black box design', and interpretable output is applicable to dual-sensor, three-sensor, multi-sensor scenarios. The proposed system combines the learning capabilities exhibited by neural networks and SOMs in efficiently representing time series with efficient correlation learning and multi-sensor fusion mechanisms. This combination allows for real-time learning and relearning. Since the system has learning capabilities, it does not require explicit coding of sensor associations and sensor models. The system can transmit the learned correlation information to other positions or intersections, and can predict the road traffic flow through other available sensor data without reconfiguration, so that the related cost of equipping all intersections with expensive sensors can be reduced to the greatest extent.

The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.

Claims (12)

1. A road traffic flow prediction system, the road traffic flow prediction system configured to receive data from a first sensor configured to acquire traffic flow data in a first data domain and a second sensor and a third sensor configured to acquire data in a second data domain, the first sensor and the second sensor located at a first location and the third sensor located at a second location spatially remote from the first location, wherein the first location and the second location are a first traffic intersection and a second traffic intersection, respectively, the first sensor is a camera, and each of the second sensor and the third sensor includes one of a weather sensor, a pollution sensor, and a CO sensor;

the system is used for:

receiving data from the first sensor, the second sensor, and the third sensor;

processing data received from the third sensor in accordance with data received from the first sensor and the second sensor to estimate traffic flow at the second location;

Wherein the second sensor and the third sensor are for acquiring data relating to the level of an environmental attribute at the first location and the second location, respectively;

wherein the third sensor is configured to acquire data related to the same environmental attribute as the second sensor.

2. The system of claim 1, wherein the data received from each of the sensors comprises a time series of values.

3. The system of claim 1, wherein the first data field is one or more images of a vehicle at the location.

4. A system according to any one of claims 1 to 3, characterized in that the system is adapted to perform the processing by implementing a learned artificial intelligence model.

5. The system of claim 4, wherein the artificial intelligence model is a neural network.

6. The system of claim 4, wherein the system is configured to learn a mapping from the second data domain to the first data domain.

7. The system of claim 5, wherein the system is configured to iteratively update parameters of the model over time based on data received from the first sensor and the second sensor.

8. A system according to any one of claims 1 to 3, further for processing data received from the third sensor in dependence on data received from at least one other sensor to estimate traffic flow at the second location.

9. A system according to any one of claims 1 to 3, wherein the system is further adapted to:

receiving data from a fourth sensor located at the first location and a fifth sensor located at the second location, the fourth sensor and the fifth sensor for acquiring data in a third data domain;

processing data received from the third sensor and the fifth sensor based on data received from the first sensor, the second sensor, and the fourth sensor to estimate traffic flow at the second location;

wherein the fifth sensor is configured to acquire data related to the same environmental attribute as the fourth sensor.

10. The system of claim 1, further configured to generate a time plan for respective sets of traffic signals at the first and second traffic intersections.

11. A method for implementation at a road traffic flow prediction system, the road traffic flow prediction system configured to receive data from a first sensor configured to acquire traffic flow data in a first data domain and a second sensor and a third sensor configured to acquire data in a second data domain, the first sensor and the second sensor being located at a first location and the third sensor being located at a second location spatially distant from the first location, wherein the first location and the second location are a first traffic intersection and a second traffic intersection, respectively, the first sensor is a camera, and each of the second sensor and the third sensor includes one of a weather sensor, a pollution sensor, and a CO sensor;

wherein the method comprises the following steps:

receiving data from the first sensor, the second sensor, and the third sensor;

processing data received from the third sensor in accordance with data received from the first sensor and the second sensor to estimate traffic flow at the second location;

Wherein the data received from the second and third sensors is data relating to the level of an environmental attribute at the first and second locations;

wherein the third sensor is configured to acquire data related to the same environmental attribute as the second sensor.

12. The method of claim 11, wherein the processing includes implementing the learned artificial intelligence model.