CN102800197B - Preprocessing method of road section dynamic traffic stream essential data of urban road - Google Patents

Abstract

The invention discloses a preprocessing method of road section dynamic traffic stream essential data of an urban road. The method comprises the following steps of: acquiring the traffic stream essential data dynamically for time warping; detecting the effectiveness of divided lane traffic stream essential data; collecting the time of divided lane traffic stream essential data; collecting the fracture surface space at individual vehicle driving direction of the divided lane traffic stream essential data; and estimating the dynamic traffic stream loss data at individual vehicle driving direction. According to the invention, based on the characteristics of collected data of a fixed-type vehicle detector, the essential idea of the traffic stream theory is combined by adopting a threshold method, a set of data effectiveness detecting method is provided; aiming at different data loss patterns, history data is utilized reasonably, and the continuity and integrity of data are ensured furthest; the algorithm related to data preprocessing considers the timeliness and accuracy requirements at the same time, and the data analysis processing capability is high; and the preprocessing method of road section dynamic traffic stream essential data of the urban road plays positive significance to urban intelligent transportation system construction, road traffic informationization level improvement, and road operation management level improvement.

Description

A kind of preprocess method of Urban road Dynamic Traffic Flow basic data

Technical field

The present invention relates to intelligent transportation application, particularly a kind of preprocess method of Urban road Dynamic Traffic Flow basic data.

Background technology

Dynamic Traffic Flow basic data based on fixed wagon detector collection is important intelligent transportation system (ITS) data source.Affected by the uncertain factors such as self duty, Internet Transmission, road traffic condition and surrounding environment, there is the problems such as mistake, loss, time point drift about, noise is excessive in the data of collection.If to raw data not in addition pre-service directly apply, can affect upper strata Intelligent transport application system urban highway traffic situation and Performance Evaluating Indexes are estimated and accuracy and the reliability of prediction, cause the problems such as system operation human intervention degree is large, application system sustainability is not strong, range of application is limited.The pretreated fundamental purpose of visual data is to control the quality that road gathers traffic flow data, reduce the impact of problem data on overall data degree of accuracy, the accurate processing and the safety applications that guarantee road traffic flow data, also provide feedback information and theoretical foundation for improving urban road Dynamic Traffic Flow data acquisition system (DAS).

Two aspects of the pre-service most critical of Dynamic Traffic Flow basic data are that data validation and missing data are estimated.Data validation method has experienced from simple threshold test method to threshold test method the evolution in conjunction with the comprehensive test method of traffic flow theory, and typical algorithm has Washington algorithm (Washington Algorithm), daily statistic algorithm (Daily Statistics Algorithm) etc.; About the estimation of missing data, early stage ITS system is not carried out estimation process to missing data, but adopts the method for ignoring, and directly removes some suspicious or misdata.Obviously ignore the sample size that missing data has not only reduced traffic flow data, also probably lose the valuable information of part.Based on this, research and engineering staff recognize theory and the realistic meaning that missing data is estimated both at home and abroad, and disappearance algorithm for estimating has been carried out to a large amount of further investigations.Traffic flow missing data method of estimation roughly can be divided into method of estimation, method of estimation, artificial intelligence method of estimation and combined type method of estimation four classes based on statistics based on experience at present.Wherein the method for estimation based on experience is the method that traffic expert uses the traffic knowledge and experience oneself grasped to sum up out, as the historical method of average, and the traffic missing data method of estimation distributing based on track etc.; Method of estimation based on statistics mainly comprises time series analysis method, EM/DA model, linear regression model (LRM), Kalman filter model etc.; Artificial intelligence method of estimation mainly comprises all kinds of neural network algorithms and genetic algorithm etc.; Combined type method of estimation is the fusion application to several different methods, as combinational algorithm average based on history and that time series analysis is sent out, the combinational algorithm of model, genetic algorithm and neural network or regression model etc. that matches.

Sum up the domestic and international research and practice at Dynamic Traffic Flow basic data preprocess method at present, also exist following several problem:

1, the algorithm that available data pre-service relates to cannot be taken into account real-time and the accuracy requirement of ITS system applies simultaneously, lacks the support of traffic flow theory, and data analysis processing power is weak, excavation level is lower;

2, data validation does not form a set of comprehensive, comprehensive rule system, and inspection rule itself lacks certain level;

3, missing data is estimated to lack to the real data disappearance identification of pattern and the utilization of historical data, and the composition that comprises empirical is more, and adopts the effect of single algorithm undesirable.

Summary of the invention

Goal of the invention

The object of the invention is to, based on fixed wagon detector image data feature, adopt threshold method in conjunction with the basic thought of traffic flow theory, form a set of comprehensively, comprehensive, the data validation method with level; For different shortage of data patterns, rationally utilize historical data, adopt relatively optimum method to estimate missing data, guarantee to greatest extent continuity and the integrality of data; On this basis, consider real-time and the accuracy requirement of preprocessing algorithms.

Technical scheme

The object of the invention is to realize as follows:

A preprocess method for Urban road Dynamic Traffic Flow basic data, comprises the steps:

1) the divided lane traffic flow basic data gathering with the fixed wagon detector of certain hour interval acquiring, whole minute that the timestamp attribute of regular data is nearest neighbor is constantly;

2) divided lane traffic flow basic data validity check, carries out validity check to the traffic flow basic data in each track successively:

2-1) data non-NULL check, if data are empty, is labeled as invalid data, proceeds to step 2-7);

2-2) date and time stamp validity check, if date and time stamp misdata is labeled as invalid data, proceeds to step 2-7);

2-3) non-repeating data check, if repeating data is labeled as invalid data, proceeds to step 2-7);

2-4) automobile storage, in check, if traffic flow basic data attribute variable value is zero, is labeled as invalid data, proceeds to step 2-7);

2-5) Single-Lane Traffic basic data threshold method check;

2-6) Single-Lane Traffic basic data traffic flow theory method check;

If 2-7) all track traffic flow basic datas have all completed validity check, proceed to step 3), otherwise carry out next track traffic flow basic data validity check;

3) judge whether raw data accumulative total acquisition time interval equals interval binding time, if meet, made zero in raw data accumulative total acquisition time interval, and proceed to step 4);

4) the divided lane traffic flow basic data time is collected, and obtains effective traffic flow basic data at interval binding time, each track;

5) divided lane traffic flow basic data single unit vehicle travel direction sectional space collects, if it is valid data that result is collected in space, obtain travel effective traffic flow basic data at section interval binding time of single unit vehicle, if it is invalid data that result is collected in space, be regarded as missing data, proceed to step 6);

6) single unit vehicle travel direction section traffic flow missing data is estimated, obtains the travel estimation traffic flow basic data at section interval binding time of single unit vehicle:

6-1) the judgement consecutive miss time interval, if the time interval is less than 15 minutes, proceed to step 6-2), otherwise proceed to step 6-3);

6-2) adopt walk random method to estimate missing data;

If 6-3) consecutive miss, interval greater than equaling 15 minutes, is less than or equal to 30 minutes, proceed to step 6-4), otherwise proceed to step 6-5);

6-4) adopt the real-time adjustment method of historical average combination to estimate missing data;

6-5) adopt the historical method of average to estimate missing data.

The above-mentioned consecutive miss time interval, the timing node of selecting 15 minutes, 30 minutes be because:

This is to be determined by the effect of missing data method of estimation, and when the consecutive miss time interval is less than 15 minutes, walk random method is optimal estimation method; When consecutive miss is interval greater than equaling to be less than or equal to 30 minutes in 15 minutes, historical average combination in real time adjustment method is optimal estimation method; When consecutive miss is during interval greater than 30 minutes, the historical method of average is optimal estimation method.

In step 1), certain hour is spaced apart the raw data acquisition time interval of fixed wagon detector; Dynamic Traffic Flow basic data is the general name of the magnitude of traffic flow, Vehicle Driving Cycle average velocity, three attribute variable's data of occupation rate averaging time;

Step 2-2), in, date and time stamp misdata is defined as the data that Data Date timestamp is not equal to effective value;

Step 2-3), in, repeating data was defined as with the traffic flow basic data attribute variable in a upper raw data acquisition time interval and is worth identical data;

Described valid data and invalid data refer to effective traffic flow basic data and invalid traffic flow basic data, and traffic flow basic data attribute variable has consistent validity.

Step 2-5), in, the method for Single-Lane Traffic basic data threshold method check is as follows:

A) magnitude of traffic flow threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7);

B) Vehicle Driving Cycle average velocity threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7);

C) averaging time occupation rate threshold test, if surpass threshold range, be labeled as invalid data, proceed to step 2-7);

D) adjacent percentage speed variation threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7), (t-1, t) percentage speed variation τ of the adjacent time interval wherein _{t-1, t}computing formula as follows:

${\mathrm{\τ}}_{t-1,t}=\mathrm{Min}\{\frac{X_t}{X_{t-1}},\frac{X_{t-1}}{X_t}\}$

In formula, X _t---represent Vehicle Driving Cycle average velocity sequence, t=1,2,3 The object of adjacent percentage speed variation threshold test is Expressway Traffic Flow basic data;

Above-mentioned steps a)～d) in, each threshold range need to the statistics based on historical traffic flow basic data be demarcated.

Step 2-6), in, the method for Single-Lane Traffic basic data traffic flow theory method check is as follows:

A) judge whether to meet that Vehicle Driving Cycle average speed value equals zero and traffic flow value is not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

B) judge whether to meet that traffic flow value equals zero and Vehicle Driving Cycle average speed value is not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

C) judge whether to meet traffic flow value and Vehicle Driving Cycle average speed value all equal zero and averaging time occupation rate value be not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

D) judging whether to meet averaging time occupation rate value is zero and traffic flow value is greater than maximum possible traffic value, if meet, is labeled as invalid data, proceeds to step 2-7), maximum possible volume of traffic q wherein _maxcomputing formula as follows:

$q_{\max }=\frac{10\×v\×{\mathrm{\Δ}}_o}{l}$

In formula, q _max---represent the maximum possible volume of traffic (/ hour) when averaging time, occupation rate was zero in the raw data acquisition time interval, v---represent Vehicle Driving Cycle average velocity (thousand ms/h), Δ _o---represent occupation rate precision averaging time (%), l---represent driving vehicle average effective vehicle commander (rice);

E) judge whether traffic flow density calculation value in track surpasses maximum traffic flow density, if meet, is labeled as invalid data, proceeds to step 2-7), wherein the computing formula of track traffic flow density k is as follows:

k=q/v

In formula, k---expression track traffic flow density (/ track/km), q---represents the track magnitude of traffic flow (/ hour), v---and represents Vehicle Driving Cycle average velocity (thousand ms/h); Maximum traffic flow density need to the statistics based on historical traffic flow basic data be demarcated;

F) if averaging time, occupation rate was non-vanishing, judge that whether average effective vehicle commander calculated value surpasses certain threshold range, if meet, this data recording of mark is invalid, proceeds to step 2-7), wherein the computing formula of average effective vehicle commander l is as follows:

$l=\frac{10\×O\×v}{q}$

In formula, l---represent average effective vehicle commander (rice), O---represent occupation rate averaging time (%), v---represent Vehicle Driving Cycle average velocity (thousand ms/h), q---represent the track magnitude of traffic flow (/ hour); Average effective vehicle commander's threshold range need to statistics and traffic flow formation based on historical traffic flow basic data be demarcated.

In step 3), binding time, interval can be defined as the integral multiple in any raw data acquisition time interval as required.

In step 4), time is collected and refers on the basis of validity check, the divided lane traffic flow basic data in the raw data acquisition time interval is accumulated to the divided lane traffic flow basic data at long period interval, concrete grammar is: successively the traffic flow basic data time of carrying out in each track is collected, if there are not valid data in interval in binding time, to collect result be invalid data this track time of mark, if there are valid data, to collect computing formula as follows the Single-Lane Traffic basic data time:

$q=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i\×\frac{N}{n}$

$v=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(q_i\×v_i)}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i}$

$o=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{o}_i}{n}$

In formula, q---represent the long period interval inside lane magnitude of traffic flow (/ hour), q _i---represent effective magnitude of traffic flow (/ hour) in i the raw data acquisition time interval in long period interval, n---the valid data that represent the raw data acquisition time interval in long period interval record number, N---the raw data acquisition time interval data that represents expectation in long period interval records number, v---represent long period interval inside lane Vehicle Driving Cycle average velocity (thousand ms/h), v _i---represent effective Vehicle Driving Cycle average velocity (thousand ms/h) in i the raw data acquisition time interval in long period interval, o---represent long period interval inside lane occupation rate averaging time (%), o _i---represent occupation rate effective time (%) in i the raw data acquisition time interval in long period interval.

In step 5), space is collected to refer in the time and is collected on basis, divided lane traffic flow basic data is accumulated to the traffic flow basic data of single unit vehicle travel direction section, and concrete grammar is as follows:

If the divided lane traffic flow basic data of a) carrying out collecting in space is invalid data, to collect result be invalid data to this single unit vehicle travel direction sectional space of mark, if there are valid data, proceeds to b);

B) to collect computing formula as follows the divided lane traffic flow basic data space based on valid data:

$q=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i\×\frac{N}{n}$

$v=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(q_i\×v_i)}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i}$

$o=\frac{q\×\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\left(\frac{o_i\×v_i/q_i}{n}\right)}{v}$

In formula, q---represent the single unit vehicle travel direction section magnitude of traffic flow (/ hour), q _i---represent effective magnitude of traffic flow (/ hour) in i track, n---represent that track valid data record number, N---represent the total number of track-lines of single unit vehicle travel direction section, v---represent single unit vehicle travel direction section Vehicle Driving Cycle average velocity (thousand ms/h), v _i---represent effective Vehicle Driving Cycle average velocity (thousand ms/h) in i track, o---represent the single unit vehicle average time occupancy of travel direction section (%), o _i---represent occupation rate effective time (%) in i track.

Step 6-2), in, walk random method estimates that the expression formula of missing data is:

${\hat{X}}_t=X_{t-1}$

In formula, ---represent the estimated value of t traffic flow constantly basic data, X _t-1---represent the actual value of t-1 traffic flow constantly basic data;

Step 6-4), in, the historical average expression formula in conjunction with the missing data of adjustment method estimation is in real time:

${\hat{X}}_{t,d_i}=X_{(t-1),d_i}\×\frac{\mathrm{HIST}{\hat{X}}_{t,d_i}}{\mathrm{HIST}{\hat{X}}_{(t-1),d_i}}$

$\mathrm{HIST}{\hat{X}}_{(t-1),d_i}=\left[\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{X}_{(t-1),d_{(i-j)}}\right]/N$

$\mathrm{HIST}{\hat{X}}_{t,d_i}=\left[\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\hat{X}}_{t,d_{(i-j)}}\right]/N$

In formula, ---represent d _ithe estimated value of it t traffic flow constantly basic data, ---represent d _ithe actual value of it t-1 traffic flow constantly basic data, with ---be expressed as d _ithe history average of it t-1 and t traffic flow constantly basic data, with ---represent respectively d _(i-j)the actual value of it t-1 and t traffic flow constantly basic data, N---represent set time length of window;

Step 6-5), in, the historical method of average estimates that the expression formula of missing data is:

${\hat{X}}_{t,d_i}=\left[\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{X}_{t,d_{(i-j)}}\right]/N$

In formula, ---represent d _ithe estimated value of it t traffic flow constantly basic data, ---represent d _(i-j)the actual value of it t traffic flow constantly basic data, N---represent set time length of window.

Technique effect:

The invention provides a kind of preprocess method of Urban road Dynamic Traffic Flow basic data, for the fixed wagon detector image data of Urban road and different types of road traffic flow operation characteristic, determined that threshold value index and the traffic flow with statistical significance move the restriction relation between three parameters (flow, speed, occupation rate), on this basis, provide a set of comprehensive data validation rule; Data are collected to processing from time and two, space dimension, generated the traffic flow basic data of multi-level type, for the data, services of different application demand is laid a good foundation; By the identification in the shortage of data time interval in various degree, adopt relatively optimum method to estimate missing data, guaranteed to greatest extent integrality and the reliability of data.The inventive method for the pre-service of Urban road traffic flow data provide a set of comprehensively, comprehensive, the solution with level, can meet the fields such as traffic operation and management, traffic programme decision-making, scientific research, public service for the demand of traffic flow basic data, also for building urban road traffic information acquisition system and traffic integration information platform, provide technical support simultaneously.

Accompanying drawing explanation

Fig. 1 is a kind of preprocess method process flow diagram of Urban road Dynamic Traffic Flow basic data;

Fig. 2 is image data cross section place figure;

Fig. 3 is No. 2035 section time period flow-speed time series charts;

West 2nd Ring Road, Tu4Shi Beijing 2046 section flow-speed time series chart at noon on September 4th, 2009;

West 2nd Ring Road, Tu5Shi Beijing 2041 section in September, 2009 11-12 day speed-average effective vehicle commander graph of a relation;

Fig. 6 is Vehicle Driving Cycle average velocity MAE figure;

Fig. 7 is speed MAPE figure;

Fig. 8 is speed RMSE figure.

Embodiment

Below in conjunction with the drawings and specific embodiments, the invention will be further described.

The present invention adopts the through street of Road Transportation in Beijing data acquisition system (DAS) microwave vehicle detecting device collection and the case verification that major trunk roads traffic flow basic data is carried out a kind of preprocess method of Urban road Dynamic Traffic Flow basic data.Traffic flow basic data is the general name of the magnitude of traffic flow, Vehicle Driving Cycle average velocity, three attribute variable's data of occupation rate averaging time, the magnitude of traffic flow is the traffic value in unit interval, Vehicle Driving Cycle average velocity is the average place of vehicle instantaneous velocity in unit interval, and averaging time, occupation rate was to have time that car passes through to account for the number percent at wagon detector acquisition time interval.The raw data acquisition time interval of Road Transportation in Beijing data acquisition system (DAS) setting is two minutes, through street data acquisition system (DAS) to averaging time occupation rate be less than 1% data and be all recorded as 1%, major trunk roads data acquisition system (DAS) to averaging time occupation rate be less than 1% data and be all recorded as zero.

As shown in Figure 1, concrete implementation step is as follows:

Step 1. was obtained the divided lane Dynamic Traffic Flow basic data of city expressway and major trunk roads with two minutes raw data acquisition time intervals, the line time of going forward side by side is regular, by raw data concentrate the timestamp attribute of entanglement regular be whole minute of nearest neighbor constantly, two minute datas of generation standard.

Step 2. divided lane traffic flow basic data validity check, successively the traffic flow basic data in each track is carried out to validity check, its assay comprises two class data, be respectively effective traffic flow basic data and invalid traffic flow basic data, referred to as valid data and invalid data, and traffic flow basic data attribute variable has consistent validity.The present embodiment is distinguished by data recording is increased to a data validation attribute, is defaulted as valid data, and property value is 1, if assay is invalid data, property value becomes 0, and concrete steps are as follows:

2-1) data non-NULL check, if data are empty, is labeled as invalid data, proceeds to step 2-7);

2-2) date and time stamp validity check, if date and time stamp misdata, be labeled as invalid data, proceed to step 2-7), wherein date and time stamp misdata is defined as the data that Data Date timestamp is not equal to effective value, the effective value of data time stamp, is worth reference time, is the predefined data upload of data acquisition system (DAS) sequence constantly;

2-3) non-repeating data check, if repeating data is labeled as invalid data, proceeds to step 2-7), wherein repeating data was defined as with the traffic flow basic data attribute variable in a upper raw data acquisition time interval and is worth identical data;

2-4) automobile storage, in check, if traffic flow basic data attribute variable value is zero, is labeled as invalid data, proceeds to step 2-7);

2-5) Single-Lane Traffic basic data threshold method check.The core concept of threshold method check is the traffic flow data based on actual acquisition, gathers traffic flow basic data attribute variable's very big and minimal value according to the fixed wagon detector of mathematical statistics rule rational, comprises following inspection rule:

A) magnitude of traffic flow threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7);

B) Vehicle Driving Cycle average velocity threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7);

C) averaging time occupation rate threshold test, if surpass threshold range, be labeled as invalid data, proceed to step 2-7);

D) adjacent percentage speed variation threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7), (t-1, t) percentage speed variation τ of the adjacent time interval wherein _{t-1, t}computing formula as follows:

${\mathrm{\τ}}_{t-1,t}=\mathrm{Min}\{\frac{X_t}{X_{t-1}},\frac{X_{t-1}}{X_t}\}$

In formula, X _t---represent Vehicle Driving Cycle average velocity sequence, t=1,2,3

The statistics of the historical traffic flow data of the present embodiment based on Beijing's city expressway and the collection of major trunk roads microwave vehicle detecting device is demarcated each threshold range, the maximum of bicycle road, through street Dynamic Traffic Flow basic data and the minimum volume of traffic (), Vehicle Driving Cycle average velocity (thousand ms/h), averaging time occupation rate check (%), adjacent percentage speed variation (dimensionless) threshold values scope be respectively [0,80], [0,120], [0,100], [0.4,1]; The maximum of major trunk roads bicycle road Dynamic Traffic Flow basic data and the minimum volume of traffic, Vehicle Driving Cycle average velocity, averaging time, occupation rate threshold values scope was respectively [0,70], [0,120], [0,100].Because the traffic flow of city expressway belongs to continuous traffic flow category, have in adjacent short period interval, traffic flow modes changes less feature, therefore set adjacent percentage speed variation inspection rule, and the traffic flow of major urban arterial highway belongs to discontinuity traffic flow category, traffic flow modes undulatory property is larger, inapplicable this inspection rule;

2-6) Single-Lane Traffic basic data traffic flow theory method check.The basic thought of the Dynamic Traffic Flow data validation based on traffic flow theory is to utilize the comformity relation between traffic flow operational factor to build dynamic traffic data validity check rule, the traffic flow basic data of fixed wagon detector actual acquisition except some attribute variables be zero, other attribute variable can not be for outside zero situation, also exist three attribute variable's values all non-vanishing, but inconsistent situation between attribute variable.The method of concrete test stage is as follows:

A) judge whether to meet that Vehicle Driving Cycle average speed value equals zero and traffic flow value is not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

B) judge whether to meet that traffic flow value equals zero and Vehicle Driving Cycle average speed value is not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

C) judge whether to meet traffic flow value and Vehicle Driving Cycle average speed value all equal zero and averaging time occupation rate value be not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

D) judging whether to meet averaging time occupation rate value is zero and traffic flow value is greater than maximum possible traffic value, if meet, be labeled as invalid data, proceed to step 2-7), this rule be for the not enough magnitude of traffic flow producing of fixed wagon detector occupation rate acquisition precision be greater than zero and averaging time occupation rate be zero situation setting, as Beijing's major trunk roads data acquisition system (DAS) to averaging time occupation rate be less than 1% data and be all recorded as zero.Maximum possible volume of traffic q wherein _maxcomputing formula as follows:

$q_{\max }=\frac{10\×v\×{\mathrm{\Δ}}_o}{l}$

In formula, q _max---represent the maximum possible volume of traffic (/ hour) when averaging time, occupation rate was zero in the raw data acquisition time interval, v---represent Vehicle Driving Cycle average velocity (thousand ms/h), Δ _o---represent occupation rate precision averaging time (%), l---represent driving vehicle average effective vehicle commander (rice);

E) judge whether traffic flow density calculation value in track surpasses maximum traffic flow density, if meet, is labeled as invalid data, proceeds to step 2-7), wherein the computing formula of track traffic flow density k is as follows:

k=q/v

In formula, k---expression track traffic flow density (/ track/km), q---represents the track magnitude of traffic flow (/ hour), v---and represents Vehicle Driving Cycle average velocity (thousand ms/h);

F) if averaging time occupation rate non-vanishing, judge that whether average effective vehicle commander calculated value surpasses certain threshold range, if meet, be labeled as invalid data, proceed to step 2-7), wherein the computing formula of average effective vehicle commander l is as follows:

$l=\frac{10\×O\×v}{q}$

In formula, l---represent average effective vehicle commander (rice), O---represent occupation rate averaging time (%), v---represent Vehicle Driving Cycle average velocity (thousand ms/h), q---represent the track magnitude of traffic flow (/ hour);

Above-mentioned steps d), in, parameter l is demarcated as 2.4 meters of the car average effective vehicle commanders that Beijing's major trunk roads travel, parameter Δ _obe demarcated as 1;

Above-mentioned steps e) in, the statistics of the historical traffic flow data based on Beijing's city expressway and the collection of major trunk roads microwave vehicle detecting device, the maximum traffic flow density value in through street is demarcated as 100/track/km, and the maximum traffic flow density value of major trunk roads is demarcated as 70/track/km;

Above-mentioned steps f) in, based on Beijing's city expressway and the historical traffic flow data of major trunk roads microwave vehicle detecting device collection and the vehicle component analysis of through street, Beijing and major trunk roads, through street and major trunk roads average effective vehicle commander threshold values scope are demarcated as [2.4,18];

If 2-7) all track traffic flow basic datas have all completed validity check, proceed to step 3, otherwise carry out next track traffic flow basic data validity check.

Step 3. judges whether raw data accumulative total acquisition time interval equals interval binding time, if meet, made zero in raw data accumulative total acquisition time interval, and proceed to step 4, wherein binding time, interval can be defined as the integral multiple in any raw data acquisition time interval as required.

The step 4. divided lane traffic flow basic data time is collected, obtain effective traffic flow basic data at each interval binding time, track, the traffic flow data time is collected the magnitude of traffic flow, Vehicle Driving Cycle average velocity, the time occupancy data that refer to the short period interval (as 1 minute) of fixed wagon detector collection and collects the magnitude of traffic flow, Vehicle Driving Cycle average velocity data and the time occupancy data for long period interval (as 2 minutes or 5 minutes).The divided lane traffic flow basic data in the raw data acquisition time interval is accumulated to the divided lane traffic flow basic data at long period interval, concrete grammar is: successively the traffic flow basic data time of carrying out in each track is collected, if there are not valid data in interval in binding time, to collect result be invalid data this track time of mark, if there are valid data, to collect computing formula as follows the Single-Lane Traffic basic data time:

$q=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i\×\frac{N}{n}$

$v=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(q_i\×v_i)}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i}$

$o=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{o}_i}{n}$

In formula, q---represent the long period interval inside lane magnitude of traffic flow (/ hour), q _i---represent effective magnitude of traffic flow (/ hour) in i the raw data acquisition time interval in long period interval, n---the valid data that represent the raw data acquisition time interval in long period interval record number, N---the raw data acquisition time interval data that represents expectation in long period interval records number, v---represent long period interval inside lane Vehicle Driving Cycle average velocity (thousand ms/h), v _i---represent effective Vehicle Driving Cycle average velocity (thousand ms/h) in i the raw data acquisition time interval in long period interval, o---represent long period interval inside lane occupation rate averaging time (%), o _ioccupation rate effective time (%) in i the raw data acquisition time interval in-expression long period interval.

Step 5. divided lane traffic flow basic data single unit vehicle travel direction sectional space collects.If it is invalid data that result is collected in space, obtain travel effective traffic flow basic data at section interval binding time of single unit vehicle, if result is collected in space, be invalid data, be regarded as missing data, proceed to step 6.Traffic flow basic data space is collected to refer in the time and is collected on basis, by the divided lane magnitude of traffic flow at interval binding time, Vehicle Driving Cycle average velocity data, averaging time occupation rate data accumulate single unit vehicle travel direction section the magnitude of traffic flow, Vehicle Driving Cycle average velocity and averaging time occupation rate, concrete grammar is as follows:

If the divided lane traffic flow basic data of a) carrying out collecting in space is invalid data, to collect result be invalid data to this single unit vehicle travel direction sectional space of mark, if there are valid data, otherwise proceeds to b);

B) to collect computing formula as follows the divided lane traffic flow basic data space based on valid data:

$q=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i\×\frac{N}{n}$

$v=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(q_i\×v_i)}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i}$

$o=\frac{q\×\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\left(\frac{o_i\×v_i/q_i}{n}\right)}{v}$

In formula, q---represent the single unit vehicle travel direction section magnitude of traffic flow (/ hour), q _i---represent effective magnitude of traffic flow (/ hour) in i track, n---represent that track valid data record number, N---represent the total number of track-lines of single unit vehicle travel direction section, v---represent single unit vehicle travel direction section Vehicle Driving Cycle average velocity (thousand ms/h), v _i---represent effective Vehicle Driving Cycle average velocity (thousand ms/h) in i track, o---represent the single unit vehicle average time occupancy of travel direction section (%), o _i---represent occupation rate effective time (%) in i track.

Step 6. single unit vehicle travel direction section Dynamic Traffic Flow missing data estimates, obtains the travel estimation traffic flow basic data at section interval binding time of single unit vehicle, and specific implementation method is as follows:

6-1) the judgement consecutive miss time interval, if the time interval is less than 15 minutes, proceed to step 6-2), otherwise proceed to step 6-3); .

6-2) adopt walk random method to estimate missing data, its calculation expression is:

${\hat{X}}_t=X_{t-1}$

In formula, ---represent the estimated value of t traffic flow constantly basic data, X _t-1---represent the actual value of t-1 traffic flow constantly basic data;

If 6-3) consecutive miss, interval greater than equaling 15 minutes, is less than or equal to 30 minutes, proceed to step 6-4), otherwise proceed to step 6-5);

6-4) adopt the real-time adjustment method of historical average combination to estimate missing data, its calculation expression is:

${\hat{X}}_{t,d_i}=X_{(t-1),d_i}\×\frac{\mathrm{HIST}{\hat{X}}_{t,d_i}}{\mathrm{HIST}{\hat{X}}_{(t-1),d_i}}$

$\mathrm{HIST}{\hat{X}}_{(t-1),d_i}=\left[\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{X}_{(t-1),d_{(i-j)}}\right]/N$

$\mathrm{HIST}{\hat{X}}_{t,d_i}=\left[\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\hat{X}}_{t,d_{(i-j)}}\right]/N$

In formula, ---represent d _ithe estimated value of it t traffic flow constantly basic data, ---represent the actual value of di days t-1 traffic flow constantly basic datas, with ---be expressed as d _ithe history average of it t-1 and t traffic flow constantly basic data, with ---represent respectively d _(i-j)the actual value of it t-1 and t traffic flow constantly basic data, N---represent set time length of window;

6-5) adopt the historical method of average to estimate missing data, its calculation expression is:

${\hat{X}}_{t,d_i}=\left[\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{X}_{t,d_{(i-j)}}\right]/N$

In formula, ---represent d _ithe estimated value of it t traffic flow constantly basic data, ---represent d _(i-j)the actual value of it t traffic flow constantly basic data, N---represent set time length of window;

Set time length of window in the present embodiment definition real-time adjustment method of historical average combination and the historical method of average is 30 days.

Based on actual acquired data, following content by from data validation result, Vehicle Driving Cycle average velocity threshold test effect, adjacent percentage speed variation threshold test effect, average effective vehicle commander's threshold test effect and the different disappearance time interval four aspects of missing data method of estimation recruitment evaluation further illustrate and verify a kind of preprocess method of Urban road Dynamic Traffic Flow basic data.

Data validation result

In order to check the performance of Dynamic Traffic Flow basic data validity check rule, the present embodiment has been enumerated through street, Beijing (being numbered 2010,2014,2017,2035) and major trunk roads (being numbered 22002,22005,24003,24005) as shown in Figure 2, the data validity rule test result in 8 section on September 1st, 2009 to September 30 altogether, as shown in table 1:

8 section Dynamic Traffic Flow in September basic data validity check result summary sheets of through street, table 1 Beijing and major trunk roads

Rule 4, rule 5 and rule 6 are easier to check the data that make mistake as can be seen from Table 1, and the amount of error that other rule tests go out is relatively less.To stress the effect of the average effective vehicle commander's check in the Vehicle Driving Cycle average velocity threshold test in threshold method check, the check of adjacent percentage speed variation and the check of traffic flow method below:

Vehicle Driving Cycle average velocity threshold test effect

Table 2 and a mistake! Do not find Reference source.3 are respectively traffic flow basic data and the corresponding flow speed figure of ring direction to Beijing first lane 16:06:00 to 17:00:00 on September 8th, 2009 in the section of No. 2035, through street, Beijing.Wherein.

3 that in table 2, are labeled as grey are recorded as the wrong or suspicious data that Vehicle Driving Cycle average velocity threshold rule is checked out, and corresponding to 3 points in the middle of Fig. 3 circle.

Ring direction to Beijing first lane traffic flow base data table in 2035 sections of through street, table 2 Beijing

Adjacent percentage speed variation threshold test effect

It is traffic flow basic data and flow-speed time series graph of a relation of No. 2035 sections microwave vehicle detecting device actual acquisition at noon on September 4th, 2009 that table 3 and Fig. 4 show.Wherein, two that in table 4, are labeled as grey are recorded as mistake or the suspicious data occurring after adjacent time percentage speed variation threshold test, and corresponding to 2 points in the middle of Fig. 4 circle.

Table 5 real time data and different traffic classification cluster centre Euclidean distance table

Average effective vehicle commander's threshold test effect

A mistake! Do not find Reference source.5 be No. 2035 sections on September 11st, 2009 and September 12 Vehicle Driving Cycle average velocity and average effective vehicle commander between loose some graph of a relation.As seen from the figure, the average effective vehicle commander of actual travel vehicle is seldom greater than 18 meters or be less than 2.4 meters, the data that triangle sign represents are the mistake or the suspicious data that after average effective vehicle commander threshold test, occur, and table 6 has been enumerated the misdata that part average effective vehicle commander checks rear appearance.

West 2nd Ring Road, table 7 Beijing 2041 section September 11 in 2009, per day effective vehicle commander checked misdata example

Missing data method of estimation recruitment evaluation under the different disappearance time intervals

In order to assess the estimated performance of each class methods under the different disappearance time interval, the present embodiment is guaranteeing on the basis of selected section data integrity in September, extract the selected section partial data in October, 11 kinds of different consecutive miss levels of artificial generation, be respectively consecutive miss 10 minutes, 15 minutes, 20 minutes, 30 minutes, 45 minutes, 1 hour, 3 hours, 5 hours, 8 hours, 12 hours and 1 day, and estimated result and True Data are contrasted, calculate respectively and estimate mean absolute error (MAE), average absolute percentage error (MAPE) and mean absolute error standard deviation (RMSE) value, some numerical results is as Fig. 6, 7, shown in 8.

According to chart, comprehensively analyze, draw the 7th consecutive miss level (3 hours) before, walk random and historical average combination in real time adjustment method are totally better than the historical method of average; Before the 2nd consecutive miss level, walk random is better than historical average in conjunction with real-time adjustment method simultaneously; Between 7 consecutive miss levels of the 2nd consecutive miss level to the, historical average combination in real time adjustment method is totally better than walk random model; After the 7th consecutive miss level, the historical on average estimation technique is better than the walk random estimation technique and historical average in conjunction with real-time adjustment method.

Claims (3)

1. a preprocess method for Urban road Dynamic Traffic Flow basic data, is characterized in that comprising the steps:

1) the divided lane traffic flow basic data gathering with the fixed wagon detector of certain hour interval acquiring, whole minute that the timestamp attribute of regular data is nearest neighbor is constantly;

2) divided lane traffic flow basic data validity check, carries out validity check to the traffic flow basic data in each track successively:

2-1) data non-NULL check, if data are empty, is labeled as invalid data, proceeds to step 2-7);

2-2) date and time stamp validity check, if date and time stamp misdata is labeled as invalid data, proceeds to step 2-7);

2-3) non-repeating data check, if repeating data is labeled as invalid data, proceeds to step 2-7);

2-4) automobile storage, in check, if traffic flow basic data attribute variable value is zero, is labeled as invalid data, proceeds to step 2-7);

2-5) Single-Lane Traffic basic data threshold method check;

2-6) Single-Lane Traffic basic data traffic flow theory method check;

If 2-7) all track traffic flow basic datas have all completed validity check, proceed to step 3), otherwise carry out next track traffic flow basic data validity check;

3) judge whether raw data accumulative total acquisition time interval equals interval binding time, if meet, made zero in raw data accumulative total acquisition time interval, and proceed to step 4);

4) the divided lane traffic flow basic data time is collected, and obtains effective traffic flow basic data at interval binding time, each track;

5) divided lane traffic flow basic data single unit vehicle travel direction sectional space collects, if it is valid data that result is collected in space, obtain travel effective traffic flow basic data at section interval binding time of single unit vehicle, if it is invalid data that result is collected in space, be regarded as missing data, proceed to step 6);

6) single unit vehicle travel direction section traffic flow missing data is estimated, obtains the travel estimation traffic flow basic data at section interval binding time of single unit vehicle:

6-1) the judgement consecutive miss time interval, if the time interval is less than 15 minutes, proceed to step 6-2), otherwise proceed to step 6-3);

6-2) adopt walk random method to estimate missing data;

If 6-3) consecutive miss, interval greater than equaling 15 minutes, is less than or equal to 30 minutes, proceed to step 6-4), otherwise proceed to step 6-5);

6-4) adopt the real-time adjustment method of historical average combination to estimate missing data;

6-5) adopt the historical method of average to estimate missing data;

Step 2-5), in, the method for Single-Lane Traffic basic data threshold method check is as follows:

A) magnitude of traffic flow threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7);

B) Vehicle Driving Cycle average velocity threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7);

C) averaging time occupation rate threshold test, if surpass threshold range, be labeled as invalid data, proceed to step 2-7);

D) adjacent percentage speed variation threshold test, if surpass threshold range, is labeled as invalid data, proceeds to step 2-7), (t-1, t) percentage speed variation τ of the adjacent time interval wherein _{t-1, t}computing formula as follows:

${\mathrm{\τ}}_{t-1,t}=\mathrm{Min}\{\frac{X_t}{X_{t-1}},\frac{X_{t-1}}{X_t}\}$

In formula, X _t---represent Vehicle Driving Cycle average velocity sequence, t=1,2,3 The object of adjacent percentage speed variation threshold test is Expressway Traffic Flow basic data;

Above-mentioned steps a)～d) in, each threshold range need to the statistics based on historical traffic flow basic data be demarcated;

Step 2-6), in, the method for Single-Lane Traffic basic data traffic flow theory method check is as follows:

A) judge whether to meet that Vehicle Driving Cycle average speed value equals zero and traffic flow value is not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

B) judge whether to meet that traffic flow value equals zero and Vehicle Driving Cycle average speed value is not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

C) judge whether to meet traffic flow value and Vehicle Driving Cycle average speed value all equal zero and averaging time occupation rate value be not equal to zero, if meet, be labeled as invalid data, proceed to step 2-7);

D) judging whether to meet averaging time occupation rate value is zero and traffic flow value is greater than maximum possible traffic value, if meet, is labeled as invalid data, proceeds to step 2-7), maximum possible volume of traffic q wherein _maxcomputing formula as follows:

$q_{\max }=\frac{10\×v\×{\mathrm{\Δ}}_o}{l}$

In formula, q _max---represent maximum possible volume of traffic when averaging time, occupation rate was zero in the raw data acquisition time interval, unit is/hour; V---represent Vehicle Driving Cycle average velocity, unit is thousand ms/h; Δ _o---represent occupation rate precision averaging time, unit is %; L---represent driving vehicle average effective vehicle commander, unit is rice;

E) judge whether traffic flow density calculation value in track surpasses maximum traffic flow density, if meet, is labeled as invalid data, proceeds to step 2-7), wherein the computing formula of track traffic flow density k is as follows:

k＝q/v

In formula, k---represent track traffic flow density, unit is/track/km; Q---represent the track magnitude of traffic flow, unit is/hour; V---represent Vehicle Driving Cycle average velocity, unit is thousand ms/h; Maximum traffic flow density need to the statistics based on historical traffic flow basic data be demarcated;

F) if averaging time, occupation rate was non-vanishing, judge that whether average effective vehicle commander calculated value surpasses certain threshold range, if meet, this data recording of mark is invalid, proceeds to step 2-7), wherein the computing formula of average effective vehicle commander l is as follows:

$l=\frac{10\×o\×v}{q}$

In formula, l---represent average effective vehicle commander, unit is rice; O---represent occupation rate averaging time, unit is %; V---represent that Vehicle Driving Cycle average velocity unit is thousand ms/h; Q---represent the track magnitude of traffic flow, unit is/hour; Average effective vehicle commander's threshold range need to statistics and traffic flow formation based on historical traffic flow basic data be demarcated;

Described step 4) in, time is collected and refers on the basis of validity check, the divided lane traffic flow basic data in the raw data acquisition time interval is accumulated to the divided lane traffic flow basic data at long period interval, concrete grammar is: successively the traffic flow basic data time of carrying out in each track is collected, if there are not valid data in interval in binding time, to collect result be invalid data this track time of mark, if there are valid data, to collect computing formula as follows the Single-Lane Traffic basic data time:

$q_T=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i\×\frac{N}{n}$

$v_T=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}(q_i\×v_i)}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{q}_i}$

$o_T=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{o}_i}{n}$

In formula, q _t---represent the long period interval T inside lane magnitude of traffic flow, unit is/hour; q _i---represent effective magnitude of traffic flow in i the raw data acquisition time interval in long period interval T, unit is/hour; The raw data acquisition time interval data that represents expectation in long period interval T records number, v to n---valid data that represent the raw data acquisition time interval in long period interval T record number, N--- _t---represent long period interval T inside lane Vehicle Driving Cycle average velocity, unit is thousand ms/h; v _i---represent effective Vehicle Driving Cycle average velocity in i the raw data acquisition time interval in long period interval T, unit is thousand ms/h; o _t---represent long period interval T inside lane occupation rate averaging time, unit is %; o _i---represent occupation rate effective time in i the raw data acquisition time interval in long period interval T, unit is %;

Described step 5) in, space is collected to refer in the time and is collected on basis, divided lane traffic flow basic data is accumulated to the traffic flow basic data of single unit vehicle travel direction section, and concrete grammar is as follows:

If the divided lane traffic flow basic data of a) carrying out collecting in space is invalid data, to collect result be invalid data to this single unit vehicle travel direction sectional space of mark, if there are valid data, proceeds to b);

B) to collect computing formula as follows the divided lane traffic flow basic data space based on valid data:

$\stackrel{\‾}{q}=\underset{j=1}{\overset{m}{\mathrm{\Σ}}}{q}_j\×\frac{M}{m}$

$\stackrel{\‾}{v}=\frac{\underset{j=1}{\overset{m}{\mathrm{\Σ}}}(q_j\×v_j)}{\underset{j=1}{\overset{m}{\mathrm{\Σ}}}{q}_j}$

$\stackrel{\‾}{o}=\frac{\stackrel{\‾}{q}\×\underset{j=1}{\overset{m}{\mathrm{\Σ}}}\left(\frac{o_j\×v_j/q_j}{m}\right)}{\stackrel{\‾}{v}}$

In formula, ---represent the single unit vehicle travel direction section magnitude of traffic flow, unit is/hour; q _j---represent effective magnitude of traffic flow in j track, unit is/hour; M---expression track valid data record number, M---represents the total number of track-lines of single unit vehicle travel direction section, ---represent single unit vehicle travel direction section Vehicle Driving Cycle average velocity, unit is thousand ms/h; v _j---represent effective Vehicle Driving Cycle average velocity in j track, unit is thousand ms/h; ---represent the average time occupancy of single unit vehicle travel direction section, unit is %; o _j---represent occupation rate effective time in j track, unit is %;

Described step 6-2), in, walk random method estimates that the expression formula of missing data is:

${\hat{X}}_t=X_{t-1}$

In formula, ---represent the estimated value of t traffic flow constantly basic data, X _t-1---represent the actual value of t-1 traffic flow constantly basic data;

Described step 6-4), in, the historical average expression formula in conjunction with the missing data of adjustment method estimation is in real time:

${\hat{X}}_{t,d_i}=X_{(t-1),d_i}\×\frac{{\mathrm{HIST}\hat{X}}_{t,d_i}}{\mathrm{HIST}{\hat{X}}_{(t-1),d_i}}$

$\mathrm{HIST}{\hat{X}}_{(t-1),d_i}=\left[\underset{j=1}{\overset{K}{\mathrm{\Σ}}}{X}_{(t-1),d_{(i-j)}}\right]/K$

$\mathrm{HIST}{\hat{X}}_{t,d_i}=\left[\underset{j=1}{\overset{K}{\mathrm{\Σ}}}{\hat{X}}_{t,d_{(i-j)}}\right]/K$

In formula, ---represent d _ithe estimated value of it t traffic flow constantly basic data, ---represent d _ithe actual value of it t-1 traffic flow constantly basic data, with ---be expressed as d _ithe history average of it t-1 and t traffic flow constantly basic data, with ---represent respectively d _(i-j)the actual value of it t-1 and t traffic flow constantly basic data, K---represent set time length of window;

Described step 6-5), in, the historical method of average estimates that the expression formula of missing data is:

${\hat{X}}_{t,d_i}=\left[\underset{j=1}{\overset{K}{\mathrm{\Σ}}}{X}_{t,d_{(i-j)}}\right]/K$

In formula, ---represent d _ithe estimated value of it t traffic flow constantly basic data, ---represent d _(i-j)the actual value of it t traffic flow constantly basic data, K---represent set time length of window.

2. the preprocess method of a kind of Urban road Dynamic Traffic Flow basic data according to claim 1, is characterized by:

Step 1), in, certain hour is spaced apart the raw data acquisition time interval of fixed wagon detector; Dynamic Traffic Flow basic data is the general name of the magnitude of traffic flow, Vehicle Driving Cycle average velocity, three attribute variable's data of occupation rate averaging time;

Step 2-2), in, date and time stamp misdata is defined as the data that Data Date timestamp is not equal to effective value;

Step 2-3), in, repeating data was defined as with the traffic flow basic data attribute variable in a upper raw data acquisition time interval and is worth identical data;

Described valid data and invalid data refer to effective traffic flow basic data and invalid traffic flow basic data, and traffic flow basic data attribute variable has consistent validity.

3. the preprocess method of a kind of Urban road Dynamic Traffic Flow basic data according to claim 1, is characterized by step 3) in, binding time, interval can be defined as the integral multiple in any raw data acquisition time interval as required.