CN107609107A - A kind of trip co-occurrence phenomenon visual analysis method based on multi-source Urban Data - Google Patents

Abstract

The invention belongs to city mobile data analysis technical field, discloses a kind of trip co-occurrence phenomenon visual analysis method based on multi-source Urban Data, and step is：Region division is carried out to city first with road network basic data and emulation tool, then to being modeled for interregional co-occurrence, rule digging is associated to region based on taxi track data using the parameter of the setting of the model and user, city interest point data excavation regions function is combined afterwards, it is final to visualize co-occurrence Result and regional function.The present invention can utilize multi-source Urban Data：Taxi track data, city road network data, POI data, visual analyzing with multi-angle is carried out in all directions to region co-occurrence phenomenon and urban area function and explored, effective information is provided for Urban Traffic Planning, there is to be easy to analyze data internal association, workable.

Description

A visual analysis method for travel co-occurrence phenomena based on multi-source urban data

technical field

The invention belongs to the technical field of urban mobile data analysis, and in particular relates to a method for visual analysis of travel co-occurrence phenomena based on multi-source urban data.

Background technique

With the rapid development of urban traffic, a large amount of mobile data has been generated. These mobile data have rich time attributes and spatial attributes, and these attributes can truly reflect the mobility of urban humans. As an important part of urban mobile transportation, taxis provide great convenience for urban residents to travel. According to the taxi trajectory data, it can be found that there is a certain regular travel pattern in the city, which is of great significance for understanding the urban structure. We define co-occurrence phenomenon as: if people from area A and area B visit area C during the same time interval, we say "area A and area B co-occur in area C". We can say that region A and region B are involved in a co-occurrence event. The law of all co-occurrence events in a city is our analysis subject - co-occurrence phenomenon. Based on the analysis of co-occurrence phenomena, we can obtain valuable information on urban planning, business strategy formulation, and the spread of contagious infectious diseases. Road network data is the most commonly used geographical data in urban research, usually presented in the form of graphs. Nodes in the graph represent intersections, which have unique geographic coordinates; edges represent road segments, connecting two nodes; other attributes, such as length, speed limit, road type, number of lanes, etc., are related to edges. Point Of Interest (POI) data (such as restaurants, shopping malls) usually consists of names, addresses, categories, and geographic coordinates, which briefly introduces the basic attributes of each geographic unit. This type of data is mainly obtained through map data providers. Marked manually or freely edited by netizens on open source online map websites.

However, the complexity and abstraction of taxi trajectory data make it difficult to mine information from these data. Visualization technology combines the display form of visual charts and human-computer interaction to simplify the analysis process. Users modify the parameters of the analysis model through interaction, thereby Generate new visualization results, and through visual analysis, more valuable information can be mined from taxi trajectory data.

This method uses urban taxi trajectory data, road network data and POI data, aiming to use multi-source urban data to jointly explore the co-occurrence phenomenon of travel from various aspects and tap the hidden value of this phenomenon.

Contents of the invention

The purpose of the present invention is to propose a visual analysis method for travel co-occurrence phenomena based on multi-source city data mainly for the inconvenience of the above-mentioned data analysis. Divide regions based on road network data, and extract co-occurrence data that can reflect the connection between regions by processing taxi trajectory data; combine urban POI data to mine regional functions, and finally visualize co-occurrence results and regional function mining . Provide useful information for understanding urban structure.

Technical scheme of the present invention:

A method for visual analysis of travel co-occurrence phenomena based on multi-source urban data, the steps are as follows:

S1: Preprocessing the raw data

S1.1: Cleaning of taxi operation trajectory data and standardization of taxi basic data;

S1.2: Cleaning of original POI data and normalization of POI data;

S2: Carry out time and area division on the data obtained by the preprocessing of step S1

S2.1: Time division: Divide a day into T periods according to the characteristics of driving rules;

S2.2: Regional division: divide the urban space into R areas according to the urban road network;

S3: Perform regional function mining on the data divided in step S2

S3.1: Division of regional functions, classify urban regional functions into category F

S3.2: Calculate the occurrence frequency of various types of POIs in each area, use the symbol TF _i,j to represent the frequency of occurrence of the jth type of POI data in area r _i , the calculation formula is as follows:

Among them, n _i,j represent the number of POIs of the jth category in the area r _i , and F represents the number of categories of POIs;

S3.3: Calculate the inverse document frequency of POI data of the jth type, represented by IDF _j , where R represents the total number of regions, and the calculation formula is as follows:

S3.4: The multiplication of TF _i,j and IDF _j is the TF-IDF value of region r _i to the jth POI, which indicates the static function distribution of the region. The calculation formula is as follows:

TF-IDF _i,j =TF _i,j ×IDF _j

S3.5: Use the LDA topic model algorithm to carry out topic mining on the OD data in step S2, and use the final result It represents the dynamic function distribution of the region r _i , where z _i,k represents the proportion of the kth type of regional function in the region r _i ;

S3.6: Calculate the dynamic functional similarity between the region r _i and the region r _m , denoted as λ _i,m , cos represents the cosine value between vectors, the calculation formula is as follows:

S3.7: Define the following cost function J, that is, the objective function, which represents the deviation between the actual functional status of the region and its performance in both static and dynamic aspects, and calculates the minimum value of the cost function. The formula of the cost function is as follows:

Among them, R represents the total number of regions, Represents the real function distribution of the region r _i , which is also the final requirement, Represents the distribution of POIs in the region r _j ;

S4: Mining co-occurrence events through the association rule mining algorithm on the data divided in step S2

S4.1: extracting co-occurrence transactions from the data described in step S2;

S4.2: Mining frequent itemsets through the association rule Apriori algorithm for the data extracted in S4.1;

S4.3: Carry out correlation statistics calculation on the data obtained in step S4.2, each correlation statistics between area A and area B supports support, confidence degree confidence, full confidence degree all_confidence, maximum confidence degree max_confidence, Lift degree lift, Kulczynski measure Kulc, imbalance ratio IR, and cosine calculation formula are as follows, where P represents probability:

support=P(A∪B)

S5: Visual display of co-occurrence results

S5.1: According to the regional function map calculated in S3.7, use different colors to mark different regional functions on the map;

S5.2: Draw a global co-occurrence map based on the frequent itemsets mined in S4.2. The map is drawn based on two aspects: co-occurrence relationship and co-occurrence participation;

S5.3: Draw a regional co-occurrence ring heat map based on the frequent itemsets mined in S4.2. The ring heat map focuses on analyzing the co-occurrence law between regions and regions;

S5.4: Draw a parallel coordinates graph based on the statistical data calculated in S4.3. The parallel coordinates graph uses indicators to measure the correlation between two regions.

Beneficial effects of the present invention: the present invention can utilize multi-source urban data: taxi track data, urban road network data, POI data, carry out all-round and multi-angle visual analysis and exploration on regional co-occurrence phenomena and urban regional functions, providing urban traffic Planning provides effective information, which is easy to analyze the internal correlation of data and has strong operability.

Description of drawings

Fig. 1 is the structural diagram of this method;

Fig. 2 is a data processing flowchart of a visual analysis method for travel co-occurrence phenomena based on multi-source city data;

Figure 3 is a regional division diagram of a visual analysis method for travel co-occurrence phenomena based on multi-source city data;

Fig. 4 is the global visualization effect of the co-occurrence event after the co-occurrence mining is carried out using the taxi data of Shanghai in April 2015 in the implementation case of the present invention;

Fig. 5 is the global visualization effect of the co-occurrence heat after the co-occurrence mining is performed using the taxi data in Shanghai in April 2015 in the implementation case of the present invention;

Fig. 6 is the partial visualization effect of regional co-occurrence heat after co-occurrence mining using the taxi data in Shanghai in April 2015 in the implementation case of the present invention;

Fig. 7 is the visualization effect of regional function mining using Shanghai POI data and taxi data in April 2015 for the implementation of the present invention;

Fig. 8 is the visualization effect of regional similarity statistics analysis after co-occurrence mining using the taxi data of Shanghai in April 2015 in the implementation case of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the specific implementation manners of the present invention will be further described in detail below.

The embodiment of the present invention provides a method for visual analysis of travel co-occurrence phenomena based on multi-source city data. The system flow is shown in Figure 1, and the data processing flow is shown in Figure 2. The method includes:

S1: Extract useful data based on the original data set, the steps are as follows:

S1.1: The cleaning of the taxi operation trajectory data is aimed at the 30-day Shanghai taxi trajectory data from April 1, 2015 to April 30, 2015. Based on the research on the co-occurrence phenomenon, it is obvious that we need OD data of passenger taxis, so we need to extract the time of getting on and off of taxis carrying passengers, the latitude and longitude of getting on and off locations from the original data set, and the attributes of OD data include , as shown in Table 1:

Table 1

Since the distance used in the original data set is a straight-line distance, but the roads in the city are basically regular. After further analysis and comparison of the distance in the city, we discard the original distance and calculate the Manhattan distance between two points based on the original latitude and longitude. When taxis are not carrying passengers, they will slow down to find passengers, which has little impact on the law of urban movement. Therefore, we choose to filter out unloaded driving trajectories to significantly reduce the amount of data and facilitate subsequent analysis and calculation. The extracted data attributes are as follows:

Table 2

After data extraction, it was found that some of the trajectories took a long time, but the distance was very short. We judged this kind of data as abnormal data. The cleaning method was to calculate the average speed and delete the ones whose speed was too small. The calculation method of the average speed is Manhattan distance/travel time; where the travel time is calculated by the passenger boarding and alighting time, and the Manhattan distance is calculated by the latitude and longitude, and then the following attributes are added to the dataset:

table 3

Numbering name note 9 interval driving time 10 speed average speed

S1.2: The cleaning of the original POI data is mainly to extract useful information from the original data, and correct some misclassified data, while ensuring the integrity of the records. The extracted information is shown in Table 3.

Table 4

Numbering name note 1 Numbering The value is 0-110769, which uniquely identifies a piece of POI data 2 name The name of the POI data 3 latitude GPS latitude of POI 4 longitude GPS Longitude of POI 5 Third-level directory Three-level directory of POI categories

S2: For the data obtained in S1, it is necessary to divide the data by time and region, and the steps are as follows:

S2.1 The steps for time division of data are as follows:

By counting the driving time of each OD, the duration of the divided time is determined through statistical laws. For this reason, we counted the driving time of a week from April 4th to April 10th. Through statistics, we can find that about 85% of ODs have a driving time of less than 30 minutes. Therefore, we choose 30 minutes as the length of time division, so that A day is divided into 48 time zones, and the number from 0:00 to 0:30 is 0, and so on, and the number from 23:30 to 0:00 is 47. Each OD calculates its own time slice based on the boarding time of passengers. At this time, add the attribute label_time[0-47] in the data set to indicate the time slice to which the OD belongs.

table 5

Numbering name note 11 label_time OD belongs to the time period number

S2.2 The steps to divide the data into regions are as follows:

Regional division is to divide the entire research area into different areas, so that the taxi OD can be mapped to the OD between urban areas, and the distribution law of co-occurrence phenomena in urban space can be intuitively presented. In order to achieve the above goals, our algorithm must have two functions: 1) divide the urban space into regions on a plane, and number each region; 2) map it to the divided region by giving a latitude and longitude middle.

We know that urban roads are planned and constructed by the city, which divides the city into regular blocks, and these blocks often show a bias in urban functions, that is, blocks gather similar functional points. Therefore, it is reasonable to divide the city spatially by urban roads.

We selected urban roads of the second grade and above in the range of N31.15-N31.37, E121.31-E121.84 in Shanghai to divide the above ranges. The specific steps are as follows:

1) Expand the image to remove the small gaps between road intersections;

2) Thinning the expanded picture, and thinning the width of the road into one pixel;

3) Number the refined image, the algorithm numbers each pixel, and the pixels in the same area have the same number;

4) Remove the pixels representing the road from the numbered image, and the processing method is to incorporate it into the adjacent area;

Through the above processing, we obtained the regional division data of Shanghai. This division divided Shanghai into 541 regions. The division effect is shown in Figure 3. After that, we need to add two attributes of start and end area numbers to the initial OD data in S1, and add area number attributes to each POI.

Table 6

Numbering name note 12 label_start OD start area number 13 label_end OD end area number

S3: Based on the data obtained in S2, complete the mining of regional functions, the steps are as follows:

S3.1: Division of regional functions, classify regional functions into 6 categories (residential, work, education, commercial, public, service and scenic spots), and each piece of POI data will be classified into a certain category according to the three-level directory.

S3.2: Calculate the frequency of occurrence of various types of POIs in each area, use the symbol TF _i,j to indicate the frequency of occurrence of the jth type of POI data in area r _i , the calculation formula is as follows (n _i,j represents area r The number of POIs of the jth category in _i , and F represents the number of categories of POIs):

S3.3: Calculate the inverse document frequency of POI data of the jth type, represented by IDF _j , where R represents the total number of regions, and the calculation formula is as follows:

S3.4: The multiplication of TF _i,j and IDF _j is the TF-IDF value of region r _i to the jth POI, which indicates the static function distribution of the region. The calculation formula is as follows:

TF-IDF _i,j =TF _i,j ×IDF _j

S3.5: The integration of regional OD data is more in line with the real situation than taking half an hour as time slice information, and integrating OD data at an indefinite time according to driving rules. Table 7 shows the time division of working days, and Table 8 shows the time division of rest days.

Table 7

peak section start time stop the time 1 02:30:00 04:29:59 2 04:30:00 07:29:59 3 07:30:00 10:29:59 4 10:30:00 14:59:59 5 15:00:00 16:59:59 6 19:30:00 02:29:59

Table 8

Taking a certain area as the benchmark, time is the column (there are 18 time periods in total to distinguish OD inflow and outflow), and the other 541 areas are rows, and a 541*18 matrix can be obtained, and 541 such matrices can be obtained. Then merge these 541 matrices to get a matrix of 541*9738 (541*18), which is recorded as matrix D;

S3.6: Use the LDA topic model algorithm to perform topic mining on the matrix D obtained in S3.5, and use the final result It represents the dynamic function distribution of the region r _i , where z _i,k represents the proportion of the kth type of regional function in the region r _i ;

S3.7: Calculate the dynamic functional similarity between the region r _i and the region r _m , denoted as λ _i,m , cos represents the cosine value between vectors, the calculation formula is as follows:

S3.8:: Define the following cost function J, that is, the objective function, which represents the deviation between the actual functional status of the region and its performance in both static and dynamic aspects, and calculates the minimum value of the cost function. The formula of the cost function is as follows ( R represents the total number of regions, Represents the real function distribution of the region r _i , which is also the final requirement, Represents the distribution of POIs in the area r _j ):

S4: Based on the data obtained in S2, the mining of co-occurrence events is completed through the association rule mining algorithm, and the steps are as follows:

S4.1: Fetch transactions. According to the label_start, label_end, label_time in the data set, the transaction is extracted. The transaction at this time represents the area number that reached the same area within the same time period, that is:

select label_start where label_time=0 and label_start=1

The above statement extracts one transaction, so there will be 541 transactions in each time period;

S4.2: Mining frequent itemsets through the association rule Apriori algorithm for the data extracted in S4.1, the specific steps are as follows:

1) Given a support threshold q. The support threshold is to tell the algorithm what itemsets are recorded as frequent itemsets, and the itemsets whose support count is not less than the support threshold are frequent itemsets; where the support is the number of transactions in which the items in the itemset appear in a transaction at the same time. The frequent itemsets are the co-occurrence events to be mined.

2) Mining frequent 1-itemsets. A frequent 1-itemset is a frequent itemset whose item is 1. The method is to traverse all transactions, count the number of transactions in which all items appear in the transaction, and mark the 1-itemset with count >=q as frequent 1-itemset.

3) Mining frequent n-itemsets. Candidate itemsets are obtained by merging frequent n-1 itemsets in pairs. When scanning transactions, check whether the support of candidate itemsets is >=q, and if so, mark it as frequent n itemsets.

4) Cyclic mining of frequent n-itemsets until the n-itemsets have no frequent itemsets to end the cycle.

5) Save the mined co-occurrence events into files by date, and store them into frequent itemsets and their support counts through time stamps.

S4.3: Calculation of relevant statistics, representative meanings and calculation formulas are as follows:

The degree of support indicates the proportion of the number of co-occurrences between region A and region B in the total transactions, where P represents the probability. which is

support=P(A∪B)

2) Confidence, which indicates the proportion of the co-occurrence events that occur in area B and area A and the co-occurrence events that area A participates in, which is called the confidence of A->B. which is

3) full confidence, Confidence and The smaller value of the confidence. which is

4) Maximum confidence, Confidence and The larger value of the confidence. which is

5) Lift, which means the ratio of the probability of containing B under the condition of containing A to the probability of containing B under the condition of not containing A. Indicates the correlation between area A and area B. When Lift is greater than 1, it means that the two areas are positively correlated. If it is less than 1, it means that the two areas are negatively correlated. If it is equal to 1, it means that the two areas are not related. independent. which is

6) Kulc, Confidence and Confidence average. which is

7) IR, Confidence and Confidence ratio. which is

8) Cosine, which represents the probability of co-occurrence of region A and region B and the geometric mean value of the probability of occurrence of A and B.

which is

S5: Visually display the data mined from S3 and S4, the steps are as follows:

S5.1: According to the regional function map calculated in S3.7, use different colors to mark different regional functions on the map, as shown in Figure 4 and Figure 7;

S5.2: Draw a global co-occurrence map based on the frequent itemsets mined in S4.2. The map is based on the co-occurrence relationship, as shown in Figure 4, and the degree of co-occurrence participation, as shown in Figure 5, drawn from two aspects;

S5.3: Draw a regional co-occurrence ring heat map based on the frequent itemsets mined in S4.2. The ring heat map focuses on analyzing the co-occurrence law between regions and regions, as shown in Figure 6;

S5.4: Draw a parallel coordinate graph based on the statistical data calculated in S4.3. The parallel coordinate graph uses indicators to measure the correlation between the two regions, as shown in Figure 8.

The above descriptions are the specific embodiments of the present invention and the technical principles used. If the changes made according to the conception of the present invention do not exceed the spirit covered by the description and accompanying drawings, they should still be Belong to the protection scope of the present invention.

Claims (1)

1. A travel co-occurrence phenomenon visualization analysis method based on multi-source city data is characterized by comprising the following steps:

s1: preprocessing raw data

S1.1: cleaning taxi operation track data and carrying out standardized processing on taxi basic data;

s1.2: cleaning original POI data and carrying out normalized processing on the POI data;

s2: carrying out time and area division on the data obtained by the preprocessing in the step S1

S2.1: time division: dividing one day into T time intervals according to the characteristics of the driving rule;

s2.2: area division: dividing an urban space into R areas according to an urban road network;

s3: performing regional function mining on the data divided in the step S2

S3.1: regional function division, classifying urban regional functions into F class

S3.2: calculating the frequency of the POI in each region, and using the symbol TF _i,j Is shown in the region r _i The calculation formula of the frequency of the j-th POI data is as follows:

wherein n is _i,j Representative region r _i The number of POIs in the j-th category, and F represents the number of categories of the POIs;

s3.3: calculating inverse document frequency of the j-th POI data by using IDF _j Where R represents the total number of regions, the calculation formula is as follows:

S3.4：TF _i,j and IDF _j Multiplication is the region r _i For the TF-IDF value of the j-th POI, the static function distribution state of the area is represented, and the calculation formula is as follows:

TF-IDF _i,j ＝TF _i,j ×IDF _j

s3.5: performing theme mining on the OD data in the step S2 by using an LDA theme model algorithm, and using a final resultIs shown, it shows the region r _i Dynamic functional distribution of (2), wherein z _i,k Indicates that the kth class region function is in region r _i The ratio of (A) to (B);

s3.6: calculating the region r _i And region r _m The dynamic function similarity between them is recorded as lambda _i,m Cos represents the cosine value between vectors, and the calculation formula is as follows:

s3.7: defining a cost function J, namely an objective function, representing the deviation of the actually executed function status of the area from the phenomenon represented in both static and dynamic aspects, and calculating the minimum value of the cost function, wherein the formula of the cost function is as follows:

wherein, R represents the total number of regions,representative region r _i The true functional distribution of (a), is also the final requirement,representative region r _j POI distribution status of (1);

s4: mining co-occurrence events of the data divided in the step S2 through an association rule mining algorithm

S4.1: carrying out co-occurrence transaction extraction on the data in the step S2;

s4.2: mining a frequent item set of the data extracted in the S4.1 through an association rule Apriori algorithm;

s4.3: and (4) carrying out correlation statistic calculation on the data obtained in the step (S4.2), wherein the support degree and confidence degree of each correlation statistic between the region A and the region B, the full confidence all _ confidence, the maximum confidence max _ confidence, the lift, the Kulczynski metric Kulc, the imbalance ratio IR, and the cosine calculation formula are as follows, where P represents the probability:

support＝P(A∪B)

s5: visualization display co-occurrence result

S5.1: according to the regional function graph obtained by calculation in S3.7, different regional functions are marked on the map by using different colors;

s5.2: drawing a global co-occurrence map according to the frequent item set mined in the S4.2, wherein the map is drawn based on two aspects of co-occurrence relation and co-occurrence participation;

s5.3: drawing an area co-occurrence annular heat map according to the frequent item set mined in the S4.2, wherein the annular heat map focuses on analyzing the co-occurrence rule between areas;

s5.4: and drawing a parallel coordinate graph according to the statistic data obtained by calculation in the S4.3, wherein the parallel coordinate graph measures the correlation between the two areas by indexes.