CN112990284B - Individual trip behavior prediction method, system and terminal based on XGboost algorithm - Google Patents

Abstract

The invention belongs to the technical field of big data model prediction, and in particular relates to a method, system and terminal for individual travel behavior prediction based on XGBoost algorithm. The method includes the following steps: S1: Obtaining historical data used to characterize user behavior characteristics; S2: Preprocessing the acquired historical data to obtain a sample data set; S3: Constructing a prediction model of individual travel behavior based on the XGBoost algorithm; S4 : Set the hyperparameters of the prediction model, adjust the hyperparameters of the prediction model through the hierarchical three-fold cross difference device, and use the training data set to train the prediction model until the prediction model meets the requirements of the evaluation index; S5: adopt The prediction data set is used as the input sample, and the trained prediction model is used to output the sample output, and the prediction result about the user's travel behavior decision is obtained from the output of the prediction model. The method, system or terminal can solve the problem that the prior art cannot accurately predict individual travel behavior decisions.

Description

A method, system and terminal for predicting individual travel behavior based on XGBoost algorithm

technical field

The invention belongs to the technical field of big data model prediction, and in particular relates to a method, system and terminal for individual travel behavior prediction based on XGBoost algorithm.

Background technique

In recent years, with the vigorous development of mobile Internet and information technology, the importance of data has become increasingly prominent. In the process of creating a digital economic highland, data flow boosts technology flow, capital flow, talent flow, and material flow, and plays an important role in promoting social productivity and promoting innovation and development. For example, using data can help predict individual behavior or needs, which has been widely used in commercial promotion and advertisement distribution. Theoretically speaking, it should be possible to predict the travel behavior of the human body through big data, so as to provide assistance in handling some public management affairs.

In some specific scenarios, predicting the travel behavior of individuals in the crowd, so as to scientifically plan ahead for social management and other affairs based on the prediction results, has become an important issue that the relevant management departments need to solve; Gathering has an impact on transportation, catering, tourism and other industries, and it is of great significance to prevent safety incidents caused by excessive crowd gathering. However, there is currently no better method that can use existing data to achieve accurate prediction of individual travel behavior decisions.

Contents of the invention

Aiming at the problems in the prior art, the present invention provides an individual travel behavior prediction method, system and terminal based on the XGBoost algorithm. The method, system or terminal can solve the problem that the prior art cannot accurately predict individual travel behavior decisions.

In order to achieve the above object, the present invention is achieved through the following technical solutions:

A method for predicting individual travel behavior based on the XGBoost algorithm, the method comprising the following steps:

S1: Obtain historical data used to characterize user behavior characteristics. Historical data includes basic user information, user call behavior data in the past three months, user online behavior data in the past three months, and user trajectory behavior data in the past three months ;

S2: Preprocessing the acquired historical data to obtain a sample data set, and using a part of the sample data set as a pre-training data set, and the rest as a prediction data set; wherein, the pre-training data set includes a training set and a test set;

S3: Construct the prediction model of the individual travel behavior based on XGBoost algorithm; The construction method of described prediction model comprises the steps:

S31: Use the CART regression tree to construct the classifier of XGBoot; add the number of trees by continuously performing feature splitting, so as to learn new functions and fit the residual predicted by the upper side;

S32: Accumulate the scores of child nodes in a certain tree in the CART tree to obtain the sum of scores of a certain tree, and then accumulate the scores of all trees to obtain the predicted value of the sample;

S33: Construct an objective function of the algorithm model, the objective function includes a loss function part and a regular term part;

S34: Use the addition training distribution to optimize the objective function, optimize each tree in the CART in turn, and minimize the objective function on the basis of the optimal tree, and complete the construction of the loss function part;

S35: complete the definition of the regularization term in the objective function by redefining the CART tree, and determine the function of the regularization term;

S36: use the aforementioned function to obtain the optimal value of each leaf node in the CART tree, and the value of the current objective function;

S4: setting the hyperparameters of the prediction model, adjusting the hyperparameters of the prediction model through the hierarchical three-fold cross difference device, and using the pre-training data set to train the prediction model until the prediction model meets the requirements of the evaluation index;

S5: Use the prediction data set as the input sample, use the trained prediction model to output the sample output, and obtain the prediction result of the user's travel behavior decision from the output of the prediction model.

Further, the source of historical data acquired in step S1 is the government's open government data and real user data collected by communication operators.

Further, in step S2, the preprocessing process of historical data includes the following steps:

S21: Table processing of basic user information:

S211: find the abnormal value in the data variable, and perform null value processing on the abnormal value less than 0 in the column of network duration;

S212: Find the collinearity between the characteristic variables through the heat map of the characteristic variables, and delete all the columns whose collinear value exceeds 0.9 in the data;

S213: numerically process the text information in the data, and uniformly divide the terminal operating system types into Android and IOS systems, which are represented by 0 and 1 respectively;

S214: According to the Pearson correlation coefficients of all feature variables and target variables, delete the column with Pearson correlation coefficient of 0; and delete the related columns of terminal model, terminal brand and terminal first use time that are not related to training;

S215: Perform summary statistics on the characteristic variable information according to the ID of the user, and fill in the empty values with the mean or mode;

S22: Table processing of user call behavior:

S221: Delete the relevant columns of the peer number number and the call start time that are not related to the training;

S222: According to the user ID, the sum or mode of the characteristic variable is used for summary statistics, and the abnormal values in the column of the home office and the long-distance area code of the corresponding terminal number are digitized; the missing values are filled with the mode;

S23: Table processing of the user's online behavior:

S231: Delete relevant columns of application names and data dates that are not related to training; and use one-hot encoding for application classification-related columns;

S232: Summarize the number of visits and visit traffic in the characteristic variables according to the user ID, and perform summary statistics on the mode of all characteristic variables to which classification one-hot encoding is applied;

S233: Using the triple interquartile range capping method to process the outliers in the found characteristic variables;

S234: filling the null values existing in each feature variable with the mode number;

S24: Table processing of user track behavior:

S241: Use the data warehouse tool hive to mark whether the current latitude and longitude is a scenic spot according to the user ID and the two columns of whether it is a 4A-level or above scenic spot in the province, and based on this, derive the total stay time of all users, the stay time in scenic spots and the time in non-scenic spots. column data;

S242: Find the outliers in the characteristic variables, and use the three-times quarter-distance capping method to deal with the outliers

S243: filling the null values existing in the characteristic variables with the mode number;

Wherein, in the step of the preprocessing method of the related data, the abnormal value of each characteristic variable is found by drawing a box plot of the related variable.

In step S31, the CART regression tree is a hypothetical binary tree, and its expression is:

R ₁ (j,s)={x|x ^(j) ≤s} and R ₂ (j,s)={x|x ^(j) ＞s}

In the above formula, R ₁ (j,s) and R ₂ (j,s) represent the left subtree and the right subtree respectively, j represents the jth feature in the data, and s represents the segmentation point;

In the decision binary tree, if the tree node is split based on the jth eigenvalue in the data, when the eigenvalue is less than s, the sample is divided into the left subtree, and when the eigenvalue is greater than s, the sample is divided into the right subtree.

Further, in step S32, the score of a certain tree is calculated using the following function:

表示第i个样本的预测值，K表示树的棵树，F表示所有的CART树，f表示某一棵具体的CART树，f_k(x_i)为样本在某棵树中叶子节点得到的分数。In the above formula,

Represents the predicted value of the i-th sample, K represents the tree of the tree, F represents all CART trees, f represents a specific CART tree, and f _k ( _xi ) is obtained from the leaf node of the sample in a tree Fraction.

Further, in steps S33-S35, the expression of the objective function is:

In the above formula, l represents the empirical loss function of the tree model, y _i represents the true value of the i-th sample, and Ω represents the regression tree regularization term;

Among them, the left side of the above formula represents the loss function, and the right side is the regularization term;

The functional expression of the loss function is:

In the above formula, g _i represents the first-order partial derivative of the i-th leaf node, and h _{i is} the second-order partial derivative of the i-th leaf node;

The regularization term expression is:

In the above formula, γ and λ represent trade-off factors, ω _j represents the output mean of the jth leaf node, and T represents the number of leaf nodes.

Further, in step S36, the optimal value of each leaf node is calculated using the following formula:

In the above formula, G _j and H _j represent the cumulative sum of first-order partial derivatives and second-order partial derivatives of the samples contained in leaf node j respectively, both of which are constants;

The value of the objective function at this time is calculated by the following formula:

In the above formula, T represents the number of leaf nodes.

Further, in step S4, the hyperparameters that need to be set in the prediction model include iterative model category, loss function category, learning rate, depth of tree, _L1 regularization parameters, and number of iterations;

The evaluation indicators of the prediction model include the accuracy rate P, the recall rate R and the F1 value. The calculation formulas of the three are as follows:

In the above formula, TP represents the number of samples that are actually predicted as positive samples, FP represents the number of samples that are actually predicted as negative samples but are positive samples, and FN is actually the number of samples that are predicted to be negative samples from positive samples.

The present invention also includes an individual travel behavior prediction system based on the XGBoost algorithm, which uses the aforementioned individual travel behavior prediction method based on the XGBoost algorithm to realize the result prediction of the individual travel behavior; the prediction system includes:

The data acquisition module is used to obtain historical data used to characterize the user's recent behavior characteristics. The historical data includes user basic information, user call behavior data in the past three months, user online behavior data in the past three months, and user online behavior data in the past three months. The user trajectory behavior data; the collected historical data is output to the preprocessing module;

A preprocessing module, which is used to preprocess the historical data acquired by the data acquisition module to obtain the required sample data set; the sample data set is output to the behavior prediction module; and

The prediction module is used to train the prediction model based on the pre-training data set in the sample data set, and uses the prediction data set in the sample data set as input to obtain an output including the prediction result of the user's travel behavior.

The present invention also includes a personal travel behavior prediction terminal based on the XGBoost algorithm, the terminal includes a memory, a processor and a computer program stored on the memory and operable on the processor, and the processor executes the aforementioned XGBoost algorithm based individual travel behavior prediction method.

A method, system and terminal for predicting individual travel behavior based on the XGBoost algorithm provided by the present invention have the following beneficial effects:

1. The present invention builds the required forecasting model based on the XGBoost algorithm, and utilizes the user's real information big data to predict the user's future travel within the province; the original user data used for prediction is preprocessed to extract the key points required for model prediction attributes, so the accuracy and reliability of the obtained prediction results are higher, and the recall rate and F1 value of the prediction model have better performance.

2. The prediction model used in the present invention is a weighted regression model, which does not require normalization of features during the application process, can select features by itself, and can adapt to various loss functions, so it has better operability and can Reduce the workload and computational burden of data processing.

3. In the preprocessing process in the present invention, the problem of outliers and missing values in the original data is considered, and the influence of this part of data on the prediction results is reduced; when the attribute is selected, the vast majority that can be obtained through big data is considered. Part of the characteristic variables, at the same time, optimized and improved the construction of the prediction model, and optimized some parameters of the model; the above work content provided a guarantee for the accuracy and reliability of the prediction results.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the attached picture:

Fig. 1 is the method flowchart of the individual travel behavior prediction method based on XGBoost algorithm in embodiment 1 of the present invention;

Fig. 2 is the feature quantity box plot of long column when the network time of embodiment 2 of the present invention;

FIG. 3 is a thermal map of the user basic information table in Embodiment 2 of the present invention;

Fig. 4 is the table of part data in the user's online behavior data table of embodiment 2 of the present invention;

Fig. 5 is the box-whisker plot of the number of visits class in the user's online behavior data table in Embodiment 2 of the present invention;

Fig. 6 is a table of some data in the user trajectory behavior data table of Embodiment 2 of the present invention;

7 is a table of some data in the user trajectory behavior derived data table in Embodiment 2 of the present invention;

Fig. 8 is the box plot of the total dwell time in the user track behavior data of embodiment 2 of the present invention;

Fig. 9 is the change curve of F1 value with the number of iterations in the test results of the individual travel behavior prediction method based on XGBoost algorithm in Example 2 of the present invention;

Fig. 10 is a schematic module diagram of an individual travel behavior prediction system based on the XGBoost algorithm in Embodiment 3 of the present invention;

The marks in the figure are: 1. Data acquisition module; 2. Preprocessing module; 3. Prediction module.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example 1

As shown in Figure 1, the present embodiment provides a method for predicting individual travel behavior based on the XGBoost algorithm, the method includes the following steps:

S1: Obtain historical data used to characterize user behavior characteristics. Historical data includes basic user information, user call behavior data in the past three months, user online behavior data in the past three months, and user trajectory behavior data in the past three months ; The source of historical data obtained is the government's open government data and real user data collected by communication operators.

S2: Preprocess the acquired historical data to obtain a sample data set, and use part of the sample data set as a pre-training data set, and the rest as a prediction data set; wherein, the pre-training data set includes a training set and a test set; historical The data preprocessing process includes the following steps:

S21: Table processing of basic user information:

S211: find the abnormal value in the data variable, and perform null value processing on the abnormal value less than 0 in the column of network duration;

S212: Find the collinearity between the characteristic variables through the heat map of the characteristic variables, and delete all the columns whose collinear value exceeds 0.9 in the data;

S213: numerically process the text information in the data, and uniformly divide the terminal operating system types into Android and IOS systems, which are represented by 0 and 1 respectively;

S214: According to the Pearson correlation coefficients of all feature variables and target variables, delete the column with Pearson correlation coefficient of 0; and delete the related columns of terminal model, terminal brand and terminal first use time that are not related to training;

S215: Perform summary statistics on the characteristic variable information according to the ID of the user, and fill in the empty values with the mean or mode;

S22: Table processing of user call behavior:

S221: Delete the relevant columns of the peer number number and the call start time that are not related to the training;

S222: According to the user ID, the sum or mode of the characteristic variable is used for summary statistics, and the abnormal values in the column of the home office and the long-distance area code of the corresponding terminal number are digitized; the missing values are filled with the mode;

S23: Table processing of the user's online behavior:

S231: Delete relevant columns of application names and data dates that are not related to training; and use one-hot encoding for application classification-related columns;

S232: Summarize the number of visits and visit traffic in the characteristic variables according to the user ID, and perform summary statistics on the mode of all characteristic variables to which classification one-hot encoding is applied;

S233: Using the triple interquartile range capping method to process the outliers in the found characteristic variables;

S234: filling the null values existing in each feature variable with the mode number;

S24: Table processing of user track behavior:

S241: Use the data warehouse tool hive to mark whether the current latitude and longitude is a scenic spot according to the user ID and the two columns of whether it is a 4A-level or above scenic spot in the province, and based on this, derive the total stay time of all users, the stay time in scenic spots and the time in non-scenic spots. column data;

S242: Find the outliers in the characteristic variables, and use the three-times quarter-distance capping method to deal with the outliers

S243: filling the null values existing in the characteristic variables with the mode number;

Wherein, in the step of the preprocessing method of the relevant data, the outliers of each characteristic variable are found by drawing a box plot of the relevant variable.

S3: Build a prediction model of individual travel behavior based on XGBoost algorithm; XGBoost algorithm is an improved learning algorithm based on gradient boosting algorithm and decision tree. The principle is to use the idea of iterative operation to convert a large number of weak classifiers into strong classifiers to achieve accurate classification results. XGBoost is a classic method in Boosting. The purpose of the Boosting algorithm is to construct a strong classifier by integrating many weak classifiers. In this embodiment, XGBoost uses a CART regression tree.

The construction method of described predictive model comprises the following steps:

S31: The CART regression tree assumes a binary tree, which can continuously split features. If the tree node is split based on the jth eigenvalue in the data, when the eigenvalue is less than s, the sample is divided into the left subtree. When the feature When the value is greater than s, the sample is divided into right subtrees, and the expression of the CART regression tree is as follows:

R ₁ (j,s)={x|x ^(j) ≤s} and R ₂ (j,s)={x|x ^j ＞s}

In the above formula, R ₁ (j,s) and R ₂ (j,s) represent the left subtree and the right subtree respectively, j represents the jth feature in the data, and s represents the segmentation point;

S32: The idea of the XGBoost algorithm is to continuously add trees and continuously perform feature splitting to grow a tree. Through each addition, a new function can be learned to fit the residual error of the previous prediction. After training K trees, when it is necessary to predict the score of a certain sample, each tree will get a score of a child node according to the characteristics of the sample, and finally add up the corresponding scores in each tree to get the The predicted value of the sample; where the score value in a certain tree is determined by:

表示第i个样本的预测值，K表示树的棵树，F表示所有的CART树，f表示某一棵具体的CART树，f_k(x_i)为样本在某棵树中叶子节点得到的分数。In the above formula,

Represents the predicted value of the i-th sample, K represents the tree of the tree, F represents all CART trees, f represents a specific CART tree, and f _k ( _xi ) is obtained from the leaf node of the sample in a tree Fraction.

S33: The objective function is usually set to determine whether each parameter of the algorithm is optimal. In this example, the XGBoost objective function is defined as:

In the above formula, l represents the empirical loss function of the tree model, y _i represents the true value of the i-th sample, and Ω represents the regression tree regularization term;

The formula contains two parts, the left is the loss function, and the right is the regular term. The loss function ensures the difference between the measured score and the real score.

S34: After the model is established, the data needs to be trained, and the optimal parameters are found by minimizing the objective function. In this example, the additive training distribution is used to optimize the objective function. The steps are as follows:

S341: first optimize the first tree of CART, then optimize the second tree, until finally the K trees are optimized, the optimization objective function is as follows:

表示第t次迭代后样本i的预测分数，

表示前t-1棵树的预In the above formula,

Denotes the predicted score of sample i after the t-th iteration,

Indicates the prediction of the previous t-1 trees

score, f _t ( _xi ) represents the functional form of the tth tree;

S342: In the last step of optimization, an optimal CART tree f _t (xi ₎ can be obtained, which minimizes the objective function on the basis of f _t-1 ( _xi ), which satisfies the following formula:

In the above formula, constant is the complexity of the first t-1 trees.

S343: When considering that the loss function used is MSE, the above expression becomes:

As for the general function, we expand it to the second order of Taylor, then the expression of the above formula further becomes:

In the above formula:

S344: The goal of XGBoost is to minimize the objective function, so the constant item is removed, and the expression of the constructed loss function is:

In the above formula, g _i represents the first-order partial derivative of the i-th leaf node, and h _{i is} the second-order partial derivative of the i-th leaf node.

S35: Determine the definition of the regularization item in XGBoost by redefining the CART tree;

Define a CART tree as the following expression:

f _t (x)＝ω _q(x) ,ω∈R ^T ,q:R ^d →{1,2,…,T}

In the above formula, T represents the number of leaf nodes in a tree, and these values form a T-dimensional vector ω, and q(x) is a map of ω, which divides samples into a certain leaf node; ω _q(x) Indicates the predicted value of this tree for the sample.

Under the above definition, the regularization term of XGBoost is defined as:

In the above formula, γ and λ represent trade-off factors, ω _j represents the output mean value of the jth leaf node, and T represents the number of leaf nodes;

When applying the XGBoost algorithm model, the values of γ and λ can be set artificially. The larger the values of γ and λ, the simpler the model.

S35: Under the new definition, the objective function is further transformed into:

表示第i棵树对样本的预测值；In the above formula,

Indicates the predicted value of the i-th tree for the sample;

Simplify the above expression to get:

In the above formula, G _j and H _j represent the cumulative sum of first-order partial derivatives and second-order partial derivatives of the samples contained in leaf node j respectively, both of which are constants, γ and λ represent trade-off factors, and ω _j represents the jth leaf node The output mean of , T represents the number of leaf nodes;

When the structure of the t-th CART tree is determined, both G _j and H _j are determined, so the optimal value of each leaf node can be obtained by the following formula:

In the above formula, G _j and H _j represent the cumulative sum of first-order partial derivatives and second-order partial derivatives of the samples contained in leaf node j respectively, both of which are constants;

And find the value of the objective function at this time:

In the above formula, T represents the number of leaf nodes.

S4: setting the hyperparameters of the prediction model, adjusting the hyperparameters of the prediction model through the hierarchical three-fold cross difference device, and using the pre-training data set to train the prediction model until the prediction model meets the requirements of the evaluation index;

The hyperparameters that need to be set in the prediction model include iterative model category, loss function category, learning rate, tree depth, _L1 regularization parameter, and number of iterations;

The evaluation indicators of the prediction model include the accuracy rate P, the recall rate R and the F1 value. The calculation formulas of the three are as follows:

In the above formula, TP represents the number of samples that are actually predicted as positive samples, FP represents the number of samples that are actually predicted as negative samples but are positive samples, and FN is actually the number of samples that are predicted to be negative samples from positive samples.

S5: Use the prediction data set as the input sample, use the trained prediction model to output the sample output, and obtain the prediction result about the user's travel behavior decision from the output of the prediction model.

Example 2

This embodiment provides a method for predicting individual travel behavior based on the XGBoost algorithm of the embodiment, and performs a simulation experiment on the basis of implementing the method in Example 1 (in other embodiments, the simulation experiment may not be performed, or Experiment with other experimental protocols to determine relevant parameters and predictive performance for individual travel behavior).

1. Data source

In this example, the actual real operating data of Anhui operator users is used. The data for 10,000 users is provided, of which the relevant data of 7,000 users is used for training, and the relevant data of 3,000 users is used for prediction, and the training set and test set are divided from 7,000 users by ourselves.

Data files and their descriptions:

In this embodiment, the data files in the prediction model include the following components:

(1).DataPlus_Public_UserInfo_travel.csv

Explanation: This data package contains the basic information of 10,000 users, each user records one line per month, and all users have records.

(2).DataPlus_Public_Comm_travel.csv

Explanation: This data packet contains the call data of 10,000 users, each user has multiple records, and some users may have no records.

(3).DataPlus_Public_Net_travel.csv

Explanation: This data package contains the Internet access data of 10,000 users, each user has multiple records, and some users may have no records.

(4).Dataplus_Travel_Train_Trail.csv

Explanation: This data package contains the trajectory information of 10,000 users in the first 3 months (including whether they appear in the scenic spot and the name of the scenic spot). Each user has multiple rows of records, and some users may have no records. The data dictionary of this packet is shown in Table 1:

Table 1: Data dictionary for .Dataplus_Travel_Train_Trail.csv

user_id User ID Sampling & field desensitization come_time Entry time Granularity down to minutes leave_time departure time Granularity down to minutes longitude Longitude (WGS84) Field desensitization, retain 3 digits after the decimal point latitude Latitude (WGS84) Field desensitization, retain 3 digits after the decimal point poi_tag Whether it is a 4A level or above scenic spot in the province 0: no 1: yes poi_name Names of 4A-level and above scenic spots in the province

(5).Dataplus_Travel_Train_User.csv

Explanation: This data package contains 7,000 users’ travel information within the province for the next 10 days. Each user has a row of records, and all users have records; the data dictionary of this data package is shown in Table 2:

Table 2: Data dictionary of .Dataplus_Travel_Train_User.csv:

user_id User ID Sampling & field desensitization in_flag Intra-province travel results 0: no travel; 1: travel

2. Preprocessing of raw data

2.1 User basic information table processing

Each record in the original user basic information data includes features such as user ID, customer age, city of origin, and attribution display. Among them, each user has one record per month, with a total of 30,000 rows*40 columns of records.

In this embodiment, as shown in FIG. 2 , the boxplot of the data set is drawn, and it can be seen that there are abnormal values in some variables, and the initial null value processing is performed on the abnormal values less than 0 in the online time column.

Draw the heat map of all the characteristic variables of the data set shown in Figure 3. It can be seen that the collinearity between some characteristic variables is high. When processing the data, delete all columns with a collinearity value exceeding 0.9, and only keep one of them. Reduce Dimensions of the matrix.

The text information in the data set is numerically processed, and the mobile terminal operating system is divided into two major systems, Android and IOS, which are represented by 0 and 1 respectively. According to the Pearson correlation coefficient of all feature variables and target variables, the column with a Pearson correlation coefficient of 0 is deleted, and the three columns of terminal model, terminal brand and terminal first use time irrelevant to training are deleted. Finally, according to the user ID, the characteristic variable information is summarized and counted, and the empty value is filled with the average value or mode. After processing, a complete Dataframe with 7000 rows*30 columns is obtained.

2.2 User call behavior table processing

Each record in the original user call behavior data contains features such as user ID, peer number code, and call duration. Each user has multiple records, a total of 3,703,119 rows*10 columns of records, and 9,879 user records.

In this example, delete the two columns of peer number number and call start time that are not related to training, and use the sum or mode of the characteristic variable information to conduct summary statistics based on the user ID, among which the home office and peer number long-distance area code columns The outliers in are digitized, and the missing values are filled with the majority. Finally, we get a complete Dataframe with 7000 rows*8 columns.

2.3 User online behavior table processing

The original user online behavior data is shown in Figure 4. Each record contains features such as user ID, application name, and application classification. Among them, each user has multiple records, a total of 2,246,083 rows*6 columns of records, and 8,879 user records .

In this embodiment, two columns of application name and data date irrelevant to training are deleted, and the column of application classification is changed to 23 columns by one-hot encoding. According to the user's ID, the two columns of the feature variable visit times and visit traffic are summed and summarized, and all the feature variables that apply classification one-hot encoding use the mode to perform summary statistics. Draw the box plot of the characteristic data as shown in Figure 5, and find outliers with the highest number of visits reaching 6,000,000 times, and use the triple interquartile range capping method to deal with them. Finally, the existing null values are filled with the mode, and a complete Dataframe with 7000 rows*26 columns is obtained after processing.

2.4 User Track Behavior Table Processing

The original user trajectory behavior data is shown in Figure 6. Each record contains user ID, entry time, exit time and other characteristics. Each user has multiple records, with a total of 68332348 rows*7 columns of records and 9933 user records.

Considering that the trajectory behavior data set is too large, the data warehouse tool hive is used to mark whether the current latitude and longitude is a scenic spot according to the two columns of user ID and whether it is a scenic spot above 4A level in the province (poi_tag=1). As shown in Figure 7, the total stay time of all users, the stay time in scenic spots and the stay time in non-scenic spots can be derived.

Draw a boxplot of the characteristic variables in the data, as shown in Figure 8. In the boxplot, it is found that there is an outlier with the highest total residence time of 26490441.0 minutes, which has been far more than three months, and triple quartiles are used It is dealt with by spacing capping method. Finally, the existing null values are filled with the mode, and a complete Dataframe with 7000 rows*4 columns is obtained after processing.

3. Model parameter determination

In order to make the experimental results more general, this embodiment divides the data set into 20% test set and 80% training set. By adjusting the hyperparameters of the XGBoost algorithm using a hierarchical three-fold crossover differencer, the model is made more stable. The specific parameter settings are shown in Table 3:

Table 3: Hyperparameter settings for the XGBoost model

Fourth, the comparison of the results of the prediction test

In this example, XGBoost, BDT, LR and GBDT+LR are used to fuse four prediction models to conduct a control experiment of prediction results. After training, the evaluation index parameters of each model are shown in Table 2 below:

Table 2 Comparison between this embodiment and other algorithm model evaluation indicators

model type Accuracy P Recall R F1 value XGBoost 0.8809 0.9429 0.9108 LR 0.5515 0.8002 0.6530 GBDT 0.8378 0.9239 0.8787 GBDT+LR 0.8394 0.9248 0.8800

Analyzing the above test results, it is found that the prediction model based on the user’s real information big data provided by this embodiment to predict the user’s future travel in the province, compared with the three models selected for the fusion of GBDT, LR and GBDT+LR in the control group, the experiment In the results, various evaluation indicators including accuracy rate, recall rate and F1 value are better than other models. Therefore, it can be considered that the prediction method provided by this embodiment has indeed solved the problem of predicting individual travel behaviors in the prior art, and at the same time, the prediction conclusion obtained by this method has relatively high accuracy and reliability.

Draw the change curve of the F1 value of the prediction model in this embodiment as the number of iterations increases. The number of times reaches the maximum around 1100.

Example 3

As shown in Figure 10, the present embodiment provides an individual travel behavior prediction system based on the XGBoost algorithm, which uses the individual travel behavior prediction method based on the XGBoost algorithm as in Embodiment 1 to realize the result prediction of the individual travel behavior; Forecasting systems include:

The data acquisition module is used to obtain historical data used to characterize the user's recent behavior characteristics. The historical data includes user basic information, user call behavior data in the past three months, user online behavior data in the past three months, and user online behavior data in the past three months. The user trajectory behavior data; the collected historical data is output to the preprocessing module;

A preprocessing module, which is used to preprocess the historical data acquired by the data acquisition module to obtain the required sample data set; the sample data set is output to the behavior prediction module; and

The prediction module is used for the prediction model based on construction, using the training data set in the sample data set to train the prediction model, and using the prediction data set in the sample data set as input to obtain an output including the prediction result of the user's travel behavior.

Example 4

This embodiment provides an individual travel behavior prediction terminal based on the XGBoost algorithm. The terminal includes a memory, a processor, and a computer program stored in the memory and operable on the processor. The processor executes the method based on Individual travel behavior prediction method based on XGBoost algorithm.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still understand the foregoing embodiments The technical scheme recorded is modified. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (9)

1. An individual trip behavior prediction method based on an XGboost algorithm is characterized by comprising the following steps:

s1: acquiring historical data for representing user behavior characteristics, wherein the historical data comprises user basic information, user conversation behavior data of nearly three months, user internet behavior data of nearly three months and user track behavior data of nearly three months;

s2: preprocessing the acquired historical data to obtain a sample data set, using part of the sample data set as a pre-training data set, and using the rest of the sample data set as a prediction data set; wherein the pre-training data set comprises a training set and a test set;

the preprocessing process of the historical data comprises the following steps:

s21: table processing of user basic information:

s211: finding abnormal values in the data variables, and performing null value processing on the abnormal values smaller than 0 in a network time length column;

s212: finding the colinearity among the characteristic variables through the thermodynamic diagram of the characteristic variables, and deleting all columns with the colinear value exceeding 0.9 in the data;

s213: performing numerical processing on text information in data, uniformly dividing the type of a terminal operating system into an android system and an IOS system, and respectively representing the types by 0 and 1;

s214: deleting columns with the Pearson correlation coefficient of 0 according to the Pearson correlation coefficients of all the characteristic variables and the target variable; deleting the terminal model, the terminal brand and the related column of the first use time of the terminal which are irrelevant to training;

s215: summarizing and counting the characteristic variable information according to the ID of the user, and filling null values in the characteristic variable information by using an average value or a mode;

s22: table processing of user call behaviors:

s221: deleting the number of the opposite terminal and the related column of the call starting time which are irrelevant to the training;

s222: summarizing and counting the characteristic variables by adopting a sum or a mode according to the user ID, and carrying out digital processing on abnormal values in the columns of the long-distance area numbers of the home office and the opposite terminal number; filling missing values by adopting mode;

s23: and (3) table processing of user internet behavior:

s231: deleting the related columns of application names and data dates which are irrelevant to training; and the application classification related column is processed by single-hot coding;

s232: summing and summarizing the access times and the access flow in the characteristic variables according to the user ID, and summarizing and counting the use mode of all the characteristic variables applying the classified one-hot codes;

s233: processing abnormal values in the found characteristic variables by adopting a three-time quartile spacing capping method;

s234: filling null values existing in each characteristic variable by adopting a mode;

s24: table processing of user trajectory behavior:

s241: whether the current longitude and latitude is the scenic spot is marked by a data warehouse tool hive according to the user ID and whether the province is in two scenic spots above the level 4A, and three columns of data of the total stay time of all users, the scenic spot stay time and the non-scenic spot stay time are derived based on the current longitude and latitude;

s242: finding abnormal values in the characteristic variables, and processing the abnormal values by adopting a triple-quarter-pitch capping method

S243: filling null values existing in the characteristic variables by adopting a mode;

in the relevant data preprocessing method step, abnormal values of all characteristic variables are discovered by a method of drawing a relevant variable box line graph;

s3: constructing a prediction model of individual travel behaviors based on an XGboost algorithm; the construction method of the prediction model comprises the following steps:

s31: constructing a classifier of the XGboot by using a CART regression tree; adding the number of trees by continuously performing feature splitting, so as to learn a new function and fit a predicted residual error of the previous layer;

s32: accumulating the scores of sub-nodes in a certain tree in the CART tree to obtain the score sum of the certain tree, and accumulating the scores of all the trees to obtain the predicted value of the sample;

s33: constructing an objective function of an algorithm model, wherein the objective function comprises a loss function part and a regular term part;

s34: using addition training distribution to optimize a target function, sequentially optimizing each tree in the CART, minimizing the target function on the basis of an optimal tree, and completing the construction of a loss function part;

s35: the definition of a regularization item in the objective function is completed through the redefinition of the CART tree, and the function of the regularization item part is determined;

s36: the function is utilized to obtain the optimal value of each leaf node in the CART tree and the value of the current objective function;

s4: setting a hyper-parameter of a prediction model, carrying out hyper-parameter adjustment on the prediction model through a layered three-fold cross differentiator, and training the prediction model by utilizing a pre-training data set until the prediction model meets the requirement of an evaluation index;

s5: and (3) adopting the prediction data set as an input sample, outputting the sample output by using the trained prediction model, and obtaining a prediction result about the user trip behavior decision from the output of the prediction model.

2. The individual travel behavior prediction method based on the XGBoost algorithm of claim 1, characterized in that: the historical data sources acquired in the step S1 are government open data and real user data collected by a communication carrier.

3. The individual travel behavior prediction method based on the XGBoost algorithm of claim 1, characterized in that: in step S31, the CART regression tree is an assumed binary tree, and its expression is:

R ₁ (m,s)＝{xx ^(m) ≤s}and R ₂ (m,s)＝{xx ^(m) ＞s}

in the above formula, R ₁ (m, s) and R ₂ (m, s) respectively represent a left sub-tree and a right sub-tree, m represents the mth feature in the data, and s represents a cut point;

in the decision binary tree, if the tree node is split based on the mth feature in the data, when the feature value is smaller than s, the sample is divided into a left subtree, and when the feature value is greater than s, the sample is divided into a right subtree.

4. The individual travel behavior prediction method based on the XGBoost algorithm of claim 1, characterized in that: in step S32, the score of a certain tree is calculated by using the following function:

in the above formula, the first and second carbon atoms are,

represents the ith sampleIn the predicted value, K represents a tree of the tree, F represents all CART trees, F represents a specific CART tree, F _k (x _i ) The scores obtained for leaf nodes of a sample in a certain tree.

5. The individual travel behavior prediction method based on the XGBoost algorithm of claim 1, characterized in that: in the steps S33 to S35, the expression of the objective function is:

in the above formula, l represents the empirical loss function of the tree model, y _i Representing the real value of the ith sample, and omega represents a regression tree regularization item;

wherein, the left side of the above formula represents a loss function, and the right side is a regularization term;

the function expression of the loss function is:

in the above formula, g _i Represents the first order partial derivative of the ith leaf node, h _i Second order partial derivatives of the ith leaf node;

the regularization term expression is:

in the above formula, γ and λ represent trade-off factors, ω _j Represents the output average value of the jth leaf node, and T represents the number of leaf nodes.

6. The individual travel behavior prediction method based on the XGBoost algorithm of claim 1, characterized in that: in step S36, the optimal value of each leaf node is calculated by using the following formula:

in the above formula, G _j And H _j Respectively representing the sum of the first-order partial derivatives and the second-order partial derivatives of the samples contained in the leaf node j, wherein the sum is a constant;

the value of the objective function at this time is calculated by the following equation:

in the above equation, T represents the number of leaf nodes.

7. The individual travel behavior prediction method based on the XGBoost algorithm of claim 1, characterized in that: in step S4, the hyper-parameters to be set in the prediction model include an iteration model category, a loss function category, a learning rate, a tree depth, and L ₁ Regularization parameters and iteration times;

the evaluation indexes of the prediction model comprise an accuracy P, a recall rate R and an F1 value, and the calculation formulas of the accuracy P, the recall rate R and the F1 value are as follows:

in the above equation, TP represents the number of samples for which positive samples are actually predicted as positive samples, FP represents the number of samples for which negative samples are actually predicted as positive samples, and FN represents the number of samples for which positive samples are actually predicted as negative samples.

8. An individual travel behavior prediction system based on an XGboost algorithm is characterized in that the system adopts the individual travel behavior prediction method based on the XGboost algorithm according to any one of claims 1-7 to realize result prediction of individual travel behaviors; the prediction system comprises:

the data acquisition module is used for acquiring historical data for representing recent behavior characteristics of the user, wherein the historical data comprises basic information of the user, conversation behavior data of the user in nearly three months, internet surfing behavior data of the user in nearly three months and track behavior data of the user in nearly three months; outputting the historical data to a preprocessing module;

the preprocessing module is used for preprocessing the historical data acquired by the data acquisition module to obtain a required sample data set; the sample data set is output to a behavior prediction module; and

and the prediction module is used for training the prediction model by adopting a pre-training data set in the sample data set based on the constructed prediction model, and acquiring output containing a user travel behavior prediction result by adopting the prediction data set in the sample data set as input.

9. An individual travel behavior prediction terminal based on an XGboost algorithm, which is characterized by comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, and is characterized in that: the processor executes the individual travel behavior prediction method based on the XGboost algorithm according to any one of claims 1 to 7.

Images