CN115169705A - Distribution time length prediction method and device, storage medium and computer equipment - Google Patents
Distribution time length prediction method and device, storage medium and computer equipment Download PDFInfo
- Publication number
- CN115169705A CN115169705A CN202210822160.9A CN202210822160A CN115169705A CN 115169705 A CN115169705 A CN 115169705A CN 202210822160 A CN202210822160 A CN 202210822160A CN 115169705 A CN115169705 A CN 115169705A
- Authority
- CN
- China
- Prior art keywords
- model
- order
- delivery
- characteristic data
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/067—Enterprise or organisation modelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/109—Time management, e.g. calendars, reminders, meetings or time accounting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
- G06Q30/0635—Processing of requisition or of purchase orders
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- Software Systems (AREA)
- Game Theory and Decision Science (AREA)
- Data Mining & Analysis (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Educational Administration (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a distribution duration prediction method, a distribution duration prediction device, a storage medium and computer equipment, and relates to the technical field of instant distribution. The method comprises the following steps: acquiring characteristic data of a sample order, wherein the characteristic data comprises characteristic data corresponding to at least one distribution estimation model and/or newly added characteristic data; training a two-classification decision tree model based on the characteristic data of the sample order to obtain an order classification model and the characteristic importance of at least one type of characteristic data; performing optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery estimation model; obtaining order information of the target order, and obtaining a prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery estimation model. The method can effectively improve the iteration speed and the iteration effect of the delivery prediction model and improve the prediction accuracy of the delivery duration.
Description
Technical Field
The invention relates to the technical field of instant delivery, in particular to a delivery duration prediction method, a delivery duration prediction device, a storage medium and computer equipment.
Background
The distribution time length of instant distribution is commonly influenced by a plurality of time length estimation models and a dispatching assignment model, and in the models, various models such as an organic learning model and a deep learning model are limited at present, and the distribution time length estimation can be more accurate only by continuously updating iteration. When the models need to be iterated, the main reason for inaccurate estimation of the delivery duration needs to be found out, and the reason needs to be positioned on a specific model or even a specific characteristic.
In the prior art, problem root cause analysis is usually performed by adopting a statistical attribution mode, namely, during statistical attribution, statistics is performed on various factors such as the real segment time length, the estimated segment deviation, the types of the products and the like of an overtime order, then the reason of inaccurate estimation of the distribution time length is reversely deduced according to the statistical result, and finally each model is optimized according to the deduced reason. However, the factors attributed to the statistics are independent, and a large amount of manual work is required to analyze the statistical data, which results in low efficiency of the root cause analysis, and in addition, the attributed analysis result also difficultly gives a credible specific reason with primary and secondary relationships, which results in low accuracy of the root cause analysis, slow progress of model iteration and unsatisfactory iteration effect, and also results in that the estimated instant delivery time length of the optimized model is still inaccurate.
Disclosure of Invention
In view of this, the present application provides a method, an apparatus, a storage medium, and a computer device for predicting a delivery duration, and mainly aims to solve the technical problem that an instant delivery duration prediction is inaccurate.
According to a first aspect of the present invention, there is provided a delivery duration prediction method, the method comprising:
acquiring characteristic data of a sample order, wherein the characteristic data comprises characteristic data corresponding to at least one distribution estimation model and/or newly added characteristic data;
training a two-classification decision tree model based on the characteristic data of the sample order to obtain an order classification model and the characteristic importance of at least one type of characteristic data;
performing optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery estimation model;
obtaining order information of the target order, and obtaining a prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery estimation model.
Optionally, the delivery prediction model includes at least one of a segmented delivery duration prediction model, an overall delivery duration prediction model and a scheduling assignment model; the obtaining of the characteristic data of the sample order includes: the method comprises the steps of obtaining a plurality of orders in a preset time period as sample orders, dividing overtime orders in the sample orders into positive samples, and dividing non-overtime orders in the sample orders into negative samples; acquiring characteristic data and/or at least one newly added characteristic data corresponding to at least one of the segmental delivery time length estimation model, the overall delivery time length estimation model and the scheduling assignment model; and extracting the feature data in the sample order according to the feature data corresponding to the at least one model and/or the newly added feature data to obtain the feature data of the sample order.
Optionally, the training a two-classification decision tree model based on the feature data of the sample order to obtain an order classification model and a feature importance of at least one of the feature data includes: dividing the sample order into a training set and a verification set; training the two classification decision tree models based on the characteristic data of the sample orders in the training set to obtain an initial order classification model; carrying out classification prediction on the sample orders in the verification set through the initial order classification model to obtain a classification prediction result of the sample orders in the verification set; calculating a model index of the initial order classification model according to the classification prediction result of the sample orders in the verification set; and performing optimization training on the initial order classification model according to the model indexes of the initial order classification model to obtain the order classification model and the feature importance of at least one type of feature data.
Optionally, the performing optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery estimation model includes: screening at least one important feature data from the feature data according to the feature importance of the feature data and the model structure of the order classification model; and optimizing and training at least one distribution estimation model of the segmented distribution time length estimation model, the overall distribution time length estimation model and the scheduling assignment model based on the important characteristic data to obtain an optimized distribution estimation model.
Optionally, the screening, according to the feature importance of the feature data and the model structure of the order classification model, at least one important feature data from the feature data includes: sorting the feature data according to the feature importance of the feature data, and obtaining at least one pre-selected feature data according to a sorting result; converting the model structure of the order classification model into a tree structure, and counting the positive sample proportion of each branch structure in the tree structure; and verifying the feature importance of the preselected feature data according to the positive sample proportion of each branch structure in the tree structure, and obtaining the important feature data according to a verification result.
Optionally, the performing optimization training on at least one delivery prediction model of the segmented delivery duration prediction model, the overall delivery duration prediction model, and the scheduling assignment model based on the important feature data to obtain an optimized delivery prediction model includes: carrying out optimization training on parameters of a delivery estimation model with the model output as the important characteristic data to obtain the optimized delivery estimation model; and/or adding the important characteristic data into a delivery prediction model corresponding to the important characteristic data, and carrying out optimization training on the delivery prediction model to obtain the optimized delivery prediction model.
Optionally, the obtaining the order information of the target order and obtaining the prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery prediction model include: acquiring order information of a target order and characteristic data corresponding to the optimized delivery estimation model; extracting characteristic data in the target order according to the characteristic data corresponding to the optimized delivery estimation model; and inputting the characteristic data of the target order into the optimized delivery estimation model to obtain a prediction result of the delivery duration of the target order.
According to a second aspect of the present invention, there is provided a delivery duration prediction apparatus comprising:
the system comprises a characteristic acquisition module, a characteristic analysis module and a characteristic analysis module, wherein the characteristic acquisition module is used for acquiring characteristic data of a sample order, and the characteristic data comprises at least one type of characteristic data corresponding to a distribution estimation model and/or newly added characteristic data;
the characteristic evaluation module is used for training a two-classification decision tree model based on the characteristic data of the sample order to obtain an order classification model and the characteristic importance of at least one type of characteristic data;
the model optimization module is used for carrying out optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery estimation model;
and the duration prediction module is used for acquiring order information of the target order and obtaining a prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery prediction model.
Optionally, the delivery prediction model includes at least one of a segmented delivery duration prediction model, an overall delivery duration prediction model and a scheduling assignment model; the characteristic obtaining module is specifically configured to obtain a plurality of orders within a preset time period as sample orders, divide overtime orders in the sample orders into positive samples, and divide non-overtime orders in the sample orders into negative samples; acquiring characteristic data and/or at least one newly added characteristic data corresponding to at least one of the segmental delivery time length estimation model, the overall delivery time length estimation model and the scheduling assignment model; and extracting the characteristic data in the sample order according to the characteristic data corresponding to the at least one model and/or the newly added characteristic data to obtain the characteristic data of the sample order.
Optionally, the feature evaluation module is specifically configured to divide the sample order into a training set and a verification set; training the two-classification decision tree model based on the characteristic data of the sample orders in the training set to obtain an initial order classification model; carrying out classification prediction on the sample orders in the verification set through the initial order classification model to obtain a classification prediction result of the sample orders in the verification set; calculating a model index of the initial order classification model according to the classification prediction result of the sample orders in the verification set; and performing optimization training on the initial order classification model according to the model indexes of the initial order classification model to obtain the order classification model and the feature importance of at least one type of feature data.
Optionally, the model optimization module is specifically configured to screen at least one important feature data from the feature data according to the feature importance of the feature data and the model structure of the order classification model; and performing optimization training on at least one delivery estimation model of the segmented delivery time length estimation model, the overall delivery time length estimation model and the scheduling assignment model based on the important characteristic data to obtain an optimized delivery estimation model.
Optionally, the model optimization module is specifically configured to sort the feature data according to the feature importance of the feature data, and obtain at least one piece of preselected feature data according to a result of the sorting; converting the model structure of the order classification model into a tree structure, and counting the positive sample proportion of each branch structure in the tree structure; and verifying the feature importance of the preselected feature data according to the positive sample ratio of each branch structure in the tree structure, and obtaining the important feature data according to a verification result.
Optionally, the model optimization module is specifically configured to perform optimization training on parameters of a delivery prediction model, which is output by the model as the important feature data, to obtain the optimized delivery prediction model; and/or adding the important characteristic data into a delivery prediction model corresponding to the important characteristic data, and carrying out optimization training on the delivery prediction model to obtain the optimized delivery prediction model.
Optionally, the duration prediction module is specifically configured to obtain order information of the target order and feature data corresponding to the optimized delivery prediction model; extracting characteristic data in the target order according to the characteristic data corresponding to the optimized delivery estimation model; and inputting the characteristic data of the target order into the optimized delivery estimation model to obtain a prediction result of the delivery duration of the target order.
According to a third aspect of the present invention, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the delivery duration prediction method described above.
According to a fourth aspect of the present invention, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the delivery duration prediction method when executing the program.
According to the distribution duration prediction method, the distribution duration prediction device, the storage medium and the computer equipment, firstly, the feature data of a sample order is obtained, based on the features currently used by a distribution prediction model and other newly-added features which may cause inaccurate predicted distribution time, a two-classification decision tree model with interpretability is trained to obtain an order classification model, then the reason of the inaccurate model prediction is positioned on a specific model and specific features by combining the feature importance and the model structure of the order classification model, so that a relevant model is optimized, and finally, the optimized model is used for predicting a target order to obtain the prediction result of the distribution duration of the target order. The method effectively improves the efficiency and accuracy of root cause analysis of the delivery estimation model, especially improves the efficiency and accuracy of root cause analysis under a multi-model scene, thereby providing optimization direction and priority for optimization of the delivery estimation model, effectively improving the iteration speed and the iteration effect of the delivery estimation model, and finally improving the prediction accuracy of delivery duration.
The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical solutions of the present application more clearly understood, the present application may be implemented in accordance with the content of the description, and in order to make the foregoing and other objects, features, and advantages of the present application more clearly understood, the following detailed description of the present application is provided.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention to a lesser extent. In the drawings:
fig. 1 is a schematic flowchart illustrating a delivery duration prediction method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a model structure of an order classification model according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram illustrating a delivery duration prediction apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
In one embodiment, as shown in fig. 1, a delivery duration prediction method is provided, which is described by taking the method as an example applied to a computer device, and includes the following steps:
101. and acquiring characteristic data of the sample order, wherein the characteristic data comprises characteristic data corresponding to at least one distribution estimation model and/or newly added characteristic data.
The feature data is used to describe the attributes of the sample, and generally, one sample has a plurality of feature data. For the sample order, there are a plurality of characteristic data, some of which may be obtained from order information of the sample order, such as order time, order delivery address, user information, commodity category, merchant information, distributor information, etc., and some of which may be obtained after calculation and missing value filling, such as distributor pressure coefficient, distributor one week history order quantity, etc.
In this embodiment, the feature data to be obtained may include feature data corresponding to at least one distribution estimation model and/or newly added feature data, where the newly added feature data refers to feature data that is not used by all distribution estimation models. Specifically, at least one delivery prediction model required for prediction of delivery duration can be combed in advance, including a meal delivery duration prediction model, an overall delivery duration prediction model, a scheduling assignment model and the like, then the currently used characteristics of each delivery prediction model are combed, and new characteristics which may cause order overtime and are not used by the delivery prediction model are combed according to actual conditions, such as current meal delivery blocking degree of a merchant, meal delivery duration prediction deviation, delivery section duration prediction deviation and the like.
Further, a plurality of order data may be obtained, and the order which does not meet the sample requirements, such as the order which cannot calculate the real meal delivery time length, the order with unreal delivery time, the reserved order, and the like, in the sample order may be filtered, so as to keep the order which can calculate the real meal delivery time length as the sample order, and then, the sample order may be divided into a positive sample and a negative sample according to a certain attribute of the order, in this embodiment, the sample order may be divided according to whether the order is overtime, for example, the overtime order may be divided into the positive sample, the order which is not overtime divided into the negative sample, and the ratio of the positive sample and the negative sample may be adjusted, for example, the ratio of the positive sample and the negative sample may be adjusted to 1:1, and further, the feature data in the sample order may be extracted, calculated, and filled with missing values according to the feature data corresponding to the screened at least one delivery pre-estimation model and/or the newly added feature data, so as to obtain a plurality of feature data of the plurality of sample orders.
102. And training a two-classification decision tree model based on the characteristic data of the sample order to obtain the order classification model and the characteristic importance of at least one characteristic data.
The binary Decision Tree model refers to a classification model for processing a binary classification task, such as a Gradient Boost Decision Tree (GBDT), a Gradient descent Decision Tree, and the like. In this embodiment, the order classification model can be obtained by training a two-classification decision tree model, so that the order classification model has good interpretability, where the interpretability refers to how much a human can understand how the model predicts. Further, through the trained order classification model, the feature importance of at least one feature data can be obtained, wherein the feature importance refers to the importance of the feature data in the order classification model, and the higher the feature importance of one feature data is, the greater the effect of the feature data on the order classification model is.
Specifically, a plurality of sample orders can be split into a training set and a verification set, a two-class decision tree model is trained by using the sample orders in the training set to obtain an initial order classification model, then result prediction is carried out based on the sample orders in the verification set, effect evaluation is carried out based on the prediction result, model indexes such as AUC (index for evaluating classification performance of the model), KS (index for evaluating discrimination of the model), F1 Score (harmonic mean of accuracy and recall) and the like are obtained, finally, the classification performance, the discrimination, the accuracy and the recall ratio and the like of the initial order classification model are evaluated through each model index, whether the model is continuously optimized or not is determined according to the performance evaluation result, and if the model does not need to be optimized, the initial order classification model can be used as an order classification model, and the feature importance of each feature data is obtained; if the model needs to be optimized, the optimized model can be used as an order classification model, and the feature importance of each feature data is obtained.
In this embodiment, in the training process of the order classification model, the feature importance of various feature data may be obtained, and in a normal case, the feature importance of the feature data with the feature importance ranking within the top 200 names may be queried, for example, after query, the feature importance of the actual meal delivery duration may be 0.50493008, the feature importance of the estimated deviation of the duration of the delivery section may be 0.11467534, the feature importance of the single historical amount of the rider for one week may be 0.07034242, the feature importance of the rider pressure coefficient may be 0.06854081, the feature importance of the estimated meal duration may be 0.0610223, and the like. In this embodiment, according to the ranking result of the feature importance, several feature data with higher feature importance may be taken as the optimization direction for distributing the predictive model.
103. And carrying out optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain the optimized delivery estimation model.
Specifically, important feature data which influences order overtime can be determined based on feature data with high feature importance, then the model structure of the order classification model is converted into a readable tree structure, and the importance of each important feature data in the model is verified through the overtime order proportion of each branch in the tree structure. For example, the tree structure after the model structure conversion of the order classification model may be as shown in fig. 2, and by expanding the tree structure, it can be analyzed that: when the estimated minimum of the distribution section is more than 10min and the meal delivery time is smaller, the probability of overtime orders is increased by 0.328; when the estimated size of the distribution section is smaller than 10min and the meal delivery time is larger than 2min, the probability of overtime orders is increased by 0.11; when the estimated delivery section is smaller than 20min, the overtime order probability is increased by 0.231. It can be seen from the above that, the estimated deviation value of the delivery section and the actual meal delivery time both have a great influence on the probability of the overtime order, and therefore, both the estimated deviation value of the delivery section and the actual meal delivery time can be used as important characteristic data influencing the overtime of the order.
Furthermore, by combining the feature importance of each feature data and the importance of the feature data in the order classification model, a plurality of feature data influencing order overtime can be obtained, so that the optimization training can be performed on at least one delivery estimation model according to the plurality of feature data influencing order overtime. In this embodiment, the delivery prediction model may include a meal length prediction model, an overall delivery length prediction model, a scheduling assignment model, and the like, and the importance of the feature data in the order classification model may be obtained according to the overtime order proportion of each branch in the tree structure. For example, by combining the importance of the features and the influence of the features on the overtime order proportion in the order classification model, the main reasons of order overtime can be found out to be the actual meal delivery duration and the estimated variation of the meal delivery duration, and then the estimated variation of the delivery duration, the familiarity of a rider to a user area, the rider pressure and the like. Furthermore, when the existing distribution estimation model is optimized according to the characteristic data, the characteristic data which cannot be changed and is available, such as the actual meal time, the rider pressure and other characteristic data, can not be set as an optimization target; the newly added features can be added into a delivery estimation model related to the features, such as adding the familiarity of a rider to a user area into a scheduling assignment model and optimizing the scheduling assignment model; the parameters of the model with the model output as important characteristic data can be optimized, for example, the parameters of the meal delivery duration estimation model with the meal delivery duration estimation deviation output by the optimization model, the parameters of the overall delivery duration estimation model with the delivery section duration estimation deviation output by the optimization model, and the like.
It should be noted that the method for optimizing the delivery forecast model through the feature data is not limited to the above contents, and a technician may select a specific optimization scheme of the delivery forecast model according to an actual situation, wherein the model subjected to optimization training may be one or more models in the delivery forecast model, and the finally obtained optimized delivery forecast model includes all models for predicting delivery duration, including but not limited to various segmental delivery duration forecast models, an overall delivery duration forecast model, a scheduling assignment model, and the like. By the accurate problem root cause analysis method covering multiple models, the root cause of the order overtime problem can be positioned on a specific model or even a specific characteristic.
104. And obtaining order information of the target order, and obtaining a prediction result of the distribution duration of the target order according to the order information of the target order and the optimized distribution prediction model.
Specifically, after the delivery estimation models are optimized, the feature data in the target order can be extracted, calculated, filled with missing values and the like according to the features corresponding to the optimized delivery estimation models, and then the feature data of the target order is input into the optimized delivery estimation models to obtain the prediction result of the delivery duration of the target order, namely the predicted delivery duration of the target order.
The delivery duration prediction method provided by this embodiment includes obtaining feature data of a sample order, training an interpretable binary decision tree model based on features currently used by a delivery estimation model and other newly-added features that may cause inaccurate delivery duration prediction, obtaining an order classification model, locating the cause of the inaccurate model prediction to a specific model and specific features by combining feature importance and a model structure of the order classification model, optimizing a related model, and predicting a target order by using the optimized model, thereby obtaining a prediction result of the delivery duration of the target order. The method effectively improves the efficiency and accuracy of root cause analysis of the distribution pre-estimation model, particularly improves the efficiency and accuracy of root cause analysis under a multi-model scene, thereby providing the optimization direction and priority for the optimization of the distribution pre-estimation model, effectively improving the iteration speed and the iteration effect of the distribution pre-estimation model, and finally improving the long prediction accuracy during distribution.
In an embodiment, step 101 may be specifically implemented by the following method: the method comprises the steps of obtaining a plurality of orders in a preset time period as sample orders, dividing overtime orders in the sample orders into positive samples, and dividing non-overtime orders in the sample orders into negative samples; acquiring characteristic data and/or at least one newly added characteristic data corresponding to at least one of the segmental delivery time length estimation model, the overall delivery time length estimation model and the scheduling assignment model; and extracting the characteristic data in the sample order according to the characteristic data corresponding to the at least one model and/or the newly added characteristic data to obtain the characteristic data of the sample order.
In the above embodiment, the overtime order and the non-overtime order in the last period (for example, in the last 2 weeks) may be selected as the positive sample and the negative sample, respectively, and then the obtained sample orders may be filtered. Specifically, when filtering sample orders, the team orders can be retained, the reservation orders are filtered, fraud orders are filtered, orders arriving at a store and with unreal delivery time are filtered, orders with unreal delivery time are filtered, and orders with unreal delivery time can be retained, wherein the orders with real delivery time can be calculated. Further, the ratio of positive and negative samples may be counted, and if the ratio of positive and negative samples is not uniform, the sampling ratio may be determined in advance, for example, in an actual online order, the ratio of an overtime order to an untime order is about 1:9, and by removing most untime orders, the ratio of an overtime order to an untime order may be adjusted to 1:1. Furthermore, at least one delivery estimation model required for delivery duration estimation can be combed out in advance, the delivery estimation model comprises a sectional duration estimation model, an overall delivery duration estimation model, a scheduling assignment model and the like, then the current use characteristics of each delivery estimation model are combed out, and in addition, other new characteristics possibly causing order overtime can be combed out by combining with actual conditions, such as the current meal delivery blocking degree of a merchant, meal delivery duration estimation deviation, delivery section duration estimation deviation, rider pressure and the like. Furthermore, according to the feature data corresponding to each screened delivery estimation model and/or the newly added feature data, the feature data in the sample order can be extracted, calculated and filled with missing values, so that various feature data of a plurality of sample orders can be obtained.
In the embodiment, the types of the feature data can be as many as possible, so that more accurate feature data influencing the distribution prediction model can be found in the subsequent steps, and more optimization directions are provided for the optimization of the subsequent distribution prediction model. In addition, each delivery estimation model of the segmented delivery duration estimation model, the overall delivery duration estimation model and the scheduling assignment model may further include a plurality of models, for example, the segmented delivery duration estimation model may further include a meal duration estimation model, a delivery segment duration estimation model and the like, and each model may also have a certain connection relationship, for example, the output of the scheduling assignment model may be used as the input of the delivery segment duration estimation model and the like. In the above embodiment, the non-timeout order may be used as the positive sample, the timeout order may be used as the positive sample, and other steps may be unchanged.
In one embodiment, step 102 may be specifically implemented by the following method: dividing the sample order into a training set and a verification set; training the two-classification decision tree model based on the characteristic data of the sample orders in the training set to obtain an initial order classification model; carrying out classification prediction on the sample orders in the verification set through the initial order classification model to obtain a classification prediction result of the sample orders in the verification set; calculating a model index of the initial order classification model according to the classification prediction result of the sample orders in the verification set; and performing optimization training on the initial order classification model according to the model indexes of the initial order classification model to obtain the order classification model and the feature importance of at least one type of feature data. In the above embodiment, the sample orders are divided into the training set and the verification set, and the two-class decision tree model is trained and evaluated by using the sample orders in the training set and the verification set, respectively, so that the classification performance of the order classification model can be improved, and the feature importance evaluation result of the feature data is more accurate.
In one embodiment, step 103 may be specifically implemented by the following method: screening at least one important feature data from the feature data according to the feature importance of the feature data and the model structure of the order classification model; and performing optimization training on at least one delivery estimation model of the segmented delivery duration estimation model, the overall delivery duration estimation model and the scheduling assignment model based on the important characteristic data to obtain an optimized delivery estimation model. In this embodiment, at least one feature data may be screened out as the important feature data according to the feature importance of each feature data and the importance of the feature data in the order classification model, for example, the feature importance may be screened out to be greater than a first threshold, and one or more feature data whose overtime order percentage exceeds a second threshold may be changed in the order classification model as the important feature data, and then one or more models in the delivery prediction model may be optimized according to the relationship between the important feature data and each delivery prediction model to obtain the optimized delivery prediction model. In this embodiment, both the screening method of the important feature data and the optimization method of the distribution estimation model may be adjusted and freely selected according to the actual situation, and this embodiment is not particularly limited.
In one embodiment, the method for screening the important feature data in step 103 can be implemented by the following steps: sorting the feature data according to the feature importance of the feature data, and obtaining at least one piece of preselected feature data according to a sorting result; converting the model structure of the order classification model into a tree structure, and counting the positive sample proportion of each branch structure in the tree structure; and verifying the feature importance of the preselected feature data according to the positive sample proportion of each branch structure in the tree structure, and obtaining the important feature data according to a verification result. In the embodiment, a plurality of preselected feature data can be screened out quickly by sequencing the feature data according to the feature importance, the influence of each preselected feature data on the positive sample ratio can be checked more intuitively by converting the model structure of the order classification model into the tree structure and counting the positive sample ratio of each branch structure in the tree structure, so that whether each preselected feature data is an important factor influencing order overtime can be verified, if the change of a certain preselected feature data on the positive sample ratio exceeds a preset threshold value, the preselected feature data can be determined as the important feature data, otherwise, the preselected feature data can be removed from the important feature data.
In one embodiment, the method for optimizing the delivery forecast model in step 103 may be implemented by the following steps: optimizing and training parameters of a delivery estimation model with the model output as the important characteristic data to obtain the optimized delivery estimation model; and/or adding the important characteristic data into a delivery estimation model corresponding to the important characteristic data, and performing optimization training on the delivery estimation model to obtain the optimized delivery estimation model. In the above embodiment, important feature data that cannot be input into the model as a feature or cannot be an optimization target, such as an actual meal length, may not be processed; for important feature data used in the distribution estimation model, such as rider pressure, the feature can be temporarily not processed, or the model using the feature can be optimized to improve the priority of the feature in the model; for newly added important feature data, namely features which are not used by each model, the feature data can be added into the model corresponding to the feature and the model is optimized, for example, the familiarity of a rider to a user area can be added into the scheduling assignment model, and the scheduling assignment model is optimized, so that the accuracy of the scheduling assignment model is improved, and the accuracy of the distribution duration prediction is improved; for a model with output of the model as important characteristic data, parameters of the model can be optimized to enable the output of the model to be more accurate, generally, the model is an intermediate model, that is, the output of the model can be used as input of another model, if the output of the intermediate model is important characteristic data, the intermediate model needs to be optimized to enable the output of the model to be more accurate, for example, if estimated deviation of meal delivery duration is important characteristic data, parameters of a meal delivery duration estimation model which outputs estimated deviation of meal delivery duration need to be optimized to enable estimated deviation of meal delivery duration output by the model to be more accurate, and therefore accuracy of delivery duration prediction is improved. It is understood that the optimization-trained model may be one or more delivery prediction models, and all models affecting the delivery duration prediction result as a whole may be referred to as an optimized delivery prediction model, where at least one model of the optimized delivery prediction models is subjected to the optimization training process.
In one embodiment, step 104 may be specifically implemented by the following method: acquiring order information of a target order and characteristic data corresponding to the optimized delivery estimation model; extracting characteristic data in the target order according to the characteristic data corresponding to the optimized delivery estimation model; and inputting the characteristic data of the target order into the optimized delivery estimation model to obtain a prediction result of the delivery duration of the target order. In this embodiment, the optimized delivery forecast model may include a plurality of models as a whole, and each delivery forecast model has a certain connection relationship. Under the combined action of the models, the prediction result of the delivery duration of the target order can be finally obtained. Further, when the plurality of models are used for predicting the distribution time length, the feature data corresponding to each model can be obtained, then the feature data are extracted from the target order and input into each model, so that the predicted distribution time length of the target order is obtained. Because one or more models in the distribution prediction models are subjected to optimization training again according to the important characteristic data, the prediction results of a single model subjected to optimization training are more accurate, and correspondingly, the prediction results of all the models are more accurate.
Further, as a specific implementation of the method shown in fig. 1 and fig. 2, the present embodiment provides a device for predicting a delivery duration, as shown in fig. 3, the device includes: a feature acquisition module 21, a feature evaluation module 22, a model optimization module 23, and a duration prediction module 24.
The characteristic obtaining module 21 is configured to obtain characteristic data of a sample order, where the characteristic data includes at least one type of characteristic data corresponding to a delivery estimation model and/or newly added characteristic data;
the characteristic evaluation module 22 is configured to train a two-class decision tree model based on the characteristic data of the sample order, so as to obtain an order class model and a characteristic importance of at least one of the characteristic data;
the model optimization module 23 is configured to perform optimization training on at least one delivery prediction model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery prediction model;
and the duration prediction module 24 is configured to obtain order information of the target order, and obtain a prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery prediction model.
In a specific application scenario, the delivery prediction model comprises at least one of a segmented delivery duration prediction model, an overall delivery duration prediction model and a scheduling assignment model; the characteristic obtaining module 21 is specifically configured to obtain a plurality of orders within a preset time period as sample orders, divide an overtime order in the sample orders into positive samples, and divide an unexpired order in the sample orders into negative samples; acquiring characteristic data and/or at least one newly added characteristic data corresponding to at least one of the sectional delivery time length estimation model, the overall delivery time length estimation model and the scheduling assignment model; and extracting the characteristic data in the sample order according to the characteristic data corresponding to the at least one model and/or the newly added characteristic data to obtain the characteristic data of the sample order.
In a specific application scenario, the feature evaluation module 22 may be specifically configured to divide the sample order into a training set and a verification set; training the two-classification decision tree model based on the characteristic data of the sample orders in the training set to obtain an initial order classification model; carrying out classification prediction on the sample orders in the verification set through the initial order classification model to obtain a classification prediction result of the sample orders in the verification set; calculating a model index of the initial order classification model according to the classification prediction result of the sample orders in the verification set; and performing optimization training on the initial order classification model according to the model indexes of the initial order classification model to obtain the order classification model and the feature importance of at least one feature data.
In a specific application scenario, the model optimization module 23 is specifically configured to screen at least one important feature data from the feature data according to the feature importance of the feature data and the model structure of the order classification model; and performing optimization training on at least one delivery estimation model of the segmented delivery duration estimation model, the overall delivery duration estimation model and the scheduling assignment model based on the important characteristic data to obtain an optimized delivery estimation model.
In a specific application scenario, the model optimization module 23 is specifically configured to sort the feature data according to the feature importance of the feature data, and obtain at least one piece of preselected feature data according to a sorting result; converting the model structure of the order classification model into a tree structure, and counting the positive sample proportion of each branch structure in the tree structure; and verifying the feature importance of the preselected feature data according to the positive sample proportion of each branch structure in the tree structure, and obtaining the important feature data according to a verification result.
In a specific application scenario, the model optimization module 23 is specifically configured to perform optimization training on parameters of a delivery estimation model in which the model output is the important feature data, so as to obtain the optimized delivery estimation model; and/or adding the important characteristic data into a delivery prediction model corresponding to the important characteristic data, and carrying out optimization training on the delivery prediction model to obtain the optimized delivery prediction model.
In a specific application scenario, the duration prediction module 24 may be specifically configured to obtain order information of a target order and feature data corresponding to the optimized delivery estimation model; extracting characteristic data in the target order according to the characteristic data corresponding to the optimized delivery estimation model; and inputting the characteristic data of the target order into the optimized delivery estimation model to obtain a prediction result of the delivery duration of the target order.
It should be noted that other corresponding descriptions of the functional units related to the delivery duration prediction apparatus provided in this embodiment may refer to the corresponding descriptions in fig. 1 and fig. 2, and are not described herein again.
Based on the methods shown in fig. 1 and fig. 2, the present embodiment further provides a storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the delivery duration prediction method shown in fig. 1 and fig. 2.
Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, and the software product to be identified may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, or the like), and include several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the method according to the implementation scenarios of the present application.
Based on the foregoing methods shown in fig. 1 and fig. 2 and the delivery duration prediction apparatus embodiment shown in fig. 3, to achieve the foregoing object, this embodiment further provides a computer device for predicting delivery duration, which may specifically be a personal computer, a server, a smart phone, a tablet computer, a smart watch, or other network devices, and the computer device includes a storage medium and a processor; a storage medium for storing a computer program and an operating system; a processor for executing the computer program to implement the above-mentioned methods as shown in fig. 1 and fig. 2.
Optionally, the computer device may further include an internal memory, a communication interface, a network interface, a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, a Display (Display), an input device such as a Keyboard (Keyboard), and the like, and optionally, the communication interface may further include a USB interface, a card reader interface, and the like. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), etc.
Those skilled in the art will appreciate that the example computer apparatus configurations providing identification of operational acts are not intended to be limiting of the computer apparatus, and may include more or fewer components, or a combination of certain components, or a different arrangement of components.
The storage medium may further include an operating system and a network communication module. The operating system is a program that manages the hardware of the computer device and the software resources to be identified, and supports the execution of the information processing program and other software and/or programs to be identified. The network communication module is used for realizing communication among components in the storage medium and communication with other hardware and software in the information processing computer equipment.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present application can be implemented by software plus a necessary general hardware platform, and can also be implemented by hardware. By applying the technical scheme of the application, firstly, the characteristic data of the sample order is obtained, the interpretable binary classification decision tree model is trained on the basis of the current used characteristics of the delivery pre-estimation model and other newly-added characteristics which may cause inaccurate delivery time prediction, the order classification model is obtained, then the importance of the characteristics and the model structure of the order classification model are combined, the reason of the inaccurate model prediction is positioned on the specific model and the specific characteristics, so that the related model is optimized, and finally, the optimized model is used for predicting the target order, and the delivery time prediction result of the target order is obtained. Compared with the prior art, the method effectively improves the efficiency and accuracy of the root cause analysis of the delivery prediction model, particularly improves the efficiency and accuracy of the root cause analysis under a multi-model scene, thereby providing the optimization direction and priority for the optimization of the delivery prediction model, effectively improving the iteration speed and the iteration effect of the delivery prediction model, and finally improving the prediction accuracy of the delivery duration.
Those skilled in the art will appreciate that the figures are merely schematic representations of one preferred implementation scenario and that the blocks or processes in the figures are not necessarily required to practice the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario may be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the present implementation scenario with corresponding changes. The modules of the implementation scenario may be combined into one module, or may be further split into a plurality of sub-modules.
The above application serial numbers are for description purposes only and do not represent the superiority or inferiority of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present application.
Claims (10)
1. A delivery duration prediction method, the method comprising:
acquiring characteristic data of a sample order, wherein the characteristic data comprises characteristic data corresponding to at least one distribution estimation model and/or newly added characteristic data;
training a two-classification decision tree model based on the characteristic data of the sample order to obtain an order classification model and the characteristic importance of at least one type of characteristic data;
performing optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery estimation model;
obtaining order information of the target order, and obtaining a prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery estimation model.
2. The method of claim 1, wherein the delivery forecast model comprises at least one of a segmented delivery duration forecast model, an overall delivery duration forecast model, and a scheduling assignment model; the obtaining of the characteristic data of the sample order includes:
the method comprises the steps of obtaining a plurality of orders in a preset time period as sample orders, dividing overtime orders in the sample orders into positive samples, and dividing non-overtime orders in the sample orders into negative samples;
acquiring characteristic data and/or at least one newly added characteristic data corresponding to at least one of the segmental delivery time length estimation model, the overall delivery time length estimation model and the scheduling assignment model;
and extracting the characteristic data in the sample order according to the characteristic data corresponding to the at least one model and/or the newly added characteristic data to obtain the characteristic data of the sample order.
3. The method of claim 1, wherein training a two-class decision tree model based on the feature data of the sample order to obtain an order classification model and a feature importance of at least one of the feature data comprises:
dividing the sample order into a training set and a verification set;
training the two-classification decision tree model based on the characteristic data of the sample orders in the training set to obtain an initial order classification model;
carrying out classification prediction on the sample orders in the verification set through the initial order classification model to obtain a classification prediction result of the sample orders in the verification set;
calculating a model index of the initial order classification model according to the classification prediction result of the sample orders in the verification set;
and performing optimization training on the initial order classification model according to the model indexes of the initial order classification model to obtain the order classification model and the feature importance of at least one type of feature data.
4. The method according to claim 2, wherein the performing optimization training on at least one delivery estimation model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery estimation model comprises:
screening at least one important feature data from the feature data according to the feature importance of the feature data and the model structure of the order classification model;
and optimizing and training at least one delivery estimation model of the segmented delivery duration estimation model, the overall delivery duration estimation model and the scheduling assignment model based on the important characteristic data to obtain an optimized delivery estimation model.
5. The method of claim 4, wherein the step of screening out at least one important feature data from the feature data according to the feature importance of the feature data and the model structure of the order classification model comprises:
sorting the feature data according to the feature importance of the feature data, and obtaining at least one piece of preselected feature data according to a sorting result;
converting the model structure of the order classification model into a tree structure, and counting the positive sample proportion of each branch structure in the tree structure;
and verifying the feature importance of the preselected feature data according to the positive sample ratio of each branch structure in the tree structure, and obtaining the important feature data according to a verification result.
6. The method according to claim 4, wherein the optimally training at least one of the segmental delivery duration estimation model, the overall delivery duration estimation model and the scheduling assignment model based on the important feature data to obtain an optimized delivery estimation model comprises:
carrying out optimization training on parameters of a delivery estimation model with the model output as the important characteristic data to obtain the optimized delivery estimation model; and/or
And adding the important characteristic data into a delivery prediction model corresponding to the important characteristic data, and carrying out optimization training on the delivery prediction model to obtain the optimized delivery prediction model.
7. The method according to claim 1, wherein the obtaining the order information of the target order and obtaining the result of predicting the delivery duration of the target order according to the order information of the target order and the optimized delivery prediction model comprises:
acquiring order information of a target order and characteristic data corresponding to the optimized delivery estimation model;
extracting characteristic data in the target order according to the characteristic data corresponding to the optimized delivery estimation model;
and inputting the characteristic data of the target order into the optimized delivery estimation model to obtain a prediction result of the delivery duration of the target order.
8. A delivery duration prediction apparatus, characterized in that the apparatus comprises:
the system comprises a characteristic acquisition module, a characteristic analysis module and a characteristic analysis module, wherein the characteristic acquisition module is used for acquiring characteristic data of a sample order, and the characteristic data comprises at least one type of characteristic data corresponding to a distribution estimation model and/or newly added characteristic data;
the characteristic evaluation module is used for training a two-classification decision tree model based on the characteristic data of the sample order to obtain an order classification model and the characteristic importance of at least one type of characteristic data;
the model optimization module is used for carrying out optimization training on at least one delivery prediction model based on the feature importance of the feature data and the model structure of the order classification model to obtain an optimized delivery prediction model;
and the duration prediction module is used for acquiring the order information of the target order and obtaining the prediction result of the delivery duration of the target order according to the order information of the target order and the optimized delivery prediction model.
9. A storage medium having a computer program stored thereon, the computer program, when being executed by a processor, realizing the steps of the method of any one of claims 1 to 7.
10. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 7 when executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210822160.9A CN115169705A (en) | 2022-07-13 | 2022-07-13 | Distribution time length prediction method and device, storage medium and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210822160.9A CN115169705A (en) | 2022-07-13 | 2022-07-13 | Distribution time length prediction method and device, storage medium and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115169705A true CN115169705A (en) | 2022-10-11 |
Family
ID=83492795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210822160.9A Pending CN115169705A (en) | 2022-07-13 | 2022-07-13 | Distribution time length prediction method and device, storage medium and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115169705A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115345716A (en) * | 2022-10-17 | 2022-11-15 | 北京永辉科技有限公司 | Method, system, medium and electronic device for estimating order fulfillment duration |
-
2022
- 2022-07-13 CN CN202210822160.9A patent/CN115169705A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115345716A (en) * | 2022-10-17 | 2022-11-15 | 北京永辉科技有限公司 | Method, system, medium and electronic device for estimating order fulfillment duration |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564286B (en) | Artificial intelligent financial wind-control credit assessment method and system based on big data credit investigation | |
| JP7090936B2 (en) | ESG-based corporate evaluation execution device and its operation method | |
| CN101556553B (en) | Defect prediction method and system based on requirement change | |
| US11915104B2 (en) | Normalizing text attributes for machine learning models | |
| CN107689008A (en) | A kind of user insures the method and device of behavior prediction | |
| CN113435602A (en) | Method and system for determining feature importance of machine learning sample | |
| CN111311338A (en) | User value prediction method and user value prediction model training method | |
| CN112329816A (en) | Data classification method and device, electronic equipment and readable storage medium | |
| CN110647995A (en) | Rule training method, device, equipment and storage medium | |
| CN111680085A (en) | Data processing task analysis method and device, electronic equipment and readable storage medium | |
| CN111582488A (en) | Event deduction method and device | |
| CN115169705A (en) | Distribution time length prediction method and device, storage medium and computer equipment | |
| CN111768247A (en) | Order-placing rate prediction method, device and readable storage medium | |
| CN113298119B (en) | Method and device for evaluating putting strategy of machine learning model and electronic equipment | |
| CN110968802A (en) | User characteristic analysis method, analysis device and readable storage medium | |
| CN113191533A (en) | Warehouse employment prediction method, device, equipment and storage medium | |
| CN110852322B (en) | Method and device for determining region of interest | |
| CN112465321A (en) | Task analysis method, device, equipment and storage medium | |
| CN112749202A (en) | Information operation strategy determination method, device, equipment and storage medium | |
| CN111950753A (en) | Scenic spot passenger flow prediction method and device | |
| CN115439928A (en) | Operation behavior identification method and device | |
| CN111654853B (en) | Data analysis method based on user information | |
| CN114936204A (en) | Feature screening method and device, storage medium and electronic equipment | |
| CN114637917A (en) | Information head bar recommendation method and device based on artificial intelligence | |
| CN111753992A (en) | Screening method and screening system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |