CN111563775A - Crowd division method and device - Google Patents

Abstract

The invention discloses a crowd division method and device. Wherein, the method comprises the following steps: receiving behavior data to be evaluated, wherein the behavior data is data for operating target information; and inputting the behavior data into an evaluation model, and outputting scores of the behavior data by the evaluation model, wherein the evaluation model is a machine learning model, the evaluation model is obtained by training multiple groups of training data, and each group of training data comprises historical behavior data and corresponding scores. The invention solves the technical problem that the actual condition of the crowd is difficult to reflect due to the fact that certain standards are lacked in the crowd division according to the labels in the related technology.

Description

Crowd division method and device

technical field

The present invention relates to the field of crowd division, and in particular, to a crowd division method and device.

Background technique

In the process of advertising, there needs to be a certain group of people to be divided, and when the group is divided, there is often a lack of certain standards or based on certain fixed labels, which makes it difficult to reflect the current situation of the crowd. A score based on the real-time behavior of the crowd can better reflect this value.

For the above problems, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a method and apparatus for classifying a crowd, so as to at least solve the technical problem in the related art that there is a lack of certain standards for classifying crowds according to tags, which makes it difficult to reflect the actual situation of the crowd.

According to an aspect of the embodiments of the present invention, a method for classifying a crowd is provided, which includes: receiving behavior data of a crowd, wherein the behavior data is data of the crowd operating on target information put in; The evaluation model is input, and the evaluation model outputs the score of the behavior data, wherein the evaluation model is a machine learning model, and the evaluation model is obtained by training multiple sets of training data, and each set of the training data includes historical behavior data and corresponding scores; classify the population according to the scores.

Optionally, inputting the behavior data into the evaluation model, and before outputting the score of the behavior data by the evaluation model, includes: selecting a model algorithm according to the data characteristics of the historical behavior data, and constructing a basic model; The historical behavior data and corresponding scores are used to establish multiple sets of training data to train the basic model; the trained basic model is optimized to determine the evaluation model.

Optionally, optimizing the trained basic model, and determining that the evaluation model includes at least one of the following: optimizing and adjusting the parameters of the trained basic model through a model parameter adjustment algorithm, and determining the evaluation model. model; split the training data, combine them into different training data, retrain the evaluation model to determine the evaluation model; fuse multiple different models trained for multiple times to determine the evaluation model .

Optionally, dividing the training data and combining them into different training data, and retraining the evaluation model includes: dividing the training data by means of cross-validation, and combining them into different training data. data, and retrain the evaluation model; wherein, the cross-validation includes at least one of the following: a simple cross-validation method, an S-fold cross-validation method, and a leave-one-out cross-validation method.

Optionally, fusing multiple different models trained for multiple times includes at least one of the following: fusing multiple different models by means of weighted average; Fusion is performed; through the secondary learners, a plurality of different said models, which are multiple primary learners, are fused.

Optionally, the behavior data is input into the evaluation model, and after the evaluation model outputs the score of the behavior data, the method further includes: determining a predicted scoring curve of the behavior data; scoring according to the historical behavior data The curve calibrates the predicted score curve to determine a calibrated score for the behavioral data.

Optionally, classifying the crowd according to the score includes: according to the score and a preset score level, determining that the crowd belongs to the crowd category corresponding to the score level to which the behavior data falls, wherein the predetermined It is assumed that there are multiple rating levels.

According to another aspect of the embodiments of the present invention, there is also provided an apparatus for classifying a crowd, including: a receiving module configured to receive behavior data of a crowd, wherein the behavior data is obtained by the crowd operating on the posted target information data; a scoring module for inputting the behavioral data into an evaluation model, and the evaluation model outputs a score for the behavioral data, wherein the evaluation model is a machine learning model, and the evaluation model is trained through multiple sets of training data It is obtained that each group of the training data includes historical behavior data and corresponding scores; a classification module is used to classify the crowd according to the scores.

According to another aspect of the embodiments of the present invention, a storage medium is further provided, and the storage medium includes a stored program, wherein when the program runs, the device where the storage medium is located is controlled to execute any one of the above population segmentation method.

According to another aspect of the embodiments of the present invention, a processor is further provided, and the processor is configured to run a program, wherein when the program runs, any one of the above-mentioned methods for dividing a crowd is executed.

In the embodiment of the present invention, the behavior data of the receiving crowd is used, wherein the behavior data is the data that the crowd operates on the target information put in; the behavior data is input into the evaluation model, and the evaluation model outputs the score of the behavior data, wherein the evaluation model It is a machine learning model, and the evaluation model is obtained by training multiple sets of training data, and each set of training data includes historical behavior data and corresponding scores; the way to classify the crowd according to the score, through the behavior data of the crowd, through the evaluation model to score, The crowd is classified according to the scores of the crowd's behavior data, which achieves the purpose of accurately classifying the crowd, thereby achieving the technical effect of improving the accuracy of crowd classification, and solving the lack of certain standards for classifying crowds based on labels in related technologies, resulting in Technical problems that are difficult to reflect the actual situation of the population.

Description of drawings

The accompanying drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached image:

1 is a flowchart of a method for dividing a crowd according to an embodiment of the present invention;

2 is a schematic diagram of a crowd division scheme according to an embodiment of the present invention;

3 is a flowchart of a crowd division scheme according to an embodiment of the present invention;

4 is a schematic diagram of a predicted distribution curve according to an embodiment of the present invention;

5 is a schematic diagram of a calibrated prediction distribution curve according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of an apparatus for dividing a crowd according to an embodiment of the present invention.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

According to an embodiment of the present invention, a method embodiment of a method for dividing a crowd is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and, Although a logical order is shown in the flowcharts, in some cases steps shown or described may be performed in an order different from that herein.

FIG. 1 is a flowchart of a method for dividing a crowd according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

Step S102, receiving behavior data of the crowd, wherein the behavior data is the data of the crowd operating the target information put in;

Step S104, input the behavior data into the evaluation model, and the evaluation model outputs the score of the behavior data, wherein the evaluation model is a machine learning model, and the evaluation model is obtained by training multiple sets of training data, and each group of training data includes historical behavior data and corresponding scores. ;

Step S106, classify the crowd according to the score.

Through the above steps, the behavior data of the receiving crowd is adopted, wherein the behavior data is the data that the crowd operates on the target information put in; the behavior data is input into the evaluation model, and the evaluation model outputs the score of the behavior data, wherein the evaluation model is machine learning The model and the evaluation model are obtained by training multiple sets of training data, each set of training data includes historical behavior data and corresponding scores; the way to classify the crowd according to the score, through the behavior data of the crowd, through the evaluation model to score, according to the crowd's behavior data. The score of behavioral data classifies the crowd, achieves the purpose of accurately classifying the crowd, thereby achieving the technical effect of improving the accuracy of crowd classification, and solving the problem of the lack of certain standards for classifying crowds based on labels in related technologies, which makes it difficult to reflect the crowd. the actual situation of technical problems.

The above-mentioned group of people may include multiple user accounts, the target information of the above-mentioned delivery may be advertisement information, and the above-mentioned behavior data may be the operation information after the user account of the above-mentioned group of people receives the advertisement information, such as exposure, click, collection, saving, etc. The above behavior data can determine whether the above user account has value for the above advertisement information. For example, the longer the exposure time to the advertisement information and the more exposure times, it means that the user account is more interested in the above advertisement information, and the more likely it is Sexual purchase, resulting in the benefits of advertising. For example, if the click rate is higher and the advertisement information is collected or saved, it can indicate that the user account is interested in the above advertisement, which means that the user account has a higher advertising value.

The above evaluation model can be a machine learning model, which can include a machine learning network, a deep learning network, a convolutional neural network, etc., and can include an input layer, an intermediate layer, and an output layer. The data is used to train the above machine learning model, and each set of training data includes historical behavior data and corresponding scores. The score of the above-mentioned historical behavior data may be determined by the value generated by the historical behavior data to the above-mentioned advertisement information, and the higher the generated value, the higher the above-mentioned score.

The behavior data is input according to the above evaluation model, and the behavior belongs to the score output by the above evaluation model, so as to score the behavior data of the crowd, and then determine the value of the behavior data of the crowd to the advertising information. According to this value, the crowd can be divided into , in order to carry out different information delivery strategies according to different value groups. For example, the lower the delivery value, the lower the delivery frequency and the shorter the delivery time.

The above-mentioned classification of crowds based on scores improves the technical effect of the accuracy of crowd classification compared to the method of classifying crowds based on tags in the related art, thereby solving the lack of certain standards for classifying crowds based on tags in related technologies, resulting in Technical problems that are difficult to reflect the actual situation of the population.

Optionally, the behavior data is input into the evaluation model, and before the evaluation model outputs the score of the behavior data, it includes: selecting a model algorithm according to the data characteristics of the historical behavior data, and constructing a basic model; Group training data to train the basic model; optimize the trained basic model to determine the evaluation model.

Specifically, in the above-mentioned scenario of crowd classification for advertisement information delivery, the model algorithm is selected according to the data characteristics of historical behavior data, and it can be considered that the basic data characteristics are not high-dimensional, sparse and relatively dense, based on the inherent advantage of the algorithm: regularization; For the characteristics of parallel training of large-scale data, flexibility, and missing value processing, the XGBoost algorithm is selected in this embodiment, and the model AUC interval is between 0.75 and 0.82 on different data sets.

The above-mentioned historical behavior data and corresponding scores can be stored in a database, and the historical behavior data and corresponding scores in the database can be updated at any time with use to ensure the validity of the data. Select a preset number of historical behavior data and corresponding scores from the above databases, establish multiple sets of training data, each set of training data includes historical behavior data and corresponding scores, train the basic model, and optimize the trained basic model , determine the evaluation model.

Optionally, optimize the trained basic model, and determine that the evaluation model includes at least one of the following: optimize and adjust the parameters of the trained basic model through a model parameter adjustment algorithm, and determine the evaluation model; segment the training data , combined into different training data, retrain the evaluation model to determine the evaluation model; fuse multiple different models trained for multiple times to determine the evaluation model.

In the above-mentioned scenario of crowd classification for advertising information, the above-mentioned model parameter adjustment algorithm is used to optimize and adjust the parameters of the basic model after training. GridSearchCV (a parameter adjustment algorithm) can be used for multiple verifications to optimize the model parameters: The optimal value of n_estimators is 400, the value of max_depth is 10, the value of min_child_weight is 5, the value of colsample_bytree is 0.3, the value of learning rate is 0.1, etc.

The above-mentioned segmentation of the training data is combined into different training data, and the evaluation model is retrained, that is, the above-mentioned evaluation model is cross-validated. Specifically, the data is used repeatedly, and the obtained sample data is divided, Combined into different training sets and test sets, use the training set to train the model, and use the test set to evaluate how well the model predicts.

Optionally, dividing the training data, combining them into different training data, and retraining the evaluation model includes: dividing the training data by means of cross-validation, combining them into different training data, and performing the evaluation on the evaluation model. Retraining; wherein, the cross-validation includes at least one of the following: a simple cross-validation method, an S-fold cross-validation method, and a leave-one-out cross-validation method.

In the above-mentioned scenario of the above-mentioned classification of the people who put advertisement information in this embodiment, the S-fold cross-validation method is selected to improve the normalization ability of the evaluation model and find the optimal model parameters.

Optionally, fusing multiple different models trained for multiple times includes at least one of the following: fusing multiple different models by means of weighted average; fusing multiple different models by means of weighted voting; Secondary learners, which fuse multiple different models as multiple primary learners.

In the above-mentioned scenario of the above-mentioned classification of the people who put advertisement information in this embodiment, the voting method is selected, which contributes about 2% to the AUC of the evaluation model, thereby optimizing the model parameters of the evaluation model.

Optionally, the behavior data is input into the evaluation model, and after the evaluation model outputs the score of the behavior data, it further includes: determining the predicted score curve of the behavior data; calibrating the predicted score curve according to the score curve of the historical behavior data, and determining the behavior data. The calibrated score.

The distribution of predicted values has a great relationship with the proportion of positive and negative samples, and the corresponding calibration is performed according to the real distribution, thereby improving the accuracy of predicted values and thus the accuracy of population division.

Optionally, classifying the crowd according to the score includes: according to the score and a preset score level, determining that the crowd belongs to a crowd category corresponding to the score level in which the behavior data falls, wherein there are multiple preset score levels.

It should be noted that this embodiment also provides an optional implementation manner, which will be described in detail below.

This embodiment provides a score based on the real-time behavior of the crowd, which can better reflect this value. When performing crowd real-time scoring, the main focus is on the behavioral data of the crowd and some other related data, and a comprehensive model is constructed to score. The specific steps can be divided into data collection, data processing, scoring model construction, scoring model use and so on.

Fig. 2 is a schematic diagram of a crowd division scheme according to an embodiment of the present invention; Fig. 3 is a flow chart of a crowd division scheme according to an embodiment of the present invention; as shown in Figs. Features are not high-dimensional, sparse and relatively dense, based on the inherent advantages of the algorithm: 1. Regularization. 2. Parallel training of large-scale data. 3. Features such as flexibility and missing value processing, select the XGBoost algorithm, and the model AUC interval is between 0.75 and 0.82 on different data sets.

The optimization model includes:

1) Model parameter optimization

GridSearchCV is used for multiple verifications, and the model parameters are optimized: the optimal value of n_estimators is 400, the value of max_depth is 10, the value of min_child_weight is 5, the value of colsample_bytree is 0.3, and the value of learning rate is 0.1.

2) Cross-validation

Cross-validation is to repeatedly use data, divide the obtained sample data, combine them into different training sets and test sets, use the training set to train the model, and use the test set to evaluate the quality of the model prediction, on this basis Multiple sets of different training sets and test sets can be obtained. Generally, there are simple cross-validation methods, S-fold cross-validation methods, and leave-one-out cross-validation methods. Here we try to use the S-fold cross-validation method to improve the normalization ability of the model and find the optimal model parameters.

3) Model fusion

By fusing multiple different models, the performance of machine learning can be improved. Common model fusion methods are:

1. Average method: The average method has general evaluation and weighted average. For the average method, it is generally used in regression prediction models. In the Boosting series of fusion models, weighted average fusion is generally used.

2. Voting method: there is an absolute majority vote (more than half of the votes), a relative majority vote (the most votes), and a weighted vote. Generally used for classification models, used in bagging models.

3. Learning method: A more powerful combination strategy is to use the "learning method", which is to combine through another learner, and the individual learners are called primary learners, and the learners used for combination are called secondary learners. Learner or meta-learner.

Here, we choose the voting method, which contributes about 2% to the AUC of the model.

4) Calibration

It is calibrated according to the real data distribution and the predicted data distribution.

FIG. 4 is a schematic diagram of a prediction distribution curve according to an embodiment of the present invention. As shown in FIG. 4 is the distribution of prediction values of two data sets. The distribution of prediction values is closely related to the ratio of positive and negative samples. The corresponding calibration is shown in FIG. 5 , which is a schematic diagram of the predicted distribution curve after calibration according to an embodiment of the present invention.

In this embodiment, the above-mentioned model applications include:

Regularly generate tags for large-scale data to build user portrait services, and divide users at different levels: high-value users, medium-value risk, low-value users, and non-value users. Follow-up measures can be adopted according to different value levels.

The measurement of user value quantification in this embodiment is beneficial to subsequent operations of different users.

FIG. 6 is a schematic diagram of an apparatus for dividing a crowd according to an embodiment of the present invention. As shown in FIG. 6 , according to another aspect of the embodiment of the present invention, an apparatus for dividing a crowd is also provided, including: a receiving module 62 , a scoring module 64 and a classification module 66, the device will be described in detail below.

The receiving module 62 is used to receive the behavioral data of the crowd, wherein the behavioral data is the data that the crowd operates on the target information put in; the scoring module 64 is connected with the above-mentioned receiving module 62, and is used for inputting the behavioral data into the evaluation model, which is evaluated by the evaluation model. The scoring of the model output behavior data, wherein, the evaluation model is a machine learning model, and the evaluation model is obtained by training multiple groups of training data, and each group of training data includes historical behavior data and corresponding scores; The classification module 66 is connected to the above-mentioned scoring module 64, Used to classify groups of people based on ratings.

Through the above device, the receiving module 62 is used to receive the behavior data of the crowd, wherein the behavior data is the data that the crowd operates on the target information put in; the scoring module 64 inputs the behavior data into the evaluation model, and the evaluation model outputs the score of the behavior data, wherein , the evaluation model is a machine learning model, and the evaluation model is obtained by training multiple groups of training data, and each group of training data includes historical behavior data and corresponding scores; the classification module 66 classifies the crowd according to the scores. The evaluation model is used for scoring, and the crowd is classified according to the score of the behavior data of the crowd, so as to achieve the purpose of accurately classifying the crowd, thereby achieving the technical effect of improving the accuracy of crowd classification, and solving the problem of classifying crowds according to labels in related technologies. The lack of certain standards for the division leads to technical problems that cannot reflect the actual situation of the population.

According to another aspect of the embodiments of the present invention, a storage medium is further provided, the storage medium includes a stored program, wherein when the program is executed, a device where the storage medium is located is controlled to execute any one of the above-mentioned methods for classifying people.

According to another aspect of the embodiments of the present invention, a processor is further provided, where the processor is configured to run a program, wherein when the program is run, any one of the above-mentioned methods for dividing a crowd is executed.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative, for example, the division of the units may be a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (10)

1. A method of crowd division, comprising:

receiving behavior data of a crowd, wherein the behavior data is data of the crowd operating target information to be put;

inputting the behavior data into an evaluation model, and outputting scores of the behavior data by the evaluation model, wherein the evaluation model is a machine learning model, the evaluation model is obtained by training multiple groups of training data, and each group of training data comprises historical behavior data and corresponding scores;

classifying the population according to the score.

2. The method of claim 1, wherein inputting the behavior data into the assessment model, prior to outputting the score for the behavior data by the assessment model, comprises:

selecting a model algorithm according to the data characteristics of the historical behavior data to construct a basic model;

establishing a plurality of groups of training data through the historical behavior data and the corresponding scores, and training the basic model;

and optimizing the trained basic model, and determining the evaluation model.

3. The method of claim 2, wherein the trained base model is optimized and determining the evaluation model comprises at least one of:

optimizing and adjusting the parameters of the trained basic model through a model parameter adjusting algorithm to determine the evaluation model;

segmenting the training data, combining the training data into different training data, and training the evaluation model again to determine the evaluation model;

and fusing a plurality of different models trained for a plurality of times to determine the evaluation model.

4. The method of claim 3, wherein the training data is segmented and combined into different training data, and wherein retraining the evaluation model comprises:

segmenting the training data in a cross validation mode, combining the training data into different training data, and training the evaluation model again;

wherein the cross-validation comprises at least one of: a simple cross verification method, an S-turn cross verification method and a left cross verification method.

5. The method of claim 3, wherein fusing the plurality of different models trained a plurality of times comprises at least one of:

fusing a plurality of different models in a weighted average mode;

fusing a plurality of different models in a weighted voting mode;

a plurality of different said models, being a plurality of primary learners, are fused by a secondary learner.

6. The method of claim 1, wherein inputting the behavior data into the assessment model, after outputting the score for the behavior data by the assessment model, further comprises:

determining a predictive scoring curve for the behavioral data;

and calibrating the prediction scoring curve according to the scoring curve of the historical behavior data, and determining the calibrated score of the behavior data.

7. The method of claim 1, wherein classifying the population according to the score comprises:

and determining that the crowd belongs to the crowd category corresponding to the grading level in which the behavior data falls according to the grading and a preset grading level, wherein the preset grading level is multiple.

8. A crowd-sourcing device, comprising:

the system comprises a receiving module, a display module and a display module, wherein the receiving module is used for receiving behavior data of a crowd, and the behavior data is data of the crowd operating target information to be released;

the evaluation module is used for inputting the behavior data into an evaluation model and outputting the scores of the behavior data by the evaluation model, wherein the evaluation model is a machine learning model, the evaluation model is obtained by training a plurality of groups of training data, and each group of training data comprises historical behavior data and corresponding scores;

and the classification module is used for classifying the crowd according to the scores.

9. A storage medium comprising a stored program, wherein the program, when executed, controls a device on which the storage medium is located to perform the crowd division method according to any one of claims 1 to 7.

10. A processor configured to run a program, wherein the program is configured to perform the crowd division method according to any one of claims 1 to 7 when running.

Images