CN110704706A

CN110704706A - Training method and classification method of classification model, related equipment and classification system

Info

Publication number: CN110704706A
Application number: CN201910860182.2A
Authority: CN
Inventors: 郑方园; 杨舒宁; 杜文滔; 吕雪芬; 李艳龙
Original assignee: Beijing Haiyi Tongzhan Information Technology Co Ltd
Current assignee: Jingdong Technology Information Technology Co Ltd
Priority date: 2019-09-11
Filing date: 2019-09-11
Publication date: 2020-01-17
Anticipated expiration: 2039-09-11
Also published as: CN110704706B

Abstract

The embodiment of the application discloses a training method and a classification method of a classification model, a related device and a classification system, wherein the classification method comprises the following steps: obtaining data to be classified, wherein the data to be classified comprises at least two users to be classified and a plurality of characteristic information of the users; obtaining a first parameter corresponding to each feature information, wherein the first parameter is a parameter characterized by the contribution degree of the feature information to the realization of user classification; classifying users needing to push or put in multimedia information and users not needing to push or put in multimedia information according to the characteristic information of each user to be classified and a first parameter corresponding to the characteristic information; and determining whether to push or release the multimedia information for each user to be classified according to the classification result of the user to be classified.

Description

Training method and classification method of classification model, related equipment and classification system

Technical Field

The present application relates to the field of prediction, and in particular, to a training method, a classification method, related devices, and a classification system for a classification model.

Background

For the e-commerce and the multimedia information promotion platform such as the advertisement platform, in order to obtain greater economic benefits, the e-commerce and the advertisement platform generally agree, the display of the advertisement page is carried out in the display page of the e-commerce platform, and/or the advertisement platform can launch the advertisement related to the e-commerce. In practical application, different users have different purchasing demands, purchasing abilities, preferences and the like, and for a piece of advertisement information to be pushed, the advertisement information cannot be welcomed by each user, and if the advertisement platform pushes the advertisement information to all users, a large amount of pushing resources of the advertisement platform are wasted. If the advertisement platform can be pushed and pushed to users who may need the advertisement in a targeted manner, the targeted pushing of multimedia information is necessarily realized, and the waste of pushing resources is reduced. It can be seen that how to implement classification of potential users (users who may need the advertisement and/or make a purchase of goods through the advertisement) and non-potential users becomes a research hotspot. In the related technology, the characteristics of the user in several aspects of purchase demand, purchase capacity, preference and the like are mostly combined to predict or identify whether the user can become a potential user, so that the classification of the potential user and a non-potential user is realized, and the classification method is too rough and cannot ensure the accuracy.

Disclosure of Invention

In order to solve the existing technical problems, embodiments of the present application provide a training method, a classification method, related devices, and a classification system for a classification model, which can at least achieve accurate distinction between potential users and non-potential users, and improve classification accuracy.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a training method of a classification model, which comprises the following steps:

obtaining first data, wherein the first data is characterized by a plurality of users and a plurality of characteristic information of each user;

obtaining second data, wherein the second data is characterized by feedback data generated by a user who is pre-pushed or pre-released with multimedia information aiming at the pre-pushed or pre-released multimedia information;

constructing training data according to the first data and the second data;

preprocessing the constructed training data to obtain target training data;

training the classification model based on target training data;

and classifying users who need to push or release the multimedia information and users who do not need to push or release the multimedia information by using the classification model obtained based on the target training data.

In the foregoing scheme, the preprocessing the constructed training data to obtain target training data includes:

screening users according to the characteristic values of at least part of the characteristic information in the plurality of characteristic information to obtain a plurality of target users, wherein the target users are users who are pre-pushed or pre-released with the multimedia information;

and obtaining the target training data according to the characteristic information of the target user and feedback data generated by the target user aiming at the multimedia information.

for any one of the individual pieces of feature information,

calculating the proportion of users with characteristic values of the characteristic information in all the users;

screening target characteristics from each characteristic information according to the proportion;

and obtaining the target training data according to the target characteristics and the second data.

In the foregoing aspect, the method further includes:

for any one of the individual pieces of feature information,

obtaining an evidence weight WOE value and/or an information value IV value of each characteristic information;

determining the target characteristics of each user according to the WOE value and/or the IV value of the characteristic information;

and obtaining the target training data according to the target characteristics of each user and the second data.

In the foregoing solution, the classifying the user who needs to push or deliver the multimedia information and the user who does not need to push or deliver the multimedia information by using the classification model obtained based on the target training data includes:

obtaining data to be classified, wherein the data to be classified comprises at least two users to be classified and a plurality of characteristic information of the users;

obtaining a first parameter corresponding to each feature information, wherein the classification model is at least used for indicating the first parameter, and the first parameter is a parameter characterized by contribution degree of the feature information to user classification;

classifying users needing to push or put in multimedia information and users not needing to push or put in multimedia information according to the characteristic information of each user to be classified and first parameters corresponding to the characteristic information;

and determining whether to push or release the multimedia information for each user to be classified according to the classification result of the user to be classified.

In the foregoing solution, after obtaining the data to be classified, the method includes:

extracting target characteristics of each user to be classified in the data to be classified, wherein the target characteristics are characteristic information of all users, and the ratio of users with characteristic values of the characteristic information to all users reaches a preset ratio;

correspondingly, the classifying the users to be classified into the users who need to push or release the multimedia information and the users who do not need to push or release the multimedia information based on the feature information of each user to be classified and the first parameter corresponding to each feature information includes:

and classifying users needing to push or release multimedia information and users not needing to push or release multimedia information based on the target characteristics of the users to be classified and the first parameters corresponding to the target characteristics.

In the foregoing aspect, the method includes:

determining target characteristics of the users to be classified according to the WOE values and/or IV values of the characteristic information;

correspondingly, the classifying the users who need to push or release multimedia information and the users who do not need to push or release multimedia information based on the target characteristics of the users to be classified and the first parameters corresponding to the target characteristics includes:

according to the target characteristics of the users to be classified and the first parameters corresponding to the target characteristics, the users needing to push or put in the multimedia information and the users not needing to push or put in the multimedia information are classified.

The embodiment of the application provides a classification method, which comprises the following steps:

obtaining a first parameter corresponding to each feature information, wherein the first parameter is a parameter characterized by the contribution degree of the feature information to the realization of user classification;

classifying users needing to push or put in multimedia information and users not needing to push or put in multimedia information according to the characteristic information of each user to be classified and a first parameter corresponding to the characteristic information;

correspondingly, the classifying, based on the feature information of each user to be classified and the first parameter corresponding to the feature information, the users who need to push or deliver the multimedia information and the users who do not need to push or deliver the multimedia information to the users to be classified includes:

In the foregoing aspect, the method further includes:

determining target characteristics of the users to be classified according to the WOE value and/or the IV value of each characteristic information;

correspondingly, the classifying the users who need to push or release the multimedia information and the users who do not need to push or release the multimedia information based on the feature information of each user to be classified and the first parameter corresponding to the feature information includes:

The embodiment of the application provides a training device for classification models, which comprises:

a first obtaining unit, configured to obtain first data, where the first data includes a plurality of users and a plurality of feature information of each user;

a second obtaining unit, configured to obtain second data, where the second data is characterized by feedback data generated by a user who is pre-pushed or pre-dropped with multimedia information for the pre-pushed or pre-dropped multimedia information;

the construction unit is used for constructing training data according to the first data and the second data;

the preprocessing unit is used for preprocessing the constructed training data to obtain target training data;

a training unit for training the classification model based on target training data;

and the classification unit is used for classifying at least users needing to push or release the multimedia information and users not needing to push or release the multimedia information by utilizing the classification model obtained based on the target training data.

The embodiment of the application provides a classification device, including:

the device comprises a first obtaining unit, a second obtaining unit and a judging unit, wherein the first obtaining unit is used for obtaining data to be classified, and the data to be classified comprises at least two users to be classified and a plurality of characteristic information of the users;

a second obtaining unit, configured to obtain a first parameter corresponding to each piece of feature information, where the first parameter is a parameter that is characterized by a degree of contribution of the feature information to user classification;

the classification unit is used for classifying users needing to push or put in multimedia information and users not needing to push or put in multimedia information based on the characteristic information of each user to be classified and a first parameter corresponding to the characteristic information;

and the determining unit is used for determining whether to push or release the multimedia information to each user to be classified according to the classification result of the user to be classified.

The present application provides a computer readable storage medium, on which a computer program is stored, wherein the program is implemented, when executed by a processor, to implement the steps of the training method of the classification model and/or the steps of the classification method.

The classification system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the program to implement the steps of the training method of the classification model and/or the steps of the classification method.

The embodiment of the application provides a classification system, which comprises the training equipment and the classification equipment of the classification model.

The embodiment of the application provides a training method, a classification method, related equipment and a classification system for a classification model, wherein the classification method comprises the following steps: obtaining data to be classified, wherein the data to be classified comprises at least two users to be classified and a plurality of characteristic information of the users; classifying users needing to push or put in multimedia information and users not needing to push or put in multimedia information for the users to be classified based on the characteristic information of each user to be classified; and determining whether to push or release the multimedia information for each user to be classified according to the classification result of the user to be classified. And predicting whether the user is a potential user or a non-potential user by combining the user characteristic information and the contribution degree parameter of the characteristic information to the realization of the user classification, so as to realize the accurate classification of the user.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.

Fig. 1 is a schematic flow chart illustrating an implementation of a first embodiment of a classification method provided in the present application;

FIG. 2 is a schematic diagram illustrating an implementation flow of an embodiment of a method for training a classification model provided in the present application;

fig. 3 is a schematic flow chart illustrating an implementation of a second embodiment of the classification method provided in the present application;

FIG. 4 is a schematic diagram of an application scenario provided by the present application;

FIG. 5 is a schematic diagram of a structure of a training apparatus for a classification model provided in the present application;

FIG. 6 is a schematic diagram of the structure of the sorting device provided in the present application;

FIG. 7 is a schematic diagram of the structure of the classification system provided in the present application;

fig. 8 is a schematic hardware configuration diagram of a classification device, a training device of a classification model, and/or a classification system provided in the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. In the present application, the embodiments and features of the embodiments may be arbitrarily combined with each other without conflict. The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

The present application provides a first embodiment of a classification method, which is applied to a classification device, as shown in fig. 1, the method includes:

step (S) 101: obtaining data to be classified, wherein the data to be classified comprises at least two users to be classified and a plurality of characteristic information of the users;

s102: obtaining a first parameter corresponding to each feature information, wherein the first parameter is a parameter characterized by the contribution degree of the feature information to the realization of user classification;

s103: classifying users needing to push or put in multimedia information and users not needing to push or put in multimedia information according to the characteristic information of each user to be classified and a first parameter corresponding to the characteristic information;

s104: and determining whether to push or release the multimedia information for each user to be classified according to the classification result of the user to be classified.

It is understood that if it is obtained through the step S103 that some user (S) is (are) a user (S) who needs to push or deliver multimedia information, S104 pushes or delivers multimedia information to these users. If it is obtained from step S103 that some user (S) is (are) a user who does not need to push or deliver multimedia information, then S104 does not need to push or deliver multimedia information to these users.

In the foregoing solution, the feature information of each user mainly includes the following parts: basic information of the user, purchasing ability information of the user, preference information of the user, and the like. The basic information of the user includes user identification (such as a user login account and/or a password), gender, age, occupation, income and the like. The information of the purchasing power of the user includes: the price of items that are habitually purchased (e.g., to frequently purchase a cell phone), the amount spent purchasing items over a period of time (e.g., the amount spent within a month), etc. The preference information of the user includes: the preference of the user is obtained through counting the types of the commodities which the user is accustomed to purchasing, or the willingness of the user to purchase the commodities is predicted according to the commodity pages which are frequently browsed by the user recently, and the like.

It can be understood that, in the embodiment of the present application, the above feature information and the contribution degree of the feature information of the user are used to distinguish the user from the potential user and the non-potential user, where the first parameter indicates the contribution degree of the corresponding feature information to the distinguishing of the potential user or the non-potential user, the method at least indicates the contribution of each feature information to the distinguishing of the user, such as which feature information has a larger influence on the distinguishing of the user and which feature information has a smaller influence on the distinguishing of the user, and this scheme of distinguishing the user by combining the feature information of the user and the contribution degree of the feature information can improve the distinguishing accuracy, and at least can realize accurate pushing or delivering, targeted pushing or delivering of multimedia information, such as advertisements. Compared with the scheme of non-targeted delivery in the related art, the delivery resource can be effectively saved.

In an optional embodiment, after obtaining the data to be classified, the method comprises: extracting first target characteristics of each user to be classified in the data to be classified, wherein the first target characteristics are characteristic information of all users, and the proportion of users with characteristic values in the characteristic information accounts for a preset proportion; accordingly, S103 is: and classifying users needing to push or release the multimedia information and users not needing to push or release the multimedia information based on the first target characteristics of the users to be classified and the first parameters corresponding to the first target characteristics. In this alternative, considering that the scheme shown in fig. 1 uses a lot of feature information, not all feature information is valid information for distinguishing users, it is necessary to delete feature information that does not contribute to distinguishing users from all feature information, select feature information that contributes to distinguishing users as a (first) target feature, and distinguish users by combining the contribution degrees of the (first) target feature and the (first) target feature of the users, so that the distinguishing accuracy and accuracy can be improved.

In an optional embodiment, the method further comprises: obtaining an evidence weight WOE value and/or an information value IV value of each first target feature; determining a second target characteristic of the user to be classified according to the WOE value and/or the IV value of the first target characteristic; correspondingly, the classifying the users who need to push or release multimedia information and the users who do not need to push or release multimedia information based on the first target characteristics of the users to be classified and the first parameters corresponding to the first target characteristics includes: and classifying users needing to push or put in the multimedia information and users not needing to push or put in the multimedia information according to the second target characteristics of the users to be classified and the first parameters corresponding to the second target characteristics. In this alternative, among the (first) target features selected to contribute to the user discrimination, the feature information that may contribute significantly to the user discrimination is filtered based on the WOE value and/or IV value of the (first) target feature, and the feature information filtered from the (first) target features is used as the second target feature. The second target feature of the user and the contribution degree of the second target feature are combined to distinguish the user, and the distinguishing accuracy and accuracy can be further improved by distinguishing the user by using the feature information with the larger contribution degree as the second target feature is the feature information with the larger contribution degree.

In the foregoing classification method, the first parameter characterized as the contribution degree can be known from a trained classification model, and the classification model in the embodiment of the present application can be obtained by the following classification model training method.

The application provides an embodiment of a training method of a classification model, which is applied to training equipment, and as shown in fig. 2, the method includes:

s201: obtaining first data, wherein the first data is characterized by a plurality of users and a plurality of characteristic information of each user;

s202: obtaining second data, wherein the second data is characterized by feedback data generated by a user who is pre-pushed or pre-released with multimedia information aiming at the pre-pushed or pre-released multimedia information;

s203: constructing training data according to the first data and the second data;

in this step, the training data may be regarded as a set of the first data and the second data.

S204: preprocessing the constructed training data to obtain target training data;

in this step, the user and/or the feature information of the user may be considered to be screened to obtain the target training data.

S205: training the classification model based on target training data;

in this step, the first data is used as the input of the classification model, the second data is used as the output of the classification model, and the training of the classification model is carried out until the training is finished.

S206: and classifying users who need to push or release the multimedia information and users who do not need to push or release the multimedia information by using the classification model obtained based on the target training data.

Therefore, in the embodiment of the application, multimedia information is pre-pushed or pre-released to a user, feedback information generated by the user aiming at the pre-pushed or pre-released multimedia information is obtained, and the feedback information generated by the user aiming at the pre-pushed or pre-released multimedia information is combined with the characteristic information of the user to train the classification model. Under a normal condition, the user feeds back the pre-pushed or pre-dropped multimedia information by combining with the actual demand of the user, that is, the technical scheme takes the use demand of each user in the actual application into consideration and trains the classification model by combining with the characteristic information, so that the obtained classification model is more accurate. Parameters which are obtained from the classification model and are characterized by contribution degrees are more accurate, and the accuracy of distinguishing the users can be further improved.

As will be appreciated by those skilled in the art, the scheme of S201-S206 can be regarded as a training process for the classification model. In practical application, the classification model needs to be trained at least twice in the above training process. If the aforementioned S201 to S206 are regarded as the nth training of the classification model, in an optional embodiment, after classifying the user who needs to push or deliver the multimedia information and the user who does not need to push or deliver the multimedia information by using the classification model obtained by the nth training, the method further includes:

s207: acquiring input data of an N +1 training, wherein the input data of the N +1 training at least comprises input data of an nth training (characteristic information of users and users) and a plurality of characteristic information of each user and each user included in the N +1 training;

s208: obtaining user feedback information obtained by aiming at the (N + 1) th training, wherein the user feedback information is feedback data generated by a user who is pre-pushed or pre-thrown with multimedia information aiming at the pre-pushed or pre-thrown multimedia information;

s209: constructing Nth training data according to the first data and the second data;

s210: preprocessing the constructed Nth training data to obtain Nth target training data;

s211: training the classification model for the Nth time based on the Nth target training data;

s212: and classifying the users who need to push or release the multimedia information and the users who do not need to push or release the multimedia information for the Nth time by using the classification model obtained based on the target training data.

In the foregoing optional embodiment, N is a positive integer greater than or equal to 1, preferably N is a positive integer greater than or equal to 2, and is flexibly set as the case may be. The aforementioned S207 to S212 can be regarded as a scheme for training the classification model N +1 times. And (5) circulating S201-S206, S207-S212 or S201-S212 for multiple times to obtain a trained classification model. The classification model is obtained through at least one training, and the training accuracy of the classification model can be guaranteed. In order to ensure that the classification model is more and more accurate along with the increase of the training times, the size of the selected input data of the (N + 1) th training is usually larger than the input data amount of the Nth training, and the input data amount is gradually increased along with the increase of the training times so as to achieve the beneficial effect that the classification model is more and more accurate along with the increase of the training times.

In an optional embodiment, the preprocessing the constructed training data to obtain target training data includes: screening users according to the characteristic values of at least part of the characteristic information in the plurality of characteristic information to obtain a plurality of target users, wherein the target users are users who are pre-pushed or pre-released with the multimedia information; and obtaining the target training data according to the characteristic information of the target user and feedback data generated by the target user aiming at the multimedia information. For the present alternative embodiment, it can be understood that a user that is helpful for training the classification model may be selected from all users included in the first data as the target user. Further, in all users, if the feature information of the user with the feature value reaches a certain proportion of the total number of the feature information, the user can be considered as a target user. Considering that the characteristic value of the characteristic information of the user may be null, the alternative scheme deletes the user with the null value of most characteristic information, the characteristic information with the null value does not help the training process, and the user with the value of most characteristic information is kept as the target user. The training of the classification model by using the target user and the characteristic information of the target user can ensure the calculation accuracy of the classification model on one hand, and can greatly reduce the calculation amount compared with a scheme of training by using all users on the other hand.

The foregoing alternative is to select target users, and in addition, not all the users or all the feature information of all the target users are helpful to the training process of the classification model, and feature information screening is also required.

In an optional embodiment, the obtaining of the target characteristics of all users or the target characteristics of the target user in the first data may be implemented in the following one and/or two ways: for each feature information of the respective users (each user or each target user in the first data),

the first method is as follows: and under the condition that the proportion of the users with the characteristic values in the characteristic information to all target users reaches a preset proportion, determining the characteristic information as the target characteristic.

The second method comprises the following steps: and calculating an evidence weight WOE value and/or an information value IV value of each feature information, and determining the feature information of which the WOE value reaches a first threshold value and/or the IV value reaches a second threshold value as the target feature.

In the first mode, the feature information is screened according to the characteristic that whether the proportion of users with values in all users reaches a preset proportion or not, so that the reserved feature information is the feature information of which most users have the feature values, and great help is provided for distinguishing the users. In the second mode, the characteristic information is screened through the evidence weight WOE value and/or the information value IV value of the characteristic information, and the characteristic information helpful for distinguishing the users can be reserved.

The scheme of combining the first mode and the second mode is as follows: calculating the proportion of users with characteristic values in all users according to any one characteristic information in all the characteristic information; screening out first target characteristics from each characteristic information according to the proportion; obtaining a WOE value and/or an IV value for each first target feature; determining a second target characteristic of each user according to the WOE value and/or the IV value of the first target characteristic; and obtaining the target training data according to the second target characteristics of each user and the second data. It can be understood that this scheme is equivalent to first performing a preliminary screening of the target features (obtaining the first target features) in a first manner, and then obtaining the second target features from the first target features obtained by the preliminary screening in a second manner, so that feature information which is helpful for distinguishing users can be further screened out. No matter which mode is adopted for screening the characteristic information or the two modes are adopted for screening the characteristic information at the same time, the screened characteristic information can be ensured to be more beneficial to distinguishing potential users from non-potential users.

It should be understood that the user attribute in the embodiment of the present application refers to whether the user is a potential user or a non-potential user, and the potential user may be understood as a user who has a viewing demand for multimedia information such as an automobile advertisement and/or a user who can make a purchase of a pushed or delivered product such as an automobile through the delivery or pushing of the multimedia information such as an automobile advertisement.

The present application will be described in further detail with reference to the following drawings and specific embodiments.

In the application scenario, taking the delivery scenario of the automobile advertisement as an example, a machine learning technology is adopted to predict which users in the platform are possible to be potential users and to buy the automobile, and which users are not possible to be predicted for the potential users, and then the pushing or the delivery of the automobile advertisement is performed on the potential users, and the pushing or the delivery is not performed on the non-potential users, so that the targeted pushing or the delivery is realized.

In the application scenario, for convenience of description, the description is divided into two stages: training phase and prediction phase. It is understood that the prediction phase is used for predicting whether the user is a potential user or a non-potential user, and then classification of the user is achieved. The parameters which are used in the prediction process and are characterized by the contribution degree of the characteristic information are obtained by training in the training stage. Here, a Logistic Regression (LR) model in the machine learning technology is used to predict the potential users, that is, the classification model in the present application scenario is the LR model. It will be understood by those skilled in the art that the mathematical expression for the LR model may be as follows: y ═ f (x); wherein X represents the input of the model-the characteristic information of the user; y represents the output of the model-the result of the prediction made to the user; predicting whether the user is a potential user (e.g., Y ═ 1) or a non-potential user (Y ═ 0); f () represents a mapping function from input to output, representing a functional relationship between X and Y. The weight parameters in the f () expression form need to be known before prediction with the LR model. In the scenario of the present application

Where M is the feature information X or the number of target features, X_jAs jth feature information, A_jIs the weight parameter corresponding to the jth characteristic information. The training phase in this application scenario is intended to be performed by inputting known feature information (M X's)_j) Training the model with known predictions (M Y), i.e. by using known characteristic information (M X)_j) And the known predictors (M Y) learn the respective a in the function. Each a corresponding to each X in the function can be regarded as a contribution degree of the feature information X, and for convenience of description, it is referred to as a weight parameter corresponding to the feature information X, that is, the foregoing first parameter in the application scenario is specifically a weight parameter.

Those skilled in the art will understand that if there is feedback from the user on the pre-pushed or delivered multimedia information, the preliminary identification is a potential user (Y ═ 1); if there is no feedback, then the preliminary identification is a non-potential user (Y ═ 0).

In the application scenario, as shown in fig. 4, the database, the front-end server, and the background server are involved; the background server is functionally divided into a server (prediction server) for predicting user attributes and a server (label creation server) for creating labels according to prediction results. Of course, the prediction server and the label creation server may be implemented by the same server.

Specifically, the training phase is completed by the following process:

s301: collecting user characteristic information of each platform, and constructing pre-released or pushed crowds;

in this step, the background server, specifically the prediction server, collects the user characteristic information of each platform and stores the user characteristic information in the database. The platforms refer to platforms such as e-commerce platforms for user characteristic information summarized by accessing pages, browsing pages and purchasing actions such as adding shopping carts, placing orders and the like of a user through a terminal (such as a mobile phone, a computer, a tablet computer PAD) in a period of time. The user characteristic information comprises basic information of the user, purchasing ability information of the user, preference information of the user and the like; the method specifically comprises the following steps: user identification (such as a user login account number and/or a password), gender, age, occupation, income, price of commodities which are habitually purchased, types of commodities which are liked to be purchased, and the like.

In the application scenario, for all collected users, if the advertisement is pre-delivered or pre-pushed, a large amount of resources must be consumed. Part of the users (target users) are selected from the information to carry out pre-release or pre-push. In practical application, part of the feature information of the user may not take a value, such as taking a value as null, for example, for the feature information of gender, the gender feature information of some users takes a value as null (it is considered that the feature information does not have a feature value), and the gender feature information of some users has a feature value, such as taking a value as "male" or "female". If the number of the characteristic information without the characteristic value reaches a certain proportion of the total number of the characteristic information, the user needs to be deleted. For example, assuming that each user has 10 pieces of feature information, and there is one user whose 10 pieces of feature information have 7 pieces of feature information null values and the number of pieces of feature information null values is greater than 10 × 40% (predetermined ratio) — 4, the user is deleted and is not qualified as a target user. It can be understood that the number of feature information with feature values of the remaining target users needs to be large, which is beneficial to the subsequent training of the classification model.

It can be understood that, in practical applications, the target users may also be screened from other angles or in combination with other angles, for example, from the two user characteristics of gender and age, in order to achieve coverage of people of different age groups and different genders and ensure balance of data selection, users with approximately the same number are selected from different age groups, that is, the number of the selected users in each age group is the same or approximately the same; and/or, from the user characteristic of gender, the number of male users and female users may be approximately the same. And selecting users selected from different user characteristic angles as pre-released or pre-pushed crowds (target users).

S302: primarily screening target characteristics from all characteristic information of a target user;

in this step, feature information with high feature value coverage (most target users have the feature value of the feature information) is selected according to the delivery requirements of the advertisement service. It is understood that, in the user feature information collected from the platform, not all feature information carry the value (feature value) of the feature information. For example, taking the characteristic information "gender" as an example, the value (characteristic value) of the characteristic information is "male" or "female", and the collected gender characteristic information of the user may be null and have no characteristic value. In this case, the background server, specifically, the prediction server calculates, for each piece of feature information, a ratio of the user having the feature value to all target users among all target users, and if the ratio reaches a predetermined ratio for calculation of one piece of feature information, retains the piece of feature information as the target feature. For example, taking the feature information of "sex" as an example, the number of users having the feature value of "male" and the feature value of "female" in the feature information of sex is calculated, the number is divided by the total number of target users to obtain a ratio, the ratio is determined to be 70% or 80% of the predetermined ratio, and when the obtained ratio reaches the predetermined ratio, the feature information of "sex" is retained from the plurality of feature information as the target feature. If the obtained proportion value does not reach the preset proportion, the characteristic information needs to be deleted. Therefore, the feature information with high coverage rate can be screened from all the feature information, and most of the feature information without feature values of users is deleted. The foregoing may be regarded as a preliminary screening scheme for feature information according to a coverage rate of the feature information in the embodiment of the present application, and the feature information obtained by the preliminary screening may be regarded as the foregoing first target feature.

It is understood that, in an alternative embodiment, for a target user, if there is more feature information without feature values, for example, the number of feature information without feature values reaches a certain percentage, for example, 50%, of the total number of feature information, the target user is deleted, and such target user does not greatly help the training of the LR model because of too much missing feature information. In addition, the balance of the feature values of the features of the target user is also considered, if the deviation is too large, the structure of the target user, such as the feature information of gender, can be adjusted through a certain algorithm, and the gender proportions of all the users remained after the target user is deleted due to the lack of the feature values can be balanced as much as possible through adjustment. In practical application, the number of the screened target users is preferably ten thousand levels, about 10 thousands of people in the application scene, and the number of the target users can be selected to ensure the applicability and the generalization of the LR model.

S303: the front-end server carries out Nth pre-push or pre-release of the advertisement to the pre-release or push crowd;

as will be appreciated by those skilled in the art, pre-push or pre-placement is used herein to train the LR model, i.e., placement or pushing of advertisements to obtain weighting parameters.

S304: the front-end server records feedback data generated by users in the pre-delivery or push crowd aiming at the (Nth time) pre-delivery or pre-delivery advertisement; the prediction server classifies the users according to the feedback data;

in S303, the front-end server performs trial (pre) delivery or push of the car advertisement to the pre-delivered or pushed crowd through at least one channel of short message, email, app information push, and the like, so as to achieve reaching of the crowd package. In S304, after receiving the pre-delivered or pre-pushed advertisement information, the crowd may generate a series of behaviors according to the needs of the crowd, such as browsing or not browsing the advertisement, browsing duration, whether there is an advertisement link clicking behavior, whether there is a behavior of joining a shopping cart, whether there is an order placing behavior, and whether there is a payment behavior, the front-end server records the behaviors of each user in the crowd, and transmits the recorded result to the background server, specifically, the prediction server. And the prediction server classifies the users according to the recording result. The users who have feedback, such as the users who browse the advertisement, have a long browsing time, click on an advertisement link, join a shopping cart, and the like, are considered to watch the pre-pushed or pre-delivered advertisement and may have a shopping intention, and are identified as a first numerical value, such as a numerical value 1. For users without feedback, such as users without browsing advertisements, without clicking links, and the like, which are considered to be not required to watch pre-pushed or pre-dropped advertisements, without car buying intentions, data identification is also performed, and the identification is a second numerical value different from the first numerical value, such as a numerical value 0. Further, it is known which users have an intention to buy the car and which users do not have an intention to buy the car in the case of pre-placing the advertisement.

It should be understood by those skilled in the art that, in the foregoing solutions, the selection of the target user is described first and then the selection of the target feature is described, and in the application scenario, there is no strict order for how to select the target user from the collected users and how to select the target feature from the collected feature information, and these may also be performed simultaneously. That is, the scheme for selecting the target user and the steps 303 and 304 have no strict sequence, and can be performed simultaneously.

S305: the background server, particularly the prediction server selects out the features which can be input into the LR model from the preliminarily screened target features; inputting positive and negative samples containing characteristics capable of being input into an LR model, and training the LR model to obtain weight parameters;

in this step, generally, the number of (first) target features preliminarily selected is large, not every (first) target feature is used in the training stage, and in order to ensure the training accuracy of the model, features which are helpful for the training of the LR model need to be selected from the (first) target features preliminarily selected again as second target features; and constructing sample data input to the LR model according to the second target characteristics. The concrete implementation is as follows:

since the Value Of the Evidence Weight (WOE) and the Value Of the Information (IV) can both represent the Value Of one feature, the WOE Value and/or the IV Value Of each target feature are calculated in the application scenario, and the features which can be input to the LR model are screened out again from the preliminarily screened target features as second target features according to the calculated WOE Value and/or IV Value and the respective threshold values. In the case where the WOE value of a target feature reaches the first threshold, the target feature may be considered to be a feature that may be subjected to LR model training. And/or in the case that the IV value of a target feature reaches the second threshold, the target feature may be considered as a feature that can be subjected to LR model training. That is, in the present application scenario, the feature that can be subjected to LR model training may be selected only by the WOE value of the target feature, the feature that can be subjected to LR model training may be selected only by the IV value of the target feature, or the feature that can be subjected to LR model training may be selected by both the WOE value and the IV value of the target feature. The first threshold and the second threshold are set flexibly according to experience.

Here, each target feature may be regarded as a feature variable X, since the feature variables may take different values among different users, such as "gender" may take a value of "male" or "female", and "age" may take a value of 20, 30, or 40, etc. In order to facilitate the calculation of the WOE value and the IV value of the characteristic variable, the characteristic variable is divided into at least two groups according to different values of the characteristic variable. For example, a characteristic variable of "sex X" is classified as male (X)₁) Female (x)₂) Two groups; the characteristic variable of 'age' is divided into 6 groups, 0-10 years oldGroups 1, 11-20, groups 2 … 50-60, group 6 (assuming only users under 60 years of age are considered), and so on. After grouping, for the characteristic variable x of the ith group_iThe WOE value is calculated as follows:

wherein x is_iIndicating grouping of characteristic variables X, WOE (X)_i) Grouping the characteristic variables X into ith group of WOE encoded values; n is_i0Represents the number of samples grouped in the ith group and falling at the value identified as 0 (negative sample); n is_*0Represents the total number of samples falling at the label 0 (negative samples); n is_i1Represents the number of samples grouped in the ith group and falling within the sample number identified as 1 (positive sample); n is_*1Represents the total number of samples falling on the label 1 (positive samples); ln () is a logarithmic function. Wherein, the positive sample is a user (i.e. the user identified as the first value 1) having feedback for the pre-delivered or pre-pushed advertisement information and its (second) target feature information; the negative examples are users for which no feedback exists for the pre-delivered or pre-pushed advertisement information (i.e. the aforementioned user identified with the second value 0) and their (second) target characteristic information. In practical application, in order to ensure the training accuracy of the model, the proportion of the negative samples is generally performed according to the number of the positive samples, so that the positive samples and the negative samples are balanced as much as possible.

Wherein n is_i0/n_*0It can be understood as the ratio of non-responding clients (users who have no feedback on pre-paid or pre-pushed advertising information) in the ith group to a negative sample; n is_i1/_n*1It can be understood as the ratio of responding users (users with feedback on pre-paid or pre-pushed advertising information) to the positive sample in the ith group. The logarithm (ln) of the two ratios is taken again to obtain the WOE value of the characteristic variable X. The larger the WOE, the greater the difference between the two ratios, and the greater the likelihood of a sample response in the packet; the smaller the WOE, the smaller the difference between the two ratios, and the less likely the sample in this packet will respond.

The WOE value of the characteristic variable X is calculated as follows:

where n is the number of packets. Whether a feature variable X can enter the LR model for weight parameter training depends on the magnitude relationship between WOE (X) and a first threshold, and if WOE (X) is greater than the first threshold, such as 3, the feature variable can be selected into the LR model. The first threshold is preset and is any reasonable value. The selection of the characteristic variables according to the woe (x) is referred to the subsequent selection according to the IV value, which is not described in detail.

The IV value is calculated on the basis of the WOE value, for the characteristic variable x of the i-th group_iThe IV value is calculated as follows:

the IV value of the characteristic variable X is calculated as follows:

taking the characteristic variable iv (X) of age X as an example, the ages are divided into the above-mentioned 6 groups (n is 6), and the number of age groups, such as the 1 st age group (1 st group) X, in each of the 6 age groups is calculated₁Age group 2 (group 2) x₂To age group 6 (group 6) x₆IV value of (a). For the age, it is qualified to enter the LR model for training of the weight parameter, depending on the magnitude relationship between the sum of the 6 IV values and the second threshold, for example, 0.2, and if the sum of the 6 IV values is greater than 0.2, the age can be selected in the LR model. The second threshold is preset and is any reasonable value. Inputting positive and negative samples containing features that can be selected into the LR model

In (1), the characteristic variables input to the LR model are assumedIs M, the LR model is trained to obtain A₁To A_MThe value of (a). For LR model

In the training process, X_jAs is known, Y is known (Y of the feedback user is 1, and Y of the non-feedback user is 0), which corresponds to the fact that both the input and the output of the model are known, and in the case of the input and the output, the solution for each a is carried out_j。

S306: the front-end server carries out the (N + 1) th advertisement delivery, returns to S304 and continues to execute until the loss function value of the LR model is minimum;

s301 to S304 are schemes for training the weight parameters in the LR model N-th time, for example, N-th time equal to 1 time, and it should be understood by those skilled in the art that training is usually performed a plurality of times to improve the accuracy of the LR model. For example, each time the car advertisement is pre-delivered or pre-pushed once or more times, training is performed for the N +1 th time, such as the 2 nd training, and after S305, steps S306, S304 to S305 are continuously performed until the loss function value of the LR model is minimum. The loss function value of the LR model is minimum, which indicates that the LR model is trained well or completely.

In this application scenario, the LR model

The loss function of (c) may employ a minimum error (MSE) function representing an expectation of using a trained LR model of the square of the difference between the predicted outcome of whether the user is a potential user and the actual value of whether the user actually purchases a car. The weight parameter in the case where the MSE is minimized is convergence (close to an ideal value), and the training of the LR model may be stopped, and the LR model training may be completed or trained.

It can be understood that, each pre-placement or pre-pushing of the car advertisement is performed for one or more times, each training is equivalent to one adjustment of the weight parameter in the LR model, and the multiple pre-placement or pre-pushing of the car advertisement adjusts the weight parameter according to the feedback data of the user for the pre-placement or pre-pushing of the advertisement each time, and can be regarded as a scheme for dynamically adjusting the weight parameter by combining the result of the dynamic feedback of the user. According to the scheme, on one hand, the feedback result of the user is combined, and the actual requirement of the user is adapted; on the other hand, dynamic adjustment of the weight parameters can lead to trained weight parameters

In addition, in the process of training the LR model multiple times, in order to ensure that the weight parameter is closer to the ideal state as the training times increase, the size of the input data of the selected N +1 th training is usually larger than the input data amount of the nth training, for example, in the N +1 th training, not only the feature information of the target user used in the nth training is used as one part of input data, but also the feature information of the part of users not used in the nth training is additionally read from the database as another part of input data, the two parts of data are collected as the input of the LR model, and the LR model is trained for the N +1 th time. Along with the increase of the training times, the input data amount is gradually increased, the weight parameters are converged, and the beneficial effect that the weight parameters are more and more accurate along with the increase of the training times can be achieved.

It will be appreciated by those skilled in the art that the foregoing scheme is based on the LR model

This mathematical expression is an example for illustration, and in addition, any other reasonable mathematical expression may fall into the scope of the embodiments of the present application. Of course, the loss function may also adopt other functions, such as a root mean square error function, an average error function, and the like, which are not described in detail.

Those skilled in the art should also understand that the aforementioned scheme is a training process of the weight parameters by taking the training model as the LR model as an example, and other reasonable models such as the LR model integrated with the Xgboost algorithm, the deep neural network model, and the like may also be used, which is not described in detail. Furthermore, the LR model is

Description of the preferred embodimentsIn addition, any reasonable mathematical expression can be used as a specific example of the LR model, such as that the output Y and the input X have a logarithmic function relationship, an exponential function relationship, and the like, which can be flexibly set according to specific situations.

S307: the background server, particularly the prediction server, tests the users in the test set by using the trained LR model;

it is understood that the well-trained LR model is the model obtained from the last 1 training. In order to avoid the problem that whether the user is a potential user cannot be accurately predicted due to the error of the model training, in practical application, whether the LR model can enter the prediction stage can be judged according to the importance evaluation parameters of the characteristic information and/or the accuracy of the prediction result obtained in the test stage.

The scheme of judging whether the prediction result can be entered according to the importance evaluation of the feature information can be understood as follows: in the feature information of the user, if one feature information is positively correlated with the prediction of the potential user, the weight parameter corresponding to the feature information should be a positive number, and if one feature information is negatively correlated with the prediction of the potential user, the weight parameter corresponding to the feature information should be a negative number. For example, a characteristic variable expressed as a time difference value may exist in the characteristic information of the user, which represents a difference value between the time of the last 1 purchase of the user and the time corresponding to the current training time. If the time difference is larger, the frequency of purchasing is lower, and the possibility of purchasing is lower. Based on this business logic, in an ideal state, the weight parameter corresponding to the time difference characteristic variable in the LR model should be a negative number, and if the weight parameter corresponding to the time difference characteristic variable in the trained LR model should be a positive number, it indicates that this LR model is not accurate enough and needs to be retrained. Wherein, positive correlation or negative correlation is a specific representation form of the importance evaluation parameter of the characteristic information.

The scheme of judging whether the prediction result can be entered or not for the accuracy of the prediction result obtained in the test stage can be understood as follows: collecting users who have actually purchased automobiles through automobile advertisements and characteristic information thereof, users who have not actually purchased automobiles and characteristic information thereof from the users collected by each platform, selecting characteristic information-target characteristics which can be used by an LR model from the information to predict whether the users are potential users, comparing the prediction results with the results which are actually purchased or not purchased to judge the accuracy of the results obtained by predicting through the LR model, and if the accuracy reaches a set threshold value such as 60% or 70%, considering the LR model as an available model and entering the subsequent prediction stage.

The scheme of S307 may further ensure the accuracy of the LR model to help achieve accurate prediction of user attributes.

Specifically, the prediction phase is completed by the following process:

s308: the background server, particularly the prediction server, can predict whether the user to be predicted is a potential user by using an available LR model;

here, the background server, specifically, the prediction server, reads or collects the feature information of the user to be predicted, further reads or collects the feature information of the user to be predicted corresponding to the feature information used in the LR model, that is, reads or collects the target feature information of the user to be predicted, and calculates the Y value by using each target feature information and the corresponding weight parameter. It will be appreciated that the calculation of Y using the LR model is typically 1 probability value (0-1), and after calculating the probability values for different users to be tested, a decision as to whether a potential user is required based on the probability values. For example, in the case that the calculated probability value of a certain user to be predicted is greater than or equal to a third threshold value, such as 0.5, the predicted user is regarded as a potential user. And under the condition that the calculated probability value of a certain user to be predicted is less than 0.5, the predicted user is regarded as a non-potential user, and then the potential user and the non-potential user are distinguished. The third threshold may also take any other reasonable value, such as 0.4, 0.6, or 0.8, and is not particularly limited.

It can be understood that the users in the platform are usually dynamically changed, and the users to be predicted in the application scenario may be users collected from the platform or users newly added to the platform, which is not limited specifically.

The number of the users to be predicted can be single or two or more; preferably two or more. In the case that the number of the users to be predicted is two or more, in practical application, thousands, tens of thousands, millions, and the like can be used as the order of magnitude of the users to be predicted, and the order of magnitude is usually preferred to be ten thousands. The potential user prediction is carried out on the users to be predicted, so that the prediction of a large number of users is realized, and the prediction throughput is high.

S309: the method comprises the steps of putting or pushing automobile advertisements for users predicted as potential users;

in the step, the potential users are released or pushed, and the non-potential users are not released or pushed, so that targeted release or pushing is realized, and release or pushing resources can be effectively saved.

In an optional embodiment, the application scenario may further include:

s310: the background server, particularly the label creation server, at least stores the user information determined as the potential user.

In this step, the background server, specifically the tag creation server, creates a tag for indicating that the potential user is the potential user for the potential user according to the determination result of the potential user and the non-potential user, and stores at least basic information of the potential user, such as a user name or an account number, age, income, and the like, and of course, may also store preference information of the user, and the like, which is flexibly set according to specific situations.

It can be understood that the role of the tag creation server to identify the potential user and store the relevant information of the potential user is to regard the tag creation server as a resource pool, and in some application scenarios, if a demand is used, the demand-oriented user is preferentially found from the resource pool. For example, when a new advertisement is to be pushed or delivered, the delivery may be given priority to users listed in the resource pool.

S309 and S310 are not in strict sequence, and may be performed simultaneously.

In the foregoing embodiment, the feature information is screened twice as an example. In addition, the feature information may be screened only once, the screening process is as shown in the first or second manner of the screening scheme, and the scheme for screening the feature information once may refer to the foregoing related description, which is not repeated.

In the scheme of the application scenario, in the training stage, the LR model is obtained through multiple times of training, the LR model obtained through multiple times of training has a larger input data amount, and a more accurate weight parameter of the LR model can be obtained. In addition, the machine learning technology is utilized to train the weight parameters, the model is generally robust and is not easy to deviate due to external influence, and the calculation accuracy of the weight parameters can be further ensured. The accurate calculation of the weight parameters can greatly improve the accuracy of the prediction of the user attributes. In this case, not all the feature information of the user is trained with the weight parameter, the feature information with a large prediction aid is selected from the coverage, the WOE value, and/or the IV value, and the weight parameter of each target feature information is trained using the feature information with a large prediction aid and the already-fed user feedback result, so that the contribution degree (weight parameter) of each target feature information can be obtained. In the prediction stage, the user attributes are predicted by combining the target characteristics of the user and the contribution degree of the target characteristics, so that the prediction accuracy and the accuracy can be improved, and at least accurate prediction of potential users and accurate pushing or putting of multimedia information such as advertisements can be realized. Compared with the scheme of needleless targeted delivery in the related technology, the targeted delivery or delivery is realized, and the delivery resources are effectively saved.

It will be appreciated by those skilled in the art that the database in the present application scenario may be stored separately, or may be stored in a server, such as a front-end server or a back-end server, preferably a back-end server. In the case of storage in a background server, the database may be stored using a single server, or may be stored in a single server with the prediction server or the tag server. And is not particularly limited.

In another embodiment of the present application, as shown in fig. 5, the training apparatus for classification models includes: a first obtaining unit 501, a second obtaining unit 502, a constructing unit 503, a preprocessing unit 504, a training unit 505 and a classifying unit 506; wherein the content of the first and second substances,

a first obtaining unit 501, configured to obtain first data, where the first data includes multiple users and multiple pieces of feature information of each user;

a second obtaining unit 502, configured to obtain second data, where the second data is represented by feedback data generated by a user who is pre-pushed or pre-dropped with multimedia information for the pre-pushed or pre-dropped multimedia information;

a constructing unit 503, configured to construct training data according to the first data and the second data;

a preprocessing unit 504, configured to preprocess the constructed training data to obtain target training data;

a training unit 505, configured to train the classification model based on target training data;

a classifying unit 506, configured to classify, by using the classification model obtained based on the target training data, at least a user who needs to push or deliver the multimedia information and a user who does not need to push or deliver the multimedia information.

In an alternative embodiment of the method according to the invention,

the preprocessing unit 504 is further configured to perform user screening according to feature values of at least part of the feature information in the plurality of feature information to obtain a plurality of target users, where the target users are users who are pre-pushed or pre-dropped with the multimedia information; and obtaining the target training data according to the characteristic information of the target user and feedback data generated by the target user aiming at the multimedia information.

In an alternative embodiment of the method according to the invention,

the preprocessing unit 504 is further configured to, for any one of the feature information,

In an alternative embodiment of the method according to the invention,

the classifying unit 506 is further configured to: obtaining data to be classified, wherein the data to be classified comprises at least two users to be classified and a plurality of characteristic information of the users;

In an alternative embodiment of the method according to the invention,

the classifying unit 506 is further configured to: extracting first target characteristics of each user to be classified in the data to be classified, wherein the first target characteristics are characteristic information of all users, and the proportion of users with characteristic values in the characteristic information accounts for a preset proportion; and classifying users needing to push or release the multimedia information and users not needing to push or release the multimedia information based on the first target characteristics of the users to be classified and the first parameters corresponding to the first target characteristics.

In an alternative embodiment of the method according to the invention,

the classifying unit 506 is further configured to: obtaining an evidence weight WOE value and/or an information value IV value of each first target feature; determining a second target characteristic of the user to be classified according to the WOE value and/or the IV value of the first target characteristic; and classifying users needing to push or put in the multimedia information and users not needing to push or put in the multimedia information according to the second target characteristics of the users to be classified and the first parameters corresponding to the second target characteristics.

It is understood that the first obtaining Unit 501, the second obtaining Unit 502, the constructing Unit 503, the preprocessing Unit 504, the training Unit 505, and the classifying Unit 506 in the training apparatus of the classification model may be implemented by a Central Processing Unit (CPU), a Digital Signal Processor (DSP), a Micro Control Unit (MCU), or a programmable gate Array (FPGA) in practical applications. It is further understood that the training device may be the aforementioned background server, and may also be specifically a prediction server.

An embodiment of a sorting apparatus, as shown in fig. 6, includes: a first obtaining unit 601, a second obtaining unit 602, a classifying unit 603, and a determining unit 604; wherein the content of the first and second substances,

a first obtaining unit 601, configured to obtain data to be classified, where the data to be classified includes at least two users to be classified and a plurality of feature information thereof;

a second obtaining unit 602, configured to obtain a first parameter corresponding to each piece of feature information, where the first parameter is a parameter that is characterized by a degree of contribution of the feature information to user classification;

a classifying unit 603, configured to classify, based on feature information of each user to be classified and a first parameter corresponding to the feature information, a user who needs to push or deliver multimedia information and a user who does not need to push or deliver multimedia information for the user to be classified;

a determining unit 604, configured to determine whether to push or deliver the multimedia information to each user to be classified according to the classification result of the user to be classified.

In an alternative embodiment of the method according to the invention,

the second obtaining unit 602 is further configured to extract a first target feature of each user to be classified in the data to be classified, where the first target feature is feature information in which a proportion of users whose feature information has a feature value in all users accounts for a predetermined proportion of all users; correspondingly, the classifying unit 603 is further configured to classify the user who needs to push or deliver the multimedia information and the user who does not need to push or deliver the multimedia information based on the first target feature of each user to be classified and the first parameter corresponding to the first target feature.

In an alternative embodiment of the method according to the invention,

the second obtaining unit 602 is further configured to obtain an evidence weight WOE value and/or an information value IV value of each feature information; determining target characteristics of the users to be classified according to the WOE value and/or the IV value of each characteristic information; correspondingly, the classifying unit 603 is further configured to classify the user needing to push or deliver the multimedia information and the user not needing to push or deliver the multimedia information according to the target feature of the user to be classified and the first parameter corresponding to the target feature.

It can be understood that the first obtaining unit 601, the second obtaining unit 602, the classifying unit 603, and the determining unit 604 in the classifying device can be implemented by a CPU, a DSP, an MCU, or an FPGA in practical application. It is further understood that the prediction device may be the aforementioned background server, and may also be specifically a prediction server.

The present application further provides a classification system, as shown in fig. 7, which at least includes the training device and the classification device of the classification model; the system also comprises the front-end server, a database and a prediction server; the system can also comprise the front-end server, the database, the prediction server and the label creation server.

An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is configured to, when executed by a processor, perform at least the steps of the method shown in any one of fig. 1 to 4. The computer readable storage medium may be specifically a memory. The memory may be a memory 82 as shown in fig. 8.

The embodiment of the application also provides training equipment, classification equipment and a classification system of the classification model. Fig. 8 is a schematic hardware structure diagram of a related device or a classification system according to an embodiment of the present application, and as shown in fig. 8, the related device or the classification system includes: a communication component 83 for data transmission, at least one processor 81 and a memory 82 for storing computer programs capable of running on the processor 81. The various components in the terminal are coupled together by a bus system 84. It will be appreciated that the bus system 84 is used to enable communications among the components. The bus system 84 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 84 in fig. 8.

Wherein the processor 81 executes the computer program to perform at least the steps of the method of any of fig. 1 to 4.

It will be appreciated that the memory 82 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memory 82 described in embodiments herein is intended to comprise, without being limited to, these and any other suitable types of memory.

The method disclosed in the embodiment of the present application can be applied to the processor 81 or implemented by the processor 81. The processor 81 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 81. The processor 81 described above may be a general purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. Processor 81 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 82, and the processor 81 reads the information in the memory 82 and performs the steps of the aforementioned methods in conjunction with its hardware.

In an exemplary embodiment, the related Device or classification system may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, microprocessors (microprocessors), or other electronic components for performing the aforementioned training method and/or classification method of the classification model.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.

The methods disclosed in the several method embodiments provided in the present application may be combined arbitrarily without conflict to obtain new method embodiments.

Features disclosed in several of the product embodiments provided in the present application may be combined in any combination to yield new product embodiments without conflict.

The features disclosed in the several method or apparatus embodiments provided in the present application may be combined arbitrarily, without conflict, to arrive at new method embodiments or apparatus embodiments.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A training method of a classification model is characterized by comprising the following steps:

constructing training data according to the first data and the second data;

preprocessing the constructed training data to obtain target training data;

training the classification model based on target training data;

2. The method of claim 1, wherein preprocessing the constructed training data to obtain target training data comprises:

3. The method according to claim 1 or 2, wherein the preprocessing the constructed training data to obtain target training data comprises:

for any one of the individual pieces of feature information,

4. The method according to claim 1 or 2, characterized in that the method further comprises:

for any one of the individual pieces of feature information,

5. The method according to claim 1 or 2, wherein the classifying the users who need to push or deliver the multimedia information and the users who do not need to push or deliver the multimedia information by using the classification model obtained based on the target training data comprises:

6. The method according to claim 5, characterized in that after obtaining the data to be classified, the method comprises:

7. The method according to claim 5 or 6, characterized in that it comprises:

8. A method of classification, comprising:

9. The method according to claim 8, wherein after obtaining data to be classified, the method comprises:

10. The method according to claim 8 or 9, characterized in that the method further comprises:

11. Training apparatus for classification models, comprising:

12. A sorting apparatus, comprising:

13. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7 and/or the steps of the method of any one of claims 8 to 10.

14. A classification system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 7 and/or the steps of the method of any one of claims 8 to 10 when executing the program.

15. A classification system comprising a training apparatus of the classification model of claim 11 and a classification apparatus of claim 12.