CN112541776A

CN112541776A - Data processing method and device, electronic equipment and storage medium

Info

Publication number: CN112541776A
Application number: CN201910894670.5A
Authority: CN
Inventors: 袁德东
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2019-09-20
Filing date: 2019-09-20
Publication date: 2021-03-23

Abstract

The disclosure relates to a data processing method, a data processing device, an electronic device and a storage medium. The method comprises the following steps: acquiring a plurality of user behavior logs corresponding to a video to be processed, and extracting characteristic data from each user behavior log respectively; the feature data comprises a feature class; determining the effectiveness of the current feature class and the importance of the current feature class aiming at each feature class; acquiring feature categories of which the validity degrees meet a preset first threshold condition and the importance degrees meet a preset second threshold condition, and taking the acquired feature categories as representative feature categories of the video to be processed; the representative feature classes are used as feature classes used when training a feedback prediction model by using the video to be processed. On one hand, the data volume of the representative feature category relative to all feature data is smaller, so that the model training efficiency can be improved; on the other hand, the representative feature class plays a greater role in model training, so that the accuracy of the model can be improved by training the representative feature class.

Description

Data processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of internet technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.

Background

With the rapid development of internet technology, users increasingly rely on obtaining various information through networks. In order to promote goods, various merchants usually perform video (such as advertisement video) delivery via the internet. And recommending videos released by merchants to the user when requested by the user.

In a video recommendation scene, a technology of predicting various feedback information through a feedback prediction model is very important. In a request of a user, feedback information such as click rate, conversion rate and the like of each video can be estimated through the feedback estimation model, and the video delivery engine selects a proper video to recommend to the user according to the feedback information.

An LR (Logistic Regression) model can be used as a feedback prediction model. However, as the number of users increases rapidly, there is a very obvious characteristic of the data of the user behavior log: the scene is sparse, and the data is massive. The scenes are sparse, that is, the behaviors of the user in different scenes are sparse, for example, a user only can act on a small part of videos in a massive commodity video library, and the records of the behaviors are sparse. The data volume, i.e. the behavior data of the user, is huge, for example, each sliding and clicking of the user will generate one record, which results in a very high data volume recorded each day. For this case the LR model is no longer able to meet the requirements. Therefore, considering that the DNN model has better generalization capability than the LR model, the use of the DNN (Deep neural networks) model as the feedback prediction model has appeared.

In the related art, when the DNN model is trained, all feature data in the user behavior log corresponding to the sample video are used for training. However, the training of the model is inefficient due to the large amount of data and the DNN model training itself is slower than the LR model, and the blind training with feature data also results in a less accurate model.

Disclosure of Invention

The present disclosure provides a data processing method, an apparatus, an electronic device, and a storage medium method, an apparatus, and a system, so as to at least solve the problems of low model training efficiency and low model accuracy in the related art. The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a data processing method, including:

acquiring a plurality of user behavior logs corresponding to a video to be processed, and extracting characteristic data from each user behavior log respectively; the characteristic data comprises characteristic categories, and the video to be processed is a video for training a feedback estimation model;

determining the effectiveness of the current feature class and the importance of the current feature class aiming at each feature class; the effectiveness degree is used for representing the correlation degree of the current characteristic category and the feedback parameter estimated by the feedback estimation model; the importance is used for representing the degree of association between the current characteristic category and an area under the curve AUC parameter corresponding to the feedback prediction model;

acquiring feature categories of which the validity degrees meet a preset first threshold condition and the importance degrees meet a preset second threshold condition, and taking the acquired feature categories as representative feature categories of the video to be processed; the representative feature classes are used as feature classes used when training a feedback prediction model by using the video to be processed.

Optionally, the determining the validity of the current feature class includes: calculating the coverage rate of the current feature category, and taking the coverage rate as the effectiveness degree of the current feature category; the coverage rate is used for characterizing the probability of the current feature category appearing in the plurality of user behavior logs; the validity degree meets a preset first threshold condition, specifically, the coverage rate is greater than a preset coverage rate threshold.

Optionally, the step of calculating the coverage of the current feature class includes: acquiring the occurrence times of the current feature category in the user behavior logs and the number of the user behavior logs; and calculating the coverage rate of the current feature category according to the occurrence frequency of the current feature category in the user behavior logs and the number of the user behavior logs.

Optionally, the determining the validity of the current feature class includes: acquiring a feedback rate curve corresponding to the current characteristic category, acquiring the discrimination of a preset interval in the feedback rate curve, and taking the discrimination as the effectiveness of the current characteristic category; the feedback rate is used to characterize a probability that the current feature class appears in the plurality of user behavior logs and is fed back; the discrimination is used for representing the difference value between the maximum feedback rate and the minimum feedback rate in the interval; the validity degree meets a preset first threshold condition, specifically, the discrimination is greater than a preset discrimination threshold.

Optionally, the feature data further comprises feature values of the feature classes; the step of obtaining the feedback rate curve corresponding to the current characteristic category comprises: if the number of the characteristic values of the current characteristic category is smaller than a preset number threshold, respectively obtaining a feedback rate corresponding to each characteristic value of the current characteristic category; if the number of the characteristic values of the current characteristic category is larger than or equal to the number threshold, dividing the characteristic values of the current characteristic category into a plurality of sets, and respectively obtaining the feedback rate corresponding to each set; and generating a feedback rate curve corresponding to the current characteristic category according to the feedback rate.

Optionally, the step of respectively obtaining the feedback rate corresponding to each feature value of the current feature category includes: aiming at each characteristic value of the current characteristic category, acquiring the times of the current characteristic value appearing in the plurality of user behavior logs and the times of the current characteristic value feeding back in the plurality of user behavior logs; and determining the feedback rate corresponding to the current characteristic value according to the occurrence times and the feedback times.

Optionally, the step of respectively obtaining the feedback rate corresponding to each set includes: for each set, acquiring the total times of appearance of the characteristic values contained in the current set in the user behavior logs and the total feedback times of the characteristic values contained in the current set in the user behavior logs; and determining the feedback rate corresponding to the current set according to the total occurrence times and the total feedback times.

Optionally, the feature data further comprises feature values of the feature classes; the step of determining the importance of the current feature class comprises the following steps: training an initial deep neural network model by using the characteristic data; estimating the video to be processed by using the trained deep neural network model, and randomly replacing the characteristic value of the current characteristic category in the estimation process to obtain estimated feedback parameters; determining an area under the curve AUC parameter corresponding to the current characteristic category after replacement by using the real feedback parameter and the pre-estimated feedback parameter of the video to be processed, and taking the AUC parameter as the importance of the current characteristic category; the importance degree meets a preset second threshold condition, specifically, the AUC parameter is smaller than a preset AUC threshold.

Optionally, the feature data further comprises feature values of the feature classes; after the step of taking the obtained feature category as the representative feature category of the video to be processed, the method further comprises the following steps: training an initial deep neural network model by using the representative feature classes and the feature values of the representative feature classes, and obtaining the feedback estimation model after the training is finished; and monitoring the characteristic data corresponding to the displayed video after the feedback estimation model is used for performing feedback estimation on the displayed video.

According to a second aspect of the embodiments of the present disclosure, there is provided a data processing apparatus including:

the extraction module is configured to execute the steps of obtaining a plurality of user behavior logs corresponding to the video to be processed and respectively extracting characteristic data from each user behavior log; the characteristic data comprises characteristic categories, and the video to be processed is a video for training a feedback estimation model;

a determination module configured to perform, for each feature class, determining a validity degree of the current feature class and determining an importance degree of the current feature class; the effectiveness degree is used for representing the correlation degree of the current characteristic category and the feedback parameter estimated by the feedback estimation model; the importance is used for representing the degree of association between the current characteristic category and an area under the curve AUC parameter corresponding to the feedback prediction model;

the processing module is configured to execute the steps of obtaining the feature categories of which the validity degrees meet a preset first threshold condition and the importance degrees meet a preset second threshold condition, and taking the obtained feature categories as representative feature categories of the video to be processed; the representative feature classes are used as feature classes used when training a feedback prediction model by using the video to be processed.

Optionally, the determining module includes: a calculation submodule configured to perform calculation of a coverage of the current feature class, the coverage being taken as the validity of the current feature class; the coverage rate is used for characterizing the probability of the current feature category appearing in the plurality of user behavior logs; the validity degree meets a preset first threshold condition, specifically, the coverage rate is greater than a preset coverage rate threshold.

Optionally, the computation submodule includes: a parameter obtaining unit configured to perform obtaining of the number of times the current feature category appears in the plurality of user behavior logs and the number of the plurality of user behavior logs; a coverage calculation unit configured to perform calculation of a coverage of the current feature category depending on the number of occurrences of the current feature category in the plurality of user behavior logs and the number of the plurality of user behavior logs.

Optionally, the determining module includes: the obtaining submodule is configured to execute obtaining of a feedback rate curve corresponding to the current feature type, obtain a discrimination of a preset interval in the feedback rate curve, and use the discrimination as an effectiveness of the current feature type; the feedback rate is used to characterize a probability that the current feature class appears in the plurality of user behavior logs and is fed back; the discrimination is used for representing the difference value between the maximum feedback rate and the minimum feedback rate in the interval; the validity degree meets a preset first threshold condition, specifically, the discrimination is greater than a preset discrimination threshold.

Optionally, the feature data further comprises feature values of the feature classes; the acquisition sub-module includes: a first obtaining unit configured to perform, if the number of the feature values of the current feature category is smaller than a preset number threshold, obtaining a feedback rate corresponding to each feature value of the current feature category respectively; a second obtaining unit, configured to perform, if the number of the feature values of the current feature category is greater than or equal to the number threshold, dividing the feature values of the current feature category into a plurality of sets, and obtaining a feedback rate corresponding to each set respectively; and the generating unit is configured to generate a feedback rate curve corresponding to the current feature category according to the feedback rate.

Optionally, the first obtaining unit includes: a first parameter obtaining subunit configured to perform, for each feature value of the current feature category, obtaining the number of times that a current feature value appears in the plurality of user behavior logs and the number of times that a current feature value is fed back in the plurality of user behavior logs; and the first feedback rate determining subunit is configured to determine a feedback rate corresponding to the current characteristic value according to the occurrence times and the feedback times.

Optionally, the second obtaining unit includes: a second parameter obtaining subunit configured to perform, for each set, obtaining a total number of times that the feature values included in the current set appear in the plurality of user behavior logs and a total number of times that the feature values included in the current set feed back in the plurality of user behavior logs; and the second feedback rate determining subunit is configured to determine the feedback rate corresponding to the current set according to the total occurrence times and the total feedback times.

Optionally, the feature data further comprises feature values of the feature classes; the determining module comprises: a training sub-module configured to perform training of an initial deep neural network model using the feature data; the characteristic replacement submodule is configured to perform estimation on the video to be processed by using the trained deep neural network model, and randomly replace the characteristic value of the current characteristic category in the estimation process to obtain an estimated feedback parameter; an AUC determining submodule configured to determine an area under the curve AUC parameter corresponding to the current feature category after replacement by using the real feedback parameter and the estimated feedback parameter of the video to be processed, and take the AUC parameter as the importance of the current feature category; the importance degree meets a preset second threshold condition, specifically, the AUC parameter is smaller than a preset AUC threshold.

Optionally, the feature data further comprises feature values of the feature classes; the device further comprises: the model training module is configured to perform training on an initial deep neural network model by using the representative feature categories and the feature values of the representative feature categories, and obtain the feedback prediction model after the training is completed; and the monitoring module is configured to monitor the characteristic data corresponding to the displayed video after performing feedback estimation on the displayed video by using the feedback estimation model.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus, including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data processing method as described above.

According to a fourth aspect of embodiments of the present disclosure, there is provided a storage medium, in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method as described above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising readable program code which, when run on a computing device, causes the computing device to perform the data processing method as described above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

in the embodiment of the disclosure, a plurality of user behavior logs corresponding to a video to be processed are obtained, and feature data are respectively extracted from each user behavior log, wherein the feature data comprise feature categories; determining the effectiveness of the current feature class and the importance of the current feature class aiming at each feature class; and acquiring the feature categories of which the validity meets a preset first threshold condition and the importance meets a preset second threshold condition, and taking the acquired feature categories as representative feature categories of the video to be processed, wherein the representative feature categories are used for training a feedback estimation model by utilizing the video to be processed. Therefore, in the embodiment of the disclosure, the feedback prediction model is not trained blindly by using all the feature data corresponding to the video to be processed, but the representative feature category of the video to be processed is selected from the feedback prediction model for use. On one hand, the data volume of the representative feature class relative to all feature data is smaller, so that the model training efficiency can be improved; on the other hand, the representative feature category is the feature category of which the validity meets a preset first threshold condition and the importance meets a preset second threshold condition, the validity is used for representing the degree of association between the feature category and the feedback parameters estimated by the feedback estimation model, and the importance is used for representing the degree of association between the current feature category and the AUC parameters corresponding to the feedback estimation model, so that the degree of association between the representative feature category and the feedback estimation model is larger, the effect of the representative feature category in model training is larger, and the accuracy of the model can be improved by training the representative feature category.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a flow chart illustrating a method of data processing according to an exemplary embodiment.

FIG. 2 is a flow chart illustrating a method of data processing according to an exemplary embodiment.

Fig. 3 is a diagram illustrating a distribution of coverage according to an example embodiment.

FIG. 4 is a diagram illustrating a feedback rate curve according to an exemplary embodiment.

FIG. 5 is a diagram illustrating a determination of importance for a current feature class in accordance with an exemplary embodiment.

FIG. 6 is a schematic diagram illustrating a system process for feeding back a predictive model according to an exemplary embodiment.

FIG. 7 is a block diagram illustrating a data processing apparatus according to an example embodiment.

FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

FIG. 1 is a flow chart illustrating a method of data processing according to an exemplary embodiment. The data processing method is used in electronic equipment, and the electronic equipment can be provided as a server. As shown in fig. 1, the data processing method includes the following steps.

In step S11, a plurality of user behavior logs corresponding to the video to be processed are obtained, and feature data is extracted from each user behavior log, where the feature data includes a feature category.

The data processing method in the embodiment can be applied to the characteristic engineering before the feedback prediction model is trained. Feature engineering refers to the process of converting raw data into training data of a model, and aims to obtain better training data features. The feature engineering is to find out useful features from the original features, so that the performance of the model can be improved. Therefore, the video to be processed in this embodiment may be a sample video acquired from the internet, such as an advertisement video, and the sample video is a sample video used for training the feedback prediction model. There are a plurality of videos to be processed, and the data processing process for each video to be processed is basically similar, and in this embodiment, one video to be processed is described as an example.

After the pending video is presented to the user, the user may generate some actions for the pending video, such as sliding, clicking, reporting, and the like, which form a user action log. Since a plurality of users may generate behaviors for the video to be processed and one user may generate a plurality of behaviors for the video to be processed, one video to be processed corresponds to a plurality of user behavior logs.

The user behavior logs comprise characteristic data, and after a plurality of user behavior logs corresponding to the video to be processed are obtained, the characteristic data are respectively extracted from each user behavior log. The feature data may include feature classes and feature values corresponding to the feature classes. The feature category refers to attributes of the feature, such as the feature category may include identification, gender, age, behavior, preferences, title of the video, size, image information, and so on of the user. The feature value of the feature category refers to specific information of the feature category, for example, the feature value of gender may be male or female, the feature value of age may be specific age, the preferred feature value may be a video clicked historically, and the like.

In step S12, for each feature class, the effectiveness of the current feature class is determined, and the importance of the current feature class is determined.

The validity is used for representing the correlation degree of the current characteristic category and the feedback parameters estimated by the feedback estimation model. The greater the validity, the greater the degree of association between the current feature category and the feedback parameter estimated by the feedback estimation model. For example, the effectiveness of the feature class may be determined based on the coverage rate, click rate distribution, and the like of the feature class. The greater the coverage of the feature class, the more effective the feature class is for model training. The more obvious the degree of distinction of the click rate distribution of the feature categories, the more effective the feature categories are for model training.

The importance is used for representing the degree of association between the current characteristic category and an AUC (Area Under Curve) parameter corresponding to the feedback prediction model. The greater the importance degree is, the greater the association degree between the current characteristic category and the AUC parameter corresponding to the feedback estimation model is. For example, the importance of the feature type may be determined based on the amount of information carried by the feature type. The larger the amount of information carried by the feature class is, the more important the feature class is to model training.

In step S13, a feature category whose validity satisfies a preset first threshold condition and whose importance satisfies a preset second threshold condition is obtained, and the obtained feature category is used as a representative feature category of the video to be processed.

If the validity of the current feature class meets a preset first threshold condition and the importance meets a preset second threshold condition, it can be shown that the role of the current feature class in making a contract when training the feedback estimation model is large, so that the current feature class is taken as the representative feature class of the video to be processed. And when the to-be-processed video is utilized to train the feedback estimation model, the representative characteristic category of the to-be-processed video is used for training.

In the embodiment of the disclosure, the feedback prediction model is not trained blindly by using all the characteristic data corresponding to the video to be processed, but the representative characteristic category of the video to be processed is selected from the feedback prediction model for use. On one hand, the data volume of the representative feature class relative to all feature data is smaller, so that the model training efficiency can be improved; on the other hand, the representative feature category is the feature category of which the validity meets a preset first threshold condition and the importance meets a preset second threshold condition, the validity is used for representing the degree of association between the feature category and the feedback parameters estimated by the feedback estimation model, and the importance is used for representing the degree of association between the current feature category and the AUC parameters corresponding to the feedback estimation model, so that the degree of association between the representative feature category and the feedback estimation model is larger, the effect of the representative feature category in model training is larger, and the accuracy of the model can be improved by training the representative feature category.

FIG. 2 is a flow chart illustrating a method of data processing according to an exemplary embodiment. As shown in fig. 2, the data processing method includes the following steps.

In step S21, a plurality of user behavior logs corresponding to the video to be processed are obtained, and feature data is extracted from each user behavior log, where the feature data includes a feature category and a feature value of the feature category.

In step S22, for each feature class, the effectiveness of the current feature class is determined, and the importance of the current feature class is determined.

In an alternative embodiment, the step of determining the validity of the current feature class may comprise: and calculating the coverage rate of the current feature category, and taking the coverage rate as the effectiveness degree of the current feature category.

The coverage rate is used to characterize a probability of the current feature class occurring in the plurality of user behavior logs. The coverage rate indicates the information scale covered by the feature classes, and the higher the coverage rate, the greater the effect of the feature classes.

In an alternative embodiment, the step of calculating the coverage of the current feature class may include steps a11 to a 12:

step a11, obtaining the number of times that the current feature category appears in the user behavior logs and the number of the user behavior logs.

The number of times the current feature class appears in the plurality of user behavior logs is not affected by the number of feature values of the current feature class. That is, whether the number of feature values of the current feature class is one or more may be regarded as the current feature class appearing once. For example, for the feature category of the user's preference, if the feature value of the user's preference includes videos a, B, and C of the historical clicks in a certain user behavior log, the feature category of the user's preference is considered to appear once in the user behavior log.

Step a12, calculating the coverage of the current feature category according to the number of times the current feature category appears in the user behavior logs and the number of the user behavior logs.

And calculating the quotient of the occurrence frequency of the current feature category in the plurality of user behavior logs divided by the number of the plurality of user behavior logs, and taking the quotient as the coverage rate of the current feature category. For example, the number of the user behavior logs is 100, and the current feature class appears in all 80 of the user behavior logs, the number of times that the current feature class appears in the plurality of user behavior logs is 80, so that the coverage rate of the current feature class is 80/100 ═ 0.8.

Fig. 3 is a diagram illustrating a distribution of coverage according to an example embodiment. In fig. 3, the distribution of the coverage rates obtained by sorting all the feature types in descending order of the coverage rate is shown, the horizontal axis represents the ID of the feature type (for example, a unique ID may be set for each feature type in advance), and the vertical axis represents the coverage rate of the feature type. The distribution of the coverage of each feature class can be visually seen from fig. 3.

If the coverage of the current feature category is taken as the validity of the current feature category, the condition that the validity meets a preset first threshold may specifically be that the coverage is greater than a preset coverage threshold. For the specific value of the preset coverage threshold, a person skilled in the art may set any suitable value according to practical experience, for example, the value may be set to 0.6, 0.7, 0.8, and the like, which is not limited in this embodiment.

In an alternative embodiment, the step of determining the validity of the current feature class may comprise: and acquiring a feedback rate curve corresponding to the current characteristic category, acquiring the discrimination of a preset interval in the feedback rate curve, and taking the discrimination as the effectiveness of the current characteristic category.

The feedback rate is used for representing the probability that the current characteristic category appears in a plurality of user behavior logs and is fed back; the discrimination is used for representing the difference value between the maximum feedback rate and the minimum feedback rate in the interval. The feedback rate curve indicates the distribution of the feedback rate corresponding to each characteristic value in the characteristic category. The larger the difference in feedback between different feature values, the greater the contribution of the feature class.

In an alternative embodiment, the step of obtaining the feedback rate curve corresponding to the current feature class may include steps a21 to a 23:

a21, if the number of the characteristic values of the current characteristic category is smaller than a preset number threshold, respectively obtaining a feedback rate corresponding to each characteristic value of the current characteristic category.

If the number of feature values of the current feature class is small, a feedback rate may be obtained for each feature value. For the specific value of the preset number threshold, any suitable value may be set by those skilled in the art according to practical experience, for example, the value may be set to 30, 40, 50, and so on, and the present embodiment does not limit this.

The step of respectively obtaining the feedback rate corresponding to each feature value of the current feature category may include: aiming at each characteristic value of the current characteristic category, acquiring the times of the current characteristic value appearing in the plurality of user behavior logs and the times of the current characteristic value feeding back in the plurality of user behavior logs; and determining the feedback rate corresponding to the current characteristic value according to the times of the current characteristic value appearing in the user behavior logs and the feedback times of the current characteristic value in the user behavior logs.

If the user behavior log includes feedback behaviors of the user, such as clicks, conversions, and other feedback behaviors, the number of times of feedback of each feature value included in the user behavior log is 1.

And calculating a quotient obtained by dividing the feedback times of the current characteristic value in the plurality of user behavior logs by the times of the current characteristic value appearing in the plurality of user behavior logs, and taking the quotient as the feedback rate corresponding to the current characteristic value. The feedback Rate may be a Statistical Click Through Rate (sCTR), a Statistical Conversion Rate (cvr), or the like.

For example, for the feedback behavior of clicking, the number of clicks of the current feature value in the user behavior logs is 30, the number of occurrences of the current feature value in the user behavior logs is 50, and then the sCTR corresponding to the current feature value is 30/50 ═ 0.6. For the conversion of the feedback behavior, the conversion times of the current feature value in the user behavior logs are 10, the occurrence times of the current feature value in the user behavior logs are 50, and then the svvr corresponding to the current feature value is 10/50 ═ 0.2.

And A22, if the number of the characteristic values of the current characteristic category is greater than or equal to the number threshold, dividing the characteristic values of the current characteristic category into a plurality of sets, and respectively acquiring the feedback rate corresponding to each set.

If the number of the feature values of the current feature category is large, the calculation amount for obtaining the feedback rate for each feature value is high, so that the feature values can be firstly divided into sets, the feedback rate is obtained for each set, and the calculation amount is reduced.

In implementation, the feature values of the current feature class may be divided into a plurality of sets in an equal frequency or K-means clustering manner. The manner of equalizing frequency may be to order the eigenvalues, so that the total times of appearance of the eigenvalues in each set divided according to the order of the eigenvalues in the plurality of user behavior logs are approximately the same. For example, for the age of the user, if the total number of occurrences of the 10 feature values of 1-10 years old in the user behavior logs is 10, and the total number of occurrences of the 2 feature values of 11-12 years old in the user behavior logs is 10, dividing the 1-10 years old into a set, and dividing the 11-12 years old into a set. The K-means clustering mode means that firstly, K objects are randomly selected from n data objects to serve as initial clustering centers; for the other objects left, they are respectively assigned to the most similar clusters (represented by the cluster centers) according to their similarity (distance) to the cluster centers; then calculating the cluster center of each obtained new cluster (the mean value of all objects in the cluster); this process is repeated until the standard measure function begins to converge.

The step of respectively obtaining the feedback rate corresponding to each set may include: for each set, acquiring the total times of appearance of the characteristic values contained in the current set in the user behavior logs and the total feedback times of the characteristic values contained in the current set in the user behavior logs; and determining the feedback rate corresponding to the current set according to the total times of the characteristic values contained in the current set appearing in the user behavior logs and the total feedback times of the characteristic values contained in the current set in the user behavior logs.

And calculating the quotient of the total feedback times of the characteristic values contained in the current set in the user behavior logs divided by the total times of the characteristic values contained in the current set in the user behavior logs, wherein the quotient is used as the feedback rate corresponding to the current set.

And A23, generating a feedback rate curve corresponding to the current feature type according to the feedback rate.

If the feedback rate corresponding to each characteristic value of the current characteristic category is respectively obtained, a feedback rate curve corresponding to the current characteristic category can be obtained according to the feedback rate corresponding to each characteristic value. If the feedback rate corresponding to each set is obtained, a feedback rate curve corresponding to the current feature category can be obtained according to the feedback rate corresponding to each set. The horizontal axis of the graph represents the characteristic value or the set of characteristic values, and the vertical axis represents the feedback rate. If the eigenvalues are unordered, they may be assumed to be in-order. For example, for a male and a female in gender, the order may be assumed to be male and female, or female and male.

If the discrimination of a preset interval in the feedback rate curve corresponding to the current feature category is taken as the validity of the current feature category, the validity meeting a preset first threshold condition is specifically that the discrimination is greater than a preset discrimination threshold. The interval has a larger discrimination, which indicates that the difference of the feedback rates of different feature values of the current feature type is larger, and the more effective the current feature type is. For example, for the characteristic category of gender, if the feedback rate of the characteristic value male is 0.8 and the feedback rate of the characteristic value female is 0.1, the distinction degree of the two characteristic values of male and female is large, and the value of the characteristic category of gender in training the model is large. For the specific value of the preset discrimination threshold, a person skilled in the art may set any suitable value according to practical experience, and this embodiment does not limit this.

FIG. 4 is a diagram illustrating a feedback rate curve according to an exemplary embodiment. Fig. 4 includes feedback rate curves corresponding to two feature categories, and as can be seen from fig. 2, the feedback rate curve far from the horizontal axis has a large fluctuation, wherein there is a section with a large degree of discrimination, while the feedback rate curve near the horizontal axis is substantially equal, wherein there is no section with a large degree of discrimination, so that the feature category validity corresponding to the feedback rate curve far from the horizontal axis satisfies the preset first threshold condition.

In an alternative implementation, referring to FIG. 5, a schematic diagram illustrating determining importance of a current feature class according to an exemplary embodiment is shown. As can be seen from fig. 5, the process of determining the importance of the current feature class may include training → replacing feature values to obtain the predictive feedback parameters → AUC estimation.

Specifically, the step of determining the importance of the current feature class may include steps B1-B3:

and step B1, training the initial deep neural network model by using the characteristic data.

The initial deep neural network model may be a model that is pre-constructed by a developer. In determining the importance of the current feature class, the initial deep neural network model may be trained first simply using the feature data of the video(s) to be processed. The trained deep neural network model is mainly used for judging the importance degree, but not used for a subsequent online feedback evaluation model.

And B2, estimating the video to be processed by using the trained deep neural network model, and randomly replacing the characteristic value of the current characteristic category in the estimation process to obtain estimated feedback parameters.

In the replacement, the feature value of the current feature class may be randomly replaced with other feature values of the current feature class. For example, the feature data corresponding to 3 videos to be processed is [ x ]₁₁，x₁₂，x₁₃]，[x₂₁，x₂₂，x₂₃]，[x₃₁，x₃₂，x₃₃]Each column corresponds to a feature class, and the process of randomly replacing the feature value of the first feature class may be to replace x by x₁₁Is replaced by x₃₁X is to be₂₁Is replaced by x₁₁X is to be₃₁Is replaced by x₂₁As follows:

and step B3, determining an AUC parameter corresponding to the current characteristic category by using the real feedback parameter and the estimated feedback parameter of the video to be processed, and taking the AUC parameter as the importance of the current characteristic category.

The AUC is defined as the area under the ROC (Receiver Operating Characteristic) curve. The ROC curve is a comprehensive index reflecting continuous variables of sensitivity and specificity, and each point on the ROC curve reflects the sensitivity to the same signal stimulus. The abscissa represents a False Positive class Rate (FPR), i.e., the proportion of samples predicted to be Positive but actually negative to all negative samples; the ordinate represents the True Positive Rate (TPR), i.e. the proportion of samples predicted to be Positive and actually Positive to all Positive examples. In a two-class model, assuming a logistic regression classifier is used that gives the probability of being a positive class for each instance, by setting a threshold such as 0.6, positive classes are assigned a probability greater than or equal to 0.6, and negative classes are assigned a probability less than 0.6. Accordingly, a set of (FPR, TPR) points is calculated, and corresponding coordinate points are obtained in the plane. Each point on the ROC curve corresponds to a threshold value. After the ROC curve is obtained, AUC is calculated from the ROC curve.

If the AUC parameter corresponding to the current feature category is taken as the importance of the current feature category, the importance meeting the preset second threshold condition is specifically that the AUC parameter is smaller than the preset AUC threshold.

If the characteristic value of the current characteristic category is not changed, the AUC parameter corresponding to the current characteristic category can be obtained after the current characteristic category is predicted by using the trained deep neural network model. If the characteristic value of the current characteristic category is randomly replaced, the trained deep neural network model is used for predicting the replaced current characteristic category, and AUC parameters corresponding to the replaced current characteristic category can be obtained.

Under the condition that the characteristic value of the characteristic category is not changed, the AUC parameter corresponding to each characteristic category of the video to be processed is the same. And the AUC parameter corresponding to each feature category after replacement is smaller than the AUC parameter corresponding to the feature category before replacement. The smaller the AUC parameter corresponding to the replaced feature category is, the richer the information carried by the feature category is, and the larger the influence of the feature category after being damaged is. Therefore, an AUC threshold may be preset, and when the AUC parameter corresponding to the current feature category after replacement is smaller than the preset AUC threshold, it is determined that the importance of the current feature category satisfies the preset second threshold condition. For the specific value of the AUC threshold, a person skilled in the art may set any suitable value according to practical experience, and this embodiment does not limit this, for example, the AUC threshold may be set to be 0.7, 0.8, and so on.

For example, for a plurality of feature classes, the AUC corresponding to the feature class after replacement is shown in the following table:

feature classes	AUC
		938	0.774
942	0.782
		919	0.779
926	0.786
		940	0.791
......	......
		939	0.791
918	0.821
		913	0.819
935	0.824
		915	0.825

If the AUC threshold is 0.8, it can be determined that the importance of the feature categories 938, 942, 919, 926, 940, 939 all satisfy a preset second threshold condition.

It should be noted that, in this embodiment, the order of determining the validity of the current feature class and determining the importance of the current feature class is not limited, and the validity of the current feature class may be determined first and then the importance of the current feature class is determined, or the importance of the current feature class may be determined first and then the validity of the current feature class is determined.

If the validity of the current feature type is determined not to meet the preset first threshold condition, data verification can be performed on the current feature type, for example, whether a problem occurs in the feature extraction process or not, whether a problem occurs in an intermediate link or not is verified, and the like. If the feature category of which the importance does not meet the preset second threshold condition is determined, feature transformation and optimization can be performed on the current feature category, for example, the current feature category is discretized, noise points are removed, and the like.

In step S23, a feature category whose validity satisfies a preset first threshold condition and whose importance satisfies a preset second threshold condition is obtained, and the obtained feature category is used as a representative feature category of the video to be processed.

If the validity of the current feature class meets a preset first threshold condition and the importance meets a feature class of a preset second threshold condition, it can be shown that the effect of the current feature class in training the feedback prediction model is large, so that the current feature class is used as a representative feature class of the video to be processed. Otherwise, the current feature category is not taken as the representative feature category of the video to be processed.

In step S24, an initial deep neural network model is trained by using the representative feature class and the feature value of the representative feature class, and the feedback prediction model is obtained after the training is completed.

The model training here refers to training a subsequent online feedback estimation model.

After the representative feature category of each sample video (i.e., the video to be processed) is obtained in the above manner, the initial deep neural network model is trained by using the representative feature categories of the plurality of sample videos and the feature values of the representative feature categories during model training, and a feedback prediction model is obtained after the training is completed. For a specific training process, a person skilled in the art may perform related processing according to actual experience, and this embodiment is not discussed in detail here.

In step S25, after performing feedback estimation on the displayed video by using the feedback estimation model, monitoring the feature data corresponding to the displayed video.

After the feedback prediction model is on line, the feedback prediction model can be used for performing feedback prediction on the displayed video. In this embodiment, the feature data corresponding to the displayed video may also be monitored. In the implementation, a user behavior log corresponding to a displayed video can be obtained at regular time, feature data can be extracted from the user behavior log, feedback parameters (such as click rate, conversion rate and the like) corresponding to each feature type are obtained, the change of the feedback parameters is monitored, and if the feedback parameters are abnormal, abnormal conditions can be quickly positioned and analyzed. For example, if the click rate of gender male is 30%, the click rate of gender female is 5%, and the click rate of gender male is 5% and the click rate of gender female is 30%, the feedback parameter of gender is abnormal. The characteristic monitoring plays an important role in model effect investigation and problem analysis.

FIG. 6 is a schematic diagram illustrating a system process for feeding back a predictive model according to an exemplary embodiment. As can be seen from fig. 6, the processing procedure may include: collecting a user behavior log; performing sample processing, wherein the sample processing can comprise the selection of a sample, and selecting a proper user behavior log as the sample; feature extraction is performed from the sample (corresponding to step S21 in the present embodiment); after the feature extraction, steps S22 to S23 (not shown in fig. 6) of the present embodiment are executed; then obtaining a training sample and a testing sample, and performing estimation and evaluation on the off-line model (which is equivalent to step S24 of the embodiment); training the model, and then putting the model on line; when a user has a request, recalling, and then performing online estimation sequencing by using the model; collecting user behavior logs after online estimation and sorting, namely landing the logs, and collecting the user behavior logs through a Distributed File System (HDFS); step S25 (not shown in fig. 6) of the present embodiment is also performed after the model comes online.

The embodiment has wide application range and can be applied to modules related to feature engineering in a computing system. The time for carrying out feature mining is saved by determining the representative feature category, and the abnormity can be timely positioned and analyzed by feature monitoring when the problem troubleshooting is involved, so that a guidance scheme is given, and the transparency of the problem troubleshooting is improved.

FIG. 7 is a block diagram illustrating a data processing apparatus according to an example embodiment. Referring to fig. 7, the apparatus includes an extraction module 701, a determination module 702, and a processing module 703.

The extracting module 701 is configured to perform obtaining of a plurality of user behavior logs corresponding to a video to be processed, and extract feature data from each user behavior log respectively. The feature data comprises feature types, and the video to be processed is a video for training a feedback prediction model.

A determining module 702 configured to perform, for each feature class, determining a validity degree of the current feature class and determining an importance degree of the current feature class. The significance is used for representing the degree of association between the current characteristic category and the feedback parameter estimated by the feedback estimation model, and the importance is used for representing the degree of association between the current characteristic category and the area under the curve AUC parameter corresponding to the feedback estimation model.

The processing module 703 is configured to perform obtaining a feature category of which the validity meets a preset first threshold condition and the importance meets a preset second threshold condition, and take the obtained feature category as a representative feature category of the video to be processed. The representative feature classes are used as feature classes used when training a feedback prediction model by using the video to be processed.

In an alternative embodiment, the determining module 702 includes: a calculation sub-module configured to perform calculating a coverage of the current feature class, the coverage being taken as the validity of the current feature class. The coverage rate is used to characterize a probability of the current feature class appearing in the plurality of user behavior logs. The validity degree meets a preset first threshold condition, specifically, the coverage rate is greater than a preset coverage rate threshold.

In an alternative embodiment, the computation submodule includes: a parameter obtaining unit configured to perform obtaining of the number of times the current feature category appears in the plurality of user behavior logs and the number of the plurality of user behavior logs; a coverage calculation unit configured to perform calculation of a coverage of the current feature category depending on the number of occurrences of the current feature category in the plurality of user behavior logs and the number of the plurality of user behavior logs.

In an alternative embodiment, the determining module 702 includes: and the obtaining sub-module is configured to execute obtaining of a feedback rate curve corresponding to the current feature type, obtain the discrimination of a preset interval in the feedback rate curve, and use the discrimination as the validity of the current feature type. The feedback rate is used to characterize a probability that the current feature class appears in the plurality of user behavior logs and is fed back; and the discrimination is used for representing the difference value between the maximum feedback rate and the minimum feedback rate in the interval. The validity degree meets a preset first threshold condition, specifically, the discrimination is greater than a preset discrimination threshold.

In an alternative embodiment, the feature data further comprises feature values of the feature classes; the acquisition sub-module includes: a first obtaining unit configured to perform, if the number of the feature values of the current feature category is smaller than a preset number threshold, obtaining a feedback rate corresponding to each feature value of the current feature category respectively; a second obtaining unit, configured to perform, if the number of the feature values of the current feature category is greater than or equal to the number threshold, dividing the feature values of the current feature category into a plurality of sets, and obtaining a feedback rate corresponding to each set respectively; and the generating unit is configured to generate a feedback rate curve corresponding to the current feature category according to the feedback rate.

In an optional implementation, the first obtaining unit includes: a first parameter obtaining subunit configured to perform, for each feature value of the current feature category, obtaining the number of times that a current feature value appears in the plurality of user behavior logs and the number of times that a current feature value is fed back in the plurality of user behavior logs; and the first feedback rate determining subunit is configured to determine a feedback rate corresponding to the current characteristic value according to the occurrence times and the feedback times.

In an optional implementation, the second obtaining unit includes: a second parameter obtaining subunit configured to perform, for each set, obtaining a total number of times that the feature values included in the current set appear in the plurality of user behavior logs and a total number of times that the feature values included in the current set feed back in the plurality of user behavior logs; and the second feedback rate determining subunit is configured to determine the feedback rate corresponding to the current set according to the total occurrence times and the total feedback times.

In an alternative embodiment, the feature data further comprises feature values of the feature classes; the determining module 702 comprises: a training sub-module configured to perform training of an initial deep neural network model using the feature data; the characteristic replacement submodule is configured to perform estimation on the video to be processed by using the trained deep neural network model, and randomly replace the characteristic value of the current characteristic category in the estimation process to obtain an estimated feedback parameter; an AUC determining submodule configured to determine an area under the curve AUC parameter corresponding to the current feature category after replacement by using the real feedback parameter and the estimated feedback parameter of the video to be processed, and take the AUC parameter as the importance of the current feature category; the importance degree meets a preset second threshold condition, specifically, the AUC parameter is smaller than a preset AUC threshold.

In an alternative embodiment, the feature data further comprises feature values of the feature classes; the device further comprises: the model training module is configured to perform training on an initial deep neural network model by using the representative feature categories and the feature values of the representative feature categories, and obtain the feedback prediction model after the training is completed; and the monitoring module is configured to monitor the characteristic data corresponding to the displayed video after performing feedback estimation on the displayed video by using the feedback estimation model.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 8 is a block diagram illustrating an electronic device 800 in accordance with an example embodiment. For example, the electronic device 800 may be a server.

Referring to fig. 8, electronic device 800 includes a processing component 822, which further includes one or more processors, and memory resources, represented by memory 832, for storing instructions, such as applications, that are executable by processing component 822. The application programs stored in memory 832 may include one or more modules that each correspond to a set of instructions. Further, the processing component 822 is configured to execute instructions to perform the above-described methods.

The electronic device 800 may also include a power component 826 configured to perform power management of the electronic device 800, a wired or wireless network interface 850 configured to connect the electronic device 800 to a network, and an input/output (I/O) interface 858. The electronic device 800 may operate based on an operating system stored in the memory 832, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

In an exemplary embodiment, a storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of the electronic device 800 to perform the above-described method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, which comprises readable program code executable by a processor of the electronic device 800 to perform the above-described method. Alternatively, the program code may be stored in a storage medium of the electronic device 800, which may be a non-transitory computer-readable storage medium, such as a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method, comprising:

acquiring feature categories of which the validity degrees meet a preset first threshold condition and the importance degrees meet a preset second threshold condition, and taking the acquired feature categories as representative feature categories of the video to be processed; the representative feature classes are used as feature classes used in training the feedback prediction model by using the video to be processed.

2. The data processing method of claim 1, wherein the determining the validity of the current feature class step comprises:

calculating the coverage rate of the current feature category, and taking the coverage rate as the effectiveness degree of the current feature category; the coverage rate is used for characterizing the probability of the current feature category appearing in the plurality of user behavior logs;

the validity degree meets a preset first threshold condition, specifically, the coverage rate is greater than a preset coverage rate threshold.

3. The data processing method of claim 2, wherein the step of calculating the coverage of the current feature class comprises:

acquiring the occurrence times of the current feature category in the user behavior logs and the number of the user behavior logs;

and calculating the coverage rate of the current feature category according to the occurrence frequency of the current feature category in the user behavior logs and the number of the user behavior logs.

4. The data processing method of claim 1, wherein the determining the validity of the current feature class step comprises: acquiring a feedback rate curve corresponding to the current characteristic category, acquiring the discrimination of a preset interval in the feedback rate curve, and taking the discrimination as the effectiveness of the current characteristic category; the feedback rate is used to characterize a probability that the current feature class appears in the plurality of user behavior logs and is fed back; the discrimination is used for representing the difference value between the maximum feedback rate and the minimum feedback rate in the interval;

the validity degree meets a preset first threshold condition, specifically, the discrimination is greater than a preset discrimination threshold.

5. The data processing method according to claim 4, wherein the feature data further includes feature values of the feature classes; the step of obtaining the feedback rate curve corresponding to the current characteristic category comprises:

if the number of the characteristic values of the current characteristic category is smaller than a preset number threshold, respectively obtaining a feedback rate corresponding to each characteristic value of the current characteristic category;

if the number of the characteristic values of the current characteristic category is larger than or equal to the number threshold, dividing the characteristic values of the current characteristic category into a plurality of sets, and respectively obtaining the feedback rate corresponding to each set;

and generating a feedback rate curve corresponding to the current characteristic category according to the feedback rate.

6. The data processing method according to claim 5, wherein the step of obtaining the feedback rate corresponding to each feature value of the current feature class comprises:

aiming at each characteristic value of the current characteristic category, acquiring the times of the current characteristic value appearing in the plurality of user behavior logs and the times of the current characteristic value feeding back in the plurality of user behavior logs;

and determining the feedback rate corresponding to the current characteristic value according to the occurrence times and the feedback times.

7. The data processing method of claim 5, wherein the step of obtaining the feedback rate corresponding to each set comprises:

for each set, acquiring the total times of appearance of the characteristic values contained in the current set in the user behavior logs and the total feedback times of the characteristic values contained in the current set in the user behavior logs;

and determining the feedback rate corresponding to the current set according to the total occurrence times and the total feedback times.

8. A data processing apparatus, comprising:

the processing module is configured to execute the steps of obtaining the feature categories of which the validity degrees meet a preset first threshold condition and the importance degrees meet a preset second threshold condition, and taking the obtained feature categories as representative feature categories of the video to be processed; the representative feature classes are used as feature classes used in training the feedback prediction model by using the video to be processed.

9. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the data processing method of any one of claims 1 to 7.

10. A storage medium in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of any one of claims 1 to 7.