CN112541705B - Method, device, equipment and storage medium for generating user behavior evaluation model - Google Patents
Method, device, equipment and storage medium for generating user behavior evaluation model Download PDFInfo
- Publication number
- CN112541705B CN112541705B CN202011543938.XA CN202011543938A CN112541705B CN 112541705 B CN112541705 B CN 112541705B CN 202011543938 A CN202011543938 A CN 202011543938A CN 112541705 B CN112541705 B CN 112541705B
- Authority
- CN
- China
- Prior art keywords
- user behavior
- user
- model
- evaluation
- sample set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000013210 evaluation model Methods 0.000 title claims abstract description 42
- 230000006399 behavior Effects 0.000 claims abstract description 225
- 238000011156 evaluation Methods 0.000 claims abstract description 85
- 230000008859 change Effects 0.000 claims abstract description 73
- 238000012549 training Methods 0.000 claims abstract description 69
- 238000013507 mapping Methods 0.000 claims abstract description 16
- 238000012360 testing method Methods 0.000 claims description 37
- 238000010801 machine learning Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 6
- 238000003066 decision tree Methods 0.000 claims description 4
- 238000007477 logistic regression Methods 0.000 claims description 4
- 238000007637 random forest analysis Methods 0.000 claims description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 6
- 238000013135 deep learning Methods 0.000 abstract description 3
- 239000000047 product Substances 0.000 description 56
- 230000000875 corresponding effect Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000002354 daily effect Effects 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000012512 characterization method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 235000007688 Lycopersicon esculentum Nutrition 0.000 description 1
- 244000046095 Psophocarpus tetragonolobus Species 0.000 description 1
- 240000003768 Solanum lycopersicum Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001364 causal effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003924 mental process Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000000551 statistical hypothesis test Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The disclosure discloses a method, a device, equipment and a storage medium for generating a user behavior evaluation model, which relate to the field of artificial intelligence, in particular to the technical field of user portraits and deep learning. The specific scheme is as follows: constructing at least one association pair of user behaviors and preset evaluation indexes, wherein the association pair comprises: mapping pairs of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes; acquiring a training sample set, wherein samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes; selecting training samples from the sample set corresponding to any user behavior, and performing the training steps of: and inputting the training sample into the initial model to perform regression fitting to obtain a trained user behavior evaluation model, so that a model capable of evaluating the user behavior value can be obtained, and the value of different user behaviors in product evaluation can be quantized.
Description
Technical Field
The present disclosure relates to the field of computer technology, and in particular, to the field of artificial intelligence such as user portraits and deep learning, and more particularly, to a method, apparatus, device, and storage medium for generating a user behavior assessment model.
Background
More and more internet products are subjected to auxiliary design and product iteration acceleration by using user feedback data, so that the new internet products are continuously promoted, and the user experience of the products is improved. It can be said that the research on user behavior has become a technical trend for accelerating the development of products in the internet industry.
During the process of using the product, a user can directly evaluate the content of the product, such as scoring (e.g. bean paste scoring, rotten tomato scoring, etc.), so as to express the personal preference degree of the product content. However, in addition to direct scoring, users may also interact with the product content in a variety of ways (e.g., praise, comment, share, etc.), which may themselves represent some objective rating of the product by the user.
Disclosure of Invention
The present disclosure provides a method, apparatus, device, and storage medium for generating a user behavior assessment model.
According to an aspect of the present disclosure, there is provided a method of generating a user behavior assessment model, including: constructing at least one association pair of user behaviors and preset evaluation indexes, wherein the association pair comprises: mapping pairs of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes; acquiring a training sample set, wherein samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes; selecting training samples from the sample set corresponding to any user behavior, and performing the training steps of: and inputting the training sample into the initial model to perform regression fitting to obtain the trained user behavior evaluation model.
According to another aspect of the present disclosure, there is provided an apparatus for generating a user behavior evaluation model, including: the construction module is configured to construct at least one correlation pair of user behaviors and preset evaluation indexes, wherein the correlation pair comprises: mapping pairs of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes; the first acquisition module is configured to acquire a training sample set, wherein samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes; a training module configured to select training samples from the sample set corresponding to any user behavior, and perform the training steps of: and inputting the training sample into the initial model to perform regression fitting to obtain the trained user behavior evaluation model.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of the implementations of the first aspect.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform a method as described in any one of the implementations of the first aspect.
According to a fifth aspect of the present disclosure, a computer program product is presented, comprising a computer program which, when executed by a processor, implements a method as described in any of the implementations of the first aspect.
The method, device, equipment and storage medium for generating the user behavior evaluation model firstly construct at least one correlation pair of user behaviors and preset evaluation indexes, wherein the correlation pair comprises the following steps: mapping pairs of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes; then obtaining a training sample set, wherein samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes; finally, training samples corresponding to any user behavior are selected from the sample set, and the following training steps are executed: and inputting the training sample into the initial model to perform regression fitting to obtain a trained user behavior evaluation model, so that a model capable of evaluating the user behavior value can be obtained, and the value of different user behaviors in product evaluation can be quantized.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a method of generating a user behavior assessment model according to the present application;
FIG. 3 is a flow diagram of another embodiment of a method of generating a user behavior assessment model according to the present application;
FIG. 4 is a flow diagram of yet another embodiment of a method of generating a user behavior assessment model according to the present application;
FIG. 5 is a schematic diagram of one embodiment of a final estimate of user behavior output by a user behavior assessment model according to the present application;
FIG. 6 is an application scenario diagram of one embodiment of a method of generating a user behavior assessment model according to the present application;
FIG. 7 is a schematic structural diagram of one embodiment of an apparatus for generating a user behavior assessment model of the present application;
FIG. 8 is a block diagram of an electronic device for implementing a method of generating a user behavior assessment model in accordance with an embodiment of the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 illustrates an exemplary system architecture 100 to which embodiments of the methods of generating a user behavior assessment model or apparatus of generating a user behavior assessment model of the present application may be applied.
As shown in fig. 1, the system architecture 100 may include terminals 101, 102, a network 103, a database server 104, and a server 105. The network 103 serves as a medium for providing a communication link between the terminals 101, 102, the database server 104 and the server 105. The network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user 110 may interact with the server 105 via the network 103 using the terminals 101, 102 to receive or send messages or the like. The terminals 101, 102 may have various client applications installed thereon, such as model training class applications, product class applications, etc., including, but not limited to, news class applications, shopping class applications, payment class applications, web browsers, instant messaging tools, etc.
The terminals 101 and 102 may be hardware or software. When the terminals 101, 102 are hardware, they may be various electronic devices with display screens, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video experts compression standard audio layer 3), laptop and desktop computers, and the like. When the terminals 101, 102 are software, they can be installed in the above-listed electronic devices. Which may be implemented as multiple software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein.
The user 110 may perform various interactions (e.g., clicking, commenting, browsing, etc.) on the product class applications installed in the terminals 101, 102, and the terminals 101, 102 may record and communicate the various interactions performed by the user to the database server 104 and/or the server 105 via the network 103.
Database server 104 may be a database server that provides various services. For example, a database server may have stored therein a sample set. The sample set contains a large number of samples. The sample may include at least one correlation pair of user behavior and a preset evaluation index. Thus, the user 110 may also select samples from the sample set stored by the database server 104 via the terminals 101, 102.
The server 105 may also be a server providing various services, such as a background server providing support for various applications displayed on the terminals 101, 102. The background server may train the initial model using the samples in the sample set sent by the terminals 101, 102, and may send training results (e.g., the generated user behavior assessment model) to the terminals 101, 102. In this way, the user can apply the generated user behavior evaluation model to evaluate the user behavior value.
The database server 104 and the server 105 may be hardware or software. When they are hardware, they may be implemented as a distributed server cluster composed of a plurality of servers, or as a single server. When they are software, they may be implemented as a plurality of software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein.
It should be noted that, the method for generating the user behavior evaluation model provided in the embodiment of the present application is generally performed by the server 105. Accordingly, means for generating a user behavior assessment model are typically also provided in the server 105.
It should be noted that the database server 104 may not be provided in the system architecture 100 in cases where the server 105 may implement the relevant functions of the database server 104.
It should be understood that the number of terminals, networks, database servers, and servers in fig. 1 are merely illustrative. There may be any number of terminals, networks, database servers, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a method of generating a user behavior assessment model in accordance with the present application is shown. The method comprises the following steps:
step 201, constructing an association pair of at least one user behavior and a preset evaluation index.
In this embodiment, the execution subject (e.g., the server 105 shown in fig. 1) of the method for generating the user behavior evaluation model may construct at least one correlation pair of the user behavior and the preset evaluation index.
The association pair comprises a mapping pair of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes.
Wherein, the data before and after the product change can be obtained first, for example, the content list displayed to the user before the product change includes: music + football + entertainment, the list of content presented to the user after the product change includes: military, basketball and music, and then obtaining the change of preset evaluation indexes before and after the product change. The preset evaluation index may be a standard for characterizing the quality of the product, such as the number of active users (Daily Active User, DAU), the number of active users in the month (Monthly Active User, MAU), the click rate, the click-through ratio, and the like. The change data of the preset evaluation indexes before and after the product is changed can be obtained by the following modes: collecting user behavior data corresponding to a preset evaluation index, such as data of daily login, browsing and clicking of a user; then generating corresponding preset evaluation indexes, such as DAU data, based on the collected user behavior data; and finally, obtaining the change data of preset evaluation indexes before and after the product is changed.
The data of the user behavior and action before and after the product change can be collected by calling a specific data collection software development kit (Software Development Kit, SDK), such as a personal computer (Personal Computer, PC) end log SDK, a Wise log SDK, a Feed information stream product SDK and the like of the search product.
The terminals 101 and 102 carrying the products can log specific user behaviors, for example, the user slides down to browse news, and records a sliding behavior of a list page; the user clicks on a result, which is recorded as "list page click"; the user browses comments or posts comments on a landing page, and the actions such as comment browsing or comment posting are recorded. After collecting the user behavior data, the terminals 101, 102 carrying the product will periodically transmit the user behavior records back to the server ends 104, 105.
The execution body may process the obtained user behavior data and extract the user behavior. Wherein, the user behavior comprises single behavior, such as browsing, clicking and commenting, etc.; specific behavior patterns (e.g., sequences of user behaviors) such as "slide-click" and "slide-click-praise-share" may also be included. In this embodiment, the method is not limited to a single dimension, but can also be used for evaluating a product more comprehensively and systematically by combining a user behavior sequence.
Any specific user behavior (single behavior or user behavior sequence) can be selected as a study object, the change values of the user behavior before and after the product change and the change values of the preset evaluation indexes before and after the product change are quantized, and a mapping pair of the quantized change values of the user behavior before and after the product change and the quantized change values of the preset evaluation indexes is constructed. For example, click= [ (Δ=0.56), DAU (Δ=0.8) ], comment= [ (Δ=0.75), DAU (Δ=0.8) ].
Step 202, a training sample set is obtained.
In this embodiment, the executing body may acquire a training sample set. The samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes. Wherein, the association pair is based on the relationship data established in the step 201.
Step 203, selecting training samples corresponding to any user behavior from the sample set, and performing the following training steps: and inputting the training sample into the initial model to perform regression fitting to obtain the trained user behavior evaluation model.
In this embodiment, the executing body may select a training sample corresponding to any user behavior from the sample set and execute the training step of the user behavior evaluation model.
The initial model may be an untrained initial model or an untrained initial model, and each layer of the initial model may be provided with initial parameters, which may be continuously adjusted during the training process of the initial model. The initial model may be various types of untrained or untrained artificial neural networks or a model obtained by combining a plurality of untrained or untrained artificial neural networks, for example, the initial model may be an untrained convolutional neural network, or a model obtained by combining an untrained convolutional neural network, and an untrained fully connected layer.
Among them, regression fitting is a predictive modeling technique that studies the relationship between dependent variables (e.g., quantized variance values of preset user evaluation indexes) and independent variables (e.g., quantized variance values of user behaviors) for finding causal relationships between variables and the influence intensities of the independent variables on the dependent variables. Wherein the variables include numeric variables and classification variables. The user behaviors to be researched can be selected as independent variables to be modeled, and feature coefficients of the user behaviors are obtained. Taking a logic return model as an example, taking user behavior as an independent variable, and taking a preset user evaluation index as an independent variable for modeling, the result can be obtained: p (P) i /1-P i ＝α 0 +α i1 *x i1 +α i2 *x i2 +α i3 *x i3 … …. Wherein x is i1 、x i2 、x i3 Representing arguments (e.g., click, comment, praise), α i1 、α i2 、α i3 E R represents a characteristic coefficient, P i /1-P i Representing the probability of the dependent variable. The characteristic coefficients of the user behaviors can represent the contribution values of the user behaviors to the product under a preset evaluation index. The positive characteristic coefficient represents positive correlation of the user behavior to the product evaluation, and the negative characteristic coefficient represents negative correlation of the user behavior to the product evaluation.
Alternatively, after the user behavior assessment model training is completed, model tuning may be performed by learning curve control variance and bias magnitude. The learning curve refers to a relation curve of model training precision and sample number, and the accuracy and recall rate of the model are adjusted by observing the change of the learning curve, so that the model is prevented from being overfitted.
The method for generating the user behavior evaluation model provided by the embodiment of the application can obtain a model capable of evaluating the user behavior value, quantify the value of different user behaviors in product evaluation, has wide application range, is compatible with more user behaviors, and can score corresponding products based on the implicit behaviors of the users.
With further reference to FIG. 3, there is shown a flow chart of another embodiment of a method of generating a user behavior assessment model according to the present application, the method comprising the steps of:
step 301, constructing an association pair of at least one user behavior and a preset evaluation index.
Step 301 is substantially the same as step 201 and will not be described again.
Step 302, a training sample set is obtained.
Step 302 is substantially the same as step 202 and will not be described in detail.
Step 303, selecting training samples corresponding to any user behavior from the sample set, and performing the following training steps: and inputting the training sample into the initial model to perform regression fitting to obtain the trained user behavior evaluation model.
Step 303 is substantially the same as step 203 and will not be described in detail.
Step 304, a test sample set is obtained.
The samples in the test sample set comprise correlation pairs of user behaviors and preset evaluation indexes. Wherein, the association pair is based on the relationship data established in step 301. Alternatively, the correlated pair of any user behavior constructed in step 301 above may be divided into training samples and test samples in a certain ratio (e.g., 70%: 30%), thereby forming a training sample set and a test sample set.
Step 305, selecting a test sample from the test sample set, and performing the following test steps: and inputting the test sample into the trained user behavior evaluation model, and adjusting parameters of the trained user behavior evaluation model according to the loss value to obtain the tested user behavior evaluation model.
The step is used for testing the user behavior evaluation model trained in step 303, so that the user behavior evaluation model can be continuously optimized according to the test result.
Optionally, after the user behavior evaluation model test is completed, model tuning can be performed by verifying the curve control variance and deviation. The verification curve refers to a relation curve of the precision and the number of samples of the model test, and the accuracy and recall rate of the model are adjusted by observing the change of the verification curve, so that the over-fitting result of the model is prevented.
With further reference to FIG. 4, there is shown a flow chart of yet another embodiment of a method of generating a user behavior assessment model according to the present application, the method comprising the steps of:
in step 401, an association pair of at least one user behavior and a preset evaluation index is constructed.
Step 401 is substantially the same as step 201 and will not be described again.
Step 402, a training sample set is obtained.
Step 402 is substantially the same as step 202 and will not be described in detail.
Step 403, selecting training samples corresponding to any user behavior from the sample set, and performing the following training steps: and inputting the training sample into the initial model to perform regression fitting to obtain the trained user behavior evaluation model.
Step 403 is substantially the same as step 203 and will not be described again.
Step 404, an original estimate of the user's behavior is obtained.
Wherein an initial estimate may be set for each different user behavior based on evaluation experience, for example: click=1 point, slide=1.5 points, comment=2 points, praise=3 points, share=3 points, etc. For an evaluation object to be a user behavior sequence pattern, then according to the extracted pattern: mode a=x, e.g.: "slide-click=2 points, slide-click-praise-share=4 points" and so on give the corresponding behavior pattern raw estimates.
Step 405, calculating a final evaluation value of the user behavior under a preset evaluation index according to the feature coefficients based on the feature coefficients of the user behavior obtained by the original evaluation value and the trained user behavior evaluation model.
The characteristic coefficients of different user behaviors are obtained according to the output of the user behavior evaluation model, and the final user behavior evaluation score under a specific evaluation index can be calculated by combining the original evaluation value of the corresponding user behavior. For example, the final user behavior estimate score is equal to the original estimate of the corresponding user behavior multiplied by the characteristic coefficient of the user behavior.
Illustratively, FIG. 5 shows a schematic diagram of one embodiment of a final estimate of user behavior output according to the user behavior assessment model of the present application, as shown in FIG. 5, with final estimates of "click", "landing page share", "list page dislike", "comment like", "landing page dislike", "landing page praise", "landing page attention", "click-related" and "landing page attention", "landing page relevant" click (click_direct "calculated, respectively. If a behavior score deviates further above the 0 axis, the better the behavior relative to other behaviors is for the characterization of the positive evaluation of the product, the greater the contribution value, and vice versa, the lesser the contribution value; the more the deviation score under the 0 axis indicates that the better the behavior contributes value to the characterization of the negative product assessment relative to other behaviors, and vice versa.
In some optional implementations of the present embodiment, the initial model in step 203 above includes any one of the following machine learning models: logistic regression models, decision tree models, random forest models, gradient descent tree models.
Alternatively, a variety of machine learning models may be used for learning, and a model with the optimal effect may be selected as the user behavior evaluation model. The comparison of the model effect belongs to the prior art, and is not described in detail herein.
In some optional implementations of this embodiment, the association pairs in step 201 further include a mapping pair of the quantized change value of the user behavior and the saliency evaluation conclusion thereof before and after the product change and the quantized change value of the preset evaluation index and the saliency evaluation conclusion thereof before and after the product change.
Wherein "saliency check" is a Chinese translation of English Significance Test. In statistics, a significance test is one of "statistical hypothesis test" (Statistical Hypothesis Testing), which is a method for detecting whether there is a difference between an experimental group and a control group in a scientific experiment and whether the difference is significant, such as T test. The mapping pair of the quantized change value of the user behavior and the saliency evaluation conclusion thereof before and after the product change and the quantized change value of the preset evaluation index and the saliency evaluation conclusion thereof before and after the product change is as follows: click= [ (Δ=0.56, significance=false), DAU (Δ=0.8, significance=false) ], comment= [ (Δ=0.75, significance=false), DAU (Δ=0.8, significance=false) ]. Where "significance=false" means that the indicator is below some statistically significant level, the transformation is not apparent, and no signal can be understood.
In some optional implementations of this embodiment, the quantified change value of the user behavior before and after the product change in the association pair in step 201 is determined according to the change value of the user behavior before and after the product change and the number of the user behaviors.
In which the user behavior as a subject of investigation can be abstracted into the number of user behaviors X and the user behavior itself Y. The value of the user behavior itself Y can be represented by an original estimate. In consideration of the influence of the user behavior quantity X in the product experience process of the user, quantitative corresponding quantity weights of the quantized user behaviors can be given as compensation according to specific situations in the calculation process of building the model.
For ease of understanding, fig. 6 shows an application scenario schematic of one embodiment of a method of generating a user behavior assessment model according to the present application.
As shown in fig. 6, first, log dotting is completed for a specific behavior of a user, and based on the obtained log, main behaviors (single behaviors such as "browse, click and comment" and the like) and specific behavior pattern data (a sequence of behaviors within a session such as "slide-click" and "slide-click-praise-share" and the like) of the user are extracted. Specific user behavior data is then collected, such as: the user logs in, browses and clicks data every day, and then generates corresponding evaluation indexes such as: and analyzing the DAU data to obtain an evaluation conclusion of the product change. And then, establishing a correlation model between specific behaviors and evaluations of the user, namely, a specific relation mapping, wherein the part is mainly through constructing a relation between the quantized result of the specific behaviors and the evaluation result of the final product. And then, generating corresponding training and testing samples for each behavior based on the constructed relation data. And then adopting a plurality of different learning models to perform fitting, and comparing the effects of the different models, and selecting a group with better expression from the models. To achieve better model representation while preventing overfitting on the data after model determination, adjustments can be made by learning and verifying curve control variances and bias magnitudes. And finally, outputting and obtaining characteristic coefficients under different behaviors according to the tuned and optimized model, and further estimating the different behaviors.
With further reference to fig. 7, as an implementation of the method shown in the foregoing figures, the present application provides an embodiment of an apparatus for generating a user behavior assessment model, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 2, and the apparatus may be specifically applied in various electronic devices.
As shown in fig. 7, the apparatus 700 for generating a user behavior evaluation model of the present embodiment may include: a construction module 701, a first acquisition module 702, a training module 703. Wherein, the construction module 701 is configured to construct an association pair of at least one user behavior and a preset evaluation index, where the association pair includes: mapping pairs of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes; a first obtaining module 702 configured to obtain a training sample set, wherein samples in the training sample set include at least one correlation pair of user behavior and a preset evaluation index; a training module 703 configured to select training samples from the sample set corresponding to any user behavior, and to perform the following training steps: and inputting the training sample into the initial model to perform regression fitting to obtain the trained user behavior evaluation model.
In this embodiment, in the apparatus 700 for generating a user behavior evaluation model: a construction module 701, a first acquisition module 702, a training module 703. The specific processing of the construction module 701 and the technical effects thereof may refer to the relevant descriptions of steps 201 to 203 in the corresponding embodiment of fig. 2, and are not repeated herein.
In some optional implementations of this embodiment, further comprising: the second acquisition module is configured to acquire a test sample set, wherein samples in the test sample set comprise associated pairs of user behaviors and preset evaluation indexes; a test module configured to select a test sample from a set of test samples, and to perform the following test steps: and inputting the test sample into the trained user behavior evaluation model, and adjusting parameters of the trained user behavior evaluation model according to the loss value to obtain the tested user behavior evaluation model.
In some optional implementations of this embodiment, further comprising: a third acquisition module configured to acquire an original estimate of user behavior; and the estimation module is configured to calculate a final estimation of the user behavior under a preset estimation index according to the characteristic coefficient based on the original estimation and the characteristic coefficient of the user behavior obtained by the trained user behavior estimation model.
In some alternative implementations of the present embodiment, the initial model includes any one of the following machine learning models: logistic regression models, decision tree models, random forest models, gradient descent tree models.
In some optional implementations of this embodiment, the preset evaluation index includes any one of the following: the daily active user quantity index and the point display ratio index.
In some optional implementations of this embodiment, the association pair further includes a mapping pair of a quantized change value of the user behavior and a significance evaluation conclusion thereof before and after the product change and a quantized change value of a preset evaluation index and a significance evaluation conclusion thereof before and after the product change.
In some optional implementations of this embodiment, the quantified change value of the user behavior before and after the product change in the association pair is determined according to the change value of the user behavior before and after the product change and the number of user behaviors.
In some optional implementations of the present embodiment, the user behavior includes: single user behavior and sequences of user behaviors.
Fig. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 806, such as a magnetic disk, an optical disk, or the like; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 801 performs the respective methods and processes described above, for example, a subtitle generating method for a mobile terminal. For example, in some embodiments, the caption generating method for a mobile terminal may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When a computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the subtitle generating method for a mobile terminal described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the subtitle generating method for the mobile terminal by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme, at least one correlation pair of user behaviors and preset evaluation indexes is firstly constructed, wherein the correlation pair comprises: mapping pairs of quantized change values of user behaviors before and after product change and quantized change values of preset evaluation indexes; then obtaining a training sample set, wherein samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes; finally, training samples corresponding to any user behavior are selected from the sample set, and the following training steps are executed: and inputting the training sample into the initial model to perform regression fitting to obtain a trained user behavior evaluation model, so that a model capable of evaluating the user behavior value can be obtained, and the value of different user behaviors in product evaluation can be quantized.
Artificial intelligence is the discipline of studying the process of making a computer mimic certain mental processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.) of a person, both hardware-level and software-level techniques. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (18)
1. A method of generating a user behavior assessment model, comprising:
constructing at least one association pair of user behaviors and preset evaluation indexes, wherein the association pair comprises: mapping pairs of quantized change values of user behaviors before and after product content changes and quantized change values of preset evaluation indexes;
acquiring a training sample set, wherein samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes;
selecting training samples corresponding to any user behavior from the sample set, and performing the training steps of: and inputting the training sample into an initial model for regression fitting to obtain a trained user behavior evaluation model.
2. The method of claim 1, further comprising:
obtaining a test sample set, wherein samples in the test sample set comprise associated pairs of the user behaviors and preset evaluation indexes;
selecting a test sample from the set of test samples, and performing the following test steps: and inputting the test sample into a trained user behavior evaluation model, and adjusting parameters of the trained user behavior evaluation model according to the loss value to obtain the tested user behavior evaluation model.
3. The method of claim 1, the method further comprising:
acquiring an original estimated value of user behavior;
and calculating a final estimated value of the user behavior under a preset evaluation index according to the characteristic coefficient based on the original estimated value and the characteristic coefficient of the user behavior obtained by the trained user behavior evaluation model.
4. The method of claim 1, wherein the initial model comprises any one of the following machine learning models:
logistic regression models, decision tree models, random forest models, gradient descent tree models.
5. The method of claim 1, wherein the preset evaluation index comprises any one of:
the daily active user quantity index and the point display ratio index.
6. The method of claim 1, wherein the association pair further comprises a mapping pair of a quantized change value of user behavior and a significance evaluation conclusion thereof before and after the product content is changed and a quantized change value of a preset evaluation index and a significance evaluation conclusion thereof before and after the product content is changed.
7. The method of claim 1 or 6, wherein the quantified change in user behavior before and after a change in product content in the association pair is determined from the change in user behavior before and after a change in product content and the number of user behaviors.
8. The method of claim 1, wherein the user behavior comprises: single user behavior and sequences of user behaviors.
9. An apparatus for generating a user behavior assessment model, comprising:
the construction module is configured to construct an association pair of at least one user behavior and a preset evaluation index, wherein the association pair comprises: mapping pairs of quantized change values of user behaviors before and after product content changes and quantized change values of preset evaluation indexes;
the system comprises a first acquisition module, a second acquisition module and a first evaluation module, wherein the first acquisition module is configured to acquire a training sample set, and samples in the training sample set comprise at least one correlation pair of user behaviors and preset evaluation indexes;
a training module configured to select training samples from the sample set corresponding to any user behavior, and to perform the training steps of: and inputting the training sample into an initial model for regression fitting to obtain a trained user behavior evaluation model.
10. The apparatus of claim 9, further comprising:
the second acquisition module is configured to acquire a test sample set, wherein samples in the test sample set comprise associated pairs of the user behaviors and preset evaluation indexes;
a test module configured to select a test sample from the set of test samples, and to perform the following test steps: and inputting the test sample into a trained user behavior evaluation model, and adjusting parameters of the trained user behavior evaluation model according to the loss value to obtain the tested user behavior evaluation model.
11. The apparatus of claim 9, further comprising:
a third acquisition module configured to acquire an original estimate of user behavior;
and the estimation module is configured to calculate a final estimation of the user behavior under a preset estimation index according to the characteristic coefficients based on the original estimation and the characteristic coefficients of the user behavior obtained by the trained user behavior estimation model.
12. The apparatus of claim 9, wherein the initial model comprises any one of the following machine learning models:
logistic regression models, decision tree models, random forest models, gradient descent tree models.
13. The apparatus of claim 9, wherein the preset evaluation index comprises any one of:
the daily active user quantity index and the point display ratio index.
14. The apparatus of claim 9, wherein the association pair further comprises a mapping pair of a quantified variation value of user behavior and a significance evaluation conclusion thereof before and after a product content variation and a quantified variation value of a preset evaluation index and a significance evaluation conclusion thereof before and after a product content variation.
15. The apparatus of claim 9 or 14, wherein the quantified change in user behavior before and after a change in product content in the association pair is determined from the change in user behavior before and after a change in product content and the number of user behaviors.
16. The apparatus of claim 9, wherein the user behavior comprises: single user behavior and sequences of user behaviors.
17. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
18. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011543938.XA CN112541705B (en) | 2020-12-23 | 2020-12-23 | Method, device, equipment and storage medium for generating user behavior evaluation model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011543938.XA CN112541705B (en) | 2020-12-23 | 2020-12-23 | Method, device, equipment and storage medium for generating user behavior evaluation model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112541705A CN112541705A (en) | 2021-03-23 |
| CN112541705B true CN112541705B (en) | 2024-01-23 |
Family
ID=75017433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011543938.XA Active CN112541705B (en) | 2020-12-23 | 2020-12-23 | Method, device, equipment and storage medium for generating user behavior evaluation model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112541705B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919684A (en) * | 2019-03-18 | 2019-06-21 | 上海盛付通电子支付服务有限公司 | For generating method, electronic equipment and the computer readable storage medium of information prediction model |
| CN110569427A (en) * | 2019-08-07 | 2019-12-13 | 智者四海（北京）技术有限公司 | Multi-target sequencing model training and user behavior prediction method and device |
| WO2019242222A1 (en) * | 2018-06-21 | 2019-12-26 | 北京字节跳动网络技术有限公司 | Method and device for use in generating information |
| WO2020000876A1 (en) * | 2018-06-27 | 2020-01-02 | 北京字节跳动网络技术有限公司 | Model generating method and device |
| CN111488995A (en) * | 2020-04-08 | 2020-08-04 | 北京字节跳动网络技术有限公司 | Method and apparatus for evaluating a joint training model |
-
2020
- 2020-12-23 CN CN202011543938.XA patent/CN112541705B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019242222A1 (en) * | 2018-06-21 | 2019-12-26 | 北京字节跳动网络技术有限公司 | Method and device for use in generating information |
| WO2020000876A1 (en) * | 2018-06-27 | 2020-01-02 | 北京字节跳动网络技术有限公司 | Model generating method and device |
| CN109919684A (en) * | 2019-03-18 | 2019-06-21 | 上海盛付通电子支付服务有限公司 | For generating method, electronic equipment and the computer readable storage medium of information prediction model |
| CN110569427A (en) * | 2019-08-07 | 2019-12-13 | 智者四海（北京）技术有限公司 | Multi-target sequencing model training and user behavior prediction method and device |
| CN111488995A (en) * | 2020-04-08 | 2020-08-04 | 北京字节跳动网络技术有限公司 | Method and apparatus for evaluating a joint training model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112541705A (en) | 2021-03-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109783632B (en) | Customer service information pushing method and device, computer equipment and storage medium | |
| CN110503531B (en) | Dynamic social scene recommendation method based on time sequence perception | |
| CN104778173B (en) | Target user determination method, device and equipment | |
| KR20220113881A (en) | Method and apparatus for generating pre-trained model, electronic device and storage medium | |
| CN109299258A (en) | A kind of public sentiment event detecting method, device and equipment | |
| WO2018184518A1 (en) | Microblog data processing method and device, computer device and storage medium | |
| CN111242310A (en) | Feature validity evaluation method and device, electronic equipment and storage medium | |
| CN113610239A (en) | Feature processing method and feature processing system for machine learning | |
| CN113360711B (en) | Model training and executing method, device, equipment and medium for video understanding task | |
| CN116561542B (en) | Model optimization training system, method and related device | |
| CN106803092B (en) | Method and device for determining standard problem data | |
| WO2021103401A1 (en) | Data object classification method and apparatus, computer device and storage medium | |
| CN111210332A (en) | Method and device for generating post-loan management strategy and electronic equipment | |
| CN112541705B (en) | Method, device, equipment and storage medium for generating user behavior evaluation model | |
| CN113961765B (en) | Searching method, searching device, searching equipment and searching medium based on neural network model | |
| CN113051911B (en) | Method, apparatus, device, medium and program product for extracting sensitive words | |
| CN112463964B (en) | Text classification and model training method, device, equipment and storage medium | |
| CN115080856A (en) | Recommendation method and device and training method and device of recommendation model | |
| CN112200602A (en) | Neural network model training method and device for advertisement recommendation | |
| CN112905743A (en) | Text object detection method and device, electronic equipment and storage medium | |
| CN112270574A (en) | Method, device, equipment and medium for analyzing abnormal motion in motion execution process | |
| CN116340864B (en) | Model drift detection method, device, equipment and storage medium thereof | |
| CN113190779B (en) | Webpage evaluation method and device | |
| CN113836439B (en) | User matching method, computing device, and computer-readable storage medium | |
| CN114637921B (en) | Item recommendation method, device and equipment based on modeling accidental uncertainty |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |