CN109190791A - Using the appraisal procedure of recommended models, device and electronic equipment - Google Patents
Using the appraisal procedure of recommended models, device and electronic equipment Download PDFInfo
- Publication number
- CN109190791A CN109190791A CN201810829464.1A CN201810829464A CN109190791A CN 109190791 A CN109190791 A CN 109190791A CN 201810829464 A CN201810829464 A CN 201810829464A CN 109190791 A CN109190791 A CN 109190791A
- Authority
- CN
- China
- Prior art keywords
- value
- sample
- samples
- group
- samples estimates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0282—Rating or review of business operators or products
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Physics & Mathematics (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Finance (AREA)
- Educational Administration (AREA)
- Accounting & Taxation (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of appraisal procedures using recommended models, comprising: obtains more parts of Samples Estimates data of target application recommended models；Wherein, every part of Samples Estimates data include the sample predictions probability value and sample true value of a sample；To the more parts of Samples Estimates data groupings, multiple groups Samples Estimates group is obtained；It wherein, include the Samples Estimates data of no less than preset number of samples in each Samples Estimates group；The regrouping prediction value and grouping actual value of Samples Estimates group described in obtaining every group；According to the regrouping prediction value and the grouping actual value, the model evaluation value of the target application recommended models is obtained.It according to the present invention can be by assessing the relationship between the predicted value and true value using recommended models, so that the quality evaluation of target application recommended models is more comprehensively, accurately.
Description
Technical field
The present invention relates to software application technology fields, more particularly, to a kind of appraisal procedure using recommended models, dress
It sets and electronic equipment.
Background technique
Using being the software program for interacting, providing certain particular application services by human-computer interaction interface and user.
It grows at top speed with the rapid development of computer technology with Internet penetration, people have got used to mechanical, electrical using such as hand
This class of electronic devices of brain, from application searches, the application platform such as application shop of downloading, recommendation are provided, acquisition, which meets itself, to be needed
The application asked runs the application after downloading installation and obtains corresponding application service.
In this kind of application platform of application shop, what is provided is normally based on some application recommendation mould using recommendation service
Type, from the extensive application obtained for user, filter out meet user preference application recommend user.Therefore, using pushing away
The quality good or not for recommending model is the key factor for influencing to apply recommendation effect.So how accurate evaluation application recommended models,
Good application is pushed to user with vital influence on ensuring to stablize.
Currently, the assessment to application recommended models in the industry, usually using such as AUC (Area Under Curve, under curve
Area) this kind of classification of assessment superiority and inferiority of assessment method, but AUC assessment main sides focus on and apply the classification of recommended models in evaluation
Accuracy, corresponding assessment is model ranking results using recommended models, can not assess predicted value using recommended models with
Relationship between true value cannot comprehensively and accurately assess the quality using recommended models.
Summary of the invention
It is an object of the present invention to provide a kind of new solutions for application recommended models assessment.
According to the first aspect of the invention, a kind of appraisal procedure using recommended models is provided, comprising:
Obtain more parts of Samples Estimates data of target application recommended models；
Wherein, every part of Samples Estimates data include the sample predictions probability value and sample true value of a sample；
To the more parts of Samples Estimates data groupings, multiple groups Samples Estimates group is obtained；
It wherein, include the Samples Estimates data of no less than preset number of samples in each Samples Estimates group；
The regrouping prediction value and grouping actual value of Samples Estimates group described in obtaining every group；
According to the regrouping prediction value and the grouping actual value, the model for obtaining the target application recommended models is commented
Valuation.
Optionally, further includes:
When the model evaluation value is higher than preset assessment threshold value, the model matter of the target application recommended models is determined
Amount does not meet recommended requirements, triggers the update to the target application recommended models.
Optionally, the step of more parts of Samples Estimates data of acquisition target application recommended models include:
Obtain multiple samples of the target application recommended models model sample value and corresponding sample true value；
When the model sample value is not belonging to preset Samples Estimates range, by preset sample process mode to mould
Type sample value is handled, and the corresponding sample predictions probability value is obtained；
When the model sample value belongs to preset Samples Estimates range, set corresponding for the model sample value
The sample predictions probability value.
Optionally,
The sample process mode is logistic regression processing.
Optionally, include: to the step of more parts of Samples Estimates data groupings
According to the sample predictions probability value, descending sort is carried out to the more parts of Samples Estimates data；
By the more parts of Samples Estimates data after the descending sort, the sample for obtaining meeting preset grouping number is divided
This evaluation group.
Optionally, the step of regrouping prediction value and grouping actual value of Samples Estimates group described in every group of the acquisition is wrapped
It includes:
According to the sample predictions probability value of every part that includes in the Samples Estimates group Samples Estimates data, really
The fixed regrouping prediction value；
According to the sample true value of every part that includes in the Samples Estimates group Samples Estimates data, institute is determined
State grouping actual value.
Optionally,
The step of determining regrouping prediction value includes:
Sample predictions probability value summation to every part of Samples Estimates data for including in the Samples Estimates group,
The result that summation is obtained is as the regrouping prediction value；
And the step of determination grouping actual value, includes:
Sample true value summation to every part of Samples Estimates data for including in the Samples Estimates group, will ask
With obtained result as the grouping actual value.
Optionally, the step of obtaining the model evaluation value of the target application recommended models include:
It the regrouping prediction value of the Samples Estimates group according to every group, the grouping true value and corresponding is commented including sample
Estimate the number of data, determines grouping assessed value；
The grouping assessed value of the Samples Estimates group according to every group, determines the model of the target application recommended models
Assessed value.
Optionally,
The step of determining grouping assessed value includes:
According to the number of the regrouping prediction value, the grouping true value and the Samples Estimates data, determines and correspond to
The grouping difference of two squares；
According to the number of the grouping difference of two squares, regrouping prediction value and the Samples Estimates data, corresponding point is determined
Group assessed value；
And/or
The step of model evaluation value of the determination target application recommended models includes:
The grouping assessed value of Samples Estimates group described in every group is summed, the result that summation is obtained is as the model
Assessed value.
According to the second aspect of the invention, a kind of assessment device using recommended models is provided, comprising:
Data capture unit, for obtaining more parts of Samples Estimates data of target application recommended models；
Wherein, every part of Samples Estimates data include the sample predictions probability value and sample true value of a sample；
Sample packet unit, for obtaining multiple groups Samples Estimates group to the more parts of Samples Estimates data groupings；
It wherein, include the Samples Estimates data of no less than preset number of samples in each Samples Estimates group；
Packet processing unit, regrouping prediction value and grouping actual value for Samples Estimates group described in obtaining every group；
Model evaluation unit, for obtaining the target and answering according to the regrouping prediction value and the grouping actual value
With the model evaluation value of recommended models.
According to the third aspect of the invention we, a kind of electronic equipment is provided, comprising:
Memory, for storing executable instruction；
Processor runs the electronic equipment for the control according to the executable instruction to execute the present invention the
On the one hand the appraisal procedure using recommended models provided.
According to one embodiment of the disclosure, what it is by target application recommended models includes sample predictions probability value and sample
The Samples Estimates data of true value, assessment are recommended using the relationship between the predicted value and true value of recommended models, reflection application
The degree of fitting of model, so that the quality evaluation of target application recommended models is more comprehensively, accurately.
By referring to the drawings to the detailed description of exemplary embodiment of the present invention, other feature of the invention and its
Advantage will become apparent.
Detailed description of the invention
It is combined in the description and the attached drawing for constituting part of specification shows the embodiment of the present invention, and even
With its explanation together principle for explaining the present invention.
Fig. 1 is the block diagram for showing the example of hardware configuration for the computing system that can be used for realizing the embodiment of the present invention.
Fig. 2 shows the flow charts of the appraisal procedure of the embodiment of the present invention.
Fig. 3 shows the flow chart of the acquisition Samples Estimates data step of the embodiment of the present invention.
Fig. 4 shows the flow chart of the acquisition Samples Estimates group step of the embodiment of the present invention.
Fig. 5 shows the flow chart for obtaining regrouping prediction value and being grouped actual value step of the embodiment of the present invention.
Fig. 6 shows the flow chart of the acquisition model evaluation value step of the embodiment of the present invention.
Fig. 7 shows the flow chart of the acquisition grouping assessed value step of the embodiment of the present invention.
Fig. 8 shows the block diagram of the assessment device using recommended models of the embodiment of the present invention.
Fig. 9 shows the another block diagram of the assessment device using recommended models of the embodiment of the present invention.
Figure 10 shows the block diagram of the electronic equipment of the embodiment of the present invention.
Specific embodiment
Carry out the various exemplary embodiments of detailed description of the present invention now with reference to attached drawing.It should also be noted that unless in addition having
Body explanation, the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally
The range of invention.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the present invention
And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as part of specification.
It is shown here and discuss all examples in, any occurrence should be construed as merely illustratively, without
It is as limitation.Therefore, other examples of exemplary embodiment can have different values.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
<hardware configuration>
Fig. 1 is the block diagram for showing the hardware configuration for the electronic equipment 1000 that the embodiment of the present invention may be implemented.
Electronic equipment 1000 can be portable computer, desktop computer, mobile phone, tablet computer etc..As shown in Figure 1, electric
Sub- equipment 1000 may include processor 1100, memory 1200, interface arrangement 1300, communication device 1400, display device
1500, input unit 1600, loudspeaker 1700, microphone 1800 etc..Wherein, processor 1100 can be central processing unit
CPU, Micro-processor MCV etc..Memory 1200 is for example including ROM (read-only memory), RAM (random access memory), such as
The nonvolatile memory etc. of hard disk.Interface arrangement 1300 is for example including USB interface, earphone interface etc..Communication device 1400
It is such as able to carry out wired or wireless communication, specifically may include Wifi communication, Bluetooth communication, 2G/3G/4G/5G communication etc..It is aobvious
Showing device 1500 is, for example, liquid crystal display, touch display screen etc..Input unit 1600 for example may include touch screen, keyboard,
Body-sensing input etc..User can pass through 1800 inputting/outputting voice information of loudspeaker 1700 and microphone.
Electronic equipment shown in FIG. 1 is merely illustrative and is in no way intended to the invention, its application, or uses
Any restrictions.Using in an embodiment of the present invention, the memory 1200 of electronic equipment 1000 is for storing instruction, described
Instruction is operated for controlling the processor 1100 to execute any one application model provided in an embodiment of the present invention
Appraisal procedure.It will be appreciated by those skilled in the art that although showing multiple devices to electronic equipment 1000 in Fig. 1,
The present invention can only relate to partial devices therein, for example, electronic equipment 1000 pertains only to processor 1100 and storage device
1200.Technical staff can disclosed conceptual design instruction according to the present invention.How control processor is operated for instruction, this is
It is known in the art that therefore being not described in detail herein.
<embodiment>
The general plotting of the present embodiment is to provide a kind of new application recommended models evaluation scheme, is pushed away by target application
Recommend model includes the Samples Estimates data of sample predictions probability value and sample true value, and the predicted value of recommended models is applied in assessment
Relationship between true value, the degree of fitting of recommended models is applied in reflection, so that the quality evaluation of target application recommended models is more
Comprehensively, accurately.
<method>
In the present embodiment, a kind of appraisal procedure using recommended models is provided.It should be understood that as assessment object
Application recommended models, be arbitrary can be used for for user recommend meet user preference application data model.Using being
Refer to the software program for interacting, providing certain particular application services by human-computer interaction interface and user, such as: chat software
Program, payment software program, shopping software program etc..
The appraisal procedure of the Application models, as shown in Figure 2, comprising: step S2100-S2400.
Step S2100 obtains more parts of Samples Estimates data of target application recommended models.
In the present embodiment, every part of Samples Estimates data include a sample sample predictions probability value and sample it is true
Value.
Sample is used to characterize the associated statistical information that some user some application occurs some event.Event includes user
Click application, downloading application, installation application, unloading application, browse application etc..
Sample predictions probability value is the probability value based on the corresponding corresponding sample of prediction obtained using recommended models.Example
Such as, the corresponding event of sample be user click in application, sample predictions probability value be namely based on it is corresponding using recommended models, in advance
The probability value that the user for surveying the sample clicks application.
The user that sample true value is used to be characterized in the sample application of sample occurs the truth of some event.Example
Such as, the corresponding event of sample is that user clicks in application, user clicks the application, and sample true value is 1, and user does not click on this and answers
With sample true value is -1.
It in the present embodiment, can include that sample predictions probability value and sample are true by obtain target application recommended models
The Samples Estimates data of real value, the pass between true value and predicted value to combine subsequent step assessment target application recommended models
The degree of fitting of target application recommended models is reflected in system, so that the quality evaluation of target application recommended models is more comprehensively, accurately.
In one example, step S2100 can be as shown in figure 3, include S2110-S2130.
Step S2110, model sample value and the corresponding sample for obtaining multiple samples of target application recommended models are true
Real value.
Specifically, model sample value is the predicted value of the correspondence sample obtained by target application recommended models.
For example, sample I1Model sample value be 0.6, sample I2Model sample value be 1, sample I3Model sample value
It is 20, sample I4Model sample value be 0, sample I5Model sample value be -1, it is as shown in table 1 below:
Table 1
| Sample | Model sample value |
| Sample I1 | 0.6 |
| Sample I2 | 1 |
| Sample I3 | 20 |
| Sample I4 | 0 |
| Sample I5 | -1 |
Sample true value can be the true value whether event in characterization sample occurs, it is assumed that the event in sample occurs
When, sample true value is set as 1, then when the event in sample does not occur, sample true value is set as 0 or -1, and vice versa.This implementation
In example, when the event in sample occurs, sample true value is set as 1 or is set as 0 or is set as -1 without limitation.It is following in sample
Event for sample true value is 1 when occurring, sample true value is 0 or -1 when the event in sample does not occur, for example, sample
I1In event occur, then sample I1Sample true value be 1, sample I2In event do not occur, then sample I2Sample it is true
Value is 0, sample I3In event do not occur, then sample I3Sample true value be -1, it is as shown in table 2 below:
Table 2
| Sample | Sample true value |
| Sample I1 | 1 |
| Sample I2 | 0 |
| Sample I3 | -1 |
Step S2120 passes through preset sample process side when model sample value is not belonging to preset Samples Estimates range
Formula handles model sample value, obtains corresponding sample predictions probability value.
It should be understood that sample in the present embodiment is there may be multiple types, different types of sample is corresponding
Model sample value may have different numberical ranges.In this example, by the way that Samples Estimates range is arranged, according to model sample value
Whether respective handling is carried out within the scope of Samples Estimates, the model sample value of different type sample can be made in same numerical value model
Enclose it is interior assessed, further promoted apply recommended models assessment accuracy.In addition, recommending mould for assessing different applications
When type, similar method also can be used, so that based on the different model sample values obtained using recommended models in same number
It is assessed within the scope of value, it is ensured that the validity of assessment.Samples Estimates range can be set according to specific application scenarios or demand
It sets, for example, it is 0~1 that Samples Estimates range, which can be set,.
In one example, sample process mode can be logistic regression processing.When model sample value is in preset sample
When outside scope of assessment, model sample value obtained above is handled by logistic regression, the end value obtained after processing is set
It is set to corresponding sample predictions probability value.The range of the end value exported after being handled by logistic regression is 0~1, will be a wide range of
Model sample value be compressed in 0~1 range, can eliminate the influence of the variable especially to stand out, that is, eliminate the exceptional value of data.
Hypothesized model sample value is x, and the end value obtained after being handled by logistic regression is S (x), and logistic regression processing can
To use following formula:
For example, sample I1Model sample value be 0.6, after handling by logistic regression, obtain corresponding sample I1Sample
Prediction probability value is 0.65；Sample I2Model sample value be 1, after handling by logistic regression, obtain corresponding sample I2Sample
This prediction probability value is 0.73；Sample I3Model sample value be 20, after handling by logistic regression, obtain corresponding sample I3
Sample predictions probability value be 0.9999999979；Sample I4Model sample value be 0, after being handled by logistic regression, obtain
Corresponding sample I4Sample predictions probability value be 0.5；Sample I5Model sample value be -1, after being handled by logistic regression, obtain
To corresponding sample I5Sample predictions probability value be 0.26, it is as shown in table 3 below:
Table 3
| Sample | Sample predictions probability value |
| Sample I1 | 0.65 |
| Sample I2 | 0.73 |
| Sample I3 | 0.9999999979 |
| Sample I4 | 0.5 |
| Sample I5 | 0.26 |
It should be noted that above-mentioned logistic regression processing can directly pass through formulaIt is handled,
It can also be by the inclusion ofOther formula handled, optionally, " other formula " here can be for conduct
The correction of logistic regression processing or side formula can also beDeformation.
Step S2130 sets corresponding for model sample value when model sample value belongs to preset Samples Estimates range
Sample predictions probability value.
When the model sample value got by S2110 is within the scope of preset Samples Estimates, the model that will acquire
Sample value is set as corresponding sample predictions probability value.
After step S2100, enter:
Step S2200 obtains multiple groups Samples Estimates group to more parts of Samples Estimates data groupings；Wherein, each Samples Estimates
It include the Samples Estimates data of no less than preset number of samples in group.
It specifically, include multiple Samples Estimates data in each Samples Estimates group.The sample for including in Samples Estimates group is commented
The number for estimating data is no less than preset number of samples, wherein number of samples can be configured according to different needs.Example
Such as, it is 10 that number of samples, which can be set,.Assuming that more parts of Samples Estimates data are divided into 5 groups, then the sample in each Samples Estimates group
The number of samples for assessing data can be respectively 10,12,11,15,13；The number of Samples Estimates data in each Samples Estimates group
Mesh can also be respectively 10,10,10,10,10.It should be noted that the number of the Samples Estimates data in each Samples Estimates group
Mesh can be equal, can not also wait, not limit this in the present embodiment, it is only necessary to the Samples Estimates in each Samples Estimates group
The number of data is no less than preset number of samples.
In one example, step S2200 can be as shown in figure 4, include step S2210-S2220.
Step S2210 carries out descending sort to more parts of Samples Estimates data according to sample predictions probability value.
The available sample predictions probability value of S2120 or above-mentioned steps S2130 through the above steps, according to obtained sample
This prediction probability value carries out descending sort to more parts of Samples Estimates data.
Step S2220, by more parts of Samples Estimates data after descending sort, division obtains meeting preset grouping number
Samples Estimates group.
Grouping number can be arranged according to different needs.For example, can be set to 10 groups.
More parts of Samples Estimates data have been carried out descending sort by above-mentioned steps S2210, according to the total of Samples Estimates data
Number of samples, the grouping number of Samples Estimates minimum data in number of samples, each Samples Estimates group, will be more after descending sort
Part Samples Estimates data are grouped, and obtain the Samples Estimates group for meeting preset grouping number.For example, it is assumed that Samples Estimates number
According to total number of samples mesh be 95 parts, the number of samples of Samples Estimates data is no less than 5 parts in each Samples Estimates group, grouping number
It is 10 groups, then by more parts of Samples Estimates data after descending sort, Samples Estimates in each Samples Estimates group for being obtained after division
The number of samples of data can be respectively 10,10,10,10,10,10,10,10,10,5；The each Samples Estimates obtained after division
The number of samples of Samples Estimates data can also be respectively 8,11,9,13,7,12,12,10,6,7 in group.It should be noted that
The present embodiment to how to more parts of Samples Estimates data be grouped division without limitation, it is only necessary to wrapped in each Samples Estimates group
The number for the Samples Estimates group that the number of samples of the Samples Estimates data included is no less than preset number of samples and obtains meets in advance
If grouping number.In general, carrying out equivalent division to more parts of Samples Estimates data, i.e., make every group of Samples Estimates as far as possible
The number of samples of sample evaluating data in group is equal.
Attached drawing is had been combined above and example illustrates how to implementation steps S2200, is entered later:
Step S2300 obtains the regrouping prediction value and grouping actual value of every group of Samples Estimates group.
Regrouping prediction value is the sample predictions probability value of one group of Samples Estimates group.
It is grouped the sample true value that actual value is one group of Samples Estimates group.
In one example, step S2300 can be as shown in Figure 5, comprising: step S2310-S2320.
Step S2310 is determined according to the sample predictions probability value for every part of Samples Estimates data for including in Samples Estimates group
Regrouping prediction value.
It in one example, can be to the sample predictions probability value for every part of Samples Estimates data for including in Samples Estimates group
Summation, the result that summation is obtained is as regrouping prediction value.
Assuming that k-th sample I in i-th group of Samples Estimates groupiKSample predictions probability value be PiK, above-mentioned steps S2210-
According to sample predictions probability value in S2220, descending arrangement is carried out to more parts of Samples Estimates data, and to more after descending arrangement
Part Samples Estimates data divide to obtain Samples Estimates group, it is assumed that all sample predictions probability values are divided into 10 groups of (i.e. ranges of i
It is 1~10), 10 groups of Samples Estimates groups are respectively F1,…,Fi,…,F10, and include n in i-th group of Samples Estimates groupiPart sample is commented
Estimate data, then the regrouping prediction value FP of i-th group of Samples Estimates groupiAre as follows:
FPi=Pi1+Pi2+…+PiK(Pi1, Pi2..., PiK∈Fi)。
Step S2320 determines grouping according to the sample true value for every part of Samples Estimates data for including in Samples Estimates group
Actual value.
In one example, the sample true value for every part of Samples Estimates data for including in Samples Estimates group can be asked
With, will the obtained result of summation as being grouped actual value.
Assuming that k-th sample I in i-th group of Samples Estimates groupiKSample true value be AiK, above-mentioned steps S2210-S2220
It is middle according to sample predictions probability value, descending arrangement has been carried out to more parts of Samples Estimates data, and to more parts of samples after descending arrangement
This assessment data divide to obtain Samples Estimates group, it is assumed that by all sample true values be divided into 10 groups (i.e. the range of i be 1~
10), 10 groups of Samples Estimates groups are respectively F1,…,Fi,…,F10, and include n in i-th group of Samples Estimates groupiPart Samples Estimates number
According to the then grouping actual value FA of i-th group of Samples Estimates groupiAre as follows:
FAi=Ai1+Ai2+…+AiK(Ai1, Ai2..., AiK∈Fi)。
Attached drawing is had been combined above and example illustrates how to implementation steps S2300, is entered later:
Step S2400 obtains the model evaluation of target application recommended models according to regrouping prediction value and grouping actual value
Value.
In the present embodiment, model evaluation value can reflect between the predicted value and true value of target application recommended models
Degree of fitting assesses the relationship between the predicted value and true value of target application recommended models with this.Model evaluation value is smaller, meaning
Degree of fitting between the predicted value and true value of target application recommended models it is higher, target application recommends mould to a certain extent
The quality of type is better, conversely, model evaluation value is bigger, it is meant that between the predicted value and true value of target application recommended models
Degree of fitting is poorer, and the quality of target application recommended models is poorer to a certain extent.
In one example, step S2400 can be as shown in Figure 6, comprising: step S2410-S2420.
Step S2410, according to the regrouping prediction value of every group of Samples Estimates group, grouping true value and it is corresponding include sample
The number of data is assessed, determines grouping assessed value.
In one example, step S2410 can with as shown in fig. 7, comprises: step S2411-S2412.
Step S2411 is determined corresponding according to the number of regrouping prediction value, grouping true value and Samples Estimates data
It is grouped the difference of two squares.
It is grouped the number that the difference of two squares is used to according to regrouping prediction value, be grouped true value and Samples Estimates data, assesses mesh
Mark applies the degree of fitting of recommended models.
Assuming that regrouping prediction value is FPi(wherein, the group number that i is i-th group of Samples Estimates group), it is assumed that being grouped true value is
FAi, it is assumed that it include n in i-th group of Samples Estimates groupiPart Samples Estimates data, the Samples Estimates number for including in every group of Samples Estimates group
According to number be arranged according to specific application scenarios or demand, such as setting ni>=5, corresponding grouping difference of two squares S are as follows:
S=(FAi-ni×FPi)2。
It should be noted that grouping difference of two squares S can directly pass through S=(FAi-ni×FPi)2It obtains, can also pass through
Include (FAi-ni×FPi)2Other formula obtain, optionally, " other formula " here can be for as grouping square
The correction of poor S or side formula can also be (FAi-ni×FPi)2Deformation.
Step S2412 is determined corresponding according to the number of the grouping difference of two squares, regrouping prediction value and Samples Estimates data
It is grouped assessed value.
Assessed value is grouped to be used to assess mesh according to the number of the grouping difference of two squares, regrouping prediction value and Samples Estimates data
Mark applies the degree of fitting of recommended models.
According to above-mentioned steps S2411, the grouping difference of two squares is S=(FAi-ni×FPi)2, regrouping prediction value is FPi, i-th group of sample
The number of Samples Estimates data is n in this evaluation groupiPart, it is corresponding, it is grouped assessed value Q are as follows:
It should be noted that grouping assessed value Q can directly pass through
It obtains, it can also be by the inclusion ofOther formula obtain, optionally, " other formula " here can
To be that for the correction or side formula as grouping assessed value Q or can also beChange
Shape.
Step S2420 determines that the model of target application recommended models is commented according to the grouping assessed value of every group of Samples Estimates group
Valuation.
In one example, the grouping assessed value of every group of Samples Estimates group can be summed, the result that summation is obtained is made
For model evaluation value.
According to above-mentioned steps S2412, being grouped assessed value isI-th
The number of Samples Estimates data is n in group Samples Estimates groupiPart, the group number of Samples Estimates group is k group, then corresponding model evaluation
Value χ2Are as follows:
It should be noted that model evaluation value χ2Can directly it pass throughIt obtains, it can also be by the inclusion ofOther formula obtain, optionally, " other formula " here can be for conduct
Model evaluation value χ2Correction or side formula or can also beDeformation.
By the relationship between the predicted value and true value of target application recommended models, reflect target application recommended models
Degree of fitting, so that the quality evaluation of target application recommended models is more comprehensively, accurately.
After step S2400:
When model evaluation value is higher than preset assessment threshold value, determine that the model quality of target application recommended models is not met
Recommended requirements trigger the update to target application recommended models.
When assessment threshold value is whether the model quality of assessment target application recommended models meets recommended requirements, model evaluation value
Threshold value.Assessment threshold value can be configured according to different application scenarios or different demands.
When model evaluation value be higher than preset assessment threshold value when, it is meant that the predicted value of target application recommended models and really
Value difference is larger, that is, the degree of fitting of target application recommended models is poor, and the quality of target application recommended models, which is not met, to be pushed away
Demand is recommended, needs to be updated target application recommended models.It should be noted that being carried out more to target application recommended models
It newly, may include re -training target application recommended models or the model parameter for updating target application recommended models, it can be with
Target application recommended models are replaced, this is not construed as limiting in the present embodiment.
<article search device>
In the present embodiment, a kind of assessment device 3000 using recommended models is also provided, as shown in Figure 8, comprising: data
Acquiring unit 3100, sample packet unit 3200, packet processing unit 3300, model evaluation unit 3400, for implementing this reality
The appraisal procedure of any one application recommended models provided in example is applied, details are not described herein.
Data capture unit 3100, for obtaining more parts of Samples Estimates data of target application recommended models；Wherein, every part
Samples Estimates data include the sample predictions probability value and sample true value of a sample.
In one example, data capture unit 3100 is used for:
The model sample value of multiple samples of acquisition target application recommended models and corresponding sample true value；
When model sample value is not belonging to preset Samples Estimates range, by preset sample process mode to model sample
This value is handled, and corresponding sample predictions probability value is obtained；
When model sample value belongs to preset Samples Estimates range, corresponding sample predictions are set by model sample value
Probability value.
Optionally, sample process mode is logistic regression processing.
Sample packet unit 3200, for obtaining multiple groups Samples Estimates group to more parts of Samples Estimates data groupings；Wherein,
It include the Samples Estimates data of no less than preset number of samples in each Samples Estimates group.
In one example, sample packet unit 3200 is used for:
According to sample predictions probability value, descending sort is carried out to more parts of Samples Estimates data；
By more parts of Samples Estimates data after descending sort, the Samples Estimates for obtaining meeting preset grouping number are divided
Group.
Packet processing unit 3300, for obtaining the regrouping prediction value and grouping actual value of every group of Samples Estimates group.
In one example, packet processing unit 3300 is used for:
According to the sample predictions probability value for every part of Samples Estimates data for including in Samples Estimates group, regrouping prediction is determined
Value；
According to the sample true value for every part of Samples Estimates data for including in Samples Estimates group, grouping actual value is determined.
Further, in one example, packet processing unit 3300 is also used to:
Sample predictions probability value summation to every part of Samples Estimates data for including in Samples Estimates group, summation is obtained
As a result it is used as regrouping prediction value；
Sample true value summation to every part of Samples Estimates data for including in Samples Estimates group, the result that summation is obtained
As grouping actual value.
Model evaluation unit 3400, for obtaining target application and recommending mould according to regrouping prediction value and grouping actual value
The model evaluation value of type.
In one example, model evaluation unit 3400 is used for:
According to the regrouping prediction value of every group of Samples Estimates group, grouping true value and corresponding including Samples Estimates data
Number determines grouping assessed value；
According to the grouping assessed value of every group of Samples Estimates group, the model evaluation value of target application recommended models is determined.
Further, in one example, model evaluation unit 3400 is also used to:
According to the number of regrouping prediction value, grouping true value and Samples Estimates data, the corresponding grouping difference of two squares is determined；
According to the number of the grouping difference of two squares, regrouping prediction value and Samples Estimates data, corresponding grouping assessed value is determined；
And/or
By the grouping assessed value summation of every group of Samples Estimates group, the result that summation is obtained is as model evaluation value.
In one example, as shown in figure 9, using recommended models assessment device further include: update trigger unit 3500.
When model evaluation value is higher than preset assessment threshold value, trigger unit 3500 is updated, for determining that target application pushes away
The model quality for recommending model does not meet recommended requirements, triggers the update to target application recommended models.
It will be appreciated by those skilled in the art that the assessment device using recommended models can be realized by various modes
3000.For example, can realize the assessment device 3000 using recommended models by instruction configuration processor.For example, can incite somebody to action
Instruction is stored in the ROM, and when starting the device, and instruction is read in programming device from ROM and is realized using recommendation
The assessment device 3000 of model.For example, can by application recommended models assessment device 3000 be cured to dedicated devices (such as
ASIC in).The assessment device 3000 of application recommended models can be divided into mutually independent unit, or they can be merged
It realizes together.It can be realized by one of above-mentioned various implementations using the assessment device 3000 of recommended models,
Or it can be realized by the combination of two or more modes in above-mentioned various implementations.
In the present embodiment, using the assessment device 3000 of recommended models can specific various forms of implementation, for example, using
The assessment device 3000 of recommended models can be any software product for providing and applying recommendation service function, such as application shop
Deng alternatively, can be set using the assessment device 3000 of recommended models in being able to achieve any electronics using recommendation service function
In equipment, for example setting is in client server or partial function unit is arranged in client, partial function
Unit is arranged in server etc..
<electronic equipment>
In the present embodiment, a kind of electronic equipment 4000 is also provided, as shown in Figure 10, comprising:
Memory 4100, for storing executable instruction；
Processor 4200, for the control according to executable instruction, operation electronic equipment is executed as mentioned in the present embodiment
The appraisal procedure of any one application recommended models of confession.
In the present embodiment, electronic equipment 4000 is the electronic equipment that arbitrarily may be implemented using recommendation function, such as hand
Machine, tablet computer, palm PC, laptop or desktop computer etc., electronic equipment 4000 can also include that other are hard
Part device, for example, electronic equipment 1000 as shown in Figure 1.
Attached drawing is had been combined above and example describes the embodiment of the present invention, according to the present embodiment, provides a kind of application
Evaluation method, device and the electronic equipment of recommended models include sample predictions probability by obtain target application recommended models
The Samples Estimates data of value and sample true value, it is real to the regrouping prediction value and grouping that obtain every group after Samples Estimates data grouping
Actual value is obtained the model evaluation value of target application recommended models, is obtained and answered with this according to regrouping prediction value and grouping actual value
With the degree of fitting of recommended models, the relationship between the true value and predicted value of assessment target application recommended models is realized, so that mesh
Mark is more comprehensive, accurate using the quality evaluation of recommended models.
The present invention can be system, method and/or computer program product.Computer program product may include computer
Readable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the invention.
Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment
Equipment.Computer readable storage medium for example can be-- but it is not limited to-- storage device electric, magnetic storage apparatus, optical storage
Equipment, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium
More specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only deposits
It is reservoir (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable
Compact disk read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example thereon
It is stored with punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Calculating used herein above
Machine readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations lead to
It crosses the electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of waveguide or the propagation of other transmission mediums or is transmitted by electric wire
Electric signal.
Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing operation of the present invention can be assembly instruction, instruction set architecture (ISA) instructs,
Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages
The source code or object code that any combination is write, the programming language include the programming language-of object-oriented such as
Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer
Readable program instructions can be executed fully on the user computer, partly execute on the user computer, be only as one
Vertical software package executes, part executes on the remote computer or completely in remote computer on the user computer for part
Or it is executed on server.In situations involving remote computers, remote computer can pass through network-packet of any kind
It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit
It is connected with ISP by internet).In some embodiments, by utilizing computer-readable program instructions
Status information carry out personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or can
Programmed logic array (PLA) (PLA), the electronic circuit can execute computer-readable program instructions, to realize each side of the invention
Face.
Referring herein to according to the method for the embodiment of the present invention, the flow chart of device (system) and computer program product and/
Or block diagram describes various aspects of the invention.It should be appreciated that flowchart and or block diagram each box and flow chart and/
Or in block diagram each box combination, can be realized by computer-readable program instructions.
These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas
The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas
When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced
The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to
It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction
Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram
The instruction of the various aspects of defined function action.
Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other
In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce
Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment
Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.
The flow chart and block diagram in the drawings show the system of multiple embodiments according to the present invention, method and computer journeys
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use
The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box
It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel
Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or
The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic
The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.It is right
For those skilled in the art it is well known that, by hardware mode realize, by software mode realize and pass through software and
It is all of equal value that the mode of combination of hardware, which is realized,.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In principle, the practical application or to the technological improvement in market for best explaining each embodiment, or make the art its
Its those of ordinary skill can understand each embodiment disclosed herein.The scope of the present invention is defined by the appended claims.
Claims (11)
1. a kind of appraisal procedure using recommended models, wherein include:
Obtain more parts of Samples Estimates data of target application recommended models；
Wherein, every part of Samples Estimates data include the sample predictions probability value and sample true value of a sample；
To the more parts of Samples Estimates data groupings, multiple groups Samples Estimates group is obtained；
It wherein, include the Samples Estimates data of no less than preset number of samples in each Samples Estimates group；
The regrouping prediction value and grouping actual value of Samples Estimates group described in obtaining every group；
According to the regrouping prediction value and the grouping actual value, the model evaluation of the target application recommended models is obtained
Value.
2. according to the method described in claim 1, wherein, further includes:
When the model evaluation value is higher than preset assessment threshold value, the model quality of the target application recommended models is determined not
Meet recommended requirements, triggers the update to the target application recommended models.
3. according to the method described in claim 1, wherein, the more parts of Samples Estimates data for obtaining target application recommended models
The step of include:
Obtain multiple samples of the target application recommended models model sample value and corresponding sample true value；
When the model sample value is not belonging to preset Samples Estimates range, by preset sample process mode to model sample
This value is handled, and the corresponding sample predictions probability value is obtained；
When the model sample value belongs to preset Samples Estimates range, set corresponding described for the model sample value
Sample predictions probability value.
4. according to the method described in claim 3, wherein,
The sample process mode is logistic regression processing.
5. according to the method described in claim 1, wherein, including: to the step of more parts of Samples Estimates data groupings
According to the sample predictions probability value, descending sort is carried out to the more parts of Samples Estimates data；
By the more parts of Samples Estimates data after the descending sort, the sample that division obtains meeting preset grouping number is commented
Estimate group.
6. according to the method described in claim 1, wherein, it is described obtain every group described in Samples Estimates group regrouping prediction value and
Be grouped actual value the step of include:
According to the sample predictions probability value of every part that includes in the Samples Estimates group Samples Estimates data, institute is determined
State regrouping prediction value；
According to the sample true value of every part that includes in the Samples Estimates group Samples Estimates data, described point is determined
Group actual value.
7. according to the method described in claim 6, wherein,
The step of determining regrouping prediction value includes:
Sample predictions probability value summation to every part of Samples Estimates data for including in the Samples Estimates group, will ask
With obtained result as the regrouping prediction value；
And the step of determination grouping actual value, includes:
Sample true value summation to every part of Samples Estimates data for including in the Samples Estimates group, will sum
The result arrived is as the grouping actual value.
8. according to the method described in claim 1, wherein, the step of obtaining the model evaluation value of the target application recommended models
Include:
The regrouping prediction value of the Samples Estimates group according to every group, the grouping true value and it is corresponding include Samples Estimates number
According to number, determine grouping assessed value；
The grouping assessed value of the Samples Estimates group according to every group, determines the model evaluation of the target application recommended models
Value.
9. according to the method described in claim 8, wherein,
The step of determining grouping assessed value includes:
According to the number of the regrouping prediction value, the grouping true value and the Samples Estimates data, corresponding point is determined
The group difference of two squares；
According to the number of the grouping difference of two squares, regrouping prediction value and the Samples Estimates data, determine that corresponding grouping is commented
Valuation；
And/or
The step of model evaluation value of the determination target application recommended models includes:
The grouping assessed value of Samples Estimates group described in every group is summed, the result that summation is obtained is as the model evaluation
Value.
10. a kind of assessment device using recommended models, wherein include:
Data capture unit, for obtaining more parts of Samples Estimates data of target application recommended models；
Wherein, every part of Samples Estimates data include the sample predictions probability value and sample true value of a sample；
Sample packet unit, for obtaining multiple groups Samples Estimates group to the more parts of Samples Estimates data groupings；
It wherein, include the Samples Estimates data of no less than preset number of samples in each Samples Estimates group；
Packet processing unit, regrouping prediction value and grouping actual value for Samples Estimates group described in obtaining every group；
Model evaluation unit, for obtaining the target application and pushing away according to the regrouping prediction value and the grouping actual value
Recommend the model evaluation value of model.
11. a kind of electronic equipment, wherein include:
Memory, for storing executable instruction；
Processor runs the electronic equipment for the control according to the executable instruction to execute the claim
The appraisal procedure for any one application recommended models that 1-9 is provided.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810829464.1A CN109190791A (en) | 2018-07-25 | 2018-07-25 | Using the appraisal procedure of recommended models, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810829464.1A CN109190791A (en) | 2018-07-25 | 2018-07-25 | Using the appraisal procedure of recommended models, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109190791A true CN109190791A (en) | 2019-01-11 |
Family
ID=64937383
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810829464.1A Pending CN109190791A (en) | 2018-07-25 | 2018-07-25 | Using the appraisal procedure of recommended models, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109190791A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110618936A (en) * | 2019-08-29 | 2019-12-27 | 凡普数字技术有限公司 | Application performance evaluation method and device and storage medium |
| CN110825956A (en) * | 2019-09-17 | 2020-02-21 | 中国平安人寿保险股份有限公司 | Information flow recommendation method and device, computer equipment and storage medium |
| CN110866776A (en) * | 2019-10-12 | 2020-03-06 | 上海掌门科技有限公司 | Data calibration method for popularization resource, electronic device and readable storage medium |
| CN111724238A (en) * | 2020-06-23 | 2020-09-29 | 中国平安人寿保险股份有限公司 | Method, device and equipment for evaluating product recommendation accuracy and storage medium |
| EP3828804A1 (en) * | 2019-11-27 | 2021-06-02 | Tata Consultancy Services Limited | Method and system for recommender model selection |
| CN113190746A (en) * | 2021-04-27 | 2021-07-30 | 北京百度网讯科技有限公司 | Recommendation model evaluation method and device and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104616065A (en) * | 2015-02-13 | 2015-05-13 | 北京嘀嘀无限科技发展有限公司 | Method and device for processing order |
| US9224104B2 (en) * | 2013-09-24 | 2015-12-29 | International Business Machines Corporation | Generating data from imbalanced training data sets |
| CN106503386A (en) * | 2016-11-07 | 2017-03-15 | 上海思源弘瑞自动化有限公司 | The good and bad method and device of assessment luminous power prediction algorithm performance |
| CN107613022A (en) * | 2017-10-20 | 2018-01-19 | 广州优视网络科技有限公司 | Content delivery method, device and computer equipment |
| CN107909427A (en) * | 2017-10-25 | 2018-04-13 | 浙江大学 | A kind of Recognition with Recurrent Neural Network method for lifting recommended models timing driving ability |
-
2018
- 2018-07-25 CN CN201810829464.1A patent/CN109190791A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9224104B2 (en) * | 2013-09-24 | 2015-12-29 | International Business Machines Corporation | Generating data from imbalanced training data sets |
| CN104616065A (en) * | 2015-02-13 | 2015-05-13 | 北京嘀嘀无限科技发展有限公司 | Method and device for processing order |
| CN106503386A (en) * | 2016-11-07 | 2017-03-15 | 上海思源弘瑞自动化有限公司 | The good and bad method and device of assessment luminous power prediction algorithm performance |
| CN107613022A (en) * | 2017-10-20 | 2018-01-19 | 广州优视网络科技有限公司 | Content delivery method, device and computer equipment |
| CN107909427A (en) * | 2017-10-25 | 2018-04-13 | 浙江大学 | A kind of Recognition with Recurrent Neural Network method for lifting recommended models timing driving ability |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110618936A (en) * | 2019-08-29 | 2019-12-27 | 凡普数字技术有限公司 | Application performance evaluation method and device and storage medium |
| CN110825956A (en) * | 2019-09-17 | 2020-02-21 | 中国平安人寿保险股份有限公司 | Information flow recommendation method and device, computer equipment and storage medium |
| CN110866776A (en) * | 2019-10-12 | 2020-03-06 | 上海掌门科技有限公司 | Data calibration method for popularization resource, electronic device and readable storage medium |
| CN110866776B (en) * | 2019-10-12 | 2023-11-24 | 上海掌门科技有限公司 | Data calibration method for popularization resources, electronic equipment and readable storage medium |
| EP3828804A1 (en) * | 2019-11-27 | 2021-06-02 | Tata Consultancy Services Limited | Method and system for recommender model selection |
| CN111724238A (en) * | 2020-06-23 | 2020-09-29 | 中国平安人寿保险股份有限公司 | Method, device and equipment for evaluating product recommendation accuracy and storage medium |
| CN113190746A (en) * | 2021-04-27 | 2021-07-30 | 北京百度网讯科技有限公司 | Recommendation model evaluation method and device and electronic equipment |
| CN113190746B (en) * | 2021-04-27 | 2024-01-12 | 北京百度网讯科技有限公司 | Recommendation model evaluation method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109190791A (en) | Using the appraisal procedure of recommended models, device and electronic equipment | |
| US11651233B2 (en) | Candidate visualization techniques for use with genetic algorithms | |
| CN106993104B (en) | Incoming call processing method and device and terminal | |
| US20150178134A1 (en) | Hybrid Crowdsourcing Platform | |
| US11188809B2 (en) | Optimizing personality traits of virtual agents | |
| TW201928709A (en) | Method and apparatus for merging model prediction values, and device | |
| CN111061956B (en) | Method and apparatus for generating information | |
| US20170017908A1 (en) | Churn risk scoring using call network analysis | |
| CN108932320A (en) | Article search method, apparatus and electronic equipment | |
| CN109388674A (en) | Data processing method, device, equipment and readable storage medium storing program for executing | |
| CN105335875A (en) | Purchasing power prediction method and purchasing power prediction device | |
| CN116541610B (en) | Training method and device for recommendation model | |
| US20180349476A1 (en) | Evaluating theses using tree structures | |
| CN108596711A (en) | Using recommendation method, apparatus and electronic equipment | |
| CN110866625A (en) | Promotion index information generation method and device | |
| US20160019564A1 (en) | Evaluating device readiness | |
| CN109710811A (en) | Detection method, equipment and the application system of user's portrait | |
| US20150278731A1 (en) | Generating a functional performance index associated with software development | |
| CN112132323A (en) | Method and device for predicting value amount of commodity object and electronic equipment | |
| KR20210048818A (en) | Apparatus and method for trade based on artificial intelligence using fintech | |
| CN111445139A (en) | Business process simulation method and device, storage medium and electronic equipment | |
| WO2023049280A1 (en) | Systems and methods to screen a predictive model for risks of the predictive model | |
| US11182606B2 (en) | Converting chart data | |
| CN109902698A (en) | Information generating method and device | |
| CN114218496A (en) | Object recommendation method, device and equipment, medium and product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200527 Address after: 310051 room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: 510627, room 146-150, first floor, No. 07, Whampoa Avenue, Tianhe District, Guangdong, Guangzhou Applicant before: GUANGZHOU UC NETWORK TECHNOLOGY Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190111 |