CN109462751A - The appraisal procedure and device of prediction model - Google Patents
The appraisal procedure and device of prediction model Download PDFInfo
- Publication number
- CN109462751A CN109462751A CN201811224095.XA CN201811224095A CN109462751A CN 109462751 A CN109462751 A CN 109462751A CN 201811224095 A CN201811224095 A CN 201811224095A CN 109462751 A CN109462751 A CN 109462751A
- Authority
- CN
- China
- Prior art keywords
- wonderful
- prediction
- prediction model
- video
- segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
- H04N21/4756—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for rating content, e.g. scoring a recommended movie
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of appraisal procedure of prediction model and devices, considerably complicated to solve existing appraisal procedure, assess the problem of inaccuracy.Wherein method includes: to obtain multiple videos for assessment prediction model；For each video, at least one wonderful extracted from current video is obtained；Current video is predicted using the prediction model to obtain the corresponding prediction curve of current video, and extracts at least one high score segment from the prediction curve；According to the wonderful and the high score segment, determine the prediction model to the Prediction Parameters of current video；According to the prediction model to the Prediction Parameters of each video, the evaluation index of the prediction model is calculated.Present invention greatly reduces the workloads of mark personnel, simplify evaluation process, improve assessment efficiency, and evaluation process is more objective, to keep assessment result more accurate.
Description
Technical field
The present invention relates to model evaluation technical fields, appraisal procedure and a kind of prediction more particularly to a kind of prediction model
The assessment device of model.
Background technique
With the rapid development of Internet technology, people, which are increasingly dependent on, obtains various information by internet.For example it is
Meet user to the viewing demand of video, a large amount of video website occurs therewith.Video website refers to flat in perfect technology
Under platform is supported, the online smoothness of Internet user is allowed to issue, the network media of browsing and sharing video frequency works.
Video website generally can do various analyses to online video, analyze whether online video is liked by spectators, point
The part that analysis video is liked by spectators is promoted and is runed etc. to do to orient with this, the user experience of Lai Tigao video website.Video
Website can accumulate the data of many user's viewings, such as played data, dragging data, barrage data after runing a period of time
Deng can be analyzed by various user data online video.Video website calculates view after doing various video analysis
Frequency can reflect the favorite variation tendency of video, video website after these scores are drawn curve in each second score
The reason of can analyzing at curve ascendant trend and peak, and the decline of analysis curve and reason at a low ebb, in turn
Make targeted decision.
It is trained the available prediction model to video by the curve to multitude of video, is by the prediction model
It is predictable to obtain the corresponding curve of video.If making precisely effective orientation to promote, to guarantee prediction model prediction
Accuracy, the suggestion otherwise provided are invalid.The standard of prediction model prediction can be determined by assessing prediction model
True property.In the prior art, the appraisal procedure generallyd use is to look for a collection of video, after manually finishing watching video in advance, for video
Each second get score, these true scores of score got as video, with prediction model to pre- at video each second
Score is measured, error is calculated according to the score of prediction and true score, finally obtains the accuracy of prediction model.
But the problem of this assessment maximum is manually to get score to video each second to be difficult to realize.The reason is that needing
Mark personnel look first at complete video, have the understanding of entirety to entire video, video are then seen again again, in video
Each second makes judge and provides score, this workload is very big, and mark personnel will in the opposite height of the score beaten each second
The excellent degree for objectively evaluating video each second, since the time of video is longer, before mark personnel may see that back has been forgotten
Face, subsequent video with respect to front video it is excellent whether mark personnel and forgotten, objective appraisal cannot be made, into
And cannot the result based on mark prediction model done accurately have evaluated.Therefore, existing appraisal procedure is considerably complicated, and
And assessment inaccuracy.
Summary of the invention
The embodiment of the present invention provides a kind of appraisal procedure of prediction model and a kind of assessment device of prediction model, to solve
Existing appraisal procedure is considerably complicated, and assesses the problem of inaccuracy.
In order to solve the above-mentioned technical problem, the embodiment of the invention provides a kind of appraisal procedures of prediction model, comprising:
Obtain multiple videos for assessment prediction model；
For each video, at least one wonderful extracted from current video is obtained；
Predicted to obtain the corresponding prediction curve of current video to current video using the prediction model, and from described
At least one high score segment is extracted in prediction curve；
According to the wonderful and the high score segment, determine the prediction model to the Prediction Parameters of current video；
According to the prediction model to the Prediction Parameters of each video, the evaluation index of the prediction model is calculated.
Preferably, described according to the wonderful and the high score segment, determine the prediction model to current video
Prediction Parameters the step of, comprising: be directed to each wonderful, judge whether current wonderful is located at the high score segment
In；If so, determining that current wonderful is to predict accurate wonderful；Foundation predicts the number of accurate wonderful,
The prediction model is calculated to the Prediction Parameters of current video.
Preferably, the step for judging current wonderful and whether being located in the high score segment, comprising: according to respective
The corresponding beginning and ending time, judge current wonderful whether with the high score segment there are intersections；Intersection if it exists then calculates and works as
Preceding wonderful with there are the friendship of the high score segment of intersection and ratios；If described hand over and than being greater than preset threshold, it is determined that current essence
Color segment is located in the high score segment.
Preferably, the Prediction Parameters include accuracy rate and/or recall rate, and the foundation predicts accurate wonderful
Number, the step of calculating Prediction Parameters of the prediction model to current video, comprising: calculate the prediction accurately excellent
The ratio of the total number of the number and high score segment of section, as accuracy rate；And/or the calculating prediction is accurately excellent
The ratio of the number of segment and the total number of the wonderful, as recall rate.
On the other hand, the embodiment of the invention also provides a kind of assessment devices of prediction model, comprising:
Module is obtained, for obtaining multiple videos for assessment prediction model；
First extraction module obtains at least one wonderful extracted from current video for being directed to each video；
Second extraction module, it is corresponding for being predicted to obtain current video to current video using the prediction model
Prediction curve, and at least one high score segment is extracted from the prediction curve；
Determining module, for according to the wonderful and the high score segment, determining the prediction model to working as forward sight
The Prediction Parameters of frequency；
Computing module calculates the prediction model for the Prediction Parameters according to the prediction model to each video
Evaluation index.
Preferably, the determining module includes: segment judging submodule, and for being directed to each wonderful, judgement is current
Whether wonderful is located in the high score segment；Segment determines submodule, if being judged as YES for segment judging unit, really
Settled preceding wonderful is to predict accurate wonderful；Parameter computation module, for according to the accurate wonderful of prediction
Number, calculate the prediction model to the Prediction Parameters of current video.
Preferably, the segment judging submodule includes: intersection judging unit, when for according to corresponding start-stop
Between, judge current wonderful whether with the high score segment there are intersections；Ratio calculation unit, if judging for the intersection
Subelement is judged as that there are intersections, then calculates current wonderful and there are the friendship of the high score segment of intersection and ratios；Compare determination
Unit, if for the friendship and than being greater than preset threshold, it is determined that current wonderful is located in the high score segment.
Preferably, the Prediction Parameters include accuracy rate and/or recall rate, and the parameter computation module includes: first
Computing unit is made for calculating the ratio of the number of the accurate wonderful of prediction and the total number of the high score segment
For accuracy rate；And/or second computing unit, for calculating the number of the accurate wonderful of prediction and described excellent
The ratio of the total number of section, as recall rate.
In embodiments of the present invention, multiple videos for assessment prediction model are obtained first；Then it is directed to each video,
At least one wonderful extracted from current video is obtained, and current video is predicted to be worked as using prediction model
The corresponding prediction curve of preceding video, and at least one high score segment is extracted from prediction curve, according to wonderful and high fragment
Section, determines prediction model to the Prediction Parameters of current video；It is last to be calculated according to Prediction Parameters of the prediction model to each video
The evaluation index of the prediction model.It follows that no longer needing to mark per second progress of the personnel to video in the embodiment of the present invention
Marking, but wonderful is simply extracted from video, it gives a mark without to the wonderful, therefore greatly reduce
The workload of mark personnel, simplifies evaluation process, improves assessment efficiency, and evaluation process is more objective, to make
Assessment result is more accurate.
Detailed description of the invention
Fig. 1 is a kind of step flow chart of the appraisal procedure of prediction model of the embodiment of the present invention；
Fig. 2 is a kind of tendency of prediction curve of the embodiment of the present invention and the schematic diagram of discretization；
Fig. 3 is a kind of distribution schematic diagram of the wonderful and high score segment of the embodiment of the present invention after time shaft sequence；
Fig. 4 is a kind of structural block diagram of the assessment device of prediction model of the embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Referring to Fig.1, a kind of step flow chart of the appraisal procedure of prediction model of the embodiment of the present invention is shown.
The appraisal procedure of the prediction model of the embodiment of the present invention the following steps are included:
Step 101, multiple videos for assessment prediction model are obtained.
In practical applications, it can first pass through in advance and the multitude of video in video website is analyzed, obtain each video
Corresponding curve, and be trained using these curves, obtain the prediction model for carrying out video estimation.In order to guarantee to predict
The accuracy of model can assess prediction model, and what the embodiment of the present invention described is the assessment side to prediction model
Method.
In the embodiment of the present invention, multiple videos can be obtained from video website, the effect of these videos is to prediction
Model is assessed.For obtaining the particular number of the video for assessment prediction model, those skilled in the art are according to reality
Experience selects any suitable value, the embodiment of the present invention to this with no restriction.The number of videos of acquisition is bigger, the standard of assessment
True property is also bigger.For example, 100 videos can be obtained from video website for assessing the prediction model.
Step 102, for each video, at least one wonderful extracted from current video is obtained.
It, can be by marking personnel after getting multiple videos for assessment prediction model in the embodiment of the present invention
Each video is labeled, to extract at least one wonderful from video.For example, mark rule can be preset
Then, it is specified which type of segment should be used as wonderful, then organize mark personnel according to the mark rule of setting, difference needle
Wonderful therein is gone out to each video labeling, namely extracts wonderful therein.Therefore mark personnel no longer need
Specific score is provided for each second quality of video, it is only necessary to the quality after video according to video different fragments is finished watching, from
Several more excellent segments are found out in video, these segments do not need to give a mark yet, do not need to provide specific 0-100 it
Between score.The work load of mark personnel is dramatically reduced in this way, also dramatically saves time, such mark cost
It is very little.
It is analyzed respectively for each video for assessment prediction model, to obtain prediction model to each video
Prediction Parameters.Analytic process to each video is identical, therefore mainly to be directed to a video in the embodiment of the present invention
It is introduced for being analyzed.
For current video, at least one wonderful extracted from current video is obtained.Wonderful indicates mark
What personnel marked out, more excellent segment in current video.
Wherein, current video indicates the video analyzed.For example, if to the 1st in 100 videos
Video is analyzed, then current video is the 1st video；If analyzed the 2nd video in 100 videos,
Then current video is the 2nd video, etc..
Step 103, current video is predicted using the prediction model to obtain the corresponding prediction curve of current video,
And at least one high score segment is extracted from the prediction curve.
Current video is input in prediction model, current video is predicted using prediction model, it is available to work as
The corresponding prediction curve of preceding video, which is prediction result of the prediction model to current video.
In prediction curve, the horizontal axis of prediction curve indicates the time, and the longitudinal axis indicates excellent score value, therefore prediction curve is vertical
Height on axis is different, and the excellent score value indicated is different.It, can be bent from the corresponding prediction of current video in the embodiment of the present invention
At least one high score segment is extracted in line.High score segment indicates what prediction model predicted, and excellent score value is higher in current video
Segment.
In a kind of specific implementation, high score segment can be extracted by the way of by prediction curve discretization.It can be with
Using the ensemble average value of prediction curve as reference data, ensemble average value is the average excellent score value of video, in prediction curve
It is that prediction model thinks more excellent part higher than the part of average value, it in this way can be by prediction curveization point by average value
For two parts: higher than the part of average value and sub-average part.The segment that wherein will be above average value is referred to as high score
Segment, sub-average segment are referred to as low fragment section.
Specifically, the average value of whole excellent score values in prediction curve is calculated；Score value excellent in prediction curve is higher than flat
The snippet extraction of mean value goes out, these segments of extraction are high score segment, these segments are that discretization, prediction model are considered
The part of high score.
Referring to Fig. 2, a kind of tendency of prediction curve of the embodiment of the present invention and the schematic diagram of discretization are shown.In Fig. 2
Horizontal axis indicates the time, and the longitudinal axis indicates that excellent score value, curve therein indicate the excellent score value tendency of prediction curve, excellent score value 60
Straight line between to 80 indicates that the average value of all excellent score values, Blocked portion indicate that excellent score value is higher than average in prediction curve
The high score segment of value.
Step 104, according to the wonderful and the high score segment, determine the prediction model to the pre- of current video
Survey parameter.
Through the above steps 102 and step 103, the corresponding wonderful of current video and high score segment are obtained, according to essence
Color segment and high score segment, that is, can determine prediction model to the Prediction Parameters of current video.
In a kind of specific implementation, which may include:
A1 is determined according to the wonderful and the high score segment and is predicted accurate wonderful.
Wonderful is the more excellent segment that mark personnel mark out, and high score segment is the essence that prediction model predicts
The color higher segment of score value.It can determine that prediction model is directed to the prediction of each wonderful according to wonderful and high score segment
It is whether accurate.
It based on the idea that can be in the embodiment of the present invention are as follows: prediction model is directed to the prediction curve that current video is predicted
In, the mountain portions (namely high score segment) of the prediction curve should coincide with wonderful, as long as coincideing can think pre-
Survey model to the prediction of wonderful be meet it is expected.Namely, it is believed that as long as wonderful is located in high score segment, in advance
Surveying model is accurate to the prediction of the wonderful.
Therefore, step A1 can specifically include: being directed to each wonderful, judges whether current wonderful is located at institute
It states in high score segment；If so, determining that current wonderful is to predict accurate wonderful.
It, can be according to current wonderful and high score segment when judging whether current wonderful is located in high score segment
Registration judge.For example, can according to current wonderful and high score segment friendship and than (Intersection-over-
Union, IOU), to determine the registration of current wonderful Yu high score segment.Therefore, judge whether current wonderful is located at
Step in the high score segment may include:
A11 judges whether current wonderful exists with the high score segment and hands over according to corresponding beginning and ending time
Collection.
It is possible, firstly, to which high score segment and wonderful are taken intersection.When the personnel of mark extract wonderful from video,
The wonderful corresponding beginning and ending time, such as [20-120,230-350 ...] can be marked, indicates first wonderful
Beginning and ending time is the 20th second to the 120th second, and the beginning and ending time of second wonderful is the 230th second to the 350th second, etc..?
When extracting high score segment from prediction curve, the beginning and ending time of high score segment can also be obtained from the horizontal axis of prediction curve.
In the embodiment of the present invention, according to corresponding beginning and ending time, it can be determined that current wonderful whether with high score
There are intersections for segment.
It, can be according to corresponding beginning and ending time, on a timeline by wonderful in a kind of specific implementation
It sorts with high score segment.A kind of wonderful of the embodiment of the present invention is shown referring to Fig. 3 and high score segment sorts in time shaft
Distribution schematic diagram afterwards.In Fig. 3, curve indicates the corresponding prediction curve of current video, and a, b, c expression are extracted from prediction curve
3 high score segments, A, B, C, D indicate 4 wonderfuls extracted from current video.From the figure 3, it may be seen that wonderful A and height
There are intersections by fragment section a, and wonderful B and high score segment b are there are intersection, and there are intersections with high score segment d by wonderful D.
A12, if it exists intersection then calculate current wonderful and there are the friendship of the high score segment of intersection and ratios.
It hands over and the molecule of ratio is intersection, be exactly the length of high score segment and wonderful intersection segment, denominator union is exactly
The length of high score segment and wonderful union segment, the ratio of the two are to hand over and compare.
Therefore, calculate current wonderful with there are the friendship of the high score segment of intersection and than the step of may include: determination
Current wonderful with there are the intersection time spans of the high score segment of intersection, and determine current wonderful and there are intersections
High score segment union time span；The ratio for calculating the intersection time span Yu the union time span, as working as
Preceding wonderful with there are the friendship of the high score segment of intersection and ratios.
For example, the beginning and ending time of wonderful A is [30,120], the beginning and ending time of high score segment a is [20,40], then smart
The intersection time span of color segment A and high score segment a is 10, and union time span is 100, then wonderful A and high score segment a
Friendship and than be 0.1.The beginning and ending time of wonderful B is [160,250], and the beginning and ending time of high score segment b is [170,260],
Then the intersection time span of wonderful B and high score segment b is 80, and union time span is 100, then wonderful B and high score
The friendship of segment b and than being 0.8.The beginning and ending time of wonderful D is [450,530], beginning and ending time of high score segment d be [460,
550], then the intersection time span of wonderful D and high score segment d are 70, and union time span is 100, then wonderful D with
The friendship of high score segment d and than being 0.7.
A13, if described hand over and than being greater than preset threshold, it is determined that current wonderful is located in the high score segment.
It hands over and the value of ratio is bigger, it may be considered that the registration of high score segment and wonderful is higher, that is, prediction mould
The high score segment of type prediction is more consistent with true wonderful, therefore prediction model prediction is more accurate.Therefore, the present invention is real
A threshold value can be preset by applying in example, be handed over and if current wonderful is corresponding than that can determine current greater than preset threshold
Wonderful is located in high score segment, namely current wonderful is to predict accurate wonderful.
For the specific value of preset threshold, those skilled in the art select any suitable value equal based on practical experience
Can, the embodiment of the present invention to this with no restriction.
For example, can choose preset threshold is 0.5, the friendship of wonderful A and high score segment a and than being 0.1, wonderful
The friendship of B and high score segment b and than being 0.8, the friendship of wonderful D and high score segment d and than being 0.7, thus may determine that excellent
Segment B and wonderful D is to predict accurate wonderful.
A2 calculates the prediction model to the Prediction Parameters of current video according to the number for predicting accurate wonderful.
In the embodiment of the present invention, prediction model may include accuracy rate acc to the Prediction Parameters of current video and/or recall
Rate recall.Wherein, "and/or" indicates to may include any one in two, can also two include.
Therefore, it according to the number for predicting accurate wonderful, calculates the prediction model and the prediction of current video is joined
Several steps may include:
A21 calculates the ratio of the number of the accurate wonderful of prediction and the total number of the high score segment, as
Accuracy rate.
And/or
A22 calculates the ratio of the number of the accurate wonderful of prediction and the total number of the wonderful, as
Recall rate.
For example, according to above-mentioned example, this 4 wonderfuls of shared A, B, C, a D, this 3 high fragments of shared a, b, a c
Section, wonderful B and wonderful D are to predict accurate wonderful, namely the number of the accurate wonderful of prediction is 2.
Therefore, prediction model is 2/3=66.7%, recall rate 2/4=50% to the accuracy rate that current video is predicted.
Step 105, the assessment for calculating the prediction model according to Prediction Parameters of the prediction model to each video refers to
Mark.
According to the process of 102~step 104 of above-mentioned steps, prediction model can be calculated, the prediction of each video is joined
Number calculates the Prediction Parameters of each video the evaluation index of the prediction model according to prediction model.Pass through prediction model
Evaluation index, it can be learnt that the prediction effect of the prediction model.
In the embodiment of the present invention, the evaluation index of prediction model may include at least one of: accuracy rate, recall rate,
F1 score.
Wherein, the accuracy rate acc^ of prediction model is the average value for the accuracy rate that prediction model predicts all videos.In advance
The recall rate recall^ for surveying model is the average value for the recall rate that prediction model predicts all videos.The F1 of prediction model points
Number F1-score=2*acc^*recall^/(acc^+recall^).
It no longer needs to mark personnel in the embodiment of the present invention and give a mark to the per second of video, but simply from video
Wonderful is extracted, is given a mark without to the wonderful, therefore considerably reduce the workload of mark personnel, is simplified
Evaluation process, improves assessment efficiency, and evaluation process is more objective, to keep assessment result more accurate.
It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method
It closes, but those skilled in the art should understand that, embodiment of that present invention are not limited by the describe sequence of actions, because according to
According to the embodiment of the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should
Know, the embodiments described in the specification are all preferred embodiments, and the related movement not necessarily present invention is implemented
Necessary to example.
Referring to Fig. 4, a kind of structural block diagram of the assessment device of prediction model of the embodiment of the present invention is shown.
The assessment device of the prediction model of the embodiment of the present invention comprises the following modules:
Module 401 is obtained, for obtaining multiple videos for assessment prediction model；
First extraction module 402 obtains at least one excellent extracted from current video for being directed to each video
Section；
Second extraction module 403, for being predicted current video to obtain current video pair using the prediction model
The prediction curve answered, and at least one high score segment is extracted from the prediction curve；
Determining module 404, for determining the prediction model to current according to the wonderful and the high score segment
The Prediction Parameters of video；
Computing module 405 calculates the prediction model for the Prediction Parameters according to the prediction model to each video
Evaluation index.
In a preferred embodiment, the determining module includes: segment judging submodule, for for each excellent
Segment, judges whether current wonderful is located in the high score segment；Segment determines submodule, if being used for segment judging unit
It is judged as YES, it is determined that current wonderful is to predict accurate wonderful；Parameter computation module, for quasi- according to prediction
The number of true wonderful calculates the prediction model to the Prediction Parameters of current video.
In a preferred embodiment, each wonderful is corresponding with beginning and ending time, each high-segment pairs Ying Youqi
The only time.The segment judging submodule includes: intersection judging unit, for according to corresponding beginning and ending time, judgement to be worked as
Whether there are intersections with the high score segment for preceding wonderful；Ratio calculation unit, if sentencing for the intersection judgment sub-unit
Break as there are intersections, then calculates current wonderful and there are the friendship of the high score segment of intersection and ratios；Compare determination unit, is used for
If described hand over and than being greater than preset threshold, it is determined that current wonderful is located in the high score segment.
In a preferred embodiment, the ratio calculation unit is specifically used for determining current wonderful and exist
The intersection time span of the high score segment of intersection, and determine current wonderful and there are when the union of the high score segment of intersection
Between length；As current wonderful and there is friendship in the ratio for calculating the intersection time span Yu the union time span
The friendship of the high score segment of collection and ratio.
In a preferred embodiment, the Prediction Parameters include accuracy rate and/or recall rate.The parameter computation
Module includes: the first computing unit, for calculating the total of the number of the accurate wonderful of the prediction and the high score segment
The ratio of number, as accuracy rate；And/or second computing unit, for calculating the number of the accurate wonderful of prediction
With the ratio of the total number of the wonderful, as recall rate.
In a preferred embodiment, the horizontal axis of the prediction curve indicates the time, and the longitudinal axis indicates excellent score value, described
Second extraction module includes: score value computational submodule, for calculating the average value of whole excellent score values in the prediction curve；Piece
Section extracting sub-module, for extracting segment of the excellent score value higher than the average value from the prediction curve as high fragment
Section.
It no longer needs to mark personnel in the embodiment of the present invention and give a mark to the per second of video, but simply from video
Wonderful is extracted, is given a mark without to the wonderful, therefore considerably reduce the workload of mark personnel, is simplified
Evaluation process, improves assessment efficiency, and evaluation process is more objective, to keep assessment result more accurate.
For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate
Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.
The assessment device of appraisal procedure to a kind of prediction model provided by the present invention and a kind of prediction model above, into
It has gone and has been discussed in detail, used herein a specific example illustrates the principle and implementation of the invention, the above implementation
The explanation of example is merely used to help understand method and its core concept of the invention；Meanwhile for the general technology people of this field
Member, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion this explanation
Book content should not be construed as limiting the invention.
Claims (8)
1. a kind of appraisal procedure of prediction model characterized by comprising
Obtain multiple videos for assessment prediction model；
For each video, at least one wonderful extracted from current video is obtained；
Predicted to obtain the corresponding prediction curve of current video to current video using the prediction model, and from the prediction
At least one high score segment is extracted in curve；
According to the wonderful and the high score segment, determine the prediction model to the Prediction Parameters of current video；
According to the prediction model to the Prediction Parameters of each video, the evaluation index of the prediction model is calculated.
2. the method according to claim 1, wherein described according to the wonderful and the high score segment,
The step of determining Prediction Parameters of the prediction model to current video, comprising:
For each wonderful, judge whether current wonderful is located in the high score segment；
If so, determining that current wonderful is to predict accurate wonderful；
According to the number for predicting accurate wonderful, the prediction model is calculated to the Prediction Parameters of current video.
3. according to the method described in claim 2, it is characterized in that, described judge whether current wonderful is located at the high score
Step in segment, comprising:
According to corresponding beginning and ending time, judge current wonderful whether with the high score segment there are intersections；
Intersection if it exists then calculates current wonderful and there are the friendship of the high score segment of intersection and ratios；
If described hand over and than being greater than preset threshold, it is determined that current wonderful is located in the high score segment.
4. according to the method described in claim 2, it is characterized in that, the Prediction Parameters include accuracy rate and/or recall rate, institute
The step of stating according to the number for predicting accurate wonderful, calculating Prediction Parameters of the prediction model to current video, packet
It includes:
The ratio for calculating the number of the accurate wonderful of prediction and the total number of the high score segment, as accuracy rate；
And/or
The ratio for calculating the number of the accurate wonderful of prediction and the total number of the wonderful, as recall rate.
5. a kind of assessment device of prediction model characterized by comprising
Module is obtained, for obtaining multiple videos for assessment prediction model；
First extraction module obtains at least one wonderful extracted from current video for being directed to each video；
Second extraction module obtains the corresponding prediction of current video for being predicted using the prediction model current video
Curve, and at least one high score segment is extracted from the prediction curve；
Determining module, for determining the prediction model to current video according to the wonderful and the high score segment
Prediction Parameters；
Computing module calculates the assessment of the prediction model for the Prediction Parameters according to the prediction model to each video
Index.
6. device according to claim 5, which is characterized in that the determining module includes:
Segment judging submodule judges whether current wonderful is located at the high score segment for being directed to each wonderful
In；
Segment determines submodule, if being judged as YES for segment judging unit, it is determined that current wonderful is that prediction is accurate
Wonderful；
Parameter computation module, for according to the number for predicting accurate wonderful, calculating the prediction model to working as forward sight
The Prediction Parameters of frequency.
7. device according to claim 6, which is characterized in that the segment judging submodule includes:
Intersection judging unit, for according to corresponding beginning and ending time, judge current wonderful whether with the high fragment
There are intersections for section；
Ratio calculation unit, if being judged as the intersection judgment sub-unit there are intersection, calculate current wonderful with
There are the friendship of the high score segment of intersection and ratios；
Compare determination unit, if for the friendship and than being greater than preset threshold, it is determined that current wonderful is located at the high score
In segment.
8. device according to claim 6, which is characterized in that the Prediction Parameters include accuracy rate and/or recall rate, institute
Stating parameter computation module includes:
First computing unit, for calculating the number and the total number of the high score segment of the accurate wonderful of prediction
Ratio, as accuracy rate；
And/or
Second computing unit, for calculating the number and the total number of the wonderful of the accurate wonderful of prediction
Ratio, as recall rate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811224095.XA CN109462751B (en) | 2018-10-19 | 2018-10-19 | Estimation method and device of prediction model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811224095.XA CN109462751B (en) | 2018-10-19 | 2018-10-19 | Estimation method and device of prediction model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109462751A true CN109462751A (en) | 2019-03-12 |
| CN109462751B CN109462751B (en) | 2020-07-21 |
Family
ID=65607953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811224095.XA Active CN109462751B (en) | 2018-10-19 | 2018-10-19 | Estimation method and device of prediction model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109462751B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109685144A (en) * | 2018-12-26 | 2019-04-26 | 上海众源网络有限公司 | The method, apparatus and electronic equipment that a kind of pair of Video Model does to assess |
| CN110505519A (en) * | 2019-08-14 | 2019-11-26 | 咪咕文化科技有限公司 | A kind of video clipping method, electronic equipment and storage medium |
| CN113497977A (en) * | 2020-03-18 | 2021-10-12 | 阿里巴巴集团控股有限公司 | Video processing method, model training method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346372A (en) * | 2013-07-31 | 2015-02-11 | 国际商业机器公司 | Method and device for evaluating prediction models |
| US20170109583A1 (en) * | 2011-01-20 | 2017-04-20 | Verint Americas Inc. | Evaluation of models generated from objects in video |
| CN108537139A (en) * | 2018-03-20 | 2018-09-14 | 校宝在线（杭州）科技股份有限公司 | A kind of Online Video wonderful analysis method based on barrage information |
-
2018
- 2018-10-19 CN CN201811224095.XA patent/CN109462751B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170109583A1 (en) * | 2011-01-20 | 2017-04-20 | Verint Americas Inc. | Evaluation of models generated from objects in video |
| CN104346372A (en) * | 2013-07-31 | 2015-02-11 | 国际商业机器公司 | Method and device for evaluating prediction models |
| CN108537139A (en) * | 2018-03-20 | 2018-09-14 | 校宝在线（杭州）科技股份有限公司 | A kind of Online Video wonderful analysis method based on barrage information |
Non-Patent Citations (2)
| Title |
|---|
| 王硕: "篮球视频精彩事件检测方法", 《中国优秀硕士学位论文全文数据库》 * |
| 谢文娟: "足球视频精彩进球事件检测", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109685144A (en) * | 2018-12-26 | 2019-04-26 | 上海众源网络有限公司 | The method, apparatus and electronic equipment that a kind of pair of Video Model does to assess |
| CN110505519A (en) * | 2019-08-14 | 2019-11-26 | 咪咕文化科技有限公司 | A kind of video clipping method, electronic equipment and storage medium |
| CN110505519B (en) * | 2019-08-14 | 2021-12-03 | 咪咕文化科技有限公司 | Video editing method, electronic equipment and storage medium |
| CN113497977A (en) * | 2020-03-18 | 2021-10-12 | 阿里巴巴集团控股有限公司 | Video processing method, model training method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109462751B (en) | 2020-07-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109462751A (en) | The appraisal procedure and device of prediction model | |
| CN110222880B (en) | Service risk determining method, model training method and data processing method | |
| CN109299327A (en) | Video recommendation method, device, equipment and storage medium | |
| CN104965773B (en) | Terminal, Caton detection method, device and game Caton detection method, device | |
| CN102222103B (en) | Method and device for processing matching relationship of video content | |
| EP1258816A2 (en) | Image search method and apparatus | |
| CN107704525A (en) | Video searching method and device | |
| CN104991962B (en) | A kind of method and device generating recommendation information | |
| CN109087135A (en) | The method for digging and device, computer equipment and readable medium that user is intended to | |
| CN104469508A (en) | Method, server and system for performing video positioning based on bullet screen information content | |
| KR102131791B1 (en) | Method to provide recommended contents and associated contents | |
| CN105792000A (en) | Video recommendation method and device | |
| CN103383702A (en) | Method and system for recommending personalized news based on ranking of votes of users | |
| CN105302879B (en) | For determining the method and apparatus of user demand | |
| CN112287175B (en) | Video highlight segment prediction method and system | |
| KR20170041653A (en) | Prediction method of disk capacity, equipment, facilities and non-volatile computer storage media | |
| CN103500163B (en) | The method and apparatus of identification event key development | |
| CN106599047A (en) | Information pushing method and device | |
| CN102737383B (en) | Camera movement analyzing method and device in video | |
| CN112100221A (en) | Information recommendation method and device, recommendation server and storage medium | |
| EP2908283A1 (en) | Forensic system, forensic method, and forensic program | |
| CN104077361B (en) | A kind of sort method and system for big data | |
| CN105912549A (en) | Content recommendation method and device thereof | |
| CN110737497B (en) | Display control interface aided decision design optimization method | |
| CN113761271A (en) | Method and device for video recommendation and refrigerator with display screen |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |