WO2017156994A1 - 多媒体资源的质量评估方法和装置 - Google Patents

多媒体资源的质量评估方法和装置 Download PDF

Info

Publication number
WO2017156994A1
WO2017156994A1 PCT/CN2016/099358 CN2016099358W WO2017156994A1 WO 2017156994 A1 WO2017156994 A1 WO 2017156994A1 CN 2016099358 W CN2016099358 W CN 2016099358W WO 2017156994 A1 WO2017156994 A1 WO 2017156994A1
Authority
WO
WIPO (PCT)
Prior art keywords
multimedia resource
vector
quality
user behavior
multimedia
Prior art date
Application number
PCT/CN2016/099358
Other languages
English (en)
French (fr)
Inventor
魏博
齐志兵
王远图
马广续
刘宇平
尹玉宗
姚键
潘柏宇
王冀
Original Assignee
合一网络技术(北京)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 合一网络技术(北京)有限公司 filed Critical 合一网络技术(北京)有限公司
Priority to US15/758,836 priority Critical patent/US10762122B2/en
Priority to EP16894153.2A priority patent/EP3346396A4/en
Publication of WO2017156994A1 publication Critical patent/WO2017156994A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/438—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/435—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/45—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/483—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation

Definitions

  • the present invention relates to the field of Internet technologies, and in particular, to a method and an apparatus for evaluating quality of multimedia resources.
  • the quality calculation and quality assessment of the existing multimedia resources are mainly based on the original attributes (machine attributes) of the multimedia resources themselves.
  • the overall quality of multimedia resources is judged by static properties such as signal quality and distortion of audio and video.
  • some user-oriented attributes such as frame rate, sharpness, etc. are added to the quality assessment method to further determine the overall quality of the multimedia resource.
  • the prior art quality assessment method is to use the native attributes (dynamic and static) of multimedia resources as the main indicators of quality assessment.
  • the quality of multimedia resources is evaluated based on only the native attributes of multimedia resources, in many cases, the user's needs cannot be met. Therefore, the quality of multimedia resources cannot be characterized only by native attributes such as network transmission and resolution.
  • the technical problem to be solved by the present invention is to provide a method and apparatus for quality assessment of multimedia resources to more accurately characterize the quality of multimedia resources.
  • the present invention provides a method for evaluating quality of a multimedia resource, including:
  • the determining, according to the indicator data used to describe user behavior of the multimedia resource, determining a cumulative distribution vector of the multimedia resource includes:
  • a vector formed by the number corresponding to each group is determined as a cumulative distribution vector of the user behavior of the multimedia resource.
  • the dividing the indicator data of a type of user behavior into multiple groups including:
  • n is the number of groups
  • the interval max(D)-min(D) is divided into n groups.
  • the according to the cumulative distribution Determining the quality score of the multimedia resource, the upper boundary vector and the lower boundary vector, including: calculating the quality score by using Equation 1 below,
  • Score represents the quality score
  • DistanceTOTOP represents the distance of the cumulative distribution vector to the upper boundary vector
  • DistanceBetween represents the distance of the upper boundary vector to the lower boundary vector
  • the present invention provides a quality assessment apparatus for a multimedia resource, including:
  • a first determining unit configured to determine, according to indicator data used to describe user behavior of the multimedia resource, a cumulative distribution vector of the multimedia resource
  • a second determining unit connected to the first determining unit, configured to determine an upper boundary vector and a lower boundary vector of the multimedia resource according to the cumulative distribution vector;
  • a third determining unit configured to be connected to the first determining unit and the second determining unit, configured to determine a quality score of the multimedia resource according to the cumulative distribution vector, the upper boundary vector, and the lower boundary vector .
  • the first determining unit includes:
  • a statistical subunit connected to the divided subunit, for counting the number of indicator data of the user behavior included in each group;
  • the dividing subunit includes:
  • An obtaining module configured to obtain a maximum value max(D) and a minimum value min(D) of the indicator data D of the user behavior of the class;
  • a dividing module is connected to the determining module for dividing the interval max(D)-min(D) into n groups.
  • the third determining unit is specifically configured to utilize Equation 1 calculates the quality score
  • Score represents the quality score
  • DistanceTOTOP represents the distance of the cumulative distribution vector to the upper boundary vector
  • DistanceBetween represents the distance of the upper boundary vector to the lower boundary vector
  • the method and device for evaluating quality of a multimedia resource according to an embodiment of the present invention can more accurately describe the quality of a multimedia resource and has strong operability.
  • the multimedia is evaluated by using a quality assessment method for multimedia resources according to an embodiment of the present invention. After the quality of the resources, the feedback of the online results can be absorbed to dynamically sort and recommend the multimedia resources, and the final sorting and recommendation results of the multimedia resources can be given.
  • FIG. 1 is a flowchart showing a method for evaluating quality of a multimedia resource according to Embodiment 1 of the present invention
  • FIG. 2 is a flowchart showing a method for evaluating quality of a multimedia resource according to Embodiment 2 of the present invention
  • FIG. 3a is a flowchart showing a method for evaluating quality of a multimedia resource according to Embodiment 3 of the present invention
  • Figure 3b shows a play-through ratio frequency distribution histogram
  • Figure 3c shows a play complete ratio frequency distribution histogram
  • Figure 3d shows a histogram of the quality score distribution of the video
  • FIG. 4 is a structural block diagram of a quality assessment apparatus for a multimedia resource according to Embodiment 4 of the present invention.
  • FIG. 5 is a block diagram showing the structure of a quality assessment apparatus for a multimedia resource according to Embodiment 5 of the present invention.
  • FIG. 6 is a block diagram showing the structure of a quality assessment apparatus for a multimedia resource according to Embodiment 6 of the present invention.
  • FIG. 7 is a block diagram showing the structure of a quality assessment apparatus for a multimedia resource according to another embodiment of the present invention.
  • FIG. 1 is a flowchart showing a method for evaluating quality of a multimedia resource according to Embodiment 1 of the present invention.
  • the quality assessment method may mainly include:
  • Step S100 Determine a cumulative distribution vector of the multimedia resource according to the indicator data used to describe the user behavior of the multimedia resource.
  • the terminal device may be, for example, a mobile phone, a mobile Internet device (English: Mobile Internet Device, MID for short), a personal digital assistant (English: Personal Digital Assistant, PDA for short), a notebook, a desktop computer, a smart TV, or the like.
  • the multimedia resource can be, for example, video, audio, pictures, and the like.
  • multimedia resources of the present invention are not limited to the above three examples, and those skilled in the art should be able to understand that the focus of the present invention is not on multimedia resources, and any other form of multimedia resources may also be applicable to the present invention. That is, the present invention does not limit the specific form of multimedia resources.
  • the metric data can be used to characterize user behavior of multimedia resources such as video, audio, etc., and the user behavior of the multimedia resource can include multiple categories, such as topping, commenting, recommending (forwarding), collecting, playing, downloading, and the like.
  • the top step refers to the operation of the user to make a "top” or “step” on the played multimedia resource based on its own support or opposition to the played multimedia resource.
  • the topping usually includes the identification (vid) of the multimedia resource being played, the topping operation, the operator (user) related information, the operation time and IP (for example, the user's mobile phone or computer, etc.).
  • a comment is a description of a comment made at a corresponding location based on its own understanding of the content and form of the multimedia resource being played.
  • the comments usually include the identifier of the multimedia resource being played (vid), the specific content of the comment, the relevant information of the operator (user), the operation time, and the IP.
  • the collection refers to the recording operation performed by the user based on his own understanding of the content and form of the multimedia resource being played, so that the multimedia resource can be retrieved more conveniently in the future.
  • the collection usually includes the identification of the multimedia resource being played (vid), the operator (user) related information, the operation time and IP and so on.
  • Recommendation refers to an off-site push operation performed by the user based on his own understanding of the content and form of the multimedia resource being played.
  • Recommendations typically include the identity of the multimedia resource being played (vid), operator (user) related information, operating time and IP, recommendation platform, and the like.
  • Playback refers to the user's viewing behavior for multimedia resources.
  • the playback usually includes an identification (vid) of the multimedia resource being played, related information of the operator (user), operation time and IP, length of play time, and the like.
  • Downloading refers to a user's download to local operation based on his own understanding of the content and form of the multimedia resource being played.
  • the download usually includes the identifier (vid) of the multimedia resource being played, related information of the operator (user), operation time and IP, download progress, and the like.
  • the construction process of user behavior is a mapping process from problem domain to behavior domain: f:P ob oblemDo m ain ⁇ UserBehavior, where Pr oblemDo m ain represents the problem domain and UserBehavior represents the user behavior set.
  • Each business unit can select the best user behavior for assessment based on its own background data and page functions. From the actual effect, it is recommended to use user behavior that can truly reflect the user's intentions, so that the quality assessment (calculation) of multimedia resources is more accurate.
  • the indicator data can be used to measure each user behavior in each type of user behavior, and the detailed description of the indicator data of each type of user behavior is as follows.
  • the user's behavior of the top-level operation of the multimedia resource can be used as the indicator data to measure the user behavior of the top-like class. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource that occurs every time. In theory, it is hoped that the user does not step on the behavior and the top behavior occurs as early as possible (it is unreasonable to happen too early).
  • the user behavior of the comment class can be measured using the point at which the user performs a comment operation on the multimedia resource and the comment emotion as the indicator data. If the multimedia resource is played to complete the progress The calculation can record the progress of the completion of the playback of the multimedia resource each time the comment occurs. At the same time, the positive and negative emotions of the user comments can be quantified as much as possible. In theory, it is hoped that the user does not have a negative emotional comment and the comment behavior occurs as early as possible (it is also unreasonable to have a comment behavior too early).
  • the user's behavior of the collection class can be measured by using the occurrence point of the user's collection operation performed by the multimedia resource as the indicator data. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource each time the collection behavior occurs. In theory, it is hoped that users will have a collection behavior and collect behavior as soon as possible (it is unreasonable to have a collection behavior too early).
  • the user behavior of the recommended class can be measured by using the occurrence point of the recommended operation performed by the user on the multimedia resource and the ratio of the returned traffic of the recommended multimedia resource as the indicator data.
  • the lead-out flow ratio the number of times of the lead-out/the number of times of exposure refers to the number of times the recommended multimedia resource is opened twice, and the number of times of exposure refers to the recommended number of times of the recommended multimedia resource. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the recommended multimedia resource each time. At the same time, the ratio of the return flow can be calculated by crawling the relevant data of the external station. In theory, it is desirable that the user has a recommended behavior and the recommendation behavior occurs as early as possible (it is unreasonable to have the recommendation behavior too early) and the higher the ratio of the return traffic is, the better.
  • the playback behavior of the playback class can be measured by using the playback completion of the multimedia resource and the number of times the user drags the progress bar (fast reverse, fast forward) as the indicator data. It is hoped that the user's playback completion ratio is as high as possible and there is no fast forward drag but there are multiple reasonable rewind drags.
  • the user behavior of the download class can be measured using the occurrence point of the download operation performed by the user on the multimedia resource and the download completion progress as the indicator data. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource each time the download behavior occurs.
  • the download completion progress measures the determination and network status of users to download multimedia resources. In theory, it is hoped that the user will have the download behavior and the download behavior will occur as soon as possible (it is unreasonable for the download behavior to occur too early) and that it is 100% complete download.
  • the embodiments of the present invention only exemplify several types of user behaviors and their indicator data, and those skilled in the art should be able to understand that the types of user behaviors of the present invention may also be other categories, and it is not necessary to extract in actual operations.
  • the above various indicator data but can extract an appropriate amount of indicator data according to the needs of the business and whether it imposes an excessive burden on the system.
  • the construction process of the indicator data is a mapping process from user behavior to indicator data: f: UserBehavior ⁇ Indicators, where UserBehavior represents the user behavior set, and Indicators represents the indicator data set.
  • the construction process of the cumulative distribution vector is a mapping process from the index space to the vector space: f:Indicators ⁇ V n , where Indicators represent the indicator data set and V n represents the n-dimensional vector space.
  • Step S120 Determine an upper boundary vector and a lower boundary vector of the multimedia resource according to the cumulative distribution vector.
  • the optimal performance and the worst performance of the indicator data may be defined.
  • the maximum number of occurrences of a reasonable top-on operation performed by the user on the multimedia resource the maximum count of the highest ratio of the recommended return flow rate of the recommended multimedia resource, and the maximum number of users who have completely viewed the multimedia resource.
  • the upper boundary and the lower boundary of the multimedia resource on the indicator data may all be represented by a vector, that is, an upper boundary vector and a lower boundary vector.
  • Step S140 Determine a quality score of the multimedia resource according to the cumulative distribution vector, the upper boundary vector, and the lower boundary vector.
  • the quality score of the multimedia resource can be determined based on the cumulative distribution vector.
  • the farther a cumulative distribution vector is from the lower boundary vector and the closer to the upper boundary vector the better the performance of the user behavior, and the higher the quality of the multimedia resource.
  • the distance fraction can be used to define the quality score of a multimedia resource.
  • determining the quality score of the multimedia resource according to the cumulative distribution vector, the upper boundary vector, and the lower boundary vector may include: calculating the quality score by using Equation 1 below,
  • Score represents the mass score
  • Dis tan ceTOTOP represents the distance from the cumulative distribution vector to the upper boundary vector
  • Dis tan ceBetween represents the distance from the upper boundary vector to the lower boundary vector.
  • the distance between vectors can be calculated using methods such as cosine similarity or multi-dimensional Euclidean distance, and the cosine similarity and Euclidean distance can ensure that the range of the quality score Score is [0, 1].
  • the cosine similarity is that the vector is drawn into the vector space according to the coordinate value, the angle between the two vectors is obtained, and the cosine value corresponding to the angle is calculated, and the cosine value can be used to characterize the similarity between the two vectors. .
  • the smaller the angle the closer the cosine value is to 1, and the directions of the two vectors are more consistent.
  • the Euclidean distance is a commonly used distance definition and is the true distance between two points in an m-dimensional space.
  • Cosine similarity and Euclidean distance have a wide range of applications in calculating the distance between vectors, both of which are easy to understand and easy to operate. Cosine similarity is a good way to output normalized results, while Euclidean distance is the method of outputting global values. In actual operation, one of the methods can be arbitrarily selected according to actual needs.
  • the construction of the quality score of the multimedia resource is a mapping process from the cumulative distribution vector to the interval [0, 1]: f: V n ⁇ [0, 1], where V n represents the n-dimensional vector space, [0 , 1] indicates the range of values of the quality score Score.
  • the method for evaluating the quality of multimedia resources according to the embodiment of the present invention is based on user experience and loyal to the user, that is, the quality of the multimedia resource is characterized by the user behavior of the multimedia resource, which enables the present invention to more accurately describe the quality of the multimedia resource.
  • the quality assessment method of the embodiment of the present invention has strong operability, because for the Internet application, a large amount of multimedia resources are publicly available on the line, and the user can pass the daily point. Hit and watch behavior to consume these multimedia resources, and the enterprise backend can use the log system to record these user behaviors. Therefore, the service process of the system is the data preparation process for the quality assessment of multimedia resources, so the user behavior of acquiring multimedia resources is simple. OK. In contrast, the traditional quality assessment methods based on the original attributes of multimedia resources require specialized staff and systems to complete the collection and measurement of relevant indicators.
  • the quality evaluation method of the embodiment of the present invention is used to evaluate the quality of the multimedia resource, the feedback of the online result can be absorbed to perform dynamic optimization ranking and recommendation of the multimedia resource, and the final multimedia resource ranking and recommendation result can be given. . If the user's behavior on the sorting and recommendation results of the multimedia resources is not ideal, in the future iteration, the ranking of the multimedia resources and the quality score of the multimedia resources in the recommendation result are reduced, thereby sorting and recommending the original multimedia resources. The top multimedia resources in the results are automatically listed behind.
  • the quality assessment of the multimedia resource can be performed by using only the indicator data of one type of user behavior for quality assessment, or by using statistical methods such as indicator data of multiple types of user behavior. Quality assessment.
  • the quality scores of each type of user behavior may be separately calculated, and then the quality scores of user behaviors of all categories are averaged to determine the quality score of the multimedia resources.
  • FIG. 2 is a flowchart showing a method for evaluating quality of a multimedia resource according to Embodiment 2 of the present invention.
  • the quality assessment method may mainly include:
  • Step S201 dividing indicator data of a type of user behavior into a plurality of groups.
  • a non-overlapping grouping method can be used to divide the metric data of a type of user behavior into multiple
  • an overlapping grouping method may be used to divide the index data of a type of user behavior into a plurality of groups.
  • the indicator data of a type of user behavior is divided into multiple groups, including:
  • n is the number of groups
  • the interval max(D)-min(D) is divided into n groups.
  • the process of grouping is as follows: Suppose a given set of data on a real number field D, you can first obtain the maximum value max(D) and the minimum value min(D) of the data D; then divide the interval max(D)-min(D) (also called the range or full distance) into n Grouping, the corresponding segmentation interval is (also called group spacing), then n packets correspond to n grouping intervals, for example: with Two grouping intervals for the head and tail.
  • an overlapping grouping method for observing the overall change of data is used to divide the index data of a type of user behavior into a plurality of groups, the grouping process is as follows: assuming that the data D on a set of real numbers is given, it can be obtained first. The maximum value max(D) and the minimum value min(D) of the data D, the interval [min(D), max(D)] can contain the entire data D; then the interval max(D)-min(D) is divided equally. For n overlapping grouping intervals, for example: [min(D), max(D)] and The maximum and minimum two grouping intervals.
  • Step S203 Count the number of indicator data of the user behavior included in each group.
  • the number of the indicator data falling in each interval can be separately counted.
  • Step S205 Determine a vector formed by the number corresponding to each group as a cumulative distribution vector of the user behavior of the multimedia resource.
  • the cumulative distribution vector of the user's behavior can be quickly determined directly from the frequency distribution histogram.
  • Step S207 Determine an upper boundary vector and a lower boundary vector of the multimedia resource according to the cumulative distribution vector.
  • Step S209 Determine a quality score of the multimedia resource according to the cumulative distribution vector, the upper boundary vector, and the lower boundary vector.
  • step S207 and step S209 refer to the related descriptions in step S120 and step S140 in the above embodiment 1.
  • the method for evaluating the quality of multimedia resources according to the embodiment of the present invention is based on user experience and loyal to the user, that is, the quality of the multimedia resource is characterized by the user behavior of the multimedia resource, which enables the present invention to more accurately describe the quality of the multimedia resource.
  • the quality assessment method of the embodiment of the present invention has strong operability because, for Internet applications, a large amount of multimedia resources are publicly available online, and users can consume these multimedia resources through daily click and viewing behaviors.
  • the enterprise background can use the log system to record these user behaviors. Therefore, the service process of the system is the data preparation process for the quality assessment of multimedia resources, so the user behavior of obtaining multimedia resources is simple and easy.
  • the traditional quality assessment methods based on the original attributes of multimedia resources require specialized staff and systems to complete the collection and measurement of relevant indicators.
  • the feedback of the online result can be absorbed to perform dynamic optimization ranking and recommendation of the multimedia resource, and the final ranking and recommendation of the multimedia resource can be given. result. If the user's behavior on the sorting and recommendation of multimedia resources is not ideal, then in the future iteration, the row of these multimedia resources The quality scores of the multimedia resources in the sequence and recommendation results are reduced, so that the multimedia resources in the ranking and recommendation results of the original multimedia resources are automatically ranked behind.
  • FIG. 3a is a flowchart showing a method for evaluating quality of a multimedia resource according to Embodiment 3 of the present invention.
  • the quality evaluation method of the multimedia resource of the present invention is exemplified by the indicator data of the playback user behavior, that is, the playback completion ratio of the multimedia resource (for example, the video viewing completion ratio).
  • Step 301 Use a video play log of a video website as a basic data source.
  • the original video play log is a data table containing at least the following four-tuple: the four-tuple is ⁇ Vids, PlayLength, FullLength, Time ⁇ , where Vids represents the video collection being viewed; PlayLength represents the accumulation of each video view. The length of time, usually in seconds; FullLength represents the total length of time of the video being viewed; Time represents the timestamp of the occurrence of this viewing behavior.
  • Each line of the original video playback log stores the viewing behavior of the user's click-through video at that timestamp.
  • User viewing behavior data for one day, one hour, or even any time can be obtained by defining different timestamps.
  • Table 1 is an example fragment of a video viewing log data.
  • the above four-tuple ⁇ Vids, PlayLength, FullLength, Time ⁇ can be pre-processed by summarizing the video play log information of the user viewing time length.
  • the video playing data of a specific time period may be selected by defining a Time field.
  • video playing data with a Time field of “20160105” may be selected from the video playing log information.
  • Step 302 Acquire indicator data of the video according to the video play log, that is, a play completion ratio.
  • the completion of the video playback than the perc refers to the ratio of the length of the video playback time to the total length of the video, ie
  • the video view log data of Table 1 By pre-processing the video view log data of Table 1 above, information including the viewed video set Vids, the playback completion ratio perc of the viewed video, and the time stamp Time at which the viewing behavior occurs may be generated to record the user's presence.
  • the viewing on the video is complete.
  • the playback completion ratio perc of the viewed video shown in Table 2 below can be obtained by using the video viewing log data in Table 1 above. It should be noted that the time stamp Time at which the viewing behavior occurred is omitted in Table 2 for the attention of the problem itself.
  • Step 303 The indicator data playing completion of the user behavior of the playing class is divided into multiple groups and the cumulative distribution vector of the indicator data playing completion ratio perc is determined.
  • the non-overlapping grouping method described in Embodiment 2 above may be employed to complete the playback.
  • the splitting grouping method described in the above embodiment 2 can also be used to divide the playback completion into multiple groups than the perc divided into a plurality of groups.
  • the frequency distribution histogram can be used to display the frequency distribution of the playback completion ratio perc, wherein the frequency distribution histogram is the ratio of the frequency of the corresponding group to the group distance by the height of the rectangle (since the group distance is a constant, therefore In order to facilitate drawing and viewing, the height of the rectangle is directly used to represent the frequency, and the frequency distribution histogram clearly shows the distribution of the frequency of each group and it is easy to display the difference in frequency between the groups.
  • the horizontal axis of the frequency distribution histogram is the global interval where the playback completion is 0%-100% of perc.
  • the group distance of the group can be determined according to the actual situation. Among them, according to the statistical knowledge, the appropriate group spacing can reflect the distribution characteristics of the sampled data. If the group distance is too small, the number of groups of the group will be too large. If the group distance is too large, the group number of the group will be too small, which will cover up the playback. Complete the distribution characteristics than perc. In addition, the number of groups of packets caused by too small a group distance is too large, so that the subsequent completion of the playback according to the determined cumulative distribution vector, that is, the completion ratio of the cumulative distribution vector is too high, which causes a computational burden on the analysis of massive data. . Therefore, in the present embodiment, the group distance is determined to be 10%.
  • the global interval of 0%-100% of the completion of the play can be divided into 10 groups according to the group distance of 10%, so that 10 equidistant non-overlapping sampling intervals (grouping) of 0%-10%, 10%-20%, ..., 90%-100%, whereby the play completion ratio frequency distribution histogram shown in Fig. 3b can be obtained.
  • the global interval of 0%-100% of the completion of the play can be divided into 10 groups according to the group distance of 10%, so that a form such as 0 can be formed.
  • the overlap grouping method i.e., cumulative distribution count
  • the playback uses the overlap grouping method (i.e., cumulative distribution count) to divide the play completion ratio into multiple groups and determine the play completion ratio cumulative distribution vector.
  • the vector consisting of the counts corresponding to the group 0%, (0%, 10%], ..., (0%, 100%) ( f 0 , f 1 , ..., f 100/m ) is the playback completion ratio of the video than the cumulative distribution vector V vid .
  • the critical point of each group is mapped to the progress point of the actual video playing, and the user's playing completion ratio can describe the progress of the video playing.
  • Single-point grouping 0% can be understood as the number of times the video is clicked, that is, the video is counted as soon as it is clicked, and the number of times the video is recorded in the video log data can be used.
  • the number of times the playback completes m% must not be more than the number of times the playback completes c%, and the frequency of the interval (0%, m%) must not exceed the interval (0%, c
  • the frequency of % for example, the number of times the video is viewed 100% must not be more than 20% of the video. Therefore, the internal data of the playback completion than the cumulative distribution vector V vid is a non-incrementing sequence.
  • a play completion ratio frequency distribution histogram as shown in FIG. 3c can be obtained.
  • the playback completion ratio cumulative frequency vector V vid (14,13,11,10,9,8,7,6,4,2, corresponding to the frequency distribution histogram is completed. 1).
  • Step 304 Determine an upper boundary vector and a lower boundary vector of the playback completion ratio according to the playback completion ratio cumulative distribution vector Vvid .
  • V t (14, 14, 14, 14, 14, 14, 14, 14, 14)
  • V b (14, 0, 0, 0, 0, 0, 0, 0, 0, 0).
  • Step 305 After determining the upper boundary vector V t and the lower boundary vector V b of the completion ratio of the playback completion ratio than the cumulative distribution vector V vid , the playback completion ratio, the upper boundary vector of the playback completion ratio cumulative distribution vector V vid to the playback completion ratio may be calculated. The distance of V t and the distance from the upper boundary vector V t of the playback completion ratio to the lower boundary vector V b of the playback completion ratio.
  • the 11-dimensional Euclidean distance is used to calculate the above two distances separately.
  • the specific definition of 11-dimensional Euclidean distance is as follows:
  • the distance between vector X and vector Y is Where j ⁇ [1, 11], x j is the value of the vector X at the jth position, and y j is the value of the vector Y at the jth position.
  • the distance d(V t , V b ) between 0,0) is 44.272.
  • Step 306 after calculating the playback is finished vid to play completion on the distance boundary vectors V t ratio and the playback is completed on the ratio of boundary vectors V t to the player completed from the boundary vectors V b ratio than the cumulative distribution vector V, Calculate the quality score of the video using Equation 2 below:
  • a mass score of a video website can be calculated, thereby obtaining a video quality score statistics table of the video website shown in Table 3 below.
  • the third quartile has reached the maximum value, which means that at least 25% of the video quality score is 1, which is caused by the long tail effect of video playback. That is, there are a large number of videos that have only one or two play behaviors and both have a full play.
  • a mass fraction distribution histogram of the video shown in Figure 3d can be obtained.
  • the user behavior of the new category may be introduced for multiple calculations, or new indicator data of the video may be introduced for calculation, thereby preparing for subsequent video search and video recommendation.
  • the method for evaluating the quality of multimedia resources according to the embodiment of the present invention is based on user experience and loyal to the user, that is, the quality of the multimedia resource is characterized by the user behavior of the multimedia resource, which enables the present invention to more accurately describe the quality of the multimedia resource.
  • the quality assessment method of the embodiment of the present invention has strong operability because, for Internet applications, a large amount of multimedia resources are publicly available online, and users can consume these multimedia resources through daily click and viewing behaviors.
  • the enterprise backend can use the logging system to record this.
  • the service process of the system is the data preparation process for the quality assessment of multimedia resources, so the user behavior of acquiring multimedia resources is simple and easy.
  • the traditional quality assessment methods based on the original attributes of multimedia resources require specialized staff and systems to complete the collection and measurement of relevant indicators.
  • the feedback of the online result can be absorbed to perform dynamic optimization ranking and recommendation of the multimedia resource, and the final ranking and recommendation of the multimedia resource can be given. result. If the user's behavior on the sorting and recommendation results of the multimedia resources is not ideal, in the future iteration, the ranking of the multimedia resources and the quality score of the multimedia resources in the recommendation result are reduced, thereby sorting and recommending the original multimedia resources. The top multimedia resources in the results are automatically listed behind.
  • FIG. 4 is a block diagram showing the structure of a quality assessment apparatus for a multimedia resource according to Embodiment 4 of the present invention.
  • the quality evaluation device 400 provided in this embodiment is used to implement the quality evaluation method shown in FIG. 1.
  • the quality evaluation apparatus 400 can mainly include:
  • the first determining unit 410 is configured to determine a cumulative distribution vector of the multimedia resource according to the indicator data used to describe the user behavior of the multimedia resource.
  • the terminal device may be, for example, a mobile phone, a mobile Internet device (English: Mobile Internet Device, MID for short), a personal digital assistant (English: Personal Digital Assistant, PDA for short), a notebook, a desktop computer, a smart TV, or the like.
  • the multimedia resource can be, for example, video, audio, pictures, and the like.
  • multimedia resources of the present invention are not limited to the above three examples, and those skilled in the art should be able to understand that the focus of the present invention is not on multimedia resources, and any other form of multimedia resources may also be applicable to the present invention. That is, the present invention does not limit the specific form of multimedia resources.
  • Indicator data can be used to characterize user behavior of multimedia resources such as video, audio, and the like.
  • the user behavior of the multimedia resource may include multiple categories, such as topping, commenting, recommending (forwarding), collecting, playing, downloading, and the like.
  • the top step refers to the operation of the user to make a "top” or “step” on the played multimedia resource based on its own support or opposition to the played multimedia resource.
  • the topping usually includes the identification (vid) of the multimedia resource being played, the topping operation, the operator (user) related information, the operation time and IP (for example, the user's mobile phone or computer, etc.).
  • a comment is a description of a comment made at a corresponding location based on its own understanding of the content and form of the multimedia resource being played.
  • the comments usually include the identifier of the multimedia resource being played (vid), the specific content of the comment, the relevant information of the operator (user), the operation time, and the IP.
  • the collection refers to the recording operation performed by the user based on his own understanding of the content and form of the multimedia resource being played, so that the multimedia resource can be retrieved more conveniently in the future.
  • the collection usually includes the identification (vid) of the multimedia resource being played, related information of the operator (user), operation time, IP, and the like.
  • Recommendation refers to an off-site push operation performed by the user based on his own understanding of the content and form of the multimedia resource being played.
  • Recommendations typically include the identity of the multimedia resource being played (vid), operator (user) related information, operating time and IP, recommendation platform, and the like.
  • Playback refers to the user's viewing behavior for multimedia resources.
  • the playback usually includes an identification (vid) of the multimedia resource being played, related information of the operator (user), operation time and IP, length of play time, and the like.
  • Downloading refers to a user's download to local operation based on his own understanding of the content and form of the multimedia resource being played.
  • the download usually includes the identifier (vid) of the multimedia resource being played, related information of the operator (user), operation time and IP, download progress, and the like.
  • the construction process of user behavior is a mapping process from problem domain to behavior domain: f:P ob oblemDo m ain ⁇ UserBehavior, where Pr oblemDo m ain represents the problem domain and UserBehavior represents the user behavior set.
  • Each business unit can select the best user behavior for assessment based on its own background data and page functions. From the actual effect, it is recommended to use user behavior that can truly reflect the user's intentions, so that the quality assessment (calculation) of multimedia resources is more accurate.
  • the indicator data can be used to measure each user behavior in each type of user behavior, and the detailed description of the indicator data of each type of user behavior is as follows.
  • the user's behavior of the top-level operation of the multimedia resource can be used as the indicator data to measure the user behavior of the top-like class. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource that occurs every time. In theory, it is hoped that the user does not step on the behavior and the top behavior occurs as early as possible (it is unreasonable to happen too early).
  • the user behavior of the comment class can be measured using the point at which the user performs a comment operation on the multimedia resource and the comment emotion as the indicator data. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource each time the comment occurs. At the same time, the positive and negative emotions of the user comments can be quantified as much as possible. In theory, it is hoped that the user does not have a negative emotional comment and the comment behavior occurs as early as possible (it is also unreasonable to have a comment behavior too early).
  • the user's behavior of the collection class can be measured by using the occurrence point of the user's collection operation performed by the multimedia resource as the indicator data. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource each time the collection behavior occurs. In theory, it is hoped that users will have a collection behavior and collect behavior as soon as possible (it is unreasonable to have a collection behavior too early).
  • the user behavior of the recommended class can be measured by using the occurrence point of the recommended operation performed by the user on the multimedia resource and the ratio of the returned traffic of the recommended multimedia resource as the indicator data.
  • the lead-out flow ratio the number of times of the lead-out/the number of times of exposure refers to the number of times the recommended multimedia resource is opened twice, and the number of times of exposure refers to the recommended number of times of the recommended multimedia resource. If the multimedia resource playback completion progress is calculated, it is possible to record the broadcast of the recommended multimedia resource each time. Put the progress on. At the same time, the ratio of the return flow can be calculated by crawling the relevant data of the external station. In theory, it is desirable that the user has a recommended behavior and the recommendation behavior occurs as early as possible (it is unreasonable to have the recommendation behavior too early) and the higher the ratio of the return traffic is, the better.
  • the playback behavior of the playback class can be measured by using the playback completion of the multimedia resource and the number of times the user drags the progress bar (fast reverse, fast forward) as the indicator data. It is hoped that the user's playback completion ratio is as high as possible and there is no fast forward drag but there are multiple reasonable rewind drags.
  • the user behavior of the download class can be measured using the occurrence point of the download operation performed by the user on the multimedia resource and the download completion progress as the indicator data. If the multimedia resource playback completion progress is calculated, it is possible to record the progress of the completion of the playback of the multimedia resource each time the download behavior occurs.
  • the download completion progress measures the determination and network status of users to download multimedia resources. In theory, it is hoped that the user will have the download behavior and the download behavior will occur as soon as possible (it is unreasonable for the download behavior to occur too early) and that it is 100% complete download.
  • the construction process of the indicator data is a mapping process from user behavior to indicator data: f: UserBehavior ⁇ Indicators, where UserBehavior represents the user behavior set, and Indicators represents the indicator data set.
  • the construction process of the cumulative distribution vector is a mapping process from the index space to the vector space: f:Indicators ⁇ V n , where Indicators represent the indicator data set and V n represents the n-dimensional vector space.
  • the second determining unit 430 is connected to the first determining unit 410, and is configured to determine an upper boundary vector and a lower boundary vector of the multimedia resource according to the cumulative distribution vector.
  • the second determining unit 430 may define an optimal performance and a worst performing performance of the indicator data, The upper and lower boundaries. For example, the maximum number of occurrences of a reasonable top-on operation performed by the user on the multimedia resource, the maximum count of the highest ratio of the recommended return flow rate of the recommended multimedia resource, and the maximum number of users who have completely viewed the multimedia resource.
  • the upper and lower boundaries of the multimedia resource on the indicator data are represented by vectors, that is, an upper boundary vector and a lower boundary vector.
  • the third determining unit 450 is connected to the first determining unit 410 and the second determining unit 430, and is configured to determine a quality score of the multimedia resource according to the cumulative distribution vector, the upper boundary vector, and the lower boundary vector.
  • the third determining unit 450 may determine the quality score of the multimedia resource according to the cumulative distribution vector determined by the first determining unit 410 and the upper boundary vector and the lower boundary vector determined by the second determining unit 430.
  • the farther a cumulative distribution vector is from the lower boundary vector and the closer to the upper boundary vector the better the performance of the user behavior, and the higher the quality of the multimedia resource.
  • the distance fraction can be used to define the quality score of a multimedia resource.
  • the third determining unit 450 is specifically configured to calculate a quality score by using Equation 1 below.
  • Score represents the mass score
  • Dis tan ceTOTOP represents the distance from the cumulative distribution vector to the upper boundary vector
  • Dis tan ceBetween represents the distance from the upper boundary vector to the lower boundary vector.
  • the distance between vectors can be calculated using methods such as cosine similarity or multi-dimensional Euclidean distance, and the cosine similarity and Euclidean distance can ensure that the range of the quality score Score is [0, 1].
  • cosine similarity is to draw the vector into the vector space according to the coordinate value, find the angle between the two vectors and calculate the cosine value corresponding to the angle, the cosine value can be used to characterize the similarity of the two vectors. Sex. The smaller the angle, the closer the cosine value is to 1, and the directions of the two vectors are more consistent. The more similar the two vectors are.
  • the Euclidean distance is a commonly defined distance definition and is the true distance between two points in an m-dimensional space.
  • Cosine similarity and Euclidean distance have a wide range of applications in calculating the distance between vectors, both of which are easy to understand and easy to operate. Cosine similarity is a good way to output normalized results, while Euclidean distance is the method of outputting global values. In actual operation, one of the methods can be arbitrarily selected according to actual needs.
  • the construction of the quality score of the multimedia resource is a mapping process from the cumulative distribution vector to the interval [0, 1]: f: V n ⁇ [0, 1], where V n represents the n-dimensional vector space, [0 , 1] indicates the range of values of the quality score Score.
  • the quality assessment device for the multimedia resource is based on the user experience and faithful to the user, that is, the quality of the multimedia resource is characterized by the user behavior of the multimedia resource, which enables the present invention to more accurately describe the quality of the multimedia resource.
  • the quality evaluation apparatus of the embodiment of the present invention has strong operability because, for Internet applications, a large amount of multimedia resources are publicly available on the line, and users can consume these multimedia resources through daily click and viewing behaviors.
  • the enterprise background can use the log system to record these user behaviors. Therefore, the service process of the system is the data preparation process of the quality assessment of the multimedia resources, and thus the user behavior of acquiring the multimedia resources is simple and easy.
  • the traditional quality assessment device based on the original attributes of multimedia resources requires specialized staff and systems to complete the collection and measurement of relevant indicators.
  • the quality assessment device of the embodiment of the present invention since the user behavior will exhibit certain dynamic characteristics in a period of time, user behavior usually has cumulative characteristics. Therefore, after the quality assessment device of the embodiment of the present invention is used to evaluate the quality of the multimedia resource, the feedback of the online result can be absorbed to perform dynamic optimization ranking and recommendation of the multimedia resource, and the final ranking and recommendation of the multimedia resource can be given. result. If the user's behavior on the sorting and recommendation results of multimedia resources is not ideal, in the future iteration, the ranking of these multimedia resources and the quality score of the multimedia resources in the recommendation result will be Reduced, so that the original multimedia resources are sorted and the multimedia resources in the recommendation results are automatically ranked behind.
  • the quality assessment of the multimedia resource can be performed by using only the indicator data of one type of user behavior for quality assessment, or by using statistical methods such as indicator data of multiple types of user behavior. Quality assessment.
  • the quality scores of each type of user behavior may be separately calculated, and then the quality scores of user behaviors of all categories are averaged to determine the quality score of the multimedia resources.
  • FIG. 5 is a block diagram showing the structure of a quality assessment apparatus for a multimedia resource according to Embodiment 5 of the present invention.
  • the quality evaluation apparatus 500 provided in this embodiment is used to implement the quality evaluation method shown in FIG. 2.
  • the quality assessment apparatus 500 can mainly include:
  • the sub-unit 510 is configured to divide indicator data of a type of user behavior into a plurality of groups.
  • the dividing sub-unit 510 may adopt a non-overlapping grouping method to divide the index data of a type of user behavior into a plurality of groups.
  • the dividing sub-unit 510 may adopt an overlapping grouping method to divide the index data of a type of user behavior into Multiple groups.
  • the dividing subunit 510 may include:
  • the obtaining module 511 is configured to obtain a maximum value max(D) and a minimum value min(D) of the indicator data D of the user behavior of the type;
  • the determining module 513 is connected to the obtaining module 511 for Determined to be a segmentation interval, where n is the number of groups;
  • the dividing module 515 is connected to the determining module 513 for dividing the interval max(D)-min(D) into n groups.
  • the partitioning sub-unit 510 uses a very efficient and commonly used non-overlapping grouping method that characterizes data distribution characteristics to divide the index data of a type of user behavior into multiple groups
  • the grouping process is as follows: Suppose a given set of real number fields On the upper data D, the obtaining module 511 can first obtain the maximum value max(D) and the minimum value min(D) of the data D; then the dividing module 515 sets the interval max(D)-min(D) (also called the pole The difference or the full distance is divided into n groups on average, and the corresponding segmentation interval is (also called group spacing), then n packets correspond to n grouping intervals, for example: with Two grouping intervals for the head and tail.
  • the grouping process is as follows: assuming that the data D on a set of real numbers is given, Then, the obtaining module 511 may first obtain the maximum value max(D) and the minimum value min(D) of the data D, and then the interval [min(D), max(D)] can include the entire data D; then the dividing module 515 sets the interval max (D)-min(D) is equally divided into n overlapping grouping intervals, for example: [min(D), max(D)] and The maximum and minimum two grouping intervals.
  • the statistic sub-unit 530 is connected to the sub-unit 510 for counting the number of metric data of the user behavior included in each group.
  • the statistical sub-unit 530 may separately count the index data falling in each interval.
  • the determining sub-unit 550 is connected to the statistical sub-unit 530 for determining a vector formed by the number corresponding to each group as a cumulative distribution vector of the user behavior of the multimedia resource.
  • the cumulative distribution vector of the user's behavior can be quickly determined directly from the frequency distribution histogram.
  • the second determining unit 570 is connected to the determining subunit 550, and is configured to determine an upper boundary vector and a lower boundary vector of the multimedia resource according to the cumulative distribution vector.
  • the third determining unit 590 is connected to the determining subunit 550 and the second determining unit 570, and is configured to determine a quality score of the multimedia resource according to the cumulative distribution vector, the upper boundary vector, and the lower boundary vector.
  • the quality assessment device for the multimedia resource is based on the user experience and faithful to the user, that is, the quality of the multimedia resource is characterized by the user behavior of the multimedia resource, which enables the present invention to more accurately describe the quality of the multimedia resource.
  • the quality evaluation apparatus of the embodiment of the present invention has strong operability because, for Internet applications, a large amount of multimedia resources are publicly disclosed, and users can consume these multimedia resources through daily click and viewing behavior.
  • the enterprise background can use the log system to record these user behaviors. Therefore, the service process of the system is the data preparation process for the quality assessment of multimedia resources, so the user behavior of obtaining multimedia resources is simple and easy.
  • the traditional quality assessment device based on the original attributes of multimedia resources requires specialized staff and systems to complete the collection and measurement of relevant indicators.
  • the feedback of the online result can be absorbed to perform dynamic optimization ranking and recommendation of the multimedia resource, and the final ranking and recommendation of the multimedia resource can be given. result. If the user's behavior on the sorting and recommendation results of the multimedia resources is not ideal, in the future iteration, the ranking of the multimedia resources and the quality score of the multimedia resources in the recommendation result are reduced, thereby sorting and recommending the original multimedia resources. The top multimedia resources in the results are automatically listed behind.
  • FIG. 6 is a block diagram showing the structure of a quality assessment apparatus for a multimedia resource according to Embodiment 6 of the present invention.
  • the indicator data of the user behavior of the playback class that is, the broadcast of the multimedia resource
  • the completion ratio (for example, video viewing completion ratio) is used to exemplify the quality evaluation apparatus of the multimedia resource of the present invention.
  • the quality evaluation apparatus 600 provided in this embodiment is used to implement the quality assessment method shown in FIG. 3a.
  • the quality assessment apparatus 600 can mainly include:
  • the use unit 610 is used to use the video play log of a video website as a basic data source.
  • the original video play log is a data table containing at least the following four-tuple: the four-tuple is ⁇ Vids, PlayLength, FullLength, Time ⁇ , where Vids represents the video collection being viewed; PlayLength represents the accumulation of each video view. The length of time, usually in seconds; FullLength represents the total length of time of the video being viewed; Time represents the timestamp of the occurrence of this viewing behavior.
  • Each line of the original video playback log stores the viewing behavior of the user's click-through video at that timestamp.
  • User viewing behavior data for one day, one hour, or even any time can be obtained by defining different timestamps.
  • An example fragment of a video viewing log data can be found in Table 1 of Example 3 above.
  • the above four-tuple ⁇ Vids, PlayLength, FullLength, Time ⁇ can be pre-processed by summarizing the video play log information of the user viewing time length.
  • the video playing data of a specific time period may be selected by defining a Time field.
  • video playing data with a Time field of “20160105” may be selected from the video playing log information.
  • the obtaining unit 620 is connected to the using unit 610, and is configured to obtain the indicator data of the video according to the video playing log, that is, the playing completion ratio.
  • the completion of the video playback than the perc refers to the ratio of the length of the video playback time to the total length of the video, ie
  • the first determining unit 630 is connected to the obtaining unit 620, and is used to refer to the user behavior of the playing class.
  • the target data playback is completed by dividing the perc into multiple groups and determining that the indicator data playback is completed than the cumulative distribution vector of perc.
  • the first determining unit 630 may use the non-overlapping grouping method described in the foregoing Embodiment 2 to divide the playing completion ratio into multiple groups, and the first determining unit 630 may also adopt the overlapping grouping described in Embodiment 2 above.
  • the method is to divide the play completion into multiple groups than perc.
  • the frequency distribution histogram can be used to display the frequency distribution of the playback completion ratio perc, wherein the frequency distribution histogram is the ratio of the frequency of the corresponding group to the group distance by the height of the rectangle (since the group distance is a constant, therefore In order to facilitate drawing and viewing, the height of the rectangle is directly used to represent the frequency, and the frequency distribution histogram clearly shows the distribution of the frequency of each group and it is easy to display the difference in frequency between the groups.
  • the frequency distribution histogram is the ratio of the frequency of the corresponding group to the group distance by the height of the rectangle (since the group distance is a constant, therefore
  • the height of the rectangle is directly used to represent the frequency
  • the frequency distribution histogram clearly shows the distribution of the frequency of each group and it is easy to display the difference in frequency between the groups.
  • the second determining unit 640 is connected to the first determining unit 630, and is configured to determine an upper boundary vector and a lower boundary vector of the playback completion ratio according to the playback completion ratio cumulative distribution vector Vvid .
  • step 304 in Embodiment 3 For details, refer to the related description of step 304 in Embodiment 3 above.
  • the first calculating unit 650 is connected to the first determining unit 630 and the second determining unit 640.
  • the first determining unit 630 determines that the playing completion ratio is the cumulative distribution vector Vvid
  • the second determining unit 640 determines the upper boundary of the playing completion ratio. after the vector V T and the boundary vectors V b, can be calculated has finished playing vid than the cumulative distribution vector V to play completion on the distance boundary vectors V T ratio and the playback is completed on the ratio of boundary vectors V T to play completion ratio lower boundary The distance of the vector V b .
  • step 305 For details, refer to the related description of step 305 in the above third embodiment.
  • the second calculation unit 660, 650 is connected to the first calculating means for calculating on the first player to complete the unit 650 calculates the ratio of the cumulative distribution vector V vid player to complete the distance boundary vectors V t than the play and completion ratio After the boundary vector V t reaches the distance of the lower boundary vector V b of the completion ratio, the quality score of the video is calculated using Equation 2 below:
  • the quality assessment device for the multimedia resource is based on the user experience and faithful to the user, that is, the quality of the multimedia resource is characterized by the user behavior of the multimedia resource, which enables the present invention to more accurately describe the quality of the multimedia resource.
  • the quality evaluation apparatus of the embodiment of the present invention has strong operability because, for Internet applications, a large amount of multimedia resources are publicly disclosed, and users can consume these multimedia resources through daily click and viewing behavior.
  • the enterprise background can use the log system to record these user behaviors. Therefore, the service process of the system is the data preparation process for the quality assessment of multimedia resources, so the user behavior of obtaining multimedia resources is simple and easy.
  • the traditional quality assessment device based on the original attributes of multimedia resources requires specialized staff and systems to complete the collection and measurement of relevant indicators.
  • the feedback of the online result can be absorbed to perform dynamic optimization ranking and recommendation of the multimedia resource, and the final ranking and recommendation of the multimedia resource can be given. result. If the user's behavior on the sorting and recommendation results of the multimedia resources is not ideal, in the future iteration, the ranking of the multimedia resources and the quality score of the multimedia resources in the recommendation result are reduced, thereby sorting and recommending the original multimedia resources. The top multimedia resources in the results are automatically listed behind.
  • FIG. 7 is a structural block diagram of a quality assessment apparatus for a multimedia resource according to another embodiment of the present invention.
  • the quality assessment device 1100 of the multimedia resource may be a host server having a computing capability, a personal computer PC, or a portable computer or terminal that can be carried.
  • the specific embodiments of the present invention do not limit the specific implementation of the computing node.
  • the quality assessment device 1100 of the multimedia resource includes a processor 1110, a communication interface 1120, a memory 1130, and a bus 1140.
  • the processor 1110, the communication interface 1120, and the memory 1130 are connected to each other through the bus 1140. letter.
  • Communication interface 1120 is for communicating with network devices, including, for example, a virtual machine management center, shared storage, and the like.
  • the processor 1110 is configured to execute a program.
  • the processor 1110 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.
  • ASIC Application Specific Integrated Circuit
  • the memory 1130 is used to store files.
  • the memory 1130 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.
  • Memory 1130 can also be a memory array.
  • the memory 1130 may also be partitioned, and the blocks may be combined into a virtual volume according to certain rules.
  • the above program may be program code including computer operating instructions.
  • the program is specifically applicable to: performing the steps of the method described in Embodiment 1, Embodiment 2 or Embodiment 3.
  • the function is implemented in the form of computer software and sold or used as a stand-alone product, it is considered to some extent that all or part of the technical solution of the present invention (for example, a part contributing to the prior art) is It is embodied in the form of computer software products.
  • the computer software product is typically stored in a computer readable non-volatile storage medium, including instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all of the methods of various embodiments of the present invention. Or part of the steps.
  • the foregoing storage medium includes a USB flash drive, a mobile hard disk, a read-only memory (ROM), and a random access memory (RAM, Random).
  • the method and device for evaluating quality of a multimedia resource according to an embodiment of the present invention can more accurately describe the quality of a multimedia resource and has strong operability.
  • the multimedia is evaluated by using a quality assessment method for multimedia resources according to an embodiment of the present invention. After the quality of the resources, the feedback of the online results can be absorbed to dynamically sort and recommend the multimedia resources, and the final sorting and recommendation results of the multimedia resources can be given.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Library & Information Science (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

一种多媒体资源的质量评估方法和装置，该多媒体资源的质量评估方法包括：根据用于刻画多媒体资源的用户行为的指标数据，确定多媒体资源的累积分布向量（S100）；根据累积分布向量，确定多媒体资源的上边界向量和下边界向量（S120）；以及根据累积分布向量、上边界向量和下边界向量，确定多媒体资源的质量分数（S140）。该方法及装置能够更准确地刻画多媒体资源的质量并且具有很强的可操作性，另外，在利用该方法来评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。

Description

多媒体资源的质量评估方法和装置
交叉引用
本申请主张2016年3月18日提交的中国专利申请号为201610159190.0的优先权，其全部内容通过引用包含于此。
技术领域
本发明涉及互联网技术领域，尤其涉及一种多媒体资源的质量评估方法和装置。
背景技术
现有的多媒体资源的质量计算和质量评估主要是以多媒体资源自身的原生属性(机器属性)作为考核指标。例如，通过音频和视频的信号质量、失真程度等静态属性来判断多媒体资源的整体质量。或者，在该质量评估方法的基础上加入一些例如帧率、清晰度等的面向用户的属性，来进一步判断多媒体资源的整体质量。还有一些质量评估方法是通过考察多媒体资源的网络特性来判断多媒体资源的整体质量。例如，通过计算流媒体在网络上传输的数据包的特性和解码过程来判断多媒体资源的整体质量。
也就是说，现有技术中的质量评估方法是将多媒体资源的原生属性(动态和静态)作为质量评估的主要指标。然而，由于仅基于多媒体资源的原生属性来评估多媒体资源的质量在很多情况下并不能满足用户的需求，因此多媒体资源的优劣是无法仅通过网络传输、分辨率等原生属性来刻画的。
发明内容
技术问题
有鉴于此，本发明要解决的技术问题是，提供一种多媒体资源的质量评估方法和装置，以更准确地刻画多媒体资源的质量。
解决方案
为了解决上述技术问题，在第一方面，本发明提供了一种多媒体资源的质量评估方法，包括：
根据用于刻画所述多媒体资源的用户行为的指标数据，确定所述多媒体资源的累积分布向量；
根据所述累积分布向量，确定所述多媒体资源的上边界向量和下边界向量；以及
根据所述累积分布向量、所述上边界向量和所述下边界向量，确定所述多媒体资源的质量分数。
结合第一方面，在第一种可能的实现方式中，所述根据用于刻画所述多媒体资源的用户行为的指标数据，确定所述多媒体资源的累积分布向量，包括：
将一类用户行为的指标数据划分为多个组；
统计每个组所包括的该类用户行为的指标数据的个数；以及
将每个组对应的个数构成的向量确定为所述多媒体资源的该类用户行为的累积分布向量。
结合第一方面的第一种可能的实现方式，在第二种可能的实现方式中，所述将一类用户行为的指标数据划分为多个组，包括：
获取该类用户行为的指标数据D的最大值max(D)和最小值min(D)；
将
Figure PCTCN2016099358-appb-000001
确定为分割区间，其中，n为组的个数；以及
将区间max(D)-min(D)划分为n个组。
结合第一方面或第一方面的第一种可能的实现方式或第一方面的第二种可能的实现方式，在第三种可能的实施方式中，所述根据所述累积分布向 量、所述上边界向量和所述下边界向量，确定所述多媒体资源的质量分数，包括：利用下式1计算所述质量分数，
Figure PCTCN2016099358-appb-000002
其中，Score表示所述质量分数，DistanceTOTOP表示所述累积分布向量到所述上边界向量的距离，DistanceBetween表示所述上边界向量到所述下边界向量的距离。
在第二方面，本发明提供了一种多媒体资源的质量评估装置，包括：
第一确定单元，用于根据用于刻画所述多媒体资源的用户行为的指标数据，确定所述多媒体资源的累积分布向量；
第二确定单元，与所述第一确定单元连接，用于根据所述累积分布向量，确定所述多媒体资源的上边界向量和下边界向量；以及
第三确定单元，与所述第一确定单元和所述第二确定单元连接，用于根据所述累积分布向量、所述上边界向量和所述下边界向量，确定所述多媒体资源的质量分数。
结合第二方面，在第一种可能的实现方式中，所述第一确定单元包括：
划分子单元，用于将一类用户行为的指标数据划分为多个组；
统计子单元，与所述划分子单元连接，用于统计每个组所包括的该类用户行为的指标数据的个数；以及
确定子单元，与所述统计子单元连接，用于将每个组对应的个数构成的向量确定为所述多媒体资源的该类用户行为的累积分布向量。
结合第二方面的第一种可能的实现方式，在第二种可能的实现方式中，所述划分子单元包括：
获取模块，用于获取该类用户行为的指标数据D的最大值max(D)和最小值min(D)；
确定模块，与所述获取模块连接，用于将
Figure PCTCN2016099358-appb-000003
确定为分割区间，其中，n为组的个数；以及
划分模块，与所述确定模块连接，用于将区间max(D)-min(D)划分为n个组。
结合第二方面或第二方面的第一种可能的实现方式或第二方面的第二种可能的实现方式，在第三种可能的实施方式中，所述第三确定单元具体用于利用下式1计算所述质量分数，
Figure PCTCN2016099358-appb-000004
其中，Score表示所述质量分数，DistanceTOTOP表示所述累积分布向量到所述上边界向量的距离，DistanceBetween表示所述上边界向量到所述下边界向量的距离。
有益效果
本发明实施例的多媒体资源的质量评估方法和装置，能够更准确地刻画多媒体资源的质量并且具有很强的可操作性，另外，在利用本发明实施例的多媒体资源的质量评估方法评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。
根据下面参考附图对示例性实施例的详细说明，本发明的其它特征及方面将变得清楚。
附图说明
包含在说明书中并且构成说明书的一部分的附图与说明书一起示出了本发明的示例性实施例、特征和方面，并且用于解释本发明的原理。
图1示出根据本发明实施例一的多媒体资源的质量评估方法的流程图；
图2示出根据本发明实施例二的多媒体资源的质量评估方法的流程图；
图3a示出根据本发明实施例三的多媒体资源的质量评估方法的流程图；
图3b示出播放完成比频度分布直方图；
图3c示出播放完成比频度分布直方图；
图3d示出视频的质量分数分布直方图；
图4示出根据本发明实施例四的多媒体资源的质量评估装置的结构框图；
图5示出根据本发明实施例五的多媒体资源的质量评估装置的结构框图；以及
图6示出根据本发明实施例六的多媒体资源的质量评估装置的结构框图。
图7示出根据本发明另一实施例的多媒体资源的质量评估设备的结构框图。
具体实施方式
以下将参考附图详细说明本发明的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
另外，为了更好的说明本发明，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本发明同样可以实施。在另外一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本发明的主旨。
实施例1
图1示出根据本发明实施例一的多媒体资源的质量评估方法的流程图。如图1所示，该质量评估方法主要可以包括：
步骤S100、根据用于刻画多媒体资源的用户行为的指标数据，确定多媒体资源的累积分布向量。
用户可以使用终端设备来播放多媒体资源。其中，该终端设备例如可以是手机、移动互联网设备(英文：Mobile Internet Device，简称：MID)、个人数字助理(英文：Personal Digital Assistant，简称：PDA)、笔记本、台式电脑、智能电视等。该多媒体资源例如可以是视频、音频、图片等。
需要说明的是，本发明的多媒体资源不仅限于上述三种示例，本领域技术人员应能够了解，本发明的重点并不在于多媒体资源，任何其它形式的多媒体资源也可以适用于本发明。也就是说，本发明并不限制多媒体资源的具体形式。
可以使用指标数据来刻画诸如视频、音频等的多媒体资源的用户行为，并且，多媒体资源的用户行为可以包括多种类别，例如顶踩、评论、推荐(转发)、收藏、播放、下载等。
其中，顶踩是指用户基于自身对被播放的多媒体资源的支持或者反对态度，对被播放的多媒体资源作出“顶”或者“踩”的操作。顶踩通常包括被播放的多媒体资源的标识(vid)、顶踩操作、操作人(用户)的相关信息、操作时间和IP(例如，用户的手机或者电脑等)等。
评论是指用户基于自身对被播放的多媒体资源的内容和形式的理解，在相应位置处作出的评论描述。评论通常包括被播放的多媒体资源的标识(vid)、评论的具体内容、操作人(用户)的相关信息、操作时间和IP等。
收藏是指用户基于自身对被播放的多媒体资源的内容和形式的理解所进行的收录操作，以便于未来能够更方便地找回该多媒体资源。收藏通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和 IP等。
推荐(转发)是指用户基于自身对被播放的多媒体资源的内容和形式的理解所进行的站外的推送操作。推荐通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP、推荐平台等。
播放是指用户对于多媒体资源的观看行为。播放通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP、播放时间长度等。
下载是指用户基于自身对被播放的多媒体资源的内容和形式的理解所进行的下载到本地的操作。下载通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP、下载进度等。
实际上，用户行为的构建过程是一个从问题领域到行为领域的映射过程：f:Pr oblemDo m ain→UserBehavior，其中，Pr oblemDo m ain表示问题领域，UserBehavior表示用户行为集合。
每个业务部门可以根据自身的后台数据和页面功能，选择最优的用户行为来进行考核。从实际效果来看，推荐使用能够真实反映用户需求意图的用户行为，从而使得多媒体资源的质量评估(计算)更精准。
具体地，可以使用指标数据来衡量每一类用户行为中的每一个用户行为，并且，每一类用户行为的指标数据的详细说明如下。
假设一个IP针对一个多媒体资源只能操作一次顶或者踩，则可以使用用户对多媒体资源所进行的顶踩操作的发生点作为指标数据来衡量顶踩类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生顶或者踩的多媒体资源的播放完成进度。理论上，希望用户没有踩的行为并且尽早发生顶的行为(太早发生顶的行为也是不合理的)。
可以使用用户对多媒体资源所进行的评论操作的发生点以及评论情感作为指标数据来衡量评论类的用户行为。如果以多媒体资源播放完成进度来 计算，则可以记录每次发生评论的多媒体资源的播放完成进度。同时，可以对用户评论的正负情感尽量量化。理论上，希望用户没有负向情感评论并且尽早发生评论行为(太早发生评论行为也是不合理的)。
假设一个IP针对一个多媒体资源只能收藏一次，则可以使用用户对多媒体资源所进行的收藏操作的发生点作为指标数据来衡量收藏类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生收藏行为的多媒体资源的播放完成进度。理论上，希望用户有收藏行为并且尽早发生收藏行为(太早发生收藏行为也是不合理的)。
可以使用用户对多媒体资源所进行的推荐操作的发生点以及被推荐的多媒体资源的导回流量比率作为指标数据来衡量推荐类的用户行为。其中，导回流量比率＝导回次数/露出次数，导回次数是指被推荐的多媒体资源二次被打开的次数，露出次数是指被推荐的多媒体资源的被推荐次数。如果以多媒体资源播放完成进度来计算，则可以记录每次发生推荐的多媒体资源的播放完成进度。同时，可以通过爬取外站的相关数据来计算导回流量比率。理论上，希望用户有推荐行为并且尽早发生推荐行为(太早发生推荐行为也是不合理的)而且导回流量比率越高越好。
可以使用多媒体资源的播放完成比和用户拖动进度条(快退、快进)的次数作为指标数据来衡量播放类的用户行为。希望用户的播放完成比越高越好并且没有快进拖动而是有多次合理的快退拖动。
可以使用用户对多媒体资源所进行的下载操作的发生点以及下载完成进度作为指标数据来衡量下载类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生下载行为的多媒体资源的播放完成进度。下载完成进度可以衡量用户下载多媒体资源的决心和网络状况。理论上，希望用户有下载行为并且尽早发生下载行为(太早发生下载行为也是不合理的)而且希望是100％完整下载。
需要说明的是，本发明实施例仅例示了几类用户行为及其指标数据，本领域技术人员应能够理解，本发明的用户行为的种类还可以为其它类别，并且在实际操作中不是必须提取上述各种指标数据，而是可以根据自身业务需求以及是否对系统造成过大的负担等来提取适量的指标数据。
实际上，指标数据的构建过程是一个从用户行为到指标数据的映射过程：f:UserBehavior→Indicators，其中，UserBehavior表示用户行为集合，Indicators表示指标数据集合。并且，累积分布向量的构建过程是一个从指标空间到向量空间的映射过程：f:Indicators→Vn，其中，Indicators表示指标数据集合，Vn表示n维向量空间。
步骤S120、根据累积分布向量，确定多媒体资源的上边界向量和下边界向量。
具体地，在确定了用户行为的指标数据上的累积分布向量之后，可以定义该指标数据的最优表现和最差表现，即上边界和下边界。例如，用户对多媒体资源所进行的合理的顶踩操作的发生点最多的计数为多少、被推荐的多媒体资源的导回流量比率最高的最大计数是多少、最多有多少用户完整观看了多媒体资源。其中，多媒体资源在指标数据上的上边界和下边界可以均使用向量来表示，即，上边界向量和下边界向量。
步骤S140、根据累积分布向量、上边界向量和下边界向量，确定多媒体资源的质量分数。
可以根据累积分布向量来确定多媒体资源的质量分数。理论上，一个累积分布向量离下边界向量越远并且离上边界向量越近，则说明用户行为的表现越好，进而说明多媒体资源的质量越高。例如，可以使用距离占比来定义多媒体资源的质量分数。
即，在一种可能的实现方式中，根据累积分布向量、上边界向量和下边界向量，确定多媒体资源的质量分数，可以包括：利用下式1计算质量分数，
Figure PCTCN2016099358-appb-000005
其中，Score表示质量分数，Dis tan ceTOTOP表示累积分布向量到上边界向量的距离，Dis tan ceBetween表示上边界向量到下边界向量的距离。
根据上述式1可知，累积分布向量到上边界向量的距离Dis tan ceTOTOP越小，质量分数Score越大。可以使用余弦相似度或者多维欧式距离等方法来计算向量之间的距离，并且余弦相似度和欧式距离可以保证质量分数Score的取值范围为[0,1]。
其中，余弦相似度是将向量根据坐标值绘制到向量空间中，求得两个向量之间的夹角并计算夹角对应的余弦值，该余弦值可以用于表征这两个向量的相似性。夹角越小，余弦值越接近于1，这两个向量的方向更加吻合，这两个向量就越相似。欧式距离是一个通常采用的距离定义，是在m维空间中两个点之间的真实距离。例如，假设二维空间中存在点A(x1,y1)和点B(x2,y2)，则点A(x1,y1)和点B(x2,y2)之间的欧式距离为
Figure PCTCN2016099358-appb-000006
在计算向量之间的距离上，余弦相似度和欧式距离都有广泛的应用，这两种方法均易于理解且便于操作。余弦相似度是一个良好的输出归一化结果的方法，而欧式距离是输出全域取值的方法。实际操作中，根据实际需要任意选取其中一种方法即可。
实际上，多媒体资源的质量分数的构建是一个从累积分布向量到区间[0,1]的映射过程：f:Vn→[0,1]，其中，Vn表示n维向量空间，[0,1]表示质量分数Score的取值范围。
本发明实施例的多媒体资源的质量评估方法，是基于用户体验和忠实于用户的，即通过多媒体资源的用户行为来刻画多媒体资源的质量，这使得本发明能够更准确地刻画多媒体资源的质量。
并且，本发明实施例的质量评估方法具有很强的可操作性，原因在于，对于互联网应用，大量的多媒体资源是线上公开的，用户可以通过每日的点 击和观看行为来消费这些多媒体资源，而企业后台可以使用日志系统来记录这些用户行为，因此，系统的服务过程就是多媒体资源的质量评估的数据准备过程，因而获取多媒体资源的用户行为是简单易行的。与之相比较，传统的基于多媒体资源的原生属性的质量评估方法需要专门的工作人员和系统来完成相关指标的采集和度量。
另外，由于以一段时间为考察区间，用户行为会呈现一定的动态特性，因此用户行为通常具有累积特性。因而，在利用本发明实施例的质量评估方法评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。如果用户在多媒体资源的排序和推荐结果上的行为不够理想，则在未来的迭代中，这些多媒体资源的排序和推荐结果中的多媒体资源的质量分数会降低，从而把原先多媒体资源的排序和推荐结果中靠前的多媒体资源自动排在后面。
实施例2
由于用户行为可以包括多种类别，因此多媒体资源的质量评估既可以仅利用一类用户行为的指标数据来进行质量评估，也可以利用诸如统计学的方法来根据多类用户行为的指标数据来进行质量评估。
例如，可以先分别计算每一类用户行为的质量分数，再对所有类别的用户行为的质量分数进行平均，以确定多媒体资源的质量分数。
本领域普通技术人员可以理解，平均只是一种实现方式，也可以采用其它实现方式，例如加权求和等，仍可实现本发明的基本目的。
图2示出根据本发明实施例二的多媒体资源的质量评估方法的流程图。如图2所示，该质量评估方法主要可以包括：
步骤S201、将一类用户行为的指标数据划分为多个组。
例如，可以采用非重叠分组方法来将一类用户行为的指标数据划分为多 个组，又如，可以采用重叠分组方法来将一类用户行为的指标数据划分为多个组。
在一种可能的实现方式中，将一类用户行为的指标数据划分为多个组，包括：
获取该类用户行为的指标数据D的最大值max(D)和最小值min(D)；
将
Figure PCTCN2016099358-appb-000007
确定为分割区间，其中，n为组的个数；以及
将区间max(D)-min(D)划分为n个组。
例如，假设采用非常有效并且常用的刻画数据分布特点的非重叠分组方法来将一类用户行为的指标数据D划分为多个组，则分组的过程如下：假设给定一组实数域上的数据D，则可以先获得数据D的最大值max(D)和最小值min(D)；然后将区间max(D)-min(D)(也称之为极差或全距)平均划分为n个分组，对应的分割区间为
Figure PCTCN2016099358-appb-000008
(也称之为组距)，则n个分组对应n个分组区间，例如：
Figure PCTCN2016099358-appb-000009
和
Figure PCTCN2016099358-appb-000010
为头部和尾部的两个分组区间。
又如，假设采用观察数据整体变化的重叠分组方法来将一类用户行为的指标数据划分为多个组，则分组的过程如下：假设给定一组实数域上的数据D，则可以先获得数据D的最大值max(D)和最小值min(D)，则区间[min(D),max(D)]能够包含全体数据D；然后将区间max(D)-min(D)平均划分为n个重叠分组区间，例如：[min(D),max(D)]和
Figure PCTCN2016099358-appb-000011
为最大和最小的两个分组区间。
步骤S203、统计每个组所包括的该类用户行为的指标数据的个数。
在将一类用户行为的指标数据划分为多个组之后，可以分别对落在每一个区间的指标数据进行个数统计。
步骤S205、将每个组对应的个数构成的向量确定为多媒体资源的该类用户行为的累积分布向量。
如果使用作图的方法来画出直方图，其中，x轴表示分组区间并且y轴表示频度计数，则可以直接根据频度分布直方图来快速地确定出用户行为的累积分布向量。
步骤S207、根据累积分布向量，确定多媒体资源的上边界向量和下边界向量。
步骤S209、根据累积分布向量、上边界向量和下边界向量，确定多媒体资源的质量分数。
步骤S207和步骤S209的说明可以参见上述实施例1中的步骤S120和步骤S140中的相关描述。
本发明实施例的多媒体资源的质量评估方法，是基于用户体验和忠实于用户的，即通过多媒体资源的用户行为来刻画多媒体资源的质量，这使得本发明能够更准确地刻画多媒体资源的质量。
并且本发明实施例的质量评估方法具有很强的可操作性，原因在于，对于互联网应用，大量的多媒体资源是线上公开的，用户可以通过每日的点击和观看行为来消费这些多媒体资源，而企业后台可以使用日志系统来记录这些用户行为，因此，系统的服务过程就是多媒体资源的质量评估的数据准备过程，因而获取多媒体资源的用户行为是简单易行的。与之相比较，传统的基于多媒体资源的原生属性的质量评估方法需要专门的工作人员和系统来完成相关指标的采集和度量。
另外，在利用本发明实施例的质量评估方法来评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。如果用户在多媒体资源的排序和推荐结果上的行为不够理想，则在未来的迭代中，这些多媒体资源的排 序和推荐结果中的多媒体资源的质量分数会降低，从而把原先多媒体资源的排序和推荐结果中靠前的多媒体资源自动排在后面。
实施例3
图3a示出根据本发明实施例三的多媒体资源的质量评估方法的流程图。在本发明实施例中，将以播放类用户行为的指标数据即多媒体资源的播放完成比(例如，视频观看完成比)来例示本发明的多媒体资源的质量评估方法。
步骤301、使用某视频网站的视频播放日志作为基本的数据来源。原始的视频播放日志是一个至少包含以下四元组的数据表格：该四元组为{Vids,PlayLength,FullLength,Time}，其中，Vids表示被观看的视频集合；PlayLength表示每次视频观看的累积时间长度，通常以秒计；FullLength表示被观看的视频的总时间长度；Time表示发生此次观看行为的时间戳。
原始的视频播放日志的每一行记录均存储了用户在该时间戳下的点击视频的观看行为。可以通过界定不同的时间戳，获取一天、一个小时、甚至任何时刻的用户观看行为数据。表1是一个视频观看日志数据的示例片段。
表1 视频观看日志数据的示例片段
Vids PlayLength FullLength Time
… … … …
1 2 100 20160105
1 12 100 20160105
1 11 100 20160105
1 53 100 20160105
1 34 100 20160105
1 23 100 20160105
1 77 100 20160105
1 88 100 20160105
1 88 100 20160105
1 96 100 20160105
1 100 100 20160105
1 112 100 20160105
1 69 100 20160105
1 41 100 20160105
1 79 100 20160105
… … … …
通过汇总用户观看时间长度的视频播放日志信息，可以对上述四元组{Vids,PlayLength,FullLength,Time}进行预处理。举例而言，可以通过界定Time字段，选取特定时间段的视频播放数据，例如，可以从视频播放日志信息中选取Time字段为“20160105”的视频播放数据。也可以使用PlayLength/FullLength来计算每一次观看Vids字段为“1”的视频的播放完成比(也称之为视频观看完成比)，以生成Vids字段为“1”的视频的播放完成比perc字段。还可以对视频的播放完成比数据进行数据清理，例如，应该舍弃perc>100％的数据。
步骤302、根据视频播放日志获取视频的指标数据即播放完成比。其中，视频的播放完成比perc是指视频的播放时间长度与视频的总时间长度的比值，即
Figure PCTCN2016099358-appb-000012
通过对上述表1的视频观看日志数据进行预处理，可以生成包含被观看的视频集合Vids、被观看的视频的播放完成比perc和发生此次观看行为的时间戳Time的信息，以记录用户在视频上的观看完成情况。其中，使用上述表1中的视频观看日志数据可以得到下述表2所示的被观看的视频的播放完成比perc。需要说明的是，为关注问题本身而在表2中省去了发生此次观看行为的时间戳Time。
表2 视频的播放完成比perc示例
Figure PCTCN2016099358-appb-000013
步骤303、将播放类的用户行为的指标数据播放完成比perc划分为多个组并确定指标数据播放完成比perc的累积分布向量。
具体地，可以采用上述实施例2中描述的非重叠分组方法来将播放完成 比perc划分为多个组，也可以采用上述实施例2中描述的重叠分组方法来将播放完成比perc划分为多个组。并且可以使用频度分布直方图来显示播放完成比perc的频度分布，其中，频度分布直方图是通过长方形的高代表对应组的频数与组距的比值(由于组距是一个常数，因此为了便于画图和看图而直接使用长方形的高来表示频数)，并且频度分布直方图能够清楚地显示各组频数的分布情况并且易于显示各组之间的频数的差别。
频度分布直方图的横轴为播放完成比perc的0％-100％的全域区间。可以根据实际情况来确定分组的组距。其中，根据统计学的知识可知，合适的组距可以反映抽样数据的分布特性，组距太小会造成分组的组数太多，组距太大会造成分组的组数太少，这些都会掩盖播放完成比perc的分布特性。另外，组距太小所导致的分组的组数太多，使得后续根据播放完成比所确定的累积分布向量即播放完成比累积分布向量的维度过高，这对海量数据分析时造成了计算负担。因此，在本实施例中，将组距确定为10％。
如果采用非重叠分组方法来将播放完成比perc划分为多个组，则可以按照组距10％来将播放完成比perc的0％-100％的全域区间划分为10个组，这样可以形成诸如0％-10％、10％-20％、…、90％-100％的10个等距非重叠抽样区间(分组)，由此可以得到图3b所示的播放完成比频度分布直方图。
如果采用重叠分组方法来将播放完成比perc划分为多个组，则可以按照组距10％来将播放完成比perc的0％-100％的全域区间划分为10个组，这样可以形成诸如0％-10％、0％-20％、…、0％-100％的10个等距重叠抽样区间(分组)，由此可以得到图3c所示的播放完成比频度分布直方图。
当然，在本实施例中，优选使用重叠分组方法(即，累积分布计数)来将播放完成比perc划分为多个组并确定播放完成比累积分布向量。使用播放完成比perc为0％、10％、20％、30％、…、100％的抽样区间。由于视频播放的涵盖特性，因此播放完成比perc为30％的计数一定包括播放完成比Perc为 10％的计数。
假设一个视频vid在特定时间段内的播放完成比数据集是P＝{perc|0％≤perc≤100％}，对播放完成比数据集P进行组距为m/100的重叠分组统计，其中，100mod m＝0(100可以被m整除)，可以得到包含单点分组0％在内的(100/m)+1个分组数据。如果用(0％,i％]表示播放完成比i％的计数区间，则分组0％、(0％,10％]、…、(0％,100％]所对应的计数所构成的向量(f0,f1,…,f100/m)为该视频的播放完成比累积分布向量Vvid。
需要说明的是，每一个分组的临界点映射为实际视频播放的进度点，用户的播放完成比可以刻画视频播放的进度。单点分组0％可以理解为该视频被点击的次数，即视频只要被点击就会形成一次频度计数，可以采用视频播放日志数据中的该视频被记录的条数。显然，如果m％>c％，则播放完成m％的次数一定不会多于播放完成c％的次数，区间(0％,m％]的频度一定不会多于区间(0％,c％]的频度，例如，看完视频的100％的次数一定不会多于看完视频的20％的次数。因此，播放完成比累积分布向量Vvid的内部数据是一个非递增数列。
根据播放完成比累积分布向量Vvid的定义，可以得到如图3c所示的播放完成比频度分布直方图。并且，根据图3c可知，与该播放完成比频度分布直方图相对应的播放完成比累积分布向量Vvid＝(14,13,11,10,9,8,7,6,4,2,1)。
步骤304、根据播放完成比累积分布向量Vvid，确定播放完成比的上边界向量和下边界向量。
每一个视频都希望能够被完整的播放，即有多少人点击打开视频也应该有多少人最终完成观看。结合播放完成比累积分布向量Vvid，上边界向量的每一个元素都应该和完成0％的频度相同，因此，可以根据播放完成比累积分布向量Vvid定义播放完成比的上边界向量Vt、即一个视频的播放完成情况的可能最优表现。即，假设一个视频的播放完成比累积分布向量 Vvid＝(f0,f1,…,f100/m)，则该视频的播放完成比的上边界向量Vt＝(f0,f0,…,f0)并且|Vt|＝|Vvid|。
类似地，可以考察一个视频的播放完成情况的可能最差表现，即每一次点击打开视频都没有实际的观看行为。结合播放完成比累积分布向量Vvid，除了0％的对应点击次数以外，其它抽样区间的累积频度均为0，因此，可以根据播放完成比累积分布向量Vvid定义播放完成比的下边界向量Vb。即，假设一个视频的播放完成比累积分布向量Vvid＝(f0,f1,…,f100/m)，则该视频的播放完成比的下边界向量Vb＝(f0,0,…,0)并且|Vb|＝|Vvid|。
继续使用之前的例子，m的取值为10，由此可以得到播放完成比的上边界向量和下边界向量分别为：Vt＝(14,14,14,14,14,14,14,14,14,14,14)和Vb＝(14,0,0,0,0,0,0,0,0,0,0)。
步骤305、在确定出播放完成比累积分布向量Vvid、播放完成比的上边界向量Vt和下边界向量Vb之后，可以计算播放完成比累积分布向量Vvid到播放完成比的上边界向量Vt的距离以及播放完成比的上边界向量Vt到播放完成比的下边界向量Vb的距离。
由于m的取值为10，因此使用11维欧式距离来分别计算上述两个距离。其中，11维欧式距离的具体定义如下：
假设X和Y为两个向量，则向量X和向量Y之间的距离为
Figure PCTCN2016099358-appb-000014
其中，j∈[1,11]，xj为向量X在第j个位置的取值，yj为向量Y在第j个位置的取值。
利用上述距离定义可以计算出播放完成比累积分布向量Vvid＝(14,13,11,10,9,8,7,6,4,2,1)和播放完成比的上边界向量Vt＝(14,14,14,14,14,14,14,14,14,14,14)之间的距离d(Vvid,Vt)为24.759并且播放完成比的上边界向量Vt＝(14,14,14,14,14,14,14,14,14,14,14)和播放完成比的下边界向量Vb＝(14,0,0,0,0,0,0,0,0,0,0)之间的距离d(Vt,Vb)为44.272。
步骤306、在计算出播放完成比累积分布向量Vvid到播放完成比的上边界向量Vt的距离以及播放完成比的上边界向量Vt到播放完成比的下边界向量Vb的距离之后，利用下述式2来计算视频的质量分数：
Figure PCTCN2016099358-appb-000015
即，视频的质量分数
Figure PCTCN2016099358-appb-000016
例如，可以对某视频网站的海量视频进行质量分数的计算，由此可以得到下述表3所示的该视频网站的视频质量分数统计表。
表3 某视频网站的视频的质量分数统计
Figure PCTCN2016099358-appb-000017
根据上述表3可以看出，在实际数据中，第三四分位数已经达到最大值，这意味着至少有25％的视频质量分数为1，这是由于视频播放的长尾效应造成的，即有大量视频只有一次或两次播放行为且均实现了完整播放。
通过去除视频播放的长尾效应，可以得到图3d所示的视频的质量分数分布直方图。并且，得到这些视频的质量分数之后，既可以引入新类别的用户行为进行多次计算，也可以引入视频的新的指标数据来进行计算，从而为后续的视频搜索和视频推荐做准备。
本发明实施例的多媒体资源的质量评估方法，是基于用户体验和忠实于用户的，即通过多媒体资源的用户行为来刻画多媒体资源的质量，这使得本发明能够更准确地刻画多媒体资源的质量。
并且本发明实施例的质量评估方法具有很强的可操作性，原因在于，对于互联网应用，大量的多媒体资源是线上公开的，用户可以通过每日的点击和观看行为来消费这些多媒体资源，而企业后台可以使用日志系统来记录这 些用户行为，因此，系统的服务过程就是多媒体资源的质量评估的数据准备过程，因而获取多媒体资源的用户行为是简单易行的。与之相比较，传统的基于多媒体资源的原生属性的质量评估方法需要专门的工作人员和系统来完成相关指标的采集和度量。
另外，在利用本发明实施例的质量评估方法来评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。如果用户在多媒体资源的排序和推荐结果上的行为不够理想，则在未来的迭代中，这些多媒体资源的排序和推荐结果中的多媒体资源的质量分数会降低，从而把原先多媒体资源的排序和推荐结果中靠前的多媒体资源自动排在后面。
实施例4
图4示出根据本发明实施例四的多媒体资源的质量评估装置的结构框图。本实施例提供的质量评估装置400用于实现图1所示的质量评估方法。如图4所示，该质量评估装置400主要可以包括：
第一确定单元410，用于根据用于刻画多媒体资源的用户行为的指标数据，确定多媒体资源的累积分布向量。
用户可以使用终端设备来播放多媒体资源。其中，该终端设备例如可以是手机、移动互联网设备(英文：Mobile Internet Device，简称：MID)、个人数字助理(英文：Personal Digital Assistant，简称：PDA)、笔记本、台式电脑、智能电视等。该多媒体资源例如可以是视频、音频、图片等。
需要说明的是，本发明的多媒体资源不仅限于上述三种示例，本领域技术人员应能够了解，本发明的重点并不在于多媒体资源，任何其它形式的多媒体资源也可以适用于本发明。也就是说，本发明并不限制多媒体资源的具体形式。
可以使用指标数据来刻画诸如视频、音频等的多媒体资源的用户行为， 并且，多媒体资源的用户行为可以包括多种类别，例如顶踩、评论、推荐(转发)、收藏、播放、下载等。
其中，顶踩是指用户基于自身对被播放的多媒体资源的支持或者反对态度，对被播放的多媒体资源作出“顶”或者“踩”的操作。顶踩通常包括被播放的多媒体资源的标识(vid)、顶踩操作、操作人(用户)的相关信息、操作时间和IP(例如，用户的手机或者电脑等)等。
评论是指用户基于自身对被播放的多媒体资源的内容和形式的理解，在相应位置处作出的评论描述。评论通常包括被播放的多媒体资源的标识(vid)、评论的具体内容、操作人(用户)的相关信息、操作时间和IP等。
收藏是指用户基于自身对被播放的多媒体资源的内容和形式的理解所进行的收录操作，以便于未来能够更方便地找回该多媒体资源。收藏通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP等。
推荐(转发)是指用户基于自身对被播放的多媒体资源的内容和形式的理解所进行的站外的推送操作。推荐通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP、推荐平台等。
播放是指用户对于多媒体资源的观看行为。播放通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP、播放时间长度等。
下载是指用户基于自身对被播放的多媒体资源的内容和形式的理解所进行的下载到本地的操作。下载通常包括被播放的多媒体资源的标识(vid)、操作人(用户)的相关信息、操作时间和IP、下载进度等。
实际上，用户行为的构建过程是一个从问题领域到行为领域的映射过程：f:Pr oblemDo m ain→UserBehavior，其中，Pr oblemDo m ain表示问题领域，UserBehavior表示用户行为集合。
每个业务部门可以根据自身的后台数据和页面功能，选择最优的用户行为来进行考核。从实际效果来看，推荐使用能够真实反映用户需求意图的用户行为，从而使得多媒体资源的质量评估(计算)更精准。
具体地，可以使用指标数据来衡量每一类用户行为中的每一个用户行为，并且，每一类用户行为的指标数据的详细说明如下。
假设一个IP针对一个多媒体资源只能操作一次顶或者踩，则可以使用用户对多媒体资源所进行的顶踩操作的发生点作为指标数据来衡量顶踩类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生顶或者踩的多媒体资源的播放完成进度。理论上，希望用户没有踩的行为并且尽早发生顶的行为(太早发生顶的行为也是不合理的)。
可以使用用户对多媒体资源所进行的评论操作的发生点以及评论情感作为指标数据来衡量评论类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生评论的多媒体资源的播放完成进度。同时，可以对用户评论的正负情感尽量量化。理论上，希望用户没有负向情感评论并且尽早发生评论行为(太早发生评论行为也是不合理的)。
假设一个IP针对一个多媒体资源只能收藏一次，则可以使用用户对多媒体资源所进行的收藏操作的发生点作为指标数据来衡量收藏类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生收藏行为的多媒体资源的播放完成进度。理论上，希望用户有收藏行为并且尽早发生收藏行为(太早发生收藏行为也是不合理的)。
可以使用用户对多媒体资源所进行的推荐操作的发生点以及被推荐的多媒体资源的导回流量比率作为指标数据来衡量推荐类的用户行为。其中，导回流量比率＝导回次数/露出次数，导回次数是指被推荐的多媒体资源二次被打开的次数，露出次数是指被推荐的多媒体资源的被推荐次数。如果以多媒体资源播放完成进度来计算，则可以记录每次发生推荐的多媒体资源的播 放完成进度。同时，可以通过爬取外站的相关数据来计算导回流量比率。理论上，希望用户有推荐行为并且尽早发生推荐行为(太早发生推荐行为也是不合理的)而且导回流量比率越高越好。
可以使用多媒体资源的播放完成比和用户拖动进度条(快退、快进)的次数作为指标数据来衡量播放类的用户行为。希望用户的播放完成比越高越好并且没有快进拖动而是有多次合理的快退拖动。
可以使用用户对多媒体资源所进行的下载操作的发生点以及下载完成进度作为指标数据来衡量下载类的用户行为。如果以多媒体资源播放完成进度来计算，则可以记录每次发生下载行为的多媒体资源的播放完成进度。下载完成进度可以衡量用户下载多媒体资源的决心和网络状况。理论上，希望用户有下载行为并且尽早发生下载行为(太早发生下载行为也是不合理的)而且希望是100％完整下载。
需要说明的是，本发明实施例中的仅例示了几类用户行为及其指标数据，本领域技术人员应能够理解，本发明的用户行为的种类还可以为其它类别，并且在实际操作中不是必须提取上述各种指标数据，而是可以根据自身业务需求以及是否对系统造成过大的负担等来提取适量的指标数据。
实际上，指标数据的构建过程是一个从用户行为到指标数据的映射过程：f:UserBehavior→Indicators，其中，UserBehavior表示用户行为集合，Indicators表示指标数据集合。并且，累积分布向量的构建过程是一个从指标空间到向量空间的映射过程：f:Indicators→Vn，其中，Indicators表示指标数据集合，Vn表示n维向量空间。
第二确定单元430，与第一确定单元410连接，用于根据累积分布向量，确定多媒体资源的上边界向量和下边界向量。
具体地，在第一确定单元410确定了用户行为的指标数据上的累积分布向量之后，第二确定单元430可以定义该指标数据的最优表现和最差表现， 即上边界和下边界。例如，用户对多媒体资源所进行的合理的顶踩操作的发生点最多的计数为多少、被推荐的多媒体资源的导回流量比率最高的最大计数是多少、最多有多少用户完整观看了多媒体资源。其中，多媒体资源在指标数据上的上边界和下边界均使用向量来表示，即，上边界向量和下边界向量。
第三确定单元450，与第一确定单元410和第二确定单元430连接，用于根据累积分布向量、上边界向量和下边界向量，确定多媒体资源的质量分数。
第三确定单元450可以根据第一确定单元410确定出的累积分布向量和第二确定单元430确定出的上边界向量和下边界向量来确定多媒体资源的质量分数。理论上，一个累积分布向量离下边界向量越远并且离上边界向量越近，则说明用户行为的表现越好，进而说明多媒体资源的质量越高。例如，可以使用距离占比来定义多媒体资源的质量分数。
即，在一种可能的实现方式中，第三确定单元450具体用于利用下式1计算质量分数，
Figure PCTCN2016099358-appb-000018
其中，Score表示质量分数，Dis tan ceTOTOP表示累积分布向量到上边界向量的距离，Dis tan ceBetween表示上边界向量到下边界向量的距离。
根据上述式1可知，累积分布向量到上边界向量的距离Dis tan ceTOTOP越小，质量分数Score越大。可以使用余弦相似度或者多维欧式距离等方法来计算向量之间的距离，并且余弦相似度和欧式距离可以保证质量分数Score的取值范围为[0,1]。
其中，余弦相似度是将向量根据坐标值绘制到向量空间中，求得两个向量之间的夹角并计算夹角对应的余弦值，该余弦值就可以用于表征这两个向量的相似性。夹角越小，余弦值越接近于1，这两个向量的方向更加吻合，这两个向量就越相似。欧式距离是一个通常采用的距离定义，是在m维空间 中两个点之间的真实距离。例如，假设二维空间中存在点A(x1,y1)和点B(x2,y2)，则点A(x1,y1)和点B(x2,y2)之间的欧式距离为
Figure PCTCN2016099358-appb-000019
在计算向量之间的距离上，余弦相似度和欧式距离都有广泛的应用，这两种方法均易于理解且便于操作。余弦相似度是一个良好的输出归一化结果的方法，而欧式距离是输出全域取值的方法。实际操作中，根据实际需要任意选取其中一种方法即可。
实际上，多媒体资源的质量分数的构建是一个从累积分布向量到区间[0,1]的映射过程：f:Vn→[0,1]，其中，Vn表示n维向量空间，[0,1]表示质量分数Score的取值范围。
本发明实施例的多媒体资源的质量评估装置，是基于用户体验和忠实于用户的，即通过多媒体资源的用户行为来刻画多媒体资源的质量，这使得本发明能够更准确地刻画多媒体资源的质量。
并且，本发明实施例的质量评估装置具有很强的可操作性，原因在于，对于互联网应用，大量的多媒体资源是线上公开的，用户可以通过每日的点击和观看行为来消费这些多媒体资源，而企业后台可以使用日志系统来记录这些用户行为，因此，系统的服务过程就是多媒体资源的质量评估的数据准备过程，因而获取多媒体资源的用户行为是简单易行的。与之相比较，传统的基于多媒体资源的原生属性的质量评估装置需要专门的工作人员和系统来完成相关指标的采集和度量。
另外，由于以一段时间为考察区间，用户行为会呈现一定的动态特性，因此用户行为通常具有累积特性。因而，在利用本发明实施例的质量评估装置来评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。如果用户在多媒体资源的排序和推荐结果上的行为不够理想，则在未来的迭代中，这些多媒体资源的排序和推荐结果中的多媒体资源的质量分数会 降低，从而把原先多媒体资源的排序和推荐结果中靠前的多媒体资源自动排在后面。
实施例5
由于用户行为可以包括多种类别，因此多媒体资源的质量评估既可以仅利用一类用户行为的指标数据来进行质量评估，也可以利用诸如统计学的方法来根据多类用户行为的指标数据来进行质量评估。
例如，可以先分别计算每一类用户行为的质量分数，再对所有类别的用户行为的质量分数进行平均，以确定多媒体资源的质量分数。
本领域普通技术人员可以理解，平均只是一种实现方式，也可以采用其它实现方式，例如加权求和等，仍可实现本发明的基本目的。
图5示出根据本发明实施例五的多媒体资源的质量评估装置的结构框图。本实施例提供的质量评估装置500用于实现图2所示的质量评估方法。如图5所示，该质量评估装置500主要可以包括：
划分子单元510，用于将一类用户行为的指标数据划分为多个组。
例如，划分子单元510可以采用非重叠分组方法来将一类用户行为的指标数据划分为多个组，又如，划分子单元510可以采用重叠分组方法来将一类用户行为的指标数据划分为多个组。
在一种可能的实现方式中，划分子单元510可以包括：
获取模块511，用于获取该类用户行为的指标数据D的最大值max(D)和最小值min(D)；
确定模块513，与获取模块511连接，用于将
Figure PCTCN2016099358-appb-000020
确定为分割区间，其中，n为组的个数；以及
划分模块515，与确定模块513连接，用于将区间max(D)-min(D)划分为n个组。
例如，假设划分子单元510采用非常有效并且常用的刻画数据分布特点的非重叠分组方法来将一类用户行为的指标数据划分为多个组，则分组的过程如下：假设给定一组实数域上的数据D，则获取模块511可以先获得数据D的最大值max(D)和最小值min(D)；然后划分模块515将区间max(D)-min(D)(也称之为极差或全距)平均划分为n个分组，对应的分割区间为
Figure PCTCN2016099358-appb-000021
(也称之为组距)，则n个分组对应n个分组区间，例如：
Figure PCTCN2016099358-appb-000022
和
Figure PCTCN2016099358-appb-000023
为头部和尾部的两个分组区间。
又如，假设划分子单元510采用观察数据整体变化的重叠分组方法来将一类用户行为的指标数据划分为多个组，则分组的过程如下：假设给定一组实数域上的数据D，则获取模块511可以先获得数据D的最大值max(D)和最小值min(D)，则区间[min(D),max(D)]能够包含全体数据D；然后划分模块515将区间max(D)-min(D)平均划分为n个重叠分组区间，例如：[min(D),max(D)]和
Figure PCTCN2016099358-appb-000024
为最大和最小的两个分组区间。
统计子单元530，与划分子单元510连接，用于统计每个组所包括的该类用户行为的指标数据的个数。
在划分子单元510将一类用户行为的指标数据划分为多个组之后，统计子单元530可以分别对落在每一个区间的指标数据进行个数统计。
确定子单元550，与统计子单元530连接，用于将每个组对应的个数构成的向量确定为多媒体资源的该类用户行为的累积分布向量。
如果使用作图的方法来画出直方图，其中，x轴表示分组区间并且y轴表示频度计数，则可以直接根据频度分布直方图来快速地确定出用户行为的累积分布向量。
第二确定单元570，与确定子单元550连接，用于根据累积分布向量，确定多媒体资源的上边界向量和下边界向量。
第三确定单元590，与确定子单元550和第二确定单元570连接，用于根据累积分布向量、上边界向量和下边界向量，确定多媒体资源的质量分数。
第二确定单元570和第三确定单元590的说明可以参见上述实施例4中的第二确定单元430和第三确定单元450中的相关描述。
本发明实施例的多媒体资源的质量评估装置，是基于用户体验和忠实于用户的，即通过多媒体资源的用户行为来刻画多媒体资源的质量，这使得本发明能够更准确地刻画多媒体资源的质量。
并且本发明实施例的质量评估装置具有很强的可操作性，原因在于，对于互联网应用，大量的多媒体资源是线上公开的，用户可以通过每日的点击和观看行为来消费这些多媒体资源，而企业后台可以使用日志系统来记录这些用户行为，因此，系统的服务过程就是多媒体资源的质量评估的数据准备过程，因而获取多媒体资源的用户行为是简单易行的。与之相比较，传统的基于多媒体资源的原生属性的质量评估装置需要专门的工作人员和系统来完成相关指标的采集和度量。
另外，在利用本发明实施例的质量评估装置来评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。如果用户在多媒体资源的排序和推荐结果上的行为不够理想，则在未来的迭代中，这些多媒体资源的排序和推荐结果中的多媒体资源的质量分数会降低，从而把原先多媒体资源的排序和推荐结果中靠前的多媒体资源自动排在后面。
实施例6
图6示出根据本发明实施例六的多媒体资源的质量评估装置的结构框图。在本发明实施例中，将以播放类用户行为的指标数据即多媒体资源的播 放完成比(例如，视频观看完成比)来例示本发明的多媒体资源的质量评估装置。本实施例提供的质量评估装置600用于实现图3a所示的质量评估方法。如图6所示，该质量评估装置600主要可以包括：
使用单元610，用于使用某视频网站的视频播放日志作为基本的数据来源。原始的视频播放日志是一个至少包含以下四元组的数据表格：该四元组为{Vids,PlayLength,FullLength,Time}，其中，Vids表示被观看的视频集合；PlayLength表示每次视频观看的累积时间长度，通常以秒计；FullLength表示被观看的视频的总时间长度；Time表示发生此次观看行为的时间戳。
原始的视频播放日志的每一行记录均存储了用户在该时间戳下的点击视频的观看行为。可以通过界定不同的时间戳，获取一天、一个小时、甚至任何时刻的用户观看行为数据。一个视频观看日志数据的示例片段可以参见上述实施例三中的表1。
通过汇总用户观看时间长度的视频播放日志信息，可以对上述四元组{Vids,PlayLength,FullLength,Time}进行预处理。举例而言，可以通过界定Time字段，选取特定时间段的视频播放数据，例如，可以从视频播放日志信息中选取Time字段为“20160105”的视频播放数据。也可以使用PlayLength/FullLength来计算每一次观看Vids字段为“1”的视频的播放完成比(也称之为视频观看完成比)，以生成Vids字段为“1”的视频的播放完成比perc字段。还可以对视频的播放完成比数据进行数据清理，例如，应该舍弃perc>100％的数据。
获取单元620，与使用单元610连接，用于根据视频播放日志获取视频的指标数据即播放完成比。其中，视频的播放完成比perc是指视频的播放时间长度与视频的总时间长度的比值，即
Figure PCTCN2016099358-appb-000025
具体说明可以参见上述实施例三中的步骤302的相关描述。
第一确定单元630，与获取单元620连接，用于将播放类的用户行为的指 标数据播放完成比perc划分为多个组并确定指标数据播放完成比perc的累积分布向量。
具体地，第一确定单元630可以采用上述实施例2中描述的非重叠分组方法来将播放完成比perc划分为多个组，第一确定单元630也可以采用上述实施例2中描述的重叠分组方法来将播放完成比perc划分为多个组。并且可以使用频度分布直方图来显示播放完成比perc的频度分布，其中，频度分布直方图是通过长方形的高代表对应组的频数与组距的比值(由于组距是一个常数，因此为了便于画图和看图而直接使用长方形的高来表示频数)，并且频度分布直方图能够清楚地显示各组频数的分布情况并且易于显示各组之间的频数的差别。具体说明可以参见上述实施例三中的步骤303的相关描述。
第二确定单元640，与第一确定单元630连接，用于根据播放完成比累积分布向量Vvid，确定播放完成比的上边界向量和下边界向量。
具体说明可以参见上述实施例三中的步骤304的相关描述。
第一计算单元650，与第一确定单元630和第二确定单元640连接，在第一确定单元630确定出播放完成比累积分布向量Vvid、第二确定单元640确定出播放完成比的上边界向量Vt和下边界向量Vb之后，可以计算播放完成比累积分布向量Vvid到播放完成比的上边界向量Vt的距离以及播放完成比的上边界向量Vt到播放完成比的下边界向量Vb的距离。
具体说明可以参见上述实施例三中的步骤305的相关描述。
第二计算单元660，与第一计算单元650连接，用于在第一计算单元650计算出播放完成比累积分布向量Vvid到播放完成比的上边界向量Vt的距离以及播放完成比的上边界向量Vt到播放完成比的下边界向量Vb的距离之后，利用下述式2来计算视频的质量分数：
Figure PCTCN2016099358-appb-000026
具体说明可以参见上述实施例三中的步骤306的相关描述。
本发明实施例的多媒体资源的质量评估装置，是基于用户体验和忠实于用户的，即通过多媒体资源的用户行为来刻画多媒体资源的质量，这使得本发明能够更准确地刻画多媒体资源的质量。
并且本发明实施例的质量评估装置具有很强的可操作性，原因在于，对于互联网应用，大量的多媒体资源是线上公开的，用户可以通过每日的点击和观看行为来消费这些多媒体资源，而企业后台可以使用日志系统来记录这些用户行为，因此，系统的服务过程就是多媒体资源的质量评估的数据准备过程，因而获取多媒体资源的用户行为是简单易行的。与之相比较，传统的基于多媒体资源的原生属性的质量评估装置需要专门的工作人员和系统来完成相关指标的采集和度量。
另外，在利用本发明实施例的质量评估装置来评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。如果用户在多媒体资源的排序和推荐结果上的行为不够理想，则在未来的迭代中，这些多媒体资源的排序和推荐结果中的多媒体资源的质量分数会降低，从而把原先多媒体资源的排序和推荐结果中靠前的多媒体资源自动排在后面。
实施例7
图7示出了本发明的另一个实施例的一种多媒体资源的质量评估设备的结构框图。所述多媒体资源的质量评估设备1100可以是具备计算能力的主机服务器、个人计算机PC、或者可携带的便携式计算机或终端等。本发明具体实施例并不对计算节点的具体实现做限定。
所述多媒体资源的质量评估设备1100包括处理器(processor)1110、通信接口(Communications Interface)1120、存储器(memory)1130和总线1140。其中，处理器1110、通信接口1120、以及存储器1130通过总线1140完成相互间的通 信。
通信接口1120用于与网络设备通信，其中网络设备包括例如虚拟机管理中心、共享存储等。
处理器1110用于执行程序。处理器1110可能是一个中央处理器CPU，或者是专用集成电路ASIC(Application Specific Integrated Circuit)，或者是被配置成实施本发明实施例的一个或多个集成电路。
存储器1130用于存放文件。存储器1130可能包含高速RAM存储器，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1130也可以是存储器阵列。存储器1130还可能被分块，并且所述块可按一定的规则组合成虚拟卷。
在一种可能的实施方式中，上述程序可为包括计算机操作指令的程序代码。该程序具体可用于：执行实施例1、实施例2或实施例3所述的方法的各步骤。
本领域普通技术人员可以意识到，本文所描述的实施例中的各示例性单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件形式来实现，取决于技术方案的特定应用和设计约束条件。专业技术人员可以针对特定的应用选择不同的方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
如果以计算机软件的形式来实现所述功能并作为独立的产品销售或使用时，则在一定程度上可认为本发明的技术方案的全部或部分(例如对现有技术做出贡献的部分)是以计算机软件产品的形式体现的。该计算机软件产品通常存储在计算机可读取的非易失性存储介质中，包括若干指令用以使得计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本发明各实施例方法的全部或部分步骤。而前述的存储介质包括U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random  Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。
实用性
本发明实施例的多媒体资源的质量评估方法和装置，能够更准确地刻画多媒体资源的质量并且具有很强的可操作性，另外，在利用本发明实施例的多媒体资源的质量评估方法评估了多媒体资源的质量之后，可以吸收线上结果的反馈来进行多媒体资源的动态优化排序和推荐，可以给出最终的多媒体资源的排序和推荐结果。

Claims (8)

  1. 一种多媒体资源的质量评估方法，其特征在于，包括：
    根据用于刻画所述多媒体资源的用户行为的指标数据，确定所述多媒体资源的累积分布向量；
    根据所述累积分布向量，确定所述多媒体资源的上边界向量和下边界向量；以及
    根据所述累积分布向量、所述上边界向量和所述下边界向量，确定所述多媒体资源的质量分数。
  2. 根据权利要求1所述的质量评估方法，其特征在于，所述根据用于刻画所述多媒体资源的用户行为的指标数据，确定所述多媒体资源的累积分布向量，包括：
    将一类用户行为的指标数据划分为多个组；
    统计每个组所包括的该类用户行为的指标数据的个数；以及
    将每个组对应的个数构成的向量确定为所述多媒体资源的该类用户行为的累积分布向量。
  3. 根据权利要求2所述的质量评估方法，其特征在于，所述将一类用户行为的指标数据划分为多个组，包括：
    获取该类用户行为的指标数据D的最大值max(D)和最小值min(D)；
    将
    Figure PCTCN2016099358-appb-100001
    确定为分割区间，其中，n为组的个数；以及
    将区间max(D)-min(D)划分为n个组。
  4. 根据权利要求1至3中任一项所述的质量评估方法，其特征在于，所述根据所述累积分布向量、所述上边界向量和所述下边界向量，确定所述多媒体资源的质量分数，包括：利用下式1计算所述质量分数，
    Figure PCTCN2016099358-appb-100002
    其中，Score表示所述质量分数，Dis tan ceTOTOP表示所述累积分布向量 到所述上边界向量的距离，Dis tan ceBetween表示所述上边界向量到所述下边界向量的距离。
  5. 一种多媒体资源的质量评估装置，其特征在于，包括：
    第一确定单元，用于根据用于刻画所述多媒体资源的用户行为的指标数据，确定所述多媒体资源的累积分布向量；
    第二确定单元，与所述第一确定单元连接，用于根据所述累积分布向量，确定所述多媒体资源的上边界向量和下边界向量；以及
    第三确定单元，与所述第一确定单元和所述第二确定单元连接，用于根据所述累积分布向量、所述上边界向量和所述下边界向量，确定所述多媒体资源的质量分数。
  6. 根据权利要求5所述的质量评估装置，其特征在于，所述第一确定单元包括：
    划分子单元，用于将一类用户行为的指标数据划分为多个组；
    统计子单元，与所述划分子单元连接，用于统计每个组所包括的该类用户行为的指标数据的个数；以及
    确定子单元，与所述统计子单元连接，用于将每个组对应的个数构成的向量确定为所述多媒体资源的该类用户行为的累积分布向量。
  7. 根据权利要求6所述的质量评估装置，其特征在于，所述划分子单元包括：
    获取模块，用于获取该类用户行为的指标数据D的最大值max(D)和最小值min(D)；
    确定模块，与所述获取模块连接，用于将
    Figure PCTCN2016099358-appb-100003
    确定为分割区间，其中，n为组的个数；以及
    划分模块，与所述确定模块连接，用于将区间max(D)-min(D)划分为n个 组。
  8. 根据权利要求5至7中任一项所述的质量评估装置，其特征在于，所述第三确定单元具体用于利用下式1计算所述质量分数，
    Figure PCTCN2016099358-appb-100004
    其中，Score表示所述质量分数，Dis tan ceTOTOP表示所述累积分布向量到所述上边界向量的距离，Dis tan ceBetween表示所述上边界向量到所述下边界向量的距离。
PCT/CN2016/099358 2016-03-18 2016-09-19 多媒体资源的质量评估方法和装置 WO2017156994A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/758,836 US10762122B2 (en) 2016-03-18 2016-09-19 Method and device for assessing quality of multimedia resource
EP16894153.2A EP3346396A4 (en) 2016-03-18 2016-09-19 Multimedia resource quality assessment method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610159190.0 2016-03-18
CN201610159190.0A CN105843876B (zh) 2016-03-18 2016-03-18 多媒体资源的质量评估方法和装置

Publications (1)

Publication Number Publication Date
WO2017156994A1 true WO2017156994A1 (zh) 2017-09-21

Family

ID=56587563

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/099358 WO2017156994A1 (zh) 2016-03-18 2016-09-19 多媒体资源的质量评估方法和装置

Country Status (5)

Country Link
US (1) US10762122B2 (zh)
EP (1) EP3346396A4 (zh)
CN (1) CN105843876B (zh)
TW (1) TWI629899B (zh)
WO (1) WO2017156994A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105843876B (zh) * 2016-03-18 2020-07-14 阿里巴巴（中国）有限公司 多媒体资源的质量评估方法和装置
CN108200471B (zh) * 2018-01-08 2019-08-16 中国科学技术大学 一种评测加密视频QoE的标准数据集的构建方法
CN108632670B (zh) * 2018-03-15 2021-03-26 北京奇艺世纪科技有限公司 一种视频满意度确定方法及装置
US11176654B2 (en) * 2019-03-27 2021-11-16 Sharif University Of Technology Quality assessment of a video
CN110366043B (zh) * 2019-08-20 2022-02-18 北京字节跳动网络技术有限公司 视频处理方法、装置、电子设备及可读介质
CN113742564A (zh) * 2020-05-29 2021-12-03 北京沃东天骏信息技术有限公司 目标资源的推送方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080059287A1 (en) * 2002-10-03 2008-03-06 Polyphonic Human Media Interface S.L. Method and system for video and film recommendation
CN102740143A (zh) * 2012-07-03 2012-10-17 合一网络技术(北京)有限公司 一种基于用户行为的网络视频榜单生成系统及其方法
CN104182816A (zh) * 2014-07-09 2014-12-03 浙江大学 基于Vague集和改进逼近理想解的电能质量综合评估方法及其应用
CN104506894A (zh) * 2014-12-22 2015-04-08 合一网络技术(北京)有限公司 多媒体资源评估方法及其装置
CN104616215A (zh) * 2015-03-05 2015-05-13 华北电力大学 一种火电厂能效综合评价方法
CN105843876A (zh) * 2016-03-18 2016-08-10 合网络技术（北京）有限公司 多媒体资源的质量评估方法和装置

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8185543B1 (en) * 2004-11-10 2012-05-22 Google Inc. Video image-based querying for video content
US20150052155A1 (en) * 2006-10-26 2015-02-19 Cortica, Ltd. Method and system for ranking multimedia content elements
US20080127280A1 (en) * 2006-11-27 2008-05-29 Shaobo Kuang Method and system for ranking videos / movies or other objects, and inserting commercial advertisements in the objects
US8565228B1 (en) * 2007-03-28 2013-10-22 Control4 Corporation Systems and methods for selecting and ranking video streams
US20100299303A1 (en) * 2009-05-21 2010-11-25 Yahoo! Inc. Automatically Ranking Multimedia Objects Identified in Response to Search Queries
CN101887460A (zh) * 2010-07-14 2010-11-17 北京大学 一种文献质量评估方法及应用
TWI449410B (zh) * 2011-07-29 2014-08-11 Nat Univ Chung Cheng Personalized Sorting Method of Internet Audio and Video Data
US8719854B2 (en) * 2011-10-28 2014-05-06 Google Inc. User viewing data collection for generating media viewing achievements
CN103188236B (zh) * 2011-12-30 2015-12-16 华为技术有限公司 媒体传输质量的评估方法和装置
US8935581B2 (en) * 2012-04-19 2015-01-13 Netflix, Inc. Upstream fault detection
CN103379358B (zh) * 2012-04-23 2015-03-18 华为技术有限公司 评估多媒体质量的方法和装置
US20140074857A1 (en) * 2012-09-07 2014-03-13 International Business Machines Corporation Weighted ranking of video data
CN103870454A (zh) * 2012-12-07 2014-06-18 盛乐信息技术（上海）有限公司 数据推荐方法及系统
US9165069B2 (en) * 2013-03-04 2015-10-20 Facebook, Inc. Ranking videos for a user
US9405775B1 (en) * 2013-03-15 2016-08-02 Google Inc. Ranking videos based on experimental data
CN103209342B (zh) * 2013-04-01 2016-06-01 电子科技大学 一种引入视频流行度和用户兴趣变化的协作过滤推荐方法
US20150095320A1 (en) * 2013-09-27 2015-04-02 Trooclick France Apparatus, systems and methods for scoring the reliability of online information
CN104035982B (zh) * 2014-05-28 2017-10-20 小米科技有限责任公司 多媒体资源推荐方法及装置
CN104123467A (zh) * 2014-07-24 2014-10-29 合肥工业大学 一种基于专家偏好的gra—topsis模型的评价方法
US10180968B2 (en) * 2015-07-23 2019-01-15 Netflix, Inc. Gaussian ranking using matrix factorization

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080059287A1 (en) * 2002-10-03 2008-03-06 Polyphonic Human Media Interface S.L. Method and system for video and film recommendation
CN102740143A (zh) * 2012-07-03 2012-10-17 合一网络技术(北京)有限公司 一种基于用户行为的网络视频榜单生成系统及其方法
CN104182816A (zh) * 2014-07-09 2014-12-03 浙江大学 基于Vague集和改进逼近理想解的电能质量综合评估方法及其应用
CN104506894A (zh) * 2014-12-22 2015-04-08 合一网络技术(北京)有限公司 多媒体资源评估方法及其装置
CN104616215A (zh) * 2015-03-05 2015-05-13 华北电力大学 一种火电厂能效综合评价方法
CN105843876A (zh) * 2016-03-18 2016-08-10 合网络技术（北京）有限公司 多媒体资源的质量评估方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3346396A4 *

Also Published As

Publication number Publication date
TWI629899B (zh) 2018-07-11
EP3346396A1 (en) 2018-07-11
CN105843876A (zh) 2016-08-10
EP3346396A4 (en) 2018-09-26
CN105843876B (zh) 2020-07-14
TW201735654A (zh) 2017-10-01
US20200192930A1 (en) 2020-06-18
US10762122B2 (en) 2020-09-01

Similar Documents

Publication Publication Date Title
WO2017156994A1 (zh) 多媒体资源的质量评估方法和装置
US10277480B2 (en) Method, apparatus, and system for determining a location corresponding to an IP address
WO2018090793A1 (zh) 多媒体推荐方法和装置
US11190821B2 (en) Methods and apparatus for alerting users to media events of interest using social media analysis
WO2020207074A1 (zh) 一种信息推送的方法及设备
US9251406B2 (en) Method and system for detecting users' emotions when experiencing a media program
WO2020048084A1 (zh) 资源推荐方法、装置、计算机设备及计算机可读存储介质
WO2016197774A1 (zh) 一种多媒体数据推送方法、装置以及存储介质
WO2017005054A1 (zh) 视频推荐方法、服务器和存储介质
US10025785B2 (en) Method and system of automatically downloading media content in a preferred network
WO2015139538A1 (zh) 一种视频信息推送方法及装置
WO2020238502A1 (zh) 物品推荐方法及装置、电子设备及存储介质
WO2015081915A1 (zh) 文件推荐方法和装置
US10346496B2 (en) Information category obtaining method and apparatus
CN109753601A (zh) 推荐信息点击率确定方法、装置及电子设备
GB2498762A (en) Computing user traffic at the website based on user actions
CN106156351B (zh) 多媒体资源推荐信息生成方法及装置
US10733244B2 (en) Data retrieval system
CN105956061B (zh) 一种用户间相似度确定方法及装置
WO2022111095A1 (zh) 一种产品推荐方法、装置、计算机存储介质及系统
CN110309361B (zh) 一种视频评分的确定方法、推荐方法、装置及电子设备
CN109271580B (zh) 搜索方法、装置、客户端和搜索引擎
KR101984773B1 (ko) 스파스 토픽들을 위한 분류자 리콜 추정
CN117201590A (zh) 一种资源确定方法、装置、电子设备及存储介质
CN111695030A (zh) 参数获取方法、装置、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE