WO2017107571A1 - 一种基于用户管理应用行为的应用质量判别方法及系统 - Google Patents

一种基于用户管理应用行为的应用质量判别方法及系统 Download PDF

Info

Publication number
WO2017107571A1
WO2017107571A1 PCT/CN2016/098293 CN2016098293W WO2017107571A1 WO 2017107571 A1 WO2017107571 A1 WO 2017107571A1 CN 2016098293 W CN2016098293 W CN 2016098293W WO 2017107571 A1 WO2017107571 A1 WO 2017107571A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
behavior
management
user
sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/098293
Other languages
English (en)
French (fr)
Inventor
刘譞哲
黄罡
梅宏
李豁然
陆璇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to US15/746,431 priority Critical patent/US10997637B2/en
Publication of WO2017107571A1 publication Critical patent/WO2017107571A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0282—Rating or review of business operators or products
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques

Definitions

  • the present invention is a method and system for predicting the quality of an application lacking user evaluation based on the application management behavior extraction behavior characteristics of the mobile device user based on the behavior characteristics.
  • the mainstream application market basically measures the application quality by the total number of downloads of the application or the number of people who have downloaded the application.
  • the basic starting point is through the user's selection behavior of the application. To judge the quality of the application. This is the only indicator of evaluation for the applications mentioned above with less evaluation. However, relying solely on such simple indicators may not accurately reflect the quality of the application. For example, some application downloads and total downloads are both high, but this may be due to the success of the marketing work done during the application, the quality of the application itself is not high, and the user is very simple after trying it out. It will be uninstalled directly. At the same time, there may be a type of application.
  • the patent entitled CN101404650B provides a suitable network quality mark according to the application condition currently used by the user, and implements a subdivided network application service. quality.
  • the quality determination method and system of an application (application number: 201110247330.7) reflects the general quality of the application by the measured value of the quality index of the obtained application based on the convergence value of the average value reported by a large number of mobile terminals. Status that reflects the quality of the application.
  • these technologies do not effectively utilize the valuable features of the user's management behavior of the application, so that a more accurate application quality evaluation cannot be obtained.
  • the object of the present invention is to use the regression algorithm to judge the quality of the application by the characteristics of the user management behavior. Using the application with more user evaluations, the regression model training is applied to the management behavior of this part of the application, and the training model is used to predict the quality of the application with less user comments.
  • the user's application management behavior data includes three behaviors: downloading an application, uninstalling an application, and updating an application.
  • the behavior record contains three fields: the user ID that made the behavior, the identity of the application that the behavior manages (that is, the application package name), and the time at which the behavior occurred.
  • the application quality discriminating method based on user management application behavior of the invention comprises four main steps: (1) data preprocessing; (2) behavior feature extraction; (3) model training; and (4) using feature to perform application quality discriminant.
  • the user application management behavior related to the training centralized application is preprocessed as follows: “D” indicates downloading application behavior, “U” indicates uninstalling application behavior, and “P” indicates updating application behavior. Sort all management behaviors between each user and each application, and get a sequence of administrative behaviors in chronological order. For example, the sequence:
  • the sequence indicates that the user first uninstalled the application, then updated it twice, and finally uninstalled the application.
  • the addition of the identifier "S" to the beginning of the original sequence indicates the beginning of the sequence
  • the addition of the identifier "E” at the end indicates the end of the sequence, resulting in the final sequence:
  • the training set of the model is constructed according to the sequence of management behavior after the screening.
  • the behavioral feature extraction of the method will be based on the training set.
  • the characteristics of each application include the following: the number of management behavior sequences owned by the application, and the average length of the management behavior sequence.
  • the characteristics of the application may include: managing a continuous number of consecutive steps (including S and E) in a continuous sequence of management behaviors considering the length of the interval.
  • the number of the features in the application is extracted, and finally the number of the features of each management behavior sequence is obtained, which is the value of the feature (for example, there are 100 management sequences in an application). , one of its characteristics.
  • the total number of features is N.
  • an N+1 dimensional independent variable space is established by using N features.
  • the application in each training set is regarded as a point in the space.
  • the value of each independent variable dimension is the value of the feature corresponding to the dimension, and the value of the dependent variable is the linear rating of the user rating of the application to [0, 1].
  • the model is trained by multi-fold training method, and the optimal model parameters are debugged, and finally a certain regression model is obtained.
  • the management behavior data of each application to be predicted is sorted according to the above steps (1) and (2), and then the model is input, and the output of the model is the discrimination result of the application quality.
  • the application quality discriminating system based on user management application behavior of the present invention includes:
  • the data preprocessor is responsible for data preprocessing of the application management behavior of the user, and sorts all management behaviors between each user and each application in chronological order to obtain a management behavior sequence as a training set;
  • the behavior feature extractor is responsible for extracting features of the user's application management behavior by using the obtained management behavior sequence
  • the training device is responsible for constructing a multi-dimensional independent variable space by using the extracted behavior features, and the application in each training set is regarded as a point in the space, and the value of each independent variable dimension is the value of the feature corresponding to the dimension; Space model training, debugging the optimal model parameters, and finally obtaining a certain regression model;
  • the predictor is responsible for extracting its features through the data preprocessor and the behavior feature extractor, and then inputting the obtained features into the trained regression model, and the output of the model is the discrimination result of the application quality.
  • the invention extracts behavior characteristics from the application management behavior, and uses the extracted behavior characteristics to judge the quality of the application with less user evaluation, and obtains a more accurate application quality evaluation result.
  • This approach provides users with a quicker understanding of the application before downloading the app to make a more appropriate choice, a quicker understanding of the market's feedback from developers and competitors, and more accurate recommendations for app market managers. Has an important meaning.
  • the present invention is the first method to determine the quality of an application by using multiple consecutive user management behaviors and management time intervals.
  • the present invention is also the first to adopt a regression algorithm, which utilizes user management behavior corresponding A method of discriminating by quality.
  • the above characteristics make the invention more effective in removing irrelevant features in the process of discriminating the application quality, and more effectively exerting the quality expression ability of the related features. Therefore, in summary, the present invention can obtain more accurate evaluation results than the prior art.
  • FIG. 1 is a technical architecture diagram of the present invention.
  • Figure 1 is a specific technical architecture diagram.
  • Each behavior is recorded with a row of data, including all of the fields described above, namely the user ID that performed the behavior, the identity of the application that the behavior manages (ie, the application package name), and the time at which the behavior occurred.
  • the required hardware environment is a server with Ubuntu 14.04LTS. All source data and software are deployed on this server.
  • the data preprocessor consists of Python scripts, Ubuntu instructions, and a uniformly controlled script file.
  • the data preprocessor takes the raw text data as input, sorts the data, and then generates the sequence of administrative behaviors described in the previous section.
  • the data preprocessor then filters the data.
  • the purpose of application filtering is to filter out applications that are not well-defined by users, and only retain applications that users show a clear preference. Therefore, the filter performs a binomial distribution probability test for all applications based on the number of evaluations that they like and dislike, and retains the application only if the user's tendency to prove that the application is more obvious with a 95% probability. Then, for the sequence of management behaviors of the retained application, the sequence retains at least two operations except S and E, and the final training set is obtained.
  • the behavior feature extractor is implemented by a Python script to input the training set data generated by the data preprocessor, and to statistically manage the number of features in the behavior sequence.
  • the feature is a management behavior feature that contains at most 3 consecutive management actions (including S, E) while considering the time interval between actions. For the time interval between the two operations, there are four cases of less than 24 hours, 24 to 48 hours, 48 to 72 hours, and 72 hours or more. It is represented by one T every 24 hours and by * for more than 72 hours.
  • S, E management actions
  • SDTTP The first behavior of the sequence begins with downloading, and an update occurs after a period of 48 to 72 hours.
  • the statistical process is performed by means of string matching. After the statistics are finished, the averaging is performed according to the method proposed in the previous section, and the extracted feature statistics result is obtained.
  • the trainer is implemented as a Python script that internally uses the GradientBoostingRegressor class of the scikit-learn library.
  • Scikit-learn is an open source tool library for Python language development. It is mainly used in statistical learning and machine learning.
  • the GradientBoostingRegressor class in scikit-learn is the implementation of scikit-learn algorithm for gradient boosted regression tree. The latter is widely used. A regression algorithm based on the idea of gradient descent.
  • the trainer will perform parameter debugging according to the manually set value range and granularity. The specific process of debugging is: for each set of model parameters, the trainer randomly divides the data into 5 parts, and then performs 5-fold cross-validation.
  • the predictor For each application to be predicted, the predictor first extracts its features through the data preprocessor and the behavior feature extractor, and then inputs the obtained features into the trained model to generate a prediction result with a range of [0, 1]. Is the predicted result of the quality of the application.
  • the present invention is more flexible for specific implementation deployment requirements, and may be deployed in a variety of other manners in addition to the above embodiments.
  • Other hardware forms such as large-scale clusters, cloud servers, etc.
  • system software such as Windows Server, RedHat, etc.
  • Software implementations such as data preprocessors, behavioral feature extractors, trainers, and prediction periods can be written in other programming languages (such as Java, C/C++, R, etc.). Data can also be organized in other feasible ways (such as SQL, MongoDB, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Development Economics (AREA)
  • General Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Finance (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Operations Research (AREA)
  • Game Theory and Decision Science (AREA)
  • Probability & Statistics with Applications (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Algebra (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Human Resources & Organizations (AREA)
  • Psychiatry (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Social Psychology (AREA)

Abstract

一种基于用户管理应用行为的应用质量判别方法及系统。该方法包括：1)对用户的应用管理行为进行数据预处理，得到管理行为序列，作为训练集；2)利用得到的管理行为序列，对用户的应用管理行为进行特征抽取；3)利用抽取的行为特征建立多维自变量空间，每一个训练集中的应用被视作该空间当中的一个点，根据此空间进行模型训练，调试出最优的模型参数，得到一个确定的回归模型；4)将每一个待预测的应用的管理行为数据按照步骤1)和步骤2)进行整理，然后输入该回归模型，模型的输出即为对应用质量的判别结果。本基于用户管理应用行为的应用质量判别方法及系统能够更为有效地去除无关特征和发挥有关特征的质量表达能力，能够获得更为准确的应用质量评价结果。

Description

一种基于用户管理应用行为的应用质量判别方法及系统 技术领域
本发明是一种基于移动设备用户的应用管理行为抽取行为特征，然后基于该行为特征利用回归算法对缺乏用户评价的应用的质量进行预测的方法及系统。
背景技术
随着移动设备(智能手机、平板电脑、智能手表等)的发展，移动设备应用程序(即app，简称应用)的种类、数量都呈现了爆发式增长的趋势。如何能够准确的衡量一个应用的质量好坏成为了一个非常重要的问题。这对于用户在下载应用前快速了解应用从而做出更合适的选择、对于开发者更快速地了解自己和竞争对手的应用在市场中的反馈、以及对于应用市场管理人员提供更准确的推荐都有着非常重要的意义。最为直观且准确的评价标准是应用市场上用户对于应用的评价，包括自然语言评价与打分，其中又以打分最为直观有效。但是，目前的移动应用市场上普遍存在着用户不习惯于评价应用、导致应用的用户评价很少的问题。对于这部分应用，其质量很难通过用户评价进行判别。
除用户评价这种相对直观、准确的评价方式外，主流应用市场基本以应用的下载总次数或者以下载过该应用的人数来衡量应用的好坏，其基本出发点是通过用户对于应用的选择行为来判断应用质量的好坏。对于上述所提及的评价较少的应用，这成为了唯一的评价指标。但是，仅仅依靠这种简单的指标可能并不能真正准确的反映出应用的质量高低。例如，一些应用下载总量与下载总人数均较高，但是这可能是由于应用在上线期间所做的营销工作成功而导致的，应用本身质量并不高，用户在简单尝试使用之后很大部分会直接将其卸载。同时，也可能存在一类应用，它们的下载量与下载人数虽然不是非常多，但是其用户会对其保持使用，并跟随版本的迭代随时更新，这类应用往往是质量较高的。因此，单纯的通过下载量、下载人数这样简单的指标可能并不能得到准确的评价。
该指标基于用户行为的出发点是合理的，但是其存在的一个问题在于浅尝辙止，没有充分挖掘出用户行为中的信息量。例如，如果一个应用的多数用户在下载了该应用后不久就进行了卸载，那么很可能说明该应用质量不高因此大量用户在使用之后选择了放弃；另一方面，如果一个应用存在部分用户在已经卸载了该应用之后又选择了重新安装，那么很有可能说明这个应用质量较好因此用户的确需要它因此重新安装。受此启示，用户对应用的管理行为序列中可能存在很多有价值的特征。如果能够挖掘出有效特征，就可能利用这些特征得出更准 确的应用质量评价。
现有技术中，授权公告号为CN101404650B的专利“一种细分网络应用服务质量的方法和系统”，根据用户当前使用的应用程序情况，提供一个合适的网络质量标记，实现细分网络应用服务质量。此外，“一种应用程序的质量确定方法和系统”(申请号：201110247330.7)通过获取的应用程序的质量指标的衡量值基于大量移动终端上报的平均值的收敛值，体现了应用程序的通用质量状态，能够反映应用程序的质量。但这些技术都没有有效利用用户对应用的管理行为中的有价值的特征，从而不能得出更准确的应用质量评价。
综上所述，需要一种从应用管理行为中抽取行为特征，并利用抽取出的行为特征对用户评价较少的应用的质量进行判断的方法。
发明内容
本发明的目的是通过用户管理行为中的特征，利用回归算法来判断应用质量的好坏。利用含有较多用户评价的应用，针对这部分应用的管理行为进行回归模型训练，用训练得到的模型对用户评论较少的应用进行质量预测。
本发明中，用户的应用管理行为数据当中包括三种行为：下载应用、卸载应用、更新应用。行为记录中包含三个域：进行该行为的用户标识、该行为管理的应用的标识(即应用包名)、以及该行为发生的时间。
本发明的基于用户管理应用行为的应用质量判别方法，包含四个主要步骤：(1)数据预处理；(2)行为特征抽取；(3)模型训练；(4)利用特征进行应用质量判别。
(1)数据预处理：
首先根据阈值、统计分析等方法筛选出可靠的应用进入训练集。然后，对与训练集中应用相关的用户应用管理行为进行如下数据预处理：以“D”表示下载应用行为、“U”表示卸载应用行为、“P”表示更新应用行为。将每个用户与每个应用之间的所有管理行为排序，按时间先后顺序得到一个管理行为序列。例如，序列：
DPPU
该序列表示该用户先对应用进行了一次卸载，然后更新了两次，最后卸载了该应用。此外，在上述原始序列的头部添加标识“S”表示序列开始，在尾部添加标识“E”表示序列结束，得到最终的序列：
SDPPUE
为保证方法尽可能准确，需要进一步对序列数据进行筛选。最终根据筛选之后的管理行为序列构成模型的训练集。本方法的行为特征抽取工作将基于该训练集进行。
(2)行为特征抽取：
每个应用的特征包括以下两项：该应用拥有的管理行为序列数、管理行为序列的平均长度。除此两项之外，应用的特征还可以包括：管理行为序列中至多连续若干步(包括S和E在内)考虑时间间隔长短的连续管理行为。
对于每一种特征，抽取得到该应用中该种特征的个数，最后得到平均每一个管理行为序列的该种特征个数，即为此种特征的值(例如：某应用共有100个管理序列，其一个特征的。出现个数为200，则最终该特征的值为200/100＝2)。特征总数为N。
(3)模型训练：
行为特征抽取完成后，利用N个特征建立一个N+1维自变量空间。每一个训练集中的应用被视作空间当中的一个点，每个自变量维度的值即为该维度对应的特征的值，因变量的值为该应用的用户评分线性规约至[0,1]区间后的值(0为最低、1为最高)。根据此空间，用多折训练的方法对模型进行训练，调试出最优的模型参数，最终得到一个确定的回归模型。
(4)利用特征进行应用质量判别：
利用训练得到的回归模型，将每一个待预测的应用的管理行为数据按照上述步骤(1)、(2)的方式进行整理，然后输入模型，模型的输出即为对应用质量的判别结果。
本发明的基于用户管理应用行为的应用质量判别系统，其包括：
数据预处理器，负责对用户的应用管理行为进行数据预处理，将每个用户与每个应用之间的所有管理行为按照时间先后顺序进行排序，得到管理行为序列，作为训练集；
行为特征抽取器，负责利用得到的管理行为序列，对用户的应用管理行为进行特征抽取；
训练器，负责利用抽取的行为特征建立多维自变量空间，每一个训练集中的应用被视作该空间当中的一个点，每个自变量维度的值即为该维度对应的特征的值；根据此空间进行模型训练，调试出最优的模型参数，最终得到一个确定的回归模型；
预测器，负责通过数据预处理器和行为特征抽取器抽取其特征，然后将得到的特征输入训练好的回归模型，模型的输出即为对应用质量的判别结果。
本发明从应用管理行为中抽取行为特征，并利用抽取出的行为特征对用户评价较少的应用的质量进行判断，得出更准确的应用质量评价结果。该方法对于用户在下载应用前快速了解应用从而做出更合适的选择、对于开发者更快速地了解自己和竞争对手的应用在市场中的反馈、以及对于应用市场管理人员提供更准确的推荐等具有重要的意义。
与现有技术相比，本发明是首个采用多个连续用户管理行为、及管理行为间的时间间隔来判别应用质量好坏的方法。同时，本发明也是首个采用回归算法，利用用户管理行为对应 用质量进行判别的方法。以上特点使得本发明在判别应用质量的过程中可以更为有效地去除无关特征，并且更加有效地发挥有关特征的质量表达能力。因此，总结而言，本发明相较于现有技术可以获得更为准确的评价结果。
附图说明
图1为本发明的技术架构图。
具体实施方式
下面给出一个基于原始的评价数据进行应用质量判别的具体部署，包括数据描述、硬件部署、具体软件实现。图1为具体的技术架构图。
1、数据描述
本部署使用的数据来自于一个大型的Android移动应用市场，均为文本文件，包括以下两部分：
(1)应用的管理行为数据。每个行为用一行数据来记录，包括前文描述的全部域，即进行该行为的用户标识、该行为管理的应用的标识(即应用包名)、以及该行为发生的时间。
(2)用户评价数据。用户对应用的评价简单分为喜欢和不喜欢两种，因此应用的用户评分按照“喜欢的评价数/总评价数”的方式计算，取值范围为[0,1]。每个应用用一行数据来记录。
2、硬件部署
所需硬件环境为一台装有Ubuntu 14.04LTS的服务器。所有源数据与软件均部署在该服务器上。
3、软件实现
软件部署由以下部分组成：
(1)数据预处理器
数据预处理器由Python脚本、Ubuntu指令及一个统一控制的脚本文件构成。数据预处理器以原始的文本数据为输入，对数据进行排序，然后生成上节中所描述的管理行为序列。
然后，数据预处理器对数据进行筛选。应用筛选的目的是希望过滤掉用户评价不明确的应用，只保留用户展现出了明显的喜好倾向的应用。因此，过滤器对所有的应用根据其喜欢、不喜欢的评价个数进行二项分布概率测试，只有能够以95％概率证明该应用的用户倾向较为明显时才保留该应用。然后，对于被保留应用的管理行为序列，支保留除S、E外至少包含两个操作的序列，得到最后的训练集。
(2)行为特征抽取器
行为特征抽取器由Python脚本实现，以以数据预处理器生成的训练集数据为输入，从统计管理行为序列中的特征个数。
本部署中，采用的特征为至多包含3个连续管理行为(包含S、E)、同时考虑行为之间的时间间隔的管理行为特征。对于两个操作之间的时间间隔，分为不足24小时、24～48小时、48～72小时、72小时以上共四种情况。每24小时用一个T表示，超过72小时用*表示。以下为若干个符合上述定义的特征示例：
D：单次下载
DU：下载之后卸载，时间间隔不超过24小时
SDTTP：序列开始的第一个行为为下载，经过48～72小时范围内的一段时间后进行了一次更新
D*PE：下载之后经过了超过72小时的一段时间进行了卸载，然后序列结束
统计过程通过字符串匹配的方式进行，统计结束后按照上节中提出的方法进行求平均，得到提取后的特征统计结果。
(3)训练器。训练器以Python脚本实现，内部使用了scikit-learn库的GradientBoostingRegressor类。scikit-learn是一个机遇Python语言开发的开源工具库，主要用于统计学习、机器学习领域；scikit-learn中的GradientBoostingRegressor类是scikit-learn对gradient boosted regression tree算法的实现，后者是一个广泛使用的基于梯度下降思想的回归算法。训练器会根据人工预先设定好的取值范围及粒度进行参数调试。调试具体过程为：针对每一组模型参数，训练器将数据随机等分为5份，然后进行5折交叉验证。每一次实验以4份数据为训练集，1份数据为测试集，得到当前参数下预测结果与实际结果之间的kendall's tau值。5组实验全部完成后，5组的kendall's tau值的平均值即为本组参数的最终结果。尝试所有参数后，选取结果最好的一组，作为训练的最终结果，确定模型。
(4)预测器。对于每一个待预测的应用，预测器首先通过数据预处理器和行为特征抽取器抽取其特征，然后将得到的特征输入训练好的模型，产生一个取值范围为[0,1]的预测结果，即为对该应用质量的预测结果。
本发明对于具体的实施部署要求较为灵活，除上述实施方式外，也可以采取多种其它方式进行部署。硬件部署方面可采用其它硬件形式(如大规模集群、云服务器等)，亦可以采用其它适用于硬件形式的系统软件(如Windows Server、RedHat等)。数据预处理器、行为特征抽取器、训练器和预测期等软件实现，可采用其它编程语言编写(如Java、C/C++、R等)。 数据也可采用其它可行的方式进行组织(如SQL、MongoDB等)。
以上实施例仅用以说明本发明的技术方案而非对其进行限制，本领域的普通技术人员可以对本发明的技术方案进行修改或者等同替换，而不脱离本发明的精神和范围，本发明的保护范围应以权利要求书所述为准。

Claims (10)

  1. 一种基于用户管理应用行为的应用质量判别方法，其特征在于，包括如下步骤：
    1)对用户的应用管理行为进行数据预处理，将每个用户与每个应用之间的所有管理行为按照时间先后顺序进行排序，得到管理行为序列，作为训练集；
    2)利用得到的管理行为序列，对用户的应用管理行为进行特征抽取；
    3)利用抽取的行为特征建立多维自变量空间，每一个训练集中的应用被视作该空间当中的一个点，每个自变量维度的值即为该维度对应的特征的值；根据此空间进行模型训练，调试出最优的模型参数，最终得到一个确定的回归模型；
    4)利用训练得到的回归模型，将每一个待预测的应用的管理行为数据按照步骤1)和步骤2)进行整理，然后输入该回归模型，模型的输出即为对应用质量的判别结果。
  2. 如权利要求1所述的方法，其特征在于，步骤1)所述用户的应用管理行为包括三种：下载应用、卸载应用、更新应用；行为记录中包含三个域：进行该行为的用户标识、该行为管理的应用的标识、该行为发生的时间。
  3. 如权利要求2所述的方法，其特征在于，步骤1)所述管理行为序列以“D”表示下载应用行为，以“U”表示卸载应用行为，以“P”表示更新应用行为，并在序列的头部添加标识“S”表示序列开始，在尾部添加标识“E”表示序列结束。
  4. 如权利要求3所述的方法，其特征在于，步骤1)还对管理行为序列进行筛选，过滤掉用户评价不明确的应用，只保留用户展现出明显喜好倾向的应用。
  5. 如权利要求4所述的方法，其特征在于，所述进行筛选的方法是：通过过滤器对所有的应用根据其喜欢、不喜欢的评价个数进行二项分布概率测试，只有能够以95％概率证明该应用的用户倾向较为明显时才保留该应用；然后对于被保留应用的管理行为序列，只保留除S、E外至少包含两个操作的序列。
  6. 如权利要求1所述的方法，其特征在于：步骤2)中，每个应用的行为特征包括：该应用拥有的管理行为序列数、管理行为序列的平均长度、管理行为序列中至多连续若干步考虑时间间隔长短的连续管理行为。
  7. 如权利要求6所述的方法，其特征在于：步骤2)对于每一种行为特征，抽取得到该应用中该种特征的个数，最后得到平均每一个管理行为序列的该种特征个数，即为此种特征的值。
  8. 如权利要求1所述的方法，其特征在于：步骤3)所述多维自变量空间中，因变量的值为应用的用户评分线性规约至[0,1]区间后的值。
  9. 如权利要求1所述的方法，其特征在于：步骤3)采用多折训练的方法对模型进行训练。
  10. 一种基于用户管理应用行为的应用质量判别系统，其特征在于，包括：
    数据预处理器，负责对用户的应用管理行为进行数据预处理，将每个用户与每个应用之间的所有管理行为按照时间先后顺序进行排序，得到管理行为序列，作为训练集；
    行为特征抽取器，负责利用得到的管理行为序列，对用户的应用管理行为进行特征抽取；
    训练器，负责利用抽取的行为特征建立多维自变量空间，每一个训练集中的应用被视作该空间当中的一个点，每个自变量维度的值即为该维度对应的特征的值；根据此空间进行模型训练，调试出最优的模型参数，最终得到一个确定的回归模型；
    预测器，负责通过数据预处理器和行为特征抽取器抽取其特征，然后将得到的特征输入训练好的回归模型，模型的输出即为对应用质量的判别结果。
PCT/CN2016/098293 2015-12-24 2016-09-07 一种基于用户管理应用行为的应用质量判别方法及系统 Ceased WO2017107571A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/746,431 US10997637B2 (en) 2015-12-24 2016-09-07 Method and system for determining quality of application based on user behaviors of application management

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510983204.6A CN105630957B (zh) 2015-12-24 2015-12-24 一种基于用户管理应用行为的应用质量判别方法及系统
CN201510983204.6 2015-12-24

Publications (1)

Publication Number Publication Date
WO2017107571A1 true WO2017107571A1 (zh) 2017-06-29

Family

ID=56045890

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/098293 Ceased WO2017107571A1 (zh) 2015-12-24 2016-09-07 一种基于用户管理应用行为的应用质量判别方法及系统

Country Status (3)

Country Link
US (1) US10997637B2 (zh)
CN (1) CN105630957B (zh)
WO (1) WO2017107571A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111143743A (zh) * 2019-12-26 2020-05-12 杭州迪普科技股份有限公司 一种自动扩充应用识别库的方法及装置

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105630957B (zh) * 2015-12-24 2019-05-21 北京大学 一种基于用户管理应用行为的应用质量判别方法及系统
CN107563679B (zh) * 2017-10-17 2020-05-22 广东小天才科技有限公司 一种应用软件的检测方法及服务设备
CN107943535B (zh) * 2017-11-14 2020-10-09 Oppo广东移动通信有限公司 应用清理方法、装置、存储介质及电子设备
CN107832429A (zh) * 2017-11-14 2018-03-23 广州供电局有限公司 审计数据处理方法和系统
TWI645350B (zh) * 2017-11-24 2018-12-21 財團法人工業技術研究院 決策因素分析裝置與決策因素分析方法
CN107967330A (zh) * 2017-11-27 2018-04-27 中山大学 基于下载行为数据及向量表征学习的移动应用推荐方法
CN110322334A (zh) * 2018-03-29 2019-10-11 上海麦子资产管理集团有限公司 信用评级方法及装置、计算机可读存储介质、终端
CN108763460A (zh) * 2018-05-28 2018-11-06 成都优易数据有限公司 一种基于sql的机器学习方法和系统
US11928451B2 (en) 2021-10-13 2024-03-12 International Business Machines Corporation Dynamic management of application uninstallation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102567477A (zh) * 2011-06-16 2012-07-11 北京亿赞普网络技术有限公司 网站价值评估方法及装置
CN102663617A (zh) * 2012-03-20 2012-09-12 亿赞普(北京)科技有限公司 一种广告的点击率预测方法及系统
CN104599159A (zh) * 2015-02-06 2015-05-06 浪潮集团有限公司 一种基于支持向量机原理的用户行为识别方法
CN105630957A (zh) * 2015-12-24 2016-06-01 北京大学 一种基于用户管理应用行为的应用质量判别方法及系统

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6714925B1 (en) * 1999-05-01 2004-03-30 Barnhill Technologies, Llc System for identifying patterns in biological data using a distributed network
US7080057B2 (en) * 2000-08-03 2006-07-18 Unicru, Inc. Electronic employee selection systems and methods
US7451065B2 (en) * 2002-03-11 2008-11-11 International Business Machines Corporation Method for constructing segmentation-based predictive models
US9818136B1 (en) * 2003-02-05 2017-11-14 Steven M. Hoffberg System and method for determining contingent relevance
US8170841B2 (en) * 2004-04-16 2012-05-01 Knowledgebase Marketing, Inc. Predictive model validation
US7730003B2 (en) * 2004-04-16 2010-06-01 Fortelligent, Inc. Predictive model augmentation by variable transformation
US7499897B2 (en) * 2004-04-16 2009-03-03 Fortelligent, Inc. Predictive model variable management
US8165853B2 (en) * 2004-04-16 2012-04-24 Knowledgebase Marketing, Inc. Dimension reduction in predictive model development
US7562058B2 (en) * 2004-04-16 2009-07-14 Fortelligent, Inc. Predictive model management using a re-entrant process
US7933762B2 (en) * 2004-04-16 2011-04-26 Fortelligent, Inc. Predictive model generation
US7725300B2 (en) * 2004-04-16 2010-05-25 Fortelligent, Inc. Target profiling in predictive modeling
US20050234761A1 (en) * 2004-04-16 2005-10-20 Pinto Stephen K Predictive model development
CN101377739A (zh) * 2007-08-31 2009-03-04 兰雨晴 基础软件平台的质量评估方法
US20090094090A1 (en) * 2007-10-05 2009-04-09 Scott Dow Lean staffing methodology
US8417715B1 (en) * 2007-12-19 2013-04-09 Tilmann Bruckhaus Platform independent plug-in methods and systems for data mining and analytics
CN101261604B (zh) * 2008-04-09 2010-09-29 中兴通讯股份有限公司 一种软件质量评价装置及软件质量评价的定量分析方法
US8788655B2 (en) * 2008-12-19 2014-07-22 Openpeak Inc. Systems for accepting and approving applications and methods of operation of same
CN101448025A (zh) * 2008-12-19 2009-06-03 北京大学 度量服务质量的方法
WO2012148293A1 (en) * 2011-04-28 2012-11-01 Google Inc. Using feedback reports to determine performance of an application in a geographic location
US9110452B2 (en) * 2011-09-19 2015-08-18 Fisher-Rosemount Systems, Inc. Inferential process modeling, quality prediction and fault detection using multi-stage data segregation
US8763131B2 (en) * 2012-05-22 2014-06-24 Verizon Patent And Licensing Inc. Mobile application security score calculation
CN102880501B (zh) * 2012-07-24 2016-05-25 北京奇虎科技有限公司 应用推荐的实现方法、装置和系统
WO2014099127A1 (en) * 2012-12-20 2014-06-26 Aha! Software LLC Dynamic model data facility and automated operational model building and usage
US20140351046A1 (en) * 2013-05-21 2014-11-27 IgnitionOne, Inc, System and Method for Predicting an Outcome By a User in a Single Score
US9702716B2 (en) * 2013-08-28 2017-07-11 Verizon Telematics Inc. Traffic score determination
US9852448B2 (en) * 2013-09-19 2017-12-26 Samsung Electronics Co., Ltd. Identifying gaps in search results
US9430476B2 (en) * 2014-03-13 2016-08-30 GM Global Technology Operations LLC Method and apparatus of user recommendation system for in-vehicle apps
US20150262198A1 (en) * 2014-03-13 2015-09-17 GM Global Technology Operations LLC Method and apparatus of tracking and predicting usage trend of in-vehicle apps
US9432395B2 (en) * 2014-04-28 2016-08-30 Quixey, Inc. Application spam detector
US10324985B2 (en) * 2014-12-19 2019-06-18 Samsung Electronics Co., Ltd. Device-specific search results
US10349205B2 (en) * 2014-12-31 2019-07-09 Samsung Electronics Co., Ltd. Location-based application search
US9785535B2 (en) * 2015-08-10 2017-10-10 Accenture Global Services Limited Multi-data analysis based proactive defect detection and resolution
US10332245B1 (en) * 2018-12-11 2019-06-25 Capital One Services, Llc Systems and methods for quality assurance of image recognition model
US11521132B2 (en) * 2019-02-12 2022-12-06 Nec Corporation Method and system for adaptive online meta learning from data streams
DE112020001253T5 (de) * 2019-03-15 2021-12-09 Nvidia Corporation Techniken zum Trainieren eines neuronalen Netzes unter Verwendung von Transformationen

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102567477A (zh) * 2011-06-16 2012-07-11 北京亿赞普网络技术有限公司 网站价值评估方法及装置
CN102663617A (zh) * 2012-03-20 2012-09-12 亿赞普(北京)科技有限公司 一种广告的点击率预测方法及系统
CN104599159A (zh) * 2015-02-06 2015-05-06 浪潮集团有限公司 一种基于支持向量机原理的用户行为识别方法
CN105630957A (zh) * 2015-12-24 2016-06-01 北京大学 一种基于用户管理应用行为的应用质量判别方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111143743A (zh) * 2019-12-26 2020-05-12 杭州迪普科技股份有限公司 一种自动扩充应用识别库的方法及装置
CN111143743B (zh) * 2019-12-26 2023-09-26 杭州迪普科技股份有限公司 一种自动扩充应用识别库的方法及装置

Also Published As

Publication number Publication date
US10997637B2 (en) 2021-05-04
US20180285947A1 (en) 2018-10-04
CN105630957A (zh) 2016-06-01
CN105630957B (zh) 2019-05-21

Similar Documents

Publication Publication Date Title
WO2017107571A1 (zh) 一种基于用户管理应用行为的应用质量判别方法及系统
US10275601B2 (en) Flaw attribution and correlation
US9734043B2 (en) Test selection
CN109918437A (zh) 分布式数据处理方法、装置及数据资产管理系统
CN113626241A (zh) 应用程序的异常处理方法、装置、设备及存储介质
CN106909510A (zh) 一种获取测试用例的方法以及服务器
CN105302706B (zh) 应用程序测试方法和装置
CN108846695A (zh) 终端更换周期的预测方法及装置
US8180762B2 (en) Database tuning methods
CN105630797B (zh) 数据处理方法及系统
CN110298007A (zh) 用户行为统计方法、装置、电子设备及计算机可读存储介质
CN111258876A (zh) 一种微服务架构下的精确回归测试方法及装置
KR20210055934A (ko) 기계 학습 모델을 개발하기 위한 자가학습 시스템
WO2017023299A1 (en) Composing future tests
CN108681505B (zh) 一种基于决策树的测试用例排序方法和装置
CN107430590B (zh) 用于数据比较的系统和方法
CN117714159A (zh) 一种基于行为树的自动化渗透测试方法及设备
WO2016032531A1 (en) Improvement message based on element score
CN109063040B (zh) 客户端程序数据采集方法及系统
Omori et al. Comparing concept drift detection with process mining tools
Guerrouj et al. Investigating the android apps' success: An empirical study
CN107431646A (zh) 负载测试
CN117081666B (zh) 故障预测方法、装置、电子设备、存储介质及程序产品
CN112035126A (zh) 用于灰度发布的处理方法和装置、电子设备和存储介质
US10740119B2 (en) Identifying a common action flow

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16877382

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15746431

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16877382

Country of ref document: EP

Kind code of ref document: A1