CN106919841A - A kind of efficient Android malware detection model DroidDet based on rotation forest - Google Patents

Abstract

The invention discloses a kind of efficient Android malware Static Detection model DroidDet based on rotation forest, belong to field of information security technology.By the present invention in that carrying out decompiling to the Apk files in Android application programs with decompiling instrument, and analyze and extract four groups of combinations of feature as the input for building rotation forest classified device model.Advantages of the present invention：1) present invention rotates forest algorithm using integrated learning approach first, each base learner is trained additionally by use PCA principal component analysis and self-service sampling method and using whole training sets, therefore the model that the present invention is created possesses significantly superior Generalization Capability；2) the characteristics of rotation forest algorithm possesses selection " optimal characteristics ", it is to avoid noise data.Verify that the present invention creates the validity of model by ten folding cross-validation methods, the predictablity rate of model proposed by the invention reaches 88.26%, the accuracy rate than the 84.93% of typical support vector machines grader is high by 3.13%.The present invention has broad application prospects in user information safety field.

Description

A kind of efficient Android malware detection model based on rotation forest DroidDet

Technical field

The present invention utilizes PCA PCAs and self-service sampling based on the rotation forest algorithm in machine learning The detection model for being used for Android malware that method Bootstrap builds, belongs to field of information security technology.

Background technology

International Data Corporation (IDC) (International Data Corporation, IDC) was at 2016 the one of the second quarter Show that the Android operation system of Google has become a smart mobile phone operation for possessing 87.6% share in part survey report The leader of System Market, and the IOS of apple turns into second largest market main body with 11.7% share, but the IOS market shares have been stopped Only go up and gradually on a declining curve.Developer's exploitation, release application program and user download Android platform for convenience With install application program, Android platform use increase income strategy and the safety verification in release application program give It is greatly tolerant.Additionally, Android platform has also set up a positive reponse system for showing popular App, from And further attract more clients that most popular App is downloaded from these platforms, in turn, developer's exploitation is encouraged again More attractive App.The thing followed is that Android platform becomes to be becoming increasingly popular.

Everything all has dual character, with the unprecedented success of Android platform, because the following features of Android platform, make One of its target architecture for becoming Malware invasion：1) third-party application shop (unofficial) and website allow hair Cloth and download application program；2) digital certificate for using in the application, should by each Android not by strict control It is that cannot trace back to developer with the signature of program；3) because of the loose policy of checking before issue, make its malware author can Easily to issue cracking programs, wooden horse or Malware be packaged into a normal application.

The invasion of Malware has caused the extensive concern of academia and industry in Android platform.By A Erka A report display that special Lucent is given：Android malware sample size was counted in the second quarter in 2016, with The first half of the year in 2015 is compared to growth twice.Some of them are than more typical and very disruptive Malware such as： Kasandra.Blooks, apparently it is normal and safe, but in fact, it can support the private of free access user People's information, some information (such as contacts list, bank certificate, short message, message registration etc.) stored on the mobile phone attacked Will be uploaded on remote server；SMSTracker, can provide a remote phone Tracking monitoring system to hacker； Spymob.A, is a kind of business monitoring tools, for stealing some personal information for storing in user mobile phone and by these information Upload onto the server；NotCompatible, it is possible to use infected equipment supports the service of anonymity proxy paying Web browser. In general, these Malwares are by stealing the user profile on infected mobile phone (position, electronics as monitored victim Mail, contact person, phone and short message etc.) so as to make some destructive behaviors.

Recently, Android malware detection has constituted an active research field.Researcher proposes all Many solutions are come damaging of avoiding user from being subjected to that Android malware brings.For example, some detection methods are based on content label Name, with known malware signature be compared each application program by they.However, the testing mechanism based on signature can only Detect those Malwares being successfully identified, but the Malware of variation can not be solved or be not successfully identified Malware.Therefore, this conventional art is difficult to keep up with the paces that Malware emerges in large numbers.Additionally, authority is also standing for malice Software detection, but these methods fail to get a desired effect, and concrete reason is as follows：1) excessively application authority is Android's It is common in manifest files；2) many authorities of application are frequently not that application program is used but needed for advertisement bag in itself Want；3) request permissions is not the necessary condition that Malware is performed.These methods belong to static analysis, as the term suggests it Application program is not performed during Malware is detected.

Except above mentioned Static Analysis Method, dynamic analysis such as droidbox instruments are by extracting IP address and moving The behavior monitor in real time application program of state analysis Android application programs is also a kind of important detection approach of Malware.Separately Outward, CPU, power supply are called using, system, these dynamic behaviours such as internal memory service condition are also commonly used for detecting Malware. Although dynamic behaviour of the dynamic analysis by detecting application program can provide one and more fully detect, in fact-finding process In environmental structure and manpower interference etc. cost be all extremely expensive.Importantly, dynamic analysis is compared with static analysis It is increasingly complex, and the problems such as often cause accuracy rate relatively low because of undertrained.

As far as we know, up to the present the present invention is to detect that Android malice is soft using rotation forest algorithm first Part.Rotation forest algorithm is a kind of new Integrated Algorithm, and the otherness and precision between base grader in Integrated Algorithm are It is two vital factors for influenceing algorithm performance.That is an integrated classifier is wanted to obtain satisfactory result, its base Grader application has precision higher, while the otherness between each base grader is also big as far as possible.

The present invention by Static Analysis Method, using various features combination it is quick, efficiently, low cost and with multi-angle from More features (being possible to characterize application program malicious act) are extracted in each application program.We pass through True Data collection (2136 samples) verifies the validity of the invention with reference to ten folding cross-validation methods, test result indicate that, the invention can be real Existing accuracy rate is up to 88.26%.It is believed that the invention is a kind of reliable method for protecting mobile subscriber soft from malice The destruction of part.In order that our inventive method has more convincingness, the present invention is created into mould under equal experiment condition Type is compared with SVMs (SVM).Comparative result shows that the performance that the invention creates rotation forest model is substantially better than SVM models.

The content of the invention

For the problem that presently, there are, it is soft that the present invention proposes a kind of efficient Android malice based on rotation forest Part detection model DroidDet, the new detection model is based on authority, system detectio event, sensitive API and authority rate and ties Close an Android malware detection model of rotation forest algorithm structure in machine learning method.

The present invention is used as the android system security evaluation instrument for automating, it is considered to which Android platform is believed in mobile subscriber Security threat in terms of breath, helps Android user to complete automatic, the comprehensive and efficient detection of conventional application program, and And ensure objectivity and accuracy that the Android application program new to predicts the outcome.

The technical solution for realizing the object of the invention is：A kind of efficient Android malice based on rotation forest is soft Part detection model DroidDet, its detecting step is as follows：

Step 1)：From Android using shop and a Virus Sample storehouse http dedicated for research of foreign countries:// Virusshare.com/ obtains normal Android application programs and each 1068 of malicious application；

Step 2)：Using the APK file of these samples of apktool.jar decompilings, normal software and Malware are respectively selected Take 600 carries out statistical analysis composing training collection as research object, and remaining APK file constitutes test set；

Step 3)：Extract all authorities occurred in malice App and normal App, system event, the API conducts of request Feature, is calculated the frequency of occurrence of each feature and is calculated certain feature and disliked 600 using methods such as TF-IDF or cosine similarities The number of times and the ratio of the frequency occurred in 600 normal softwares occurred in meaning software, select the feature of ratios as structure Used when building forecast model, authority, system detectio event, sensitive API and authority rate are extracted altogether and makees four groups of features, wherein this hair Bright mentioned authority rate is defined as follows：Wherein pr is the authority rate of definition, and pn is the authority number of APK applications Mesh, ss is sizes (unit be MB) of the APK by smali files after decompiling, and the proposition of authority rate is based on following false If the first Malware generally has privilege abuse, the authority of second usual one normal software application is more, corresponding work( Can be also more, smali files are also bigger after decompiling, therefore the authority rate of a software is also a soft detection malice The characteristic feature of part；

Step 4)：Rotate the foundation and inspection of forest model, the present invention by the 3) step the form of characteristic vector is determined Afterwards, the file after 600 normal softwares and 600 decompilings of Malware is scanned for, there is features described above, the dimension Degree is 1, non-existent to be 0, then calculates the authority rate of each APK, by each APK extract a feature for 28 dimensions to Measure, altogether 1400 characteristic vectors；

Step 5)：Realized using matlab language, PCA algorithms and rotation forest algorithm is realized setting up model, to remaining Sample using same method composition test set, carry out model checking.

As an optimal technical scheme of the invention, step 4 of the present invention) described in rotation forest model it is real by the following method It is existing：

Assuming that X=(X₁,X₂,...,X_N)^TA training sample set with d feature is represented, with Y=(Y₁,Y₂,..., Y_N)^TCorresponding sample class label in training sample set X is represented, F represents feature set, { T₁,T₂,...,T_LRepresent L base classification Device；Rotation forest model construction step is as follows：

Step 1)：It is S sub- feature set F by feature set F random divisions_i,j(j=1,2 ..., S), then every sub- property set Comprising about M=d/S feature；

Step 2)：For j=1,2 ..., S makes X_i, j represents training sample set X in corresponding character subset F_i, the sample in j This；To X_i, j carries out self-service sampling, and sampled data output is the 75% of raw data set, and the data sampled out are represented with X' Collection；Feature extraction is carried out to data set X' with PCA.

Step 3)：Repeat step 2) coefficient matrix D can be got_i,j, and sparse spin matrix R can be obtained_iIt is as follows：

According to the order to primitive character collection by the rearrangement matrix, spin matrix is obtainedAccordingly grader T_i's Training set is

In forecast period, for each sample x, ifPresentation class device T_iPrediction x belongs to class y_iPossibility, The confidence level that then x belongs to certain class is：

With the classification μ that confidence level is maximum_max(x) as x classification.

The present invention compared with prior art, its remarkable advantage：

1) high efficiency：Authority, system monitoring event, sensitive API and authority rate etc. are extracted by Static Analysis Technology many Group feature, can be with one Malware of comprehensive characterization, with reference to rotation forest algorithm and principal component analysis in machine learning method Algorithm is so as to efficiently detect Android malware；

2) easily extension：In the environment of Android platform is supported, be can detect for emerging any application；

3) it is intelligent：Accuracy rate when being detected to unknown software for overcoming traditional stationary detection technique is relatively low (i.e. The universality of characteristic of malware is poor), to extract information less and the problems such as extract in feature comprising noise, and compensate for moving State detection technique process is relative complex, consumption resource is more and undertrained causes the defects such as rate of false alarm high.

Brief description of the drawings

Fig. 1 is general frame schematic diagram of the invention.

Fig. 2 is that the character subset number of rotation forest model in the present invention adjusts ginseng process.

Fig. 3 is that base grader number adjusts ginseng process during forest model is rotated in the present invention.

Fig. 4 is the ROC curve of the rotation forest model that the present invention is created.

Fig. 5 is the ROC curve of the SVM models as reference model in the present invention.

Specific embodiment

The present invention is further described through below in conjunction with the accompanying drawings.

Understand that the present invention is mainly two parts work from Fig. 1 general frames of the invention, a part is another feature extraction Part is grader.In characteristic extraction part, by apktool.jar decompiling instruments to the APK texts of App in all training sets Part carries out decompiling, is then analyzed and extracts authority, system and monitor the features such as event, sensitive API and authority rate, to this A little features extracted carry out PCA and process the most strong principal component in standardization and keeping characteristics so as to reach feature so as to avoid The interference of noise in extracted feature, carries out sample interference so as to reach base using the self-service sampling methods of Bootstrap in addition Practise the maximum diversity of device.In grader part, Android malware forecast model is built by rotating dense algorithm.

A kind of efficient Android malware detection model DroidDet based on rotation forest, its detecting step is such as Under：

Step 1)：From Android using shop and a Virus Sample storehouse http dedicated for research of foreign countries:// Virusshare.com/ obtains normal Android application programs and each 1068 of malicious application；

Step 2)：Using the APK file of these samples of apktool.jar decompilings, normal software and Malware are respectively selected Take 600 carries out statistical analysis composing training collection as research object, and remaining APK file constitutes test set；

Step 3)：Extract all authorities occurred in malice App and normal App, system event, the API conducts of request Feature, is calculated the frequency of occurrence of each feature and is calculated certain feature and disliked 600 using methods such as TF-IDF or cosine similarities The number of times and the ratio of the frequency occurred in 600 normal softwares occurred in meaning software, select the feature of ratios as structure Used when building forecast model, authority, system detectio event, sensitive API and authority rate are extracted altogether and makees four groups of features.

Table 1 is the Partial Feature in four groups of features that the present invention is extracted.

Table 1

Wherein, the authority rate mentioned by the present invention is defined as follows：Wherein pr is the authority rate of definition, and pn is should The authority number of APK applications, ss is sizes (unit be MB) of the APK by smali files after decompiling, and authority rate is carried Go out be based on the assumption that, generally there is privilege abuse, second usual one power of normal software application in the first Malware Limit is more, and corresponding function is also more, and smali files are also bigger after decompiling, therefore an authority rate for software It is a characteristic feature for detection Malware；

Step 4)：Rotate the foundation and inspection of forest model, the present invention by the 3) step the form of characteristic vector is determined Afterwards, the file after 600 normal softwares and 600 decompilings of Malware is scanned for, there is features described above, the dimension Degree is 1, non-existent to be 0, then calculates the authority rate of each APK, by each APK extract a feature for 28 dimensions to Measure, altogether 1400 characteristic vectors.Realized using matlab language, PCA algorithms and rotation forest algorithm realized setting up model, Test set is constituted using same method to remaining sample, model checking is carried out, the accuracy for obtaining model inspection is 88.26%.

Table 2 is that the present invention creates the prediction effect of model and the comparing of SVM forecast result of model.

Table 2

Rotation forest model of the present invention, realizes by the following method：

Assuming that X=(X₁,X₂,...,X_N)^TA training sample set with d feature is represented, with Y=(Y₁,Y₂,..., Y_N)^TCorresponding sample class label in training sample set X is represented, F represents feature set, { T₁,T₂,...,T_LRepresent L base classification Device.Wherein base grader number situation as shown in figure 3, during as L=23 accuracy rate close to 90%, therefore the present invention is used Base grader number obtains L=23, and rotation forest model construction step is as follows：

Step 1)：It is S sub- feature set F by feature set F random divisions_i,j(j=1,2 ..., S), then every sub- property set Comprising about M=d/S feature, specific feature set selection situation as shown in Fig. 2 it can be seen from Fig. 2, when character subset number is Less, consider the character subset number of situations such as calculating cost and accuracy rate present invention use is difference when 4,7,14,28 7；

Step 2)：For j=1,2 ..., S makes X_i,jRepresent training sample set X in corresponding character subset F_i,jIn sample This；To X_i,jSelf-service sampling is carried out, sampled data output is the 75% of raw data set, and the data set sampled out is represented with X'； Feature extraction is carried out to X' with PCA.

Step 3)：Repeat step 2 can get coefficient matrix D_i,j, and sparse spin matrix R can be obtained_iIt is as follows：

According to the order to primitive character collection by this matrix of rearrangement, spin matrix can be obtainedAccordingly classify Device T_iTraining set be

In forecast period, for each sample x, ifPresentation class device T_iPrediction x belongs to class y_iPossibility, The confidence level that then x belongs to certain class is：

With the classification μ that confidence level is maximum_max(x) as x classification.

Confidence evaluation criteria：

In order to assess high estimated performance of the Random Forest model for being created of the invention to Android malware, we The evaluation criteria of introducing is respectively：Susceptibility (Sensitivity), precision (Precision), accuracy rate (Accuracy), horse Repair this relative coefficient (Matthews Correlation Coefficient, MCC) and TG-AUC (Area under Curve, AUC), they are defined as follows：

Wherein, TP (true Positive)：Real example, is to be classified the positive sample that device is correctly classified；TN(True Negative)：Example is really born, refers to be classified the negative sample that device is correctly classified；FP(False Positive)：Refer to by wrong terrestrial reference It is designated as the negative sample of positive sample；FN(False Negative)：It is labeled erroneously as the positive sample of negative sample.Other AUC and ROC (Receiver Operating Characteristic) curve generally combines and is used to evaluate a two-value classification The quality of device (binary classifier), because ROC curve has individual excellent characteristics：When the positive negative sample in test set When changes in distribution, ROC curve can keep constant.Class imbalance (class often occurs in actual data set Imbalance) phenomenon, i.e. negative sample are many more than positive sample (or opposite), and positive negative sample in test data point Cloth is likely to change over time.By classics in the ROC curve and Fig. 5 of Fig. 4 rotation forest models proposed by the present invention The ROC curve figure of SVM models understand, the present invention be better than SVM models, the AUC of model proposed by the invention up to 88.85%, and The AUC of SVM models is 85.96%, i.e. the present invention improves 2.89% to AUC.

The present invention rotate forest Integrated Algorithm in, by the random division to sample characteristics collection, using eigentransformation plan Slightly (PCA) effectively increases the otherness and accuracy of base grader, to obtain more preferable integrated result.Though So rotation forest algorithm is not yet applied to Android malware detection, but there are the algorithm following features to become One of hypothetic algorithm of malware detection, 1) forest algorithm and its PCA principal component analysis phases are rotated when observational characteristic is more With reference to good result can be reached；2) rotation forest algorithm has preferable predictive ability, even if test set is made an uproar comprising substantial amounts of Sound data；3) rotation forest algorithm not easily leads to over-fitting problem；4) rotation forest algorithm can well process many classification Problem and make continuous prediction；5) the features such as rotation forest algorithm has high accuracy, easily realization and computing cost are small；6) Rotation forest algorithm is by using PCA principal component analysis (Principal Component Analysis, PCA) and self-service sampling Method (Bootstrap) simultaneously trains each base learner using whole training sets, makes its base grader " well different "；7) revolve Turn the characteristics of forest algorithm possesses selection " optimal characteristics " so as to avoid noise data.

Claims (2)

1. a kind of efficient based on the Android malware detection model DroidDet for rotating forest, it is characterised in that：Detection Step is as follows：

Step 1)：Normal Android application programs and each 1068 of malice Android application programs are obtained, as sample；

Step 2)：Using the APK file of apktool.jar decompiling samples, Android application programs and malice Android should 600 are respectively chosen with program carries out statistical analysis composing training collection as research object, and remaining APK file constitutes test set；

Step 3)：Extract it is all in malicious application and normal application occur authorities, system event, request API calculates the frequency of occurrence of each feature and calculates certain feature as feature using methods such as TF-IDF or cosine similarities The number of times and the ratio of the frequency occurred in 600 normal softwares occurred in 600 Malwares, select the spy of ratios Levy and used as when building forecast model, authority, system detectio event, sensitive API and authority rate are extracted altogether and makees four groups of features；

Wherein, authority rate is defined as follows：Wherein pr is the authority rate of definition, and pn is the authority number of APK applications, Ss is sizes of the APK by smali files after decompiling；

Step 4)：The foundation and inspection of forest model are rotated, by step 3) form of characteristic vector is determined after, to 600 File after normal software and 600 decompilings of Malware is scanned for, and there is features described above, and the dimension is 1, no As 0 for existing, then the authority rate of each APK is calculated, each APK is extracted into a characteristic vector for 28 dimensions, altogether 1400 characteristic vectors；

Step 5)：Realized using matlab language, PCA algorithms and rotation forest algorithm is realized setting up model, to remaining sample This carries out model checking using same method composition test set.

2. according to claim 1 a kind of efficient based on the Android malware detection model for rotating forest DroidDet, it is characterised in that：Step 4) described in rotation forest model, realize by the following method：

Assuming that X=(X₁,X₂,...,X_N)^TA training sample set with d feature is represented, with Y=(Y₁,Y₂,...,Y_N)^TTable Show corresponding sample class label in training sample set X, F represents feature set, { T₁,T₂,...,T_LRepresent L base grader；Rotation Forest model construction step is as follows：

Step 1)：It is S sub- feature set F by feature set F random divisions_i,j(j=1,2 ..., S), then every sub- property set include About M=d/S feature；

Step 2)：For j=1,2 ..., S makes X_i,jRepresent training sample set X in corresponding character subset F_i,jIn sample；It is right X_i,jSelf-service sampling is carried out, sampled data output is the 75% of raw data set, and the data set sampled out is represented with X'；Use PCA Feature extraction is carried out to data set X'.

Step 3)：Repeat step 2) coefficient matrix D can be got_i,j, and sparse spin matrix R can be obtained_iIt is as follows：

According to the order to primitive character collection by the rearrangement matrix, spin matrix is obtainedAccordingly grader T_iTraining set For

In forecast period, for each sample x, ifPresentation class device T_iPrediction x belongs to class y_iPossibility, then x category It is in the confidence level of certain class：

${\mathrm{\μ}}_j\left(x\right)=\frac{1}{L}{\mathrm{\Σ}}_{i=1}^L{d}_{i,j}\left({\mathrm{XR}}_i^a\right)$

With the classification μ that confidence level is maximum_max(x) as x classification.