CN115795467A

CN115795467A - Intelligent evaluation method for computer software bugs

Info

Publication number: CN115795467A
Application number: CN202111065320.1A
Authority: CN
Inventors: 梁小毅; 梁慎恒
Original assignee: HSBC Software Development Guangdong Ltd
Current assignee: HSBC Software Development Guangdong Ltd
Priority date: 2021-09-12
Filing date: 2021-09-12
Publication date: 2023-03-14

Abstract

The invention provides a method for intelligently evaluating computer software bugs. The method for intelligently evaluating the computer software vulnerability comprises the following steps: the system comprises a data collection module, a planning learning module, a system maintenance module and a result updating module; the output end of the data collection module is connected with the input end of the planning learning module, the output end of the planning learning module is connected with the input end of the result updating module, the output end of the system maintenance module is connected with the input end of the data collection module, and the output end of the system maintenance module is connected with the input end of the planning learning module. According to the method for intelligently evaluating the computer software vulnerability, the possibility of vulnerability existence of similar code segments is automatically judged by combining an SAST tool with an intelligent learning method, the use of the SAST tool is optimized, and safety personnel/code examiner are assisted to evaluate the code safety.

Description

Intelligent evaluation method for computer software bugs

Technical Field

The invention relates to the field of internet security, in particular to a method for intelligently evaluating computer software bugs.

Background

According to the IT research and counselor Gartner statistics data, 75% of the hacking attacks occur in the application layer. While statistics by NIST show that 92% of vulnerabilities belong to the application layer and not the network layer. More and more safety product manufacturers also consider the whole process of paying attention to software development, and integrate safety detection and monitoring into each stage of demand analysis, summary design, detailed design, coding, testing and the like so as to comprehensively ensure application safety.

The detection of the application security is currently mostly implemented by means of tests. The test is largely divided into a black box test and a white box test. The black box test generally uses a penetration method, which still has obvious defects of the black box test itself, needs a large number of test cases for coverage, and still cannot ensure whether the software still has risks after the test is completed. At present, source code scanning in white box testing is increasingly becoming a popular technology, and code scanning is carried out on software by using a source code scanning product, so that potential risks can be found out, the software is detected from the inside, the coverage rate of testing can be effectively ensured, and modern code bugs are developed more early from a deeper level.

Static Application Security Test (SAST), also known as static analysis, is a test method that finds security holes that easily allow an organization's applications to be attacked by analyzing source code. The SAST scans the application before compiling the code. Since the number of developers is much greater than the number of security personnel, the amount of code generated by their day-to-day work is not easy for security personnel or teams to perform code review even if resources are found to perform code review for a small fraction of applications. One key advantage of the SAST tool is the ability to analyze 100% of the codebase. Moreover, it is much faster than manual security code review. These tools take only a few minutes to scan millions of lines of code. The SAST tool can automatically identify key vulnerabilities, such as buffer overflow, SQL injection, cross-site scripting, and the like.

However, the tool is always a tool. The tool also makes errors, which we call false positives (false positives). False positives occur when the problem reported by the tool is not a real problem at all. In contrast, when the tool fails to detect the due error, a false negative is generated. The simplest reasons for the large number of false alarms are: the tool cannot be analyzed like a human because it lacks the context in which the application is located; therefore, it must be done with caution in an attempt to draw the user's attention to those potential problems. On the other hand, code similarity calculation is a problem that has been a 40 year study history. The application of the method mainly focuses on detecting code plagiarism and searching similar codes in software maintenance, is not combined with a safety tool, and cannot solve the problem of code safety independently.

The automatic SAST can generate a large amount of suspected leak records in a short time, and manual review is an essential link, but the automatic SAST has the characteristics of high cost and low speed, and cannot give full play to the advantages of tools. Individual adjustment rules or parameters attempting to control the ratio of false-positives to false-positives may have an unexevable effect on the final accuracy (e.g., overfitting results in more false-positives).

Therefore, it is necessary to provide a method for intelligently evaluating computer software vulnerabilities to solve the above technical problems.

Disclosure of Invention

The invention provides a method for intelligently evaluating computer software bugs, which solves the problems of high analysis difficulty and inconvenient operation.

In order to solve the technical problem, the method for intelligently evaluating the computer software vulnerability provided by the invention comprises the following steps of: the system comprises a data collection module, a planning learning module, a system maintenance module and a result updating module;

the output end of the data collection module is connected with the input end of the planning learning module, the output end of the planning learning module is connected with the input end of the result updating module, the output end of the system maintenance module is connected with the input end of the data collection module, the output end of the system maintenance module is connected with the input end of the planning learning module, and the output end of the system maintenance module is connected with the input end of the result updating module.

Preferably, the data collection module provides a perfect interface by the currently mainstream SAST tool in the industry, and the data collection module obtains the related scan result by calling the interface, analyzes the interface return data, and stores the interface return data in the database in a structured manner.

Preferably, the rule learning module is configured to extract all records that have been manually determined, such as result status fields, typical common statuses such as to-be-confirmed, unavailable, false-positive, confirmed, and the like. As the actual proportion of various common loopholes is different, sampling is recommended according to the proportion of samples in an equal proportion in order to ensure that a classifier can identify various loopholes, so that the total of the loopholes conforms to the principle of twenty-eight, namely 80% of records are used as a training set, and 20% of records are used as a test set.

Preferably, the rule learning module may obtain a classifier by calculating similarity between the target code segment and the sample code segment by using deep learning methods such as DECKARD, rtvNN, CDLH, etc. for the code segments corresponding to the training set, and the most similar code segments are considered to have the same result status. In order to improve the operation efficiency, the vulnerability to be confirmed does not need to be compared with all records in the training set in a distance calculation manner, only needs to be compared with samples of the same vulnerability type (such as OWASP Top 10A 1-A10) in the same platform language (such as C/Java/Python/Javascript and the like) in a calculation manner, when the total false alarm and missed alarm ratio of the test set both accord with a given threshold level, the training is considered to be effective, otherwise, the sampling training is repeated.

Preferably, the system maintenance module has more and more records which are manually verified (labeled) in actual work, the training set should be updated in time, and the records of the labels, whether manual or automatic, may need to be adjusted again in the follow-up process and the classifier is updated in time.

Preferably, the system maintenance module may trigger repeated calling of the rule learning module for the increased overall samples according to given parameters, such as the number of days in interval, the increment of the number of markers, and the like, and may also support manual intervention and adjustment followed by resampling (omission), training, and verification steps.

Preferably, the result updating module applies the classifier passing the test set inspection to all the records to be classified to obtain a preliminary discrimination result of the intelligent classifier, updates the state in the report through an API of the SAST tool, and displays a score (By Automatic classifier) obtained By the Automatic classifier for further evaluation or processing.

Preferably, the result updating module includes an input unit, a data result analyzing unit, a data result integrating unit, a data backup unit, and an output unit, where the data result analyzing unit is configured to perform statistical analysis on the evaluation result, and the data structure integrating unit is configured to perform integration and classification on the analyzed evaluation result.

Preferably, the number of the backups in the data backup unit is fifty, and when the number of the backups exceeds fifty, the first backup is eliminated.

Preferably, the data in the data backup unit can be manually set as star data, the data backup unit does not delete the star data, and when the number of the star data exceeds fifty, the data backup unit reminds the user to select the star data manually.

Compared with the related technology, the method for intelligently evaluating the computer software vulnerability provided by the invention has the following beneficial effects:

the invention provides a method for intelligently evaluating computer software bugs, which is characterized in that a machine learning method is introduced for intelligent evaluation, a training set is periodically and automatically updated, the experience durably accumulated by professionals is timely converted into a judgment rule, a confidence level (reliability) level is given to a judgment result, so that the professionals can more reasonably arrange resources for confirming and repairing bugs, manual links can be reduced for highly reliable automatic judgment results, an SAST tool is combined with an intelligent learning method for automatically judging the possibility of bugs existing in similar code segments, the use of the SAST tool is optimized, and safety personnel/code examiners are assisted in evaluating code safety.

Drawings

FIG. 1 is a vulnerability description diagram of the intelligent computer software vulnerability assessment method provided by the present invention;

FIG. 2 is a schematic flow chart of a method for intelligently evaluating computer software vulnerabilities according to the present invention;

fig. 3 is a block diagram of a result update module according to the present invention.

Detailed Description

The invention is further described below with reference to the drawings and the embodiments.

Please refer to fig. 1, fig. 2, and fig. 3 in combination, wherein fig. 1 is a vulnerability description diagram of the intelligent computer software vulnerability assessment method provided by the present invention; FIG. 2 is a schematic flow chart of a method for intelligently evaluating computer software vulnerabilities according to the present invention; fig. 3 is a block diagram of a result update module according to the present invention. The method for intelligently evaluating the computer software vulnerability comprises the following steps: the system comprises a data collection module, a planning learning module, a system maintenance module and a result updating module;

The data collection module provides a perfect interface by the current SAST tool in the industry, the data collection module obtains the related scanning result by calling the interface, analyzes the data returned by the interface, and stores the data in a database in a structured manner.

The rule learning module is used for extracting all records which are manually judged, such as result state fields, and typical common states, such as to-be-confirmed, unavailable, false-alarm, confirmation and the like. Because the actual occupation ratios of various common vulnerability types are different, in order to ensure that the classifier can identify various vulnerabilities, sampling is recommended according to the proportion of samples in equal proportion, so that the total accords with the principle of two eight, namely 80% of records are used as a training set, and 20% of records are used as a test set.

In the process of analyzing the source code, the scanning tool does not deeply understand the source code enough, so that a lot of key information is lost, and in order to prevent missing report, the principle of 'killing by mistake and not putting by mistake' is adopted, so that only as many alarms as possible can be reported, and finally, a large amount of false reports are caused.

Each language has some characteristics of itself, and if not supported, the object pointed by the identifier cannot be confirmed (for example, the variable declaration of JavaScript is lifted) when the taint propagation analysis is carried out. Such language characteristics are not exhaustive, each language is different, and unfortunately most vendors do not give adequate support in this regard.

There is also a main reason that some inspectors only use in-process analysis to reduce complexity, and lose context information, so that some objects pointed by the identifiers correctly cannot be accurately located, and finally false alarm or false alarm is caused.

And manually auditing the scanning result through a rule learning module, and mainly performing classification evaluation on false reports and false reports.

The rule learning module can obtain a classifier by calculating the similarity of the target code segment and the sample code segment by using deep learning methods such as DECKARD, rtvNN, CDLH and the like for the code segments corresponding to the training set, and the most similar code segment is considered to have the same result state. In order to improve the operation efficiency, the vulnerability to be confirmed does not need to be compared with all records in the training set in a distance calculation mode, only needs to be compared with samples of the same vulnerability type (such as OWASP Top 10A 1-A10) in the same platform language (such as C/Java/Python/Javascript and the like) in a calculation mode, when the overall false report and missed report proportion of the testing set accord with a given threshold level, the training is considered to be effective, and otherwise, the sampling training is repeated.

In the actual work, the system maintenance module has more and more records which are manually verified (labeled), the training set is updated in time, and whether the records are manually or automatically labeled, the original label may need to be adjusted again in the follow-up process and the classifier is updated in time.

The system maintenance module can trigger repeated calling of the rule learning module for the increased overall samples according to given parameters, such as interval days, mark quantity increment and the like, and can also support the steps of resampling (neglect), training and verification after manual intervention and adjustment.

And the result updating module applies the classifiers which pass the test set inspection to all the records to be classified to obtain a preliminary judgment result of the intelligent classifier, updates the state in the report through an API (application program interface) of the SAST tool, displays the score (By Automatic classifier, maximum similarity) obtained By the Automatic classifier, and waits for further evaluation or processing.

The result updating module comprises an input unit, a data result analyzing unit, a data result integrating unit, a data backup unit and an output unit, wherein the data result analyzing unit is used for carrying out statistical analysis on the evaluation results, and the data structure integrating unit is used for carrying out integration and classification on the analyzed evaluation results.

Fifty backup numbers are arranged in the data backup unit, and when the storage number exceeds fifty, the first backup data is removed.

The data in the data backup unit can be manually set as star data, the data backup unit does not delete the star data, and when the number of the star data exceeds fifty, the data backup unit reminds the user to select the star data manually.

The working principle of the method for intelligently evaluating the computer software bugs provided by the invention is as follows:

by introducing intelligent evaluation of a machine learning method, a training set is automatically updated regularly, the experience durably accumulated by professionals is converted into a judgment rule in time, and a confidence level (reliability) level is given to a judgment result, so that the professionals can more reasonably arrange resources to confirm and repair the bug, and the labor link can be reduced for the highly reliable automatic judgment result.

by introducing intelligent evaluation of a machine learning method, a training set is automatically updated regularly, the experience durably accumulated by professionals is converted into a judgment rule in time, a confidence level (reliability) level is given to a judgment result, so that the professionals can more reasonably arrange resources to confirm and repair the bug, manual links can be reduced for the highly reliable automatic judgment result, an SAST tool is combined with the intelligent learning method to automatically judge the possibility of the bug existing in similar code segments, the use of the SAST tool is optimized, and safety personnel/code examiners are assisted in evaluating the code safety.

The above description is only an embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are included in the scope of the present invention.

Claims

1. A method for intelligently evaluating computer software vulnerabilities is characterized by comprising the following steps: the system comprises a data collection module, a planning learning module, a system maintenance module and a result updating module;

2. The method of claim 1, wherein the data collection module provides a complete interface with the currently mainstream SAST tools in the industry, and the data collection module obtains the related scanning results by calling the interface, analyzes the data returned from the interface, and stores the data in the database in a structured manner, and if the interface is not directly opened by the respective SAST tool, the data collection module needs to consider obtaining from the security platform such as SIEM, SOC, SPLUNK, etc.

3. The method for intelligent evaluation of computer software vulnerabilities according to claim 1, characterized in that the rule learning module is used to extract all records that have been manually determined, such as result status fields, typical common states such as to-be-confirmed, unavailable, false-positive, confirmed, etc.

4. Because the actual occupation ratios of various common vulnerability types are different, in order to ensure that the classifier can identify various vulnerabilities, sampling is recommended according to the proportion of samples in equal proportion, so that the total accords with the principle of two eight, namely 80% of records are used as a training set, and 20% of records are used as a test set.

5. The method of claim 3, wherein the rule learning module obtains the classifier by calculating the similarity between the target code segment and the sample code segment for the code segments corresponding to the training set by using deep learning methods such as DECKARD, rtvNN, CDLH, and the most similar code segments are considered to have the same result status.

6. In order to improve the operation efficiency, the vulnerability to be confirmed does not need to be compared with all records in the training set in a distance calculation manner, only needs to be compared with samples of the same vulnerability type (such as OWASP Top 10A 1-A10) in the same platform language (such as C/Java/Python/Javascript and the like) in a calculation manner, when the total false alarm and missed alarm ratio of the test set both accord with a given threshold level, the training is considered to be effective, otherwise, the sampling training is repeated.

7. The method for intelligent evaluation of computer software vulnerability according to claim 1, wherein the system maintenance module has more and more records verified (labeled) by human in actual work, the training set should be updated in time, and there is a possibility that the original label needs to be adjusted again in the following and the classifier needs to be updated in time no matter the records of the labels are manually or automatically.

8. The method for intelligent evaluation of computer software vulnerability according to claim 5, wherein the system maintenance module can trigger repeated calling of the rule learning module for the added overall samples according to given parameters, such as number of days in interval, increment of number of markers, etc., and can also support steps of re-sampling (optional), training, and verification after manual intervention and adjustment.

9. The method according to claim 1, wherein the result updating module applies the classifier passing the test set test to all the records to be classified to obtain a preliminary discrimination result By the intelligent classifier, updates the status in the report through an API of an SAST tool, and displays a score (By Automatic classifier, maximum similarity) obtained By the Automatic classifier to be further evaluated or processed.

10. The method for intelligent evaluation of computer software vulnerabilities according to claim 1, wherein the result updating module includes an input unit, a data result analysis unit, a data result integration unit, a data backup unit, and an output unit, the data result analysis unit is used for performing statistical analysis on the evaluation results, and the data structure integration unit is used for performing integration and classification on the analyzed evaluation results.

11. The method according to claim 8, wherein the number of the backup in the data backup unit is fifty, and when the number of the backup in the data backup unit exceeds fifty, the first backup is removed.

12. The method for intelligent assessment of computer software vulnerabilities according to claim 9, wherein the data in the data backup unit is manually set as star data, the data backup unit does not delete the star data, and when the number of the star data exceeds fifty, a prompt is given, and the selection is manually made.