US9747187B2 - Simulating black box test results using information from white box testing - Google Patents

Simulating black box test results using information from white box testing Download PDF

Info

Publication number
US9747187B2
US9747187B2 US12/913,314 US91331410A US9747187B2 US 9747187 B2 US9747187 B2 US 9747187B2 US 91331410 A US91331410 A US 91331410A US 9747187 B2 US9747187 B2 US 9747187B2
Authority
US
United States
Prior art keywords
computer software
computer
potential vulnerability
milestone
entry point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/913,314
Other versions
US20120110551A1 (en
Inventor
Stephen Fink
Yinnon A. Haviv
Roee Hay
Marco Pistoia
Ory Segal
Adi Sharabani
Manu Sridharan
Frank Tip
Omer Tripp
Omri Weisman
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HCL Technologies Ltd
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US12/913,314 priority Critical patent/US9747187B2/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HAY, ROEE, SHARABANI, ADI, WEISMAN, OMRI, HAVIV, YINNON A., SEGAL, ORY, TRIPP, OMER, SRIDHARAN, MANU, FINK, STEPHEN, PISTOIA, MARCO, TIP, FRANK
Publication of US20120110551A1 publication Critical patent/US20120110551A1/en
Application granted granted Critical
Publication of US9747187B2 publication Critical patent/US9747187B2/en
Assigned to HCL TECHNOLOGIES LIMITED reassignment HCL TECHNOLOGIES LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTERNATIONAL BUSINESS MACHINES CORPORATION
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3604Software analysis for verifying properties of programs
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/362Software debugging
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577Assessing vulnerabilities and evaluating computer system security
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44589Program code verification, e.g. Java bytecode verification, proof-carrying code
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/033Test or assess software

Abstract

Systems, methods are program products for simulating black box test results using information obtained from white box testing, including analyzing computer software (e.g., an application) to identify a potential vulnerability within the computer software application and a plurality of milestones associated with the potential vulnerability, where each of the milestones indicates a location within the computer software application, tracing a path from a first one of the milestones to an entry point into the computer software application, identifying an input to the entry point that would result in a control flow from the entry point and through each of the milestones, describing the potential vulnerability in a description indicating the entry point and the input, and presenting the description via a computer-controlled output medium.

Description

BACKGROUND

Software testing methods fall into two general categories: white box testing and black box testing. White box testing methods entail exposing and statically analyzing the internal workings of a software application, without executing the application. Black box testing methods, on the other hand, entail testing an application during its execution and without relying on any knowledge of the internal structure of the application. Although white box testing provides the advantage of revealing the precise nature and location of a potential vulnerability of an application, as the application has not been executed, any such exposed vulnerability is theoretical, providing no proof of vulnerability to the developer. Additionally, white box testing expresses potential vulnerabilities in terms of locations within an application's programming instructions that are intelligible only to application developers. Conversely, having executed an application, black box testing both provides the developer with proof of vulnerability, and exposes susceptibilities to attacks from a user's point of view. However, black box testing is often computationally expensive as the tester is ‘blind’ to the internal structure of the application, requiring many execution attempts before a vulnerability is discovered.

SUMMARY

In one aspect of the present invention a system is provided for simulating black box test results using information obtained from white box testing, the system including a white box tester configured to analyze computer software (e.g., an application) to identify a potential vulnerability within the computer software application and a plurality of milestones associated with the potential vulnerability, where each of the milestones indicates a location within the computer software application, an entry point tracer configured to trace a path from a first one of the milestones to an entry point into the computer software application, an input analyzer configured to identify an input to the entry point that would result in a control flow from the entry point and through each of the milestones, and a black box simulator configured to describe the potential vulnerability in a description indicating the entry point and the input and present the description via a computer-controlled output medium.

In another aspect of the invention a method is provided for simulating black box test results using information obtained from white box testing, the method including analyzing computer software (e.g., an application) to identify a potential vulnerability within the computer software application and a plurality of milestones associated with the potential vulnerability, where each of the milestones indicates a location within the computer software application, tracing a path from a first one of the milestones to an entry point into the computer software application, identifying an input to the entry point that would result in a control flow from the entry point and through each of the milestones, describing the potential vulnerability in a description indicating the entry point and the input, and presenting the description via a computer-controlled output medium.

A computer program product embodying the invention is also provided.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be understood and appreciated more fully from the following detailed description taken in conjunction with the appended drawings in which:

FIG. 1 is a simplified conceptual illustration of a system for simulating black box test results using information obtained from white box testing, constructed and operative in accordance with an embodiment of the invention;

FIG. 2 is a simplified flowchart illustration of an exemplary method of operation of the system of FIG. 1, operative in accordance with an embodiment of the invention;

FIG. 3, which is an exemplary code sample useful in understanding the system and method of FIGS. 1 and 2; and

FIG. 4 is a simplified block diagram illustration of an exemplary hardware implementation of a computing system, constructed and operative in accordance with an embodiment of the invention.

DETAILED DESCRIPTION

The invention is now described within the context of one or more embodiments, although the description is intended to be illustrative of the invention as a whole, and is not to be construed as limiting the invention to the embodiments shown. It is appreciated that various modifications may occur to those skilled in the art that, while not specifically shown herein, are nevertheless within the true spirit and scope of the invention.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical data storage device, a magnetic data storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

Reference is now made to FIG. 1 which is a simplified conceptual illustration of a system for simulating black box test results using information obtained from white box testing, constructed and operative in accordance with an embodiment of the invention. In the system of FIG. 1, a white box tester 100 is configured to use conventional methods, such as static analysis, to analyze the instructions and related resource files of computer software (e.g., an application), such as where the instructions are in the form of source code or bytecode, to identify potential vulnerabilities within the application. White box tester 100 preferably describes each identified potential vulnerability by indicating the type of vulnerability identified, the location or locations within the application instructions and related resource files where the vulnerability was identified, each location now referred to herein as a “milestone,” and one or more variables related to the vulnerability. Below is an example of an identified potential vulnerability and its white-box description:

    • SQL injection vulnerability at line 38 of file DatabaseUtil.java. Variable x in statement sqlStatement.executeQuery(x); might contain malicious data.

An entry point tracer 102 is configured to trace one or more paths, using conventional techniques, from the first milestone of a given potential vulnerability identified during white box testing to one or more entry points into the application, where an entry point is defined as a location within an application where an interface is exposed to receive input to the application from a source that is external to the application. Entry point tracer 102 may, for example, construct a call graph of method invocations within the application, using conventional techniques, and utilize the call graph to trace the paths.

An input analyzer 104 is configured to identify, using conventional techniques, the inputs to an entry point identified by entry point tracer 102 for the given potential vulnerability. Input analyzer 104 may, for example, employ constant-propagation and string-analysis techniques to extract the values of non-constant variables that participate in computations at the entry point. Input analyzer 104 then determines which of the identified inputs would result in a control flow from the entry point and through each of the milestones of the given potential vulnerability, such as by employing control-flow and data-flow analysis techniques, as well as constant-propagation and string-analysis techniques, where each milestone is analyzed to identify any constraints it places on an input in order to allow the control flow to reach the milestone given the input.

A black box simulator 106 is configured to take a given potential vulnerability, identified during white box testing and described in terms of the location or locations within the application instructions and related resource files where the vulnerability was identified, and describe the potential vulnerability in terms of its associated entry points as identified by entry point tracer 102 and associated inputs as identified by input analyzer 104, thereby creating a black box view of the potential vulnerability identified during white box testing. Thus, for example, the following white box description of an identified potential vulnerability:

    • SQL injection vulnerability at line 38 of file DatabaseUtil.java. Variable x in statement sqlStatement.executeQuery(x); might contain malicious data.
      becomes:
    • SQL injection vulnerability at URL http://www.my.site/members/search.jsp. Parameter searchTerm flows into an SQL query without validation.

Black box simulator 106 preferably presents the black box view of the potential vulnerability via a computer-controlled output medium, such as a computer display or printout.

Any of the elements shown in FIG. 1 are preferably executed by or otherwise made accessible to a computer 108, such as by implementing any of the elements in computer hardware and/or in computer software embodied in a physically-tangible, computer-readable medium in accordance with conventional techniques.

Reference is now made to FIG. 2 which is a simplified flowchart illustration of an exemplary method of operation of the system of FIG. 1, operative in accordance with an embodiment of the invention. In the method of FIG. 2, white box testing is performed on the instructions and related resource files of a computer software application to identify potential vulnerabilities within the application (step 200). For any, and preferably every, identified potential vulnerability, a path is traced from the first milestone of the identified potential vulnerability being processed to one or more entry points into the application (step 202). For any, and preferably every, identified entry point, the inputs to the entry point are identified (step 204). It is then determined which of the identified inputs would result in a control flow from the entry point and through each of the milestones of the given potential vulnerability being processed (step 206). The identified potential vulnerability being processed is then described in terms of its associated entry points and associated inputs (step 208), thereby creating a black box view of the potential vulnerability identified during white box testing. The black box view of the potential vulnerability is provided via a computer-controlled output medium, such as a computer display or printout (step 210).

Additional reference is now made to FIG. 3, which is an exemplary code sample useful in understanding the system and method of FIGS. 1 and 2. In the example of FIG. 3, the Java™ code shown is stored in a file named “UsernameServlet.java,” for which white box testing produces the following report:

    • XSS vulnerability at line 12 of file UsernameServlet.java. Variable username in statement w.println(“username is”+username); might contain malicious data.

The first milestone for this report is the statement at line 8, which reads potentially malicious data via the method call req.getParameter(“username”). From this milestone, entry point tracer 102 identifies the UsernameServlet.doGet( ) method as a corresponding entry point. Then, input analyzer 104 discovers that the HTTP request parameter “username” contains the potentially malicious data by performing a string constant analysis of the parameter to the req.getParameter( ) call on line 8. Finally, black box simulator 106 consults the web.xml file for the application to discover a URL corresponding to the UsernameServlet class, such as http://www.my.site/username.html. Black box simulator 106 then presents the black box view of the potential vulnerability as XSS vulnerability at URL http://www.my.site/username.html. Parameter username is added to the HTTP response without validation.

Referring now to FIG. 4, block diagram 400 illustrates an exemplary hardware implementation of a computing system in accordance with which one or more components/methodologies of the invention (e.g., components/methodologies described in the context of FIGS. 1-2) may be implemented, according to an embodiment of the invention.

As shown, the techniques for controlling access to at least one resource may be implemented in accordance with a processor 410, a memory 412, I/O devices 414 and a network interface 416, coupled via a computer bus 418 or alternate connection arrangement.

It is to be appreciated that the term “processor” as used herein is intended to include any processing device, such as, for example, one that includes a CPU (central processing unit) and/or other processing circuitry. It is also to be understood that the term “processor” may refer to more than one processing device and that various elements associated with a processing device may be shared by other processing devices.

The term “memory” as used herein is intended to include memory associated with a processor or CPU, such as, for example, RAM, ROM, a fixed memory device (e.g., hard drive), a removable memory device (e.g., diskette), flash memory, etc. Such memory may be considered a computer readable storage medium.

In addition, the phrase “input/output devices” or “I/O devices” as used herein is intended to include, for example, one or more input devices (e.g., keyboard, mouse, scanner, etc.) for entering data to the processing unit, and/or one or more output devices (e.g., speaker, display, printer, etc.) for presenting results associated with the processing unit.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality and operation of possible implementations of systems, methods and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It will be appreciated that any of the elements described hereinabove may be implemented as a computer program product embodied in a computer-readable medium, such as in the form of computer program instructions stored on magnetic or optical storage media or embedded within computer hardware, and may be executed by or otherwise accessible to a computer (not shown).

While the methods and apparatus herein may or may not have been described with reference to specific computer hardware or software, it is appreciated that the methods and apparatus described herein may be readily implemented in computer hardware or software using conventional techniques.

While the invention has been described with reference to one or more specific embodiments, the description is intended to be illustrative of the invention as a whole and is not to be construed as limiting the invention to the embodiments shown. It is appreciated that various modifications may occur to those skilled in the art that, while not specifically shown herein, are nevertheless within the true spirit and scope of the invention.

Claims (8)

What is claimed is:
1. A system, comprising:
a white box tester that statically analyzes computer software to identify a plurality of milestones, including a first milestone, associated with a potential vulnerability within the computer software, wherein each of the milestones indicates a location of a method call of a source code statement within the computer software at which a data item can be accessed and modified and wherein the first milestone indicates a first source code location within the computer software at which the data item can be accessed and modified without validation, resulting in the potential vulnerability;
an entry point tracer that identifies one or more entry points into the computer software associated with the potential vulnerability by tracing paths from the source code statement of the first milestone, wherein each entry point provides a method location where an interface of the computer software is exposed to receive input external to the computer software;
an input analyzer that identifies one or more HTTP request parameter inputs to at least a first one of the one or more entry points that results in a control flow from the first entry point to the first milestone; and
a black box simulator that:
automatically identifies, from a consultation of an Extensible Markup Language (XML) configuration file for a web server executing the computer software, a uniform resource locator (URL) of a class representing the computer software having the potential vulnerability based on the first entry point and the one or more identified HTTP request parameter inputs; and
presents a simulated black box test result for the computer software by generating a description of the potential vulnerability indicating the identified URL exposing the potential vulnerability, and one or more of the identified HTTP request parameter inputs that have not been validated.
2. The system of claim 1 wherein the entry point tracer is configured to construct a call graph of method invocations within the computer software and utilize the call graph to trace the path.
3. The system of claim 1 wherein the input analyzer is configured to analyze each of the milestones to identify any constraints the milestone places on the input in order to allow the control flow to reach first the milestone given the input.
4. The system of claim 1 wherein the computer software is an application.
5. A computer program product, comprising:
a non-transitory computer-readable storage medium; and
computer-readable program code embodied in the computer-readable storage medium, wherein the computer-readable program code:
statically analyzes computer software to identify a plurality of milestones, including a first milestone, associated with a potential vulnerability within the computer software, wherein each of the milestones indicates a respective location of a method call within a respective source code statement of the computer software which accesses a data variable associated with the potential vulnerability and the first milestone indicates a first source code location of the method call within the computer software which allows the data variable to be accessed and modified without validation, resulting in the potential vulnerability,
identifies one or more entry points into the computer software associated with the potential vulnerability by tracing paths from the source code statement associated with the first milestone, wherein each entry point is a location where an interface of the computer software is exposed to receive input to the computer software from a source that is external to the computer software,
identifies one or more HTTP request parameter inputs to at least a first one of the one or more entry points that results in a control flow from the first entry point to the first milestone,
automatically identifies, from a consultation of an Extensible Markup Language (XML) configuration file for a web server executing the computer software, a uniform resource locator (URL) of a class representing the computer software having the potential vulnerability based on the first entry point and the one or more HTTP request parameter inputs, and
presents a simulated black box test result detailing, for the computer software, a description of the potential vulnerability, the identified URL exposing the potential vulnerability, and one or more of the identified HTTP request parameter inputs that have not been validated.
6. The computer program product of claim 5, wherein the computer-readable program code is configured to construct a call graph of method invocations within the computer software and utilizing the call graph to trace the path.
7. The computer program product of claim 5, wherein the computer-readable program code is configured to analyze each of the milestones to identify any constraints the milestone places on the input in order to allow the control flow to reach the first milestone given the input.
8. The computer program product of claim 5 wherein the computer software is an application.
US12/913,314 2010-10-27 2010-10-27 Simulating black box test results using information from white box testing Active 2032-05-20 US9747187B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/913,314 US9747187B2 (en) 2010-10-27 2010-10-27 Simulating black box test results using information from white box testing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/913,314 US9747187B2 (en) 2010-10-27 2010-10-27 Simulating black box test results using information from white box testing
US13/493,067 US9720798B2 (en) 2010-10-27 2012-06-11 Simulating black box test results using information from white box testing

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/493,067 Continuation US9720798B2 (en) 2010-10-27 2012-06-11 Simulating black box test results using information from white box testing

Publications (2)

Publication Number Publication Date
US20120110551A1 US20120110551A1 (en) 2012-05-03
US9747187B2 true US9747187B2 (en) 2017-08-29

Family

ID=45998094

Family Applications (2)

Application Number Title Priority Date Filing Date
US12/913,314 Active 2032-05-20 US9747187B2 (en) 2010-10-27 2010-10-27 Simulating black box test results using information from white box testing
US13/493,067 Active 2031-01-09 US9720798B2 (en) 2010-10-27 2012-06-11 Simulating black box test results using information from white box testing

Family Applications After (1)

Application Number Title Priority Date Filing Date
US13/493,067 Active 2031-01-09 US9720798B2 (en) 2010-10-27 2012-06-11 Simulating black box test results using information from white box testing

Country Status (1)

Country Link
US (2) US9747187B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10394704B2 (en) * 2016-10-07 2019-08-27 Ford Global Technologies, Llc Method and device for testing a software program

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9747187B2 (en) 2010-10-27 2017-08-29 International Business Machines Corporation Simulating black box test results using information from white box testing
US9971896B2 (en) 2011-12-30 2018-05-15 International Business Machines Corporation Targeted security testing
JP5982575B2 (en) * 2012-08-29 2016-08-31 ヒューレット−パッカード デベロップメント カンパニー エル.ピー.Hewlett‐Packard Development Company, L.P. Security scan based on dynamic taint
CN102968375B (en) * 2012-11-30 2015-10-28 中国矿业大学 Based on the infeasible paths detection method of association rule mining
US8984495B2 (en) 2013-01-03 2015-03-17 International Business Machines Corporation Enhanced string analysis that improves accuracy of static analysis
US9569334B2 (en) * 2013-03-14 2017-02-14 Whitehat Security, Inc. Techniques for traversing representations of source code
US9135152B2 (en) 2013-05-29 2015-09-15 International Business Machines Corporation Optimizing test data payload selection for testing computer software applications via computer networks
US9195570B2 (en) 2013-09-27 2015-11-24 International Business Machines Corporation Progressive black-box testing of computer software applications
US9413780B1 (en) 2014-05-06 2016-08-09 Synack, Inc. Security assessment incentive method for promoting discovery of computer software vulnerabilities
CN104063326A (en) * 2014-07-14 2014-09-24 上海零一拼装信息技术有限公司 Software testing data analysis oscilloscope and method
US10437707B2 (en) * 2015-03-19 2019-10-08 Teachers Insurance And Annuity Association Of America Evaluating and presenting software testing project status indicators
US9792102B2 (en) * 2015-09-04 2017-10-17 Quest Software Inc. Identifying issues prior to deploying software
US10310849B2 (en) 2015-11-24 2019-06-04 Teachers Insurance And Annuity Association Of America Visual presentation of metrics reflecting lifecycle events of software artifacts
WO2017173289A2 (en) 2016-04-01 2017-10-05 Massachusetts Institute Of Technology Apparatus, systems, and methods of transparent displays
US10210076B2 (en) 2016-09-26 2019-02-19 International Business Machines Corporation White box testing
US20190340366A1 (en) * 2018-05-04 2019-11-07 Raytheon Company Cybersecurity by i/o inferred from execution traces

Citations (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6006328A (en) 1995-07-14 1999-12-21 Christopher N. Drake Computer software authentication, protection, and security system
US6249882B1 (en) 1998-06-15 2001-06-19 Hewlett-Packard Company Methods and systems for automated software testing
US20030046029A1 (en) 2001-09-05 2003-03-06 Wiener Jay Stuart Method for merging white box and black box testing
US20030140337A1 (en) * 2001-12-21 2003-07-24 Celoxica Ltd. System, method, and article of manufacture for data transfer reporting for an application
US20040230956A1 (en) * 2002-11-07 2004-11-18 Cirne Lewis K. Simple method optimization
US6959431B1 (en) 1999-05-13 2005-10-25 Compuware Corporation System and method to measure and report on effectiveness of software program testing
US20050273861A1 (en) * 2004-06-04 2005-12-08 Brian Chess Apparatus and method for monitoring secure software
US6978228B1 (en) * 1998-12-18 2005-12-20 Graham Paul Gordon Method of performing a system reverse engineering process
US20050283768A1 (en) * 2004-06-21 2005-12-22 Sanyo Electric Co., Ltd. Data flow graph processing method, reconfigurable circuit and processing apparatus
US20060010497A1 (en) * 2004-05-21 2006-01-12 O'brien Darci System and method for providing remediation management
US7028223B1 (en) 2001-08-13 2006-04-11 Parasoft Corporation System and method for testing of web services
US20070061625A1 (en) 2005-09-15 2007-03-15 Acosta Juan Jr Automation structure for software verification testing
US20070226794A1 (en) * 2006-03-21 2007-09-27 Sbc Knowledge Ventures L.P. Security scanning system and method
US20080184208A1 (en) * 2007-01-30 2008-07-31 Sreedhar Vugranam C Method and apparatus for detecting vulnerabilities and bugs in software applications
US20080244321A1 (en) 2007-03-08 2008-10-02 Tim Kelso Program Test System
US20090119777A1 (en) * 2008-02-22 2009-05-07 Nhn Corporation Method and system of determining vulnerability of web application
US20090132861A1 (en) * 2007-11-16 2009-05-21 Microsoft Corporation Privacy Enhanced Error Reports
US7596778B2 (en) 2003-07-03 2009-09-29 Parasoft Corporation Method and system for automatic error prevention for computer software
US7603660B2 (en) 2005-06-01 2009-10-13 Microsoft Corporation Code coverage test selection
US20090282480A1 (en) 2008-05-08 2009-11-12 Edward Lee Apparatus and Method for Monitoring Program Invariants to Identify Security Anomalies
US7665072B2 (en) * 2005-04-21 2010-02-16 Microsoft Corporation Generating test cases for software with complex preconditions
US7685575B1 (en) * 2004-06-08 2010-03-23 Sun Microsystems, Inc. Method and apparatus for analyzing an application
US20100088683A1 (en) * 2000-03-03 2010-04-08 Identify Software, Ltd. System and method for software diagnostics using a combination of visual and dynamic tracing
US20100229158A1 (en) * 2009-03-06 2010-09-09 Fujitsu Limited Computer-readable recording medium storing verification support program, information processor, and verification support method
US7900193B1 (en) 2005-05-25 2011-03-01 Parasoft Corporation System and method for detecting defects in a computer program using data and control flow analysis
US20110055813A1 (en) * 2009-09-03 2011-03-03 Inaternational Business Machines Corporation Black Box Testing Optimization Using Information from White Box Testing
US7987390B2 (en) * 2004-09-24 2011-07-26 Oracle International Corporation Techniques for automatically tracking software errors
US20110191855A1 (en) 2010-01-29 2011-08-04 International Business Machines Corporation In-development vulnerability response management
US8024807B2 (en) 2004-06-17 2011-09-20 Trend Micro Incorporated Probabilistic mechanism to determine level of security for a software package
US20110231936A1 (en) 2010-03-19 2011-09-22 Aspect Security Inc. Detection of vulnerabilities in computer systems
US20120011493A1 (en) 2010-07-08 2012-01-12 Microsoft Corporation Binary code change vulnerability prioritization
US8122438B2 (en) * 2006-06-09 2012-02-21 International Business Machines Corporation Computer implemented method and system for accurate, efficient and adaptive calling context profiling
US8122439B2 (en) * 2007-08-09 2012-02-21 International Business Machines Corporation Method and computer program product for dynamically and precisely discovering deliquent memory operations
US8136095B2 (en) 2007-12-19 2012-03-13 Microsoft Corporation Relations in fuzzing data
US8141158B2 (en) 2008-12-31 2012-03-20 International Business Machines Corporation Measuring coverage of application inputs for advanced web application security testing
US20120110551A1 (en) 2010-10-27 2012-05-03 International Business Machines Corporation Simulating black box test results using information from white box testing
US20120110549A1 (en) 2010-10-29 2012-05-03 International Business Machines Corporation Code Breakage Detection Using Source Code History Background
US8276123B1 (en) 2008-07-22 2012-09-25 Juniper Networks, Inc. Adaptive regression test selection within testing environments
US20120266248A1 (en) 2011-04-13 2012-10-18 International Business Machines Corporation Pinpointing security vulnerabilities in computer software applications
US8296735B2 (en) * 2010-02-19 2012-10-23 National Ict Australia Limited Inter-procedural analysis of computer programs
WO2012166120A1 (en) 2011-05-31 2012-12-06 Hewlett-Packard Development Company, L.P. Application security testing
US20130024846A1 (en) 2011-07-22 2013-01-24 Microsoft Corporation Real-Time Code Coverage Results in AD-HOC Testing
US8370798B2 (en) * 2004-05-05 2013-02-05 Silverdata Limited Analytical software design system
US8397300B2 (en) 2009-09-22 2013-03-12 International Business Machines Corporation Detecting security vulnerabilities relating to cryptographically-sensitive information carriers when testing computer software
US8484732B1 (en) 2012-02-01 2013-07-09 Trend Micro Incorporated Protecting computers against virtual machine exploits
US8528095B2 (en) 2010-06-28 2013-09-03 International Business Machines Corporation Injection context based static analysis of computer software applications
US8539585B2 (en) 2006-11-30 2013-09-17 Microsoft Corporation Systematic approach to uncover visual ambiguity vulnerabilities
US20150095888A1 (en) 2013-09-27 2015-04-02 International Business Machines Corporation Progressive black-box testing of computer software applications

Patent Citations (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6006328A (en) 1995-07-14 1999-12-21 Christopher N. Drake Computer software authentication, protection, and security system
US6249882B1 (en) 1998-06-15 2001-06-19 Hewlett-Packard Company Methods and systems for automated software testing
US6978228B1 (en) * 1998-12-18 2005-12-20 Graham Paul Gordon Method of performing a system reverse engineering process
US6959431B1 (en) 1999-05-13 2005-10-25 Compuware Corporation System and method to measure and report on effectiveness of software program testing
US20100088683A1 (en) * 2000-03-03 2010-04-08 Identify Software, Ltd. System and method for software diagnostics using a combination of visual and dynamic tracing
US7028223B1 (en) 2001-08-13 2006-04-11 Parasoft Corporation System and method for testing of web services
US20030046029A1 (en) 2001-09-05 2003-03-06 Wiener Jay Stuart Method for merging white box and black box testing
US20030140337A1 (en) * 2001-12-21 2003-07-24 Celoxica Ltd. System, method, and article of manufacture for data transfer reporting for an application
US20040230956A1 (en) * 2002-11-07 2004-11-18 Cirne Lewis K. Simple method optimization
US7596778B2 (en) 2003-07-03 2009-09-29 Parasoft Corporation Method and system for automatic error prevention for computer software
US8370798B2 (en) * 2004-05-05 2013-02-05 Silverdata Limited Analytical software design system
US20060010497A1 (en) * 2004-05-21 2006-01-12 O'brien Darci System and method for providing remediation management
US20050273861A1 (en) * 2004-06-04 2005-12-08 Brian Chess Apparatus and method for monitoring secure software
US7685575B1 (en) * 2004-06-08 2010-03-23 Sun Microsystems, Inc. Method and apparatus for analyzing an application
US8024807B2 (en) 2004-06-17 2011-09-20 Trend Micro Incorporated Probabilistic mechanism to determine level of security for a software package
US20050283768A1 (en) * 2004-06-21 2005-12-22 Sanyo Electric Co., Ltd. Data flow graph processing method, reconfigurable circuit and processing apparatus
US7987390B2 (en) * 2004-09-24 2011-07-26 Oracle International Corporation Techniques for automatically tracking software errors
US7665072B2 (en) * 2005-04-21 2010-02-16 Microsoft Corporation Generating test cases for software with complex preconditions
US7900193B1 (en) 2005-05-25 2011-03-01 Parasoft Corporation System and method for detecting defects in a computer program using data and control flow analysis
US7603660B2 (en) 2005-06-01 2009-10-13 Microsoft Corporation Code coverage test selection
US20070061625A1 (en) 2005-09-15 2007-03-15 Acosta Juan Jr Automation structure for software verification testing
US20070226794A1 (en) * 2006-03-21 2007-09-27 Sbc Knowledge Ventures L.P. Security scanning system and method
US8122438B2 (en) * 2006-06-09 2012-02-21 International Business Machines Corporation Computer implemented method and system for accurate, efficient and adaptive calling context profiling
US8539585B2 (en) 2006-11-30 2013-09-17 Microsoft Corporation Systematic approach to uncover visual ambiguity vulnerabilities
US20080184208A1 (en) * 2007-01-30 2008-07-31 Sreedhar Vugranam C Method and apparatus for detecting vulnerabilities and bugs in software applications
US20080244321A1 (en) 2007-03-08 2008-10-02 Tim Kelso Program Test System
US8122439B2 (en) * 2007-08-09 2012-02-21 International Business Machines Corporation Method and computer program product for dynamically and precisely discovering deliquent memory operations
US20090132861A1 (en) * 2007-11-16 2009-05-21 Microsoft Corporation Privacy Enhanced Error Reports
US8136095B2 (en) 2007-12-19 2012-03-13 Microsoft Corporation Relations in fuzzing data
US20090119777A1 (en) * 2008-02-22 2009-05-07 Nhn Corporation Method and system of determining vulnerability of web application
US20090282480A1 (en) 2008-05-08 2009-11-12 Edward Lee Apparatus and Method for Monitoring Program Invariants to Identify Security Anomalies
US8276123B1 (en) 2008-07-22 2012-09-25 Juniper Networks, Inc. Adaptive regression test selection within testing environments
US8141158B2 (en) 2008-12-31 2012-03-20 International Business Machines Corporation Measuring coverage of application inputs for advanced web application security testing
US20100229158A1 (en) * 2009-03-06 2010-09-09 Fujitsu Limited Computer-readable recording medium storing verification support program, information processor, and verification support method
US20110055813A1 (en) * 2009-09-03 2011-03-03 Inaternational Business Machines Corporation Black Box Testing Optimization Using Information from White Box Testing
US8397300B2 (en) 2009-09-22 2013-03-12 International Business Machines Corporation Detecting security vulnerabilities relating to cryptographically-sensitive information carriers when testing computer software
US20110191855A1 (en) 2010-01-29 2011-08-04 International Business Machines Corporation In-development vulnerability response management
US8296735B2 (en) * 2010-02-19 2012-10-23 National Ict Australia Limited Inter-procedural analysis of computer programs
US20110231936A1 (en) 2010-03-19 2011-09-22 Aspect Security Inc. Detection of vulnerabilities in computer systems
US8528095B2 (en) 2010-06-28 2013-09-03 International Business Machines Corporation Injection context based static analysis of computer software applications
US20120011493A1 (en) 2010-07-08 2012-01-12 Microsoft Corporation Binary code change vulnerability prioritization
US20120254839A1 (en) 2010-10-27 2012-10-04 International Business Machines Corporation Simulating black box test results using information from white box testing
US20120110551A1 (en) 2010-10-27 2012-05-03 International Business Machines Corporation Simulating black box test results using information from white box testing
US20120110549A1 (en) 2010-10-29 2012-05-03 International Business Machines Corporation Code Breakage Detection Using Source Code History Background
US20120266248A1 (en) 2011-04-13 2012-10-18 International Business Machines Corporation Pinpointing security vulnerabilities in computer software applications
WO2012166120A1 (en) 2011-05-31 2012-12-06 Hewlett-Packard Development Company, L.P. Application security testing
US20130024846A1 (en) 2011-07-22 2013-01-24 Microsoft Corporation Real-Time Code Coverage Results in AD-HOC Testing
US8484732B1 (en) 2012-02-01 2013-07-09 Trend Micro Incorporated Protecting computers against virtual machine exploits
US20150095886A1 (en) 2013-09-27 2015-04-02 International Business Machines Corporation Progressive black-box testing of computer software applications
US20150095888A1 (en) 2013-09-27 2015-04-02 International Business Machines Corporation Progressive black-box testing of computer software applications

Non-Patent Citations (14)

* Cited by examiner, † Cited by third party
Title
Dewhurst Ryan, Implementing basic static code analysis into integrated development environments (ides) to reduce software vulnerablitilies, A Report submitted in partial fulfillment of the regulations governing the award of the Degree of BSc (Honours) Ethical Hacking for Computer Security at the University of Northumbria at Newcastle, 2011-2012.
E. Steegmans et al., "Black & White Testing: Bridging Black Box Testing and White Box Testing", Conferentie Software Testing: Beheers Optimaal de Risico's van IT in uw Business, location:Leuven, Belgium, date: Jan. 20-21, 2004, pp. 1-12, <<https://lirias.kuleuven.be/bitstream/123456789/134277/1/Paper.pdf>>.
Fink et al., "Snugglebug: A Powerful Approach to Weakest Preconditions," ACM, Jun. 15-20, 2009, pp. 363-374.
Galan, E., et al., A Multi-agent Scanner to Detect Stored-XSS Vulnerabilities, 2010 International Confere3nce for Internet Technology and Secured Transactions, Nov. 2010, pp. 1-6, IEEE, Piscataway, United States.
Gebre, Misganaw, et al., A Robust Defense Against Content-Sniffing XSS Attacks, 6th International Conference on Digital Content, Aug. 2010, pp. 315-320, IEEE, Piscataway, United States.
James Skene, "CS 230 Software Design and Construction-Part 3: Introduction to software engineering-Topic 6: Verification", Department of Computer Science, University of Auckland, New Zealand, May 2009, <<http://www.cs.auckland.ac.nz/compsci230s1c/lectures/james/cs230-2009-6.pdf>>.
James Skene, "CS 230 Software Design and Construction—Part 3: Introduction to software engineering—Topic 6: Verification", Department of Computer Science, University of Auckland, New Zealand, May 2009, <<http://www.cs.auckland.ac.nz/compsci230s1c/lectures/james/cs230-2009-6.pdf>>.
Jörg Herrmann et al., "Variants of Validity and Their Impact on the Overall Test Space", Proceedings of the Eleventh International Florida Artificial Intelligence Research Society Conference, pp. 472-477, Year of Publication: 1998, Publisher: AAAI Press, <<http://www.aaai.org/Papers/FLAIRS/1998/FLAIRS98-091.pdf>>.
Michal Chmielewski et al., "Find and Fix Vulnerabilities Before Your Application Ships", MSDN Magazine, Nov. 2007 Issue, Publisher: Microsoft Corporation, <<http://msdn.microsoft.com/en-us/magazine/cc163312.aspx>>.
Nicolas Kicillof et al., "Achieving Both Model and Code Coverage W Ith Automated Gray-Box Testing", Proceedings of the 3rd International Workshop on Advances in Model-Based Testing, London, United Kingdom, Jul. 9-12, 2007, pp. 1-11, Publisher: Association for Computing Machinery, New York, NY, USA.
Stephen H. Edwards, "Black-Box Testing Using Flow Graphs: An Experimental Assessment of Effectiveness and Automation Potential", Software Testing, Verification and Reliability, vol. 10, No. 4, pp. 249-262, Dec. 2000, <<http://people.cs.vt.edu/˜edwards/downloads/edwards-stvr-0012.pdf>>.
T. Y. Chen et al., "White on Black: A White-Box-Oriented Approach for Selecting Black-Box-Generated Test Cases", Proceedings of the First Asia-Pacific Conference on Quality Software, Oct. 30-31, 2000, Hong Kong, China, pp. 275-284.
U.S. Appl. No. 13/493,067 entitled "Simulating Black Box Test Results Using Information From White Box Testing," , filed Jun. 11, 2012.
Yoon H, Choi B; Inter-class test technique between black-box-class and white-box-class for component customization failures; Dec. 1999; Proceedings of the 1999 Asia Pacific Software Engineering Conference (APSEC '99); pp. 162-165. *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10394704B2 (en) * 2016-10-07 2019-08-27 Ford Global Technologies, Llc Method and device for testing a software program

Also Published As

Publication number Publication date
US20120254839A1 (en) 2012-10-04
US9720798B2 (en) 2017-08-01
US20120110551A1 (en) 2012-05-03

Similar Documents

Publication Publication Date Title
Feng et al. Apposcopy: Semantics-based detection of android malware through static analysis
Klieber et al. Android taint flow analysis for app sets
Li et al. Static analysis of android apps: A systematic literature review
KR20130051116A (en) Apparatus for automatically inspecting security of applications and method thereof
US8370945B2 (en) Identifying security breaches caused by web-enabled software applications
US9081961B2 (en) System and method for analyzing malicious code using a static analyzer
US7849509B2 (en) Detection of security vulnerabilities in computer programs
KR20090017598A (en) Iterative static and dynamic software analysis
Walden et al. Predicting vulnerable components: Software metrics vs text mining
US8479188B2 (en) Binary code change vulnerability prioritization
Mahmood et al. A whitebox approach for automated security testing of Android applications on the cloud
US8510842B2 (en) Pinpointing security vulnerabilities in computer software applications
US8893102B2 (en) Method and system for performing backward-driven path-sensitive dataflow analysis
US8850581B2 (en) Identification of malware detection signature candidate code
WO2008103286A2 (en) Assessment and analysis of software security flaws
Sui et al. SVF: interprocedural static value-flow analysis in LLVM
KR20090065277A (en) Apparatus and method for automatically analyzing a program for detecting malicious codes triggered under an certain event/context
US8776239B2 (en) In-development vulnerability response management
US20130167123A1 (en) Application debugging
Kwon et al. Mantis: Automatic performance prediction for smartphone applications
US9372785B2 (en) Identifying implicit assumptions associated with a software product
CA2777434C (en) Verifying application security vulnerabilities
US9904614B2 (en) Source code inspection and verification
Coker et al. Program transformations to fix C integers
US8584246B2 (en) Eliminating false reports of security vulnerabilities when testing computer software

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FINK, STEPHEN;HAVIV, YINNON A.;HAY, ROEE;AND OTHERS;SIGNING DATES FROM 20100922 TO 20101020;REEL/FRAME:025205/0847

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: HCL TECHNOLOGIES LIMITED, INDIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:050374/0781

Effective date: 20190811