CN115470138A - Debugger defect detection method based on different debugging levels cross validation - Google Patents
Debugger defect detection method based on different debugging levels cross validation Download PDFInfo
- Publication number
- CN115470138A CN115470138A CN202211161594.5A CN202211161594A CN115470138A CN 115470138 A CN115470138 A CN 115470138A CN 202211161594 A CN202211161594 A CN 202211161594A CN 115470138 A CN115470138 A CN 115470138A
- Authority
- CN
- China
- Prior art keywords
- debugging
- debugger
- program
- levels
- different
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 54
- 238000001514 detection method Methods 0.000 title claims abstract description 22
- 238000002790 cross-validation Methods 0.000 title claims abstract description 21
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000005457 optimization Methods 0.000 claims abstract description 15
- 230000009471 action Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims 2
- 238000012795 verification Methods 0.000 claims 2
- 230000003542 behavioural effect Effects 0.000 claims 1
- 230000002349 favourable effect Effects 0.000 claims 1
- 238000007689 inspection Methods 0.000 claims 1
- 238000012360 testing method Methods 0.000 description 9
- 230000006399 behavior Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000009377 nuclear transmutation Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3696—Methods or tools to render software testable
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a debugger defect detection method based on cross validation of different debugging levels, which mainly comprises the following steps: 1) Compiling the source code program into an executable program by using a compiler, and appointing any optimization option and starting a debugging option; 2) Respectively debugging and running the executable program in a debugger at a source code level and an instruction level in a single step; 3) Tracking and recording the state of the program when the single step debugging is carried out each time, namely dynamic debugging information; 4) And checking whether the dynamic debugging information recorded by the source code level single step debugging and the instruction level single step debugging violates a predefined relation, and if so, regarding the dynamic debugging information as a defect in the debugger. The invention solves the problems that the defect of a single debugger cannot be detected in the defect detection of the debugger and the false alarm rate is high, and provides a set of brand-new solution for the defect detection of the debugger based on the method of cross validation of different debugging levels.
Description
Technical Field
The invention relates to the field of software defect detection, in particular to defect detection of a debugger, which is a debugger defect detection method based on cross validation of different debugging levels.
Background
A debugger is the core foundation software in software development activities. Software developers rely on a debugger to debug running program codes and track the behavior of the program in dynamic running so as to quickly diagnose defects in the program. However, as a complex software system with a large scale, the debugger itself has a large number of defects. The defects hidden in the debugger inevitably affect the diagnosis and troubleshooting of software failures by developers. Therefore, timely and effective detection of the defects hidden in the debugger is of great significance for supporting the construction of high-reliability software products. The main function of the debugger is to debug and run the binary program and diagnose the error source of the program according to the debugging information. Thus, the input to the debugger is the structured binary program and the interactive debug script, and the output is the debug information of the program at each breakpoint. However, constructing an accurate debugger expected output is difficult for a given binary program and debug script. Detecting defects in a debugger presents a significant challenge due to the lack of an effective expected output. For this reason, recent researchers have proposed debugger defect detection methods based on differential testing and transmutation testing, respectively. The method based on differential testing detects defects of a debugger by debugging the same code by a plurality of debuggers, and comparing the differences to find inconsistencies in the generated debugging information. The metamorphic test-based method can be used for defect detection of a single debugger, multiple binary programs are generated by compiling the same source code at different optimization levels, and debugging information generated by the debugger for debugging the binary programs generated at different optimization levels is compared to detect defects of the debugger. However, the existing methods have certain limitations. Firstly, different debuggers implement inconsistency, and different interpretations are made on the output of debugging information, so that a great amount of false alarms may exist in the method based on the differential test. Secondly, due to the compiling optimization, the debugging information of the generated binary program mapped to the same source code cannot be directly compared, so that a large amount of debugging information cannot be verified, and the metamorphic test-based method has more false positives. Therefore, the invention provides a debugger defect detection method based on cross validation of different debugging levels to solve the existing limitation. The invention uses different debugging levels to debug the same executable program for differential comparison, supports the differential debugging of the executable program under any optimization level, namely supports the detection of the defects of a debugger when the debugger debugs the executable program with any optimization, and can more effectively detect the defects in the debugger.
Disclosure of Invention
The invention provides a debugger defect detection method based on cross validation of different debugging levels, which solves the key problem that whether the debugger behavior is correct or not can not be effectively verified due to lack of expected output in debugger test. For a single debugger, debugging information generated by debugging the same binary program at different debugging levels is used, and the defects in the debugger are exposed by comparing and searching inconsistency in the debugging information, so that the reliability and the quality of the debugger are improved.
In order to effectively detect the defects of the debugger, the invention discloses a debugger defect detection method based on different debugging level differences, which specifically comprises the following steps:
step 1, compiling a source code program by using a compiler to generate an executable program containing debugging information;
step 2, using a debugger to debug and run the program in a single step by taking a line of source codes as a step length, and recording dynamic information of the program when executing each step, wherein the dynamic information comprises information such as a hit address, the source codes and values of variables;
step 3, using a debugger to debug the running program in a single step by taking a line of instructions as a step length, and recording dynamic information of the program when executing each step, wherein the dynamic information comprises information such as hit addresses, source codes and values of variables;
step 4, cross-verifying the information of the source code level stepping debugging operation record and the information of the instruction level stepping debugging operation record, and searching for inconsistency;
and 5, regarding the inconsistency found in the debugging information collected by the debugging running program at different debugging levels as the defect of the debugger.
In step 1, when compiling the source code program into an executable program containing debugging information, debugging options of a compiler need to be started, and meanwhile, starting of any compiling optimization option is supported. For example, when using GCC and LLVM compilers, the "-g-O0" or "-g-O3" options, etc. can be turned on. Here, "-g" is the debug option, "-O0", "-O3", etc. is the compile optimization option. When the debug option is enabled to compile the source code, the compiler adds additional information to the executable program, allowing the debugger to control the execution of the program.
In step 2 and step 3, for each executable program containing debugging information, the program is executed in a debugger in a single step by taking a source code level and an instruction level as step sizes respectively. Specifically, for the source code level, the step size of a single step is one row of source code, and the debugger debugs and executes the program step by step until the program runs to the end. For the instruction level, the step size of a single step is one row of instructions. For example, taking GDB as an example, the sequence of debug operations of a single-step execution program at the source code level is "start → while (true) { info frame → step }", the program is run line by line until the program exits, and the corresponding sequence of debug operations at the instruction level is "start → while (true) { info frame → stepi }". Here, the only difference between two different debug level debug execution programs is that the former uses step debug commands, while the latter uses stepi debug commands to perform single step debugging.
In step 2 and step 3, when the program is run by single step debugging, the dynamic information of each step of program execution is recorded. The dynamic information here is as follows but is not limited to the following information: and the next step is to proxy the action of the debugger according to the information such as the line number of the source code to be run, the address of the program counter, the value of the variable and the like. Specifically, the debugger is used to look up the current call stack to obtain the position information when the debugger is executed each time when the debugger is paused, that is, the line number of the source code to be executed next and the address of the next instruction to be executed, and the value of each variable is recorded. For example, the GDB can use the "bt-frame-info location-and-address" command to check the stack, and "info args/locals" to obtain the parameters and local variables of the current function.
In step 4, the debugging information of different debugging levels is compared, and whether the debugging information violates the predefined relationship is checked, so that the defect in the debugger is found. Here, the predefined relationship is as follows but is not limited to the following three categories: 1) The line number of a certain source code in the program is hit when stepping at the source level, and is also hit if and only if it is stepping at the instruction level; 2) If an instruction's address hits in a source-level step, it must also hit in an instruction-level step; 3) When a certain position of the program is reached for the first time in source and instruction level stepping, the value of each variable should be equal. If any of the predefined relationships are violated, a potential defect is reported.
The invention respectively debugs and runs the same program by using different debugging levels, records the dynamic information when the program is debugged and runs, and cross-verifies the dynamic information under different debugging levels to detect the debugging defect, thereby solving the key problem that the debugger behavior can not be verified to be correct due to lack of expected output in the debugger defect detection technology. The program debugged and run by the debugger can be compiled and generated by any optimization option, namely, the defect of the debugger when debugging and running any optimization program can be detected. The invention can automatically test the debugger, has simple and efficient execution process, can effectively detect the defects of the debugger and improves the reliability and quality of the debugger.
Has the beneficial effects that: the method can effectively detect the defects of the debugger, uses a single debugger to debug and run debugging behaviors of the same program at different debugging levels for cross validation, effectively solves the key challenge that expected output is absent in the debugger test, can effectively detect the defects in the debugger, and provides a brand-new solution for detecting the defects of the debugger.
Drawings
The above and other advantages of the present invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.
FIG. 1 is a flow chart of a debugger defect detection method based on different debug level cross validation.
FIG. 2 is a flow diagram of a single step debugger and collecting debug information at different debug levels.
FIG. 3 is a flow diagram for cross-validating debug information at different debug levels.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail with reference to the accompanying drawings.
FIG. 1 is a flowchart of a debugger defect detection method based on cross validation at different debug levels according to the present invention. The method comprises 5 steps, and the process is as follows:
step 1, compiling a source code program by using a compiler to generate an executable program containing debugging information;
step 2, using a debugger to debug the execution program in a single step by taking a line of source codes as a step length, and recording dynamic information of the program when executing one step, wherein the dynamic information comprises information such as a hit address, the source codes and values of variables;
step 3, a debugger is used for debugging and executing the program in a single step by step mode with a line of instructions as step lengths, and dynamic information of the program in each step of execution is recorded, wherein the dynamic information comprises information of hit addresses, source codes, values of variables and the like;
step 4, cross-verifying the information of the source code level single step stepping debugging operation record and the information of the instruction level single step stepping debugging operation record, and searching for inconsistency;
and 5, regarding the inconsistency found in the debugging information collected by the debugging running program at different debugging levels as the defect of the debugger.
Fig. 2 is a flowchart of step 2 and step 3 for stepping through the execution program and recording the dynamic information of each step of the program execution, the only difference between the two different debugging levels is the step size of stepping through the debugging execution program, and the process is as follows:
and 2-1, performing the test program in a debugger by taking the source code level as a step size in a single step. If the program is executed, finishing debugging; otherwise, the step 2-2 is entered.
Step 2-2, recording dynamic information of each step of program execution, including but not limited to the following information: and taking the information of the line number of the source code to be operated, the address of the program counter, the value of the variable and the like as the behavior of the debugger. And (5) repeating the step 2-1.
And 3-1, executing the test program in a debugger by taking the instruction level as a step in a single step. If the program is executed, finishing debugging; otherwise, the step 3-2 is entered.
Step 3-2, recording dynamic information of each step of program execution, including but not limited to the following information: and taking the information of the line number of the source code to be operated, the address of the program counter, the value of the variable and the like as the behavior of the debugger. And (5) repeating the step 3-1.
FIG. 3 is a flowchart of step 4 cross-validating debug information collected at different debug levels compared to step 2 and step 3. Step 2 and step 3 record the behavior information of the same debugger for debugging the same program under different debugging levels, and step 4 carries out cross validation on the information according to the predefined relation to find out inconsistent information. The process is as follows:
and 4-1, predefining the relation between the instruction level stepping and the source code level stepping for cross-verifying the debugging information of different debugging levels. For example, a line number of a certain source code in a program is hit when stepping at source level, and if and only if it is also hit when stepping at instruction level, this relationship can be used to cross-verify source code line information that hits at different debug levels.
And 4-2, comparing the debugging information at different debugging levels, and checking whether the debugging information violates the relationship predefined in the step 4-1, thereby discovering inconsistent debugging information. If any of the predefined relationships are violated, a potential bug in the debugger is reported.
The present invention provides a debugger defect detection method based on cross validation at different debugging levels, and a plurality of methods and ways for implementing the technical solution are provided, and the above description is only a preferred embodiment of the present invention, it should be noted that, for those skilled in the art, a plurality of improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. All the components not specified in this embodiment can be implemented by the prior art.
Claims (7)
1. A debugger defect detection method based on different debugging levels and cross validation is characterized in that a debugger is used for respectively debugging and running the same program at different debugging levels, and the debugging information recorded when the same program is debugged and run at different debugging levels is cross-compared to validate the debugger so as to detect defects in the debugger. It should be noted that, the cross-validation debugger may be a single debugger, or multiple debuggers that respectively debug and run the same program at different debugging levels. That is to say, the method supports both the self-verification of a single debugger and the verification of a plurality of debuggers, solves the key problem that whether the debugger behavior is correct or not cannot be effectively verified due to lack of expected output in the debugger defect detection technology, and the debugged program input by the debugger can be an executable program generated by compiling by using a compiler and any optimization option. The method can automatically detect the defects of the debugger, and is favorable for improving the correctness and reliability of the debugger. The method mainly comprises the following steps:
1) Compiling to generate an executable program containing debugging information:
for a given source code program, the source code program is compiled into an executable program containing debug information using a compiler to turn on any compile optimization options and enable debug options. Such as the "-g-O0" or "-g-O3" options in GCC and LLVM compilers, enable the compiler to add debug information to the executable program and allow the debugger to control the execution of the program, so that the program generated by compilation with any optimization option can be used as input to the debugger, thereby enabling detection of behavioral errors by the debugger while debugging any program.
2) Respectively debugging the running programs based on different debugging levels:
for any debug-enabled executable program, the program is run as a single step debug in the debugger at the source code level and instruction level, respectively. Specifically, the debugger starts running the executable program, and then steps the program in a single step until the executable program exits. For example, taking GDB as an example, the source code level step-by-step debugging running program operation sequence is "start → while (true), { info frame → step }", the step-by-step debugging running program is stepped until the program exits, and the instruction level step-by-step debugging running program operation sequence is "start → while (true) { info frame → step }", where the only difference between the source code level and the instruction level step-by-step debugging running program is to choose to use step and step debugging commands for step debugging, respectively.
3) Recording dynamic debugging information when the program executes each step:
when the program is run by single step debugging, the line number of the hit source code, the address of the program counter and the value of the variable are recorded to monitor the action of the debugger. Specifically, the position information of each pause of the single step debugging program, namely the line number of the next source code to be executed and the address of the next instruction, is obtained by checking the current call stack, and the current value of each variable is obtained. For example, the stack is checked using the command "bt-frame-info" of GDB, and "info areas/locals" obtains the parameters and local variable values of the current function.
4) Cross validation compares debugging information of different debugging levels:
the difference comparison uses the debugging information generated when the single-step debugging programs of different debugging levels are used to check whether the debugging information violates the predefined relation, so that the defects in the debugger are exposed. There are three predefined relationships: 1) The source line number in the program is hit in the source level single step debug run if and only if it is also hit in the instruction level single step debug run. 2) If an instruction address hits in a source-level step, it must also hit in an instruction-level step. 3) When a certain position of the program is reached for the first time in source and instruction level stepping, the value of each variable should be equal. If any of the predefined relationships are violated at the time of inspection, then a potential defect is deemed.
2. The method for detecting the defect of the debugger based on the cross validation at different debugging levels as claimed in claim 1, wherein in step 1), the input of the debugger of the method supports the compiler to enable any optimization option to compile and generate the binary program with the debugging information. That is, the compiler compiles the generated binary program with debug information using any optimization option can be used as the input of the debugger. Thus, the method supports verifying that the debugger behaves as expected when debugging an executable program with arbitrary optimizations.
3. The method for detecting the defect of the debugger based on the cross validation with different debugging levels as claimed in claim 1, wherein in step 2), the method supports both the debugging and running of the same program with different debugging levels by using a single debugger and the debugging and running of the same program with different debugging levels by a plurality of debuggers, and the method supports both the self validation of a single debugger and the mutual validation of a plurality of debuggers.
4. The method for detecting the defect of the debugger based on the cross validation with different debugging levels as claimed in claim 1, wherein in step 2), the method cross validates with each other using different debugging levels, the debugger uses different debugging levels to debug and run the same binary program respectively, and the debugging information is tracked and recorded to validate with each other. Here, the debugging level of the debugger is active code level single step debugging and instruction level single step debugging, that is, a line of source code or a line of instruction is used as a step size single step debugging running program.
5. The method for detecting the defect of the debugger based on the cross validation with different debugging levels as claimed in claim 1, wherein in step 2), the starting point of the debugger debugging the same binary program using different debugging levels respectively can be any position of the debugged program. That is, the debugged program can be run to a certain position of the program in an arbitrary manner, and then the running program is debugged respectively by using different debugging levels, and then the debugging information corresponding to the different debugging levels is tracked and recorded.
6. The method for detecting the defect of the debugger based on the cross validation with different debugging levels as claimed in claim 1, wherein in step 3), the debugger debugs the running program by one step, and tracks and records the dynamic debugging information after each step of executing the program. Here, the traced and recorded dynamic debugging information includes, but is not limited to, the hit source code line number, instruction address, and variable value, etc., so that the recorded debugging information of different debugging levels is cross-validated in step 4) to detect the defect of the debugger.
7. The method for detecting the defect of the debugger based on the cross validation of different debugging levels as claimed in claim 1, wherein in step 4), the method uses the predefined relationship between the source code level single step debugging and the instruction level single step debugging to validate the hit source code line number, instruction address, variable value, etc., and if the predefined relationship is violated, the debugger is considered to be the defect of the debugger. Here, the predefined relationship between different debugging levels may be a relationship that needs to be maintained between any two different debugging levels for debugging dynamic debugging information running the same program record.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211161594.5A CN115470138A (en) | 2022-09-22 | 2022-09-22 | Debugger defect detection method based on different debugging levels cross validation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211161594.5A CN115470138A (en) | 2022-09-22 | 2022-09-22 | Debugger defect detection method based on different debugging levels cross validation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115470138A true CN115470138A (en) | 2022-12-13 |
Family
ID=84335869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211161594.5A Pending CN115470138A (en) | 2022-09-22 | 2022-09-22 | Debugger defect detection method based on different debugging levels cross validation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115470138A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116126698A (en) * | 2022-12-29 | 2023-05-16 | 中国人民解放军国防科技大学 | Run-time configuration updating defect detection method based on metamorphic test |
| CN116955118A (en) * | 2023-09-19 | 2023-10-27 | 统信软件技术有限公司 | Performance analysis method, system, computing device and storage medium |
-
2022
- 2022-09-22 CN CN202211161594.5A patent/CN115470138A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116126698A (en) * | 2022-12-29 | 2023-05-16 | 中国人民解放军国防科技大学 | Run-time configuration updating defect detection method based on metamorphic test |
| CN116955118A (en) * | 2023-09-19 | 2023-10-27 | 统信软件技术有限公司 | Performance analysis method, system, computing device and storage medium |
| CN116955118B (en) * | 2023-09-19 | 2023-12-29 | 统信软件技术有限公司 | Performance analysis method, system, computing device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gyimesi et al. | Bugsjs: a benchmark of javascript bugs | |
| CN115470138A (en) | Debugger defect detection method based on different debugging levels cross validation | |
| US7503037B2 (en) | System and method for identifying bugs in software source code, using information from code coverage tools and source control tools to determine bugs introduced within a time or edit interval | |
| Chen et al. | Star: Stack trace based automatic crash reproduction via symbolic execution | |
| Skarin et al. | GOOFI-2: A tool for experimental dependability assessment | |
| Leitner et al. | Contract driven development= test driven development-writing test cases | |
| Küçük et al. | Improving fault localization by integrating value and predicate based causal inference techniques | |
| US20080256393A1 (en) | Detecting unexpected impact of software changes using coverage analysis | |
| Yang et al. | Hunting for bugs in code coverage tools via randomized differential testing | |
| CN110688313B (en) | Fault injection method for software testing under VxWorks operating system | |
| CN110704314A (en) | Fault injection method for embedded software test | |
| Silva et al. | Flacoco: Fault localization for java based on industry-grade coverage | |
| Laurent et al. | The MPI bugs initiative: a framework for MPI verification tools evaluation | |
| CN110704315B (en) | Fault injection device for embedded software test | |
| Zhang et al. | Automated Breakpoint Generation for Debugging. | |
| Yang et al. | Automatic self-validation for code coverage profilers | |
| Zamfir et al. | Automated debugging for arbitrarily long executions | |
| CN110633199A (en) | Testing apparatus, method, and medium for block chain supporting intelligent contract | |
| CN105630678A (en) | Intelligent ammeter software reliability detector and detection method for intelligent ammeter software reliability | |
| Ficco et al. | Bug Localization in Test‐Driven Development | |
| CN112685320A (en) | Software defect repairing method and device based on multiple candidate programs | |
| Chen et al. | UnitFL: A fault localization tool integrated with unit test | |
| CN115080978A (en) | Runtime vulnerability detection method and system based on fuzzy test | |
| CN115658507A (en) | Analysis system for real-time embedded software source code coverage rate | |
| CN113836023A (en) | Compiler security testing method based on system structure cross check |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |