US20020010879A1 - Apparatus and method for fast code coverage analysis - Google Patents
Apparatus and method for fast code coverage analysis Download PDFInfo
- Publication number
- US20020010879A1 US20020010879A1 US09/153,389 US15338998A US2002010879A1 US 20020010879 A1 US20020010879 A1 US 20020010879A1 US 15338998 A US15338998 A US 15338998A US 2002010879 A1 US2002010879 A1 US 2002010879A1
- Authority
- US
- United States
- Prior art keywords
- code
- blocks
- executed
- time
- computer program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
Definitions
- the present invention generally relates to an apparatus and method for efficient and fast code coverage analysis.
- code coverage analysis is a technique that checks which program code blocks have been executed during testing.
- the result of a code coverage analysis can be processed in many different ways.
- the one percentage number can be used directly.
- Another way is a map of where the covered code is located.
- the percentages cannot be combined by adding the results as there will be a large amount of duplicated coverage.
- a map of the code blocks hit must be kept for the first run case and updates must be made to the map when new test results are combined. The percentage can then be computed after obtaining cumulated results in the map.
- One of the methods to check if all the code in the program has been tested is to utilize the inclusion of a display message indicating when each block of code is entered, thereby enabling the developer to test which blocks of code have been entered and tested.
- the greatest problem with the inclusion of display messages in the first line of each block of code is the developer's time to include a display statement at the beginning of each block of code, and the time to actually execute the modified program. These extra instructions slow down the execution a lot due to input/output (I/O) operations and plainly due to more instructions having to be executed.
- I/O input/output
- a further overhead factor is that each program must be recompiled or reassembled and linked with the additional display commands.
- another problem with this method is the additional paper resource that is required to trace where the execution of the program is currently situated.
- Another method of code coverage analysis is to include instead a write to file a statement in the beginning of each code block.
- the output file shows each portion of the software program entered.
- the present invention is generally directed to an apparatus and method for fast and efficient code coverage analysis.
- the present invention for fast code coverage analysis utilizes a technique that provides for capturing an event whenever a block of code is visited for the first time. This allows for generating an event only once during numerous executions of a code block. The generation of only one event will provide for execution time close to the speed of the original source code. The speed of the code coverage analysis execution is now provided in an efficient way.
- the present invention further utilizes an apparatus and method where each time a branch is executed, a high level control looks up the branch target program counter in the code cache in order to see if there is a code block corresponding to the branch target program counter. If the program block is found within the code cache, then the high level control determines if that code block has already been executed and proceeds with further execution of the copied source code in code cache.
- the high level control determines that this is the first time that that code block has been executed in the original source program and copies that code block from the original source program into the code cache and provides notification of the first execution of the block of code. This allows a developer to calculate the amount of lines executed in the test run of the program as compared to the total amount of code lines in the original source code.
- FIG. 1 is a block diagram of a user system showing the high level control of the present invention and original source program within the memory area.
- FIG. 2 is a flow chart of a well known prior art method to perform code coverage analysis.
- FIG. 3 is a block diagram of the system of the present invention showing the interaction between the program counter, original source code, new program counter, code cache, and the high level control as shown in FIG. 1.
- FIG. 4 is a flow chart of the preferred method to perform the fast code coverage analysis within the high level control as shown in FIGS. 1 and 4.
- FIG. 5 is a flow chart of the process that translates the program code and puts the program code into the code cache as shown in FIG. 4.
- FIG. 6 is a flow chart of the process that determines if a target address within the code cache has been established and some branches can be backpatched.
- FIG. 7A is a block diagram showing the translation of the original source code to the code that is put in the code cache for a direct unconditional branch.
- FIG. 7B is a block diagram showing the translation from the original source code to the code put in the code cache for a direct conditional branch.
- FIG. 7C is a block diagram showing the original program source code and the translated code put into the code cache for an indirect unconditional branch.
- FIG. 7D is a block diagram showing the original source code and translated code placed into the code cache for an indirect conditional branch.
- FIG. 8A is a block diagram showing the original source code for an example of a direct, unconditional branch of the present invention.
- FIG. 8B is a block diagram showing the contents of the code cache of the translation of part of the source code for the example of a direct, unconditional branch as shown in FIG. 8A.
- FIG. 8C is a block diagram showing the translated code within the code cache when the translation for the instruction at T 0 is added to the code for the example of a direct, unconditional branch as shown in FIG. 8A.
- FIG. 8D is a block diagram of the translated original source code within the code cache where a branch is backpatched for the example of a direct, unconditional branch as shown in FIG. 8A.
- FIG. 9A is the original source code for an example of an indirect conditional branch.
- FIG. 9B is a block diagram showing the contents of the code cache when part of the original source code has been translated for the example of an indirect conditional branch as shown in FIG. 9A.
- FIG. 9C shows the translated original source code within the code cache prior to a branch statement being backpatched for the example of an indirect conditional branch as shown in FIG. 9A.
- FIG. 9D is a block diagram of the original source code within the code cache after a branch statement is backpatched for the example of an indirect conditional branch as shown in FIG. 9A.
- computer systems 12 today generally comprise a processor 21 , and memory 31 (e.g., RAM, ROM, hard disk, CD-ROM, etc.) with an operating system 32 .
- the processor 21 accepts program code 62 and data from the memory 31 over the local interface 23 , for example, a bus(es).
- Direction from the user can be signaled by using input devices, for example, a mouse 24 and a keyboard 25 .
- the actions input and result output are displayed on the display terminal 26 .
- FIG. 2 Illustrated in FIG. 2 is the prior art methodology of performing a code coverage analysis.
- the original source code is preprocessed to add a write to file or display instruction at each basic block of code at step 51 .
- This can be very time consuming if performed by the program developer or it may be possible to automate the process by utilizing a special routine to insert the write to file instruction within each basic block of code.
- the modified source code is executed at step 52 .
- a comparison is then performed with the output file or display, and the actual program source code to determine a percentages of coverage for the program test at step 53 .
- the write to file or display methods causes increased developer time to insert the write to file or display instructions into the program as well as time required to recompile or reassemble and link the program.
- the modified program will always runs much slower than the original program as a result of the additional instructions and input/output processing required for each block.
- High level control 60 acquires the address of the next instruction to be executed from the program counter 61 .
- the original source code 62 is provided to the high level control 60 , which performs a translation of the original source code 62 and inserts this transformed program code into the code cache 63 .
- a new program counter (NPC) 64 which points to the beginning of the translation is returned for execution.
- the method of the present invention performed by the high level control is hereinafter explained in detail with regard to FIGS. 4, 5 and 6 .
- FIG. 4 is a flow chart of the code coverage analysis method of the present invention.
- the high level control 60 process initializes the backpatch pointer “B” to null at step 71 . Backpatching is further explained hereinafter.
- the high level control 60 then performs a lookup of the translation for program counter 61 in the code cache 63 to get a corresponding new program counter 64 . If no translation is found in the code cache 63 , the high level control 60 translates the original source code 62 and places the translated code into code cache 63 at step 76 .
- the translation of the original source code and placing of the translated code within the code cache is herein further defined with reference to FIG. 5.
- the high level control 60 checks if backpatching the “B” pointer is necessary at step 81 .
- Backpatching involves the process of modifying the destination of a branch address to a different location so that future lookup of the translation for the destination is avoided. The determination of backpatching is herein further defined with reference to FIG. 6.
- the high level control 60 allows the execution of the translated code pointed to by the new program counter 64 within the code cache 63 until reaching a branch that is not backpatched at step 82 .
- the high level control 60 next moves the branch target to the program counter 61 at step 84 and returns to step 72 to repeat the process.
- FIG. 5 Illustrated in FIG. 5 is the translation of the original source code 62 (step 76 of FIG. 4) and the placing of the translated code into the code cache 63 .
- the high level control 60 copies a basic block of the original source code 62 into the code cache 63 at step 91 .
- the high level control 60 changes the branch target address of the branch instruction at the end of the basic block of source code to an address within the high level control 60 at step 92 .
- the high level control 60 adds an instruction before the branch to push the original target address on to the stack at step 93 .
- the high level control 60 registers the translated code in the code cache's lookup table at step 94 , and provides a notification event to mark the first execution of the current block of code at step 95 .
- the notification can be done in the form of writing to a file of the address range or the source line range of the block.
- This file which can be called the verification file, can be used in post processing to figure out lines of source code executed.
- An alternative approach involves dumping the content of the code cache's lookup table, which has one entry for each executed code block, to the verification file after the program being tested finishes execution.
- the high level control 60 has the new program counter 64 point to the translated copy of the original source code 62 residing within the code cache 63 at step 99 , and returns to step 81 in FIG. 4, for further processing.
- FIG. 6 Illustrated in FIG. 6 is a flow chart of the process for backpatching ( 81 of FIG. 4) the “B” pointer if necessary.
- the high level control 60 first tests if the backpatch pointer “B” is equal to null at step 101 . If the backpatch pointer “B” is equal to null, the high level control 60 continues the execution of the translated source code by returning to step 82 from step 109 .
- the backpatch pointer “B” is not equal to null, then the high level control backpatches the target address of branch “B” to be the address pointed to by the new program counter 64 .
- the backpatch testing process then returns to FIG. 4 at step 82 .
- FIGS. 7A through 7D show examples of four respective branches and the translated source code for each of the four different types of branch conditions.
- FIG. 7A shows an example of a translation of a direct unconditional branch.
- the direct unconditional branch within the original source code 62 is shown to be a jump to the instruction referenced by label T 1 at block 111 .
- the jump to the instruction at the address referenced by label T 1 in the original source code at block 111 is translated to pushing the address of the instruction referenced by the label T 1 onto the stack and jumping to the high level control 60 , at step 84 (IG. 4 ), in block 112 .
- the stack is popped to reveal the address at label T 1 , and then the high level control 60 jumps to step 72 (FIG. 4) for continued processing.
- FIG. 7B shows an example of a translation of a direct conditional branch translation.
- a direct conditional branch is a jump to an instruction at the address referenced by label T 1 if Register 3 is less than zero, such as the instruction within the original source code 62 illustrated in block 113 .
- the translated code in the code cache is shown in block 114 .
- the translated code tests if R 3 is less than zero, and if so, the address of the instruction at the address referenced by label T 1 is placed on the stack and the method of the program step jumps to the high level control 60 , at step 84 (FIG. 4), in block 114 .
- step 84 the stack is popped to reveal either the address at label T 1 or the fall-through target address, and then the high level control 60 jumps to step 72 (FIG. 4) for continued processing.
- FIG. 7C shows an example of a translation of an indirect unconditional branch.
- Block 115 illustrates a jump to the address pointed to by the contents of Register 1 .
- This indirect unconditional branch is translated to put the address that R 1 points to on the stack and jumping to the high level control 60 in block 116 .
- the stack is popped to reveal the address pointed to by R 1 , and then the high level control 60 jumps to step 72 (FIG. 4) for continued processing.
- FIG. 7D shows an example of a translation of an indirect conditional branch.
- Block 117 shows a jump to the address that R 1 points to if R 3 is less than 0.
- This code is translated and is reflected in the code displayed in block 118 .
- the code directly translates to the instructions'conditional statement that if R 3 is less than zero, then the address that R 1 points to is pushed onto the stack and the program then jumps to the high level control 60 . If R 3 is not less than zero, then the program pushes the address of the next instruction in the original source code onto the stack and then jumps to the high level control 60 in block 118 .
- the stack is popped to reveal the address at label T 1 or the fall-through target address, and then the high level control 60 jumps to step 72 (FIG. 4) for continued processing.
- FIGS. 8A through 8D show an example of a direct unconditional branch.
- the direct unconditional branch in the original source code is shown in FIG. 8A.
- How the direct unconditional branch is translated into the code cache is shown in FIG. 8B.
- the execution of the translated code in the code cache is shown in FIG. 8C.
- the backpatching within the translated code of the code cache is shown in FIG. 8D.
- FIG. 8A Illustrated in FIG. 8A is the original source code 122 of an example program for a direct unconditional branch.
- the original source code within the example for the direct unconditional branch in FIG. 8A is translated and placed into the contents of code cache 63 and is illustrated in FIG. 8B.
- FIG. 8B Illustrated in FIG. 8B is the content of the code cache after the code block T 1 of the original source code, for the example program, for a direct unconditional branch in FIG. 8A, is translated 124 and placed into the code cache 63 .
- the tested example program is about to execute a statement at line item T 0 (i.e., data label T 0 ) of the original source code, therefore the PC 61 points to T 0 when executing step 72 of FIG. 4.
- step 72 the lookup table shows that there is no translation for T 0 so the process moves to 73 to see if the new program counter is found.
- step 76 is executed. Step 76 translates the source code 62 and puts the translated code into the code cache 63 as defined with regard to FIG. 5.
- FIG. 8C Illustrated in FIG. 8C is the content of the code cache after the code starting at instruction T 0 is translated and placed into the code cache 63 . This translation causes the addition of the instructions at S 201 through S 213 as shown in FIG. 8C.
- the new program counter 64 is changed to point to the S 201 instruction and then the system proceeds with the execution at step 81 in FIG. 4. Then the new program counter 64 is backpatched to the last branch that has T 0 as the target at step 81 .
- the translated code is pointed to by the new program counter 64 which points to S 201 within FIG. 8C and causes the code analysis system to determine that a branch that is not backpatched has been encountered and the program flows to step 84 which is shown as S 213 within FIG. 8C.
- step 84 the backpatch pointer “B” points to S 210 and the branch targeted by instruction at label T 1 is popped from the stack and assigned to the program counter 61 .
- the high level control lookup program 60 counter in the code cache 63 next receives the corresponding new program counter at step 72 .
- step 72 the program looks-up the translation for the instruction T 1 item and finds the translation at S 101 as shown in FIG. 8B through 8D.
- the high level control 60 then assigns the address at label T 1 to the new program counter 64 and executes the check at step 73 . Since the new program counter 64 is found within the code cache 63 at step 73 , the high level control 60 proceeds to step 81 to check if backpatching of a branch is necessary.
- the high level control 60 backpatches the branch pointed to by backpatch pointer “B”, which is in this example is S 210 , using the new program counter 64 which points to S 101 .
- the code cache has the contents as shown in FIG. 8D. Note the change at label location S 210 between FIGS. 8C and 8D.
- the changes at label location S 210 include changing the jump location from S 211 , which in the original unexecuted translated code, to now jump to the instruction location pointed to by S 101 , since the code block at S 101 has been executed at least once.
- the high level control 60 executes the program pointed to by the new program counter 64 which points to S 101 and continues to loop through the code. From now on, every time the high level control 60 executes the translation for T 0 at S 201 , the executed test program jumps to instruction location pointed to by S 101 at S 210 without the need of a lookup for data label T 1 .
- FIG. 9A Illustrated in FIG. 9A is an example program 132 of the original source code 62 , including an indirect conditional branch, and in FIG. 9B is the translated source code as a content of the code cache 63 .
- the code cache In the code cache is a translation of the instruction label T 1 at S 101 and a translation of label T 2 at S 201 .
- the program counter 61 points to the data label T 0 during execution of step 72 in FIG. 4.
- the lookup table shows no translation for the label T 0 . Since the data label T 0 is not found in the new program counter 64 , the high level control 60 determines that this is the first time this new block of code has been executed, and moves to step 76 to translate the new block of code. The translated new block of code is written into the code cache as defined in detail with regard to FIG. 5.
- FIG. 9C Illustrated in FIG. 9C, is the high level control 60 translation of the block of code starting with the label T 0 .
- the high level control 60 puts in the instruction translation at labels S 301 to S 317 at step 76 .
- the high level control 60 also changes the new program counter 64 to point to the label S 301 and then moves to step 81 to decide if backpatching is necessary.
- the new program counter 64 is backpatched to the last branch instruction that has T 0 as the target address at step 81 .
- the high level control 60 executes the block of code pointed to by the new program counter 64 which points to label S 30 1 .
- the example program encounters an indirect conditional branch and must determine if R 3 is less than zero at label S 310 .
- R 3 is not less than zero and the high level control 60 falls through to the instruction at S 311 .
- the high level control 60 then jumps from label S 311 to the instruction at S 315 , because R 3 is greater than or equal to zero.
- the instruction at S 315 pushes the address pointed to by the label T 1 , the fall-through target address, onto the stack and proceed to assign the address S 311 to backpatch pointer “B” and jump to step 84 of FIG. 4, as shown in step S 315 of FIG. 9C.
- the backpatch pointer “B” points to address S 311 and the high level control 60 pops the branch target from the stack into the program counter 61 .
- the high level control next executes step 72 which causes a lookup of the translation for label T 1 and finds the translation at address S 101 .
- the high level control 60 then assigns label T 1 to the new program counter 64 , and the processing proceeds to step 81 of FIG. 4.
- the high level control 60 backpatches the branch pointed to by the backpatch “B” pointer which contains the pointer to S 311 using the new program counter which points to the instruction at S 101 .
- the code cache now has the contents shown in FIG. 9D.
- the fast code coverage analysis program which comprises an ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
- a “computer-readable medium” can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- the computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a nonexhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (magnetic), a read-only memory (ROM) (magnetic), an erasable programmable read-only memory (EPROM or Flash memory) (magnetic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical).
- an electrical connection electronic having one or more wires
- a portable computer diskette magnetic
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- CDROM portable compact disc read-only memory
- the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Devices For Executing Special Programs (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
Description
- 1. Field of the Invention
- The present invention generally relates to an apparatus and method for efficient and fast code coverage analysis. Generally, code coverage analysis is a technique that checks which program code blocks have been executed during testing.
- 2. Description of Related Art
- As is known in the computer and software arts, when a software program is developed, it will contain errors. As a consequence, developers of the software program utilize many techniques in which to check the correctness of the program and to diagnose errors. One of the techniques that developers will often use is a benchmark test to check the code coverage of the program. The best benchmark tests check as many different code blocks and combinations as possible (i.e., code coverage).
- The result of a code coverage analysis can be processed in many different ways. One is a single percentage number showing the amount of code covered in the test. For a single test run, the one percentage number can be used directly.
- Another way is a map of where the covered code is located. For multiple test runs, the percentages cannot be combined by adding the results as there will be a large amount of duplicated coverage. In multiple test run cases, a map of the code blocks hit must be kept for the first run case and updates must be made to the map when new test results are combined. The percentage can then be computed after obtaining cumulated results in the map.
- One of the methods to check if all the code in the program has been tested (i.e., code coverage) is to utilize the inclusion of a display message indicating when each block of code is entered, thereby enabling the developer to test which blocks of code have been entered and tested. The greatest problem with the inclusion of display messages in the first line of each block of code is the developer's time to include a display statement at the beginning of each block of code, and the time to actually execute the modified program. These extra instructions slow down the execution a lot due to input/output (I/O) operations and plainly due to more instructions having to be executed. A further overhead factor is that each program must be recompiled or reassembled and linked with the additional display commands. Furthermore, another problem with this method is the additional paper resource that is required to trace where the execution of the program is currently situated.
- Another method of code coverage analysis, similar to the displaying of a message at the beginning of each block in the software program, is to include instead a write to file a statement in the beginning of each code block. The output file shows each portion of the software program entered.
- Problems with this write to file method, again as stated above for the display technique, include the extended program execution time, the developer's time to include a write to file statement at the beginning of each block of code, and the time required to recompile, reassemble, and execute a link program. Also, the amount of disk storage required for such write to file could be an enormous resource demand.
- Heretofore, software developers have lacked an apparatus and method for accomplishing code coverage analysis in an efficient way.
- Certain objects, advantages and novel features of the invention will be set forth in part in the description that follows and in part will become apparent to those skilled in the art upon examination of the following description or may be learned with the practice of the invention. The objects, features and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly recited in the appended claims.
- To achieve the advantages and novel features, the present invention is generally directed to an apparatus and method for fast and efficient code coverage analysis. The present invention for fast code coverage analysis utilizes a technique that provides for capturing an event whenever a block of code is visited for the first time. This allows for generating an event only once during numerous executions of a code block. The generation of only one event will provide for execution time close to the speed of the original source code. The speed of the code coverage analysis execution is now provided in an efficient way.
- The present invention further utilizes an apparatus and method where each time a branch is executed, a high level control looks up the branch target program counter in the code cache in order to see if there is a code block corresponding to the branch target program counter. If the program block is found within the code cache, then the high level control determines if that code block has already been executed and proceeds with further execution of the copied source code in code cache.
- If the high level control does not find the code within the code cache, then the high level control determines that this is the first time that that code block has been executed in the original source program and copies that code block from the original source program into the code cache and provides notification of the first execution of the block of code. This allows a developer to calculate the amount of lines executed in the test run of the program as compared to the total amount of code lines in the original source code.
- The accompanying drawings incorporated in and forming a part of the specification illustrate several aspects of the present invention, and together with the description, serve to explain the principles of the invention. In the drawings:
- FIG. 1 is a block diagram of a user system showing the high level control of the present invention and original source program within the memory area.
- FIG. 2 is a flow chart of a well known prior art method to perform code coverage analysis.
- FIG. 3 is a block diagram of the system of the present invention showing the interaction between the program counter, original source code, new program counter, code cache, and the high level control as shown in FIG. 1.
- FIG. 4 is a flow chart of the preferred method to perform the fast code coverage analysis within the high level control as shown in FIGS. 1 and 4.
- FIG. 5 is a flow chart of the process that translates the program code and puts the program code into the code cache as shown in FIG. 4.
- FIG. 6 is a flow chart of the process that determines if a target address within the code cache has been established and some branches can be backpatched.
- FIG. 7A is a block diagram showing the translation of the original source code to the code that is put in the code cache for a direct unconditional branch.
- FIG. 7B is a block diagram showing the translation from the original source code to the code put in the code cache for a direct conditional branch.
- FIG. 7C is a block diagram showing the original program source code and the translated code put into the code cache for an indirect unconditional branch.
- FIG. 7D is a block diagram showing the original source code and translated code placed into the code cache for an indirect conditional branch.
- FIG. 8A is a block diagram showing the original source code for an example of a direct, unconditional branch of the present invention.
- FIG. 8B is a block diagram showing the contents of the code cache of the translation of part of the source code for the example of a direct, unconditional branch as shown in FIG. 8A.
- FIG. 8C is a block diagram showing the translated code within the code cache when the translation for the instruction at T0 is added to the code for the example of a direct, unconditional branch as shown in FIG. 8A.
- FIG. 8D is a block diagram of the translated original source code within the code cache where a branch is backpatched for the example of a direct, unconditional branch as shown in FIG. 8A.
- FIG. 9A is the original source code for an example of an indirect conditional branch.
- FIG. 9B is a block diagram showing the contents of the code cache when part of the original source code has been translated for the example of an indirect conditional branch as shown in FIG. 9A.
- FIG. 9C shows the translated original source code within the code cache prior to a branch statement being backpatched for the example of an indirect conditional branch as shown in FIG. 9A.
- FIG. 9D is a block diagram of the original source code within the code cache after a branch statement is backpatched for the example of an indirect conditional branch as shown in FIG. 9A.
- Reference will now be made in detail to the description of the invention as illustrated in the drawings. Although the invention will be described in connection with these drawings, there is no intent to limit it to the embodiment or embodiments disclosed therein. On the contrary, the intent is to include all alternatives, modifications, and equivalents included within the spirit and scope of the invention as defined by the appended claims.
- As illustrated in FIG. 1,
computer systems 12 today generally comprise aprocessor 21, and memory 31 (e.g., RAM, ROM, hard disk, CD-ROM, etc.) with anoperating system 32. Theprocessor 21 acceptsprogram code 62 and data from thememory 31 over thelocal interface 23, for example, a bus(es). Direction from the user can be signaled by using input devices, for example, amouse 24 and akeyboard 25. The actions input and result output are displayed on thedisplay terminal 26. - Illustrated in FIG. 2 is the prior art methodology of performing a code coverage analysis. First, the original source code is preprocessed to add a write to file or display instruction at each basic block of code at
step 51. This can be very time consuming if performed by the program developer or it may be possible to automate the process by utilizing a special routine to insert the write to file instruction within each basic block of code. - Next, the modified source code is executed at
step 52. A comparison is then performed with the output file or display, and the actual program source code to determine a percentages of coverage for the program test atstep 53. - As noted above, the write to file or display methods causes increased developer time to insert the write to file or display instructions into the program as well as time required to recompile or reassemble and link the program. The modified program will always runs much slower than the original program as a result of the additional instructions and input/output processing required for each block. There is also a resource requirement of a large amount of disk storage to hold the write to file documentation.
- Illustrated in FIG. 3 is a block diagram of the components of the present invention.
High level control 60 acquires the address of the next instruction to be executed from theprogram counter 61. Theoriginal source code 62 is provided to thehigh level control 60, which performs a translation of theoriginal source code 62 and inserts this transformed program code into thecode cache 63. A new program counter (NPC) 64 which points to the beginning of the translation is returned for execution. The method of the present invention performed by the high level control is hereinafter explained in detail with regard to FIGS. 4, 5 and 6. - FIG. 4 is a flow chart of the code coverage analysis method of the present invention. First, the
high level control 60 process initializes the backpatch pointer “B” to null atstep 71. Backpatching is further explained hereinafter. Thehigh level control 60 then performs a lookup of the translation forprogram counter 61 in thecode cache 63 to get a correspondingnew program counter 64. If no translation is found in thecode cache 63, thehigh level control 60 translates theoriginal source code 62 and places the translated code intocode cache 63 atstep 76. The translation of the original source code and placing of the translated code within the code cache is herein further defined with reference to FIG. 5. - If the translation for the
program counter 61 is found in thecode cache 63 atstep 73, thehigh level control 60 then checks if backpatching the “B” pointer is necessary atstep 81. Backpatching involves the process of modifying the destination of a branch address to a different location so that future lookup of the translation for the destination is avoided. The determination of backpatching is herein further defined with reference to FIG. 6. - Next, the
high level control 60 allows the execution of the translated code pointed to by thenew program counter 64 within thecode cache 63 until reaching a branch that is not backpatched atstep 82. Thehigh level control 60 next moves the branch target to theprogram counter 61 atstep 84 and returns to step 72 to repeat the process. - Illustrated in FIG. 5 is the translation of the original source code62 (
step 76 of FIG. 4) and the placing of the translated code into thecode cache 63. First, thehigh level control 60 copies a basic block of theoriginal source code 62 into thecode cache 63 atstep 91. Thehigh level control 60 changes the branch target address of the branch instruction at the end of the basic block of source code to an address within thehigh level control 60 atstep 92. Thehigh level control 60 adds an instruction before the branch to push the original target address on to the stack atstep 93. - Next, the
high level control 60 registers the translated code in the code cache's lookup table atstep 94, and provides a notification event to mark the first execution of the current block of code atstep 95. The notification can be done in the form of writing to a file of the address range or the source line range of the block. This file, which can be called the verification file, can be used in post processing to figure out lines of source code executed. - An alternative approach involves dumping the content of the code cache's lookup table, which has one entry for each executed code block, to the verification file after the program being tested finishes execution. The
high level control 60 has thenew program counter 64 point to the translated copy of theoriginal source code 62 residing within thecode cache 63 atstep 99, and returns to step 81 in FIG. 4, for further processing. - Illustrated in FIG. 6 is a flow chart of the process for backpatching (81 of FIG. 4) the “B” pointer if necessary. The
high level control 60 first tests if the backpatch pointer “B” is equal to null atstep 101. If the backpatch pointer “B” is equal to null, thehigh level control 60 continues the execution of the translated source code by returning to step 82 fromstep 109. - If the backpatch pointer “B” is not equal to null, then the high level control backpatches the target address of branch “B” to be the address pointed to by the
new program counter 64. The backpatch testing process then returns to FIG. 4 atstep 82. - FIGS. 7A through 7D show examples of four respective branches and the translated source code for each of the four different types of branch conditions.
- FIG. 7A shows an example of a translation of a direct unconditional branch. The direct unconditional branch within the
original source code 62 is shown to be a jump to the instruction referenced by label T1 atblock 111. The jump to the instruction at the address referenced by label T1 in the original source code atblock 111 is translated to pushing the address of the instruction referenced by the label T1 onto the stack and jumping to thehigh level control 60, at step 84 (IG. 4), inblock 112. Atmethod step 84, the stack is popped to reveal the address at label T1, and then thehigh level control 60 jumps to step 72 (FIG. 4) for continued processing. - FIG. 7B shows an example of a translation of a direct conditional branch translation. A direct conditional branch is a jump to an instruction at the address referenced by label T1 if
Register 3 is less than zero, such as the instruction within theoriginal source code 62 illustrated inblock 113. The translated code in the code cache is shown inblock 114. The translated code tests if R3 is less than zero, and if so, the address of the instruction at the address referenced by label T1 is placed on the stack and the method of the program step jumps to thehigh level control 60, at step 84 (FIG. 4), inblock 114. If R3 is not less than zero, then the instruction pushes the address of the next instruction in the original source code on the stack and jumps to thehigh level control 60, at step 84 (FIG. 4), inblock 114. Atmethod step 84, the stack is popped to reveal either the address at label T1 or the fall-through target address, and then thehigh level control 60 jumps to step 72 (FIG. 4) for continued processing. - FIG. 7C shows an example of a translation of an indirect unconditional branch.
Block 115 illustrates a jump to the address pointed to by the contents ofRegister 1. This indirect unconditional branch is translated to put the address that R1 points to on the stack and jumping to thehigh level control 60 inblock 116. Atmethod step 84, the stack is popped to reveal the address pointed to by R1, and then thehigh level control 60 jumps to step 72 (FIG. 4) for continued processing. - FIG. 7D shows an example of a translation of an indirect conditional branch.
Block 117 shows a jump to the address that R1 points to if R3 is less than 0. This code is translated and is reflected in the code displayed inblock 118. The code directly translates to the instructions'conditional statement that if R3 is less than zero, then the address that R1 points to is pushed onto the stack and the program then jumps to thehigh level control 60. If R3 is not less than zero, then the program pushes the address of the next instruction in the original source code onto the stack and then jumps to thehigh level control 60 inblock 118. Atmethod step 84, the stack is popped to reveal the address at label T1 or the fall-through target address, and then thehigh level control 60 jumps to step 72 (FIG. 4) for continued processing. - FIGS. 8A through 8D show an example of a direct unconditional branch. The direct unconditional branch in the original source code is shown in FIG. 8A. How the direct unconditional branch is translated into the code cache is shown in FIG. 8B. The execution of the translated code in the code cache is shown in FIG. 8C. The backpatching within the translated code of the code cache is shown in FIG. 8D.
- Illustrated in FIG. 8A is the
original source code 122 of an example program for a direct unconditional branch. First, the original source code within the example for the direct unconditional branch in FIG. 8A is translated and placed into the contents ofcode cache 63 and is illustrated in FIG. 8B. - Illustrated in FIG. 8B is the content of the code cache after the code block T1 of the original source code, for the example program, for a direct unconditional branch in FIG. 8A, is translated 124 and placed into the
code cache 63. In this example for direct unconditional branch, it will be assumed that the tested example program is about to execute a statement at line item T0 (i.e., data label T0) of the original source code, therefore thePC 61 points to T0 when executingstep 72 of FIG. 4. - At
step 72 the lookup table shows that there is no translation for T0 so the process moves to 73 to see if the new program counter is found. Upon finding the new program counter is not being found because there is no translation of the line T0 item in the lookup table,step 76 is executed.Step 76 translates thesource code 62 and puts the translated code into thecode cache 63 as defined with regard to FIG. 5. - Illustrated in FIG. 8C is the content of the code cache after the code starting at instruction T0 is translated and placed into the
code cache 63. This translation causes the addition of the instructions at S201 through S213 as shown in FIG. 8C. - The
new program counter 64 is changed to point to the S201 instruction and then the system proceeds with the execution atstep 81 in FIG. 4. Then thenew program counter 64 is backpatched to the last branch that has T0 as the target atstep 81. Atstep 82, the translated code is pointed to by thenew program counter 64 which points to S201 within FIG. 8C and causes the code analysis system to determine that a branch that is not backpatched has been encountered and the program flows to step 84 which is shown as S213 within FIG. 8C. - In
step 84, the backpatch pointer “B” points to S210 and the branch targeted by instruction at label T1 is popped from the stack and assigned to theprogram counter 61. The high levelcontrol lookup program 60 counter in thecode cache 63 next receives the corresponding new program counter atstep 72. - At
step 72 the program looks-up the translation for the instruction T1 item and finds the translation at S101 as shown in FIG. 8B through 8D. Thehigh level control 60 then assigns the address at label T1 to thenew program counter 64 and executes the check atstep 73. Since thenew program counter 64 is found within thecode cache 63 atstep 73, thehigh level control 60 proceeds to step 81 to check if backpatching of a branch is necessary. - At
step 81 thehigh level control 60 backpatches the branch pointed to by backpatch pointer “B”, which is in this example is S210, using thenew program counter 64 which points to S101. Now the code cache has the contents as shown in FIG. 8D. Note the change at label location S210 between FIGS. 8C and 8D. The changes at label location S210, include changing the jump location from S211, which in the original unexecuted translated code, to now jump to the instruction location pointed to by S101, since the code block at S101 has been executed at least once. - At
step 82 in FIG. 4 thehigh level control 60 executes the program pointed to by thenew program counter 64 which points to S101 and continues to loop through the code. From now on, every time thehigh level control 60 executes the translation for T0 at S201, the executed test program jumps to instruction location pointed to by S101 at S210 without the need of a lookup for data label T1. - Illustrated in FIG. 9A is an
example program 132 of theoriginal source code 62, including an indirect conditional branch, and in FIG. 9B is the translated source code as a content of thecode cache 63. In the code cache is a translation of the instruction label T1 at S101 and a translation of label T2 at S201. Within an example of a tested program that is about to execute the statement at label T0, and therefore theprogram counter 61 points to the data label T0 during execution ofstep 72 in FIG. 4. - At
step 72 in FIG. 4, the lookup table shows no translation for the label T0. Since the data label T0 is not found in thenew program counter 64, thehigh level control 60 determines that this is the first time this new block of code has been executed, and moves to step 76 to translate the new block of code. The translated new block of code is written into the code cache as defined in detail with regard to FIG. 5. - Illustrated in FIG. 9C, is the
high level control 60 translation of the block of code starting with the label T0. After translation thehigh level control 60 puts in the instruction translation at labels S301 to S317 atstep 76. Thehigh level control 60 also changes thenew program counter 64 to point to the label S301 and then moves to step 81 to decide if backpatching is necessary. Thenew program counter 64 is backpatched to the last branch instruction that has T0 as the target address atstep 81. Atstep 82, thehigh level control 60 executes the block of code pointed to by thenew program counter 64 which points tolabel S30 1. - During the execution of the next block of code, the example program encounters an indirect conditional branch and must determine if R3 is less than zero at label S310. For purposes of this example, it will be assumed that R3 is not less than zero and the
high level control 60 falls through to the instruction at S311. Thehigh level control 60 then jumps from label S311 to the instruction at S315, because R3 is greater than or equal to zero. The instruction at S315 pushes the address pointed to by the label T1, the fall-through target address, onto the stack and proceed to assign the address S311 to backpatch pointer “B” and jump to step 84 of FIG. 4, as shown in step S315 of FIG. 9C. - At this point, the backpatch pointer “B” points to address S311 and the
high level control 60 pops the branch target from the stack into theprogram counter 61. The high level control next executesstep 72 which causes a lookup of the translation for label T1 and finds the translation at address S101. Thehigh level control 60 then assigns label T1 to thenew program counter 64, and the processing proceeds to step 81 of FIG. 4. Atstep 81, thehigh level control 60 backpatches the branch pointed to by the backpatch “B” pointer which contains the pointer to S311 using the new program counter which points to the instruction at S101. At this point, the code cache now has the contents shown in FIG. 9D. - Note the change of the jump destination at instruction S311 between FIGS. 9C and 9D. While executing at
step 82, thehigh level control 60 executes code pointed to by thenew program counter 64 at address S101 and proceeds as before. At this point in the example, every time the code analysis executes the translation for label T0 at address S301 and if R3 is not less than zero, the program jumps to address SIOI from address S311 without the need of a lookup of label T1. If on the other hand, R3 is less than zero at instruction at address S310 and R1 points to T2, thehigh level control 60 will not backpatch address S310 even though the code lookup has a translation for T2 at address S201. This is because the contents of R1 can be changed every time the execution processes the address at address S312. - The fast code coverage analysis program, which comprises an ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “computer-readable medium” can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a nonexhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (magnetic), a read-only memory (ROM) (magnetic), an erasable programmable read-only memory (EPROM or Flash memory) (magnetic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical).
- Note that the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
- The foregoing description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obvious modifications or variations are possible in light of the above teachings.
- The embodiment or embodiments discussed were chosen and described to provide the best illustration of the principles of the invention and its practical application to enable one of ordinary skill in the art to utilize the invention in various embodiments and with various modifications as are suited to the particular use contemplated. All such modifications and variations are within the scope of the invention as determined by the appended claims when interpreted in accordance with the breadth to which they are fairly and legally entitled.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/153,389 US6438725B2 (en) | 1998-09-15 | 1998-09-15 | Apparatus and method for fast code coverage analysis |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/153,389 US6438725B2 (en) | 1998-09-15 | 1998-09-15 | Apparatus and method for fast code coverage analysis |
Publications (2)
Publication Number | Publication Date |
---|---|
US20020010879A1 true US20020010879A1 (en) | 2002-01-24 |
US6438725B2 US6438725B2 (en) | 2002-08-20 |
Family
ID=22547016
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/153,389 Expired - Fee Related US6438725B2 (en) | 1998-09-15 | 1998-09-15 | Apparatus and method for fast code coverage analysis |
Country Status (1)
Country | Link |
---|---|
US (1) | US6438725B2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070094001A1 (en) * | 2005-06-28 | 2007-04-26 | Intel Corporation | Method and system for non-intrusive code coverage |
US7493521B1 (en) * | 2005-06-23 | 2009-02-17 | Netapp, Inc. | Apparatus and method for estimating the testing proficiency of a software test according to EMS messages extracted from a code base |
US20100146340A1 (en) * | 2008-12-09 | 2010-06-10 | International Business Machines Corporation | Analyzing Coverage of Code Changes |
US20110138231A1 (en) * | 2009-12-03 | 2011-06-09 | Infineon Technologies Ag | High compression program flow trace |
CN107623863A (en) * | 2017-09-21 | 2018-01-23 | 广州华多网络科技有限公司 | Test of heuristics method, apparatus and server |
CN117171057A (en) * | 2023-11-02 | 2023-12-05 | 沐曦集成电路(上海)有限公司 | System for determining software code coverage rate of chip software and hardware joint simulation stage |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7240303B1 (en) | 1999-11-30 | 2007-07-03 | Synplicity, Inc. | Hardware/software co-debugging in a hardware description language |
US7222315B2 (en) | 2000-11-28 | 2007-05-22 | Synplicity, Inc. | Hardware-based HDL code coverage and design analysis |
US20040025081A1 (en) * | 2002-07-31 | 2004-02-05 | Jorge Gonzalez | System and method for collecting code coverage information before file system is available |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5212794A (en) * | 1990-06-01 | 1993-05-18 | Hewlett-Packard Company | Method for optimizing computer code to provide more efficient execution on computers having cache memories |
US5694540A (en) * | 1994-12-15 | 1997-12-02 | Lucent Technologies Inc. | Automated software regression test and compilation system |
US5758061A (en) * | 1995-12-15 | 1998-05-26 | Plum; Thomas S. | Computer software testing method and apparatus |
US5805795A (en) * | 1996-01-05 | 1998-09-08 | Sun Microsystems, Inc. | Method and computer program product for generating a computer program product test that includes an optimized set of computer program product test cases, and method for selecting same |
US5732210A (en) * | 1996-03-15 | 1998-03-24 | Hewlett-Packard Company | Use of dynamic translation to provide fast debug event checks |
-
1998
- 1998-09-15 US US09/153,389 patent/US6438725B2/en not_active Expired - Fee Related
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7493521B1 (en) * | 2005-06-23 | 2009-02-17 | Netapp, Inc. | Apparatus and method for estimating the testing proficiency of a software test according to EMS messages extracted from a code base |
US20070094001A1 (en) * | 2005-06-28 | 2007-04-26 | Intel Corporation | Method and system for non-intrusive code coverage |
US20100146340A1 (en) * | 2008-12-09 | 2010-06-10 | International Business Machines Corporation | Analyzing Coverage of Code Changes |
US20110138231A1 (en) * | 2009-12-03 | 2011-06-09 | Infineon Technologies Ag | High compression program flow trace |
US8484517B2 (en) * | 2009-12-03 | 2013-07-09 | Infineon Technologies Ag | High compression program flow trace |
CN107623863A (en) * | 2017-09-21 | 2018-01-23 | 广州华多网络科技有限公司 | Test of heuristics method, apparatus and server |
CN117171057A (en) * | 2023-11-02 | 2023-12-05 | 沐曦集成电路(上海)有限公司 | System for determining software code coverage rate of chip software and hardware joint simulation stage |
Also Published As
Publication number | Publication date |
---|---|
US6438725B2 (en) | 2002-08-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7752613B2 (en) | Disambiguation in dynamic binary translation | |
US6662359B1 (en) | System and method for injecting hooks into Java classes to handle exception and finalization processing | |
US7100152B1 (en) | Software analysis system having an apparatus for selectively collecting analysis data from a target system executing software instrumented with tag statements and method for use thereof | |
US7571427B2 (en) | Methods for comparing versions of a program | |
US20030126590A1 (en) | System and method for dynamic data-type checking | |
US20110047532A1 (en) | Methods and apparatuses for selective code coverage | |
US20050187740A1 (en) | System and method for proactive computer virus protection | |
US6298479B1 (en) | Method and system for compiling and linking source files | |
US5634023A (en) | Software mechanism for accurately handling exceptions generated by speculatively scheduled instructions | |
JP2000181725A (en) | Method and system for altering executable code and giving addition function | |
US20050257092A1 (en) | Method and apparatus for identifying access states for variables | |
JPH04225431A (en) | Method for compiling computer instruction for increasing instruction-cache efficiency | |
US20020170034A1 (en) | Method for debugging a dynamic program compiler, interpreter, or optimizer | |
US20050246696A1 (en) | Method and apparatus for hardware awareness of data types | |
US20060277371A1 (en) | System and method to instrument references to shared memory | |
US7500230B2 (en) | Raising native software code | |
US6810519B1 (en) | Achieving tight binding for dynamically loaded software modules via intermodule copying | |
US6438725B2 (en) | Apparatus and method for fast code coverage analysis | |
US6625807B1 (en) | Apparatus and method for efficiently obtaining and utilizing register usage information during software binary translation | |
US7409677B1 (en) | Method and system for creation and use of embedded trace description | |
US5812828A (en) | Function simulation | |
US6374368B1 (en) | Weakest precondition analysis | |
US6728951B1 (en) | System and method for performing automated incremental compilation of computer programs | |
US20020129336A1 (en) | Automatic symbol table selection in a multi-cell environment | |
US10133655B1 (en) | Emulation of target system using JIT compiler and bypassing translation of selected target code blocks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HEWLETT-PACKARD COMPANY, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CHEN, DING-KAI;REEL/FRAME:009542/0404 Effective date: 19981020 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD COMPANY;REEL/FRAME:026945/0699 Effective date: 20030131 |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees | ||
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Expired due to failure to pay maintenance fee |
Effective date: 20140820 |