JP4745728B2 - Debugging support program - Google Patents

Description

  The present invention relates to a technique for supporting debugging of software.

  As a technique for improving the efficiency of development of various software (including firmware), an automated system for software testing has been disclosed for the purpose of improving the efficiency of the testing process among the development processes (for example, Patent Document 1). In the technique described in Patent Document 1, processing such as extraction of test items, creation of a test procedure manual, determination of test results, and the like is automatically executed among tasks in a unit test by a software test automation system.

  By the way, in general software development, the development is divided into a plurality of stages, and the functions to be realized in each stage are determined and sequentially added to the software. In each development phase test, it is confirmed that the added functions work correctly.

  If functions are added step by step, even if an appropriate return value is obtained as a test result at a certain development stage, the new function will be incorporated into the software at the next stage and tested. Results may differ from the test. In the following description, such a phenomenon is referred to as a degradation failure.

  To find the location that caused this degradation failure, compare the source code at the time when the degradation failure was detected with the version of the source code before the failure was detected, and track the relevant location. I have to do it.

Various systems and commands are used as tools for software testing, including degraded troubleshooting. For example, “wget”, a command for downloading source to the test machine from the source code development environment, “cvs / bitkeeper / subversion”, a distributed source management system, and a system for building test execution formats "Ports / portage" and "Autobuild", the system "JUNIT / NUNIT / DejaGNU / Rational Manual Tester" for automatically executing tests, the system "cxref" for taking the difference from the source code of the previous version / global "has been used as a debugging support tool in development.
Japanese Patent Laid-Open No. 9-244921 (abstract, FIG. 1, paragraphs 0013 to 0015)

  In the debugging environment of the prior art, management of source code modification and management of test results are performed separately and are not associated with each other. For this reason, even if the degradation failure can be traced using the above-mentioned support tools, etc., what source code should be actually compared, and where is the location related to degradation in the source code? For example, the system developer had to investigate manually, and it took a lot of time for these operations.

  An object of the present invention is to provide a technique capable of improving the efficiency of software debugging.

  In order to solve the above-described problem, according to the present invention, there is provided a program for supporting a software debugging process, wherein a first procedure for storing source code in a source code storage means is performed prior to the source code storage means. A second procedure for generating and storing comparison result information by comparing the first source code stored in the first source code and the second source code newly stored by the first procedure; A third procedure for converting a source code of the above into an execution format and performing a predetermined test, a test result information obtained by the test, and a test time information indicating the time when the test was performed are stored in association with each other. The fifth procedure for repeatedly executing the first to fourth procedures, the comparison result information corresponding to the designated test result information, the test time information and each source code Program for executing a sixth step to output the acquired utilizing stored time information representing the storage time of the serial source code storing unit, to a computer is provided.

  When the source code (second source code) is stored in the source code storage means, a difference from the previously stored source code (first source code) is generated and held as comparison result information. Keep it. When the result of the test performed on the second source code is not normal despite the result of the test performed on the first source code being normal, 1 and 2 output comparison result information representing the difference between the second source codes. A plurality of versions of source code stored in the source code storage means can be managed in association with the test results for each source code appropriately.

  The comparison result information may be generated when the second source code is stored in the source code storage unit. By using the new source code stored as a trigger for generating the comparison result information, it is possible to reliably compare the source code versions.

  The sixth procedure is such that when the result of the first test is normal and the result of the second test performed after the first test is abnormal, the first test is Comparison result information between the source code stored in the source code storage means immediately before the execution time and the source code stored in the source code storage means immediately before the time when the second test is executed May be output. It is possible to improve the efficiency of the work of identifying the portion where the degradation occurs in the difference between the first source code and the second source code.

  In addition, the sixth procedure may be output by notifying the mail receiving side using electronic mail, or may be output by displaying on a web browser.

  According to the present invention, the efficiency of software debugging can be improved by managing the source code and the test result of the source code in association with each other.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.
FIG. 1 is a system configuration diagram including a debugging support apparatus 1 according to the present embodiment. The system 10 of FIG. 1 includes, for example, a debug support apparatus 1, a source repository 2, an in-house repository 3, a target machine 4 (4A, 4B, 4C), and a test result reference machine 5.

  The debug support apparatus 1 includes a test execution host computer 11, a test source code repository 12, a debug support information storage unit 13, a test script storage unit 14, a target machine execution format storage unit 15, a test result storage unit 16, a test A periodic execution program 17 and a cross referencer 18 are included.

  The source repository 2 and the in-house repository 3 store source code to be tested in the host computer 11 of the debug support apparatus 1. For example, when a company develops a program and uses a part of the source code obtained through the Internet or the like for development, the source code stored in the source repository 2 is generally The source code released to a specific user and stored in the in-house repository 3 is source code developed in the company. The source repository 2 provides necessary source code to the debugging support apparatus 1 according to the present embodiment via a network such as the Internet, and the internal repository 3 via a LAN or the like.

  The target machine 4 is a device for actually executing the developed source code. In the example of the system configuration shown in FIG. 1, the configuration includes three target machines. For example, an evaluation board 4A, an actual machine 4B, and the like are connected to the debug support apparatus 1 according to the present embodiment via a hub. The Tests are executed in these target machines 4A, 4B, and 4C, and the test results are transferred to the debug support apparatus 1 via the hub.

  The test result reference machine 5 includes a terminal device for referring to the test result in the target machine 4 transferred to the debug support apparatus 1 and the analysis result of the test result in the debug support apparatus 1. As shown in FIG. 1, for example, the result may be output to a printer, a monitor, or the like.

  The host computer 11 in the debug support apparatus 1 manages the version of the source code downloaded from the source repository 2 or the in-house repository 3, the source code creation date and time, and the test result for the source code in association with each other. The test source code repository 12 is a secondary storage for storing the downloaded source code. The debug support information storage unit 13 holds time information such as the difference between versions of the downloaded source code and the date and time when each source code is stored in the test source code repository 12, and is a secondary storage for management. It is. The test script storage unit 14 is a secondary storage for storing a script describing a test procedure for the source code stored in the test source code repository 12. The execution format storage unit 15 is secondary storage for storing an execution format created for executing a test in the target machine 4. The test result storage unit 16 is secondary storage for storing the test result transferred from the target machine 4. The test periodic execution program 17 is a program for executing a test at a predetermined time interval and obtaining the result. The cross referencer 18 is a program for taking a difference between versions of the source code.

  According to the system configuration shown in FIG. 1, the developed source code is stored in the source repository 2 or the internal repository 3, and the test for the developed source code is executed in the target machine 4. It is not limited. For example, a system configuration in which the developed source code is directly stored in the debug support apparatus 1 or a configuration in which a test is executed in the debug support apparatus 1 may be used.

  Further, in the configuration example of FIG. 1, the source code is downloaded from the source code repository 2 or the internal repository 3, and a plurality of versions of the source code are managed based on the time information registered in the test source code repository 12. However, it is not limited to this. For example, it may be managed using the time when the source code is stored in the source code repository 2 or the internal repository 3. In the following description, for simplicity, the repository is a test source code repository 12 in accordance with the configuration example of FIG.

  FIG. 2 shows an example of information necessary for managing a plurality of versions of source code registered in the repository. For example, by displaying a screen as shown in FIG. 2 on the test execution host computer 11 or the monitor of the test result reference machine 5, the information for each version of the source code registered in the repository is displayed to the user. Is shown.

  In the example of FIG. 2, information necessary for each version is shown for certain software versions “1.1.1.1”, “1.2”, and “1.3”. For example, storage date / time information 22, correction content information 23, and difference source information 24 are displayed for version information 21.

  Of the plurality of versions shown in FIG. 2, version “1.3” is taken as an example. The storage date / time information 22 represents the date / time when the source code of the version “1.3” was first stored in the test source code repository 12, that is, the date / time when the software was updated in the debug support apparatus 1. The modification content information 23 displays the amount of change compared to the previous version of software. Here, compared with the software version “1.2”, there are 41 added lines and 0 deleted lines. The difference source information 24 displays a portion where a difference exists when compared with the source code of the previous version. Here, it is displayed that a difference from version “1.2” is generated.

  As described so far, the debug support apparatus 1 can store and manage the source code for each version in the repository, and holds the difference between the source codes as cross reference information. The display of the cross-reference information is performed based on a user designation. Alternatively, it may be executed every time a test result is downloaded from the target machine 4 or every predetermined period, for example, every 24 hours.

FIG. 3 is a flowchart of debugging support processing according to the present embodiment. An outline of processing executed by the debug support apparatus 1 according to the present embodiment will be described with reference to FIG.
First, in step S1, source code is downloaded from a repository that stores source code necessary for testing, such as the source repository 2. Next, in step S2, the cross referencer 18 is activated, and a difference is taken between the downloaded source code and the source code already stored in the test source code repository 12, and the difference is used as cross reference information. It is stored in the cross reference information storage unit 13. At this time, the cross referencer 18 may generate cross reference information between the newly acquired source code and each previously acquired source code, or the newly acquired source code and You may make it produce | generate the cross reference information between only the source codes acquired previously. In step S3, an execution format is constructed from the downloaded source code, and in step S4, a test on the target machine 4 is executed. Finally, in step S5, the test execution result in the target machine 4 is stored in the test result storage unit 16. At this time, the test execution time (test time information) is stored in association with the test result.

  A series of processing shown in FIG. 3 from downloading the source code to storing the test execution result in the secondary storage is periodically and automatically executed, for example, at predetermined time intervals. Alternatively, it may be executed when the debugging support apparatus 1 recognizes that a new source code has been downloaded.

  FIG. 4 is a flowchart of processing for setting source code acquisition date / time information. With reference to FIG. 4, a method for setting information (storage time information) related to the date and time when the debug support apparatus 1 acquires the source code will be described.

  In step S11, connection is made to the source repository 2 (or in-house repository 3) of the download destination described in the download destination definition file using the connection procedure described in the download destination definition file, and in step S12, the download is performed. The file specified in the destination definition file is downloaded to the host computer 11.

  FIG. 5 is an example of a download definition file. In the figure, the module name <ma>, the connection procedure <ca>, and the download destination address <da> of the program to be tested are designated by the command <5A>. In the command <5A>, the module “binutils” is specified to be downloaded from the address indicated by “anoncvs@sources.abcdef.com: / cvs / src” of the computer “pserver” using “cvs”. For the command <5B> specified for the module “gcc”, the module name <mb>, the connection procedure <mc>, and the download destination address <db> are specified in the same format.

  In step S13, the downloaded source code file is stored in the test source code repository 12. In step S14, a difference is taken between the source code file downloaded this time and the previously downloaded source code file, and the generated cross-reference information is stored in the debug support information storage unit 13. In step S15, the date and time when the cross-reference information is generated is set in correspondence with the cross-reference information as generation time information, and the process ends. Note that the series of processing shown in FIG. 4 corresponds to the processing of step S1 and step S2 in FIG.

  FIG. 6 is an example of the generated cross reference information. In step S13, the source code stored in the test source repository 12 is compared with the source code stored in advance, and each portion having a difference is shown. As shown in the example of FIG. 6, for example, it is possible to further link each difference portion and set and display the actual source code for each link destination.

  In the example illustrated in FIG. 6, “task_struct” is input as a keyword in the box B1. And the location where the difference was detected is displayed among the source codes containing the keyword input into this box B1. However, in addition to this form, for example, when no keyword is input, it is possible to display all of the extracted portions.

  FIG. 7 is an example of a scenario table. The scenario table is a file that lists the names of directories that store programs to be executed or the names of programs to be executed. When a directory described in the scenario table is designated, files under that directory are executed in order. In the example shown in FIG. 7, a file to be tested is stored in the directory “/ home / areas / scripts / FJe / ia32”.

  FIG. 8 is a flowchart of processing for executing a test based on the scenario table. In the test execution process, a list of execution formats and test scripts is read from the execution format storage unit 15 and the test script storage unit 14, respectively, in step S21. In step S22, tests are sequentially executed on the files described in the read test script list. In step S23, it is determined whether or not the test has been executed for all files. If not completed, the process returns to step S22 to test a file that has not been executed. If all the files have been completed, the process ends.

FIG. 9 is a flowchart of the operation process when the test script is executed. The process shown in FIG. 9 corresponds to the process of steps S4 and S5 in FIG.
First, in step S31, the target machine 4 to be tested is logged in, and in step S32, a test program is started in the target machine 4. In the present embodiment, the execution result of the test program is stored in the target machine 4 in a text format, for example. In step S33, the execution result of the test program is acquired from the target machine 4. In step S34, the acquired test result is referred to, and it is determined whether or not the result is an output during normal operation. If normal operation is confirmed (step S34: Yes), it is recorded in the test result that the test is successful in step S35, and the process is terminated. On the other hand, when the normal operation is not confirmed (step S34: No), it is recorded in the test result that the test has failed in step S36, and the process is terminated.

  FIG. 10 is an example of a test script. In FIG. 10, <10A> is a command for logging in to the target machine 4. <10B> is a command for starting the test program, and <10C> is a command for determining the test result from the output result. <10D> is a command for outputting and displaying the test result, which is output to the monitor of the test result reference machine 5 in FIG.

  FIG. 11 is an example of a screen on which the test result is displayed on the monitor of the test result reference machine 5. The test results are displayed including, for example, a test name 51, a test execution date 52, a test start time 53, and a test result list 55. In the example of FIG. 11, the execution list 50 and the test result 55 are displayed for the test “pq3-test” executed at 15: 6: 12 on April 18, 2005.

  In FIG. 11, only one test result is displayed, but information related to all past tests is registered in the list 50. In this case, when a desired one is selected from a plurality of tests registered in the list 50, a corresponding test result 55 is displayed on the monitor.

  The test result 55 includes the name of the tested program 56 and the number of items of the test items corresponding to each program divided into normal and abnormal return values. For example, as a test result of the program “tool sysvinit”, it is indicated that there are 4 items for which the test has succeeded (PASS) and 4 items for which the test has failed (FAIL).

  When the user presses the result display button (“results” button) 54 in the screen shown in FIG. 11, the test result 55 is displayed on the monitor of the test result reference machine 5. FIG. 12 is an example of a test execution result. In the test results shown in FIG. 12, taking the first line as an example, it can be seen that the item “MAKEDEV-> MAKEDEV” has failed (FAIL).

  FIG. 13 is an example of a result extracted for an item for which the test has failed. FIG. 13 is displayed when the user clicks on the link 57 for the four items for which the test result for the “tool sysinit” program in FIG. 11 has failed (FAIL).

  FIG. 14 is an example of a difference between test results. In the present embodiment, the “diff” command is used to obtain the difference from the test result file when the return values are all normal (PASS) as a result of the test. The difference is output in a text format as shown in FIG. 14, for example.

  In the seventh and eighth lines in FIG. 14, symbols “−” and “+” are output between the number of lines and the difference extraction result, respectively. This means that the test of the program that has already passed the test on the 7th line is successful (PASS), but the test of the program that is compared with this on the 8th line is failed (FAIL). . That is, it can be seen that degradation has occurred in this item.

  The debugging support apparatus 1 according to the present embodiment executes a test on a plurality of versions of a certain software, the date and time when each test was executed, and the version of each source code used for the test, that is, each source code, Management is performed in association with the date and time stored in the source code repository 12. As a result, when a difference is extracted between the test result for a certain source code and the test result for a later version, as illustrated in FIG. 14, information on the test execution date and time and the storage date and time of the source code is obtained. With reference to the above information, the cross-reference information corresponding to these pieces of information can be output.

FIG. 15 is a flowchart of processing in which the debug support apparatus 1 according to this embodiment outputs a difference between source codes.
In step S41, the last execution date / time is searched among the execution dates / times of the test in which the normal operation is obtained. The test execution date and time searched here is based on the test execution date 52 and the test start time 53 shown in FIG. In step S42, the source code (file) registered in the source code repository 12 before the last execution date and time detected by the search in step S41 is searched. In this step, the source code through which all test items pass in the test is searched. In step S43, a search is performed for source code that has been tested after the last execution date and time in step S41.

  In the process of step S42, the source code of the version that has finally been successfully tested is searched from among the plurality of source codes registered in the repository. In the process of step S43, the test-executed source code registered in the repository after the last execution date and time searched in step S41 is searched from among the plurality of source codes.

  In step S44, it is determined whether or not the source code corresponding to step S43 has been obtained as a search result. If such source code is obtained, the process proceeds to step S45, the difference between the two source codes obtained by the search in step S42 and step S43 is output and displayed on a monitor or the like, and the process ends. In the case of No in step S44, the difference between the latest source code stored in the test source code repository 12 and the source code detected in step S42 is output, and the process ends.

  FIG. 16 shows an output display example of the difference in step S45 (or step S46 depending on the determination result in step S44). For example, the source code detected in step S42 in FIG. 15 is on the right side, the source code detected in step S43 is on the left side, and the screen is divided to display two source codes simultaneously.

  In the figure, “*” marks are places where differences exist between two different versions of source code shown on the left and right. For example, it is preferable to display the difference portion with other portions by color-coded display, highlight display, or the like. From the viewpoint of improving the efficiency of the debugging process by the user, as shown in FIG. 16, for example, the source code before and after the difference part, the entire file including the difference part, and the like are displayed including the part having no difference. preferable.

  In the flowchart of FIG. 15, the difference between the two source codes as shown in FIG. 16 is displayed in step S45 and step S46, but the present invention is not limited to this. For example, cross reference information as shown in FIG. 6 may be displayed.

  Further, in the flowchart of FIG. 15, the difference is taken between the source code whose normal operation was finally confirmed and the source code detected in step S43 or the latest source code, but this is not restrictive. . For example, on the test result display screen as shown in FIG. 11, when a plurality of results are displayed in the test result list 55 and any two test results are selected, the test execution date 52 and the start time 53, Based on the storage time information of the source code, the corresponding source code may be detected and the difference between them may be displayed. It is preferable to be able to set for each user which source code the difference is displayed.

  Alternatively, immediately before the time when the first test is executed when the result of the first test is normal and the result of the second test executed after the first test is abnormal The comparison result information between the source code stored in the repository and the source code stored in the repository immediately before the time when the second test is executed may be output.

  FIG. 17 is a system configuration diagram including a debugging support apparatus 1 according to another embodiment. The system including the debugging support device 1 is different from the configuration of FIG. 1 in that a mail server 6 is further provided. The other configurations are the same as those in FIG. 1 and are therefore given the same reference numerals, and the description thereof is omitted here.

  The mail server 6 is provided for reporting necessary information and transmitting it to a predetermined mail address when the test execution result or the like is confirmed through the test result reference machine 5, for example. The transmitted report includes, for example, a source code or a difference extracted between a plurality of source codes, a test execution result file, a difference between test execution result files, or information on detection of a degradation.

  FIG. 18 is an example of a mail delivery destination address held in the debug support apparatus 1. The mail delivery destination can be set for each program or each version, for example.

  Alternatively, a test execution result or the like may be displayed on a Web browser or the like. In this embodiment, the same information as the information included in the report delivered by the above-mentioned mail, that is, the information related to the detection of the degradation such as the source code, the difference extracted between the source codes or the execution result file of the test And stored in the debug support apparatus 1 in the HTML format. These pieces of information are displayed on the monitor of the test result reference machine 5, for example, at a timing designated by the user. As an example of information displayed on the Web browser, for example, a difference between source codes as shown in FIG.

  As described above, according to the debug support apparatus 1 according to the present embodiment, the source code is version-managed and registered in the repository. When the source code is registered in the repository, cross reference information representing a difference from the previously registered source code is generated. The registered source code is tested, and the test result is managed in association with the version of the source code. For this reason, when a degradation is detected by this test, the cross-reference information with the source code registered previously is referred for the version of the source code. By referring to the cross-reference information, it is easier to grasp which source code is modified due to the modification. As a result, the user can streamline the task of tracking degradation failures.

(Appendix 1)
A program for supporting software debugging processing,
A first procedure for storing source code in the source code storage means;
The comparison result information is generated and stored by comparing the first source code previously stored in the source code storage means and the second source code newly stored by the first procedure. 2 steps,
A third procedure for converting the second source code into an executable format and performing a predetermined test;
A fourth procedure for storing the test result information obtained by the test and the test time information indicating the time when the test was performed in association with each other;
A fifth procedure for repeatedly executing the first to fourth procedures;
Sixth procedure for obtaining and outputting comparison result information corresponding to designated test result information using the test time information and storage time information indicating the storage time of each source code in the source code storage means ,
A program that causes a computer to execute.
(Appendix 2)
The program according to claim 1, wherein the comparison result information is generated when the second source code is stored in the source code storage unit.
(Appendix 3)
The fifth procedure repeatedly executes the first to fourth procedures at predetermined time intervals.
The program according to appendix 1, characterized by:
(Appendix 4)
The sixth procedure includes:
When the result of the first test is normal and the result of the second test executed after the first test is abnormal, immediately before the time when the first test is executed Including a process of outputting comparison result information between the source code stored in the source code storage means and the source code stored in the source code storage means immediately before the time when the second test is executed. The program according to appendix 1, characterized by:
(Appendix 5)
The program according to appendix 4, wherein the sixth procedure is outputted by notifying a mail receiving side using an electronic mail.
(Appendix 6)
The program according to appendix 4, wherein the sixth procedure outputs the program by displaying it on a web browser.
(Appendix 7)
The test is executed by transferring the program converted into the execution format to a target machine for executing the test,
The program according to appendix 4, wherein the test result information is obtained from the target machine.
(Appendix 8)
A debugging support method for supporting software debugging processing,
A first procedure for storing source code in the source code storage means;
The comparison result information is generated and stored by comparing the first source code previously stored in the source code storage means and the second source code newly stored by the first procedure. 2 steps,
A third procedure for converting the second source code into an executable format and performing a predetermined test;
A fourth procedure for storing the test result information obtained by the test and the test time information indicating the time when the test was performed in association with each other;
A fifth procedure for repeatedly executing the first to fourth procedures;
Sixth procedure for obtaining and outputting comparison result information corresponding to designated test result information using the test time information and storage time information indicating the storage time of each source code in the source code storage means ,
A debugging support method comprising:
(Appendix 9)
A debugging support device for supporting software debugging processing,
Source code storage means;
Comparison in which comparison result information is generated and stored by comparing the first source code previously stored in the source code storage means and the second source code newly stored in the source code storage means Means,
Test execution means for converting the second source code into an execution format and performing a predetermined test;
Test result management means for storing test result information obtained by the above test and test time information indicating the time when the test was performed in association with each other;
Control means for causing the comparison means, the test execution means, and the test result management means to repeatedly execute the above-described processing;
Output means for obtaining and outputting comparison result information corresponding to the designated test result information, using the test time information and storage time information representing the storage time of each source code in the source code storage means;
A debugging support apparatus comprising:

1 is a system configuration diagram including a debugging support apparatus according to the present embodiment. This is an output example of information necessary for managing multiple versions of source code. It is a flowchart about the process of the debugging assistance which concerns on this embodiment. It is a flowchart of the process which sets the acquisition date information of a source code. It is an example of a download definition file. It is an example of the produced | generated cross reference information. It is an Example of a scenario table. It is a flowchart of the process which performs a test. It is a flowchart about the operation processing at the time of test script execution. It is an example of a test script. It is an example of the screen which displayed the test result. It is an example of a test execution result. It is an example of the result extracted about the item in which the test failed. It is an example of the difference between test results. It is a flowchart of the process which outputs the difference between source codes. It is an example of a difference output display. It is a system block diagram containing the debug assistance apparatus which concerns on other embodiment. It is an example of the address of the delivery destination of mail.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Debug support apparatus 2 Source repository 3 Internal repository 4, 4A, 4B, 4C Target machine 5 Test result reference machine 11 Test execution host computer 12 Test source code repository 13 Debug support information storage unit 14 Test script storage unit 15 Target machine Execution format storage unit 16 Test result storage unit 17 Periodic execution program 18 Cross referencer

Claims (4)

A program for supporting software debugging processing,
A first procedure for storing the source code in the source code storage means in association with the storage time information indicating the storage time ;
And generates a comparison result information by comparing the second source code newly stored by the source code storing unit previously the first source code stored in the first procedure, the second A second procedure for storing in correspondence with the source code of 2;
A third procedure for converting the second source code into an executable format and performing a predetermined test;
Test result information obtained by the test and test time information indicating the time when the test was performed, a fourth procedure for storing the second source code in association with each other;
A fifth procedure for repeatedly executing the first to fourth procedures;
As comparison result information corresponding to the designated test result information, by using the test time information and the storage time information, the comparison result information testing is associated with the second source code made Causing the computer to execute a sixth procedure for obtaining and outputting;
The sixth procedure includes:
When the result of the first test is normal and the result of the second test executed after the first test is abnormal, immediately before the time when the first test is executed Processing for outputting comparison result information between the source code stored in the source code storage means and the source code stored in the source code storage means immediately before the time when the second test is executed <br / > A program that contains 2. The program according to claim 1, wherein the comparison result information is generated when the second source code is stored in the source code storage unit. The program according to claim 1 , wherein the sixth procedure is outputted by notifying a mail receiving side using an electronic mail. The program according to claim 1 , wherein the sixth procedure is output by being displayed on a web browser.