JP2011181034A - System and method for evaluation of software effective performance, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for evaluation of software effective performance, etc. capable of finding effective performance of software. <P>SOLUTION: The system 1 for evaluation of software effective performance is composed as follows: an execution environment information acquisition section acquires execution environment information indicative of specifications and performance of a software execution environment; an execution environment information storage section stores the execution environment information; a program analysis section analyzes nests of a program code of the software and performance of each nest; an analysis result storage section stores information on the nest performance analyzed by the analysis section; a selection section selects a nest for evaluating performance from the nests included in the program code; and a performance acquisition section acquires a value indicative of software effective performance based on the execution environment information and the nest performance information of the nest selected by the selection section. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for evaluating the effective performance of software.

  In order to measure the effective performance of software, it is common to deploy measurement target software in an execution environment, actually operate the software, and collect data using various tools for performance measurement. For this reason, the measurer must perform complicated work every time the measurement is performed, and when measuring the effective performance of a process that requires a long time, the measurer must wait every time until the process is completed. There was a problem.

  Furthermore, if the software is greatly affected by the performance and specifications of the execution environment such as hardware, middleware, and network, it is necessary to re-measure with the same configuration as the execution environment every time the execution environment is changed. I had to spend a lot of time and time each time.

  In particular, hardware and middleware, an OS (Operating System), a network, and the like, which are platforms for executing software, are diversified year by year, and their performance and specifications are also various. As for middleware, OSS (Open Source Software) that can withstand commercial use has appeared, and the number of companies that use these OSS middleware to reduce development costs is increasing. Compared with paid middleware, In many cases, the performance is inferior, and in order to introduce it, it is necessary to carefully compare the performance of various OSS. In addition, as for software development forms, demand for service-oriented applications such as SaaS (Software as a Service) and SOA (Service-Oriented Architecture) pursuing reusability and flexibility is increasing. In these service-oriented applications, compared to conventional applications, individual components deployed in various execution environments often cooperate via a network, and the performance of the application execution environment is the performance of the application. It has a big influence on.

  Under such circumstances, various methods have been proposed for evaluating the effective performance of software. For example, Patent Document 1 discloses a technique for evaluating the performance of software by measuring the number of steps of the software program code before the program is installed. In addition, there is a method for measuring the effective performance of software that automatically measures the performance by comparing the number of clocks of the processor in the execution environment with the number of steps of the entire program code (for example, Patent Document 2).

JP 2001-273174 A Japanese Patent Application Laid-Open No. 2006-4221

  However, in the above method, only the number of clocks of the processor and the number of steps of the entire program code are simply compared, and the performance including the OS and middleware on which the program operates, the specifications and performance of the network are calculated, It is not possible to specify a configuration for performance measurement. Further, the above-described method targets the number of steps of the entire program code, and there is a problem that it is difficult to perform a test that passes only a specific branch block and examine the performance of only the block portion.

  In addition, the measurement method of the effective performance of the software and the various tools used for the measurement are not unified, and if different personnel perform performance measurement using their own method, the performance measurement results for each personnel will be inconsistent. There was a possibility.

  The present invention has been made in view of the above problems, and a software effective performance evaluation system and software effective that can determine the effective performance of only a specific block in a predetermined execution environment without executing a software program. An object is to provide a performance evaluation method and program.

A software effective performance evaluation system according to the first aspect of the present invention that achieves the above object is as follows:
An execution environment information acquisition unit for acquiring execution environment information indicating the specifications and performance of the software execution environment;
An execution environment information storage unit for storing the execution environment information;
A program analysis unit for analyzing the nesting of the program code of the software and the performance of each nest;
An analysis result storage unit that stores nest performance information indicating the performance of the nest analyzed by the program analysis unit;
A selection unit for selecting a nest to be evaluated from a nest included in the program code;
The nest performance information of the nest selected by the selection unit is read from the analysis result storage unit, and the effective performance of the software is indicated based on the nest performance information and the execution environment information stored in the execution environment information storage unit A performance acquisition unit for acquiring values;
It is characterized by providing.

The software effective performance evaluation method according to the second aspect of the present invention is as follows.
An execution environment information acquisition unit, an execution environment information storage unit, a program analysis unit, an analysis result storage unit, a selection unit, and a software acquisition performance evaluation method performed by a device including a performance acquisition unit,
The execution environment information acquisition unit acquires execution environment information indicating execution environment specifications and performance of software, and
An execution environment information storage step in which the execution environment information storage unit stores the execution environment information;
The program analysis unit analyzes a program code nest of the software and the performance of each nest;
An analysis result storage step in which the analysis result storage unit stores nest performance information indicating the performance of the nest analyzed by the program analysis unit;
A selection step in which the selection unit selects a nest for performance evaluation from the nest included in the program code;
The performance acquisition unit reads the nest performance information of the nest selected by the selection unit from the analysis result storage unit, and based on the nest performance information and the execution environment information stored in the execution environment information storage unit, software A performance acquisition step of acquiring a value indicating the effective performance of
It is characterized by providing.

A program according to the third aspect of the present invention is:
Computer
An execution environment information acquisition unit for acquiring execution environment information indicating the specifications and performance of the software execution environment;
An execution environment information storage unit for storing the execution environment information;
A program analysis unit for analyzing the nesting of the program code of the software and the performance of each nest;
An analysis result storage unit that stores nest performance information indicating the performance of the nest analyzed by the program analysis unit;
A selection unit for selecting a nest for performance evaluation from the nest included in the program code;
The nest performance information of the nest selected by the selection unit is read from the analysis result storage unit, and the effective performance of the software is indicated based on the nest performance information and the execution environment information stored in the execution environment information storage unit A performance acquisition unit that acquires values,
It is made to function as.

  According to the present invention, there are provided a software effective performance evaluation system, a software effective performance evaluation method, and a program capable of obtaining the effective performance of only a specific block in a predetermined execution environment without executing the software program. can do.

It is a figure showing the whole software effective performance evaluation system composition concerning an embodiment of the present invention. It is a figure which shows an example of execution environment information data ((a) CPU, (b) AP server, (c) DBMS server, (d) OS, (e) LAN card, (f) DLL) of execution environment. It is a figure which shows an example of the performance information data ((a) CPU, (b) AP server, (c) DBMS server, (d) OS, (e) LAN card, (f) DLL) of execution environment. It is a figure for demonstrating the nested structure of the software which evaluates effective performance. It is a figure for demonstrating software source code information. It is a figure for demonstrating a performance evaluation object script. It is a figure which shows the evaluation result of the effective performance of software. It is a flowchart for demonstrating the process which the execution environment analyzer of embodiment performs. It is a flowchart for demonstrating the process which the software analysis apparatus of this embodiment performs. It is a flowchart for demonstrating the process which the effective performance measuring apparatus of embodiment performs.

(Embodiment)
The software effective performance evaluation system 1 according to the embodiment of the present invention acquires i) information related to the specification and performance of the software execution environment, registers the information in the database, and ii) software nesting and steps in each nesting Iii) accepts execution environment specifications and software nesting specifications, and iv) values indicating the effective performance of software based on the information stored in the database and the number of steps in the selected nesting Ask for.
Hereinafter, the software effective performance evaluation system 1 will be described in detail with reference to the drawings.

  As shown in FIG. 1, the software effective performance evaluation system 1 includes an execution environment analysis apparatus 100, an execution environment database 200, a software analysis apparatus 300, and an effective performance measurement apparatus 400. The execution environment analysis apparatus 100 and the execution environment database 200 can exchange data with each other by, for example, a LAN (Local Area Network). Similarly, the execution environment database 200 and the effective performance measurement device 400, and the software analysis device 300 and the effective performance measurement device 400 can mutually transmit and receive data via a LAN, for example. The execution environment analysis apparatus 100 communicates with computers constituting software execution environments 601 to 60n that can execute software via a network 500 such as the Internet.

  The software execution environments 601 to 60n (n: arbitrary) are configured by, for example, hardware, middleware, OS (Operating System), network, runtime library, and the like. The hardware, middleware, OS, network, and runtime library constituting the software execution environment 601 to 60n each have one or more components. Specifically, hardware is a component such as a CPU, middleware is a component such as an AP server and DBMS (Date Base Management System), a network is a component such as a LAN card, a runtime library is a DLL (Dynamic Link Library), etc. It has the following components.

  The execution environment analysis apparatus 100 acquires i) information related to the specifications and performance of the software execution environment (software execution environments 601 to 60n), and registers the information in the database (execution environment database 200).

  The execution environment analysis apparatus 100 is realized by a general computer, and functionally includes a communication unit 101, a control unit 102, a storage unit 103, and an I / F unit 104.

  The communication unit 101 transmits and receives data to and from the computers constituting the software execution environments 601 to 60n via the first network 500 by a predetermined communication method. The communication unit 101 transmits and receives data to and from the execution environment database 200 by a predetermined communication method.

  The control unit 102 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). According to a software program stored in the ROM or the like, the entire execution environment analysis apparatus 100 is controlled. Take control. The control unit 102 has a function of detecting specifications and performance of the software execution environments 601 to 60n at a predetermined cycle. For example, the control unit 102 performs the execution environment analysis processing shown in the flowchart of FIG. 8 according to the internal timer, and configures the hardware, middleware, OS, network, and runtime library components constituting the software execution environment 601 to 60n. “Execution environment information” indicating specifications and “performance information” indicating performance are acquired, and the acquired execution environment information and performance information are stored in the execution environment database 200.

  When the I / F unit 104 receives an instruction to start the process of acquiring the specifications and performance of the software execution environments 601 to 60n from the user, the control unit 102 performs the execution environment analysis process shown in the flowchart of FIG. You may make it perform.

  In addition, the control unit 102 has a function of detecting a change in the specification instead of detecting the specification and performance of the software execution environments 601 to 60n at a predetermined cycle. The execution environment analysis process shown in the flowchart may be executed to detect the specifications and performance of the software execution environments 601 to 60n. For example, the control unit 102 reads configuration files of hardware, middleware, OS, network, and runtime library that configure the software execution environments 601 to 60n at a predetermined cycle, and stores the read results in the storage unit 103 or the like. Then, the control unit 102 compares the previously read configuration file stored in the storage unit 103 or the like with the currently read configuration file, and if a difference is detected between the two, the software execution environment 601- Assuming that the 60n specification has been changed, execution environment analysis processing is executed.

  Hereinafter, specific examples of execution environment information and performance information for each component of hardware, middleware, OS, network, and runtime library will be shown.

  The CPU execution environment information includes information indicating “CPU name (CPU name)” and “processor number” indicating the CPU model number, and the CPU performance information includes “cache memory capacity”, “operating frequency”, and the like. It includes information indicating “FSB (Front Side Bus) operating frequency” and the like.

  The AP server and DBMS execution environment information includes information indicating “AP / DBMS name (AP / DBMS name)” and “version”, and the AP server and DBMS performance information is executed by the AP server and DBMS. Each “execution command”, “number of steps” of each execution command, “TAT (Turn Around Time)”, “CPU usage rate”, “memory usage rate” required to execute each execution command in a predetermined software execution environment ”And“ network bandwidth usage rate ”and the like.

  The OS execution environment information includes information indicating “OS name (OS name)” and “version”, and the OS performance information includes “recommended clock speed” of the OS, “recommended RAM capacity” of the OS, and It includes information indicating the OS performance coefficient and the like. Here, the performance coefficient is a value indicating how much the type of OS affects the software effective performance. For example, in the test environment having the same hardware configuration and software configuration in advance, the values of the software effective performance (TAT, CPU usage rate, memory usage rate, TAT, and network) when the test program is executed on various OSs Bandwidth utilization). Then, “1” is set when each effective performance value exceeds a predetermined threshold value, and “0” is set when it does not exceed the predetermined threshold value. The method of determining the performance coefficient is arbitrary, and may be a value that is proportional to the value of the software effective performance obtained by the test, for example.

  The LAN card execution environment information includes information indicating “LAN card name (LAN card name)” and the like, and the LAN card performance information includes “transmission speed”, “Jumbo Frame” size and “transmission distance”. And the like information.

  The DLL execution environment information includes information indicating “DLL file name (DLL name)” and “version”, and the DLL performance information includes “number of steps” of each DLL file, It includes information indicating “CPU usage rate”, “memory usage rate”, “TAT”, “network bandwidth usage rate”, and the like required when executing the DLL file.

  Then, the control unit 102 causes the communication unit 101 to send the acquired execution environment information and performance information to the execution environment database 200.

  The storage unit 103 includes a storage device such as a hard disk or a flash memory. For example, the storage unit 103 temporarily stores execution environment information and performance information acquired by the control unit 102.

  The I / F unit 104 includes a display device such as a monitor and an input device such as a keyboard. For example, the I / F unit 104 receives an instruction to start execution environment analysis processing for a predetermined software execution environment.

  The execution environment database 200 includes a storage device such as a hard disk. When the execution environment database 200 receives execution environment information and performance information acquired by the execution environment analysis apparatus 100 (for example, the software execution environment 601 (SE1)), the execution environment information is stored in the execution environment information table 201 (FIG. 2). The performance information is registered in the performance information table 202 (FIG. 3). The execution environment database 200 registers the execution environment information and performance information of other software execution environments (software execution environments 602 to 60n) in the execution environment information table 201 and the performance information table 202 in the same manner.

  The software analysis apparatus 300 ii) analyzes the software nesting structure and the performance of each nesting.

  The software analysis apparatus 300 is realized by a general computer, and functionally includes a communication unit 301, a control unit 302, a storage unit 303, and an I / F unit 304.

  The communication unit 301 transmits and receives data to and from the effective performance measurement device 400 using a predetermined communication method.

  The control unit 302 includes a CPU, a RAM, and a ROM, and controls the entire software analysis apparatus 300 according to a software program stored in the ROM or the like. For example, when receiving an instruction to start the software analysis process from the effective performance measurement device 400, the control unit 302 performs the process (software analysis process) shown in the flowchart of FIG. Get the number of steps. Further, for example, when the I / F unit 304 receives an instruction to start the software analysis process by the user, the process shown in the flowchart of FIG. 9 may be performed.

  For example, assume that software analysis processing is performed on the software source code SW1 shown in FIG. The control unit 302 analyzes the nest structure of the software source code SW1, and each nest a1 (nest representing a class), nest b1 (nest representing a method or function), nest c1 / c2 (nest representing a branch process), nest d1 (nest representing repetition) is extracted. Then, the control unit 302 obtains the number of steps from the source code included in each nest. At this time, when iterative processing is performed in the nest, the number of steps actually executed is obtained by multiplying the number of repetitions, not the number of steps from the number of lines of the source code. The control unit 302 stores the analyzed result in the storage unit 303.

  The storage unit 303 includes a storage device such as a hard disk or a flash memory. The storage unit 303 stores the software nesting structure and the number of steps in the nesting acquired by the control unit 302 as shown in FIG. 5 (software source code information 303a), for example. The software source code information 303a is included in each nest, “software” to be evaluated, “nest” included in the software, “higher nest” of each nest, “lower nest” of each nest. Information indicating “number of steps” is included.

  The I / F unit 304 includes a display device such as a monitor and an input device such as a keyboard. For example, the I / F unit 304 receives an instruction to start the software analysis process and designation of the software source code SW1 to be subjected to the effective performance analysis, and monitors the result of the software analysis process (software source code information in FIG. 5). Output to.

  The effective performance measuring device 400 accepts the specification of the execution environment and the software nesting, and iv) determines the effective performance of the software based on the information stored in the database and the number of steps in the selected nesting. Find the value shown.

  The effective performance measuring apparatus 400 is realized by a general computer, and functionally includes a communication unit 401, a control unit 402, a storage unit 403, and an I / F unit 404.

  The communication unit 401 transmits and receives data to and from the execution environment database 200 and the software analysis device 300 by a predetermined communication method.

  The control unit 402 includes a CPU, a RAM, and a ROM, and controls the entire effective performance measuring apparatus 400 according to a software program stored in the ROM or the like. For example, when the I / F unit 404 receives an instruction to start measuring the effective performance of software, the control unit 402 performs the process (effective performance measurement process) shown in the flowchart of FIG. That is, the performance information of the corresponding execution environment is read from the execution environment database 200 based on the information (CPU ID, AP server ID, etc.) indicating the execution environment specification specified by the user, and the software specified by the user is nested. The number of steps in the nest is read from the software source code information 303a stored in the software analysis device 300. Then, a value indicating the effective performance of the software is obtained from the read performance information and the number of steps. Hereinafter, the specification of the execution environment specified by the user and the nesting of software are collectively referred to as “evaluation target specification information”.

  Here, the value indicating the effective performance of the software is, for example, TAT, CPU usage rate, memory usage rate, or network usage rate, and can be obtained as follows based on the evaluation target designation information.

TAT, CPU usage rate, and memory usage measured when a program including X steps is executed in a predetermined execution environment having components of CPU, AP server, DBMS, OS, LAN card, and DLL Let the rate and the network usage rate be (t, u, v, w), respectively. The performance information shown in FIG. 3 for a plurality of execution environments in advance, and measured values of the TAT, CPU usage rate, memory usage rate, and network usage rate when software having X number of steps is executed in the execution environment (t ₁ , u ₁ , v ₁ , w ₁ ) to (t _n , u _n , v _n , w _n ). Then, using the performance information value (FIG. 3) and the number of software steps as a dependent variable, a logical model (formula) representing TAT, CPU usage rate, memory usage rate, or network usage rate is obtained by regression analysis.

For example, each coefficient a, b, c, d, e, f, g of the following TAT logic model is obtained by regression analysis.
TAT = a × operation frequency + b × number of steps (AP server) + c × number of steps (DBMS) + d × performance coefficient + e × transmission speed + f × number of steps (DLL) + g × X (number of software steps)
Note that the combination of performance information is arbitrary and can be selected as appropriate by the user.

  The logical expression obtained in this way is stored in the storage unit 403, and when the user designates the evaluation target designation information, the control unit 402 obtains the value of the dependent variable of the logical expression and substitutes it into the logical expression. Obtain a value that indicates the effective performance of the software.

  The storage unit 403 includes a storage device such as a hard disk or a flash memory, and stores, for example, a logical expression for calculating the effective performance of the software and a value indicating the effective performance of the software obtained by the control unit 402.

  The I / F unit 404 includes a display device such as a monitor and an input device such as a keyboard. For example, the I / F unit 104 receives an instruction to start the effective performance measurement process and displays a script (evaluation target designation script in FIG. 6) for designating evaluation target designation information on the monitor. When the user edits the evaluation target specifying script with a keyboard or the like and specifies evaluation target specifying information, the control unit 402 performs an effective performance measurement process and monitors the obtained result (FIG. 7, effective performance measurement result table). Output to.

  Next, operations (FIGS. 8 to 10) performed by the execution environment analysis device 100, the software analysis device 300, and the effective performance measurement device 400 of the software effective performance evaluation system 1 having the above-described configuration will be described. In the following description, the execution environment analysis apparatus 100 performs an execution environment analysis process at a predetermined cycle according to an internal timer. When the software analysis apparatus 300 receives an instruction to start the software analysis process from the effective performance measurement apparatus 400, the software analysis process To start. When the effective performance measurement apparatus 400 receives an instruction to start the effective performance measurement process from the user, the effective performance measurement apparatus 400 starts the effective performance measurement process. At this time, the effective performance measurement device 400 transmits an instruction to start software analysis processing to the software analysis device 300.

  The execution environment analysis apparatus 100 starts the execution environment analysis process shown in FIG. 8 at a predetermined cycle according to the internal timer.

  The control unit 102 acquires information regarding the software execution environments 601 to 60n (Step S11). For example, it communicates with the computers configuring the software execution environments 601 to 60n, and reads the configuration file that the computer has.

  Next, the control unit 102 obtains execution environment information and performance information related to the hardware, middleware, OS, network, and components of the runtime library that constitute the acquired software execution environment 601 to 60n (step S12). For example, information such as “XXX” as the “CPU name” of the CPU included in the software execution environment 601 (SE1) and “E8600” as the “processor number” is acquired from the configuration file. Similarly, the “AP server name” of the AP server is “XX Server”, the “version” is “6.0.1”, the “DBMS name” of the DBMS is “XX DB”, and the “version” is “1.2. 1 ”, the OS“ OS name ”is“ XXXX ”, the“ version ”is“ YY ”, the LAN card“ LAN card name ”is“ XXX-XXX ”, and the DLL“ DLL name ”is“ XXX.dll ”, “Version” obtains information such as “11.0.6568”.

  Further, the control unit 102 acquires performance information of each component. For example, the “cache memory capacity” of the CPU (“XXX”) is “6MB L2”, the “operating frequency” is “3.33 GHz”, the “FSB” is “1333 MHz”, and the AP server (“XX Server”) The “execution command” to be executed is “callEJB ()”, the “number of steps” is “1000”, the “CPU usage rate” is “1%”, the “memory usage rate” is “1%”, and the “TAT” is “ “7000 μs”, “Network Bandwidth Utilization” is “1%”, “Execution Command” executed by the DBMS (“XX DB”) is “select”, “CPU Utilization” is “1%”, “Memory Usage” “Rate” is “1%”, “TAT” is “300 μs”, “Network Bandwidth Utilization” is “1%”, “Recommended Clock Speed” of the OS (“XXXX”) is “300 MHz”, “Recommended RA” The “M capacity” is “128 MB”, the “HDD capacity” is “2.1 GB”, the “transmission speed” of the LAN card (“XXX-XXX”) is “1,000 Mbps”, and the “Jumbo Frame” is “7,154 bytes”. ”,“ Transmission distance ”is“ 100 m ”,“ Number of steps ”in the DLL (“ XXX.dll ”) is“ 500 ”,“ CPU usage rate ”is“ 1% ”, and“ Memory usage rate ”is“ 1% ” “,” “TAT”, “300 μs”, “Network Bandwidth Utilization”, “1%”, etc. are acquired.

  Then, the control unit 102 outputs the execution environment information and performance information of the hardware, middleware, OS, network, and runtime library as a result of the analysis to the execution environment database 200 (step S13). As shown in FIGS. 2A to 2F (execution environment information table 201), the execution environment database 200 includes IDs (“CPU ID”, “AP server ID”, ("DBMS ID", "OS ID", "LAN card ID", "DLL ID")) and the execution environment information is stored in a table for each component. Further, as shown in FIGS. 3A to 3F (performance information table 202), the performance information acquired by the control unit 102 is stored in a table for each component. At this time, the same ID as that used in the execution environment information table is assigned to the performance information corresponding to the execution environment information. Thereby, the execution environment information and performance information of each component can be read using the ID as a key. For example, if the user selects a CPU whose “CPU ID” is “CPU-001”, information such as “CPU name” and “cache memory capacity” of the CPU is read using “CPU-001” as a key. be able to.

  Next, processing operations performed by the software analysis apparatus 300 will be described. When receiving an instruction to start the software analysis process from the effective performance measuring apparatus 400, the software analysis apparatus 300 starts the software analysis process shown in FIG.

  When the software source code targeted for performance analysis is designated by the user of the effective performance measuring apparatus 400, the control unit 302 reads the designated software source code (step S21). Next, the control unit 302 analyzes the nested structure of the read software source code and extracts each nest (step S22). For example, if the control unit 302 reads the software source code SW1 of FIG. 4, the control unit 302 extracts a nest a1, a nest b1, a nest c1 / c2, and a nest d1.

  Next, the control unit 302 analyzes the software source code included in each nest and obtains the number of execution steps for each nest (step S23). For example, the number of execution steps included in the nest d1 of the software source code SW1 is obtained as 20001 (= 1 (repetition definition line) +2 (line in the loop) × 10000 (repetition count)). The number of execution steps of the nest c1 that is higher than the nest d1 is obtained as 20101, by adding the number of execution steps included in the nest d1 (20001) and the number of steps included in the nest d1 (for example, 100).

  Then, the control unit 302 causes the I / F unit 304 to output the analyzed result. For example, the control unit 302 causes the monitor to display the software code information shown in FIG. The software code information of this embodiment is included in the “software” indicating the analyzed software, the “nest” included in the software, the “upper nest” and the “lower nest” of the nest, and the nest And “step number”.

  When the I / F unit 304 receives an instruction to start the performance measurement process from the user, the effective performance measurement apparatus 400 starts the performance measurement process illustrated in FIG.

  First, the control unit 402 designates software for evaluating effective performance, communicates with the software analysis apparatus 300, and causes the software analysis apparatus 300 to start the software analysis processing shown in FIG. Then, the control unit 402 acquires software source code information analyzed by the software analysis apparatus 300 (step S31). For example, if the software for evaluating the effective performance is the software source code SW1 in FIG. 4, the control unit 402 acquires the software source code information 303a in FIG. 5 from the software analysis apparatus 300 and temporarily stores it in the RAM or the like of the control unit 402. Store.

  Next, the control unit 402 causes the I / F unit 104 to display an evaluation target designation script (FIG. 6), and the user edits the evaluation target designation script using a keyboard or the like to designate evaluation target designation information. . The control unit 402 acquires evaluation target designation information from the evaluation target designation script (step S32). For example, if the user specifies nest c1 of software source code SW1 and CPU-001, AP-001, DB-001, OS-001, LAN-001, and DLL-001 as evaluation target specifying information, Temporarily stored in the RAM or the like of the control unit 402.

  Next, the control unit 402 acquires performance information from the execution environment database 200 based on the ID written in the performance evaluation target information read in step S32 (step S33). For example, the performance information of CPU-001, AP-001, DB-001, OS-001, LAN-001, and DLL-001 shown in FIG.

  The control unit 402 calculates the performance of the software from the software source code information and the performance information read from the execution environment database 200 according to the designated evaluation target designation information (step S34). For example, the control unit 402 displays the number of steps of the nest c1 of the designated software source code SW1 and the performance information of the CPU-001, AP-001, DB-001, OS-001, LAN-001, and DLL-001. Substituting into a regression equation prepared in advance as a dependent variable, the TAT of the nest c1 of the software source code SW1 is obtained.

  Then, the control unit 402 causes the I / F unit 404 to display the result of performance measurement of software (performance measurement result data, FIG. 7). For example, the performance measurement result data includes the type of “software” to be evaluated and the “nesting” of the software, the ID of each execution environment specified by the user, and an index (for example, the performance of the specified nest of the software) , “TAT”) and the like. The example of FIG. 7 shows the performance when the nest c1 of the software source code SW1 is executed in a software execution environment including CPU-001, AP-001, DB-001, OS-001, LAN-001, and DLL-001. (TAT) is shown.

  As described above, according to the software effective performance evaluation system 1, by providing a mechanism for automatically analyzing and collecting the specifications and performance of the software execution environment and registering them in the database, complicated configurations and configurations can be realized. Even when it is frequently changed, information necessary for performance measurement can be quickly collected, and the man-hours required for performance measurement can be reduced. In addition, since it is possible to save the trouble of actually deploying and executing software in the execution environment, it is possible to reduce the man-hours required for performance measurement. Furthermore, it is possible to measure performance without grasping detailed specifications regarding various execution environments, and it is possible to reduce learning costs necessary for performance measurement.

  In addition, by analyzing the number of effective steps in the nesting unit of the software source code, it is possible to measure the performance in the nesting unit that is actually executed, and it is possible to measure data close to the actual performance under various conditions It becomes. By using the present invention, it becomes possible to perform performance measurement according to a certain procedure for each environment, so even if the person in charge of performance measurement changes, the performance measurement method may differ, and the performance measurement result may be different. Can be less likely to occur.

  The software effective performance evaluation system of the present invention is not limited to the configuration shown in FIG. For example, the execution environment analysis device 100, the execution environment database 200, the software analysis device 300, and the effective performance measurement device 400 may be realized by the same device. Further, each device may be communicable via the Internet instead of the LAN.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1) An execution environment information acquisition unit that acquires execution environment information indicating the specifications and performance of the software execution environment, an execution environment information storage unit that stores the execution environment information, and a nesting of program code of the software A program analysis unit for analyzing the performance of the nest, an analysis result storage unit for storing nest performance information indicating the performance of the nest analyzed by the program analysis unit, and a nest to be evaluated from the nest included in the program code A selection unit that selects the nest performance information of the nest selected by the selection unit from the analysis result storage unit, and based on the nest performance information and the execution environment information stored in the execution environment information storage unit And a performance acquisition unit for acquiring a value indicating the effective performance of the software. Value system.

(Additional remark 2) The said execution environment information is comprised from the OS (Operating System) of the apparatus which performs software, hardware, middleware, a network, or the information which shows the specification and performance of a runtime library, It is characterized by the above-mentioned The software effective performance evaluation system according to 1.

(Supplementary note 3) The software effective performance evaluation system according to supplementary note 1 or 2, wherein the nesting performance information includes information indicating the number of steps of the program code.

(Supplementary note 4) The software effective performance evaluation system according to supplementary note 3, wherein the number of steps of the program code is the total number of steps in consideration of the number of repetitions when the program code includes repetition processing.

(Additional remark 5) The specification change detection part which detects the change of the specification of the execution environment of the said software is further provided, The said execution environment information acquisition part, when the said specification change detection part detects the change of a specification, The software effective performance evaluation system according to any one of appendices 1 to 4, wherein execution environment information of the execution environment is acquired.

(Supplementary note 6) A software effective performance evaluation method performed by an apparatus including an execution environment information acquisition unit, an execution environment information storage unit, a program analysis unit, an analysis result storage unit, a selection unit, and a performance acquisition unit. An execution environment information acquisition step in which the execution environment information acquisition unit acquires execution environment information indicating the specifications and performance of the execution environment of the software, and an execution environment in which the execution environment information storage unit stores the execution environment information. An information storage step, a program analysis step in which the program analysis unit analyzes the nest of the program code of the software and the performance of each nest, and the analysis result storage unit determines the performance of the nest analyzed by the program analysis unit. The analysis result storage step for storing the nest performance information to be shown, and the nest to be selected as a performance evaluation target from the nest included in the program code by the selection unit The selection process to be selected, and the performance acquisition unit reads the nest performance information of the nest selected by the selection unit from the analysis result storage unit, and the execution environment information stored in the execution performance information storage unit. And a performance acquisition step of acquiring a value indicating the effective performance of the software based on the above.

(Supplementary Note 7) An execution environment information acquisition unit that acquires execution environment information indicating the specifications and performance of software execution environment, an execution environment information storage unit that stores the execution environment information, and a nesting of program code of the software A program analysis unit that analyzes the performance of each nest, an analysis result storage unit that stores nest performance information indicating the performance of the nest analyzed by the program analysis unit, and a nest that is subject to performance evaluation from the nests included in the program code The selection unit to be selected, the nest performance information of the nest selected by the selection unit is read from the analysis result storage unit, and based on the nest performance information and the execution environment information stored in the execution environment information storage unit, software A program that functions as a performance acquisition unit that acquires a value indicating the effective performance of .

DESCRIPTION OF SYMBOLS 1 Software effective performance evaluation system 100 Execution environment analysis apparatus 101 Communication part 102 Control part 103 Storage part 104 I / F part 200 Execution environment database 201 Execution environment information table 202 Performance information table 300 Software analysis apparatus 301 Communication part 302 Control part 303 Storage Unit 303a software source code information 304 I / F unit 400 effective performance measuring device 401 communication unit 402 control unit 403 storage unit 404 I / F unit 500 network 601 to 60n software execution environment

Claims (7)

An execution environment information acquisition unit for acquiring execution environment information indicating the specifications and performance of the software execution environment;
An execution environment information storage unit for storing the execution environment information;
A program analysis unit for analyzing the nesting of the program code of the software and the performance of each nest;
An analysis result storage unit that stores nest performance information indicating the performance of the nest analyzed by the program analysis unit;
A selection unit for selecting a nest to be evaluated from a nest included in the program code;
The nest performance information of the nest selected by the selection unit is read from the analysis result storage unit, and the effective performance of the software is indicated based on the nest performance information and the execution environment information stored in the execution environment information storage unit A performance acquisition unit for acquiring values;
A software effective performance evaluation system comprising: The said execution environment information is comprised from the information which shows the specification and performance of OS (Operating System) of an apparatus which performs software, hardware, middleware, a network, or a runtime library. Software effective performance evaluation system. 3. The software effective performance evaluation system according to claim 1, wherein the nesting performance information includes information indicating the number of steps of the program code. The software effective performance evaluation system according to claim 3, wherein the number of steps of the program code is a total number of steps in consideration of the number of repetitions when the program code includes repetition processing. A specification change detection unit for detecting a change in the specification of the execution environment of the software;
The execution environment information acquisition unit acquires the execution environment information of the execution environment of the software when the specification change detection unit detects a change in the specification. Software effective performance evaluation system described in the section. An execution environment information acquisition unit, an execution environment information storage unit, a program analysis unit, an analysis result storage unit, a selection unit, and a software acquisition performance evaluation method performed by a device including a performance acquisition unit,
The execution environment information acquisition unit acquires execution environment information indicating execution environment specifications and performance of software, and
An execution environment information storage step in which the execution environment information storage unit stores the execution environment information;
The program analysis unit analyzes a program code nest of the software and the performance of each nest;
An analysis result storage step in which the analysis result storage unit stores nest performance information indicating the performance of the nest analyzed by the program analysis unit;
A selection step in which the selection unit selects a nest for performance evaluation from the nest included in the program code;
The performance acquisition unit reads the nest performance information of the nest selected by the selection unit from the analysis result storage unit, and based on the nest performance information and the execution environment information stored in the execution environment information storage unit, software A performance acquisition step of acquiring a value indicating the effective performance of
A software effective performance evaluation method comprising: Computer
An execution environment information acquisition unit for acquiring execution environment information indicating the specifications and performance of the software execution environment;
An execution environment information storage unit for storing the execution environment information;
A program analysis unit for analyzing the nesting of the program code of the software and the performance of each nest;
An analysis result storage unit that stores nest performance information indicating the performance of the nest analyzed by the program analysis unit;
A selection unit for selecting a nest for performance evaluation from the nest included in the program code;
The nest performance information of the nest selected by the selection unit is read from the analysis result storage unit, and the effective performance of the software is indicated based on the nest performance information and the execution environment information stored in the execution environment information storage unit A performance acquisition unit that acquires values,
A program characterized by functioning as

Images