JPH0619749A - Log data compression device - Google Patents

Abstract

PURPOSE:To provide a log data compression device where a storage area can be saved on the storage of log data collected for performance evaluation of a parallel processing system executing a processing by means of distributing a load in plural central processing units. CONSTITUTION:The compression device compressing log data 2i collected on the respective central processing units 1i is provided with a conversion means 6 generating an index table 8 to which an instruction executed at a prescribed time is registered and a symbol table 10 to which an identification code corresponding to the instruction name is registered and converting the number of the execution times of the instruction into a relative value by reducing it at a prescribed rate and a registration means 7 which sequentially reads the instruction name and the number of the execution times from log data 2i and registers the identification code for the instruction name obtained from the symbol table 10 and the relative value with respect to the number of execution times, which is converted in the conversion means 6, in a prescribed registration position of a data table 9.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention compresses log data collected for performance evaluation of a parallel processing system in which loads are distributed to a plurality of central processing units to store the data in a storage area. Log data compression device.

[0002]

2. Description of the Related Art A parallel processing system having a plurality of central processing units (hereinafter referred to as "processors") or a system composed of a plurality of workstations connected to a network operates this system. In developing a program for this, it is required to efficiently execute the distributed processors to improve the processing speed of the entire system. Ideally, N
If one processor is used, N times speed improvement can be expected (generally called “number effect”), and normally the program developer repeats performance evaluation work in order to produce this number effect. There is.

Under the present circumstances, in order to identify a portion where a sufficient effect cannot be obtained, the execution status of the program for each processor is sampled, and after execution, the sampled log data is analyzed to evaluate the performance.

Further, in a parallel processing system having a plurality of processors, in order to prevent the processing amount (hereinafter referred to as "load") from concentrating on a specific processor as the number of processors increases, the program It is necessary to divide the load related to execution into units that can be executed in parallel and to allocate the load to each processor so that the load becomes uniform. However, in reality, because of the dependency relationships in the programs and the exchange of data between the programs, it is difficult to effectively allocate the load evenly, and it is difficult to achieve a significant increase in processing speed. . Therefore, in order to equalize the load, a work of sampling the execution status of the program for each processor and analyzing and evaluating the sampled log data after execution is performed.

Here, as an example of the log data, UN
FIG. 20 shows log data obtained when a C language program is executed under IX and profile is performed. In this example, the log is for one processor, and is aggregated for each instruction. In the figure,% time is the ratio of the time required for each instruction to the total execution time, cumsec
s is the time required for each instruction, #call is the number of times the instruction is executed, ms / call is the time required for one instruction, name
Indicates an instruction name.

[0006]

By the way, the rough movement of the processor can be known from the information of the above log data. However, when this is analyzed and evaluated to be applied to the load allocation of the parallel processing system, As it is, the information is insufficient and it is necessary to sample more detailed execution information. Further, in order to perform effective analysis and evaluation, it is necessary to grasp which instruction was executed by which processor as time as a standard for program creation.

However, if an attempt is made to secure a sufficient amount of information, the amount of log data to be sampled will increase, and the log data will increase significantly as the number of processors increases, which requires a large amount of storage area. There is a problem that comes. The present invention has been made in view of the above problems, and provides a log data compression device capable of saving a storage area by compressing and storing a large amount of log data for performance evaluation of a processor. It was done for the purpose.

[0008]

In order to solve the above technical problems, in the first means relating to the log data compression apparatus of the present invention, as shown in FIG.
For the performance evaluation in the parallel processing system in which the load is distributed to (i = 1,2 ...), the data are collected for each central processing unit, and each central processing unit 1i collects at a constant time interval. Using log data 2i (i = 1,2 ~) that records the number of executions of various instructions executed within this fixed time, the log data is provided with a compression device that compresses this log data and stores it in a predetermined storage area. A compression device, wherein the compression device is
Index table creation for creating an index table 8 in which the types of instructions executed by the central processing units 1i at the fixed time are registered for each central processing unit 1i and for each fixed time based on the log data 2i. Means 4
And a symbol table creating means 5 for creating a symbol table 10 in which an instruction name included in the log data 2i and a predetermined identification code corresponding to the instruction name are registered.
Then, the maximum number of executions of one type of instruction executed within the above fixed time is checked, this maximum value is reduced by a predetermined ratio and a relative value is set, and other executions have the same ratio. Conversion means 6 for converting into a relative value of the log data 2
The instruction name and the number of executions thereof are sequentially read from i, and the relative value to the identification code for the instruction name obtained from the symbol table 10 and the number of executions converted by the converting means 6 is determined based on the index table 8. And a registration means 7 for registering the data table 9 at a predetermined registration position.

Further, as shown in FIG. 1, the compression device in the second means examines the maximum value of the number of times of execution of one type of instruction executed within the fixed time, and reduces this maximum value by a predetermined ratio. Then, the relative value is set, and the conversion means 6 for converting the relative number of other execution times into the relative value of the same ratio, and a fixed holding area is secured for each of the command names, and the holding area has An instruction table 1 in which the number of executions of the instruction is registered for each central processing unit 1i and for each of the fixed time
2 is provided, the instruction name and the number of executions thereof are sequentially read from the log data 2i, and the number of executions converted into a relative value by the conversion means 6 is held in the instruction table 12 corresponding to the instruction name. The instruction table registration means 11 for registering at a predetermined registration position in the area is provided.

In the third means, the log data compression apparatus according to the present invention is provided with a number assigned to each fixed time unit at each fixed time interval as shown in FIG. A graph in which the number of executions of an instruction in time is the Y axis, an area of a portion surrounded by the X axis and parallel lines to the Y axes at the left and right ends of this graph, and an X axis including the maximum value of the number of executions of this instruction When comparing the area of the parallel graph and the area surrounded by the X axis and the parallel lines with the Y axis at the left and right ends of this graph, the former area is larger than the latter area by a predetermined ratio. If there is such a compression device according to the second means, if it is not selected, the selection device 3 for selecting the compression device according to the first means is provided.

[0011]

Now, the process of thinking of the present invention having the above construction will be described. When evaluating the performance of a program that runs a parallel processing system, we collect the number of times and which instruction was executed in a certain fixed time unit (hereinafter referred to as "cycle") in each processor during the execution of the program. Hold. 5 and 6 show examples of data held in each processor at this time.
The meaning of PE Info included in this data is shown below.

PE Info = profile (Cycle1, Info1), prof
ile (Cycle2, Info2), ... Cycle i: Cycle number Info i = {Rutin1, Call1}, {Rutin2, Call2}, ... Rutin i: Instruction name Call i: Number of executions

The sampled log data is held for each processor, but after the program is finished, it is collectively stored in the main memory for analysis. FIG. 6 shows an example in which the log data is simply stored in the main storage unit. Profile included in this data
The meaning of Info is as follows. Profile Info = profile (Cycle1, PEInfo2), profile (Cy
cle2, PEInfo),… Cycle i: Cycle number PEInfo i ＝ pe (PENum1, RutinInfo1), pe (PENum2, RutinIn
fo2),… PENun i = processor number RutinInfo i = {Rutin1, Call1}, {Rutin2, Call
2},… Rutin i: Instruction name Call i: Execution count

Normally, the numerical value in the log data (number of executions)
Is represented by a number from 32 to 64 bits. However, when analyzing a parallel system, it is not necessary to have a detailed number of executions, and it is sufficient to understand the overall tendency. Therefore, in relative terms, which processor has a biased load, and in which cycle the load is large, the biased processing is narrowed down. Also, by not holding the exact value of the number of executions, but only by holding the ratio of the number of executions that occurs to the maximum value, it is possible to express all the number of executions in 4 to 8 bits. About eight pieces of information can be compressed. On the other hand, when restoring, use the above maximum value,
The rough execution count can also be obtained from the ratio to this maximum value.

As a method for compressing the above information, there is a method (method A) using an index table and a symbol table (see FIG. 7). This is to compress symbols (character information) in log data.
For example, a command name appears many times in log data, and this command name is usually represented by 8 bits x number of characters. In this case, the number of characters is
If there are about 10 characters, 8 × 10 = 80 bits are required.

Therefore, a table (hereinafter referred to as a "symbol table") in which the above command names are associated with numerical IDs that can be identified so as to require a small amount of information is created. As a result, the symbol can be represented by a 16-bit numerical ID, so that about 1/5 of the information can be compressed. Further, when the log data is read after the execution, the symbol table is referred to.

Furthermore, various delimiters (separation codes) are included in the log data. Even in the example shown in FIG.
If the total number of cycles and the total number of processors used are fixed, it can be decoded without a delimiter, but since it differs depending on the program, the profile is used to identify the cycle.
(Cycle number,…), pe (P
The delimiter E number,…) is used. Since these delimiters also require 8 bits for each character, it is necessary to identify the delimiter with as little information as possible when compressing. Then, if the total number of cycles, the total number of processors, and the number of appearing instructions for each cycle in each processor are determined, the delimiter is not necessary. Therefore, we analyze the log data held for each processor,
By providing an index table and holding the necessary information in the index table, the log data can be represented by a string of numbers without delimiters.

Another method for compressing the above information is a method using an instruction table (method B) (see FIG. 7). Since the data is held for each instruction, the symbol table and index table shown above are unnecessary. This instruction table prepares a table for holding data of total number of processors × total number of cycles for each instruction name, and registers how many times this instruction is executed in cycle c of processor p.

When the instructions appear substantially evenly in each cycle on each processor, it is considered that the storage area can be further reduced by compressing with the method B using the instruction table. On the other hand, if there is unevenness in the number of appearances of instructions for each processor and for each cycle, it is better to use method A for compression. In the method B, an area of total number of processors × total number of cycles is prepared in advance for each instruction name, so that there is a disadvantage that a wasteful area increases when the number of appearances is uneven. On the other hand, since method A holds only the instructions that have appeared, it seems that the drawbacks of method B can be covered.

Now, the operation of the first means will be described. During operation of the parallel processing system, each central processing unit 1i (hereinafter referred to as a "processor" and each processor is provided with a "processor number") uses the fixed time as a unit (hereinafter, the unit of the fixed time is The "cycle number" is provided in sequence, and the "cycle number" is provided in sequence.) The type of instruction executed by each processor 1i and the number of times this instruction is executed are collected for each cycle, and this collection is performed for all predetermined cycles. The log data 2i is held for each processor 1i.

Next, in compressing the retained log data 2i, first, the index table creating means 4 reads the log data 2i for each processor 1i, and for each cycle in the processor 1i, an instruction executed in that cycle. Types are sequentially registered in the index table 8, and this is performed for all the processors 1i. Further, in the symbol table creating means 5, the log data 2i is searched, the command name as the character information contained therein is read, a predetermined identification code is given in correspondence with this, and these are registered in the symbol table 10. Simultaneously with this search, the conversion means 6 preferentially retains a large execution count of the same instruction executed in each cycle to obtain the maximum execution count.

Next, the registration means 7 creates the data table 9. At this time, the instruction name and the number of executions thereof are sequentially read from the log data 2i in the order of cycles, the identification code for the instruction name is obtained using the symbol table 10, and the conversion means 6 uses a relative value for the number of executions. Ask for. Then, the identification code and the relative value are cumulatively added, for example, to the number of types of instructions registered in the index table 8 and registered in the registration position of the data table 9 corresponding to this value. Then, these processes are performed for all cycles and for all processors 1i.
Log data 2i. In this way, the data table 9 in which the contents of the log data 2i are compressed is created, and this is stored in the predetermined storage area.

Next, the operation of the second means will be described.
First, the conversion means 6 searches the log data 2i, checks the number of times of execution of instructions in each cycle of all processors 1i, and obtains the maximum value of the number of times of execution of the same instruction executed during this one cycle. After that, the log data 2i for each processor 1i is referred to, and the instruction name that appears in each cycle and the number of executions of this instruction are read. When this instruction name appears for the first time, this instruction name is stored in the instruction table above. Register in the holding area. Then, the conversion means 6 converts the number of executions of this instruction into a relative value, and registers this into a registration position corresponding to the cycle number and the processor number in the holding area in which the instruction name is registered. In the same way, these processes are performed for all cycles,
This is performed for the log data 2i of all the processors 1i. In this way, the instruction table 12 in which the contents of the log data 2i are compressed is created, and this is stored in the predetermined storage area.

In the third means, either the compression device according to the first means or the compression device according to the second means is selected based on the variation in the number of executions of instructions in each cycle. Then, in the selecting means 3, it is surrounded by a graph in which the cycle number is the X-axis, the total number of executions of all instructions in each of these cycles is the Y-axis, and the X-axis and parallel lines with the Y-axis at the left and right ends of this graph. Find the area of the part. In addition, X including the maximum value of the total number of executions of the above instructions
The area of the rectangular portion surrounded by the graph parallel to the axis, the X axis, and the parallel lines to the Y axis at the left and right ends of the graph is obtained. Then, if the former area is larger than the latter area by a predetermined ratio or more, the compressor according to claim 2 is selected, and if not, the compressor according to claim 1 is selected.

[0025]

Embodiments of the log data compression apparatus according to the present invention will be described in detail below with reference to the drawings.

2 and 4 show a log data compression apparatus according to an embodiment of the present invention. This device includes a plurality of processors 21i (i =
1, 2), a first compression device 24 that creates a data table 39, a second compression device 25 that creates an instruction table, and a compression selection device 23 that selects these compression devices 24, 25. It has an aggregating unit 26 which aggregates and processes the compressed data compressed by the compression devices 24 and 25, and an output unit 27 which graphically displays or prints the compressed data aggregated by the aggregating unit 26 on a screen. . Further, each processor 21i has a log data extraction unit 22i (i = 1,2 ...) That extracts log data.

The log data compression apparatus according to the present embodiment compresses / processes data when collectively storing the log data held by each processor 21.

The overall processing of this embodiment will be described with reference to the flowchart shown in FIG. In each processor 21 in a distributed environment such as a parallel processing system or a network system, when each processor 21 executes a parallel program that operates in parallel, which instruction is executed many times by the OS (operating system) at a constant time interval. Taka is counted. Then, the log data extraction unit 22 of each processor 21 extracts the program execution information (S1). FIG. 5 shows the compression device 2
4 shows an example of data input to 4,25,
Such log data is held in each processor 21.

Next, the compression / selection device 23 refers to the log data of each processor 21 and calculates the respective ratios of the variation value S1 and the ideal area T1 relating to the processor and the variation value S2 and the ideal area T2 relating to the cycle. (S2). Then, the ratio calculated above, S1 / T1
Alternatively, S2 / T2 is compared with a predetermined threshold Limit (S
3). At this time, if the threshold Limit is smaller than any of the ratios, step (S5) is executed, otherwise step (S4) is executed.

In the step (S4), each processor 2 is operated by the compression method using the index table 37 and the symbol table 38 in the first compression device 24.
Compressed data (data table 39) is created from the log data in No. 1 and held in the storage area of the main memory. In step (S5), compressed data (data section of the instruction table 44) is created from the log data on each processor by the compression method using the instruction table in the second compression device 25, and the predetermined storage in the main memory is performed. Save to area. Thereafter, the compressed data output from the compression device 24 (or the compression device 25) is totalized by the totalization unit 26 (S
6) Further, the totalized result is printed out as a graph on the screen or printed out as a list from the output unit 27 (S7) and used as a material for performance analysis.

FIG. 4 is a detailed block diagram of the compression devices 24 and 25 and the compression selection device 23. The first compression device 24 includes an initial setting unit 32, a conversion unit 33 that converts the number of times an instruction is executed into a relative value, a registration unit 36 that registers this relative value at a predetermined registration position in the data table 39, An index table creation unit 34, a symbol table creation unit 35, and a storage unit 40 that holds compressed data and the like are provided. The storage unit 40 has an index table 37, a symbol table 38, a data table 39, and the instruction table 12.

The operation of the first compression device 24 will now be described. As shown in the step (S4), the compression device 24 creates the index table 37 and the symbol table 38, and also creates the data table 39 using these tables. First, the index table 37 created by the index table creation unit 34
The creation procedure of will be described based on the flowchart of FIG. The index table 37 holds the total number of processors PE, the total number of cycles Cycle, and the types of instructions executed by the processor 21 for each cycle.
The index table 37 shown in FIG.
It is a bit and the data length is an example of data of Cycle × PE + 2. First, the number of processors for parallel processing is checked, and this is registered in the 0th element of the table (S1
0). Further, the total number of cycles (Cycle) is checked by referring to the log data of the first 0th processor 21, and this total number of cycles is registered in the first element (S11).

Then, the index table 28 is created for all the processors 21 by counting the types of instructions in each processor cycle (S12). This is performed for each processor. For example, the processor Pth log data is searched, the types of instructions appearing in cycle C are counted, and registered in the (Cycle) × (P−1) + C + 1 element in the table. (S16). This registration work is performed for all the processors (S13 to S18). In the example of the index table 37 shown in FIG. 8, the number of appearing instructions in cycle 1 of processor 0 is 3 (second element), and the number of appearing instructions in cycle 2 of processor 0 is 4 (third element). Element).

Next, the procedure for creating the symbol table 38 created by the symbol table creating section 35 will be described with reference to the flowchart of FIG. This symbol table 3
8 is a table showing the correspondence between the instruction name and the instruction ID in which the instruction name is associated with a predetermined code as shown in FIG. At the time of creation, first, the number of processors 21 and the number of total cycles are initialized, and processing is performed for each processor and for each cycle (S20 to S23).
Then, the log data relating to each processor 21 is sequentially searched, and, for example, the instruction name (RName) appearing in the cycle C in the log data of the processor P is read (S2
4) At this time, it is checked whether the instruction name is already registered in the symbol table 38 (S25). At this time, if the instruction name appears for the first time, the identifiable ID, for example, the integer "1" is used as the instruction ID, and the instruction name and "1" are associated and registered in the symbol table 38 (S26).

Further, at the same time as the creation of the symbol table 37, the conversion unit 33 reads the execution count R of the instruction name from the log data and compares the execution count R with the maximum value Max of the execution counts up to that time, which is larger. The maximum value Max is detected from the number of executions by substituting the numerical value of the one for Max (S27). Note that when implementing, the total number of appearance instructions (RutinNum) of the second and subsequent elements in the index table is counted first, and the length is RutinN.
Create a um × 2 + 1 table. And Ruti as the 0th element
Register nNum, and the instruction name and instruction I for each two elements from the first element
Register D in association.

Next, the procedure for creating the data table 39 created by the registration unit 36 will be described with reference to the flowchart of FIG. First, when creating the data table 39, the initial setting unit 32 specifies the maximum value of the number of executions and the number of stages (N) indicating the ratio of the number of executions to the maximum value (S30). FIG. 12 shows an example of this data part. Here, the maximum value Max detected previously is registered in the 0th element, and N is registered in the first element (S31). Then, the log data is searched for each cycle of each processor 21 (S32 to S34), and the appearing instruction name RName is read (S35). Further, referring to the symbol table 38, the instruction ID corresponding to the above instruction name is read and registered in the data section of the data table 39 (S36). Also, the maximum value Max of the number of executions R that appears appears
The ratio Ratio (relative value) to is calculated by the following (Equation 1) and Ratio = R × N / Max (Equation 1), and registered in the next element of the data section (S37), and this is calculated for each processor. Do about cycle (S
38-S40). Since the number of executions is rounded up to the specified step, if you specify 10 steps, for example, 1
Represented by the numbers ~ 10.

FIG. 14 shows an example of compression of log data (see FIG. 5) when the maximum value is 1249 and the step is 10. In this log data, if you look at the instruction at cycle 1 of processor 0, the instruction name is after-rewrite-arg
So after-re referencing the index table
The command ID1 corresponding to write-arg is obtained and registered in the first element of the data section. On the other hand, the number of executions of after-rewrite-arg is 621, so when substituting this into (Equation 1) above, Ratio = 621 x 10/1249 = 4.9, so this is rounded off to obtain "5". Is registered in the second element of the data section. Similarly, processing is performed for the instructions that appear thereafter. As described above, compressed data is created by compressing log data in the data section, and these compressed data are stored in a predetermined storage area of the storage section. After that, the compressed data is read and restored, and this is analyzed to evaluate the performance of the parallel processing system.

Now, the process of restoring the compressed data will be described. When using the above-described compressed data, by referring to the index table 37, it is possible to know from which numerical element column of the data portion of the data table 39 the desired data is registered.
FIG. 15 shows a procedure for restoring data whose processor number is (Processor) and whose cycle number is C. First, referring to the index table 37, the total number of processors PE and the total number of cycles Cycle are read (S5).
0). And from the 2nd element to the Cycle x Processor + C
Add the values up to the element and get this total Skip Count (S
51). Further, the instruction type Rnum appearing in the cycle C in the processor Processor is read from the Cycle × Processor + C + 1 element of the index table (S52). Next, based on the above Rnum, data table 3
9 to read the instruction name and the number of executions (S54 to S)
60). First, the SkipCount element is read from the data table 39 (S56), and since this value is the instruction ID, the symbol table 38 is referenced to restore the corresponding instruction name (S57). Further, the value R of the next element is read (S5
8) Since this is the ratio when the maximum value Max is shown in N stages, the number of executions (R) and R = Max × R / N are calculated and restored from this (S59). In this way, the process of reading and restoring the compressed data is repeated Rnum times (S5).
5 to S60).

Here, as a concrete example, for example, the processor 3
Paying attention to the number, which instruction is executed many times is restored (see FIG. 14). First, the total Skip Count of the read numbers is calculated by sequentially referring to the second element to the 128 × 3 + 1th element in the index table 37. Also,
The 128 × 3 + 2th element is the number of appearing instructions RNum in cycle 1 in processor 3. Then, the Skip Count element of the data part of the data table 39 is read,
Since this element includes the instruction ID, the symbol table 38 is referred to retrieve the instruction name corresponding to this instruction ID. For example, if the ID is 3, the instruction name is unify (see FIG. 10). The numerical values included in the following elements are ratios when the maximum value 1249 is shown in 10 steps. For example, if this is 6, the actual number of executions is 1249 × 6/10 = 749.4.

In this way, the instruction execution count of cycle 1 is processed in the same manner as the number of appearances RNum. Further, by performing the same processing for cycles 2 to 128, it is possible to restore the log data for analyzing the instruction execution state in the processor No. 3. Next, the second compression device 25
Will be described. As shown in FIG. 4, the compression device 25 includes an initial setting unit 41, a conversion unit 42 for converting the number of executions of an instruction into a relative value, an instruction table registration unit for registering an instruction name and this relative value in an instruction table 44. 43 and the storage unit 4
0 and the storage unit 40 has the instruction table 44.

The instruction table 44 used in this apparatus
Is a data example of the total number of appearing instructions × (1 + total number of processors PE × total number of cycles Cycle) +4. The instruction table 44 holds data for each instruction, and therefore the symbol table 38 and the index table 37 shown in the first compression device 31 are not required. In the instruction table 44, a data area of the total number of processors × total number of cycles is secured for each instruction name, and the number of times the instruction is executed in the cycle c of the processor p is registered therein.

Now, the procedure for creating the instruction table in the second compression device 25 will be described with reference to the flowchart of FIG. First, in the initial setting section 41, the number of processors 21 is checked, and further the predetermined processors 21
The total number of cycles is obtained by referring to the log data regarding. Then, the log data regarding all the processors 21 is searched to find out how many kinds of instruction names are executed,
At the same time, the maximum number of executions (maximum value Max) of the number of executions of an instruction that appears in one cycle is determined, and the ratio of each number of executions to the maximum value of the number of executions is expressed as externally input data. How many steps (N) this maximum value is
Is designated (S70).

In this way, the total number of processors PE for each instruction name as shown in FIG.
× Create a table with a constant data area of the total number of cycles, Cycle. The zeroth element of the instruction table 44 is the total number of processors PE, and the first element is the total number of cycles C.
ycle, the maximum value Max is registered in the second element, N representing the stage is registered in the third element, and the data part is initialized to "0" (S).
71).

When the above initialization is completed, the instruction table registration unit 43 sequentially searches the log data held for each processor and reads the appearing instruction name (RName) (S75). If the command name appears for the first time (S76), this command name is registered in the command table 44 (S77). Further, the execution count R of this command is read out, and the conversion unit 42 calculates the ratio of the execution count R to the maximum value Max by the following formula: Ratio = R × N / Max. If the log data has the processor number P and the cycle number C, the log data is registered at the position of P × Cycle + C element in the data area corresponding to the instruction name (S78). In this way, the process of reading the log data and registering it at a predetermined position in the instruction table 44 is performed for each cycle of all the processors (S72 to S8).
0).

Here, a concrete example of the case where the instruction table 44 is created based on the log data shown in FIG. 5 will be shown. At this time, Max = 1249 and N = 10. First of all, when the instruction of cycle 1 is checked by the processor 1, the instruction name is after-
Since this is rewrite-arg, if this command name appears for the first time, it is registered in the command table 44. Also, the number of times this after-rewrite-arg is executed is 621, so if you calculate the Ratio from this, 621 × 10/1249 = 4.
It becomes 9, and "5" obtained by rounding off this is registered in the first element of the corresponding data part. Similarly, processing is performed for the instructions that appear thereafter. In addition, since "0" is assigned to all data parts in advance,
The elements for the processor number and the cycle number where the instruction does not appear are “0”. As described above, the instruction table 44 shown in FIG. 16 is created.

By the way, when the instructions appear substantially evenly in each cycle of each processor 21, the second compression device 25 is used, and the compression device 25 creates an instruction table to compress the log data. The storage area can be reduced more effectively. On the other hand, it is effective to use the first compression device 24 when the number of times the instructions of each processor 21 appear in each cycle varies. In this case, the second compression device 25 prepares an area of the total number of processors × total number of cycles for each instruction name in advance, so that a wasteful area in which “0” is registered in the data section increases. is there.

As a measure of the above-mentioned variation, a log data graph shown in FIG. 18 showing a distribution with respect to time is used. This is a graph in which the cycle number is on the x-axis and the number of appearing instructions is on the y-axis. Also,
Similarly, the distribution regarding the processor 21 is also represented by a graph with the processor number as the x-axis and the number of appearing instructions as the y-axis.
In this case, the log showing the distribution represented by the graph with no variation shown in FIG. 18A in which the instructions appear uniformly throughout the whole, compared to the graph with variation in the distribution shown in FIG. 18B. For the data, it is preferable to use the second compression device 25 when compressing the data. Also, FIG.
To determine the variation in the graph shown in FIG. 8, the area S of the portion surrounded by the x-axis and the graph is close to the ideal area T calculated by the maximum number of instruction occurrences in the graph × the number of cycles (or the number of processors). It can be said that there is not much variation.

Here, the compression selection device 23 that selects a device that can effectively reduce the storage area from the compression devices of both the first compression device 24 and the second compression device 25 described above.
Will be described. This apparatus, as shown in FIG. 4, has an initial setting unit 50, an actual area calculation unit 47 for obtaining the area of a portion surrounded by a graph showing the distribution of the number of appearing instructions, and a graph based on the maximum value of the number of appearing instructions. The ideal area calculation unit 48 for obtaining the area is compared with the both areas and the compression device 2
It has a comparison and selection section 49 for selecting 4, 25.

Next, the processing of the compression selection device 23 will be described with reference to the flowchart of FIG. First, the threshold value Limit (for example, 0.5) and the total cycle number Cycle are set in the initial setting unit 50 (S90). Then, the ideal area calculation unit 48 counts the number of appearing instructions for each cycle (S91), and checks the maximum number of appearing instructions M from all the cycles. From this, the ideal area T is calculated based on the following equation: ideal area T = M × Cycle (S92).

Further, the actual area S is calculated by the actual area calculation unit 47.
This real area S can be approximated by the following equation, where Fi is the number of instructions that appear on each cycle (or each processor) i. Therefore, the real area S = (F1 + 2 × F2 + 2 × F3 ... +2 x Fn-1 + F
The value of each Fi is substituted into n) / 2 for calculation (S93).

Then, the comparison / selection unit 49 compares the real area S obtained above with the ideal area T, and if the result S / T is larger than the threshold value Limiat (for example, 0.5), then The first compression device 24 is selected, and if not, the second compression device 25 is selected (S94 to S96), and compression is performed using the selected compression device. Also, when the compression device is selected based on the distribution regarding the processor, the variation regarding the processor is calculated in the same manner as above, and the compression device is selected.

Therefore, according to the log data compression apparatus of this embodiment, a plurality of central processing units are collected for performance evaluation of a parallel processing system in which loads are distributed to a plurality of central processing units. Since the log data in which the number of instructions executed in step 1 is collected is compressed using the first compression device 24 or the second compression device 25, the storage area for storing the compressed log data is significantly large. The effect is that this storage area can be saved.

Further, since the compression selection device 23 is provided and the compression device is selected based on the distribution of the number of execution instructions in each central processing unit, it is possible to select the compression device which is expected to have a higher compression effect. There is an effect that the storage area can be saved more effectively.

[0054]

As described above, according to the first means of the log data compression apparatus of the present invention, each central processing unit 1 is used.
Index table creating means 4 for creating an index table 8 in which a type of an instruction executed at a fixed time is registered in a compression device for compressing the log data 2i collected for i, an instruction name and an identification code corresponding to the instruction name. The symbol table creating means 5 for creating the symbol table 10 in which is registered, and the relative value is set by reducing the maximum value of the number of times of execution of the instruction at a predetermined ratio, and the other ratio of execution is the same as the relative ratio. The conversion means 6 for converting into a value, the instruction name and the number of executions thereof are sequentially read from the log data 2i, the identification code for the instruction name obtained from the symbol table 10 and the number of executions converted by the conversion means 6 are obtained. Since the configuration having the registration means 7 for registering the relative value with respect to the predetermined registration position of the data table 9 is adopted, compression is performed. Storage area for storing log data is effective such can be reduced achieving savings of the storage area of the.

Further, the compression device in the second means secures a constant holding area for each of the command names and the converting means 6, and this holding area is provided for each of the central processing unit sections 1i and the constant area. An instruction table 12 in which the number of executions of instructions at each time is registered is provided, and the instruction name and the number of executions are sequentially read from the log data 2i, and the number of executions is converted into a relative value by the conversion means 6. Since the command table registration means 11 for registering the command name at a predetermined registration position in the holding area of the command table 12 corresponding to the command name is adopted, the storage area for storing the compressed log data is reduced. There is an effect that the storage area can be saved.

In the third means, the log data compressing apparatus according to the present invention is provided with a number assigned to each unit of each fixed time interval at the above fixed time interval on the X axis, and the number of executions of an instruction in each of these fixed time intervals. , Which is the Y axis, and the area enclosed by the X axis and the parallel lines to the Y axis at the left and right ends of the graph, and the graph including the maximum number of executions of this instruction and the X axis, When the area of the part surrounded by the lines parallel to the Y axis at the left and right ends of the graph is compared with the area of the latter compared with the area of the latter by a predetermined ratio or more, the second means is used. A configuration is provided in which a selection unit 3 that selects the use of the compression device according to the first means otherwise is used, and the compression device is based on the distribution of the number of execution instructions in each cycle. Since I chose to select An effect such log data compression can save storage space can be reduced to.

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is a configuration diagram of a log data compression device according to an embodiment of the present invention.

FIG. 3 is a flowchart showing an operation of the log data compression device according to the exemplary embodiment of the present invention.

FIG. 4 is a configuration diagram of a compression device and a compression selection device according to an embodiment.

FIG. 5 is a diagram showing an example of log data collected by each processor after execution of a parallel system.

FIG. 6 is a diagram showing an example of a storage state of log data.

FIG. 7 is a diagram showing a form of data after compression of log data.

FIG. 8 is a diagram showing an index table.

FIG. 9 is a flowchart showing a procedure for creating an index table.

FIG. 10 is a diagram showing a symbol table.

FIG. 11 is a flowchart showing a procedure for creating a symbol table.

FIG. 12 is a diagram showing an example of a data table.

FIG. 13 is a flowchart showing a procedure for creating a data table.

FIG. 14 is a diagram showing an example of a table created by compressing log data.

FIG. 15 is a flowchart showing a procedure for restoring log data from a data table.

FIG. 16 is a diagram showing an example of an instruction table.

FIG. 17 is a flowchart showing a procedure for creating an instruction table.

FIG. 18 is a graph showing a distribution of variations in the number of execution instructions.

FIG. 19 is a flowchart showing the operation of the compression selection device.

FIG. 20 is a diagram showing an example of log data under UNIX.

[Explanation of symbols]

1, 21 Central processing unit (processor) 2, 22 Log data (log data extraction unit) 3, 23 Selection means (compression selection device) 4, 34 Index table creation means (index table creation unit) 5, 35 Symbol table creation Means (symbol table creation unit) 6,33,42 Conversion means (conversion unit) 7,36 Registration means (registration unit) 8,37 Index table 9,39 Data table 10,38 Symbol table 11,43 Instruction table registration means ( Instruction table registration unit) 12,44 Instruction table

Claims (3)

[Claims] 1. A plurality of central processing units (1i; i = 1,2)
To) are collected for each central processing unit (1i) for performance evaluation in a parallel processing system in which the load is distributed to each central processing unit (1i).
) Uses log data (2i; i = 1,2 ...) that records the number of executions of various instructions executed within this fixed time, and has a compression device that compresses this log data and stores it in a predetermined storage area. The log data compressing device according to claim 1, wherein the compressing device is such that, based on the log data (2i), the type of instruction executed by each central processing unit (1i) at the certain time is each central processing unit (1i). Index table creating means (4) for creating an index table (8) registered for each and every fixed time, an instruction name included in the log data (2i) and a predetermined identification corresponding to the instruction name. A symbol table creating means (5) for creating a symbol table (10) in which codes are registered, and a maximum value of the number of times of execution of one type of instruction executed within the fixed time are examined, and this maximum value is set to a predetermined value. The conversion means (6) is used to reduce the ratio and set the relative value, and to convert the other execution times into relative values of the same ratio, and the instruction name and the execution times are sequentially read from the log data (2i). At the same time, the identification code for this instruction name obtained from the symbol table (10) and the relative value for the number of executions converted by the conversion means (6) are determined based on the index table (8). 9. A log data compression device comprising: a registration means (7) for registering at a predetermined registration position of 9). 2. A plurality of central processing units (1i; i = 1,2)
To) are collected for each central processing unit (1i) for performance evaluation in a parallel processing system in which the load is distributed to each central processing unit (1i).
) Uses log data (2i; i = 1,2 ...) that records the number of executions of various instructions executed within this fixed time, and has a compression device that compresses this log data and stores it in a predetermined storage area. The log data compression apparatus, wherein the compression apparatus examines the maximum value of the number of executions of one type of instruction executed within the fixed time, reduces the maximum value by a predetermined ratio, and sets a relative value. However, for other execution times, a conversion means (6) for converting into a relative value of the same ratio as this, and a fixed holding area for each instruction name are secured, and in the holding area, each of the central processing units An instruction table (12) in which the number of executions of an instruction is registered for each (1i) and for each fixed time is provided. The instruction name and the number of executions are sequentially read from the log data (2i), and the number of executions is also read. The conversion means (6)
Log data characterized by having an instruction table registration means (11) for registering the value converted into a relative value at a predetermined registration position in the holding area of the instruction table (12) corresponding to this instruction name. Compressor. 3. A plurality of central processing units (1i; i = 1,2)
To) are collected for each central processing unit (1i) for performance evaluation in a parallel processing system in which the load is distributed to each central processing unit (1i).
) Uses log data (2i; i = 1,2 ...) that records the number of executions of various instructions executed within this fixed time, and has a compression device that compresses this log data and stores it in a predetermined storage area. In the log data compression device, a number is given to each fixed time unit at each fixed time interval as the X axis, and a graph with the Y axis representing the number of executions of instructions at each of these fixed times, the X axis, and The area of the part surrounded by the Y-axes at the left and right ends of this graph, the graph parallel to the X-axis containing the maximum number of executions of this instruction, and the part surrounded by the X-axis and the Y-axes at the left and right ends of this graph. The area of the former is larger than the area of the latter by a predetermined ratio or more, and the compressor according to claim 2 is selected. Otherwise, the compressor according to claim 1 is selected. The selection means (3) for performing is provided. Fine claim 2 log data compression apparatus as claimed.