JP2016091491A - Information processor, compression method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the storage capacity of a storage device.SOLUTION: A totaling part receives operation data needed for data processing in a web system by a reception part, and sections the data into rank values of verification completion showing a rank such that smooth probability density distribution information can be restored by smoothing in advance, and information updated by a determination part which determines which rank a processing time in the operation data belongs to, an update part which updates frequencies by ranks, and a totaling part which totaling processing times by the ranks is stored in a compressed data storage part.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing apparatus, a compression method, and a program.

  In Patent Document 1, based on target data for generating a histogram, a determination unit that determines a plurality of division candidate points that are candidates for division points for determining each section of the histogram, The change value calculation means for calculating the change value of the frequency of the target data, respectively, and the decision means determines by sampling the plurality of division candidate points based on each change value calculated by the change value calculation means Selection means for selecting a smaller number of division candidate points than the number of division candidate points that have been generated, and histogram generation means for generating a histogram of the target data using the division candidate points selected by the selection means. A histogram generation apparatus characterized by this is disclosed.

JP 2012-133608 A

  By the way, there is an information processing apparatus that stores the frequency (number) of processing times of the web system in a storage device so that the processing time required for data processing of the web system can be analyzed later.

  However, if the processing time of the web system is stored in the storage device as it is, there is a problem that a storage device having a large storage capacity is required.

  For example, the processing time from 0 second to 48 hours is divided into classes of equal class width of “0.001 second”, the frequency of the processing time “0.001 second” is “a”, and the processing time “0”. .002 seconds is “b”, the processing time is “47 hours 59 minutes 59 seconds, .999 seconds” is “c”, and the frequency is stored in the storage device at all processing times. The number of classes for the processing time is “172,800,000”, and a large storage capacity is required.

  Therefore, an object of the present invention is to provide a technique capable of reducing the storage capacity of a storage device.

  The present application includes a plurality of means for solving at least a part of the above-described problems. Examples of the means are as follows. In order to solve the above-described problem, the information processing apparatus according to the present invention divides a smoothed target width that can calculate smooth and distortion-free probability density distribution information from the frequency of each class into a class that has been verified in advance. And a totaling unit that updates the frequency of the processing time of the web system for each class and the total processing time for each class, and the upper limit values for each class are arranged approximately in a geometric sequence, and the class width of each class is The moving average for each smoothing target width is approximately the same between adjacent classes, and the representative value of the class after smoothing the class is a value having a significant number of digits in the order of seconds, minutes, or hours. It is characterized by that.

  Further, in the above information processing apparatus, a class value verification unit that verifies a class value or calculates a smoothing target width that can calculate smooth and distortion-free probability density distribution information by smoothing before driving the totaling unit. It may be characterized by having.

  The information processing apparatus further includes a determination unit that determines which of the ranks the processing time of the web system belongs to, and the rounding-up unit determines the processing time of the web system as the determination Rounded up to a multiple of the class width corresponding to the rank determined by the section.

  In addition, the compression method of the information processing apparatus according to the present invention has a smooth target width that can restore the processing time required for the data processing of the web system from the frequency of each class and the probability density distribution information without distortion. To update the frequency of the processing time of the web system and the total processing time for each class, and the upper limit value for each class is roughly arranged in a geometric sequence, The moving average for each smoothing width of the class width is approximately the same between adjacent classes, and the representative value of the class after smoothing the class has an effective number of digits of several digits in terms of seconds, minutes, or hours. It is the value which has.

  The program of the information processing apparatus according to the present invention indicates that there is a smooth target width that can restore the smooth and distortion-free probability density distribution information from the frequency of each class for the processing time required for web system data processing. The information processing apparatus is made to function as an aggregation unit that updates the frequency of the processing time of the web system and the total processing time for each class, divided into previously verified classes, and the upper limit value for each class is approximately equal. Arranged in a ratio sequence, the moving average for each smoothing target width of the class width for each class is approximately the same between adjacent classes, and the representative value of the class after smoothing the class is in seconds, minutes, or hours It is a value having an effective number of digits of about several digits.

  In the present invention, the storage capacity of the storage device can be reduced. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

It is the figure which showed the structural example of the network system to which the information processing apparatus which concerns on one Embodiment of this invention is applied. It is the figure which showed the hardware structural example of information processing apparatus. It is the figure which showed an example of the functional block of information processing apparatus. It is the figure which showed an example of the data flow between the functions relevant to information processing apparatus. It is the figure which showed the verification method of a class value. This is an expression related to the determination condition of requirement 1 of the class value. This is an expression related to the determination condition of requirement 2 of the class value. It is another formula related to the determination condition of requirement 2 of class value. It is the figure which showed the operation example of the determination process of the requirement 3 of a class value. It is a flowchart explaining operation | movement of the total part of information processing apparatus. It is a flowchart explaining the operation example of a statistical process. It is the figure which showed the example of the cumulative probability distribution data for verification. It is the figure which showed the outline | summary of the production | generation rule 1, and a requirement suitability determination result. It is the figure which showed the example of the verification progress data of the class value produced | generated by the production | generation rule 1. FIG. It is a figure explaining the analysis example of the resolution of the class value in the production | generation rule. It is the figure which showed the example of analysis of the ratio of the adjacent class value in the production rule 1. It is the figure which showed the ratio of the moving average of the class width of the smoothing object in the production rule 1. It is the figure which showed the verification example of the smoothness of the probability density data produced | generated from the class value by the production | generation rule 1. It is the figure which showed the outline | summary of the production | generation rule 2, and a requirement suitability determination result. It is the figure which showed the example of the input data format to the information processing apparatus of the class value by the production | generation rule 2. FIG. It is the figure which showed the example of the verification progress data of the class value produced | generated by the production | generation rule 2. FIG. It is a figure explaining the analysis example of the resolution of the class value in the production rule 2. It is the figure which showed the example of analysis of the ratio of the adjacent class value in the production rule 2. It is the figure which showed the ratio of the moving average of the class width of the smoothing object by the production | generation rule. It is the figure which showed the verification example of the smoothness of the probability density data produced | generated from the class value by the production | generation rule 2. It is the figure which showed the outline | summary of the production | generation rule 3, and a requirement suitability determination result. It is the figure which showed the example of the input data format to the information processing apparatus of the class value by the production | generation rule 3. FIG. It is the figure which showed the example of the verification progress data of the class value produced | generated by the production | generation rule 3. FIG. It is a figure explaining the analysis example of the resolution of the class value in the production rule 3. It is the figure which showed the example of analysis of the ratio of the adjacent class value in the production rule 3. It is the figure which showed the ratio of the moving average of the class width of the smoothing object in the production rule 3. It is the figure which showed the verification example of the smoothness of the probability density data produced | generated from the class value by the production | generation rule 3. It is the figure which showed the outline | summary of the production | generation rule 4, and a requirement compatibility determination result. This is information expressed by a formula in which the class of generation rule 4 is rounded up to a multiple of the class width for each rank. It is the figure which showed the example of the input data format to the information processing apparatus of the class value by the production | generation rule 4. FIG. It is a figure explaining the analysis example of the resolution of the class value in the production | generation rule. It is the figure which showed the example of analysis of the ratio of the adjacent class value in the production rule 4. It is the figure which showed the ratio of the moving average of the class width of the smoothing object in the production rule 4. It is the figure which showed the verification example of the smoothness of the probability density data produced | generated from the class value by the production | generation rule 4. It is the figure which showed another example of the input data format to the information processing apparatus of the class value by the production | generation rule 4. FIG. It is the figure which showed the example of operation data. It is the figure which showed the data structural example of the compression data storage part. It is the figure which showed the example of statistical process progress data. It is the figure which showed the example of cumulative probability data. It is the figure which showed the example of probability density data. It is the figure which showed the example which produced | generated many probability density data simultaneously. It is the figure which showed the example of the report containing a compliance rate etc. It is the figure which showed the example which mounted the process rounded up to the multiple of the class width for every rank in a total part.

  FIG. 1 is a diagram illustrating a configuration example of a network system to which an information processing apparatus according to an embodiment of the present invention is applied. As shown in FIG. 1, the network system includes an information processing device 1, a web system 2, a terminal device 3, and a network 4. The network 4 is, for example, the Internet.

  The web system 2 has a web server 2a. In FIG. 1, the web system 2 has one web server 2a, but it may have two or more.

  The terminal device 3 is connected to the web system 2 (web server 2a) via the network 4. The terminal device 3 transmits request data to the web system 2. In FIG. 1, only one terminal device 3 is shown, but two or more terminal devices 3 may be used.

  The request data transmitted from the terminal device 3 to the web system 2 includes, for example, a URL (Uniform Resource Locator), a request parameter, and the like. The web system 2 performs predetermined processing according to the request data transmitted from the terminal device 3 and transmits the processing result to the terminal device 3.

  The web system 2 stores operation data related to data processing performed according to the request data of the terminal device 3. The operation data includes the date and time when the web system 2 processed the request data, the identification information for distinguishing the processing content of the web system 2 that processed the request data, and the processing that the web system 2 required to process the request data. Time etc. are included.

  The information processing apparatus 1 is connected to the web system 2 via the network 4. The information processing apparatus 1 receives operation data from the web system 2 at predetermined intervals such as one day or one week. When the information processing apparatus 1 receives the operation data from the web system 2, the information processing apparatus 1 extracts predetermined information included in the operation data, compresses it (lossy compression), and stores it in a storage device such as an HDD (Hard Disk Drive). To do.

  FIG. 2 is a diagram illustrating a hardware configuration example of the information processing apparatus. The information processing apparatus 1 includes, for example, an arithmetic device 61 such as a CPU (Central Processing Unit), a main storage device 62 such as a RAM (Random Access Memory), and an auxiliary storage device 63 such as an HDD, as shown in FIG. A communication interface (I / F) 64 for connecting to a communication network by wire or wireless, an input device 65 such as a mouse, keyboard, touch sensor or touch panel, a display device 66 such as a liquid crystal display, and a DVD (Digital Versatile) And a read / write device 67 that reads / writes information from / to a portable storage medium such as a disk).

  The predetermined program may be installed from, for example, a storage medium read by the read / write device 67, or may be installed from a network via the communication I / F 64.

  The functional configuration of the information processing apparatus 1 described above is classified according to main processing contents in order to facilitate understanding of the configuration of the information processing apparatus 1. The present invention is not limited by the way of classification and names of the constituent elements. The configuration of the information processing apparatus 1 can be classified into more components depending on the processing content. Moreover, it can also classify | categorize so that one component may perform more processes. Further, the processing of each component may be executed by one hardware or may be executed by a plurality of hardware.

  FIG. 3 is a diagram illustrating an example of functional blocks of the information processing apparatus 1. As illustrated in FIG. 3, the information processing apparatus 1 includes a receiving unit 11, a totaling unit 12, a class value 13, and a compressed data storage unit 14. The counting unit 12 includes a determination unit 12a, a counting unit 12b, and a totaling unit 12c. The operation data 11d and the class value 13 are stored in a storage device (not shown).

  The receiving unit 11 receives the operation data 11 d from the web system 2. The receiving unit 11 receives the operation data 11d from the web system 2 at a predetermined interval such as one day or one week.

  For example, the function of the totalization unit 12 is realized by the arithmetic device 61 executing a predetermined program loaded from the auxiliary storage device 63 or the like to the main storage device 62. The receiving unit 11 is realized, for example, when the arithmetic device 61 uses the communication I / F 64. The class value 13 may be data referred to by the totalization unit 12 loaded from the auxiliary storage device 63 or the like to the main storage device 62, or data loaded to the main storage device 62 as part of a predetermined program. good. The compressed data storage unit 14 is realized, for example, when the arithmetic device 61 uses the main storage device 62 or the auxiliary storage device 63.

  The compressed data storage unit 14 stores the data compressed by the counting unit 12. Note that the information processing apparatus 1 may not include the compressed data storage unit 14. For example, another device connected to the network 4 may have the compressed data storage unit 14.

  FIG. 4 shows a data flow between functional blocks when the verification unit 7 of the class value 13 input to the totaling unit 12 of the information processing apparatus 1 is also provided.

  The totaling unit 12 accumulates, in the compressed data storage unit 14, data obtained by compressing the operation data 11 d supplied from the receiving unit 11 using the class value 13.

  When receiving operation data from the web system 2 every other day, the reception unit 11 receives operation data for one day. In addition, when receiving operation data from the web system 2 every other week, the receiving unit 11 receives operation data for one week.

  At a later date, when the report on the operation data 11d is desired, the statistical processing 8 generates the cumulative probability data 8a from the data accumulated in the compressed data storage unit 14 and the class value 13, and further uses the smoothing target width 7a. Then, the probability density data 8b is generated, and the report unit 9 outputs or visualizes a report 9f that can include an average value of the processing time, a compliance rate, and the like.

  The class value verification unit 7 provides information on the accuracy of data scheduled to be extracted by the statistical process 8 after the information compressed by the totaling unit 12 only once before starting the totaling unit 12 in the information processing apparatus 1. A class value 13 that can generate compressed data without impairing data and a smoothing target width 7a for generating probability density data from which noise included in the operation data is removed are calculated.

  Here, the class value verification unit 7 may be provided in the same information processing device 1 as the totaling unit 12 of the information processing device 1, or is processed prior to the execution of the totaling unit 12 of the information processing device 1. It may be a method.

  FIG. 5 is a flow showing a series of steps constituting the class value 13 verification method. In addition, when the class value verification unit 7 is implemented, it can be realized by a processing group equivalent to the functional block shown in FIG.

  In step S 79 in the class value verification process 7, a process equivalent to the “aggregation unit 12 of the information processing apparatus 1” is used as the “statistical process 12 in the class value verification process 7”.

  Next, in step S79, the statistical process 8 generates cumulative probability data 8a from the compressed data storage unit 14, and further generates probability density data 8b from the slope of the cumulative probability graph in the representative value of the class. It is verified that 8b becomes a smooth graph, and a verification result 7c is output.

  In order to prevent the lack of information in the report 9f such as the compliance rate output by the report unit 9 in FIG. 4 by this verification method, “statistical processing 8 immediately before the report unit 9 in FIG. 4” and “ It is reasonable to make the statistical process 8 ”in the verification process 7 in FIG. 5 an equivalent process.

  Here, as a method of verifying the class value, if the probability density data 8b is generated and visualized from the data generated in the verification data generation step S78, the probability density graph may be slightly distorted as shown in FIG. There is no objective judgment about the smoothness and distortion of the generated probability density data.

Therefore, in the class value verification processing 7, since the probability density data becomes a smooth graph, there exists a “smoothing target width” that satisfies the following requirements for the class value generated according to some generation rule. It is determined whether or not.
Requirement 1: The class value is approximately a geometric progression, and the standard deviation of the ratio of adjacent class values is smaller than the average of all the resolutions obtained by dividing the class width by the class value.
Requirement 2: The absolute value of the value obtained by subtracting 1 from the ratio of the adjacent class width to the total adjacent class width of the smoothing target width is smaller than the average value of the resolution obtained by dividing the class width by the class value. .
-Requirement 3: The representative value of the class smoothed with a predetermined smoothing target width should be fewer significant figures when viewed in hours, minutes, and seconds.

  The following is a procedure for objectively judging that a class value generated according to some generation rule conforms to requirements 1, 2, and 3.

  In step S71, the (k) class values (c [i]) are defined as adjacent class value differences (c [i] -c [i-1]) according to Equation 3 in FIG. An average value (D) of resolution (d [i]) for each class obtained by dividing the class width (w [i]) by the class value (c [i]) is calculated according to Equation 4 in FIG.

  For example, in the case of the production rule 2 shown in FIG. 19, as shown in the row D in the table T2, it is 1 / 59.7.

  In step S72, the provisional smoothing target width (f) is set to 2, and the provisional smoothing target width is increased until it meets the requirement 1 conformity determination in step S73 and the requirement 2 conformity determination in step S74. repeat.

  In step S73, as shown in FIG. 6, it is defined as the ratio of adjacent class values (c [i] -c [i-1]) as shown in Equation 1 (v [i]: i = f to k ), The standard deviation (S) of the values excluding the first (f) values is less than the average resolution (D) calculated according to Equation 5 according to Equation 6, and as shown in Equation 7 Therefore, it is determined that it meets the requirement 1.

  In step S74, it is determined that the moving average of the class widths of the smoothing target width (f) is substantially equal within the range of resolution expected after the smoothing between adjacent classes, thereby satisfying requirement 2.

  As an example of a more specific determination method, as shown in Expression 8 of FIG. 7, for each class excluding the first (f) items, 1 is calculated from the ratio between adjacent classes in the total of (f) class widths. It may be determined that the average value of the absolute value of the value obtained by subtracting is smaller than the value obtained by multiplying the average value of the resolution of the classes excluding the first (f) classes by the smoothing target width.

  Further, as shown in Equation 9 of FIG. 8, for each class excluding the first (f), the total of (f) class widths is calculated by the difference from the previous class value (f). Thus, an equivalent determination condition may be used.

  If step S75 does not meet either requirement 1 or requirement 2, the process proceeds to step S76, and when the provisional smoothing target width (f) is sufficiently close to the number of classes (k), requirement 1 and requirement 2 Therefore, the process can proceed to step S77.

  For example, in the generation rule 1 shown in FIG. 13, when the provisional smoothing target width is 70 classes, the standard deviation of the ratio of adjacent class values is 0.0021 as shown in the row D of FIG. It is larger than the pre-smoothing resolution shown in the row D (535.1 / 1) and does not meet requirement 1.

  For example, in the case of the generation rule 1 shown in FIG. 13, when the provisional smoothing target width is 70 classes, as shown in the “adjacent 70 class width ratio” column in the row of the class number 967 in FIG. A value obtained by subtracting 1 from the ratio of the total class width from 930 to 0.999 and the total class width from 0.931 to 1.000 is 1. 129, which is smaller than 0.1308 which is the resolution after smoothing shown in G of FIG.

  For example, in the case of the production rule 2 shown in FIG. 19, when eight classes are set as the smoothing target width as shown in the row F in the table T2, the requirement 1 is satisfied as shown in the row K in the table T2. And meets requirement 1 as shown in row N of table T2.

  In step S77 of FIG. 5, according to the resolution expected after smoothing with the provisional smoothing target width calculated in step S75, more class values that can be viewed in hours, minutes, and seconds are included. Therefore, it is determined that it meets the requirement 3.

  The internal processing in step S77 in FIG. 5 will be described in more detail with reference to FIG. In step S771, the sequence of class values includes {1 second, 10 seconds, 60 seconds (1 minute), 600 seconds (10 minutes), 3600 seconds (1 hour), 36000 seconds (10 hours)}, that is, Judging from the minutes and seconds, it is determined that a numerical value with one significant digit is included. If any numerical value is not included, the process proceeds to step S778, which does not conform to requirement 3. Judge.

  In step S772, the sequence of class values includes {0.5 seconds, 5 seconds, 30 seconds (0.5 minutes), 300 seconds (5 minutes), 1800 seconds (0.5 hours), 18000 seconds (5 hours). }, That is, it is determined that a half of a one-digit effective number is included in terms of hours, minutes, and seconds, and if any of the numbers is not included, the process proceeds to step S778. It is determined that it does not conform to Requirement 3.

  In step S773, a value obtained by multiplying the average resolution (D) before smoothing by the smoothing target width (f) is set as the resolution value expected after the smoothing target.

  Here, the resolution (D) expected after the smoothing target is a value smaller than 1 and the reciprocal thereof is larger than 1 due to the constraints of step S771 and step S772.

  For example, in the generation rule 2 shown in FIG. 19, the resolution before smoothing is 1 / 59.7 as shown in the row D of the table T2, and the smoothing shown in the row F of the D in the table T2. By multiplying the value 8 of the target width (f), “1 / 7.46” can be calculated as the value of the resolution (D) expected after smoothing shown in the row G of Table T2.

  In steps S774 to S777, {D, 2 × D,... {1 second, 10 seconds, 60 seconds (1 minute), 600 seconds (10 minutes), 3600 seconds (1 hour), 36000 seconds (10 hours)}. It is determined that a numerical value multiplied by (an integer not exceeding the reciprocal of D) × D} is included in the sequence of class values.

  For example, in the case of generation rule 2 shown in FIG. 19, the resolution (D) value expected after smoothing is “1 / 7.46”, so {1 in 7.46, 2 in 7.46, 7. 3/46/4, 7.46 / 5, 7.46 / 5, 7.46 / 6, 7.46 / 7} {1 second, 10 seconds, 60 seconds (1 minute), 600 seconds (10 minutes), 3600 seconds (1 hour), 36000 seconds (10 hours)}, which is a numerical value close to the value rounded to the effective precision (about 1.8 digits) expected after smoothing {{0.1, 1, 6, 60, 360, 3600}, {0.2, 2, 12, 120, 720, 7200}, {0.4, 4, 24, 240, 1440, 14400}, {0.5, 5, 30, 300, 1800, 18000}, {0.6, 6, 36, 360, 2160, 21600}, {0.8, 8, 48.25, 480, 288 , 28620}, the {0.9,9,54,540,3240,32400}}, because it contains the class values shown in 13d2 all Figure 20, it can be determined that they comply with the requirements 3.

  On the other hand, in the case of generation rule 3 shown in FIG. 26, since the resolution value “1 / 2.8” is expected after smoothing, {2.8 / 1, 2.8 / 2} is {1 second. , 10 seconds, 60 seconds (1 minute), 600 seconds (10 minutes), 3600 seconds (1 hour), 36000 seconds (10 hours)} is a class value close to {{0.309, 2.99]. , 18.1, 179, 1050, 10700}, {0.731, 7.07, 42.8, 434, 2560, 26000}} and are not suitable because they are not well-separated class values. Can be determined.

  Next, in step S78 in FIG. 5, one or more types of processing time data in which probability density data distribution is analytically known, such as Weibull distribution and uniform distribution, are generated as verification data.

  In step S79 of FIG. 5, the cumulative probability data 8a is generated by the same processing as the counting unit 12 and the statistical processing 8 in FIG.

  For example, in the case of the Weibull distribution, a smooth curve is obtained as shown in the graph of G17 in FIG.

  Further, in step S79, probability density data is generated from the slope of the tangent at the representative point for each class of the cumulative probability data, and the probability density data smoothed with the smoothing target width calculated up to step S77 is calculated.

  For example, when the probability value data of the Weibull distribution is calculated with the class value of the generation rule 1 and the smoothing target width is set to 70 class and visualized as a graph, smoothness near A1 is obtained as in the graph of G14 in FIG. The graph is damaged and distorted.

  On the other hand, if the smoothing target width is 8 in the class value of the generation rule 2, as shown in FIG. It becomes.

  The above verification result may be output as a verification result 7c including information as shown in FIG. Furthermore, the verification result may be such that the classes based on a plurality of types of generation rules can be compared, such as T1 in FIG. 13, T2, FIG. 19, T3 in FIG. 26, and T4 in FIG. .

Further, the data calculated in the verification process may be visualized as a graph.
• “Class value” columns for I1, I2, I3, and I4:
Class value “c [i]” for each class “i”
-Column of "class width" of I1, I2, I3, I4: Calculation result of "w [i]" in Formula 3-"Ratio of adjacent / class value" of I1, I2, I3, I4 Column: Calculation result of “v [i]” in Formula 1 • “Adjacent / Class width ratio” column of I1, I2, I3, I4: Calculation progress of Formula 8 and Formula 9 • I1, I2, I3, I4 Column of “adjacent / n-class ratio”:
In the calculation process of Expressions 8 and 9, the calculation process in which the width to be smoothed is n-class. “Resolution” column of I1, I2, I3, and I4: “d [i]” calculation result of Expression 4 • G11, G21, G31, G41: Graph of class value “c [i]” and class width “w [i]” G12, G22, G32, G42: Ratio of class value “c [i]” to adjacent / class value “ v [i] "graph G13, G23, G33, G43:
Graph of ratio of adjacent / class width indicating calculation process of Equation 8 and Equation 9 G14, G24, G34, G44: Graph visualizing probability density data 8b of FIG.

  Furthermore, the class value output in step S79 may be output as an enumeration of numerical values as shown in FIG.

  The class value 13 output from the class value verification unit 7 is a customizable process that configures the process of the totaling unit 12 in the form of an SQL sentence as shown in FIG. 40 for the same generation rule 4 as in FIG. A part of the code may be output.

  Returning from FIG. 5 to FIG. 4, the information processing apparatus 1 of the present application uses a class value 13 that has been verified as conforming to requirements 1 to 3 in the same manner as the class value verification unit 7. However, it is not always essential to perform the class value verification unit 7 mechanically.

  Next, the processing contents of the counting unit 12 in FIG. 4, which is the main processing of the present application, will be described in more detail with reference to FIG.

  The operation data 11d processed by the counting unit 12 in FIG. 10 is data as shown in FIG. 41, for example. As shown in FIG. 41, the records 21 and 22 in the operation data 11d have times 21a and 22a when the web system 2 processes the request data. Moreover, the records 21 and 22 in the operation data 11d have identification information 21b and 22b for distinguishing the processing contents in which the web system 2 processed the request data. Further, 21 and 22 in the operation data 11d have processing times 21c and 22c required for the data processing of the request data by the web system 2.

  For example, in the case of the record 21 in the operation data 11d, the web system 2 sets the processing time of “0.371 s” to perform the data processing of the data processing content indicated by the identification information 21b received at the time 21a. I need it.

  In FIG. 41, only 21 and 22 are shown as the records of the operation data 11d. However, when the information processing apparatus 1 receives operation data from the web system 2 every other day, for example, generally from several thousand. The operation data 11d of several million records is received.

  In step S1 of FIG. 10, a record in the operation data 11d as shown in FIG. 41 is read into the memory.

  In step S2, {URL 51, Query character string 52, time 53 when the corresponding record is processed} constituting the process identification information in FIG. 42 is extracted from the operation data 11d.

  Here, the time 53 is converted from the time portion 21a of the operation data 11d into ideographic identification information such as {busy period, normal time} according to the condition given as the process identification information extraction condition 12e in step S1. You may do it. Further, according to the process identification information extraction condition, only a part of the information part 21b by processing time in the operation data 21 may be extracted and used as the URL information 51 or the query character string 52. Thereafter, information consisting of {URL 51, Query character string 52, time 53} and the like extracted from the processing identification information included in the record in the operation data 11d can identify the data processing content in the web system 2. As information, it is called a process identifier.

  In step S3, the processing time portion 21c of one record in the operation data 11d as shown in FIG. 41 is quantified and compared with a numerical value column indicating the upper limit value for each class as shown in FIG. The class to which the processing time 21c belongs is selected.

  In step S4, in the data structure as shown in FIG. 42, the information (51, 52, 53) constituting the processing identifier matches the processing identifier extracted in step 2, and the column of the class value 54 is the belonging class. If there is no corresponding record, a row with a frequency column of 0 is added, and if there is a corresponding record, the value of frequency 55 is incremented by one.

  In step S5, in the data structure as shown in FIG. 42, the information (51, 52, 53) constituting the processing identifier matches the processing identifier extracted in step 2, and the column of the class value 54 is the belonging class. If there is no corresponding row, a row with a total processing time value column of 0 is added. If there is a corresponding record, the value of processing time 21c is added to the total processing time. Add to the value column.

  In step S6, it is determined whether or not there is subsequent data in the operation data 11d. If there is a subsequent record 22 of operation data in FIG. 41, the process proceeds to step S1 and the subsequent record 22 is processed by the same process as the record 21. The data in the compressed data storage unit 14 shown in FIG. 42 is further updated.

  In step S6, it is determined whether or not there is subsequent data in the operation data 11d. If there is no subsequent record in the operation data 11d, the aggregation process is terminated.

  Here, the operation data 11d is received from the WEB system 2 and is held only for a temporary period until the aggregation process 12 ends. Even if the operation data 11d is deleted from the information processing apparatus 1 after the aggregation process 12 ends. good.

  In addition, the compressed data storage unit 14 occupies the area of the auxiliary storage device 63 of the information processing apparatus 1 for a long period in preparation for the operation data aggregation processing for the next week and thereafter. It only occupies an area corresponding to the combination.

  As shown in FIG. 42, the compressed data storage unit 14 includes a URL (Uniform Resource Locator) 51, processing identification information including a Query 52, a time 53, etc., a class value 54, a frequency 55, and a total value 56 of processing times. And have. The compressed data storage unit 14 holds an amount of data corresponding to the combination of the type of processing identifier and class value. For example, when there are 200 types of processing identifiers and the class value is 953 class as in the example of generation rule 2, the data amount is equivalent to about 200,000 (190,600 = 200 × 953) records. Accordingly, it is possible to compress the operation data 11d, which can reach several million records even for one day, to a data amount equivalent to 200,000 records at most, and store the information in the compressed data storage unit 14. Furthermore, the compressed data storage unit 14 has a fixed capacity that does not depend on the number of records in the operation data 11d. Further, when a smaller number of class values (244 class) is used as in the generation rule 4, the data amount of the compressed data 14 can be further reduced and the memory can be saved.

  Further, as shown in FIG. 34, the class value of the generation rule 4 can be calculated by an arithmetic expression in which the class to which the processing time belongs is rounded up by the determination criterion for the rank of the processing time and the class width for each affiliation rank. Further, if the class value calculation formula of FIG. 34 is used, class value 11d is prepared in the form of a customizable executable statement like the SQL statement of 11d5 of FIG. 40, and another implementation of totaling unit 12 shown in FIG. However, processing equivalent to that in FIG. 10 can be performed.

  Another implementation of the counting unit 12 will be described in more detail with reference to FIG.

  In step S11, one record in the operation data is read, and in step S12, it is formatted as a database record and written in the temporary table. Steps S11 and S12 are applied to all records of daily operation data.

  When a matching or calculation function between a plurality of tables is used as in the SQL statement shown in FIG. 40, the determination condition of the processing identifier in step S14 is the condition of the WHERE phrase, the determination of the affiliation rank in step S15 is the CASE & When formula, the step Those skilled in the art will appreciate that rounding up to the upper limit of the class of S16 can be implemented with the CEIL function, and the sum of the frequency of step S17 and the sum of the processing time of step S18 can be implemented with one sentence including the Group BY clause and the SUM function. It is.

  For example, the processing time 21c of the record 21 in the operation data in FIG. 41 is 0.371 seconds. However, according to the calculation formula of FIG. 34 or FIG. Rounded up to a multiple of the class width (0.02 seconds), the class frequency 55 with an upper limit of 0.38 seconds and the total processing time 56 are updated.

  Here, statistical processing as internal processing of the class value verification unit 7 in order to verify that the class value verification unit 7 does not lose information necessary for statistical processing 8 as preprocessing of the report unit 9 8 and the statistical processing 8 as the preprocessing of the report unit 9 are reasonable to have equivalent processing contents. It is assumed that the statistical processing 8 used in combination with the counting unit 12 generates at least cumulative probability data 8a and probability density data 8b.

  Therefore, an example of processing that should be provided in common with statistical processing 8 as preprocessing of the report unit 9 and statistical processing 8 as internal processing of the class value verification unit 7 will be described with reference to FIG.

  In step S81, data having the content shown in FIG. 42 is read from the compressed data storage unit 14 into the main storage device 62, and data having a structure as shown in FIG. 43 is configured. 43, URL 51, Query 52, and time 53, which are examples of components of the processing identification information, are all information transferred from the compressed data storage unit 14. Further, in FIG. 43, the upper limit value 54, the frequency 55, and the total value of the processing time of the class are the information present for each combination of one or more types of processing identification information {51, 52, 53} and the class value {54}. It has been posted.

  In step S82, data structure information as shown in FIG. 43 is repeated for each processing identifier.

  In step S83, the sum of the frequencies 55 for each class is calculated for the same processing identifier in the information of the data structure as shown in FIG.

  Step S84 shows processing that is repeated in ascending order of class values for the same processing identifier in the information of the data structure as shown in FIG.

  In step S85, the value obtained by adding the frequency 55 in the class to the frequency up to the class and the total frequency calculated in step 83 is calculated as the cumulative probability 57 up to the class.

  In step S86, again, the process of repeating the ascending order of the class value for the same process identifier in the data structure information as shown in FIG.

  In step S87 and step S88, the pair 58 of the representative value for each class before smoothing and the probability density in FIG. 43, or the number of classes not less than the smoothing target width determined in the process of class value verification. A representative value for each class and a probability density pair 59 for each smoothing target width are calculated for each class. Here, the probability density data for each smoothing target width is defined as the slope of the tangent line in the representative value of the class of the cumulative probability data graph as shown in FIG. The probability density data is smoothed by calculating the slope obtained by dividing the difference in cumulative probability separated by (f) smoothing target widths by the total value of (f) class widths in ascending order for each class value. The moving average value can be calculated.

  When steps S87 and S88 have been repeated for all classes of the same process identification information, step S86 is processed for the next process identification information, and when processing is completed for all process identification information in step S82, Finish the process.

  The processing that constitutes the information processing apparatus 1, the class value verification procedure to be performed as a premise thereof, and the statistical processing that is assumed to be used in combination as a premise of verification have been described above.

  Below, the effect of each processing content is shown with the example of application data.

  First, the class value verification unit 7 or the class value verification unit can determine that only one of the class values of similar generation rules conforms to requirements 1 to 3, and the established density data is obtained from the compressed data storage unit 14. When calculated, it indicates that accuracy information necessary for statistical processing is not lost.

  As an example, the generation rule 1 shown in T1 of FIG. 13 and the generation rule 2 shown in T2 of FIG. This indicates that it can be determined that the probability density data is distorted because it is incompatible.

  First, the generation rule 1 outlined in the row B of T1 is a class value that covers the numerical values of three significant digits for the processing time from 0 seconds to 48 hours, and the number of classes (k) Are 5519 classes as shown in row (C) of T.

  In the process of verifying the class value shown in FIG. 5, data for each class value as shown by I1 in FIG. 14, for example, is used.

  The class resolution calculated in step S71 is defined by Equation 4, and the average resolution can be calculated according to Equation 5. As another means, as shown in the graph of G11 in FIG. 15, from the slope of the graph in which the processing time is plotted on the horizontal axis and the class width is plotted on the vertical axis, “1/100” and “1/1000” If the value is between, it may be determined by visual procedures that “typical resolution is a value around 1/535”.

  The ratio (v [i]) of adjacent class values used in the “Requirement 1 conformity determination” in step S73 is in accordance with Equation 2 for each class as shown in the column “Ratio of adjacent class values” in FIG. It can be calculated.

  In the case of generation rule 1, as shown in the row K of T1, the standard deviation of the ratio of adjacent class values shown in the row J of T1 in FIG. 13 is about 0.0021, and in the row D of T1. Since the average resolution before the smoothing target shown is “about 1/535”, that is, greater than 0.0018, it can be determined that the requirement 1 is not met.

  As shown in the graph of G12 in FIG. 16, the horizontal axis represents the class value, the vertical axis represents the ratio of the adjacent class value, and excluding the first few class values, the fluctuation range is less than the average resolution. Thus, it can be determined by visual means that the requirement 1 is met.

  The ratio of the adjacent class width in the smoothing target width (f) used in the “Requirement 2 conformity determination” in step S74 is the class as shown in the column “adjacent / 70 class width ratio” in FIG. With respect to the moving average value of the width, it can be determined according to Equation 8 or Equation 9. For example, in FIG. 14, the width of the 70th class where the representative value of the class is 0.967 is a ratio of 1.13 times and is larger than the average resolution (1/535 = 0.0018) calculated in step S71. , It can be determined that it is incompatible with requirement 2. In this way, representative values of classes that are likely to be incompatible with requirement 2 are obtained from a graph in which the horizontal axis represents the class value and the vertical axis represents the ratio of the adjacent class values as shown in G13 of FIG. The ratio of the adjacent class value is about 1 ", whereas it is visually unfit from the situation that" the class value is 10 times before and after 1, 10, 100, 1000, and 10,000 ". It can also be estimated.

  In step S79, if the Weibull distribution, which is known to be an analytically smooth probability density graph, is used as the verification distribution, the cumulative probability data is smooth data like the graph of G17 in FIG. Nevertheless, when the probability density data is generated using the class value of generation rule 1, smoothness is lost before and after the point of A1 where the class value is 100 seconds as shown in the graph of G14 in FIG. You can also visually check that the graph is distorted.

  In step S77, it is clear that the generation rule 1 satisfies the requirement 3 by including all the numerical values having three significant digits.

  As shown in each row of T1 in FIG. 13, the “classification value verification result” by a series of procedures according to FIG. 5 can determine that generation rule 1 does not conform to requirement 1 or requirement 2. .

  However, in the case of FIG. 13, since the smoothing target width was stopped at 70, the requirements 1 and 2 were not satisfied. However, the resolution that can be expected after the smoothing target is allowed to be rough, and the smoothing target width is set to the class. If it approaches the number 5519, the requirements 1 and 2 are met.

  As another example, in the generation rule 2 shown in T2 of FIG. 19, the average common ratio of the upper limit value of the class is set to (60 (1/240)), and the number of significant digits is viewed in hours, minutes, and seconds. Consider a class with an upper limit value for each class of 2 digits, or "the nearest value to (1/8) multiple of {1 second, 10 seconds, 60 seconds, 600 seconds, 3600 seconds, 36000 seconds}" . As shown by the class value 13d2 in FIG. 20, the generation rule 2 includes an arbitrary numerical value selection instead of regular synthesis.

  For the generation rule 2 as well, the information shown in I2 of FIG. 21 is calculated as class value verification progress data similar to that of FIG.

  When step S71 is applied to generation rule 2, the resolution is (1/90) to (1/50) as shown in graph G21 in FIG. 22, and as shown in the row D of T2 in FIG. The average resolution before smoothing is (about 59.7 times) or (about 0.0167).

  When the “Requirement 1 conformity determination” in step S73 is applied to generation rule 2, as shown in the row (J) of T2 in FIG. 19, the standard deviation of the ratios of adjacent class values is about 0.00920. Since it is smaller than the (D line) average resolution (0.0167), it can be determined that the requirement 1 is satisfied.

  Further, the conformity to the requirement 1 is shown in FIG. 23 by drawing a graph in which the horizontal axis is the class value and the vertical axis is the adjacent class value, and “0. It can also be judged visually from the flatness of G22 of “1399 seconds or more”.

  Applying the “Requirement 2 conformity determination” in step S74 to the generation rule 2 with the smoothing target width set to 8 classes, from the value of the column “Ratio of adjacent 8 class width” in the table I2 in FIG. 19, the maximum absolute value obtained by subtracting 1 from the ratio of the adjacent class width in the smoothing object width “8 class” shown in the row (f) of T2 in FIG. As shown in line M), about 0.125 can be calculated and is smaller than G line “0.1339”, which is the expected resolution after smoothing. The conformity determination to requirement 2 can also be visually confirmed by the flatness of the graph G23 of “0.1399 seconds or more” corresponding to the resolution expected after smoothing in FIG.

  When the “requirement 3 conformity determination” procedure of FIG. 9 is applied to the class value of generation rule 2, {1, 10, 60, 600, 3600, 36000} is included in the class value 13d2 of FIG. 20 in step S771. Therefore, the process proceeds to step S772. In step S772, since {0.5, 5, 30, 300, 1800, 18000} is included in the class value 13d2 in FIG. 20, the process proceeds to step S773.

  In step S773, as the expected resolution after smoothing, the resolution before smoothing (about 59.7 / 1) is multiplied by the smoothing target width (8) to obtain the expected resolution after smoothing ( A value of about 1 / 7.46≈0.1339) can be calculated.

  In step S774 to step S777 of FIG. 9, a sequence obtained by multiplying {1, 10, 60, 600, 3600, 36000} by 7 or less that is close to the denominator of the resolution (about 7.46 / 1) expected after the smoothing. {{0.1, 1, 6, 60, 360, 3600}, {0.2, 2, 12, 120, 720, 7200}, {0.4, 4, 24, 240, 1440 , 14400}, {0.5, 5, 30, 300, 1800, 18000}, {0.6, 6, 36, 360, 2160, 21600}, {0.8, 8, 48.25, 480, 2880 , 28620}, {0.9, 9, 54, 540, 3240, 32400}} are included in the class value 13d2 in FIG. 20, the process proceeds to step S779 to determine that the requirement 3 is met. it can.

  Furthermore, when step S79 in FIG. 5 is applied to the Weibull distribution such that the generation value 2 of the generation rule 2, the smoothing target width 8, and the cumulative probability data of G78 in FIG. 12 as verification data are applied to G24 in FIG. Probability density data can be calculated as shown, and the compressed data has a probability that the portion of the graph exceeding the resolution (about 0.13 seconds) expected after the smoothing is smoothed. It can also be confirmed that the data with accuracy that does not cause distortion in the density data could be stored.

  The verification result for the class value of generation rule 2 can also be output as information such as T2 in FIG.

  Here, when the result of compression of data according to the Weibull distribution and restoration as probability density data is compared between the generation rule 1 and the generation rule 2, the class value A1 in which the probability density data is distorted is the requirement 1 It can also be seen that it is a place of a class value that does not satisfy requirement 2.

  Therefore, Requirement 1 and Requirement 2 are necessary conditions for preventing the restored probability density data from being distorted.

  Here, the resolution expected after the smoothing by the 953 classes and the smoothing target width (8 classes) of the production rule 2 is as shown in T2 (about 7.46), When a logarithmic value with a resolution of 10 is calculated in order to convert it to the effective number of digits, it is equivalent to (approximately 0.87 digits). Similarly, with 5519 classes and the smoothing target width (70 classes) of generation rule 1, the expected resolution after smoothing is about 1 / 7.6 (as shown in T1). When converted to the number of digits, it is equivalent to (approximately 0.9). Comparing these production rule 1 and production rule 2, the number of significant digits expected after smoothing is about the same, but the number of classes before smoothing is 5519. Since the generation rule 2 in which the number of classes before conversion is 953 is less, the number of classes is smaller, so that statistical results with equivalent effective accuracy can be obtained with less memory.

  As described above, prior to the class value verification unit 7 or the totaling unit 12, the class value verification unit performs the verification process as shown in T1, FIG. 19, T2, FIG. 26, T3, and T4 in FIG. The number of classes calculated in step 1, the average resolution before smoothing, the smoothing target width, the average resolution expected after smoothing, the average value and standard deviation of the adjacent class ratio, and the value obtained by subtracting 1 from the ratio of the adjacent class width of the smoothing target width When outputting the maximum value or the like, it is also possible to select a class value with a high compression effect or a class value that provides a finer resolution after smoothing.

  As another example, among the production rule 3 shown in T3 of FIG. 26 and the production rule 4 shown in T4 of FIG. 33, the production rule 4 satisfies the requirement 3 because the production rule 3 is nonconforming. Indicates that an accurate compliance rate cannot be calculated.

  The production rule 3 is such that the class value from 0.1 second to 50 seconds is 1.08 times (= 50 to the (50th power)) so that 1 second, 50 seconds, 60 seconds, and 3600 seconds pass smoothly. For 60 seconds or more, a numerical value obtained by rounding a geometric series of 1.06 times (= 60 (1/60) power) to the third significant digit is used as a class value.

  For generation rule 3, as shown in I3 of FIG. 28, the average of the class width for each class and the resolution for each class obtained by dividing the class width by the class value is about 14.0 (about 0.0714).

  The ratio of the class values of the adjacent classes in the generation rule 3 is different with respect to the class of the processing time exceeding 60 seconds and exceeding 60 seconds, as indicated by G32 in FIG. The standard deviation (about 0.014) is below the average resolution (about 0.0714), except for the class for processing time below the resolution that can be expected after smoothing (less than about 0.07 seconds), as shown in the row of Therefore, it meets requirement 1.

  When the smoothing target width is set to the fifth class with respect to the class value of the generation rule 3, when the value of Expression 8 is calculated as shown in “Ratio of adjacent five class widths” of I3 in FIG. Since it is “about 0.200” as shown in (M) of FIG. 1 and is lower than the resolution after smoothing (about 2.8 / ≈0.357), it meets the requirement 2.

  The class value generated by the generation rule 3 is a numerical sequence indicated by 13d3 in FIG. However, the class value closest to {1,10,60,600,3600,3600} with one significant digit when viewed in hours, minutes, and seconds is {1,9.67,10.5,60,570. , 611, 3600, 36600}, it cannot be said that it includes numerical values that are well-separated in terms of hours, minutes, and seconds.

  Furthermore, in the verification procedure of Requirement 3 shown in FIG. 9, it is a numerical value that can be viewed with more hours / minutes / seconds according to the resolution (about 2.8 times) expected after smoothing. {{1, 10, 60, 600, 3600, 3600}, {0.5, 5, 30, 300, 1800, 18000}, {0.3, 3, 18, 180, 1080, 10800}, {0. 7, 42, 420, 2520, 25200}} is used as the determination condition, but the nearest class value is {{1, 9.67, 10.5, 60 , 570, 611, 3420, 3600, 34200, 36600}, {0.495, 0.535, 4.78, 5.17, 28.9, 31.3, 288, 309, 1700, 1820, 10000, 10700. }, {0.309, 2.99, 18.1 , 179, 1050, 10700}, {0.731, 7.07, 42.8, 434, 2560, 26000}}, it can be determined that the requirement 3 is not met.

  As shown in G41 of FIG. 36, the class value generated by the generation rule 4 has a resolution for each class less than 60 seconds from (1/10) to (1/20), and a resolution of a class exceeding 60 seconds. Is from (1/10) to (1/30), and as shown in the graph of G13 in FIG. 38, the ratio of adjacent class widths is less than 1.5 times in most classes, and 2.5 times at most. As shown in the enlarged view of B2 in FIG. 36 at a ratio that does not exceed the predetermined resolution, when it comes close to the graph showing the upper limit of the predetermined resolution, it is close to the graph showing the lower limit of the predetermined resolution and is almost doubled. The class width is increased in the vertical direction to the following class width, and even if the class value increases again, the class width is repeated in such a way that the class width remains unchanged until it reaches the lower limit of the predetermined resolution. Is a class value that increases the number, and is a good value that can be seen in hours, minutes, and seconds It was chosen arbitrarily to include more, a class value 13d4 shown in FIG. 35.

  As shown in FIG. 34, the class value of the generation rule 4 is the class width for each rank, as shown in FIG. 34, if the upper limit value of the class which is the stepped step in the graph B1 in FIG. It can also be expressed as an upper limit value for each class rounded up to a multiple.

  As shown in T4 of FIG. 33, the generation rule 4 is that the resolution before smoothing is “about 1/17”, and the average resolution expected after smoothing when the smoothing target width before smoothing is 6 is “ It can be calculated as “about 1.82 / 2≈0.3543”.

  The ratio of adjacent class values in production rule 4 is slightly different at 60 seconds as shown in the graph of G42 in FIG. 37, but the average ratio is 1.08 times as in the graph of G32 for production rule 3. The graph G42 of the ratio of the adjacent class value to the processing time exceeding the resolution (about 0.35 seconds) expected after smoothing in G42 in FIG. 37 is flat, or (J) in FIG. The standard deviation (about 0.0219) of the ratio of the adjacent class values shown in the row of <5> is lower than the resolution before smoothing (about 1/17 = about 0.058), so that the requirement 1 is satisfied.

  The ratio of the class width between adjacent classes in the generation rule 4 is 1.0 times except for many classes that become rank delimiters, as indicated by G43 in FIG. Then, from 1.5 times to 2.5 times, it is almost twice. This is also a difference from the generation rule 3 in which there is a class in which the ratio of the widths of adjacent classes is less than 1.

  When the smoothing target width is set to 6 in the generation rule 4, the maximum absolute value of the value obtained by subtracting 1 from the ratio between adjacent classes of the moving average value of the smoothing target width is shown in the row (M) of FIG. As shown, it is “about 0.250”, which is lower than the expected resolution “about 0.058” after smoothing, so that requirement 2 is satisfied.

  The production rule 4 is a value that is close to {1, 10, 60, 600, 3600, 3600} {1 time, 0.5 time, 1/2 times 2.82, 2 times 2.82}. 1, 10, 60, 600, 3600, 36000}, {0.5, 5, 30, 300, 1800, 18000}, {0.3, 3, 18, 180, 1080, 10800}, {0.7, 7, 42, 420, 2520, 25200}}, it can be determined that the requirement 3 is met under the determination conditions shown in FIG. 9.

  Here, the generation rule 3 has a class number of 238 and the resolution expected after smoothing is about 2.8. On the other hand, the generation rule 4 has a class number of 244 and a smoothness. The resolution that can be expected after conversion is about 2.82, and the graph G34 in FIG. 32 in which the probability density data of the Weibull distribution is restored in the process of verifying the class value of the generation rule 3 is also the class value of the generation rule 4. The graph G44 in FIG. 39 in which the probability density data is restored using the same verification data with respect to is also a smooth graph in the portion where the processing time (about 0.3 seconds) longer than the resolution expected after smoothing is exceeded. Therefore, it can be seen as if memory can be saved to the same extent while retaining information of the same degree of accuracy.

  However, when the report unit 9 in FIG. 4 assumes that the compliance rate is included in the report, the class of the generation rule 3 is inferior to the class of the generation rule 4. For example, if the target value of the compliance rate is 600 seconds and the statistical value of “the compliance rate when the processing time is 600 seconds or less” is to be reported, the class value of “600 seconds” is included in the class of the generation rule 4 Therefore, an accurate compliance rate can be calculated. On the other hand, according to generation rule 3, the nearest class value at 600 seconds is {570 seconds and 611 seconds}, so even if processing for 599 seconds occurs frequently, processing time of 600 seconds or less is reduced and processing for 610 seconds is performed. Even in the case of frequent occurrences, the frequency and total processing time for each class stored in the compressed data storage unit 14 are the same, and therefore cannot be distinguished. Therefore, in the cumulative probability data as shown in FIG. 44, the occurrence probability of the processing time of 600 seconds or less is interpolated from the cumulative occurrence probability of the processing time of 570 seconds or less and the cumulative occurrence probability of the processing time of 611 seconds or less. However, the accuracy with which the deviation before and after the target value of the compliance rate can be found is lost before recording in the compressed data storage unit 14.

  Here, whether it is FIG. 10, FIG. 48, or another equivalent implementation, the information in the compressed data storage unit 14 generated by the totaling unit 12 is the statistical processing 8 assumed in FIG. The information extracted from the operation data 11d using the class value that has been verified that the information required for the report unit 9 can be retained without loss in the accuracy range expected for the smoothing target width (f) It shows that the data is reversibly compressed and recorded in the compressed data storage unit 14.

  The process of irreversibly compressing will be described by focusing on the correspondence between the records 21 and 22 in the operation data of FIG. 41 and the compressed data 14 of FIG. 42 after execution of FIG. 10 or FIG.

  As described above, in the compressed data storage unit 14, the generation order of the records 21 and 22 which are examples of the records of the operation data shown in FIG. 41, and one of the query character strings not extracted as the process identification information are stored. Memory saving is achieved by not retaining all or part of the data.

  For example, in the example 21 of the record in the operation data in FIG. 41, the URL part includes information that can identify the processing content “/ context01 / svcName1”, but is recorded in the compressed data storage part 14 in FIG. The data to be processed may be information corresponding to many-to-one according to the conversion rule specified as the extraction condition of the process identification information, such as “/ URL1”. In addition, in the example 21 of the record in the operation data in FIG. 41, the query character string “p = 53 & aps = 1743606” following the URL is used, but the data constituting the processing identifier in FIG. 42 is “SomeQuery”. The information may correspond to many-to-one in accordance with the conversion rule specified as the process identification information extraction condition.

  Furthermore, the time 21a in the operation data is transferred to the process identifier when the hourly diurnal change is scheduled to be reported, but {busy period, normal}, etc. When only the time is scheduled to be reported, the time 53 in the information constituting the process identifier may be an ideographic symbol such as {busy period, normal time}.

  Alternatively, if there is no plan to include a process identifier in the contents of the report schedule, neither extraction nor recording of the process identifier may be necessary.

  These indicate that the data compression effect is enhanced by not performing lossless compression that can completely restore the original operation data 11d from the compressed data recording unit.

  The tabulation unit 12 using the class value that has been determined to meet the requirements 1, 2, and 3 at the class value verification stage has the following effects.

  That is, according to the requirement 1, for example, the number of classes can be reduced while maintaining effective accuracy (resolution) for each class with respect to a wide processing time distribution from a few hundredths of seconds to several hours.

  By using a series of logarithmically spaced sequences as class values, the number of classes is reduced compared to the case where all classes are equally spaced, and the auxiliary storage device 63 used for recording in the compressed data storage unit 14. In addition, the main storage device 62 for performing the aggregation process can also reduce the required amount.

  Consistent with requirement 2, the ratio of the adjacent class widths of the widths to be smoothed is approximately equal, so that distortion of probability density data that can be restored from the compressed data storage unit 14 in the statistical processing 8 is expected as the smoothing target widths Resolution can be suppressed.

  In the generation of probability density data, as shown in the graph before smoothing in FIG. 45, since noise of collected data is included, generally smoothing processing is required. An attempt was made to calculate an appropriate smoothing width before data collection. On the other hand, since the lower limit of the smoothing target width and the expected resolution after smoothing are calculated in the process of verifying conformity to requirement 2, an analysis of uneven multimodality as shown by 8b1 in FIG. On the other hand, it can also contribute to the adoption of discrimination conditions that should be regarded as a difference in superiority.

  According to Requirement 3, when the report unit 9 determines the compliance rate of the web system 2 with respect to the processing time, processing that can be designated as a compliance rate target so that the original data (data of the processing time before compression) is not used Time (for example, a time (interval) that can be recognized by human beings and a clear number) can be covered more.

  In other words, conforming to Requirement 3, depending on the resolution expected for the smoothing target width, more class values can be seen in hours, minutes, and seconds, and can be separated into more types, so By using time as the target value of the compliance rate, the compliance rate can be reported without losing accuracy compared to the operation data before compression.

  For example, in FIG. 47, the complement of the compliance rate is displayed for three types of {0.4 seconds, 4 seconds, 40 seconds} as the target values of the compliance rate. Even if the statistical process 8 is performed again as a value, it is possible to report without reading the operation data 11d again. Not having to re-read the operation data 11d makes it unnecessary to store the operation data 11d until after the report is output, thereby reducing the required amount of the auxiliary storage device 63 for storing the operation data 11d. Also contribute to doing.

  Furthermore, the class value of the generation rule 4 shown in FIG. 35 is necessary for the requirement 3 determination condition in the generation rule 2 shown in FIG. 20 {{0.1, 1, 6, 60, 360, 3600}. , {0.2, 2, 12, 120, 720, 7200}, {0.4, 4, 24, 240, 1440, 14400}, {0.5, 5, 30, 300, 1800, 18000}, { 0.6, 6, 36, 360, 2160, 21600}, {0.8, 8, 48.25, 480, 2880, 28620}, {0.9, 9, 54, 540, 3240, 32400} And includes {45 seconds, 50 seconds, 27000 seconds (7.5 hours), 28800 seconds (8 hours)} as a number close to {48.25, 28620} that is partly not included, The same amount of processing time that can be specified as a compliance rate target Therefore, if the compliance rate report is more important than the accuracy of probability density data, it is possible to save more by selecting 244 class values according to generation rule 4 than 953 class values according to generation rule 2. It can also be a memory.

  Further, from the information for each processing identifier recorded in the compressed data storage unit 14, the “average value 9f2”, “maximum value to cover 68%, maximum value to cover 95%, and 99.7% to cover” shown in FIG. Even when various pieces of statistical information such as “a plurality of percentile values 9f3 such as the maximum value” and “total processing time and total frequency 9f4 of the statistical target period” are extracted from one compressed data storage unit 14, the operation data is restored. It can be said that there is no need to re-read 11d, and the information of the operation data 11d can be held in the compressed data storage unit 14 without impairing information necessary for the report unit 9.

In the following, the compression effect is shown by the following example.
As the operation data 11, it is assumed that one record as shown in 21 of FIG. 41 has an average of 120 bytes, and an average of 100,000 records per day and about 36.5 million records per year can be generated.
The information included in the operation data that can identify the data processing content includes an average of 64 bytes, and 32-byte information is extracted as a processing identifier. That is, it is assumed that the process identifier (51, 52, 53) in FIG. 42 is 32 bytes.
・ Processing time can occur from 0.001 seconds to 48 hours, and is a numerical value with 3 decimal places.

In the case of the conventional technology, even if a report is generated once, it is guaranteed that information that can make it unnecessary to re-read raw operation data again when the target value of the compliance rate is changed is guaranteed. Since there was no information, the following information must be retained.
・ “(120 bytes) × (100,000 / day) × (365 days)” = “4,380,000,000 bytes” in order to hold data that goes back a year ago as operational data
42, as data having the structure shown in FIG. 42, each of “172,800,000” classes obtained by dividing 0 to 48 hours at equal intervals of 0.001 seconds, 32 bytes as a processing identifier, and as a class value A recording area of 4 bytes, 8 bytes as frequency, 8 bytes as the total processing time, “52 bytes” per class, and “8,985,600,000 bytes” per type of processing identifier is required.
When the operation data and the data used for intermediate processing are combined, a storage area of “13,365,600,000 bytes” is required.

On the other hand, in the present invention, for example, when a class value according to the generation rule 4 is used, sufficient information is stored so that it is not necessary to read the raw operation data again even if the target value of the compliance rate is changed. As a result, the amount of information to be retained can be reduced as shown below.
・ "(120 bytes) x (100,000 / day) x (1 day)" = "12,000,000 bytes" to hold data for one day as operating data
42. As data of the structure shown in FIG. 42, assuming that “52 bytes” per class is occupied for 244 classes of generation rule 4 as described above, “12.688 bytes” is stored per type of processing identifier. Requires space.
When the operation data and the data held in the compressed data portion are combined, “12,012,688 bytes” is enough storage area.
The compression effect in this example is “approximately 1/11212 = ((12,012,688) ÷ (13,365,600,000))”.

  The compression effect shown here becomes higher as it is assumed that a report for a longer operation period is created.

  Further, as in step S81 in the statistical process 8 of FIG. 11, the report unit 9 also includes a process for loading information recorded in the compressed data storage unit 14 from the auxiliary storage device 63 to the main storage device 62. In this case, the memory saving of the main storage device 62 makes it possible to hold the probability density data of various processing identifiers on the memory at the same time.

  For example, in the prior art, when data having a structure as shown in FIG. 43 is used in the course of statistical processing, “9 GB≈8,985,600 calculated as an example of the“ amount of storage area per one type of processing identifier ”is used. , 000 bytes ”, which exceeds the capacity (4 Gbytes) of a main storage device that can be easily secured by a 32-bit OS, it is difficult to perform statistical processing of a plurality of processing identifiers. On the other hand, in the present invention, when data having a structure as shown in FIG. 43 is used in the course of statistical processing, “13 Mbytes≈12.688 bytes” calculated as an example of the “amount of storage area per type of processing identifier”. If you use 4GB memory, there is room to perform statistical processing of up to (338,506 ≒ (4 * 1024 * 1024 * 1024 ÷ 12,688)) processing identifiers at the same time. As a result, for example, as shown in FIG. 46, the probability density graphs of various processing identifiers can be displayed in an overlapping manner, and the processing time of the probability density prominent compared to other processing identifiers can be found. The effect that secondary analysis can be further performed from the accumulated probability data and the probability density data generated in FIG.

  By satisfying all the requirements 1, 2 and 3, the compressed data storage unit without losing information necessary for the report unit 9 assumed in the class value verification unit 7 as shown in FIG. 14, since necessary information can be held, even when statistical processing is performed for a long period of time, it is not necessary to hold the operating data 11d, compared to the total amount of operating data that can be increased according to the statistical target period. The compressed data storage unit 14 having a small amount and a fixed capacity that does not correspond to the statistics target period also has an effect of facilitating longer-term statistics.

  Hereinafter, an example of generating information (for example, information shown in FIG. 35) for obtaining a class to which the processing time of the web system belongs based on four types of generation rules will be described. In the following, the information for rounding up the processing time of the web system may be referred to as a class value.

[Generation rule 1]
FIG. 13 is a diagram showing an outline of the generation rule 1 and the requirement conformity determination result. In the (B) column of the table T1 shown in FIG. 13, a generation rule 1 for generating a class value for obtaining a class to which the processing time of the web system belongs is shown.

  FIG. 14 shows the class value generated based on generation rule 1 and information indicating the class verification process.

  FIG. 15 is a diagram for explaining the resolution of the class in the verification process of the class value according to the generation rule 1. A graph G11 illustrated in FIG. 15 is obtained by plotting the class value and the class width of the information I1 illustrated in FIG. 14 in a logarithmic graph. The horizontal axis of the graph G11 indicates the class value, and the vertical axis indicates the class width.

  As shown in the graph G11, the class width of the class value generated based on the generation rule 1 changes in the horizontal direction in each rank. Therefore, the class values are arranged in an arithmetic progression for each rank. For example, in the rank of “1 second or less”, the tolerance of the class value, that is, the class width is “0.001”. In the rank of “10 seconds or less”, the tolerance of the class value is “0.01”.

  The class widths are arranged in a geometric sequence between ranks. For example, as shown in the graph G11, the class width changes in a staircase shape where the rank changes, and each step up of the staircase is approximately equally spaced in a logarithmic manner. Thus, in the case of generation rule 1, the resolution obtained by dividing the class width by the class value as shown in FIG. 15 can have a resolution in the range of about 1/100 to 1/1000.

  FIG. 16 is a diagram showing a ratio of adjacent class values according to the generation rule 1. A graph G12 illustrated in FIG. 16 is a logarithmic graph in which “the ratio of adjacent / class values” (the ratio of adjacent class values) illustrated in the information I1 of FIG. 14 is plotted. The horizontal axis of the graph G12 indicates the class value, and the vertical axis indicates the ratio. As shown in the graph G12, the ratio of adjacent class values is generally “1.010”.

  FIG. 17 is a diagram showing the ratio of adjacent moving averages of class widths according to generation rule 1. The white square in the graph G13 shown in FIG. 17 is a logarithmic graph plotting the “ratio of adjacent / 70 class width” (adjacent ratio of moving average by 70 sections) shown in the information I1 of FIG. The black rhombus in the graph G13 is obtained by plotting the “adjacent / class width ratio” (adjacent ratio of class width not moving average) shown in the information I1 of FIG. 14 in a logarithmic graph. The horizontal axis of the graph G13 indicates the class value, and the vertical axis indicates the ratio.

  As shown in the graph G13, the adjacent ratio when the class width is moving averaged (smoothed) is approximately “1” as shown by the white square. On the other hand, the adjacent ratio when the class width is not moving averaged may differ by 10 times at the transition of the rank of the staircase as shown by the black diamond.

[Generation rule 2]
FIG. 19 is a diagram showing an outline of the generation rule 2 and the requirement conformity determination result. A table T2 shown in FIG. 19 shows a generation rule 2 for generating a class value for obtaining a class to which the processing time of the web system belongs. As described in the column (B) of Table T2 in FIG. 19, the class value of the production rule 2 is based on the sequence of the average common ratio (60 to the power of (1/240)). 20 is a numerical sequence enumerated in FIG. 20 in which the number of significant digits viewed in seconds is arbitrarily selected from two digits or a value close to the hour / minute (1/8).

  Information I2 in FIG. 21 is data indicating a class verification process for the class value in FIG.

FIG. 22 is a diagram for explaining the resolution in the class verification process based on the generation rule 2. A graph G21 illustrated in FIG. 22 is obtained by plotting the class value and the class width of the information I2 illustrated in FIG. 21 in a logarithmic graph. The horizontal axis of the graph G21 indicates the class value, and the vertical axis indicates the class width.
Class values are generally arranged in an arithmetic progression for each rank.

As shown in the graph G21, the class width of the class value generated based on the generation rule 2 has a stair-like increasing tendency, but the horizontal change is interrupted, and there is a portion that is not a monotonous increase.
For example, in the rank of “0.092 seconds or less”, the tolerance of the class value, that is, the class width is “0.001”. In the rank of “0.160 seconds or less”, the tolerance of the class value is “0.003”.

  The class width from the class value “0.800 seconds” to “1.00 second” is mainly “0.015”, but the class values for the class values “0.810 seconds” and “0.910 seconds” in the middle By setting the width to “0.010”, the class values “0.900 seconds” and “1.00 seconds” and the like are included in more significant class values that can be viewed in units of seconds with one significant digit. .

  In addition, the class widths are generally arranged in a geometric sequence between ranks. For example, as shown in the graph G21, the class width changes in a staircase shape where the rank changes, and each rise of the staircase is approximately equally spaced from the logarithm. Thereby, in the case of production rule 2, the class value can have a resolution in the range of about 1/50 to 1/90.

  FIG. 23 is a diagram showing a ratio of adjacent class values according to the generation rule 2. A graph G22 shown in FIG. 23 is a logarithmic graph in which the “ratio of adjacent values / class values” (the ratio of adjacent class values) shown in the information I2 of FIG. 21 is plotted. The horizontal axis of the graph G22 indicates the class value, and the vertical axis indicates the ratio. As shown in the graph G22, the ratio of adjacent class values is approximately “1.0172”.

  FIG. 24 is a diagram showing the ratio of adjacent moving averages of class widths according to generation rule 2. A white square in the graph G23 illustrated in FIG. 24 is a logarithmic graph in which the “ratio of adjacent 8 class widths” (the adjacent ratio of the moving average by 8 sections) illustrated in the information I2 of FIG. 21 is plotted. The black rhombus in the graph G23 is obtained by plotting the “adjacent / class width ratio” (the adjacent ratio of the class width not moving average) shown in the information I2 of FIG. 21 on a logarithmic graph. The horizontal axis of the graph G23 indicates the class value, and the vertical axis indicates the ratio.

As shown in the graph G23, the adjacent ratio when the class width is moving averaged (smoothed) is approximately “1” as shown by the white square. On the other hand, the adjacent ratio when the class width is not moving average is the minimum value shown in the ratio of the adjacent class width to the black diamond of G23 or the class values “0.810 seconds” and “0.910 seconds” in FIG. Some classes have values from “0.67” to the maximum value “2.00” shown in the ratio of the adjacent / class width to the class value “0.094 seconds”.
As is apparent from the graph G23, the class width in the generation rule 2 is not a number sequence that can be generated as a simple geometric sequence, and does not increase monotonously according to the magnitude of the class value.
FIG. 25 is an example of a graph for visually confirming the “smoothness verification (Weibull distribution restoration capability verification)” column in the requirement conformity determination result of FIG.

[Generation rule 3]
FIG. 26 is a diagram showing an outline of the generation rule 3 and the requirement conformity determination result. Table T3 shown in FIG. 26 shows generation rule 3 for generating a class value for rounding the processing time of the web system.
The class value of generation rule 3 passes through {1 second, 50 seconds, 60 seconds, 3600 seconds} so that the class value of 50 seconds or less is 1.08 times (50 to the power of 1/50) ), A class value of 60 seconds or more is a value of a sequence that can be synthesized regularly, such that a geometric sequence with a common ratio of 1.06 times (60 to the 1 / 60th power) is rounded to 3 significant digits. is there.

  FIG. 27 is a diagram showing class values generated based on the generation rule 3. When class value verification data is generated for the class value in FIG. 27, information I3 in FIG. 28 is obtained.

  FIG. 29 is a diagram for explaining the resolution of the class value of the class value according to the generation rule 3. A graph G31 illustrated in FIG. 29 is obtained by plotting the class value and the class width of the information I3 illustrated in FIG. 28 in a logarithmic graph. The horizontal axis of the graph G31 indicates the class value, and the vertical axis indicates the class width.

  As shown in the graph G31, in the case of the production rule 3, the class value has a resolution in the range of about 1/10 to 1/20, and the average resolution is (about approximately as shown in the table of T3 in FIG. It can be seen that 1 / 14≈0.07144).

  FIG. 30 is a diagram showing the ratio of adjacent class values according to the generation rule 3. A graph G32 illustrated in FIG. 30 is a logarithmic graph in which “the ratio of adjacent / class values” (the ratio of adjacent class values) illustrated in the information I3 of FIG. 28 is plotted. The horizontal axis of the graph G32 indicates the class value, and the vertical axis indicates the ratio. As shown in the graph G32, the ratio of adjacent class values is “about 1.08 times” for a class of 60 seconds or less, and “about 1.07” for a class exceeding 60 seconds.

  FIG. 31 is a diagram showing a ratio of moving averages of adjacent class widths according to generation rule 3. The black circles in the graph G33 shown in FIG. 31 are obtained by plotting the “adjacent / fifth class width ratio” (the ratio of adjacent moving averages in five sections) shown in the information I3 of FIG. 28 in a logarithmic graph. The white rhombus in the graph G33 is obtained by plotting the “adjacent / class width ratio” (the adjacent ratio of the class width not moving average) shown in the information I3 of FIG. 28 in a logarithmic graph. The horizontal axis of the graph G33 indicates the class value, and the vertical axis indicates the ratio.

[Generation Rule 4]
FIG. 33 is a diagram illustrating an outline of the generation rule 4 and the requirement conformity determination result.
Table T4 shown in FIG. 33 shows generation rule 4 for generating a class value for rounding the processing time of the web system. The details of the generation rule 4 are determined as the class width for each rank as shown in FIG. 34, and the class values shown in FIG. 35 are obtained.

  FIG. 36 is a graph showing the resolution indicating the relationship between the class value and the class width of FIG. Further, the ratio of adjacent class values according to the generation rule 4 is as shown in FIG. Further, the ratio of the moving average of the fields of the classes adjacent to the class according to the generation rule 4 is as shown in FIG.

  Next, probability density calculation when the processing time of the web system is compressed with the above class value will be described.

  As described above, the processing time of the compressed web system is analyzed by the user. For example, the user calculates the probability density of the processing time of the web system using the processing time of the compressed web system.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiment. In addition, it is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention. The present invention can also be provided as a compression method for compressing information about processing time, a program for compressing information about processing time in the information processing apparatus 1, and a storage medium storing the program.

1: Information processing device 2: Web system 2a: Web server 3: Terminal device 4: Network 7: Class value verification unit 7a: Smoothing target width 11: Reception unit 11d: Operation data 12: Totaling unit 12a: Determination unit 12b: Count unit 12c: Total unit 12e: Process identification information extraction condition 13: Class value 13d2, 13d3, 13d4: Example of class value generated according to generation rule 14: Compressed data storage unit 8: Statistical processing 8a: Cumulative probability data 8a1: Tangent line at representative value for each class of cumulative probability data 8b: Probability density data 8b1: Probability density data including multimodality 8b2: Probability density data prominent in multi-process identification information 9: Report part 9f: Report 9f1: Processing Identification information 9f2: Average processing time for each processing identification information 9f3: Percentile (n%) of processing time for each processing identification information Maximum processing time to cover)
9f4: Number of accesses 9f5: Compliance rate (occurrence rate of processing exceeding the target processing time)
21, 22: Operation data 21a, 22a: Date and time 21b, 22b: Identification information 21c, 22c: Processing time 51: URL
52: Query
53: Time 54: Class value 55: Frequency 56: Total value of processing time 57: Cumulative probability 58: Probability of occurrence for each representative value of the class before smoothing 59: Probability of occurrence for each representative value of the class after smoothing G78 : Cumulative probability graph of verification data G11, G21, G31, G41: Graph of class value and class width G12, G22, G32, G42: Graph of ratio of adjacent / class value G13, G23, G33, G43: Adjacent / class Width ratio graph G14, G24, G34, G44: Probability density graph T1-T4: Table of class generation rules and suitability determination results I1-I3: Information indicating the progress of class value verification processing

Claims (5)

Dividing into a class that has been verified beforehand that there is a smoothing target width that allows smooth probability density distribution information to be calculated from the frequency for each class, the frequency for each class of the web system processing time and the total processing time for each class And a summary section that updates
The upper limit value for each class is roughly arranged in a geometric sequence, the moving average for each smoothing target width of the class width for each class is approximately the same between adjacent classes, and the representative value of the class after smoothing the class Is a value having a significant number of digits in the order of seconds, minutes, or hours,
An information processing apparatus characterized by that. The information processing apparatus according to claim 1,
Before driving the aggregation unit, it has a class value verification unit that verifies a class value that can calculate smooth probability density distribution information by smoothing or calculates a smoothing target width,
An information processing apparatus characterized by that. The information processing apparatus according to claim 1,
A determination unit for determining which of the ranks the processing time of the web system belongs to;
The round-up unit rounds up the processing time of the web system to a multiple of the class width corresponding to the rank determined by the determination unit.
An information processing apparatus characterized by that. A compression method for an information processing apparatus,
The processing time required for the data processing of the web system is divided into classes that have been verified in advance that there is a smoothing target width that can restore smooth probability density distribution information from the frequency of each class, and the processing of the web system Update the frequency of time and the total processing time for each class,
The upper limit value for each class is roughly arranged in a geometric sequence, the moving average for each smoothing target width of the class width for each class is approximately the same between adjacent classes, and the representative value of the class after smoothing the class Is a value having a significant number of digits in the order of seconds, minutes, or hours,
A compression method characterized by the above. An information processing apparatus program,
The processing time required for the data processing of the web system is divided into classes that have been verified in advance that there is a smoothing target width that can restore smooth probability density distribution information from the frequency of each class, and the processing of the web system The information processing apparatus functions as a counting unit that updates the frequency of time and the total processing time for each class,
The upper limit value for each class is roughly arranged in a geometric sequence, the moving average for each smoothing target width of the class width for each class is approximately the same between adjacent classes, and the representative value of the class after smoothing the class Is a value having a significant number of digits in the order of seconds, minutes, or hours,
A program characterized by that.

Images