JP4968092B2 - Stall detection device, stall detection method, and stall detection program - Google Patents

Description

  The present invention relates to a technical field for detecting a stall occurring in a computer such as an application server.

  Conventionally, when the user job execution time that is regarded as a stall is not set, log collection is performed to determine a stall failure, and it is impossible to determine whether a stall failure has occurred unless the log is analyzed. Analyzing whether a stall has occurred since a failure occurred. If stalled, collect the stack trace and restart the stalled process to recover and identify the cause of the failure. It was a time-consuming and time-consuming work from the time of failure to recovery. Also, when a failure was recovered without collecting the stack trace, the cause of the failure could not be identified and it was necessary to wait for a failure to be reproduced.

On the other hand, a method is known in which a user job execution time that is regarded as a stall is set, and when the set time is exceeded, the application server determines that the stall has occurred and automatically restarts the process to recover. For example, Patent Literature 1 discloses a technique for calculating a method execution time, and when the execution time exceeds the maximum execution time of the method, a timeout occurs (stall detection) and the stack information is written to a log file. . Further, Patent Document 2 stores the business process execution time of an application program, and reads the stall monitoring operation start time and stores the stored business process execution time at the time of a stall monitoring operation that is performed at a fixed period. A technique for determining a stall state when the time between reading, the stall monitoring operation start time and the business process execution time exceeds a preset stall state determination time is disclosed. Patent Document 3 describes that the maximum value of the program execution time is obtained in the test stage.
JP 2006-146678 A JP 2002-7172 A JP-A-62-256050

  However, there is a problem that it is difficult to quickly detect a stall in the method of determining a stall when the maximum execution time is exceeded as in the conventional technique.

  In addition, with the conventional technology, it is difficult to make a detailed stall determination, and the time for determining a stall must also be set to a larger value for each operation. There is a problem that it takes. For example, it was possible to specify which operation caused the stall when the stall was detected, but did not know detailed information such as which line of the source of which method was stalled.

  Further, in the conventional technology, it is difficult to set the time considered as a stall to the minimum time, and there is a problem that it takes a long time to restore it as a stall.

  The present invention has been made in view of the above problems and the like, and after a stall occurs, the stall is quickly detected, the system is quickly restored, and a log that can identify the detailed cause of failure is automatically collected. It is an object to provide a stall detection device, a stall detection method, and a stall detection program.

To solve the above problems, a first aspect of the present invention, a stall detector for detecting stalling of the stack information of the application is generated by the application of available collection systems, the stack information given of the application Stack information recording means for recording in a memory at a stack collection interval, stall detection means for analyzing the recorded stack information and detecting a stall from the user job execution time or stack output count for each method, and the stall is detected A log output means for outputting the stack information recorded in the memory to a log.

According to a second aspect of the invention, a stall detection method for detecting the stall of the stack information of the application is generated by the application of available collection systems, recorded on the memory stack information of the application at a predetermined stack sampling interval Analyzing the recorded stack information, detecting a stall when the number of times the stack information does not change exceeds a preset number of times of 2 or more, and when the stall is detected And a step of outputting the stack information recorded in the memory to a log.

Invention stall detection program according to claim 3, records the computer to detect a stall stack information of the application is generated by the application of available collection systems, the memory stack information of the application at a predetermined stack sampling interval Stack information recording means for analyzing the recorded stack information, and a stall detection means for detecting a stall when the number of times the stack information has not changed exceeds a preset number of times, and the stall information, When it is detected, it functions as log output means for outputting the stack information recorded in the memory to a log.

  According to the present invention, after a stall occurs, the user can quickly detect the stall, quickly recover the system, and automatically collect a log that can identify the detailed cause of failure without any setting or operation by the user. Can do.

  Hereinafter, the best embodiment of the present invention will be described in detail with reference to the drawings.

  The present invention is characterized in that when a stall occurs in a server application, the stall is automatically detected based on stack information without any special setting by the user.

  First, the configuration and function of an application server according to an embodiment of the present invention will be described with reference to FIG.

  FIG. 1 is a block diagram illustrating an example of a schematic configuration of an application server.

  As shown in FIG. 1, the application server 80 includes a server application 30, a stack collection request process 40, a memory 50, a user job execution time analysis process 60, and a log 70. Note that the application server 80 is composed of, for example, a server computer, and the CPU mounted on the application server 80 reads out and executes a program (including the stall detection program of the present invention) stored in the storage unit, whereby the server application 30, the stack It functions as a collection request process 40 and a user job execution time analysis process 60.

  The server application 30 has a function of receiving a request from the client 10 via the network.

  The stack collection process 40 (an example of the stack information recording means in the present invention) has a function of receiving a setting request for a stack collection interval (the operator 20 can set the stack collection interval) from the operator 20, and sends the stack collection request to the server. It has a function of sending the application 30 and recording the stack information in the memory 50.

  The user job execution time analysis process 60 (an example of a stall detection unit and a log output unit in the present invention) acquires stack information recorded in the memory 50, analyzes the user job execution time, and records it in the memory 50 have. In addition, it has a function of outputting the stack information recorded in the memory 50 to the log 70 when it is determined that the stall has occurred from the user task execution time (stall detection). Further, it has a function of calculating the user job execution time during normal operation and updating the record in the memory 50, and a function of deleting old user job execution time information.

  Next, the operation of the embodiment of the present invention will be described with reference to FIG.

  The flowchart of the stack collection request process 40 is shown in FIG. 2, the flowchart of the user job execution time analysis process 60 is shown in FIG. 3, the stack information recorded in the memory 50 is shown in FIG. The execution time information recorded in the recorded memory 50 is shown in FIG. 5, and an outline of the stall determination method is shown in FIG.

  In FIG. 1, when a request is transmitted from the client 10 to the server application 30, the server application 30 executes a user task.

  As shown in FIG. 2, the stack collection requesting process 40 can acquire the stack collection interval (stack output interval) (step S1), and the operator 20 sets the stack collection interval (step S2: YES), this is acquired and the stack collection interval is reflected (step S3). When the operator 20 has not set the stack collection interval, the operation is performed with the default value set by the application server 80.

  As shown in FIG. 2, the stack collection request process 40 periodically (at a set stack collection interval) sends a stack collection request to the server application 30 during normal operation to grasp the user job execution status. The stack information of the server application 30 is recorded in the memory 50 (steps S4 to S5). For example, as shown in FIG. 4, the stack information manages the stack for each thread number, and records the stack output time and the stack.

  Next, as shown in FIG. 3, the user job execution time analysis process 60 periodically acquires the stack information of FIG. 4 recorded in the memory 50 (step S11), analyzes the stack information, Each execution time is measured (step S12). Then, the user job execution time analysis process 60 determines whether or not a stall has occurred from the measured execution time (step S13). As a method for determining a stall, for example, the user job execution time analysis process 60, as shown in FIG. 6 (1), for example, the latest stack (new stack) and the past stack (old stack) for each thread. If the stack is changed, the user task execution time (for example, (time B-time A shown in FIG. 6A) is calculated for the method obtained from the stack information, and is recorded in the memory 50. To do.

  On the other hand, when there is no change in the stack, it is determined from the method start time that is separately measured whether the execution of the method is continued or the same method is newly executed. When it is determined that the execution of the method is continued, the user task execution time in the memory 50 is provisionally updated.

  Then, as shown in FIG. 6 (1), the user job execution time analysis process 60 determines that the user job execution time has greatly exceeded the average execution time from a statistical point of view such as dispersion of execution time ( An example in which the average execution time is equal to or greater than a predetermined value times), it is determined that a stall has occurred (stall detection) (step S14: YES).

  The stall determination can be performed not from the user job execution time but from the stack output count by analyzing the stack information as shown in FIG. For example, the occurrence of a stall is determined from the number of times the stack information has not changed (when the number of times the stack information has not changed exceeds a predetermined number, it is determined that a stall has occurred (stall detection)). For example, since the stack information shown in FIG. 4 indicates how many times the state in which the stack does not change continues, the stall is determined if the state in which the stack does not change continues indefinitely compared to the normal operation.

  When the user job execution time analysis process 60 determines that a stall has occurred, that is, when a stall is detected, the stack information recorded in the memory 50 is output to the log 70 (step S15).

  Also, for example, as shown in FIG. 3, when the user job execution time analysis process 60 determines that the execution time has been recorded in the memory 50 for the specified number of times (step S16: YES), it recalculates the average execution time. The record in the memory 50 is updated (step S17). That is, the average execution time of the user job is periodically calculated and the average execution time of the memory 50 is updated.

  Further, the user job execution time analysis process 60 deletes the past execution time and stack (step S18). That is, the latest stack information for a certain period of time is recorded in the memory 50, and the old stack information and user job execution time are deleted by the user job execution time analysis process 60 in order to save memory.

  If the operator wants to initialize the average execution time of the user job, the operator issues an initialization request to the application server 80, and the application server 80 issues an initialization command to the user job execution time monitoring process 60 to execute the user job execution. The time monitoring process 60 initializes the average execution time of user tasks recorded in the memory 50.

  When it is determined that a stall has occurred, the application server 80 automatically restarts and restores the stalled process of the server application 30. When the stalled process of the server application 30 is restarted, an error is returned to the client 10. Since the client 10 can confirm that the stall has occurred in the server application 30 from the minor number of the error, the stack output to the log 70 is confirmed to identify the stall location.

  If it is found that the error is stalled from the error returned to the client, the server application can be confirmed from the log and the user application can be corrected.

  Next, another embodiment of the present invention will be described with reference to FIG.

  In the above embodiment, when a stall occurs in the server application 30, the stack information recorded in the memory 50 is recorded in the log 70 and the client 10 confirms the log 70. However, when the stall occurs, Information output to the log 70 can be notified to the client 10.

  Further, in the above-described embodiment, the case where a stall occurs is described, but it is also possible to output slowdown information to the log 70 and notify the client 10 when a slowdown failure occurs instead of a stall failure. A slowdown failure is a failure that takes longer than normal user task execution time due to machine load or the like. This can be realized because the user job execution time analysis process 60 records the user job execution time in the memory 50 in units of methods.

  As described above, according to the above-described embodiment, it is possible to automatically collect a log that can quickly detect a stall, quickly restore the system, and identify a detailed cause of failure after the stall has occurred.

  The stall determination can be performed automatically and in a minimum time by the application server without any setting or operation by the user. In other words, if the user job execution time that is considered to be a stall has not been set, conventionally there was only a way to check whether it was stalled from the log in order to determine whether it was stalled, According to the present embodiment, since the stack information of the user application is periodically recorded as statistical information in the memory, the application server detects the stall from the user job execution time even if the time for considering the stall is not set. The process can be automatically restarted and recovered.

  In addition, according to the above embodiment, the stack during normal operation is recorded in the memory, and the user job execution time is recorded in units of methods. Therefore, when a stall occurs, which logic of which operation is stalled Can be instantly determined by the application server. There is no need to collect logs for stall determination, and a stall failure can be recovered quickly.

  Further, according to the above-described embodiment, when a stall occurs, a stack trace is automatically collected, and the stall location can be confirmed at the source line level. Conventionally, after confirming that a stall has occurred, detailed information could not be obtained unless the user manually collected a stack trace. However, in the above embodiment, the stack is automatically logged when a stall occurs. The user does not need to perform setting and work because it is output to, and it does not take time. Furthermore, since the stack information of the user application is output to the log when a stall is detected, it is possible to acquire detailed information such as which line of the source of which method is stalled.

  Further, since the stall determination is performed from the stack information, the user job execution time can be measured in units of methods, and the stall determination can be performed in a smaller unit than in the past. As a result, the time to be regarded as a stall can be set for each method, and can be quickly recovered after a stall occurs.

  In addition, it is possible to recover from a stall failure before collecting the stack trace and not to identify the cause of the failure.

  Although the present invention is described assuming that a Java (registered trademark) application is used in the application server, it can be implemented in any system capable of collecting stack information.

It is a block diagram which shows an example of schematic structure of an application server. FIG. 10 is a flowchart of a stack collection request process 40. FIG. 6 is a flowchart of a user job execution time analysis process 60. 4 is a diagram showing stack information recorded in a memory 50. FIG. 4 is a diagram showing execution time information recorded in the memory 50 recorded in the memory 50. FIG. It is a figure which shows the outline | summary of a stall determination method.

Explanation of symbols

80 Application server
30 server applications
40 Stack collection request process
50 memory
60 User job execution time analysis process
70 logs

Claims (3)

A stall detection device that detects a stall that has occurred in the application of a system capable of collecting application stack information ,
A stack information recording means for recording in the memory at a predetermined stack sampling interval stack information of the application,
Analyzing the recorded stack information, and detecting a stall when the number of times the stack information does not change exceeds a preset number of two or more;
Log output means for outputting the stack information recorded in the memory to a log when the stall is detected;
A stall detection device comprising: A stall detection method for detecting a stall that has occurred in the application of a system capable of collecting application stack information ,
And recording in the memory stack information of the application at a predetermined stack sampling interval,
Analyzing the recorded stack information and detecting a stall when the number of times the stack information has not changed exceeds a preset number of times of two or more;
If the stall is detected, outputting the stack information recorded in the memory to a log;
A stall detection method comprising: A computer that detects a stall that has occurred in the application of the system capable of collecting the stack information of the application,
Stack information recording means for recording in the memory stack information of the application at a predetermined stack sampling interval,
Analyzing the recorded stack information and detecting a stall when the number of times the stack information does not change exceeds a preset number of times of 2 or more; and
A stall detection program that functions as a log output unit that outputs stack information recorded in a memory to a log when the stall is detected.

Images