JP2001134544A - Generating method and analyzing method for common log - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a means for one common log, with which all access states of the whole system by unit time and access states by clients can accurately be analyzed in real time while a server is in operation, as for a parallel server system wherein one information relay processor decentralizes the load to WWW servers or a proxy server. SOLUTION: A means is provided which adds the time when an information relay device is passed through to the header part which stores a transmission source IP address of request data sent from a client to a server. Each server of the parallel sever system has a means which issues an access log as its communication state, and the time when the request data pass through the information relay processor is written to the access log. The access logs of respective servers are gathered and integrated into one file, and the logs are rearranged in the order of the passage of the request data through the information relay processor.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to the field of a log management method in a parallel server system in which a load is distributed among a plurality of servers by one information relay processing device in a WWW system.

[0002]

2. Description of the Related Art Japanese Patent Application Laid-Open No. Hei 10-224349 discloses a method of collectively performing access data collectively for a plurality of WWW servers and Proxy servers. This shows a method of performing a total aggregation process by relating access logs of a plurality of WWW servers, and a method of performing a total aggregation process by relating access logs of a plurality of Proxy servers. Specifically, each WWW server and Proxy server has a tallying module that counts the access logs.Each tallying module counts the access logs of its own node, sends the tallying result to the management node, and the management node collects the tally. Perform processing. Alternatively, each WWW server and Proxy server sends an access log to the management node, and the management node performs the tallying process. In this known example, the tally result is displayed in a graph or a table. The counting method obtains the number of requests by counting the number of lines in the access log. Further, the number of lines in the access log is counted for each client that has sent the request data, and the number of requests for each client is obtained.

[0003]

In the above-described known example, in the access log tallying process, the number of lines of the access log of each server at the time of tallying is simply tallyed.
Since it is not possible to guarantee that the access logs of the WWW server and the Proxy server total the access status during the same period,
The access situation per fixed period cannot be analyzed accurately.
In order to analyze access status over a certain period, each WW
A method of extracting log information in the specified time range from the access logs of the W server and the Proxy server and summing them up is conceivable, but in general, there is an error in the clock between different computers. , There is a time lag, and it is difficult to collect access logs for the same period exactly. In addition, a method may be considered in which a certain computer notifies each server of the timing of the execution of the tallying process. It is difficult to analyze accurately. Therefore, in the conventional method, it has been difficult to accurately analyze the access status for each time zone and the access status for every minute and every second. However, in order to evaluate the performance of a plurality of WWW servers and Proxy servers as a whole, and to perform appropriate performance tuning and system configuration, it is important to accurately analyze the access status per unit time (per minute, per second). .

[0004] It is an object of the present invention to provide all access statuses per unit time of a plurality of WWW servers or Proxy servers which are load-balanced by one information relay processing device, access statuses for each client, and WWW server or Proxy.
An object of the present invention is to provide a common log generation method capable of accurately analyzing the access status of each server and real-time analyzing while these servers are operating.

Another object of the present invention is to provide a plurality of WWW servers or Pro servers which are load-balanced by one information relay processing device.
A common log analysis that analyzes in real time the access status of all xy servers per unit time, the access status of each client, and the access status of each WWW server or Proxy server while the server is operating, and displays the analysis results Is to provide a way.

[0006]

SUMMARY OF THE INVENTION The present invention relates to a parallel server system in which the load is distributed to a plurality of WWW servers or Proxy servers by a single information relay processing device, and the total access status of the entire system per unit time and An object of the present invention is to provide a means for generating one common log that can accurately analyze the access status for each client and the access status for each server in real time while the server is operating. It is another object of the present invention to provide a common log analysis method for analyzing an access state using a common log and displaying an analysis result.

Specifically, the following means are provided.

[0008] 1) For the purpose of generating a common log capable of analyzing the access status per unit time in the entire parallel server system, the passing time of the information relay processing device is added to the header of request data in the information relay processing device. Means to add a request and multiple WWWs to process the requested data
Means for issuing an access log indicating its own communication status in each server or proxy server, collecting access logs of each information processing device, sorting the request data in the order of arrival at the information relay processing device, and forming one common log Provide a means to generate.

2) In the information relay processing device, the access information is arranged in the order in which the request data arrives at the information relay processing device, and a common log including address information of the client that transmitted the request data can be generated. And, even if the IP header is changed in the request data to achieve the above object, the response data transmitted from the WWW server or the Proxy server always passes through the information relay processing device, and the information relay processing. In order for the response data to be transmitted from the device to the client that transmitted the request data, in the area of the source IP address of the IP header of the request data, the time when the device arrived at the information relay processing device from the source address and the source Means for converting the degenerated value of the address and a value including information on the identifier of the information relay processing device, and the IP header of the response data In the area of the destination address, means for converting the arrival time of the request data, the degenerate value of the destination address, and the value including the identifier of the information relay processing apparatus to the address of the client that transmitted the request data.

3) The information relay processing device or WWW
One node in the server or proxy server, or
In another node different from the two types of computers, the access information is arranged in the order in which the request data arrives at the information relay processing device, and a common log in which the address of the client that transmitted the request data is specified is generated. The purpose is to convert the access log of each WWW server or Proxy server into one common format log and integrate them into one common log, and to determine the source address of the access log of each WWW server or Proxy server. A means for converting a degenerate value into a correct source address and a means for rearranging access logs in the order of passage through the information relay processing device are provided.

4) In the module for generating a common log of 3), each WWW server is periodically registered in order to analyze the processing performance of the access status in the parallel server system in real time while the system is operating. Alternatively, a request to send an access log to the Proxy server is issued, a means for collecting the access log of each WWW server or Proxy server, a means for periodically updating the common log from the collected access log, and a means for periodically updating the common log Provides a means for analysis.

5) The common log is regularly updated by the means of 4), so that the common log is periodically read out and analyzed, whereby the access per unit time in the entire parallel server system is performed. A means for counting the number of requests, a means for counting the number of request data sent per unit time for each client sending request data, and a means for displaying the count result, and a unit for each WWW server or Proxy server processing the request data per unit time A means for totalizing the number of processed request data and displaying the total result, and a means for totalizing the request data for which normal processing was not performed in the WWW server or the Proxy server according to the request data for each client and displaying the total result And the WWW
Provided is a means for totalizing request data for which normal processing was not performed in the WWW server or the Proxy server according to the request of the request data for each server or Proxy server, and displaying a totaling means.

[0013]

Embodiments of the present invention will be described in detail below.

FIG. 1 shows a configuration of a WWW system and a parallel server system which are assumed in the present invention.
As shown in FIG. 1, a parallel server system (106) premised in the present invention includes one gateway (103) as an information relay processing device and a plurality of WWW servers (103) connected to the gateway (103). 104), and common log management node (105)
It consists of. All the WWW servers (104) connected to the gateway (103) can perform the same processing, and all the same WWW servers
Have content. The assumed WWW system is
Multiple clients (101) via the Internet (102)
Sends a plurality of request data to the gateway (103), and the gateway (103) sends the request data to one node in the WWW server (104) so that the load is distributed among the plurality of WWW servers (104). Forward. Each WWW server (104) processes the received request data according to the request, transmits response data as a processing result to the gateway (103), and transmits the response data to the client (101) via the gateway (103). Is done.

In the present invention, as shown in FIG. 2, a single gateway (103), a plurality of Proxy servers (202) connected to the gateway (103), and a parallel log management node (105). -A server system (204) may be assumed. In this case, all the Proxy servers (202) connected to the gateway (103) can perform the same processing. In the WWW system, a plurality of clients (101) transmit request data to a gateway (103) via an intranet (201), and the load of the gateway (103) is distributed among a plurality of Proxy servers (202). Request data to one node in the Proxy server (202) as described above. Each Proxy server
(202) executes the processing in the Proxy server in accordance with the request of the received request data, and then transmits the response data to the gateway (103)
Or send the response data to the gateway after accessing the WWW server via the Internet (203).
Send to (103). The response data is transmitted to the client (101) via the gateway (103).

In the present invention, there is no essential difference between the embodiment in the configuration shown in FIG. 1 and the embodiment in the configuration shown in FIG. 2. Therefore, in this embodiment, the embodiment based on the configuration in FIG. Will be described below.

In this embodiment, the gateway
(103), the common log management node (10
Although 5) exists, the processing in the common log management node (105) may be executed by the gateway (103) or one of a plurality of WWW servers.

FIG. 3 shows a main module configuration of each node in the parallel server system (106) shown in FIG. As shown in FIG. 3, the gateway (103) includes a request data processing module (303), a response data processing module (304), and one or more degenerate value conversion tables (30).
5) and a common log degeneration value conversion processing module (313).

The request data conversion processing module (303)
Destination IP of request data sent from client (101)
The address and the value of the source IP address are converted. The conversion of the destination IP address is performed in the destination IP address conversion processing module (301), and the conversion of the source IP address is performed in the source IP address conversion processing module (302). The destination IP address conversion module (301) converts the destination IP address of the request data from the IP address of the gateway (103) to the IP address of the WWW server (104) that transfers this request data.
Convert to address. By this processing, the request data is transferred to one WWW server via the gateway (103). Also, the destination IP address conversion module (302)
From the IP address of the client (101) that transmitted the request data, the identifier of the gateway (103), the passing time of the gateway (103), and the client (101 ) Is converted to a value indicating the degenerate value of the IP address. By this processing, the passing time of the gateway is added to the request data. Also, this process involves referencing and updating the degeneration value conversion table (305), and the degeneration value conversion table
By referring to the information written in (305), the client IP address
It is possible to convert to an address.

The response data conversion processing module (304)
The destination IP address of the response data transmitted from the WW server (104) is converted. The destination IP address of the response data transmitted from the WWW server (104) is equal to the source IP address of the corresponding request data. As described above, the source IP address of the request data is, by the source IP address conversion processing module (302), the identifier of the gateway,
It is converted into a value indicating the passage time of the gateway and the degenerate value of the IP address of the client (101) that transmitted the request data. Therefore, the destination IP address of the response data is also a value indicating such information. The response data conversion processing module (304) refers to the degenerate value conversion table (305) to determine the destination IP address of the response data, the identifier of the gateway, the passing time of the gateway, and the request data corresponding to the response data. From the degenerate value of the IP address of the client (101) that transmitted the request, the IP address of the client (101) that transmitted the request data corresponding to this response data
Convert to address.

By this processing, the response data is transferred to the client (101) via the gateway (103).

Common log degeneration value conversion processing module (313)
Receives the degenerated value of the IP address of the client (101) transmitted from the common log management node (105), and refers to the degenerated value conversion table (305) to obtain the IP address of the client (101). And converts the converted IP address of the client (101) to the common log management node (105).

The WWW server (104) comprises a WWW server program (306), a log information transmission processing module (307), an access log (308), and a log buffer (309). The WWW server program (306) generates an access log (308) indicating its own communication status. As described above, the request data received by the WWW server is transmitted by the source IP address conversion processing module of the gateway (103) in the field of the source IP address and the passing time of the gateway and the client (101) that transmitted the request data. Since the degenerated value of the IP address is added, the access log generated by the WWW server program (306) includes these contents. Also, the log information transmission processing module (307) copies the log to be transmitted to the common log management node (105) from the access log (308) to the log buffer (309), and transmits the log from the common log management node (105). When there is a request, the contents of the log buffer (309) are transmitted to the common log management node (105).

The common log management node (105) comprises a common log generation processing module (310), a common log analysis processing module (311), and a common log (312). The common log generation processing module (311) issues a request to periodically transmit an access log to each WWW server (104), and receives a log transmitted from each WWW server (104). And each WWW server (10
4) The degeneration value of the client IP address included in the log is transmitted to the gateway (103), and the client IP address converted by the common log degeneration value conversion processing module (313) transmitted from the gateway (103) is transmitted. Receive. In addition, logs of each WWW server (104) and common logs (312)
And updates the common log (312) in the order in which the request data passed through the gateway (103). The common log analysis processing module (311) uses the common log (312),
The access status of the entire parallel server system (106), the access status of each client (101), and the access status of each WWW server (104) are quantitatively analyzed, and the analysis result is displayed.

The operation of 1) gateway 103, 2) WWW server 104, 3) common log management node 105 will be described below with reference to flowcharts.

First, the operation of the gateway (103) will be described with reference to FIGS.
This will be described with reference to FIG.

First, processing of request data transmitted from the client (101) will be described. This process
This is executed in the request data processing module (303).

FIG. 4 shows a flowchart of the operation in the request data processing module (303).

In step 402, the gateway (103)
Receives the request data from the client (101).
In step 403, the request data processing module (303)
The destination IP address conversion processing module (301) converts the destination IP address in the IP header of the request data into the address of the WWW server (104) that transfers the request data from the gateway (103). In step 404, the source IP address conversion processing module (302) in the request data processing module (303) determines the source IP address in the IP header of the request data from the address of the client, Is converted to a value having information on the degenerate value of the address of the gateway and the identifier of the gateway. In Step 405, the request data is transmitted to the WWW server having the destination IP address converted in Step 403, and Step 406
Ends with

The order of step 403 and step 404 is as follows:
May be interchanged.

In the conversion of the destination IP address in step 403, the load status of each WWW server is monitored and the load of the most load is monitored, even in the case of static address conversion for uniquely determining the WWW server to be transferred from the client IP address and port number. Low W
Either dynamic address translation to translate to the WW server IP address may be used.

The processing in the source IP address conversion processing module (302) is one of the main processes in the present invention. The source IP address conversion processing will be described with reference to FIGS.

First, FIG. 5 shows a flowchart of the source IP address conversion process.

In step 502, the gateway (103)
Is the passing time of the request data.
In step 503, the source IP address in the IP header of the request data is converted into a degenerate value by referring to the degenerate value table (305), and the degenerate value table (305) is updated. In step 504, the passing time obtained in step 502, the degenerate value of the source IP address obtained in step 503,
Embed the gateway identifier in the area of the source IP address. The process ends at step 505.

The details of the process of converting the source IP address into the degenerate value in step 503 will be described with reference to FIGS. First, FIG. 6 shows the data structure of the degeneration value conversion table used in this processing. As shown in FIG. 6, the degenerate value conversion table contains a source IP address,
An IP address field (601), a degenerate value field (602) for storing a degenerate value, and a request data passing time field for storing the passing time of the request data at the gateway (103)
(603), a response data passing time field (604) storing the passing time of the response data transmitted from the WWW server to the request data at the gateway (103), and a log generation flag field indicating whether the response data has been written to the common log (605). The number of entries (the number of rows) of one reduced value conversion table (305) is equal to the number that can be converted to a reduced value. In this embodiment, the degenerate value is assigned to a one-byte length area of the IP header. Therefore, the degenerate value table in the present embodiment
The number of entries in (305) is 256. This table (305)
In the initial state, in each of the degeneration value fields (602), degeneration values from 0 to 255 are stored in order from the head of the entry, and nothing is written in all the remaining fields. The degeneration value field (602) is not rewritten, but the other fields are
In addition, it is written or deleted when response data passes and when a common log is generated. The rewriting process of this table will be described below when describing each process described above.
In the present embodiment, an entry in which nothing is written in all the fields other than the degenerate value field (602) is hereinafter referred to as an empty entry. Also, this table
Although (305) is one in the initial state, a plurality of (305) may be present because they are newly generated when the number of entries is insufficient. This will be described in detail with reference to FIG. Further, the table (305) has a table number in the order of generation, and each degenerate value table is associated with a table number by a pointer indicating the table number shown in (606).

FIG. 7 shows a flowchart of a process for converting a source IP address into a degenerate value.

First, in step 702, the degenerate value conversion table (305) is searched for an empty entry in ascending order of table number and from the top of the table. Steps
In step 703, if an empty entry is found in the search in step 702, the process proceeds to step 704, and in step 705, the searched empty entry is selected. Step 702
If no empty entry is found in the search, the process proceeds to step 706. In step 706, a degenerate value conversion table (305) is newly generated, and by using the pointer of (606),
Associate with the new table number. The new table number is a value obtained by adding 1 to the maximum value of the table number assigned to the existing degeneration value conversion table (305). Steps
After the end of 706, in step 707, the first entry of the generated new degeneration value conversion table is selected. Step 70
5 Or, after step 707 ends, in step 708, the passing time obtained in step 502 of FIG. 5 is substituted into the request data passing time fold (603) of the selected entry. In step 709, the value of the degenerate value conversion field (602) of the selected entry is assigned to the determined area of the request data IP header. The process ends at step 710.

The operation of the source IP address conversion processing module (302) in the flowcharts of FIGS. 5 and 7 described above will be described more specifically with reference to FIGS.
First, it is assumed that request data is transmitted from the client having the IP address “133.144.5.233” to the gateway (103). The arrival time of this request data is
The clock of (103) is `` May 2, 1999 12: 7: 32 '',
This time is defined as the arrival time. In this embodiment, hereinafter,
Client's IP address "133.144.5.233", request data gateway arrival time "May 2, 1999 12:07:32
"Second" will be used as an example. Next, the degeneration value conversion table (305) is searched in the order of the table numbers, and as shown in FIG. 6, the 256th entry of the table number 3 is selected as an empty entry, and the transmission source address of this request data and the Set the time in the source IP address field of the selected entry (601) and the request data passing time field (603)
Respectively. As can be seen from FIG. 6, the value of the degenerate value field of this entry is “255”, so the source IP address is degenerated to 255.

FIG. 8 shows an example in which the data structure of the IP header in the request data and the field indicating the source IP address are converted by the source IP address conversion processing module (302). The data structure of the IP header is
(801). Source IP in IP header
The address field (802) is 4 bytes long in the case of IPv4, and is the source of the request data upon arrival at the gateway.
“133.144.5.233” is stored in the IP address field (802) (803). In this embodiment, the upper one byte of the 4-byte source IP address field is converted to the gateway identifier (8
04), the next two bytes are converted to the passing time (805), and the lower one byte is converted to the degenerate value (808) of the source IP address. In this embodiment, the identifier of the gateway is “192”, and the upper one byte of the source IP address of all request data is always converted to “192”. Therefore, as shown in FIG. 8, the upper 1 of the source IP address field (802)
Byte (804) has been converted to 192. The next two bytes correspond to the “minute” and “second” of the passing time, respectively.
07 (806) and 32 (807) indicating "7 minutes 32 seconds" are substituted.
Furthermore, in the lower 1 byte, the degenerate value of the source IP address
255 (808) has been assigned.

In this embodiment, the passing time added to the request data in the source IP address conversion processing is “minute”.
It is only "seconds" and is incomplete as information on the time to be recorded in the log. Common log generation processing module described later
(310), and common log degeneration value conversion processing module (313)
In the degeneration value conversion table (305), the request data passing time field (60
The time is converted into a more detailed passing time stored in 3), and the detailed time is recorded in the common log (312).

In this embodiment, the number of degenerate values is 25.
Although it is 6, it can be increased or decreased. To increase the number, the field of the identifier of the gateway may be reduced.

In this embodiment, IPv4 is assumed, but the present invention is not particularly limited to IPv4. It can also support IPv6. In this case, the number of degenerate values can be more than 256, and more detailed passing time can be added.

The operation of the request data processing module (303) has been described above. The request data transmitted from the client (101) passes through the gateway (102) and executes the processing in the request data processing module (303). Later, the WWW server (10
FIG. 9 shows the state transferred in 4). As shown in FIG. 9, the request data (901) is transmitted to the gateway (103) via the Internet. The destination IP address (902) of the request data upon arrival at the gateway (103) is
The IP address (907) on the Internet side in (103) “192.168.1.1” and the source IP address (903) are the IP address “133.144.5.233” of the client. Request data processed by the request data processing module (303) in the gateway (103) is shown in (904). Destination IP address
(905) is processed by the destination IP address conversion module (301).
The request data is converted into “10.0.0.1” which is the IP address of the WWW server 1 (909), and the request data is transferred to the WWW server 1 (909).

The transmission source IP address (906) is shown in FIG.
As described above with reference to FIG. 8, the value is converted into a value “192.07.32.255” indicating information of the gateway identifier, the passage time of the gateway, and the degeneration value of the IP address of the client.

By the request data processing in the gateway, the request data is transferred to any one of the WWW servers (104), and the source IP address in the IP header is rewritten, and Is converted into the degenerate value of the client IP address and the identifier of the gateway. Gateway transit time and client IP
The degenerate value of the address is information used when a common log is generated.
From the WWW server (104) log, the requested data is
One common log (312) sorted using the passing time as a key at 03) can be generated. Also, by adding the degenerate value of the address of the client (101), the address of the client (101) can be obtained from the degenerate value using the degenerate value address conversion table (305). ) Can be used to analyze the access status of each client. The method of converting the degenerate value to the client address will be described later in detail in the description of the operation of the common log management module (105). Further, it is guaranteed that response data transmitted from the WWW server is always transmitted to the gateway from the identifier of the gateway. This description will also be described later.

Next, processing of response data transmitted from the WWWW server will be described. This processing is executed in the response data processing module (304).

FIG. 10 shows a flowchart of the operation in the response data processing module (304).

First, in step 1002, the WWW server
The response data transmitted from (104) is received.

In step 1003, the destination IP address of the received response data is converted. The destination IP address field in the IP header of the response data indicates the identifier of the gateway, the time at which the request data passed through the gateway, and the information of the client that transmitted the request data by the source IP address conversion processing in the request data described above. It has been converted to a value indicating the degenerate value of the IP address. These values are converted to the IP address of the client transmitting the response data by referring to the degenerate value conversion table (305). In step 1004, response data is transmitted to the client that transmitted the request data, and the process ends in step 1005.

Details of the processing in step 1003 will be described with reference to FIGS. 6 and 11. FIG. 6 shows the data structure of the degenerate value conversion table (305) as described above. FIG. 11 is a flowchart showing a detailed processing flow in step 1003.

In step 1102, the degeneration value conversion table (305) is referred to in ascending order of table number, and
At 103, an entry is found in which the degenerate value of the client address in the destination IP address field in the IP header of the response data matches the value of the degenerate value field (602). In step 1104, the request data passing time field in the entry selected in step 1103
(603), it is checked whether the response data matches the passage time of the gateway in the destination address field in the IP header. If they do not match, the process proceeds to step 1105, where the next degeneration value conversion table (305) is referred to, and steps 1113 and 1114 are similarly executed. If they match, this response data can be said to be response data to the request data represented by the selected entry. In this case, the process proceeds to step 1106.

In step 1106, the address of the source IP address field (601) in the selected entry is assigned to the destination IP address area of the response data.
In step 1107, the current time is assigned to the response data passage time field (604) of the selected entry, and the process ends in step 1108.

Incidentally, the gateway (1) of the request data corresponding to this data added to the response data in the present embodiment.
The passing time of 03) is only information on “minutes” and “seconds”.

For this reason, the comparison in step 1104 is as follows:
The passing time added to the response data is compared with only the part representing “minute” and “second” in the request data passing time field (603) of the degeneration value conversion table (305). In the parallel server system (106) shown in FIG. 1, the time from when request data passes through the gateway (103) to when response data arrives from the WWW server (104) for this request data is: It takes only a few seconds at most, and if the gateway periodically checks the degeneration value conversion table (305) and clears unnecessary entries, a correct entry can be searched by this comparison method. Specifically, the gateway (1
03) periodically checks each entry of the degeneration value conversion table, and a sufficient time (several minutes or more) has elapsed since the time of the response data passage time field (604), and response data is no longer transmitted. If it can be determined that there will be no entry, that entry is cleared to make it an empty entry. If the response data to the request data does not return after a certain period of time (several minutes or more) after the request data has passed through the gateway, a failure has occurred in the WWW server that transferred the request data. This entry is also assumed to be an empty entry. By means such as the above, the gateway periodically performs the degeneration value conversion table (305).
By clearing the entry, information on request data for a period exceeding one hour does not remain in the entry, so that if the "minute" and "second" parts match, the request data and the response data can be correlated. It can be said that there is.

The degeneration value conversion table (305) is also used in the common log (312) generation processing. The log generation flag field (605) of the degenerate value conversion table (305) is used so that the entry is not cleared as described above before the common log is generated. In the log generation flag field (605), a flag is set as to whether or not there is a reference at the time of generating a common log, and when the entry is cleared as described above, the value of the log generation flag field (605) is referred to as a reference. Only when it is "1" as shown, it is determined that the entry is empty. The process of writing to the log generation flag field (605) will be described later in detail in the description of the operation of the common log node (105).

The operation of the response data processing module (304) has been described above. The response data transmitted from the WWW server (104) passes through the gateway (103) and is processed by the response data processing module (304). FIG. 12 shows a state after the execution, which is transmitted to the client (101) via the Internet. FIG. 12 shows the request data (90
4) arrives at the WWW server 1 (909), and after the WWW server 1 (909) executes processing on the request data (904), response data (1201) as a processing result is transmitted. . As shown in FIG. 12, the destination IP address in the IP header of the response data (1201) transmitted from the WWW server 1 (909)
(1202) is `` 192.07.32.255 '', the source IP address (120
3) is “10.0.0.1”. The destination IP address (1202)
The request data (901) is equal to the source address in the IP header of the request data (904) in FIG.
Reflects the result of the translation of the source IP address. The source IP address is the IP address of the WWW server 1 (909).
Address.

As described above, the destination IP address (1
202), the upper one byte represents the identifier of the gateway. Using this identifier, the routing and routing of each WWW server is used to ensure that response data is sent to the gateway.
Set the table (1204).

As shown in FIG. 12, the routing table (1204) of each WWW server has an IP address field (1
In the entry in which “192.XXX.XXX.XXX” is stored in 205), “192.168.2.1” is stored in the field (1206) indicating the IP address of the next router. In other words, the data sent from each WWW server is
If the upper one byte of the address is 192, it is always transmitted to the IP address “192.168.2.1”. In the present embodiment, “192,1
The address of "68.2.1" indicates the IP address of the side connected to the WWW server of the gateway. If the upper 1 byte of the destination IP address of the data transmitted from each WWW server is "192", be sure to Sent to gateway. In this embodiment, when the request data passes through the gateway, the upper 1 byte of the source IP address is always converted to "192".
The upper one byte of the IP address is always "192", so that the response data is always transmitted to the gateway (103).

The response data (1201) arrives at the gateway, and the response data processing module in the gateway (103)
The request data processed in (304) is shown in (1207). The destination IP address (1208) is stored in the response data processing module (30
According to 4), from the value `` 192.07.32.255 '' including the identifier of the gateway, the passing time of the request data, and the degenerate value of the IP address of the client that sent the request data, the IP address of the client that sent the request data is 133.
144.5.233 ". As a result, the response data (1207) is transmitted from the gateway (103) to the client that transmitted the request data. In this embodiment,
As shown in (1209), the source IP address of the response data is not converted. However, the source IP address of the response data may be converted. Source IP of response data
By converting the address, the IP address of the WWW server can be hidden from the outside.

As described above, the response data is processed in the gateway (103) by the response data processing in the gateway (103).
Indicates that the request data is transmitted to the client that transmitted the request data by rewriting the destination IP address in the IP header.

The operation of the gateway (103) has been described above. The operation of the common log degenerate value conversion processing module (313) will be described later in the description of the operation of the common log management node (105).

Next, the operation of the WWW server (104) will be described with reference to FIGS. Each WWW server (104)
It has means for executing a WWW server program (306), processing request data from a client, and transmitting response data as a result of processing the request data. Also,
The WWW server program (306) generates an access log (308) for recording the contents of the request in the request data, the processing results of the WWW server, and the like. FIG. 17 shows the access log (1701) in this embodiment. The fields (1702), (1703) and (1704) contain the contents of the source IP address field in the IP header of the request data. In the present embodiment, the request data is obtained by converting the source IP address in the gateway (103), and as shown in FIG. 17, the gateway identifier (1702), the gateway passage time (1703), the source IP address (1704) is shown. The next field (1705) indicates the date and time, and indicates the time at which the WWW server processed the request data, and indicates 12:07:34 on May 2, 1999. Further, +0900 in the field (1705) indicates a time difference from Greenwich Mean Time. Field (1
706) indicates a command requested by the client in the request data, and the command “GET” indicates that the WWW server is requesting data acquisition.
Field 1707 is a field indicating the request destination, and is "htt
"p: //" indicates the protocol, and "webserver" indicates the name of the server. In this embodiment, "webserve
"r" is the name of the parallel server system shown in FIG. The next “/index2.html” indicates the data name to be acquired. The field (1708) is a protocol for transferring data, and is assumed to be "HTTP / 1.0" in this embodiment. The field (1709) is an error code indicating the processing result for the request data, and “200” indicates that the processing has been normally performed. The field (1710) indicates the size of the transmitted data, and “index2.html” indicates “78
9 "bytes of data.

The WWW server also executes a log information transmission process. FIG. 1 shows the operation of the log information transmission processing module.
This will be described with reference to the flowchart of FIG.

In step 1302, the common log node (1
From 05), it is checked whether there is a transmission request to transmit a log. If there is a transmission request, the process proceeds to step 1302. If there is no transmission request, the process proceeds to step 1305.

If there is no transmission request, the access log shown in FIG. 17 is read in step 1305. When reading the access log for the first time, all records (all lines) are read. Thereafter, only the newly added record is read from the time of the previous reading. The read contents are copied to the log buffer (309). All of the read contents may be directly copied to the log buffer (309), or only the information necessary for the common log (312) may be copied. In this embodiment, it is assumed that the access log is copied to the log buffer in the format of the access log shown in FIG. Upon completion of copying to the log buffer (309), the flow returns to step 1302.

If there is a transmission request, the contents of the log buffer (309) are transmitted to the common log management node (105) in step 1303. After the transmission processing is completed, in step 1304, all the contents of the log buffer (309) are deleted,
Return to step 1302.

The operation of the WWW server (104) has been described above.
Each WWW server (104) has means for generating an access log (308) by the WWW server program (306). As a result, the passage time at the gateway added to the request data at the gateway is written in the access log, and each WWW
By collecting this access log in the server, it is possible to generate a common log in which the logs are sorted in the order in which the request data passes through the gateway (103). In addition, the common log management node has a means for transmitting the log, when there is a log collection request from the common log management node (105),
A new log can be transmitted to the common management node (105).

Finally, the operation of the common log management node (105) will be described with reference to FIGS.

First, a common log generation processing module (310)
Will be described with reference to FIGS.

FIG. 14 is a flowchart of the common log generation process.

At step 1402, the common log management node (105) issues a transmission request to transmit a log to each WWW server (104). With this, each WWW server (104)
Sends the contents of the log buffer (309) to the common log management node (105) as described in FIG.

At step 1403, a log transmitted from each WWW server is received. In step 1404, for each access log received in step 1403, the format of the access log is converted to a common log format. Here, the degenerate value of the client's IP address is
Convert to an address, and convert the passing time of the gateway to a more detailed time. Details will be described later with reference to FIGS. In step 1405, each access log converted into the common log format in step 1404 is integrated into one file. In step 1406, step 1405
And the common log are integrated, and the logs are sorted in the order of the passage time of the gateway (103). After a predetermined time has elapsed in step 1407, the process returns to step 1402. As a result, the common log can be updated periodically.

The details of the processing in step 1404 will be described with reference to the flowchart shown in FIG. In step 1502, the IP address of the WWW server that transmitted the access log is obtained. In step 1503, the first line of the access log sent from the WWW server is read. In the present embodiment, it is assumed that an access log (1701) in the format shown in FIG. 17 is transmitted. In step 1504, the degeneration value (1703) of the source IP address, which is the IP address of the client (101) that transmitted the request data, and the time (1702) at which the request data passed through the gateway (103)
Send to (103). These values are converted and converted by the common log degeneration value conversion processing module (313) of the gateway (103) into the IP address of the client (101) that transmitted the request data and the detailed passage time of the gateway (103). The value is transmitted to the common log management node (105).
The operation of the common log degeneracy value conversion processing module (313) will be described later in detail with reference to FIG.

At step 1505, the gateway (10
3) of the client that sent the request data sent from
The source IP address, which is the IP address, and the gateway (1
Receive the detailed passing time of 03). In step 1506, these received values are written to the currently selected line of the access log. In step 1507, when the access log has been read up to the last line, the process proceeds to step 1509,
If the last line has not been read, the process proceeds to step 1508.
If you have not read the access log to the end, step 15
At 08, read the next line in the access log and go to step 15
Return to 04. If the access log has been read to the end, in step 1509, extract information necessary for generating a common log from the access log.In step 1510, the IP address of the WWW server that generated the log obtained in step 1502, and The information necessary for generating the common log extracted in step 1509 is arranged according to the format of the common log. Step 1511
Ends with

Next, the operation of the common log degeneration value conversion processing module (313) in the gateway (103) will be described with reference to the flowchart shown in FIG.

In step 1602, the degenerate value and the passage time of the source IP address transmitted from the common log management node (105) are received. In step 1603, the degeneration value conversion table (305) of table number 1 is selected. In step 1604, the degeneration value of the received source IP address and the degeneration value field (602) of the degeneration value conversion table (305) are selected.
Is searched for an entry in the degenerate value conversion table (305) that matches. In step 1605, the request data passing time field (60
Check if 3) matches the passing time received. If they match, the process proceeds to step 1606; otherwise, the process proceeds to step 1607. If they do not match, in step 1607, the degeneration value conversion table (305) of the next table number is selected, and the flow returns to step 1604. If they match, it indicates that the request data corresponding to the selected entry matches the request data to be processed in the access log. In step 1606, the IP address of the source IP address field (601) of the selected entry and the value of the exchange request data passage time field (603) are transmitted to the common log management node (105). Step 160
In step 8, 1 is stored in the log generation field (605) of the degeneration value conversion table (305). This value indicates that the reference process at the time of generating the common log has been executed. Step 16
Ends at 09.

In this embodiment, it is assumed that the update of the common log is performed within several seconds to one hour. Gateway (1
As described in the operation description of (03), in the present embodiment, the gateway (10
Since the passing time of 3) is only information on “minutes” and “seconds”, in order for a correct entry to be selected by referring to the degeneration value conversion table, the content of the request data is set to 1 in the degeneration value conversion table entry. There must be no more than hours. Therefore, as described above, the gateway periodically checks the degeneration value conversion table (305), and a sufficient time (several minutes or more) has elapsed since the response data passed, and Clear entries that have been referenced when generating the common log. Therefore, if the update of the common log (312) is also performed at intervals of one hour or less, it is possible to guarantee that the conversion from the degenerate value to the client address is performed correctly without leaving the entry of the degenerate value conversion table for one hour or more. . In this embodiment, the common log management node (105) transmits a pair of the degenerate value of the source IP address and the passing time,
The gateway (103) receives these information one by one and executes the conversion process, but the common log management node (103)
Collectively sends the degenerate values and transit times of all source IP addresses in the access log, and sends the gateway (103)
However, such information may be received and the conversion process may be performed collectively.

As another method for converting the degenerate value of the source IP address to the source IP address and converting the passing time of the gateway into a more detailed passing time, the gateway (103) transmits the degenerated value conversion table ( 305) transmits a copy of the entry to the common log management node (105), and the common log management node converts the degeneration value and the passage time of the source IP address into the source IP address and detailed passage time, respectively. You may do it. In this case, the gateway (103) does not need the log generation flag field (605) in the degeneration value conversion table (305). The gateway (103) may clear the entry after a sufficient time has elapsed from the time written in the response data passage time field and the copy has been transmitted to the common log management node (105).

In the common log management node (105), the entry transmitted from the gateway (103) may be deleted after being used for the conversion process.

Further, in this embodiment, the common log is generated by a different node from the gateway (103).
The common log may be generated in the gateway. In this case, the conversion between the degenerate value of the source IP address and the passage time does not involve communication with another node.

FIG. 18 shows an example of the common log (1801) generated by the common log generation processing described above. A field (1802) shown in FIG. 18 is a source IP address field indicating the address of the client that transmitted the request data. The field (1803) indicates the IP address of the WWW server that has executed the processing for the requested data.
The field (1804) indicates the time when the request data arrived at the gateway.

The field (1805) indicates a command requested by the client in the request data, and the field (1806) indicates a field indicating the request destination.
The field (1807) is a protocol for transferring data, and the field (1808) is an error code indicating a processing result for the requested data, and "200" indicates that the processing was normally performed. Field (1809) indicates the size of the transmitted data. As shown in FIG. 18, the common log (1801) is arranged in the order that the request data has passed through the gateway (103). In addition, the IP address of the client (101) that transmitted the request data and the IP address of the WWW server that executed the processing for the request data are described.

As described above, it has been shown that the request data is sorted in the order in which the request data has passed through the gateway, and a common log in which the IP address of the client that transmitted the request data is described. It also showed that the common log is updated regularly.

Finally, the common log analysis processing module (31)
An example in which the access status is analyzed for each unit time using the common log (312) according to 1) will be described with reference to FIGS. Note that the analysis methods in FIGS. 19 to 22 are self-evident, and thus description thereof is omitted here.

FIG. 19 shows a display example of the analysis result of the access status for each client, in which the number of requests per second is displayed as a bar graph (1903) for each client IP address (1902). Here, the number of requests is
This is the number of request data sent by the client. FIG. 20 shows the analysis result of the access status for each WWW server, and the number of requests received per second is displayed in a bar graph (2003) for each WWW server (2002). The analysis results of FIG. 19 and FIG.
This can be used for a system configuration such as estimation of the required number of WWW servers in the system (106), performance tuning of each WWW server, and the like. In addition, using the analysis results in FIGS.
The load distribution method to the WW server can be dynamically changed. For example, when the WWW server that transfers the requested data is determined based on the address of the client, it is clear from the analysis result shown in FIG. 19 that there are two clients with particularly large numbers of requests, and that these two clients are the same.
If the request data is transferred to the same WWW server, the request data of one of these clients is searched for another WWW server having a lower load from the analysis result of FIG. If the data is changed to be forwarded to the WWW server, the load between the WWW servers can be smoothed. Also, from FIG. 20, when there is a large difference in the number of requests accepted per second between WWW servers,
By changing the load distribution method at the gateway, the load can be smoothed between WWW servers.

FIG. 21 shows an analysis result of the status of the processing result of the request data for each WWW server. This can be analyzed by referring to the status field (1809) in FIG. In FIG. 21, request data that the WWW server program returns an error code has been sent to the WWW server 2 and an illegal access has occurred from the client, or
It is possible that a failure has occurred in the WWW server 2. FIG. 22 shows the status of an error request for each client. Since a client having many error requests may have attempted unauthorized access, gateway () In step 103), it is possible to take measures such as not performing the transfer process or transmitting data including the contents of the warning to the client.

[0087]

According to the common log generation method of the present invention, it is possible to generate one common log indicating the communication status of the entire parallel server system in which the load is distributed among a plurality of servers by one information relay processing device. it can. In addition, since the common log according to the present invention is added with the passing time at the information relay processing device, even if processing is performed between different servers, the data from the client is arranged in the order in which the data arrived at the information relay processing device. Alternatively, a common log can be generated. Thus, the access status of the entire parallel server system per unit time can be accurately analyzed. Further, since the present invention has means for periodically updating the common log during the operation of the parallel server system, it is possible to analyze the access status of the entire parallel server system accurately in real time.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of a parallel server system according to the present invention assumed in an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration example of a parallel server system according to the present invention.

FIG. 3 is a diagram showing a module configuration in a gateway, a WWW server, and a common log management node of a parallel server system assumed in the embodiment of the present invention.

FIG. 4 is a flowchart in a request data header processing module of the gateway.

FIG. 5 is a flowchart of a source IP address conversion process of request data in the gateway.

FIG. 6 is a flowchart of a process for converting a source IP address of request data in a gateway into a degenerated value.

FIG. 7 is a diagram showing a data structure of a degeneration value conversion table.

FIG. 8 is a diagram showing an example in which a transmission source IP address of request data is converted in the embodiment of the present invention.

FIG. 9 is a diagram showing an example of how request data transmitted from a client is transferred to a WWW server via a gateway in the embodiment of the present invention.

FIG. 10 is a flowchart of a response data header process of the gateway.

FIG. 11 is a flowchart of a destination IP address conversion process of response data in the gateway.

FIG. 12 shows an embodiment of the present invention.
FIG. 9 is a diagram illustrating an example of a state in which response data transmitted from a WWW server is transmitted to a client via a gateway.

FIG. 13 is a flowchart of log information transmission processing in a WWW server.

FIG. 14 is a flowchart of a common log generation process in the common log management node.

FIG. 15 is a flowchart of a process performed by the common log management node to convert an access log generated by each server into a common log format.

FIG. 16 is a flowchart of a degeneration value conversion table reference process in the common log management node.

FIG. 17 is a diagram showing an example of an access log generated by a WWW server according to the embodiment of the present invention.

FIG. 18 is a diagram showing an example of a common log according to the embodiment of the present invention.

FIG. 19 is a diagram showing an analysis result display indicating the number of requests for each client in an analysis example using a common log generated in the present invention.

FIG. 20 is a diagram showing an analysis result display indicating the number of accepted requests for each WWW server in an analysis example using a common log generated in the present invention.

FIG. 21 is a diagram showing an analysis result display showing a ratio of error requests for each WWW server in an analysis example using a common log generated in the present invention.

FIG. 22 is a diagram showing an analysis result display indicating the number of error requests for each client in an analysis example using a common log generated in the present invention.

[Explanation of symbols]

101: Client, 102: Internet, 103: Gateway, 104: WWW server, 105: Common log management node, 106: Parallel server system, 201: Intranet, 202: Proxy server, 203: Internet, 204
... Parallel server system, 301 ... Destination IP address conversion processing module, 302 ... Source IP address conversion processing module, 303 ... Request data processing module, 304 ... Response data processing module, 305 ... Degenerate value conversion table, 3
06 ... WWW server program, 307 ... Log information transmission processing module, 308 ... Access log, 309 ... Log buffer,
310 ... common log generation processing module, 311 ... common log analysis processing module, 312 ... common log, 601 ... source IP address field in the degeneration value conversion table, 602 ... degeneration value field in the degeneration value conversion table, 603 ... degeneration value conversion Request data passing time field in the table, 604 ... Response data passing time field in the degeneration value conversion table, 605 ... Log generation flag field in the degeneration value conversion table.

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Shoji Kodama 1099 Ozenji Temple, Aso-ku, Kawasaki City, Kanagawa Prefecture F-term in Hitachi, Ltd. System Development Laboratory 5B042 GA12 GA18 GA34 GA39 MA08 MC09 MC40 5B045 BB42 BB50 GG04 JJ08 5B089 GA11 GA19 GB02 JA21 JA35 JA36 JB22 JB24 KA12 KB11 MC02 MC03 MC06 MC08

Claims (5)

[Claims] A plurality of information processing apparatuses for transmitting and receiving data in a bidirectional manner by a specific information communication processing mechanism, and transmitting request data for requesting data transmission; A process of relaying request data transmitted from a device group and transferring the request data to one of a plurality of information information processing device groups capable of executing the same process on the request data; An information relay processing device that executes a process of relaying response data that is a processing result for request data transmitted from the information processing device that transmitted the request data, and receiving the request data,
In a network including a plurality of information processing devices that execute a process based on a request and transmit response data as a processing result, the information relay processing device adds a passing time of the request data in the information relay processing device to the request data. Means for issuing individual communication information, which is its own communication status, in each of a plurality of information processing apparatuses for processing request data, and collecting individual communication information in each information processing apparatus, and transmitting the request data to the information relay processing apparatus. A means for generating one piece of communication information by rearranging the communication information in the order of arrival. 2. The information relay processing device according to claim 1, wherein
Means for converting the data from the source address to a value including the arrival time of the data, the degenerate value of the source address, and the identifier of the information relay processing device in the area where the source address of the request data is stored; In the area where the destination address of the response data transmitted by the information processing apparatus to be transmitted is stored, the information processing that transmitted the request data from the value including the arrival time of the request data, the degeneration value of the destination address, and the identifier of the information relay processing apparatus By having the means for converting to the address of the device, the individual communication information issued by each information processing device that processes the request data may include the information of the degeneration value of the source address and the passing time of the information relay processing device Response data is transmitted to the information processing apparatus that has transmitted the request data even if the transmission source address of the request data is changed. Common log generation method to ensure the door. 3. The information relay processing device according to claim 1, or a specific one information processing device among a group of information processing devices that process request data, or another information processing device different from the two types of devices. A common means for issuing a request to transmit each communication information to each information processing apparatus, and a means for converting a degenerated value of a source address to a source address for each received communication information. Means for converting the communication processing information for each request data and the communication processing information for each request data in the order of passage through the information relay processing apparatus, so that the communication status per unit time of the entire information processing apparatus group transmitting the response data is obtained. A common log generation method capable of generating one piece of communication processing information capable of analyzing a communication state according to each communication source address. 4. A means for periodically collecting communication information of each information processing apparatus, and means for periodically updating a common log based on communication information of each information processing apparatus collected periodically. And a means for periodically analyzing the common log, so that the processing performance and access status per unit time of the entire information processing device group can be analyzed in real time while the information processing device group is operating. A method for generating a common log, the method comprising: 5. The common log is periodically updated according to claim 4, means for periodically reading the common log, and determining the total number of requested data per unit time from the information of the periodically read common log. A requesting means, a means for periodically counting the number of transmitted request data, and a display of the number of request data transmitted per unit time for each information processing apparatus which has transmitted the request data, based on the result of totalization by the means. Means for receiving request data, and periodically summing up the number of request data processed per unit time for each information processing apparatus which executes processing based on the request of the request data; and Means for displaying the number of request data processed per unit time for each information processing apparatus that executes processing based on the request of the request data from the totalized result, Means for periodically summing up the number of request data for which the processing requested by the request data per unit time was not performed normally for each information processing apparatus that has transmitted the Means for displaying the number of request data for which the processing requested by the request data per unit time was not performed normally, for each information processing apparatus that transmitted the request data, receiving the request data, and performing the processing based on the request for the request data. For each information processing apparatus to be executed, means for periodically summing up the number of request data for which the processing requested by the request data per unit time was not performed normally, and request data based on the count result summed up by the means, Request data for which the processing requested by the request data per unit time was not performed normally for each information processing apparatus that receives and executes the processing based on the request for the request data Means for displaying request data, the processing per unit time for the entire information processing apparatus group that processes the request data, and for each information processing apparatus that has transmitted the request data, and for each information processing apparatus that processes the request data. A common log analysis method, wherein performance and access status are analyzed in real time while the information processing apparatus group is operating.