JP2013171541A - Visualization device, visualization method and visualization program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily specify a bottleneck in a large-scale distributed processing system.SOLUTION: A log extraction unit extracts a log showing communication between processes from among logs outputted in a large-scale distributed processing system. A log structuring unit generates log information which shows a transmission source process and a transmission destination process in which a processing request is transmitted/received in the communication, transmission source server and a transmission destination server in which the transmission source process and the transmission destination process are separately executed, content of the processing request and time when the communication has occurred, on the basis of information included in the log extracted by the log extraction unit. A communication drawing data generation unit and a process drawing data generation unit generate drawing data in which transmission/reception of the processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server is drawn so as to show the content and the time of the processing request, on the basis of the log information generated by the log structuring unit. A drawing control unit displays the drawing data on a display unit in a client device.

Description

  The present invention relates to a visualization device, a visualization method, and a visualization program.

  Conventionally, a large-scale distributed processing system that autonomously executes distributed processing on hundreds to thousands of servers is known. In such a large-scale distributed processing system, one process is realized by cooperation of a plurality of processes. For example, in a large-scale distributed system, processes such as reading and writing are realized by autonomous cooperation of processes having different roles.

  However, since the large-scale distributed processing system has the above-described features, there is a problem that when viewed from the host application, the inside of the system becomes a black box, and it is difficult to analyze when a problem occurs in the system. For example, when throughput (such as the number of read / write processing requests per unit time) decreases on the host application side, there are many suspected locations that cause a decrease in throughput. It is difficult to identify the invited location (bottleneck).

  In order to identify the bottleneck in such a large-scale distributed processing system, it is necessary to take a bird's-eye view of the execution state of the entire system. Generally, the log of the large-scale distributed processing system used for bottleneck identification is in text format. In addition, since the amount is enormous, it is difficult to provide an overview of the execution state of the entire system by analyzing the text log.

  On the other hand, in the large-scale distributed processing system, data transfer between nodes is expressed in a three-dimensional space, processing bias is visualized, and network traffic is visualized in real time. Techniques for incident analysis are known.

Norihiro Kawasaki and two others, "Study of a visualization tool that simultaneously displays communication and processing status of parallel programs in a large-scale distributed system," IEICE Tech. Vol.110, no. 448, NS2010-257, pp. 527- 532 Koji Nakao and 8 others, “Visualization technology of incident analysis center nicter”, Information Processing Society of Japan Vol.2006 No.81, 2006-CSCE-034, pp313-319

  However, with the above-described prior art, there are cases where a bottleneck in a large-scale distributed processing system cannot be easily identified. Specifically, in the above-described conventional technology, data movement between nodes is only visualized, and packet movement in the network is only visualized, so that it is not possible to distinguish between processes. That is, in the above-described conventional technology, it is not possible to confirm the concentration or delay of specific processing for specifying a bottleneck, and it may not be possible to easily specify a bottleneck in a large-scale distributed processing system. . Further, in the above-described conventional technology, detailed information on the internal operation of the system cannot be acquired, and there are cases where a bottleneck in a large-scale distributed processing system cannot be easily identified.

  Therefore, the technology according to the present application has been made in view of the above-described problems of the prior art, and a visualization device, a visualization method, and a visualization program that can easily identify a bottleneck in a large-scale distributed processing system. The purpose is to provide.

  In order to solve the above-described problems and achieve the object, the visualization apparatus according to the present application includes an extraction unit that extracts a log indicating communication between processes among logs output in a large-scale distributed processing system, and the extraction A transmission source process and a transmission destination process, and a transmission source server and a transmission destination server on which the transmission source process and the transmission destination process are executed, respectively. And a log information generation unit that generates log information indicating the content of the processing request and the time when the communication occurred, and the log information generated by the log information generation unit, The transmission / reception of the processing request from the transmission source process to the transmission destination process of the transmission destination server is depicted so as to indicate the content of the processing request and the time. A drawing data generator for generating drawing data, characterized by comprising a display control unit for displaying the drawing data generated by the drawing data generation unit at a predetermined display unit.

  The visualization device according to the present application makes it possible to easily identify a bottleneck in a large-scale distributed processing system.

FIG. 1 is a diagram illustrating an example of the configuration of the visualization device according to the first embodiment. FIG. 2 is a diagram illustrating an example of keywords stored in the keyword table according to the first embodiment. FIG. 3 is a diagram illustrating an example of correspondence information stored by the host name correspondence table according to the first embodiment. FIG. 4A is a diagram illustrating an example of information stored in the design information table according to the first embodiment. FIG. 4B is a diagram illustrating an example of a sequence diagram used for generating information stored in the design information table according to the first embodiment. FIG. 5 is a diagram illustrating an example of log information stored by the log table according to the first embodiment. FIG. 6 is a diagram illustrating an example of coordinate correspondence information stored by the coordinate correspondence table according to the first embodiment. FIG. 7 is a diagram illustrating an example of color correspondence information stored in the color correspondence table according to the first embodiment. FIG. 8 is a diagram illustrating an example of the size correspondence information stored by the size correspondence table according to the first embodiment. FIG. 9 is a diagram illustrating an example of a log extracted by the log extraction unit according to the first embodiment. FIG. 10 is a schematic diagram illustrating an example of processing performed by the log structuring unit according to the first embodiment. FIG. 11 is a diagram illustrating an example of communication drawing data generated by the communication drawing data generation unit according to the first embodiment. FIG. 12 is a diagram illustrating an example of process drawing data generated by the process drawing data generation unit according to the first embodiment. FIG. 13 is a diagram illustrating an example of a drawing image obtained by imaging process drawing data displayed under the control of the drawing control unit according to the first embodiment. FIG. 14 is a diagram illustrating an example of a drawing image obtained by imaging communication drawing data displayed under the control of the drawing control unit according to the first embodiment. FIG. 15 is a diagram for explaining a display example of a drawn image controlled by the drawing control unit according to the first embodiment. FIG. 16 is a flowchart of a process procedure performed by the visualization apparatus according to the first embodiment. FIG. 17 is a flowchart of a drawing process performed by the drawing control unit according to the first embodiment. FIG. 18 is a flowchart for explaining a modification. FIG. 19 is a diagram illustrating the computer that executes the visualization program according to the second embodiment.

  Exemplary embodiments of a visualization device, a visualization method, and a visualization program according to the present application will be described below in detail with reference to the accompanying drawings. Note that the visualization device, the visualization method, and the visualization program according to the present application are not limited to the following examples.

[Configuration of Visualization Device According to Embodiment 1]
First, the configuration of the visualization device according to the embodiment will be described. FIG. 1 is a diagram illustrating an example of the configuration of the visualization apparatus 100 according to the first embodiment. As shown in FIG. 1, the visualization device 100 is connected to a large-scale distributed processing system 200 and a client device 300 via a network 400. In FIG. 1, only one client device 300 is shown, but actually, a large number of client devices 300 are connected to the visualization device 100 via the network 400.

  The network 400 is a local area network (LAN) or a wide area network (WAN). The large-scale distributed processing system 200 is a system that has a large number of servers and autonomously executes distributed processing on each server. Specifically, the large-scale distributed processing system 200 is a system that distributes and processes each process related to an application that provides various services to a user with a large number of servers.

  As illustrated in FIG. 1, the client device 300 includes a communication control I / F unit 310, an input unit 320, a display unit 330, and a control unit 340. The communication control I / F unit 310 controls communication related to various types of information exchanged between the visualization device 100 and the large-scale distributed processing system 200 and the control unit 340. For example, the communication control I / F unit 310 controls communication related to transmission of a request to the visualization apparatus 100.

  The input unit 320 is, for example, a keyboard or a mouse, and accepts various information input processes by the client. For example, the input unit 320 receives a request input operation by a client. Details of the request will be described later. The display unit 330 is, for example, a display and displays and outputs the processing result to the user. For example, the display unit 330 displays and outputs the drawing data generated by the visualization device 100. The drawing data displayed by the display unit 330 will be described in detail later.

  The control unit 340 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Execute overall control. Although not shown, the client device 300 includes a storage device such as a hard disk or an optical disk, or a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, and the client device 300. Various programs executed by the program, various information, etc. are stored.

  As shown in FIG. 1, the visualization apparatus 100 includes a communication control I / F unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150. In response to a request from the client device 300, the visualization device 100 transmits drawing data that visualizes the processing generated in the large-scale distributed processing system 200 to the client device 300.

  The communication control I / F unit 110 controls communication related to various types of information exchanged between the large-scale distributed processing system 200 and the client device 300 and the control unit 150. For example, the communication control I / F unit 110 controls communication related to process log acquisition for the large-scale distributed processing system 200. Further, for example, the communication control I / F unit 110 controls communication related to a request for drawing data exchanged with the client apparatus 300 and drawing transmission / reception. The communication control I / F unit 110 controls the exchange of various information between the input unit 120 and the display unit 130 and the control unit 150.

  The input unit 120 is, for example, a keyboard or a mouse, and accepts various information input processes by the user. For example, the input unit 120 accepts input processing such as coordinate correspondence information, color correspondence information, and size correspondence information stored in the storage unit 140 described later. The coordinate correspondence information, color correspondence information, and size correspondence information will be described later. The display unit 130 is, for example, a display, and displays and outputs the processing result to the user.

  As shown in FIG. 1, the storage unit 140 includes a keyword table 141, a host name correspondence table 142, a design information table 143, a log table 144, a coordinate correspondence table 145, a color correspondence table 146, and a size correspondence table. 147. The storage unit 140 is, for example, a storage device such as a hard disk or an optical disk, or a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, and stores various programs executed by the visualization device 100. To do.

  The keyword table 141 stores keywords used for log extraction from the large-scale distributed processing system 200 by the control unit 150 described later. Specifically, the keyword table 141 stores keywords for collecting logs indicating request reception and response return from the logs output from the processes of the respective servers included in the large-scale distributed processing system 200. FIG. 2 is a diagram illustrating an example of keywords stored by the keyword table 141 according to the first embodiment.

  For example, as shown in FIG. 2, the keyword table 141 stores “Receive request” and “Send response” as keywords. Here, “Receive request” shown in FIG. 2 is a keyword for extracting a request reception log, and “Send response” is a keyword for extracting a response return log. The keyword is arbitrarily determined by the administrator of the visualization apparatus 100 and stored in advance.

  Returning to FIG. 1, the host name correspondence table 142 stores information for generating a log table by the control unit 150 described later. Specifically, the host name correspondence table 142 stores correspondence information indicating the correspondence between the host name and the IP address in each server included in the large-scale distributed processing system 200. The log table will be described in detail later. FIG. 3 is a diagram illustrating an example of correspondence information stored by the host name correspondence table 142 according to the first embodiment.

  For example, the host name correspondence table 142 stores correspondence information in which “host” is associated with “ip” as shown in FIG. Here, “host” shown in FIG. 3 indicates the host name of the server included in the large-scale distributed processing system 200, and “ip” indicates the IP address of the server. For example, as shown in FIG. 3, the host name correspondence table 142 stores correspondence information in which “ip: 192.168.116.12” is associated with “host: hostA”. In other words, the information described above indicates that the IP address of the server “hostA” is “192.168.116.12.” Similarly, the host name correspondence table 142 stores correspondence information in which host names and IP addresses are associated with each other for all servers included in the large-scale distributed processing system 200. The correspondence information is stored in advance by the administrator of the visualization apparatus 100.

  Returning to FIG. 1, the design information table 143 stores information for generating a log table by the control unit 150 described later. Specifically, the design information table 143 stores communication information between processes generated based on the design / implementation information determined at the stage when the large-scale distributed processing system 200 is operated. Here, the design / implementation information is the type of communication (internal protocol) transmitted / received by each process included in the large-scale distributed processing system 200, the sequence diagram in the design document, and the location where the log output is specified in the program. is there.

  FIG. 4A is a diagram illustrating an example of information stored in the design information table 143 according to the first embodiment. For example, as illustrated in FIG. 4A, the design information table 143 stores information in which “process From” and “process To” are associated with “request”. Here, “Request” shown in FIG. 4A indicates a request from one process to another process. In addition, “process From” illustrated in FIG. 4A indicates a process for transmitting a request, and “process To” indicates a process for receiving a request. For example, as illustrated in FIG. 4A, the design information table 143 stores information in which “process: processB, process To: processA” is associated with “request: command = OPEN”.

  Here, the information described above indicates that “request” of “command = OPEN” is transmitted from “processB” to “processA”. Note that “command = OPEN” means, for example, processing for opening a file to be written at the time of writing. Similarly, the design information table 143 stores information in which processes are associated with other requests. The information stored in the design information table 143 is stored in advance by the administrator of the visualization apparatus 100.

  Here, an example of generation of information stored in the design information table 143 will be described. As described above, the information stored in the design information table 143 includes the type of communication (internal protocol) transmitted and received by each process included in the large-scale distributed processing system 200, the sequence diagram in the design document, and the log output in the program. It is generated by the administrator based on the location that defines FIG. 4B is a diagram illustrating an example of a sequence diagram used for generating information stored in the design information table according to the first embodiment.

  For example, the design document includes a sequence diagram showing a flow of requests transmitted and received between processes as shown in FIG. 4B. The administrator acquires information that a request (command = OPEN) is transmitted from processB to processA by referring to a sequence diagram as shown in FIG. 4B. Then, the administrator generates information as shown in FIG. 4A from this information and stores it in the design information table 143.

  Returning to FIG. 1, the log table 144 stores log information generated by the control unit 150 described later. Specifically, the log table 144 stores log information related to request reception and response return output by the processes of each server included in the large-scale distributed processing system 200. FIG. 5 is a diagram illustrating an example of log information stored by the log table 144 according to the first embodiment.

  For example, as illustrated in FIG. 5, the log table 144 includes “time”, “ip”, “process” of “From (transmission side information)”, “time”, “time” of “To (reception side information)”. Log information in which “ip”, “process”, “command”, “size”, and the like are associated with each other is stored. Here, “time”, “ip”, and “process” of “From (transmission side information)” indicate a server, a process, and a transmission time at which the request is transmitted. In addition, “time”, “ip”, and “process” of “To (receiving side information)” indicate a server, a process, and a reception time at which the request is received. Further, “command” shown in FIG. 5 means a command included in the transmitted request. Also, “size” shown in FIG. 5 means the size of data in the case of a request for data exchange.

  As an example, as shown in FIG. 5, the log table 144 includes “ip: 192.168.116.10, process: processB” of “From (transmission side information)” and “To (reception side information)”. "Time: 2011-09-11 11: 36: 46.080, ip: 192.168.116.12, process: processA" and "command: OPEN" are stored. That is, the above-described information includes a request of “command = OPEN” transmitted by “processB” of the server whose IP address is “192.168.116.10”, and an IP address of “192.168.116.12”. "ProcessA" of the server at the time "2011-09-11 11: 36: 46.080". The log table 144 similarly stores other log information generated by the control unit 150 described later.

  Returning to FIG. 1, the coordinate correspondence table 145 stores coordinate information used when drawing data is generated by the control unit 150 described later. Specifically, the coordinate correspondence table 145 stores coordinates indicating positions on the two-dimensional coordinates of each process when drawing each process included in the large-scale distributed processing system 200 on the drawing data. More specifically, the coordinate correspondence table 145 is determined so that different processes on the same server are arranged at positions where the x coordinate is equal, and processes of the same role on different servers are arranged at positions where the y coordinate is equal. Coordinate correspondence information is stored. FIG. 6 is a diagram illustrating an example of coordinate correspondence information stored by the coordinate correspondence table 145 according to the first embodiment.

  For example, as shown in FIG. 6, the coordinate correspondence table 145 stores coordinate correspondence information in which “server”, “process”, and “coordinates” are associated with each other. Here, as an example, the coordinate correspondence table 145 includes, as shown in FIG. 6, “server: 192.168.116.10”, “process: processA”, and “coordinates: (10, 10)”. Is stored. That is, the above-described information means that “processA” of the server whose IP address is “192.168.116.10” is drawn at the coordinates “(10, 10)” when generating drawing data. The coordinate correspondence table 145 similarly stores other coordinate correspondence information. The coordinate correspondence information is arbitrarily determined by the administrator of the visualization apparatus 100 and stored in advance.

  Returning to FIG. 1, the color correspondence table 146 is color correspondence information for determining a color when rendering communication drawing representing communication (reception of request, return of response) transmitted and received between processes on drawing data. Remember. Specifically, the color correspondence table 146 stores information on the color assigned for each process. FIG. 7 is a diagram illustrating an example of the color correspondence information stored by the color correspondence table 146 according to the first embodiment.

  For example, the color correspondence table 146 stores color correspondence information in which “command” and “color” are associated with each other as illustrated in FIG. For example, the color correspondence table 146 stores color correspondence information in which “command: OPEN” and “color: # FFAE35” are associated with each other, as shown in FIG. In other words, the above-described information means that “command: OPEN” is expressed by the color “# FFAE35” when drawing the drawing data. The color correspondence table 146 similarly stores other processes. Note that the color correspondence information is arbitrarily determined and stored by the administrator of the visualization apparatus 100.

  Returning to FIG. 1, the size correspondence table 147 is a size correspondence for determining the size at which communication drawing expressing communication (reception of request, return of response) transmitted and received between processes is drawn in drawing data. Store information. Specifically, the size correspondence table 147 stores size correspondence information in which the size of communication drawing is associated with each data size when the request includes data exchange.

  FIG. 8 is a diagram illustrating an example of the size correspondence information stored in the size correspondence table 147 according to the first embodiment. For example, as illustrated in FIG. 8, the size correspondence table 147 stores size correspondence information in which “data size” and “pixel” are associated with each other. For example, the size correspondence table 147 stores size correspondence information in which “data size: 1K> S” and “pixel: 4” are associated with each other, as shown in FIG. That is, the above-described information indicates that communication drawing is expressed by “4 pixels” when the data size (S in the figure) is less than “1K” when data transmission / reception is included in the request. means. Similarly, the size correspondence table 147 stores size correspondence information in which pixels are associated with data sizes. The size correspondence information is arbitrarily determined by the administrator of the visualization apparatus 100 and stored in advance.

  Returning to FIG. 1, the control unit 150 includes a log extraction unit 151, a log structuring unit 152, a communication drawing data generation unit 153, a process drawing data generation unit 154, and a drawing control unit 155. The control unit 150 is, for example, an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). Execute overall control.

  The log extraction unit 151 extracts a log indicating communication between processes from the logs output in the large-scale distributed processing system 200. Specifically, the log extraction unit 151 extracts a log indicating request reception and response return. More specifically, the log extraction unit 151 performs request reception and response return based on the keywords stored in the keyword table 141 for all processes of all servers included in the large-scale distributed processing system 200. Get all the logs shown.

  FIG. 9 is a diagram illustrating an example of a log extracted by the log extraction unit 151 according to the first embodiment. For example, the log extraction unit 151 reads the keywords “receive request” and “send response” stored in the keyword table 141, and uses the read keywords for all the logs acquired from the large-scale distributed processing system 200. By executing a keyword search, a log as shown in FIG. 9 is extracted. That is, as illustrated in FIG. 9, the log extraction unit 151 extracts a log indicating request reception including “Receive request” and a log indicating response return including “Send response”.

  Returning to FIG. 1, the log structuring unit 152 determines, from the information included in the log extracted by the log extraction unit 151, the transmission source process and the transmission destination process in which processing requests are transmitted and received in the communication between processes, the transmission source Log information indicating the source server and destination server on which the process and the destination process are respectively executed, the contents of the processing request, and the time when the communication occurred is generated. Specifically, the log structuring unit 152 generates log information from the information included in the request reception and response return logs extracted by the log extraction unit 151 and stores the log information in the log table 144.

  FIG. 10 is a schematic diagram illustrating an example of processing performed by the log structuring unit 152 according to the first embodiment. Here, in FIG. 10, a request reception log “2011-09-11 11: 36: 46.080 DEBUG processA: 040205003003004 MSG_DBG_000 hostA 11336 Receive receive (TID = 1466353339, Command = OPEN, Param = 0 [0] ], From = 192.168.116.10) ”.

  For example, as illustrated in FIG. 10, the log structuring unit 152 indicates that the log for which log information is generated is a log indicating request reception, and therefore, “time (time side information)” “time” of the log table. ", The time“ 2011-09-11 11: 36: 46.080 ”included in the log is stored. If the log information generation target log is a log indicating response return, the log structuring unit 152 sets the time information included in the log to “From (transmission side information)” in the log table. Store.

  Then, as illustrated in FIG. 10, the log structuring unit 152 extracts “processA” that is the process that received the request, that is, the process that received the request, and “command = OPEN” that is the processing content from the log. And store in the log table respectively. Here, in the log indicating the request reception shown in FIG. 10, since the information of the server on which “processA” is executed is described by the host name “hostA”, the log structuring unit 152 stores the host name correspondence table. With reference to the correspondence information stored in 142, the IP address “192.168.116.12” of “hostA” is acquired. Then, the log structuring unit 152 stores the acquired IP address “192.168.116.12” in the log table.

  Further, as illustrated in FIG. 10, the log structuring unit 152 extracts the server information “From = 192.168.116.10” of the request transmission source, and obtains “From (transmission side information)”. In “ip”. Here, since the transmission source process name is not described in the request reception log shown in FIG. 10, the log structuring unit 152 refers to the information stored in the design information table 143 and reads “From (transmission). “Process” of “side information)” is stored. In other words, the log structuring unit 152 refers to the information illustrated in FIG. 4, determines that the process of transmitting “command = OPEN” to “processA” is “processB”, and sets “processB” to “processB”. Store in log table.

  As described above, the log structuring unit 152, when the information of the source or destination process is not included in the log, the source or destination process from the information generated based on the design information or the implementation information. To decide. Here, when one process of the transmission source or the transmission destination cannot be specified, the log structuring unit 152 excludes the log from the target of log information generation. As described above, the log structuring unit 152 generates log information from all the logs (request reception log and response return log) extracted by the log extraction unit 151 and stores the log information in the log table 144.

  Returning to FIG. 1, the communication drawing data generation unit 153 transmits and receives a processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server based on the log information generated by the log structuring unit 152. Is generated to indicate the content and time of the processing request. Specifically, in the communication drawing data generation unit 153, processes executed by the same server are arranged at the same coordinates on one axis, and processes having the same function are arranged at the same coordinates on the other axis. In two-dimensional coordinates, transmission / reception of processing requests is expressed by movement of a sphere between processes on the two-dimensional coordinates, and communication drawing data expressing the contents of processing requests by the color of the sphere is generated. Furthermore, when the log information generated by the log structuring unit 152 includes a data size, the communication drawing data generation unit 153 generates communication drawing data that represents the size of the data by the size of a sphere. .

  Here, the communication drawing data generation unit 153 generates the above-described communication drawing data when receiving a drawing data request that visualizes communication between processes from the client device 300. That is, the visualization device 100 generates drawing data in response to a request from a user who operates the client device 300. The request transmitted from the client device 300 includes a drawing condition that the user desires to draw (such as a time zone (start time and end time) in which communication has occurred, a server, a process, and a processing type). Each condition is selected by the user at the time of transmission.

  The communication drawing data generation unit 153 extracts log information suitable for each drawing condition included in the request from the log table 144. Then, the communication drawing data generation unit 153 generates communication drawing data from the extracted log information with reference to the information stored by the coordinate correspondence table 145, the color correspondence table 146, and the size correspondence table 147, respectively.

  FIG. 11 is a diagram illustrating an example of communication drawing data generated by the communication drawing data generation unit 153 according to the first embodiment. For example, the communication drawing data generation unit 153 performs communication drawing data (100001, (10, 10), (10, 20), # FF44EE as shown in FIG. 11 based on the log information stored in the log table 144. , 10). The communication drawing data described above indicates (time, start point coordinates, end point coordinates, sphere color, sphere size: number of pixels in radius).

  Here, the details of the communication drawing data generation will be described using the communication drawing data described above as an example. The communication drawing data generation unit 153 sets the time included in the log information extracted based on the request received from the client device 300 to the unix time. Convert to “100001”. Then, the communication drawing data generation unit 153 refers to the coordinate correspondence information stored in the coordinate correspondence table 145 and determines the start point coordinates “10, 10” from the transmission source process included in the log information. Also, the communication drawing data generation unit 153 refers to the coordinate correspondence information stored in the coordinate correspondence table 145 and determines the end point coordinates “10, 20” from the transmission destination process included in the log information. Further, the communication drawing data generation unit 153 refers to the color correspondence information stored in the color correspondence table 146 and determines the sphere color “# FF44EE” indicating the processing content included in the request. Then, the communication drawing data generation unit 153 refers to the size correspondence information stored in the size correspondence table 147 and determines the size “10” of the sphere.

  That is, the communication drawing data (100001, (10, 10), (10, 20), # FF44EE, 10) generated as described above has a color of “# FF44EE” and a size of “10” pixels. Is a data that moves from “10, 10” on two-dimensional coordinates to “10, 20” at time “100001”. Similarly, the communication drawing data generation unit 153 generates communication drawing data related to all the extracted log information. If the extracted log information does not include data size information, the communication drawing data generation unit 153 generates communication drawing data whose sphere has a predetermined size (for example, 3 pixels). Generate.

  Returning to FIG. 1, in the process drawing data generation unit 154, processes executed by the same server are arranged at the same coordinates on one axis on the two-dimensional coordinates, and the functions are the same at the same coordinates on the other axis. Process drawing data in which a certain process is arranged is generated. Specifically, the process drawing data generation unit 154 extracts a process included in the request received from the client device 300. Then, the process drawing data generation unit 154 refers to the coordinate correspondence information stored in the coordinate correspondence table 145 and determines the coordinates of the extracted process on the two-dimensional coordinates.

  FIG. 12 is a diagram illustrating an example of process drawing data generated by the process drawing data generation unit 154 according to the first embodiment. For example, the process drawing data generation unit 154 generates process drawing data “processA, ((10, 10), (12, 10), (14, 10))” as shown in FIG. That is, the process drawing data generated as described above is data in which three “processA” are arranged at “10, 10”, “12, 10”, and “14, 10”, respectively, on a two-dimensional coordinate. It means that there is. Similarly, the process drawing data generation unit 154 generates process drawing data for all processes included in the request received from the client device 300.

  Returning to FIG. 1, the drawing control unit 155 displays the communication drawing data and the process drawing data generated by the communication drawing data generation unit 153 and the process drawing data generation unit 154 on a predetermined display unit, respectively. Specifically, the drawing control unit 155 transmits communication drawing data and process drawing data to the client apparatus 300 that has transmitted a drawing request indicating communication between processes, and a drawing image obtained by imaging each drawing data. Is displayed on the browser displayed on the display unit 330 of the client device 300.

  FIG. 13 is a diagram illustrating an example of a drawing image obtained by imaging the process drawing data displayed under the control of the drawing control unit 155 according to the first embodiment. For example, as illustrated in FIG. 13, the drawing control unit 155 displays a drawing image in which a process is indicated by a rectangle on a screen where the horizontal direction of the display unit 330 is the x axis and the vertical direction is the y axis. Here, since the process drawing data is generated based on the coordinate correspondence information, the drawing image displayed by the drawing control unit 155 is a process executed on the same server as shown in FIG. Processes that are arranged on the same x-coordinate and have the same role are arranged on the same y-coordinate.

  In the drawing image displayed by the drawing control unit 155, for example, as shown in FIG. 13, a higher-level process that executes reading and writing is arranged at the top of the screen, and a distributed lock is performed from the top to the bottom of the screen. The distributed table (master), the distributed table (worker), the distributed file system (master), and the distributed file system (worker) are arranged in this order. Further, in the drawing image displayed by the drawing control unit 155, as shown in FIG. 13, the same process is expressed by the same color, and the different processes are expressed by different colors. A time axis indicating the passage of time is displayed at the bottom of the screen, and the passage of time is expressed by scrolling a square in the positive direction of the x axis on the time axis.

  Note that the distributed lock shown in FIG. 13 controls the sharing of resources between servers that perform processing in a distributed manner, exclusive access to resources, and control for maintaining system availability and consistency. It is a functional part to do. The distributed table is a functional unit that manages a large amount of data as structured data on a plurality of servers. The distributed file system is a functional unit that holds a large amount of data. The master in the distributed table and the distributed file system is an element that executes overall control in each functional unit, and the worker is an element that actually executes processing in each functional unit.

  FIG. 14 is a diagram illustrating an example of a drawing image obtained by imaging communication drawing data displayed under the control of the drawing control unit 155 according to the first embodiment. Here, FIG. 14 shows an example of a drawn image when a processing request is transmitted from a higher-level process that executes reading and writing to the distribution table (worker). For example, as illustrated in FIG. 14, the drawing control unit 155 uses a sphere that moves from a communication source process (start point coordinates in communication drawing data) to a communication destination process (end point coordinates in communication drawing data). A drawing image representing the communication is displayed. Here, the sphere displayed by the drawing control unit 155 is expressed in a different color for each processing content. Further, the sphere displayed by the drawing control unit 155 is expressed in different sizes depending on the data size. In FIG. 14, the trajectory of the sphere is shown, but the trajectory of the sphere is not actually displayed.

  Furthermore, as shown in FIG. 14, the drawing control unit 155 expresses the number of processing requests waiting to be processed at each time point by stacking a cylinder on the upper part of the rectangle indicating the process, as shown in FIG. Is possible. In such a case, in the process having the queue, the drawing control unit 155 calculates the number obtained by subtracting the number of responses returned from the number of received requests as the number of processing requests waiting to be processed, and the calculated number of cylinders is calculated. Stack on top of the process.

  The drawing speed can be arbitrarily selected by the user. For example, when the time from the start time to the end time of the time zone in which the communication to be rendered in the request from the user occurs is 10 minutes, in addition to displaying the rendered image over 10 minutes, It is possible to display in 20 minutes. The selection of the drawing speed is executed in response to a request from the user after the client device 300 receives the drawing data.

  In such a case, first, the user inputs a desired drawing speed to the client device 300. The drawing control unit 155 receives the drawing speed input from the client device 300, and causes the display unit 330 of the client device 300 to display a drawing image with the drawing time advanced by a drawing interval corresponding to the received drawing speed. For example, when the drawing interval is “T” and the input drawing speed is “V”, the drawing control unit 155 displays a drawing image at a drawing interval given by “T = C / V”. “C” is a constant such as “1”.

  FIG. 15 is a diagram for explaining a display example of a drawn image controlled by the drawing control unit 155 according to the first embodiment. For example, as shown in FIG. 15, the drawing control unit 155 equally divides a parabola with endpoints of the coordinates of the transmission source process and the coordinates of the transmission destination process by a constant (for example, equally divided into 12), and draws only the drawing interval. As the time advances, a drawing image for moving the sphere toward the coordinates of the destination process is displayed one interval at a time. Note that the movement of the sphere is not limited to being on a parabola, and may be, for example, on a straight line having the coordinates of the transmission source process and the coordinates of the transmission destination process as endpoints. In such a case, the straight line is equally divided by a constant.

  As described above, the drawing control unit 155 expresses communication between processes on the display unit 330 of the client device 300 by moving a sphere whose color is changed for each process and the size is changed according to the data size. Display a drawn image. That is, the user of the client device 300 can confirm at a glance what processing is occurring in which process of which server. Furthermore, by displaying the number of requests waiting to be processed, it is possible to check at a glance the situation where the requests are not sequentially processed and accumulated in the stack. As a result, the user can recognize the place where the problem appears to occur, and can easily identify the bottleneck in the large-scale distributed processing system.

[Processing Procedure by Visualization Device According to Embodiment 1]
Next, a processing procedure performed by the visualization apparatus 100 according to the first embodiment will be described with reference to FIG. FIG. 16 is a flowchart of a process procedure performed by the visualization apparatus 100 according to the first embodiment. As illustrated in FIG. 16, in the visualization apparatus 100 according to the first embodiment, the log extraction unit 151 extracts a request reception and response return log output by a process included in the large-scale distributed processing system 200 (Step S1). S101).

  Then, for each extracted log, the log structuring unit 152 generates log information indicating the transmission source server, the transmission source process, the transmission destination server, the transmission destination process, the processing content, and the time of the processing request, and the log table 144 (step S102). Thereafter, the communication drawing data generation unit 153 and the process drawing data generation unit 154 determine whether a request has been received from the client device 300 (step S103).

  When a request is received from the client device 300 (Yes at Step S103), the communication drawing data generation unit 153 and the process drawing data generation unit 154 generate communication drawing data and process drawing data, respectively (Step S104). . On the other hand, when the request is not received from the client device 300, the communication drawing data generation unit 153 and the process drawing data generation unit 154 enter a standby state (No at Step S103).

  When the communication drawing data and the process drawing data are respectively generated in step S104, the drawing control unit 155 determines whether or not the drawing speed is received from the client device 300 (step S105). Here, when the drawing speed is received from the client device 300 (Yes in step S105), the drawing control unit 155 displays a drawn image drawn at a drawing interval corresponding to the received drawing speed on the display unit 330 of the client device 300. It is displayed (step S106). On the other hand, when the drawing speed is not received from the client apparatus 300 (No at Step S105), the drawing control unit 155 enters a standby state.

[Procedure for Drawing Process by Drawing Controller according to Embodiment 1]
Next, a drawing process procedure performed by the drawing control unit 155 according to the first embodiment will be described. FIG. 17 is a flowchart of a drawing process performed by the drawing control unit 155 according to the first embodiment. Here, the flowchart shown in FIG. 17 corresponds to the processing in step S106 in FIG. In FIG. 17, “Ts” represents the drawing start time, “Te” represents the drawing end time, “T” represents the drawing time, and “Tb” represents the drawing interval. In FIG. 17, there are N pieces of communication drawing data, each communication drawing data is indicated as dn, and time information of dn is tn (n = 0... N−1). That is, each communication drawing data “dn” is “time information tn, (start point coordinates), (end point coordinates), sphere color, sphere size”. Note that the time information in FIG. 17 is the time after being converted to the unix time.

  As illustrated in FIG. 17, when the time is changed from the start time “Ts” to the drawing time “T” (step S <b> 201), the drawing control unit 155 determines the communication drawing data “dn” from “n = 0” to “N−1”. ”, Whether or not there is communication drawing data whose time information“ tn ”is greater than or equal to the drawing time“ T ”and less than“ T + Tb ”obtained by adding the drawing interval“ Tb ”to the drawing time“ T ”. Is determined (step S202).

  Here, if the time information “tn” is the drawing time “T” or more and there is communication drawing data less than “T + Tb” obtained by adding the drawing interval “Tb” to the drawing time “T” (step) In S202, the drawing control unit 155 starts drawing the communication drawing data “dn” (Step S203). On the other hand, when the time information “tn” is not less than the drawing time “T” and there is no communication drawing data less than “T + Tb” obtained by adding the drawing interval “Tb” to the drawing time “T” (No in step S202). ), The drawing control unit 155 does not perform drawing.

  Then, the drawing control unit 155 sets the time obtained by adding the drawing interval “Tb” to the current drawing time “T” as a new drawing time “T” (step S204), and the new drawing time “T” is set as the end time “T”. It is determined whether or not “Te” has been exceeded (step S205). If the new drawing time “T” does not exceed the end time “Te” (No at Step S205), the drawing control unit 155 returns to Step S202 to make a determination. On the other hand, when the new drawing time “T” exceeds the end time “Te” (Yes at Step S205), the drawing control unit 155 ends the drawing process.

[Effect of Example 1]
As described above, according to the first embodiment, the log extraction unit 151 extracts a log indicating communication between processes from logs output in the large-scale distributed processing system. The log structuring unit 152 executes each of a transmission source process and a transmission destination process, and a transmission source process and a transmission destination process that have transmitted and received a processing request in the communication from information included in the log extracted by the log extraction unit 151. Log information indicating the transmission source server and the transmission destination server, the content of the processing request, and the time when the communication occurred is generated. Based on the log information generated by the log structuring unit 152, the communication drawing data generation unit 153 and the process drawing data generation unit 154 receive a processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server. Drawing data is generated in which transmission / reception is drawn to indicate the content and time of a processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server. The drawing control unit 155 causes the display unit 330 of the client device 300 to display the drawing data generated by the communication drawing data generation unit 153 and the process drawing data generation unit 154. Therefore, the visualization apparatus 100 according to the first embodiment can visualize processing concentration in a specific process, delay of the specific process, and the like, and can easily identify a bottleneck in a large-scale distributed processing system. .

  Further, according to the first embodiment, the log extraction unit 151 extracts a log indicating request reception and a log indicating response return as a log indicating communication between processes. Therefore, the visualization apparatus 100 according to the first embodiment makes it possible to reliably extract processing that has occurred in the large-scale distributed processing system.

  Further, according to the first embodiment, the log structuring unit 152 can determine whether the source process or the above-described process is performed from the design information or the implementation information of the large-scale distributed processing system when the source process or destination process is not included in the log Get the destination process. Therefore, the visualization apparatus 100 according to the first embodiment can make more logs (request reception and response return logs) to be rendered, and can identify bottlenecks with higher accuracy. To do.

  Further, according to the first embodiment, the communication drawing data generation unit 153 and the process drawing data generation unit 154 arrange processes executed by the same server at the same coordinates on one axis in the two-dimensional coordinates, Processes having the same function are arranged at the same coordinates on the axis, transmission / reception of the processing request is expressed by movement of a sphere between processes on the two-dimensional coordinate, and the content of the processing request is expressed by the color of the sphere The drawing data expressed by is generated. Therefore, the visualization apparatus 100 according to the first embodiment can visually understand the exchange of data executed in the large-scale distributed processing system, and makes it easy to understand the system execution state.

  Further, for example, as described above, in the flow of processing across multiple processes, a sphere indicating data movement flows from the upper part of the screen to the lower part for the writing process, and data from the lower part to the upper part of the screen for the reading process. Since drawing is performed so that a sphere indicating movement flows, it is easy to visually understand the exchange in the large-scale distributed processing system, and it is possible to more easily understand the system execution state.

  Further, according to the first embodiment, the communication drawing data generation unit 153 expresses the data size by the size of the sphere when the log information generated by the log structuring unit 152 includes the data size. Generated drawing data. Therefore, the visualization apparatus 100 according to the first embodiment can store a large amount of information in one sphere, and enables a better understanding of the internal state of the system.

  Although the first embodiment has been described so far, the embodiment according to the present application is not limited to the first embodiment. That is, these embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made.

  In the first embodiment described above, the case where the correspondence between the data size and the size of the sphere and the correspondence between the processing type and the color are stored in the table has been described. However, the embodiment is not limited to this, and for example, it may be embedded in the program as a flow (a color or size value is hard-coded). Hereinafter, an example of a flow embedded in the program will be described with reference to FIG.

  FIG. 18 is a flowchart for explaining a modification. Here, FIG. 18 shows a flowchart when the correspondence between the data size and the size of the sphere is set in the flow. For example, as shown in FIG. 18, the communication drawing data generation unit 153 first determines whether or not the log information to be drawn includes a data size (step S301). (No in step S301), the size (radius) of the sphere is set to 3 pixels (step S302).

  If the data size is included in the log information to be drawn (Yes at Step S301), the communication drawing data generation unit 153 next determines whether or not the data size is 1K or more (Step S303). ) When the data size is less than 1K (No at Step S303), the size (radius) of the sphere is set to 4 pixels (Step S304). As described above, the communication drawing data generation unit 153 performs the data size determination and the sphere size setting step by step from step S305 to step S311. Thereby, the visualization apparatus 100 can generate communication drawing data in which the size of the sphere is changed according to the data size without providing a table indicating the correspondence between the data size and the size of the sphere. The correspondence between the processing type and the color can also be embedded in the program as a flow.

  In the first embodiment described above, the case where communication between processes is expressed by movement of a sphere has been described. However, the embodiment is not limited to this, and for example, another figure may be used. For example, a polygon, an ellipse, a block arrow, or the like may be used. Similarly, in the above-described first embodiment, the case where the process is expressed by a rectangle has been described. However, the embodiment is not limited to this, and other graphics may be used.

  Further, for example, the specific form of distribution / integration of each device (for example, the form shown in FIG. 1) is not limited to the one shown in the figure, and all or a part thereof can be arbitrarily set according to various loads or usage conditions. It can be distributed or integrated functionally or physically in units. As an example, the communication drawing data generation unit 153 and the process drawing data generation unit 154 may be integrated as one drawing data generation unit, while the log extraction unit 151 receives logs from the large-scale distributed processing system 200. You may distribute to the collection part which collects, and the extraction part which extracts the log of a request reception and a response return from the collected log.

  Alternatively, the control unit 150 may be connected as an external device of the visualization apparatus 100 via a network, or another apparatus may include a communication drawing data generation unit 153 and a process drawing data generation unit 154, respectively. The functions of the visualization device 100 described above may be realized by being connected to and cooperating with each other.

  The visualization apparatus 100 described in the above embodiment can also be realized by executing a program prepared in advance on a computer. Therefore, an example of a computer that executes a visualization program that realizes the same function as the visualization apparatus 100 illustrated in FIG. 1 will be described below.

  FIG. 19 is a diagram illustrating the computer 1000 that executes the visualization program according to the second embodiment. As shown in FIG. 19, the computer 1000 includes, for example, a memory 1010, a CPU (Central Processing Unit) 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, a network Interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 19, the hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. The visualization program according to the second embodiment is stored in, for example, the hard disk drive 1090 as a program module in which a command executed by the computer 1000 is described. Specifically, a log extraction step for executing information processing similar to the log extraction unit 151 described in the above embodiment, a log structuring step for executing information processing similar to the log structuring unit 152, and communication drawing data A communication drawing data generation step for executing information processing similar to the generation unit 153, a process drawing data generation step for executing information processing similar to the process drawing data generation unit 154, and an information processing similar to that of the drawing control unit 155 are executed. The program module describing the drawing control step to be stored is stored in the hard disk drive 1090.

  Further, like the data stored in the keyword table 141, the host name correspondence table 142, the design information table 143, the log table 144, the coordinate correspondence table 145, the color correspondence table 146, and the size correspondence table 147 described in the above embodiment, Data used for information processing by the visualization program is stored as, for example, hard disk drive 1090 as program data. Then, the CPU 1020 reads the program module and program data stored in the hard disk drive 1090 to the RAM 1012 as necessary, and performs a log extraction step, a log structuring step, a communication drawing data generation step, and a process drawing data generation step. And a drawing control step.

  Note that the program module and program data related to the visualization program are not limited to being stored in the hard disk drive 1090, but are stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Also good. Alternatively, a program module and program data related to the information transmission / reception program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and the CPU 1020 via the network interface 1070. May be read.

  These embodiments and modifications thereof are included in the invention disclosed in the claims and equivalents thereof as well as included in the technology disclosed in the present application.

DESCRIPTION OF SYMBOLS 100 Visualization device 140 Storage part 141 Keyword table 142 Host name correspondence table 143 Design information table 144 Log table 145 Coordinate correspondence table 146 Color correspondence table 147 Size correspondence table 150 Control part 151 Log extraction part 152 Log structuring part 153 Communication drawing data generation Unit 154 Process drawing data generation unit 155 Drawing control unit 200 Large scale distributed processing system 300 Client device

Claims (8)

An extraction unit that extracts a log indicating communication between processes among logs output in a large-scale distributed processing system;
From information included in the log extracted by the extraction unit, a transmission source process and a transmission destination process in which a processing request is transmitted and received in the communication, a transmission source server on which the transmission source process and the transmission destination process are executed, and A log information generating unit that generates log information indicating a destination server, the content of the processing request, and the time when the communication occurred;
Based on the log information generated by the log information generation unit, transmission / reception of the processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server, the content of the processing request and the A drawing data generation unit for generating drawing data drawn to indicate the time;
A display control unit for displaying the drawing data generated by the drawing data generation unit on a predetermined display unit;
A visualization device characterized by comprising:   The visualization apparatus according to claim 1, wherein the extraction unit extracts a log indicating request reception and a log indicating response return as a log indicating communication between the processes.   When the log does not include the transmission source process or the transmission destination process, the log information generation unit determines the transmission source process or the transmission destination process from design information or implementation information of the large-scale distributed processing system. The visualization device according to claim 1, wherein the visualization device is acquired. The log information generation unit generates log information including the size of the data when data exchange occurs in communication between the processes,
The drawing data generation unit, when the log information generated by the log information generation unit includes the size of the data, from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server The drawing data drawn to indicate the content of the processing request and the size of the data is generated for transmission / reception of the processing request to / from the computer. The visualization device described.   In the two-dimensional coordinate, the drawing data generation unit arranges a process executed by the same server at the same coordinate on one axis, arranges a process having the same function at the same coordinate on the other axis, and The processing request transmission / reception is expressed by movement of a sphere between processes on the two-dimensional coordinates, and drawing data expressing the content of the processing request by the color of the sphere is generated. The visualization device according to any one of 4 above.   The drawing data generation unit generates drawing data expressing the size of the data by the size of the sphere when the size of the data is included in the log information generated by the log information generation unit. The visualization device according to claim 5. A visualization method executed by a visualization device for visualizing movement of data in a large-scale distributed processing system,
An extraction step of extracting a log indicating communication between processes among logs output in the large-scale distributed processing system;
From the information included in the log extracted by the extraction step, a transmission source process and a transmission destination process that have received and transmitted a processing request in the communication, a transmission source server and a transmission in which the transmission source process and the transmission destination process are executed, respectively. A log information generating step for generating log information indicating a destination server, the content of the processing request, and the time when the communication occurred;
Based on the log information generated by the log information generation step, the transmission / reception of the processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server, the content of the processing request and the A drawing data generation step for generating drawing data drawn to indicate time;
A display control step of displaying the drawing data generated by the drawing data generation step on a predetermined display unit;
The visualization method characterized by including. A visualization program executed by a visualization device for visualizing movement of data in a large-scale distributed processing system,
An extraction step for extracting a log indicating communication between processes among logs output in the large-scale distributed processing system;
From the information included in the log extracted by the extraction step, a transmission source process and a transmission destination process that have received and transmitted a processing request in the communication, a transmission source server that executes the transmission source process and the transmission destination process, and A log information generating step for generating log information indicating a destination server, the content of the processing request, and the time when the communication occurred;
Based on the log information generated by the log information generation step, transmission / reception of the processing request from the transmission source process of the transmission source server to the transmission destination process of the transmission destination server, A drawing data generation step for generating drawing data drawn to indicate the time;
A display control step of displaying the drawing data generated by the drawing data generation step on a predetermined display unit;
To be executed by the visualization device.

Images