JP2019087039A - Task management system, task management method, and task management program - Google Patents

Abstract

To enable reduction in a data communication traffic between nodes.SOLUTION: A computer system 10 includes a plurality of task processing nodes 200 capable of executing tasks and a task management node 100 determining that task processing node 200 to which a new task is to be assigned, wherein each of the task processing nodes 200 includes a memory 202 capable of caching data which is used by an assignment task assigned to the local task processing node, the task management node 100 stores task assignment history information 112 including a correspondence relationship between an assignment task and that task processing node 200 to which the assignment task is assigned, and for which the data to be used is cached, and a CPU 101 of the task management node 100 is configured to determine a similarity between a new task and the assignment task, determine that task processing node 200 to which the new task is to be assigned from the task processing nodes 200 included in the task assignment history information 112 based on the similarity, and assign the new task.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a task management system or the like that determines a node to which a task is to be assigned among a plurality of nodes.

  Currently, the market for providing decision support services utilizing AI (Artificial Intelligence) is expanding. In a service that utilizes a natural language processing system AI, in order to handle a large amount of text data, distributed management and distributed processing of data by a plurality of nodes are performed.

  In recent years, for distributed processing of tasks for the purpose of AI processing, a method of distributing large-volume data to a plurality of nodes and managing them has been used. When this method is used, when processing a task, it is necessary to acquire data required by the task from the distributed managed nodes, and as the data required by the task increases, communication between nodes increases. And there is a problem that communication becomes a bottleneck and processing speed is reduced.

  On the other hand, in order to speed up processing, a task is placed on a node that processes a task, which is close to the node where data used most by the task is stored the most, and the method of reducing communication between nodes at the time of data acquisition Is used. For example, Patent Document 1 discloses a technique for determining a node to which a task is to be assigned using an actual distance between nodes.

US Patent Application Publication No. 2014/0372611

  As described above, when data is distributed to and managed by a plurality of nodes and distributed processing is performed by a plurality of nodes, it is required to reduce the amount of data between the nodes.

  For example, in the technique of Patent Document 1, it is necessary to perform processing for investigating data used by a task in order to determine a node that uses most data used by the task. Furthermore, as data to be used increases, it takes time to calculate a node to which a task is assigned. For this reason, communication between nodes for acquiring data occurs frequently, and the performance is degraded.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a technology capable of reducing the amount of data communication between nodes.

  In order to achieve the above object, a task management system according to one aspect is a task management system having a plurality of task processing nodes capable of executing tasks and a task management node for determining a task processing node to which a new task is to be assigned. Each of the plurality of task processing nodes has a memory capable of caching data used by the assignment task which is a task assigned to itself, and the task management node is assigned the assignment task and the assignment task, and is assigned The task management node stores the task assignment information including the correspondence with the task processing node in which the data used by the task is cached, and the processor of the task management node determines the similarity between the new task and the assignment task. Assign a new task from the task processing nodes included in the task assignment information based on the That task processing node determines and assigns a new task to the determined task processing node.

  According to the present invention, it is possible to reduce the amount of data communication between nodes.

FIG. 1 is an overall configuration diagram of a computer system according to the first embodiment. FIG. 2 is a diagram showing the configuration of task assignment history information according to the first embodiment. FIG. 3 is a diagram showing the configuration of cache management information according to the first embodiment. FIG. 4 is a flowchart of task management processing according to the first embodiment. FIG. 5 is a flowchart of task execution processing according to the first embodiment. FIG. 6 is a flowchart of the in-cache reorganization processing in units of tasks according to the first embodiment. FIG. 7 is a flowchart of deletion target task determination processing according to the first embodiment. FIG. 8 is a flowchart of the cache data unit cache internal processing according to the first embodiment. FIG. 9 is an overall configuration diagram of a computer system according to the second embodiment.

  Embodiments will be described with reference to the drawings. The embodiments described below do not limit the invention according to the claims, and all of the elements described in the embodiments and their combinations are essential to the solution means of the invention. There is no limit.

  In the following description, the information according to the embodiment will be described by, for example, expressing it as a data structure using a table, but such information may not necessarily be expressed using a data structure using a table, “list”, “DB ( It may be expressed by a data structure such as “database” or “queue” or the like. Therefore, "table", "list", "DB", "queue" and the like can be simply referred to as "information" in order to indicate that they do not depend on the data structure. In addition, when describing the contents of each information, it is possible to use the expressions "identification information", "identifier", "name", "name", "ID" and "number", which are mutually Substitution is possible.

  In the following description, although the “program” (for example, a program module) is described as the subject of the operation, the program uses a memory and a communication port (communication control device) to execute processing determined by being executed by the processor. However, the description may be based on the processor or may be based on the controller.

  Further, the processing disclosed with the program as the subject of operation may be processing performed by a computer (computer, information processing apparatus) such as a node. Some or all of the programs may be realized by dedicated hardware. Various programs may be installed in each node by a program distribution server or storage medium (for example, non-volatile storage medium).

  First, an overview of a computer system according to the first embodiment will be described.

  The computer system according to the first embodiment searches for similar tasks and assigns task management node 100 (see FIG. 1) and executes a task process, and also manages a plurality of data necessary for the task process. And the task processing node 200 (see FIG. 2).

  The task management node receives the task processing request, analyzes the task content, performs a similarity search with task history information, and executes a process of determining a task processing node to which a task is to be allocated on the condition of similarity between tasks etc. . The task processing node also executes processing for processing assigned tasks and managing cache of data used for the tasks.

  Next, the computer system according to the first embodiment will be described in detail.

  FIG. 1 is an overall configuration diagram of a computer system according to the first embodiment.

  A computer system 10 as an example of a task management system includes a client 50, a task management node 100, and a plurality of task processing nodes 200. The client 50, the task management node 100, and the task processing node 200 are connected via the network 300. The network 300 may be, for example, a wired LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), or a combination thereof.

  The client 50 is configured by, for example, a PC (Personal Computer), and is used by a user who requests execution of a task. The client 50 transmits, for example, a task processing request from a user to the task management node 100, receives a task processing result from the task management node 100, and displays the result on, for example, a display or the like.

  The task management node 100 is configured of, for example, a computer such as a server, and includes a central processing unit (CPU) 101 as an example of a processor, a memory 102, and a network interface 103.

  The network interface 103 is, for example, an interface such as a wired LAN card or a wireless LAN card, and communicates with other devices (for example, the client 50, the task processing node 200, and the like) via the network 300.

  The CPU 101 executes various processes by executing programs (modules) stored in the memory 102.

  The memory 102 is, for example, a RAM (RANDOM ACCESS MEMORY), and stores a program executed by the CPU 101 and necessary information.

  The memory 102 stores a task management module 111 as an example of a program and task allocation history information 112 as an example of task allocation information.

  The task management module 111 includes a task assignment module 131, a task analysis module 132, and a similarity search module 133. The task assignment module 131 is a program module that causes the CPU 101 to execute processing relating to task acceptance and task assignment. The task analysis module 132 is a program module that causes the CPU 101 to execute processing such as analysis of task contents and digitization of tasks. The similarity search module 133 is a program module that causes the CPU 101 to execute a process of comparing the information of the new task quantified and the information of the task that has already been allocated and evaluating the similarity rate. The details of processing by these modules will be described later.

  The task assignment history information 112 is information on a history of task assignment. The details of the task allocation history information 112 will be described later.

  The task processing node 200 is configured by, for example, a computer such as a server, and includes a CPU 201 as an example of a processor, a memory 202, a storage device 203, and a network interface 204.

  The network interface 204 is, for example, an interface such as a wired LAN card or a wireless LAN card, and communicates with other devices (for example, the client 50, the task management node 100, another task processing node 200, etc.) via the network 300. .

  The CPU 201 executes various processes by executing a program (module) stored in the memory 202.

  The memory 202 is, for example, a RAM, and stores a program executed by the CPU 201 and necessary information.

  The memory 202 stores a task processing module 211, a cache management module 212, a data management module 213, and cache management information 214.

  The task processing module 211 is a program module that causes the CPU 201 to process the task received from the task management node 100 and execute the process of notifying the task management node 100 of the result of the task and the completion. The task processing module 211 may cause the CPU 201 to execute natural language processing by AI as task processing. The cache management module 212 is a program module that causes the CPU 201 to manage data cached in the cache 215 when processing a task, and to execute processing (in-cache organization processing) that performs organization when the capacity of the cache 215 is insufficient. It is. The data management module 213 is a program module that causes the CPU 201 to manage the arrangement status of the data 231 in the storage devices 203 of the plurality of task processing nodes 200. By the CPU 210 executing the data management module 213, the data 231 in the storage device 203 can be distributed and managed by the storage devices 203 of one or more task processing nodes 200.

  The cache management information 214 is information on the cache status in the cache 215 and a list of data used by the task. The details of the cache management information 214 will be described later.

  The memory 202 is also an area for storing data (cache data) 221 acquired from the storage device 203 of its own task processing node 200 or another task processing node 200 for use in task processing. Have. The capacity of the cache 215 is set in advance to a predetermined capacity. The cache data 221 is maintained in the cache 215 until it is deleted from the cache 215 by the cache management module 212.

  The storage device 203 is, for example, a hard disk or a flash memory, and stores programs executed by the CPU 201 and data used by the CPU 201. In the present embodiment, the storage device 203 stores data 231 used in processing of tasks.

  Next, the task assignment history information 112 will be described.

  FIG. 2 is a diagram showing the configuration of task assignment history information according to the first embodiment.

  The task assignment history information 112 stores an entry corresponding to each task in which data used in a task is cached in the task processing node 200. The entry of the task assignment history information 112 includes fields of task 301, execution node ID 302, analysis result 303, processing status 304, comparison frequency 305, average similarity rate 306, adoption number 307, and final execution date and time 308.

  The task 301 stores an identifier (task ID) of a task (assignment task) assigned to the task processing node 200. In the present embodiment, the task 301 stores a task ID of a task whose cached state of data used by the task is maintained in the cache 215 of the task processing node 200. The execution node ID 302 stores an identifier (execution node ID) of a node (task processing node 200) to which a task corresponding to the entry is assigned. The analysis result 303 stores the result of analysis of the task (content of task) corresponding to the entry by the task analysis module 133. For example, for a task in the natural language processing system, the analysis result 303 stores the result of digitizing (vectorizing) a sentence or word of task content. The processing status 304 stores information as to whether the task corresponding to the entry is being processed or has been processed. The number of comparisons 305 stores the number of comparisons between the task (new task) newly requested by the similarity search module 132 and the task (allocation task) corresponding to the entry. The average similarity ratio 306 stores the average (average similarity ratio) of the results obtained by the similarity search module 132 comparing the new task with the assignment task (similarity ratio: an example of similarity). In the adoption number 307, the task allocation module 131 uses this entry as a factor to determine the task processing node 200 to which the task is to be allocated, that is, the task processing node 200 corresponding to this entry is determined as the new task allocation destination. The number of times (adopted number) is stored. The final execution date 308 stores the final date (for example, YYMMDD hh: mm: ss) at which the task corresponding to the entry was executed.

  Next, the cache management information 214 will be described.

  FIG. 3 is a diagram showing the configuration of cache management information according to the first embodiment.

  The cache management information 214 holds an entry corresponding to the task processing node 200. The entry of the cache management information 214 includes fields of a node ID 401, a cache capacity 402, a used cache capacity 403, a data deletion threshold 404, a cache reduction scheme 405, a cached task 406, and a cache data list 407.

  The node ID 401 stores a unique identifier (node ID) for identifying the task processing node 200. When the cache management information 214 manages only information related to the task processing node 200 in which the cache management information 214 is stored, the node ID 401 may not be provided. The cache capacity 402 stores the storage capacity of the memory 202 usable as the cache 215 of the task processing node 200 corresponding to the entry. The used cache capacity 403 stores the total capacity of the cache data 221 cached in the cache 215 of the task processing node 200 corresponding to the entry. The data deletion threshold 404 stores a threshold related to the capacity used to determine whether to delete data in the cache 215. The cache organizing method 405 stores information indicating a method for organizing cache data when the cache capacity of the cache 215 is insufficient. The cached task 406 stores an identifier of a task for which data used in the cache 215 of the task processing node 200 corresponding to the entry is cached. Cached tasks 406 may store identifiers of multiple tasks. The cache data list 407 holds a list of identifiers of the cache data 221 cached in the cache 215 among data used or used by the task corresponding to the task identifier of the cached task 406.

  Next, the processing operation in the computer system 10 will be described.

  First, task management processing by the task management node 100 will be described.

  FIG. 4 is a flowchart of task management processing according to the first embodiment.

  The task management process is started when the task management node 100 receives a task execution request from the client 50. In the present embodiment, the case where the task (new task) to be the target of the task execution request is a natural language processing task will be described as an example. The type of task is not limited to the natural language processing task.

The task management module 111 (precisely, the CPU 101 that executes the task management module 111) causes the task analysis module 132 to execute analysis of the new task included in the task processing request received from the client 50, thereby performing the task ( The task analysis module 132 analyzes the task content (step S101) Specifically, the task analysis module 132 converts (digitizes) the task word (text or word) into vector information as a method of converting the word into vector information For example, known techniques such as word2vec can be used.

  Next, the task management module 111 determines whether there is a task processing node 200 (empty node) that is not currently processing a task by referring to the processing status 304 of each entry of the task allocation history information 112 ( Step S102).

  As a result, when there is no empty node (step S102: NO), the task management module 111 uses the similarity search module 133 to calculate the similarity rate with the new task for all tasks having an entry in the task assignment history information 112. After calculation (step S104), the process proceeds to step S106. Here, the similarity search module 133 calculates the similarity between the new task and the assigned task based on the calculated vector information of the new task and the information of the analysis result 303 of the entries of all the tasks in the task assignment history information 112. calculate.

  On the other hand, when there is an empty node (step S102: YES), the task management module 111 causes the similarity search module 133 to calculate the similarity rate with the new task for all empty nodes (step S105), To step S106.

  In step S106, the task management module 111 determines, based on the similarity rate calculated by the similarity search module 133, a task processing node 200 having a task with the highest similarity rate as a node to which a new task is assigned (assignment destination node). Choose as.

  Here, the task having the highest similarity rate to the new task is likely to use the same data as the data used in the new task, and the data is also likely to be cached. Therefore, when task processing node 200 having a task with the highest similarity rate is assigned as a new task assignment node, the possibility of using cached data when executing a new task (ie, cache hit rate) Becomes higher. The possibility that the new task and the task with the highest similarity use the same data depends on the type of task, but for example, if the task is a task related to natural language processing, the possibility of using the same data is higher Tend to be

  Next, the task assignment module 131 requests task processing to the task processing node 200 selected as the assignment destination node, that is, sends a new task (step S107). As a result, in the task processing node 200 of the assignment destination node, the processing of the task is executed, and when the task processing ends, the processing result of the task and the task completion notification are transmitted to the task management node 100. .

  Next, the task management module 111 updates the task allocation history information 112 (step S108). Specifically, the task management module 111 adds an entry corresponding to the new task to the task allocation history information 112, and the added entry The identifier of the new task is set in the task 301 in step S. The identifier of the allocation destination node is set in the execution node 302, the vector information obtained in step S101 is set in the analysis result 303, and the task is executing in the processing status 304 It sets “in process” indicating that there is a certain number, stores the number (the number of similarity ratios calculated) compared with the allocation task in the comparison frequency 305, stores the average similarity ratio with the allocation task compared with the average similarity ratio 306 The accepted number 307 stores 0. In addition, the task management module 111 adds 1 to the number of comparisons of the number of comparisons 305 of the entry corresponding to the compared assignment task, and calculates the number of comparisons and the average similarity before the update in the average similarity rate 306. The latest average similarity rate calculated based on the calculated similarity rate is stored. Further, the task management module 111 adds 1 to the adopted number 307 of the entry corresponding to the task processing node 200 selected as the assignment destination node.

  Thereafter, the task management module 111 receives the task completion notification and the task processing result from the task processing node 200 (S109), and the processing status 304 of the entry of the task assignment history information 112 corresponding to the task notified of the task completion. It is updated to “processed” (S110), and the task processing result is notified to the client 50 that is the request source of the task processing request (S111).

  Next, task execution processing by the task processing node 200 will be described.

  FIG. 5 is a flowchart of task execution processing according to the first embodiment.

  The task execution process is started when the task processing node 200 receives a new task from the task management node 100.

  When the task execution process is started, the task processing module 211 of the task processing node 200 analyzes the task and investigates data (acquisition target data) necessary for task processing (S201).

  Next, the task processing module 211 repeatedly executes the processing of loop A (steps S203 to S208) with each of the acquisition target data as a processing target.

  In loop A, the task processing module 211 inquires the cache management module 212 whether or not the acquisition target data to be processed (data to be processed) exists in the cache 215 (step S203). As a result, when it exists in the cache 215 (step S203: YES), the task processing module 211 acquires cache data 221 corresponding to the processing target data from the cache 215 via the cache management module 212 (step S205). ).

  On the other hand, when the task processing module 211 does not exist in the cache 215 (step S203: NO), the task processing module 211 corresponds to the processing target data from any of the storage devices 203 of all the task processing nodes 200 via the data management module 213. The data 231 to be acquired is acquired (step S204).

  Next, the task processing module 211 inquires of the cache management module 212 whether or not there is free space in the cache 215 where the data 231 can be stored (step S206).

  As a result, when there is no free space available for storing the data 231 in the cache 215 (step S206: NO), the task processing module 211 notifies the cache management module 212 that the cache capacity is insufficient, and organizes the cache 215. In-cache organization processing to increase the free space is executed (step S207). The method of the in-cache reorganization processing differs depending on the information of the cache reorganization method 405 of the cache management information 214. The in-cache organizing process will be described later with reference to FIGS. 6 and 8.

  If there is free space (step S206: YES) or after executing the cache organization process (step S207), the task processing module 211 writes the data 231 to the cache 215 and updates the cache management information 214 (S208) . Specifically, the task processing module 211 stores the identifier of the new task in the cached task 406 of the entry corresponding to the task processing node 200 in the cache management information 214, and the identifier of the acquired data in the cache data list 407. Add and change the used cache capacity 403 to the capacity at that time.

  When the process of loop A is executed with all the acquisition target data as the process target, the task processing module 211 exits loop A and executes the process of the task (step S209).

  Next, when the task processing module 211 finishes executing the task processing, the task processing module 211 notifies the task management node 100 of the task execution result (step S210), and ends the processing.

  Next, in-cache organization processing in task units in the task processing node 200 will be described. The task-based in-cache reorganization processing is an example of the in-cache reorganization processing in step S207 of FIG.

  FIG. 6 is a flowchart of the in-cache reorganization processing in units of tasks according to the first embodiment.

  The task-based cache internalization process is started when the cache management module 212 receives a notification of cache capacity shortage from the task processing module 211.

  First, the cache management module 212 notifies the task management node 100 of the identifier of its own task processing node 200, and inquires the task management node 100 about the rank of the deletion target task (deletion priority) (step S301). As a result, the task management node 100 executes the deletion target task determination process (see FIG. 7), and as a result of this process, the cache management module 212 acquires the rank of the deletion target task.

  When the cache management module 212 receives the order of the deletion target task from the task management node 100, the cache data to be deleted from the cache data list 407 of the cache management information 214 corresponding to the deletion target task A list of cache data to be deleted is acquired (S302).

  Next, the cache management module 212 executes the processing of loop B (steps S304 and S305) with each of deletion target cache data corresponding to one deletion target task as a processing target according to the order.

  In loop B, the cache management module 212 refers to the information in the cache data list 407 of the cache management information 214 and determines whether or not other tasks use the deletion target cache data (processing target cache data) to be processed. It is determined (step S304).

  As a result, if it is determined that another task is using it (step S304: YES), deleting the process target cache data from the cache 215 will affect the other task, so the cache management module 212 Next, without deleting the processing target cache data, the processing of loop B is executed for the processing target of the deletion target cache data.

  On the other hand, when it is determined that another task is not using it (step S304: NO), the cache management module 212 deletes the cache data to be processed because there is no influence on the execution of the other task. The processing target cache data is deleted from 215 (step S305), and the processing of loop B is executed with the next deletion target cache data as the processing target.

  When the processing of loop B is executed with all the deletion target cache data as the processing target, the cache management module 212 exits loop B and the cache task 406 and the cache data list 407 corresponding to the deletion target task. Are deleted from the entry of the cache management information 214 (S306).

  Next, the cache management module 212 determines whether the capacity shortage of the cache 215 is resolved (step S307). As a result, when the capacity shortage of the cache 215 is not resolved (step S307: NO), the cache management module 212 executes the process of loop B for the next deletion target task.

  On the other hand, if the capacity shortage of the cache 215 is eliminated (step S307: YES), the cache management module 212 notifies the task management node 100 of the task whose cache data has been deleted (step S308), and ends the processing.

  As described above, according to the in-cache reorganization processing in units of tasks, the capacity of the cache 215 can be secured appropriately. In addition, data used in other tasks can be prevented from being deleted from the cache 215.

  Next, deletion target task determination processing by the task management node 100 will be described.

  FIG. 7 is a flowchart of deletion target task determination processing according to the first embodiment.

  The deletion target task determination process is started when an inquiry about the identifier of the task processing node 200 and the rank of the deletion target task is received from the task processing node 200.

  The task management module 111 determines the order of the deletion target task (step S401). Here, as a method of determining the order of the deletion target task, for example, a method of determining based on the number of comparisons of the number of comparisons 305 of the task allocation history information 112 (for example Method to make things with a large number of comparisons higher or methods to make decisions based on the date and time of the last execution date and time 308 of the task allocation history information 112 (latest ones are higher ones, oldest ones are higher ones) The method may be at least one of a method in which the executed task is performed and the average similarity rate of the average similarity rate 306 of the task allocation history information 112 is small.

  Next, the task management module 111 notifies the task processing node 200 of the determined order of the deletion target task (step S402).

  Next, when the task management module 111 receives a notification from the task processing node 200 of a task for which cache data has been deleted, the task management module 111 deletes the entry of the deleted task from the task allocation history information 112 (step S403). End the task determination process.

  According to the deletion target task determination process, the order of the deletion target task can be appropriately determined, and the entry for the task whose cache data has been deleted from the cache 215 can be deleted from the task allocation history information 112. Therefore, in the task allocation history information 112, only an entry for a task for which data used in the cache 215 of the task processing node 200 is cached is present.

  Next, the cache internal processing in cache data units in the task processing node 200 will be described. The in-cache reorganization processing in units of cache data is an example of the in-cache reorganization processing of step S207 in FIG. 5. Note that one or both of the in-cache organizing processing shown in FIG. 6 may be executed.

  FIG. 8 is a flowchart of the cache data unit cache internal processing according to the first embodiment.

  First, the cache management module 212 determines cache data to be deleted (deletion target cache data) (step S501). The cache data to be deleted may be, for example, data that is not redundantly used by a plurality of tasks, data with a small number of tasks used, data with a large data size, or data with a small data size, any method It can be determined by

  Next, the cache management module 212 deletes the deletion target cache data from the cache data list 407 of the cache management information 214 (step S502), and deletes the deletion target cache data existing in the cache 215 (step S503).

  Next, the cache management module 212 checks whether the necessary free space for the cache 215 has been secured (step S504). As a result, if the necessary free space can not be secured (step S504: NO), the cache management module 212 advances the process to step S501 and performs the process of deleting the next cache data to be deleted. On the other hand, when the necessary free space can be secured (step S504: YES), the cache management module 212 ends the cache data unit internal cache organization process.

  According to the above-described cache internal processing in cache data units, it is possible to appropriately secure the free space of the cache 215 without inquiring of the task management node 100.

  As described above, according to the computer system 10 according to the first embodiment, the task processing node 200 to which a task having a high similarity to the new task is assigned is assigned as the new task assignment destination. Data necessary for executing a new task is likely to exist in the cache 215 of the task processing node 200, and data may be read from the storage device 203 of the task processing node 200 to the memory 202, or another task may be performed. Acquisition of data from the storage device 203 of the processing node 200 by inter-node communication is reduced, and the communication amount of inter-node communication can be reduced. In addition, since the amount of communication between nodes can be reduced, the time to receive data can be reduced and the execution time of task processing can be shortened, and the user's waiting time until the result of task processing is obtained It can be shortened.

  Next, a computer system according to a second embodiment of the present invention will be described. In the following, differences from the computer system according to the first embodiment will be mainly described, and the description of the common points will be omitted or simplified.

  FIG. 9 is an overall configuration diagram of a computer system according to the second embodiment.

  The computer system 10A according to the second embodiment performs task processing on one or more data management nodes 500 including a configuration corresponding to the storage device 203 and the data management module 213 of the task processing node 200 in the computer system 10 according to the first embodiment. The data management node 500 is separate from the node 400 and connected to the network 300. Although the task processing node 400 illustrated in FIG. 9 has a configuration in which the storage device 203 and the data management module 213 are removed from the task processing node 200 according to the first embodiment, the present invention is not limited thereto. The processing nodes 200 may have the same configuration.

  The data management node 500 includes a CPU 501, a memory 502, a storage device 203, and a network interface 503.

  The network interface 503 is, for example, an interface such as a wired LAN card or a wireless LAN card, and another device (for example, the client 50, the task management node 100, the task processing node 400, the data management node 500, etc.) via the network 300. Communicate with.

  The CPU 501 executes various programs by executing programs (modules) stored in the memory 502.

  The memory 502 is, for example, a RAM, and stores programs executed by the CPU 501 and necessary information.

  The memory 502 stores a data management module 213.

  The data management module 213 is a program module that causes the CPU 501 to manage the arrangement status of the data 231 in the storage device 203 managed by the plurality of data management nodes 500.

  The computer system 10A according to the second embodiment may differ in the node that executes each process due to the difference in configuration from the computer system 10, but can execute processes similar to those in FIGS.

  According to the above-described second embodiment, the same effect as that of the first embodiment can be obtained. Further, according to the second embodiment, since the node (task processing node 400) that performs task processing and cache management and the node (data management node 500) that manages data are provided separately, for example, Since only the task processing node 400 can be expanded or only the data management node 500 can be expanded, the required performance can be realized at low cost.

  The present invention is not limited to the embodiments described above, and can be appropriately modified and implemented without departing from the spirit of the present invention.

  For example, in the task management process according to the above embodiment, when determining a node to which a task is to be allocated, as shown in steps S102 and S105 of FIG. 4, when there is an empty node, the task is allocated to the empty node. Although the nodes are determined, the present invention is not limited to this, and nodes to which tasks are to be assigned to all nodes may be determined regardless of whether there are free nodes or not.

  Further, in the above embodiment, the node (task processing node) that caches the data of the task with the highest similarity is determined as the node to which the task is assigned, but the present invention is not limited to this. Among nodes in which the number of tasks assigned is within a predetermined range, a node caching data of a task with the highest degree of similarity may be determined as a node to which tasks are assigned, and the degree of similarity is predetermined. A node that is caching data of a task that is equal to or more than a value may be determined as the node to which the task is assigned. In short, the node to which the task is assigned may be determined based on the similarity.

  10, 10A: computer system, 100: task management node, 101: CPU, 102: memory, 112: task allocation history information, 200, 400: task processing node, 201: CPU, 202: memory, 203: storage device, 214 ... Cache management information, 215 ... Cache, 300 ... Network, 500 ... Data management node

Claims (13)

A task management system comprising: a plurality of task processing nodes capable of executing tasks; and a task management node for determining a task processing node to which a new task is to be assigned,
Each of the plurality of task processing nodes has a memory capable of caching data used by an assignment task, which is a task assigned to the node;
The task management node is
Storing task assignment information including a correspondence between the assignment task and the task processing node to which the assignment task is assigned and in which data used by the assignment task is cached;
The processor of the task management node is
Determining the similarity between the new task and the assignment task;
Based on the degree of similarity, a task processing node to which the new task is to be assigned is determined from among the task processing nodes included in the task assignment information,
A task management system for assigning the new task to the determined task processing node. The task management node is
And storing the assignment task in association with an analysis result obtained by analyzing the assignment task by a predetermined analysis method,
The processor of the task management node is
Analyzing the new task by the predetermined analysis method;
The task management system according to claim 1, wherein the similarity is determined based on an analysis result of the new task and the analysis result of the assignment task. The assignment task and the new task are tasks related to natural language processing,
The task management system according to claim 2, wherein the predetermined analysis method is a method of vectorizing words or sentences of the assignment task or the new task. The processor of the task processing node is
4. The data to be used in the new task allocated to the task processing node of its own and not cached in the memory is acquired from a predetermined storage device and cached in the memory. Task management system according to any one of the above. The task processing node is
Storing information on correspondence between the assignment task and cached data among data used in the assignment task;
The processor of the task management node is
According to a predetermined condition, for the assignment task, a priority in deleting cached data is determined and notified to the task processing node;
The processor of the task processing node is
5. The task management system according to claim 4, wherein the data used in the assignment task is deleted from the memory according to the priority when there is no free space for caching the data acquired from the storage device in the memory. . The processor of the task processing node is
The task management system according to claim 5, wherein data used in the assignment task and not used in another assignment task is deleted from the memory. The processor of the task management node is
The new task is assigned to the execution node that has executed the assignment task, the last execution date of the assignment task, the similarity between the assignment task and one or more new tasks, the number of times the similarity with the new task is calculated The task management system according to claim 5 or 6, wherein the priority is determined based on at least one of the number of times. The processor of the task management node is
The information according to any one of claims 5 to 7 which deletes information which specifies said task processing node which performed said assignment task, when all data used by said assignment task are deleted from said memory. Task management system. The task processing node is
Storing information on correspondence between the assignment task and cached data among data used in the assignment task;
The processor of the task processing node is
9. The memory according to any one of claims 4 to 8, wherein when the memory has no free space for caching the data acquired from the storage device, data not used by a plurality of the assignment tasks is deleted from the memory. Task management system according to one item. The predetermined storage device is
The task management system according to any one of claims 1 to 9, provided in at least one of the plurality of task processing nodes. The predetermined storage device is
The task management system according to any one of claims 1 to 9, provided in a data management node configured separately from the task processing node. A task management method by a task management system, comprising: a plurality of task processing nodes capable of executing tasks; and a task management node for determining a task processing node to which a new task is to be assigned.
Each of the plurality of task processing nodes has a memory capable of caching data used by an assignment task, which is a task assigned to the node;
Storing task assignment information including a correspondence between the assignment task and the task processing node to which the assignment task is assigned and in which data used by the assignment task is cached;
Determining the similarity between the new task and the assignment task;
Based on the degree of similarity, a task processing node to which the new task is to be assigned is determined from among the task processing nodes included in the task assignment information,
A task management method for assigning the new task to the determined task processing node. A task management program for causing a computer constituting a task management node to determine a task processing node to which a new task is to be assigned among a plurality of task processing nodes capable of executing tasks,
The computer is
Each of the plurality of task processing nodes has a memory capable of caching data used by an assignment task, which is a task assigned to the node;
On the computer
Storing task assignment information including a correspondence between the assignment task and the task processing node to which the assignment task is assigned and in which data used by the assignment task is cached;
Have the degree of similarity between the new task and the assignment task determined
Based on the degree of similarity, a task processing node to which the new task is to be assigned is determined among the task processing nodes included in the task assignment information;
A task management program for assigning the new task to the determined task processing node.