JP6340308B2 - Parallel processing system, method, and program - Google Patents

Description

  The present invention relates to, for example, a system, method, and program for executing operations on data related to a plurality of users or data of a plurality of types at high speed by parallel processing in a system in which a plurality of data management servers operate in conjunction with each other.

  In recent years, in a cloud environment that has become widespread, a mechanism for sharing applications and data servers among a plurality of users is widely used in order to reduce resources and operation costs (multi-tenant method). For example, data related to municipal operations tends to be stored in a multi-tenant method in a cloud environment outside the municipality as part of cost reduction and disaster continuity planning (BCP) (see Non-Patent Document 1, for example) ).

  Local governments manage a wide variety of information on residents of the order of hundreds of thousands or millions over a period of several years, and the data is updated regularly or irregularly. For example, receipt information on medical care and dispensing in the National Health Insurance, health checkup information, and the like are added each time a visit is made. Therefore, when uploading local government business data related to such local governments to the cloud, it is required to be able to register and update various types of large volumes of data for multiple people efficiently and quickly while distributing them to appropriate destinations. ing. It is also desirable to minimize the labor of the operator as much as possible.

  Therefore, in an information system that manages data with a plurality of service servers built on the cloud, etc., the operator (external server) from the server owned by the local government, etc., gets to the service servers in the information system through the operator's hands. For example, a method for performing a data registration operation has been proposed (see, for example, Patent Document 1). By using the method of Patent Document 1 and automatically executing an authentication process that simulates a user after confirming the validity of the external server, each service server in the information system can legitimate access from the external server. Recognized as authenticated.

  Further, in Patent Document 1, in an information system in which a plurality of service servers operate in conjunction with each other, when the data operation for each service server is executed from an external server without intervention of the operator, the legitimacy of the access source is disclosed. It also describes how to check.

  Patent Document 1 also discloses that a certificate is appropriately managed by managing the authentication status when multiple accesses are executed from an external server and a plurality of threads are activated for processing in terms of speedup. A mechanism to make it available for multiple threads is proposed.

Japanese Patent No. 054758591

ADWORLD2 / SaaS Local government cloud, http://www.hitachi-systems.com/ind/adworld/products/cloud/

  However, in order to process a large amount of data at high speed, in addition to supporting multiple access from external servers, parallel processing is performed by dividing data operation processing requests for multiple service servers in the information system into appropriate units. There is a need to. The parallel processing is, for example, processing using multithread. Therefore, when processing a request including operation requests for various types of data for a plurality of users, such as local government business data, it is necessary to solve the following problems.

  A request from an external server includes data of a plurality of users and a plurality of types of data. In the information system, a database (DB) and a table in the service server are stored for each data type and for each service provider (local government). It is managed. Therefore, if the request data is separated according to the data type included in the request or the attribute of the user and the processing thread is not divided so that the communication processing time is minimized, there is a problem that the processing cannot be executed efficiently. is there.

  In addition, not all of the plurality of service servers in the information system are always operating normally. Some servers may be down or some processing program may be defective. Therefore, when an error occurs during request processing, it is necessary to restore at least the state before request execution by the operator or by automatically retrying. However, when multi-thread processing is executed for speeding up, there is a problem that it becomes difficult to know which data processing has failed in which service server and to know the range of the effect of the error. .

  The present invention has been made paying attention to the above circumstances, and its purpose is to make a request including operation requests for a plurality of users or various data to a plurality of servers in the system. Is to provide a system, a method, and a program for speeding up parallel processing.

In order to achieve the above object, a first aspect of the present invention includes the following components. That is, in a system that processes data in parallel using a plurality of management servers, and when an operation of data related to a plurality of users is requested from each external server to each management server, the data type or user attribute dividing the data in the request data for a plurality of management server based on the request data corresponding to each management server that is divided with different threads comprising a control means for executing the multithreaded process, the control means When a thread number including an identifier indicating at least the management server to be operated is assigned to each of the plurality of threads and an error occurs in any thread, at least the thread number corresponding to the thread in which the error has occurred Returned to an external server or output to a log.

  Further, an authentication means for authenticating the user is further provided. Further, the authentication unit issues an authentication token when the request source authentication token is required. Furthermore, the control means duplicates the authentication token and executes the duplicated authentication token for each thread in order to execute the multi-thread processing.

  The first aspect of the present invention also includes a method and program having similar components and aspects.

  That is, according to the present invention, a system, method, and program for speeding up parallel processing when a request including operation requests for a plurality of users or various types of data is made to a plurality of servers in the system Can be provided.

The figure which shows the structural example of the parallel processing system of the 1st Embodiment of this invention. The figure for demonstrating the process performed by the gateway server in the parallel processing system of the embodiment. An example of a processing sequence performed by a request processing overall management unit in the parallel processing system of the embodiment. An example of a processing sequence performed by a communication control unit in the parallel processing system of the embodiment. An example of a processing sequence performed by a communication processing unit in the parallel processing system of the embodiment. The example of the database structure which shows the information hold | maintained by the request processing supervision management part in the parallel processing system of the embodiment. The example of the database structure which shows the information hold | maintained by the communication control part of the parallel processing system of the embodiment. The example of the database structure which shows the information hold | maintained by the communication processing part of the parallel processing system which concerns on the embodiment. The example of the delivery data in the process performed when registering data in each service server from an external server by the gateway server of the parallel processing system concerning the embodiment. The figure which shows the structural example of the parallel processing system of the 2nd Embodiment of this invention. An example of a processing sequence performed by the parallel processing system of the embodiment. The figure which shows the structural example of the parallel processing system of the 3rd Embodiment of this invention. An example of a processing sequence performed by the parallel processing system of the embodiment.

(First embodiment)
A first embodiment according to the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram illustrating a configuration example of a parallel processing system 10 according to the first embodiment of this invention. The parallel processing system 10 of this embodiment includes a client terminal 50, a plurality of external servers 40-1, 40-2,..., 40-M, and a plurality of service servers 30-1, 30-2 via a network. ,..., 30-N.

  The gateway server 20 includes a request processing overall management unit 21, a communication management unit 22, and a communication processing unit 23. The gateway server 20 divides the data operation requests to the plurality of service servers 30 for each service server 30 to be a communication partner, and executes the divided data operation requests as different threads in parallel, thereby providing data to the plurality of service servers 30. In order to process an operation request at high speed, a thread for executing each process by the request processing overall management unit 21, the communication management unit 22, and the communication processing unit 23 is sequentially started, and the started threads are executed in parallel. Specifically, the gateway server 20 sequentially activates three threads corresponding to each process by the request processing overall management unit 21, the communication management unit 22, and the communication processing unit 23. Hereinafter, data operations such as data registration, update, or deletion are referred to as data operations. A data operation request is a request related to data operation.

  The request processing overall management unit 21 receives a data operation request for each service server 30 from each external server 40 or client terminal 50, and activates the main thread in response to the received data operation request. The received data operation request includes requests from a plurality of service providers. In addition, the request processing overall management unit 21 manages the entire response processing for data operation requests. The main thread is a thread for receiving a data operation request and managing the entire processing corresponding to the received data operation request.

  Further, the request processing overall management unit 21 receives data operation requests from the external servers 40 and the client terminals 50 to the service servers 30 and divides the received requests into requests to the service servers 30. More specifically, the request processing overall management unit 21 holds data location information. The request processing overall management unit 21 refers to the data location information, and divides data related to the data operation request (hereinafter referred to as request data), with the service server 30 that is a target of data operation as one unit. The data location information will be described in detail with reference to FIG. 6, but the data location information includes information necessary for dividing the request data for each service server 30.

  Further, the request processing overall management unit 21 calls the number of communication management units 22 corresponding to the number of service servers 30 to be communicated with the divided request data as an argument, and is returned from each communication management unit 22. The result is returned to the external server 40 or the client terminal 50 that is the request source.

  The communication management unit 22 activates a communication management thread in response to a call from the request processing overall management unit 21. The communication management thread is a thread for managing a response from the communication processing unit that has been divided and called up based on settings (for example, data size, connection upper limit number) for each service server 30 that is a communication partner. The communication management unit 22 manages the response process of the data operation request to the service server 30 specified when called.

  The communication management unit 22 holds service server path characteristic information. The service server path characteristic information will be described in detail with reference to FIG. 7, but the service server path characteristic information includes information for specifying an appropriate size in one communication. The communication management unit 22 refers to the service server path characteristic information, and divides the request data passed from the request processing overall management unit 21 into an appropriate size for the designated service server 30. The appropriate size is, for example, a data size based on the data volume or the like, or a size based on the number of requests. The communication management unit 22 calls the communication processing units 23 by the divided number using the divided request data as an argument, and returns the result returned from each communication processing unit 23 to the request processing overall management unit 21.

  The communication processing unit 23 activates a communication processing thread in response to a call from the communication management unit 22. The communication processing thread is a thread for connecting to the service server 30 and requesting a data operation. The communication processing unit 23 performs communication processing at least once for the service server 30 specified when the communication processing unit 23 is called.

  In addition, the communication processing unit 23 transmits the request divided by the communication management unit 22 to each service server 30 and requests the service server 30 corresponding to the specified request data to perform data operation on the specified request data. . The request data is specified when the communication processing unit 23 is called, for example. Then, the communication processing unit 23 returns the processing result to the communication management unit 22.

  The communication processing unit 23 holds service server communication information. The service server communication information will be described in detail with reference to FIG. 8, but the service server communication information includes information for communicating with each service server 30 such as an IP address, an end point reference, and a port. The communication processing unit 23 refers to the service server communication information and performs communication with the corresponding service server 30.

  Each service server 30 manages data by storing data of different data types according to instructions from the gateway server 20, and executes data operations according to requests from the client terminal 50 to store data. Process and provide services to the client terminal 50 and the like. Each service server 30 stores data of different data types, and one service server 30 stores data of a plurality of service providers.

  Each external server 40 and the client terminal 50 transmit a data operation request to the gateway server 20.

Next, an example of processing performed by the gateway server 20 will be described with reference to FIG.
FIG. 2 shows a relationship in calling the request processing overall management unit 21, the communication management unit 22, and the communication processing unit 23. 2 to 9, the case where data is registered from each external server 40 to each service server 30 as an example of data operation will be described. Data operation includes data update and data deletion as described above. But you can.

  The external server 40 sends registration data a having a plurality of data types to the request processing overall management unit 21 as request data related to the registration operation. The request processing overall management unit 21 activates the main thread in response to the data registration request received from the external server 40. Further, the request processing overall management unit 21 refers to the data location information 24, and registers the registered data a by using the registered data b-1 having the data type managed by the service server 30-1 and the service server 30-2. The data is divided into registered data b-2 having a data type to be managed.

  Then, the request processing overall management unit 21 uses the registration data b-1 and the registration data b-2 as arguments, so that the communication management thread is activated by the communication management unit 22 for each service server 30 of the communication destination. 22-1 and the communication manager 22-2 are called. In this way, the communication management unit 22 exists individually for each service server 30. Further, the request processing overall management unit 21 sends the registration data b-1 to the communication management unit 22-1, and sends the registration data b-2 to the communication management unit 22-2.

  FIG. 2 shows a case where the request processing overall management unit 21 calls two communication management units 22, but the request processing overall management unit 21 has one communication management unit when the registration data a is not divided. 22 is called.

  Next, the communication management unit 22-1 activates a communication management thread in response to a call from the request processing overall management unit 21. Further, the communication management unit 22-1 refers to the service server path characteristic information 25, registers the registration data b-1 and registration data c-1-1 having a size suitable for the path characteristic of the service server 30-1 and registration. Divide into data c-1-2.

  Then, the communication management unit 22-1 performs communication for each communication between the gateway server 20 and the service server 30-1 using the registration data c-1-1 and the registration data c-1-2 as arguments. In order to activate the processing thread, the communication processing unit 23-1-1 and the communication processing unit 23-1-2 are called. In addition, the communication management unit 22-1 sends registration data c-1-1 to the communication processing unit 23-1-1 and sends registration data c-1-2 to the communication processing unit 23-1-2.

  Similarly, the communication management unit 22-2 activates a communication management thread in response to a call from the request processing overall management unit 21. Further, the communication management unit 22-2 refers to the service server path characteristic information 25, registers the registration data b-2, registration data c-2-1 having a size suitable for the path characteristic of the service server 30-2, and registration. Divide into data c-2-2.

  Then, the communication management unit 22-2 performs communication for each communication between the gateway server 20 and the service server 30-2 using the registration data c-2-1 and the registration data c-2-2 as arguments. In order to activate the processing thread, the communication processing unit 23-2-1 and the communication processing unit 23-2-2 are called. In addition, the communication management unit 22-2 sends registration data c-2-1 to the communication processing unit 23-2-1 and sends registration data c-2-2 to the communication processing unit 23-2-2.

  2 shows an example in which two communication processing units 23 exist for one communication management unit 22, this is merely an example. For example, the communication processing units 23-1-1 and The communication processing unit 23-1-2 is shared by one communication processing unit 23-1 (not shown), and the communication processing unit 23-2-1 and the communication processing unit 23-2-2 are You may share with the communication process part 23-2 (not shown).

  The communication processing unit 23-1-1 activates a communication processing thread in response to a call from the communication management unit 22-1. And the registration operation regarding registration data c-1-1 is requested to the service server 30-1. The communication processing unit 23-1 refers to the service server communication information 26 and communicates with the service server 30-1.

  In addition, the communication processing unit 23-1-2 activates a communication processing thread in response to a call from the communication management unit 22-1. And the registration operation regarding registration data c-1-2 is requested to the service server 30-1. The communication processing unit 23-1-2 refers to the service server communication information 26 and communicates with the service server 30-1.

  Similarly, the communication processing unit 23-2-1 activates a communication processing thread in response to a call from the communication management unit 22-2. Then, a registration operation regarding the registration data c-2-1 is requested to the service server 30-2. The communication processing unit 23-2-1 refers to the service server communication information 26 and communicates with the service server 30-2.

  In addition, the communication processing unit 23-2-2 activates a communication processing thread in response to a call from the communication management unit 22-2. Then, a registration operation regarding the registration data c-2-2 is requested to the service server 30-2. The communication processing unit 23-2-2 refers to the service server communication information 26 and communicates with the service server 30-2.

Next, an example of processing executed by the request processing overall management unit 21 will be described with reference to FIG.
3 to 9, as an example, a case where three service servers 30-1, 30-2, and 30-3 are used will be described. However, the number of service servers 30 used is three. For example, two, four, or more service servers 30 may be used.

  When receiving the data registration operation request (step S20), the request processing overall management unit 21 activates the main thread (step S21). Next, the registration data a is read in the main thread (step S22). Then, in order to divide the read registration data a into registration data b-1, b-2, b-3 for each service server 30 serving as a storage destination, all the units of the registration data a, for example, the unit concerned, are recorded. The process of discriminating the service server 30 that is the storage destination based on the data type in the inside is repeated (step S23).

  The contents to be repeatedly executed for all the units included in the registration data a are as follows. First, for each unit of registered data a, the service server 30 that is a storage destination of data corresponding to the data type is determined (step S24).

  And when it determines with storing the data of the said unit of registration data a in the service server 30-1, the data of the said unit of registration data a are added to registration data b-1 (step S25-1). On the other hand, if it is determined that the unit of registration data a is stored in the service server 30-2, the unit of registration data a is added to the registration data b-2 (step S25-2). Alternatively, when it is determined that the unit of registration data a is stored in the service server 30-3, the unit of registration data a is added to the registration data b-3 (step S25-3).

  Next, it is determined whether or not the creation of registration data b-1, b-2, and b-3 is completed (step S26). In addition, the process in step S26 may be a process of determining whether or not the number of times the process in steps S23 to S26 has been performed has reached a preset number. When the creation of the registration data b-1, b-2, b-3 is completed (YES in step S26), the process of step S26 described later is performed. If the creation of the registration data b-1, b-2, b-3 has not been completed (NO in step S26), the process returns to step S23 to perform an additional process for the next unit of the registration data a. In step S23, when there is no next unit of the registration data a to be subjected to the addition process, the process proceeds to step S27 without performing the addition process that is repeatedly performed in steps S23 to S26.

  After step S23 or step S26, that is, after the creation of the registration data b-1, b-2, b-3 is completed, the request processing overall management unit 21 uses the registration data b-1 as an argument and the communication management unit 22 -1 is called (step S27). Next, the communication management unit 22-2 is called with the registration data b-2 as an argument (step S28). Next, the communication management unit 22-3 is called with the registration data b-3 as an argument (step S29). Note that the order of the processing of step S27, step S28, and step S29 may not be the order shown in FIG. For example, step S27 may be performed after step S28 or step S29.

  After step S29, the request processing overall management unit 21 waits until responses from all the communication management units 22-1, 22-2, and 22-3 that have been called are completed (step S30). When there is a response from all of the called communication management units 22-1, 22-2, and 22-3, in other words, when the communication management thread that has been activated in parallel ends, each communication management unit 22-1, 22-2 2 and 22-3, the request result returned to the external server 40 or the client terminal 50 that is the request source is returned (step S31).

Next, an example of processing executed by the communication management unit 22 will be described with reference to FIG.
In response to a call from the request processing overall management unit 21 (step S40), the communication management unit 22-n (n is, for example, 1 to 3) starts a communication management thread with the registration data b-n as an argument (step S40). S41). Next, the communication management unit 22-n divides the registration data b-n into the number of requests suitable for the service server 30, and generates registration data c-n-1, c-n-2 (step S42).

  Next, the communication processing unit 23-n-1 corresponding to the registration data cn-1 is called using the registration data cn-1 as an argument (step S43). Also, the communication management unit 22-n calls the communication processing unit 23-n-2 corresponding to the registration data cn-2 using the registration data cn-2 as an argument (step S44).

  After step S43-1 or step S43-2, the communication management unit 22-n waits until responses from all the communication processing units 23-n-1, 23-n-2 that have been called are completed (step S45). . When there is a response from all of the called communication processing units 23-n-1, 23-n-2, in other words, when the communication processing thread activated in parallel ends, the communication management unit 22-n Is returned to the request processing overall management unit 21, that is, to the main thread (step S46).

Next, an example of processing executed by the communication processing unit 23 will be described with reference to FIG.
Each of the communication processing unit 23-n-1 (n is, for example, 1 to 3) and the communication processing unit 23-n-2 responds to a call from the communication management unit 22-n (step S50), and registration data c -A communication processing thread is activated with each of n-1 and registered data c-n-2 as arguments (step S51). Each of the communication processing unit 23-n-1 and the communication processing unit 23-n-2 requests data registration from the corresponding service server 30-n, and obtains a processing result from each of the service servers 30-n (steps). S52). Each of the communication processing unit 23-n-1 and the communication processing unit 23-n-2 returns the acquired processing result to the corresponding communication management unit 22-n (step S53).

  Next, an example of information held by each unit of the gateway server 20 will be described with reference to FIGS. Each unit holds setting data necessary for processing of each unit according to each role.

First, an example of a database structure indicating information held by the request processing overall management unit 21 will be described with reference to FIG.
The request processing overall management unit 21 holds data location information 24. As described above, the data location information 24 includes information necessary for dividing the request data for each service server 30. The data location information 24 is also information for specifying the service server 30 in which each data type included in the request data is stored.

  As shown in FIG. 6, the data location information 24 includes, for example, data type information 24-a and storage destination service server information 24-b.

  The data type information 24-a is information indicating the type of data. As shown in FIG. 6, the data type information 24-a includes, for example, medical receipt information related to medical care, care receipt information related to care, and health check information related to health checkup.

  The storage destination service server information 24-b is information related to the service server 30 that is a data storage destination. The storage destination service server information 24-b includes, for example, elements of a service server 1, a service server 2, and a service server 3, as shown in FIG.

  Each element of the data type information 24-a is associated with each element of the service server information 24-b on a one-to-one basis, and indicates a data type stored by each service server 30.

Next, an example of a database structure indicating information held by the communication management unit 22 will be described with reference to FIG.
The communication management unit 22 holds service server path characteristic information 25. As described above, the service server path characteristic information 25 includes information for specifying an appropriate size in one communication. The service server route characteristic information 25 includes, for each service server identification information 25-a, information related to an optimum value based on the performance of the service server 30 (server performance element information 25-b), and the performance of the route to the service server 30. Also includes information on the optimum value (route characteristic element information 25-c).

  As the optimum value, based on the number of requests to the service server 30 held by the communication management unit 22, a value having a small number of requests among the number of requests in each service server 30 may be set as the optimum value. Alternatively, based on the appropriate data size as described above held by the communication management unit 22, for example, a value obtained by dividing the appropriate data size by the recording size of one unit of request data is set as the optimum value, and the communication management unit 22 It may be calculated by, for example.

  Specifically, as shown in FIG. 7, the service server path characteristic information 25 includes service server identification information 25-a, server performance element information 25-b, and path characteristic element information 25-c.

  The service server identification information 25-a is information for identifying each service server, and includes, for example, elements of the service server 1, the service server 2, and the service server 3 as illustrated in FIG.

  The server performance element information 25-b and the path characteristic element information 25-c are associated with each service server 30 on a one-to-one basis. For example, as illustrated in FIG. 7, the service server 1 includes the server performance element information 25-b having 100 cases as an optimum value and the route characteristic element 25-c having 200 cases as an optimum value. Associated. Similarly, the service server 2 is associated with the element of the server performance element information 25-b having 200 cases as the optimum value and the element of the path characteristic element 25-c having 200 cases as the optimum value. Further, the service server 3 is associated with an element of the server performance element information 25-b having 300 optimum values and an element of the path characteristic element 25-c having 200 optimum values.

Next, an example of a database structure indicating information held by the communication processing unit 23 will be described with reference to FIG.
The communication processing unit 23 holds service server communication information 26. As described above, the service server communication information 26 is communicated with each service server 30 such as the end point reference information 26-b, the IP address information 26-c, and the port for each service server identification information 26-a. Contains information.

  As shown in FIG. 8, each element of the service server identification information 26-a is associated with each element of the End Point Reference information 26-b and each element of the IP address information 26-c on a one-to-one basis. For example, the service server 1 is associated with an element of End Point Reference information 26-b indicated by “xxxxxxxx” and an element of IP address information 26-c indicated by “xxx.xxx.xxx.xx”. Yes.

Next, an example of data delivered by each unit of the gateway server 20 will be described with reference to FIG.
The “registration data a” that is data delivered from the external server 40 to the request processing overall management unit 21 includes data relating to a plurality of users having various user attributes and data relating to a plurality of data types.

  “Registered data a” is represented, for example, as a delivery database 90 as shown in FIG. Specifically, the delivery database 90 includes user identification information 90-a, data type information 90-b, and user attribute information 90-c. The data type information 90-b includes medical receipt information 90-d, care receipt information 90-e, and health check information 90-f. The user attribute information 90-c includes belonging insurer information 90-g and residence information 90-h.

  More specifically, for each user identification information 90-a, medical receipt information 90-d, care receipt information 90-e, health check information 90-f, belonging insurer information 90-g, and residence information 90-h Is associated. For example, in the user identification information 90-a indicated by UserA, the element of the medical receipt information 90-d indicated by “xxAA”, the element of the care reception information 90-e indicated by “yyAA”, indicated by “zzAA” Are associated with an element of health check information 90-f, an element of insurer information 90-g indicated by "National Health Insurance", and an element of residence information 90-dh indicated by "Local Government X". .

  Registration data 92 (b-1), 92 (b-2), and 92 (b-3) that are data delivered from the request processing overall management unit 21 to each of the communication management units 21-1, 21-2, and 21-3. ) Is data obtained by extracting only request data related to data types stored in the service servers 30-1, 30-2, and 30-3, respectively.

  Specifically, the registration data 92 (b-1) is data obtained by extracting the medical receipt information 90-d for each user identification information 90-a from the registration data a 90. Similarly, the registration data 92 (b-2) is data obtained by extracting the care receipt information 90-e for each user identification information 90-a from the registration data a 90. The registration data 92 (b-3) is data obtained by extracting the health checkup receipt information 90-f for each user identification information 90-a from the registration data a90.

  The registration data 94 (c-1-1) and 94 (c-1-2) are delivered from the communication management unit 22-1 to the communication processing unit 23-1-1 and the communication processing unit 23-1-2. The request data is divided into sizes suitable for the performance of the service server 30-1 and the characteristics of the route to the service server 30-1.

  Specifically, the registration data 94 (c-1-1) includes 100 pieces of medical receipt information 90-d including the medical receipt information 90-d related to UserA and the medical receipt information 90-d related to UserB. . On the other hand, the registration data 94 (c-1-2) has 100 pieces of medical receipt information 90-d including the medical receipt information 90-d related to UserC and the medical receipt information 90-d related to UserD.

  Similarly, although not shown, registration data 94 (c-2-1), which is data delivered from the communication management unit 22-2 to the communication processing unit 23-2-1 and the communication processing unit 23-2-2, and Registration data 94 (c-2-2) also exists. Similarly, each of the registration data 94 (c-2-1) and the registration data 94 (c-2-2) has a size suitable for the performance of the service server 30-2 and the characteristics of the route to the service server 30-2. It is carved into.

  Similarly, although not shown, registration data 94 (c-3-1) which is data delivered from the communication management unit 22-3 to the communication processing unit 23-3-1 and the communication processing unit 23-3-2. ) And registration data 94 (c-3-2). Similarly, each of the registration data 94 (c-3-1) and the registration data 94 (c-3-2) has a size suitable for the performance of the service server 30-3 and the characteristics of the route to the service server 30-3. It is carved into.

According to the first embodiment as described above, the following effects can be obtained.
Depending on the data type included in the request, the affiliation of the data owner, etc., it is possible to appropriately divide the unit to be processed in each thread and execute the multi-thread process, thereby minimizing the number of communication processes. Further, when communicating with the service server 30 that is a data operation target in each thread, it is possible to further perform thread division according to the capability of the partner server and the network environment to optimize the communication processing. Therefore, it is possible to increase the speed by hierarchical multithread processing.

In more detail, the number of communication processes is minimized by separating the data according to the contents of the request and hierarchically multi-threading, and depending on the ability of the communication partner that is the target of each thread. The communication processing can be optimized. This makes it possible to process data operation requests to the plurality of service servers 30 in the parallel processing system 10 at high speed. In addition, the request data size is divided according to the performance of the service server 30 that is the distribution destination, hierarchically multi-threaded, and parallel processing is executed, so that the speed can be increased.
(Second Embodiment)
Hereinafter, a second embodiment according to the present invention will be described with reference to FIGS. 10 and 11.
In the following, the description will focus on parts different from the configuration shown in the first embodiment. Therefore, the same code | symbol is attached | subjected to the element which is common in 1st Embodiment, and the overlapping description is abbreviate | omitted.

FIG. 10 is a diagram illustrating a configuration example of the parallel processing system 100 according to the second embodiment of this invention.
The parallel processing system 100 according to the second embodiment has a configuration in which an authentication server 110 is further added to the parallel processing system 10 according to the first embodiment. Also, the parallel processing system 100 of the second embodiment assumes a case where authentication is required to use the parallel processing system 100, and uses an authentication token issued by the authentication server 110.

  The authentication server 110 is connected to the gateway server 20 and the service servers 30-1, 30-2,..., 30-N, and issues an authentication token when authentication is necessary.

Next, an example of a processing sequence performed by the parallel processing system 100 according to the second embodiment will be described with reference to FIG.
The parallel processing system 100 further executes the following processing when executing multithread processing.
In response to a data operation request from the client terminal 50 or the external server 40, the gateway server 20 performs pseudo authentication when an authentication token of the requesting client terminal 50 or the external server 40 is required (step S111). The authentication token is, for example, a token indicating that the user corresponding to the client terminal 50 has been authenticated when each thread operating on the gateway server 20 is activated. The pseudo authentication is authentication performed when connecting to each external server 40 using the hashed ID / password held in the gateway server 20.

  Next, when the pseudo authentication is successful, the gateway server 20 requests the authentication server 110 to issue an authentication token (step S113). The authentication server 110 issues an authentication token in response to this request (step S115), and transmits the issued authentication token to the gateway server 20.

  Next, the gateway server 20 replicates the received authentication token for each thread so that authentication tokens used between threads operating in parallel do not compete with each other (step S117). In other words, the gateway server 20 generates an instance of an authentication token for each thread. The gateway server 20 uses the duplicated authentication token for each thread and causes each service server 30 to execute parallel processing (step S119).

With the parallel processing system 100 of the second embodiment as described above, the following effects can be obtained.
That is, in the second embodiment, even when authentication is required, by using the authentication server 110, data operation processing for a plurality of service servers 30-1, 30-2,. Can be executed in parallel at high speed. Further, even when the threads operate in parallel, the authentication token can be used without the authentication token competing. In addition, each service server 30-1, 30-2,..., 30-N can determine each external server 40 that is a request source.
(Third embodiment)
A third embodiment according to the present invention will be described below with reference to FIGS.
In the following, the description will focus on parts different from the configurations shown in the first embodiment and the second embodiment. Therefore, the same code | symbol is attached | subjected to the element which is common in 1st Embodiment and 2nd Embodiment, and the overlapping description is abbreviate | omitted.

  The parallel processing system 120 according to the third embodiment is configured to include an authentication server 130 instead of the authentication server 110 in the parallel processing system 100 according to the second embodiment. This configuration assumes that the parallel processing system 120 uses the authentication server 130 to manage user IDs in a distributed manner. More specifically, the parallel processing system 120 uses real user IDs as user identifiers to manage user IDs in communication between the service servers 30-1, 30-2,..., 30-N. Without using the mediation ID for mediation given to each user. The mediation ID is used to execute cooperation processing between the authentication server 130 and each service server 30-1, 30-2,..., 30-N.

First, a configuration example of the parallel processing system 120 will be described with reference to FIG.
The authentication server 130 includes a communication processing unit 131 and a data operation unit 132. Further, the authentication server 130 holds user information 133 and ID linkage information 134.

  The communication processing unit 131 communicates with the communication processing unit 23 and any one of the communication processing units 31-1, 31-2, ..., 31-N. The communication processing unit 131 may be connected to the data operation unit 132 and perform processing related to data operation in cooperation with the data operation unit 132. Further, the communication processing unit 131 is connected to each service server-1, 30-2,..., 30-N, and cooperates with each service server-1, 30-2,. Then, processing related to data manipulation may be performed.

  The data operation unit 132 cooperates with the user information 133 and the communication processing unit 131 to perform, for example, user registration processing, mediation ID registration processing, and data registration processing as described later.

  The user information 133 includes information related to the user ID in the authentication server 130 of a user who uses any of the service servers 30-1, 30-2, ..., 30-N.

  The ID linkage information 134 includes mapping information related to the mapping between the mediation ID used between the authentication server 130 and each of the service servers 30-1, 30-2,..., 30-N and the user ID of the authentication server 130.

  Each of the service servers 30-1, 30-2,..., 30-N has the same configuration as that of the authentication server 130. That is, the service servers 30-1, 30-2,..., 30-N are respectively connected to the communication processing units 31-1, 31-2,. , 32 -N, user information 33-1, 33-2,..., 33 -N, and ID linkage information 34-1, 34-2,. Hold -N.

  Each user information 33-1 held by the service server 30-1 includes information related to a user ID in the service server 30-1 of a user who uses the service server 30-1. The same applies to the user information 33-2, ..., 33-N.

  The ID linkage information 34-1 includes mapping information related to mapping between the same mediation ID as the ID linkage information 134 of the authentication server 130 and the user ID in the service server 30-1. The same applies to the ID linkage information 34-2,..., 34-N.

Next, an example of a processing sequence performed by the parallel processing system 120 will be described.
First, an outline of a processing sequence performed by the parallel processing system 120 will be described. The operation executed by the parallel processing system 120 is the first embodiment when the data operation request from each external server 40-1, 40-2,..., 40-M is a request related to update or deletion. This is the same operation as that executed by the parallel processing system 10. However, for example, when the data operation request is a request related to registration, the user registration processing to the authentication server 130 and each of the service servers 30-1, 30-2,. It is necessary to execute the registration process of the intermediary ID to the service servers 30-1, 30-2,..., 30-N before performing the data registration process.

Next, the parallel processing related to data registration will be specifically described with reference to FIG.
First, the gateway server 20 receives a data registration operation request from each external server 40-1, 40-2,..., 40-M (step S131). When the gateway server 20 receives the data registration operation request, the gateway server 20 executes the following processing before processing the data registration processing in parallel.

  The request processing overall management unit 21 generates user registration request data for the service servers 30-1, 30-2,..., 30-N and the authentication server 130 based on the data registration operation request data (step S133). ). The generated user registration request data is data related to the user corresponding to the user information included in the data registration operation request data.

  Next, the request processing overall management unit 21 uses a data registration process similar to the data registration process described in the first embodiment, for each of the service servers 30-1, 30-2,. The generated user registration request data is divided (step S135). Then, the request processing overall management unit 21 calls the communication management unit 22 using the divided user registration request data as an argument. Then, the communication management unit 22 requests the user registration to a size suitable for the performance of the service server 30 (any one of the service servers 30-1, 30-2,..., 30-N) as a communication destination. The data is divided and the communication processing unit 23 is called to execute parallel processing related to the user registration request data.

  Next, the request processing overall management unit 21 generates user ID cooperation request data (mediation ID registration request data) (step S137). The user ID cooperation request data is data relating to the user corresponding to the user information included in the data registration operation request data for the service servers 30-1, 30-2,..., 30-N and the authentication server 130. Further, the user ID cooperation request data is data requesting registration of the mediation ID in the authentication server 130 and each of the service servers 30-1, 30-2, ..., 30-N.

  Note that the mediation ID included in the user ID cooperation request data may be included in the data registration operation request data, or may be generated by the gateway server 20. Alternatively, the gateway server 20 requests the authentication server 130 to issue and register a mediation ID corresponding to the user ID included in the data registration operation request data, and uses the mediation ID issued by the authentication server 130. May be.

  Next, the request processing overall management unit 21 uses a data registration process similar to the data registration process described in the first embodiment, for each of the service servers 30-1, 30-2,. Then, the user ID cooperation request data is divided (step S139), and the communication management unit 22 is called using the divided user ID cooperation request data as an argument. Then, the communication management unit 22 links the user ID to a size suitable for the performance of the service server 30 (any one of the service servers 30-1, 30-2,..., 30-N) that is the communication destination. Split request data. And the communication management part 22 calls the communication process part 23, and performs the parallel process regarding user ID cooperation request data (step S141).

  After step S141, the parallel processing related to data registration described in the first embodiment is executed. For example, after step S141, the process of step S31 as shown in FIG. 3 is started.

According to the third embodiment as described above, the following effects can be obtained.
That is, according to the present embodiment, user IDs are distributed and managed, and an intermediary ID is used instead of an actual user ID between the authentication server 130 and the service servers 30-1, 30-2, ..., 30-N. Even in this case, a plurality of service servers 30-1, 30-2,..., 30 are used by using the parallel processing method in the parallel processing system 10 of the first embodiment or the parallel processing system 100 of the second embodiment. With respect to −N, the data operation processing can be executed at high speed. Further, by managing user IDs in a distributed manner, users to the service servers 30-1, 30-2,... The registration process and the ID linkage process can also be executed at high speed in one request.
(Other variations)
Next, modifications that can be applied in the first to third embodiments will be described.
(Modification 1)
In Modification 1, it is assumed that an error has occurred during execution of multi-thread processing. By issuing a return value and a log relating to an error when an error occurs, for example, even when an error occurs during multi-thread processing, an error occurs. Specify the location (error handling / log output method).

  When an error occurs, the gateway server 20 in the parallel processing system of Modification 1 outputs a function for issuing a thread number corresponding to the thread in which the error has occurred, and a log for identifying the location where the error has occurred. It has a function.

  Specifically, the gateway server 20 assigns a thread number to each of a plurality of threads in order to identify an error occurrence location. If an error occurs in any of the threads, the gateway server 20 A corresponding thread number (hereinafter referred to as an error thread number) is issued. The thread number is, for example, a systematic thread number set in advance when the thread is divided.

  Next, for example, when executing processing for each thread, the gateway server 20 outputs, as a log, data to be subjected to data operation included in the thread corresponding to the error thread number. Further, the gateway server 20 returns the error thread number to, for example, the external server 40 or the client terminal 50 that is the request source.

Next, the output log will be described.
First, when the request processing overall management unit 21 divides a thread for each thread in the hierarchical thread used in the first to third embodiments, the following systematic thread numbers are used. Is granted. The thread number is represented by “XYZ”, for example.

  Here, X represents a character string or a numerical value indicating each service server 30. X indicated by the character string is, for example, “medical”, “care”, “inquiry”, or “checkup”. The value of X is managed by the main thread.

  Y is a value indicating the number of communication management threads for each service server 30. Y is incremented every time the communication management unit 21 is called with data for the same service server 30 as an argument, in other words, every time a communication management thread is created. The value of Y is managed by the main thread.

  Z is a value indicating the number of communication processing threads generated by each communication management thread. Z is incremented each time the communication processing unit 23 is called in the communication management thread, in other words, every time the communication processing thread is activated. The value of Z is managed by a communication management thread.

According to Modification 1 as described above, the following effects can be obtained.
When making a request including operation requests for data related to a plurality of users and various types of data, such as local government business data, to a plurality of service servers 30 in a parallel processing system, processing speeding up and errors It is possible to specify the location of occurrence at the time of occurrence.

  Further, the client terminal 50 corresponding to the user who is a user can specify the error occurrence location and the range affected by the error by using the return value and log including the error thread number when an error occurs. . Further, the client terminal 50 can execute a data operation request retry for an error occurrence location, for example, based on the return value and the log as necessary. In addition, the client terminal 50 can accurately track a log regarding data operation for each request by determining that the processing is for the same request based on the log.

  In addition, by outputting the error thread number as described above as a log during processing of each thread, for example, even when processing a plurality of data operation requests, the same data operation request is processed. It is possible to associate the logs output by the parent thread and the child thread with each other, and it is possible to trace the processing log of the same data operation request.

  When an error occurs in any thread, the error thread number is returned to, for example, the external server 40 or the client terminal 50 that is the request source, and the request data and error thread number included in the thread in which the error has occurred. By outputting to the log, it is possible to clarify data processed in the same thread that may be affected by an error. For this reason, it is possible to efficiently perform a retry when an error occurs.

In addition, when unauthorized use is detected, it may be necessary to track the access history for each user using a log. In such a case, when the processing is executed sequentially, for example, by outputting the name of the executing thread to the log, even if a plurality of data operation requests are mixed, a plurality of data operation requests Logs corresponding to the same data operation request can be extracted from. However, when executing multi-thread processing, the same data operation request processing is divided into multiple threads and executed, and the thread name such as a simple serial number that does not indicate that each thread is the same data operation request In the case of outputting a log with, it becomes difficult to track using the time series regarding the execution status of the executed data operation request processing and to analyze the location where the error occurred. However, this can be prevented by executing the processing described above in the first modification of the present invention.
(Modification 2)
Modification 2 is a modification related to the thread division unit by the request processing overall management unit 21.
In a parallel processing system, for the purpose of reducing costs by sharing resources, even if a service server 30 is installed for each data type, data is managed independently for each local government in the service server 30. There is a case. Further, even if the service server 30 is installed for each municipality in order to independently manage data having different management organizations or classifications within the local government, the service server 30 may manage the data independently for each data.

  In such a case, the data location information 24 held by the request processing overall management unit 21, the destination and user attribute information 90-c regarding more detailed data recorded in the database or table in the service server 30, To map. Thereby, the request processing overall management unit 21 further divides the request data by further using the user attribute information 90-c regarding the data owner included in the request data, and independently manages the unit of the communication management thread in the service server 30. It becomes possible to match the unit. Further, by enabling the communication management unit 22 to refer to the data location information 24, when calling the communication processing unit 23 in the communication management unit 22, the request attribute is further divided using the user attribute information 90-c, The unit of the communication management thread can be matched with the unit managed independently in the service server 30.

  In consideration of retrying when an error occurs for the thread division unit in the communication management unit 22, it is desirable that the result of whether or not the multi-thread processing is successful can be grasped for each user. Therefore, when there is a lot of identical user information included in the request data, the request data divided for each service server 30 is divided for each user in the communication management unit 22 and the communication processing thread is activated. Good.

  In the first to third embodiments, the data registration is described as an example of the data operation. However, by replacing the request data with the request search expression, the data search and the data acquisition are performed as the data operation. The first to third embodiments can be applied. Therefore, for example, by collecting data with distributed management of user IDs using a protocol such as ID-WSF, parallel processing can be speeded up.

  Further, the parallel processing systems 10, 100, 120 may connect the client terminal 50 to the gateway server 20 without connecting to the external server 40, and read local request data using, for example, a GUI. Good.

  In addition, this invention is not limited to the said 1st Embodiment thru | or 3rd Embodiment. For example, the configuration in the case where the authentication server 110 is provided separately from the gateway server 30 as illustrated in FIGS. 1, 10, and 12 has been described, but the authentication server 110 may be included in the gateway server 30. Or you may comprise by more entities. For example, the parallel processing system 10 can be obtained by appropriately integrating or further distributing databases included in each of the request processing overall management unit 21, the communication management unit 22, the communication processing unit 23, the authentication server 130, and the service server 30. , 100, 120 may be realized.

  In addition, FIGS. 6 to 9 are examples of each database, and since fields can be added as appropriate, FIGS. 6 to 9 and the like are variously modified without departing from the gist of the present invention. Is possible.

  Further, as shown in other modified examples, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  DESCRIPTION OF SYMBOLS 10,100,120 ... Parallel processing system, 20 ... Gateway server, 21 ... Request processing supervision management part, 22 ... Communication management part, 23 ... Communication processing part, 24 ... Data location information, 25 ... Service server path characteristic information, 26 ... service server communication information, 30 ... service server, 40 ... external server, 50 ... client terminal, 90 ... delivery database, 92, 94 ... registration data, 110, 130 ... authentication server, 31, 131 ... communication processing unit, 32, 132: Data operation unit, 33, 133: User information, 34, 134: ID linkage information.

Claims (5)

A system for processing data in parallel using a plurality of management servers,
When an operation of data related to a plurality of users is requested from an external server to each management server, the data is divided into request data for each of the plurality of management servers based on the type of the data or the attribute of the user, a control means for executing a multi-threading as a plurality of threads having different request data corresponding to each management server that is the divided,
The control means assigns a thread number including at least an identifier indicating the management server to be operated to each of the plurality of threads, and when an error occurs in any thread, the thread in which the error has occurred A parallel processing system that returns a thread number corresponding to at least to the external server or outputs it to a log . An authentication unit for authenticating the user;
When the authentication token of the request source is necessary, the authentication means issues the authentication token,
The system according to claim 1, wherein the control means replicates the authentication token and uses the replicated authentication token for each thread to perform the multi-thread processing. A parallel processing method executed by a system that processes data in parallel using a plurality of management servers,
Control means for the system comprises If the requested operation data for a plurality of users with respect to each management server from the external server, the plurality of managing the data based on the attributes of the type or the user of the data dividing the request data for each server, the request data corresponding to the divided respective management server has to perform multithreading in a plurality of different threads,
When the control means included in the system executes the multi-thread processing, each of the plurality of threads gives a thread number including an identifier indicating the management server to be operated at least to any one of the threads. In the case where an error occurs , a parallel processing method in which at least a thread number corresponding to the thread in which the error has occurred is returned to the external server or output to a log . An authentication means included in the system authenticates the user,
When the authentication means provided in the system requires the authentication token of the request source, the authentication token is issued,
The parallel processing method according to claim 3 , wherein the control means included in the system duplicates the authentication token and uses the duplicated authentication token for each thread to execute the multi-thread processing. A program for processing data in parallel using a plurality of management servers,
When an operation of data related to a plurality of users is requested from an external server to each management server, the data is divided into request data for each of the plurality of management servers based on the type of the data or the attribute of the user, wherein the divided request data corresponding to each management server which, to execute the steps for executing the multithreaded processed as different threads in the computer,
And the procedure for executing the multi-thread process is to assign a thread number including an identifier indicating at least the management server to be operated to each of the plurality of threads, and when an error occurs in any of the threads, A program that returns at least the thread number corresponding to the thread in which the error has occurred to the external server or outputs it to a log .

Images