JP6211487B2 - Distributed system, data acquisition method, and computer program - Google Patents

Description

  The present invention relates to a distributed system, a data acquisition method, and a computer program.

  Patent Document 1 discloses a command conversion storage unit that stores a protocol name supported by a control device connected to a network, protocol command information, and profile information including a command type, and is addressed from the information processing device to the target control device. When receiving the transmission data, the command conversion storage means is accessed, and a judgment means for judging whether or not the command stored in the transmission data is a command understandable by the target control device, and judgment of the judgment means If the result can be understood, the transmission data using the received command is transmitted to the target control device. If the result cannot be understood, the transmission data generated by converting the command into an understandable command based on the profile information is controlled by the target. An information conversion server comprising means for transmitting to a device is disclosed .

JP-A-2005-260893 Japanese Patent No. 5568067

  Incidentally, in a distributed system in which a plurality of devices (referred to as “nodes”) are connected via a communication network, a method for obtaining data held by another node from another node is, for example, Request / Reply (“Req / A method of acquiring data by communication), a method of acquiring data from a shared memory, a method of acquiring data by receiving a broadcast notification relating to Publish / Subscribe (referred to as “Pub / Sub”) communication, etc. There is. In this case, a certain node needs to acquire data from other nodes by selecting an appropriate data acquisition method from these.

  However, these data acquisition methods may be different in parameters used, processing when an error occurs, and the like. For example, an error processing method or the like may differ between a data acquisition method using Req / Rep communication, a data acquisition method using Pub / Sub communication, and a data acquisition method using a shared memory.

  Accordingly, an object of the present invention is to provide a distributed system, a data acquisition method, and a computer program that enable an application program to acquire data without being aware of a difference in error processing method that may occur due to a difference in data acquisition method. is there.

The distributed system according to the present embodiment includes a management device and a plurality of nodes.
The management device has management information including a data identifier for identifying data and a data acquisition method for data corresponding to the data identifier.
Node is
When receiving an acquisition command for requesting acquisition of data, including a data identifier and an error processing method representing a response method when an error occurs in the acquisition processing of data corresponding to the data identifier, from the application program,
Acquire management information related to the data identifier included in the acquisition command from the management device,
Based on the data acquisition method included in the acquired management information, execute the data acquisition process corresponding to the data identifier,
If an error occurs in the acquisition process, execute the error process based on the error processing method included in the acquisition command,
When the data corresponding to the data identifier is acquired by the acquisition process or the error process, the acquired data is returned to the application program.

  According to the present invention, it is possible for an application program to acquire data without instructing in detail a difference in error processing content that may occur due to a difference in data acquisition method.

2 shows an example of the configuration of a distributed system according to an embodiment. A configuration example of a node is shown. An example of the functional configuration of the access middleware is shown. The structural example of a management apparatus is shown. An example of the functional configuration of a management application is shown. A configuration example of property information is shown. The structural example of a management information table is shown. The structural example of a link table is shown. The structural example of a cache table is shown. The structural example of an update information table is shown. A configuration example of an acquisition command is shown. A configuration example of the acquisition command result is shown. An example of the result code is shown. The structural example of a query command is shown. The structural example of a query command result is shown. The structural example of the registration request message of management information is shown. The structural example of the management information registration response message is shown. The structural example of the acquisition request message of management information is shown. The structural example of the acquisition response message of management information is shown. The structural example of a data acquisition request message is shown. The structural example of a data acquisition response message is shown. The structural example of a Pub / Sub message is shown. The structural example of the management information deletion request message is shown. The structural example of the deletion response message of management information is shown. It is a sequence chart showing the example of the process in which a node requests | requires registration of management information to a management apparatus. It is a flowchart showing the example of the registration process of the management information in a management apparatus. It is a sequence chart showing the example of the process in which a node acquires data from a node. It is a flowchart showing the example of the acquisition process of the management information in access middleware. It is a flowchart showing the example of a process of the management apparatus which received the acquisition request message of management information. It is a flowchart showing the example of the data acquisition process in a node. It is a flowchart showing the example of the data acquisition process by Req / Rep communication in the node of a request side. It is a flowchart showing the example of the data acquisition process by Req / Rep communication in the node of a response side. It is a flowchart showing the example of the data acquisition process by the shared memory reference in a node. It is a flowchart showing the example of the data acquisition process by Pub / Sub communication in a node. It is a sequence chart showing the example of the process which a node performs the deletion request | requirement of management information with respect to a management apparatus. It is a flowchart showing the deletion process of the management apparatus which received the deletion request message of management information.

  Hereinafter, the distributed system according to the present embodiment will be described. In the following description, information may be described in terms of “xxx table” or “xxx list”, but the information may be expressed in any data structure. That is, in order to show that the information does not depend on the data structure, the “xxx table” or the “xxx list” can be called “xxx information”.

  Further, in the following description, a program is executed by a processor (for example, a CPU (Central Processing Unit)), so that a predetermined process is appropriately performed at least one of a storage resource (for example, a memory) and a communication interface device. Therefore, the subject of the processing may be a processor and a device having the processor. Part or all of the processing performed by the processor may be performed by a hardware circuit. The computer program may be installed from a program source. The program source may be a program distribution server or a storage medium (for example, a portable storage medium).

  Also, in the following description, when different types of elements are described separately, reference numerals are used such as “node 102a” and “node 102b”, and the same types of elements are not distinguished. In some cases, only a common number among the reference symbols is used, such as “node 102”.

  FIG. 1 shows a configuration example of a distributed system 1 according to an embodiment.

  The distributed system 1 includes a plurality of nodes 102 that are a kind of computer and a management apparatus 103 that is a kind of computer. Each node 102 and the management apparatus 103 can send and receive messages (data) to and from each other through the communication network 101. There may be any number of nodes 102 and management apparatuses 103.

  Each node 102 connected to the communication network 101 may have information for accessing the management apparatus 103 (for example, an IP (Internet Protocol) address of the management apparatus 103). The predetermined node 102 and the management apparatus 103 connected to the communication network 101 may be able to use Pub / Sub communication. In the Pub / Sub communication, when a certain node 102 broadcasts a message and the other node 102 that received the message includes the transaction ID (Identification) that the message has been waiting to be received. For example, the communication method is to acquire (accept) the message.

  FIG. 2 shows a configuration example of the node 102.

  The node 102 is, for example, a controller (control device), a workstation, or a personal computer. The node 102 includes hardware 240. The hardware 240 includes a storage unit 250, a processing unit 241, and a communication I / F (Interface) 242.

  The storage unit 250 stores various data. The storage unit 250 includes, for example, a volatile memory, a nonvolatile memory, a hard disk device, or the like.

  The processing unit 241 executes a computer program and realizes various functions of the node 102. The processing unit 241 includes, for example, a CPU (Central Processing Unit).

  The communication I / F 242 is an I / F for connecting the node 102 to the communication network 101. The communication I / F 242 includes circuits that control protocols such as a physical layer, a MAC (Media Access Control) layer, an IP layer, and a TCP (Transmission Control Protocol) / UDP (User Datagram Protocol) layer, for example.

  In the hardware 240 of the node 102, an OS (Operating System) 230, an access middleware 220, and one or two or more applications 210 whose processing contents may be different for each use are executed.

  The storage unit 250 includes an internal memory 251 that is a storage area in which data is read and written. A shared memory 253 may be configured in the internal memory 251. The location of the shared memory 253 in the internal memory 251 may be specified by a memory address.

  Data stored in the shared memory 253 may be accessible from a plurality of nodes 102. For example, when a certain node 102 stores the data notified via the communication network 101 in the shared memory 253, the other nodes 102 may be able to access the data stored in the shared memory 253. Each node 102 may be able to access data in the shared memory 253 by specifying a memory address. A plurality of shared memories 253 may exist. In this case, each node 102 may specify the access destination shared memory 253 using the shared memory ID assigned to the respective shared memory 253. Each data in the shared memory 253 may have an update time of the data.

  FIG. 3 shows an example of the functional configuration of the access middleware 220.

  The application 210 is configured by a computer program, and may have different functions depending on applications. The node 102 may execute a plurality of applications 210. The application 210 may acquire data through the access middleware 220. Further, the application 210 may acquire management information from the management apparatus 103 through the access middleware 220.

  The access middleware 220 of the node 102 has a processing function for registering management information related to data held by the node 102 in the management apparatus 103. The management information includes information necessary for a certain node 102 to access data held by another node 102. The node 102 acquires management information from the management apparatus 103 and accesses data held by the other nodes 102 based on the management information. That is, the access middleware 220 accesses data held by the other node 102 in place of the application 210. As a result, the application 210 can acquire the target data without being aware of the method of accessing the data held by the other node 102.

  The management information includes a data ID corresponding to the data to be acquired, a data acquisition method that is information necessary for accessing the data corresponding to the data ID, and data representing a position where the data corresponding to the data ID exists. Contains information such as location. Details of the management information will be described later.

  The type of data and the data acquisition method may have a correspondence relationship. For example, the data stored in the internal memory 251 of the other node 102 may have a correspondence relationship with the data acquisition method using Req / Rep communication. For example, the data stored in the shared memory 253 and the data acquisition method by referring to the shared memory may have a correspondence relationship. For example, the data included in the message sent by the other node 102 (Publish) and the data acquisition method by the Pub / Sub communication may have a correspondence relationship.

  The access middleware 220 functions as a command reception unit 221, a management information registration unit 222, a management information acquisition unit 223, a data acquisition request unit 224, a data acquisition response unit 225, a message transmission unit 226, and a message reception. Unit 227 and management information operation accepting unit 228.

  The access middleware 220 configures a management information table 342, property information 410, a cache table 252, and an update information table 254 in the internal memory 251 of the storage unit 250.

  The management information 342 includes information necessary for accessing each data held by the own node 102. Details of the management information 342 will be described later (see FIG. 7).

  The property information 410 includes information related to data attributes held by the own node 102. One piece of property information 410 may be associated with one piece of management information 342. Details of the property information 410 will be described later (see FIG. 6).

  Management information acquired from the management apparatus 103 is temporarily stored in the cache table 252. The cache table 252 may preferentially hold the management information acquired last. Details of the cache table 252 will be described later (see FIG. 9).

  The update information table 254 is a table for managing update information related to data in the shared memory 253. When the data acquisition request unit 224 acquires data from the shared memory 253, the data acquisition request unit 224 generates update information including the data position to be acquired, the size of the data, and the time stamp added to the data. Update information may be registered in the update information table 254. Thereby, the data acquisition request unit 224 may determine whether or not the data acquired from the shared memory 253 is updated with respect to the previously acquired data by referring to the update information table 254 from the next time. it can. For example, the data acquisition request unit 224 determines that the data has not been updated when the update time of the data to be acquired matches the time stamp of the data having the same data position and size as the acquisition target in the update information table 254. Judgment can be made. The data acquisition request unit 224 may be set not to acquire data that has not been updated from the shared memory 253.

  The message transmission unit 226 generates a message according to the information passed from each unit 222, 223, 224, 225 and the specified format, and transmits the message to the specified destination. Information and format passed from each unit may be determined for each process. Details will be described later.

  The message receiving unit 227 analyzes the received message and extracts a message type included in the message. Then, the message receiving unit 227 passes necessary information to each unit 222, 223, 224, 225 based on the message type. When the message reception unit 227 receives a transaction ID from the data acquisition request unit 224, the message reception unit 227 monitors reception of a Pub / Sub message (broadcast notification) having the transaction ID. When the message reception unit 227 detects reception of a Pub / Sub message having the transaction ID, the message reception unit 227 passes data included in the message to the data acquisition request unit 224.

  When the management information operation accepting unit 228 receives a management information registration instruction or deletion instruction from the application 210, the management information operation accepting unit 228 passes the instruction to the management information registration unit 222. Also, the management information operation accepting unit 228 may automatically pass a registration instruction for the management information 426 to the management information registration unit 222 when the node 102 is activated. This is because the management information 426 of the node 102 newly participating in the distributed system 1 is automatically registered in the management apparatus 103.

  The management information registration unit 222 requests the management apparatus 103 to register or delete management information. For example, when receiving a registration instruction from the management information operation accepting unit 228, the management information registration unit 222 generates a management information registration request message including the property information 410 and the management information, and sends the management information registration message to the message transmission unit 226. Instructs the management apparatus 103 to transmit an information registration request message. Then, the management information registration unit 222 receives the management information ID corresponding to the transmitted management information from the message reception unit 227 that has received the management information registration response message. The management information ID is information for identifying management information in the management apparatus 103. For example, when receiving a deletion instruction from the management information operation accepting unit 228, the management information registration unit 222 generates a management information deletion request message including the management information ID, and deletes the management information to the message transmission unit 226. Instructs the management apparatus 103 to transmit a request message.

  The command reception unit 221 serves as an interface that connects the application 210 and the access middleware 220. For example, the command reception unit 221 receives a data acquisition command from the application 210, extracts necessary information from the received acquisition command, and passes it to the data acquisition request unit 224. The command reception unit 221 receives the data acquisition command result from the data acquisition request unit 224 and passes the received acquisition command result to the application 210.

  The data acquisition request unit 224 performs processing related to data acquisition. The data acquisition request unit 224 receives an acquisition command including the data ID to be acquired from the command reception unit 221. Then, the data acquisition request unit 224 requests the management information acquisition unit 223 to acquire management information related to the data ID. Then, the data acquisition request unit 224 determines a data acquisition method based on the data acquisition method and data position included in the management information.

  For example, when the data acquisition method is “Req / Rep communication”, the data acquisition request unit 224 passes information such as the data position to the message transmission unit 226, and sends a data acquisition request message to the message transmission unit 226. Instructed to transmit to the other node 102 indicated by. When the message reception unit 226 receives a data acquisition response message from the other node 102, the data acquisition request unit 224 receives data and an execution result included in the data acquisition response message from the message reception unit 227.

  For example, when the data acquisition method is “reference shared memory”, the data acquisition request unit 224 acquires data, an execution result, and the like from the memory address of the shared memory indicated by the data position.

  For example, when the data acquisition method is “Pub / Sub communication”, the data acquisition request unit 224 instructs the message reception unit 227 to monitor reception of a Pub / Sub message having a transaction ID indicated by the data position. . When the message reception unit 227 receives a message having the transaction ID, the message reception unit 227 passes the data included in the message, the execution result, and the like to the data acquisition request unit 224. The data acquisition request unit 224 passes the acquired data and the like to the command reception unit 221.

  The management information acquisition unit 223 acquires management information from the management apparatus 103. The management information acquisition unit 223 passes the data ID received from the data acquisition request unit 224 and the version of the node 102 to the message transmission unit 226 and instructs transmission of the management information acquisition request message. Then, the management information acquisition unit 223 receives the management information included in the management information acquisition response message 713 from the message reception unit 227. Then, the management information acquisition unit 223 passes the management information to the data acquisition request unit 224. The management information acquisition unit 223 may store the management information 426 in the cache table 252.

  Note that the management information acquisition unit 223 may acquire the management information from the cache table 252 when the management information corresponding to the data ID exists in the cache table 252. That is, in this case, the management information acquisition unit 223 does not need to acquire management information from the management apparatus 103 via the communication network 101.

  The data acquisition response unit 225 performs processing related to a response when the node 102 receives a data acquisition request message from another node. When the data acquisition response unit 225 receives the data position from the message reception unit 227, the data acquisition response unit 225 acquires data from the internal memory 103 based on the data position. Then, the data acquisition response unit 225 instructs the message transmission unit 226 to return a data acquisition response message including the data to the transmission source (other node) of the data acquisition request message.

  FIG. 4 shows a configuration example of the management apparatus 103.

  The management device 103 includes hardware 260. The hardware 260 includes a storage unit 270, a processing unit 261, and a communication I / F 262. The storage unit 270, the processing unit 261, and the communication I / F 262 are the same as the storage unit 250, the processing unit 241, and the communication I / F 242 of the node 102 illustrated in FIG. In the hardware 260 of the management apparatus 103, the OS 231 and the management application 310 may be executed.

  FIG. 5 shows an example of the functional configuration of the management application 310.

  The storage unit 270 stores a link table 341 and a plurality of management information 342. The management information 342 is transmitted from each node 102. Details of the management information 342 will be described later (see FIG. 7). The link table 341 has information necessary for accessing the management information 342 managed by the management apparatus 103. Details of the link table 341 will be described later (see FIG. 8).

  The management application 310 includes a registration response unit 311, an acquisition response unit 312, a message transmission unit 226, and a message reception unit 314 as functions.

  The message receiving unit 314 analyzes the received message and extracts a message type included in the message. Then, the message receiving unit 314 passes necessary information to the units 311 and 312 based on the message type.

  The registration response unit 311 performs registration and deletion of property information and management information received from the node 102. Upon receiving the property information and management information from the message receiving unit 314, the registration response unit 311 stores them in the storage unit 270. Further, the registration response unit 311 updates the link table 341 so that the stored management information 342 can be accessed. Then, the registration response unit 311 issues a management information ID corresponding to the stored management information 342 and transmits a registration response message 512 including the management information ID to the node 102 that is the transmission source of the management information. .

  The registration response unit 311 receives the management information ID extracted from the management information deletion request message from the message reception unit 314. Then, the registration response unit 311 deletes the link information (record) having the management information ID from the link table 341 and deletes the link destination management information 342 of the link information (record). Then, the registration response unit 311 returns a management information deletion response message to the node 102 that has transmitted the deletion request message.

  The acquisition response unit 312 returns a management information acquisition response message corresponding to the management information acquisition request message to the source node 102 of the acquisition request message. For example, when receiving the data ID and version extracted from the management information acquisition request message 712 from the message reception unit 314, the acquisition response unit 312 extracts a record corresponding to the data ID and version from the link table 341. Then, the acquisition response unit 312 acquires management information from the link destination of the extracted record. Then, the acquisition response unit 312 instructs the message transmission unit 226 to return a management information acquisition response message including the acquired management information to the node 102 that is the transmission source of the acquisition request message.

  FIG. 6 shows a configuration example of the property information 410.

  The property information 410 is a type of additional information in the management information table 342 (see FIG. 7). One property information 410 may be associated with one management information table 342. The property information 410 includes information regarding the node 102 that generated the management information table 342. The property information 410 may include a node name 411, a version 412, and a compatible version 413 as information.

  The node name 411 represents the name of the node 102 that generated the management information table 342 associated with the property information 410. The version 412 represents the version of the node 102 when the management information table 342 is generated. The compatible version 413 represents the oldest version that can be backward compatible with the version 412 of the node 102 when the management information table 342 is generated.

  FIG. 7 shows a configuration example of the management information table 342.

  The management information table 342 has one or more management information as a record. The management information table 342 has management information necessary for accessing each data managed in the node 102. The management information table 342 of each node 102 may be aggregated in the management apparatus 103. The management information table 342 may have a data ID 421, a data acquisition method 422, a data position 423, a data type 424, and a size 425 as field values.

  The data ID 421 is information (for example, a file name) for identifying data. In the node 102, one data ID is associated with one data. The data acquisition method 422 represents a method for acquiring data corresponding to the data ID 421. The data position 423 represents information (parameters) necessary for acquiring data corresponding to the data ID 421 by the method indicated by the data acquisition method 422. The data type 424 represents the data type corresponding to the data ID 421. A size 425 represents the size of data corresponding to the data ID 421.

  For example, in FIG. 7, in order to obtain data corresponding to the data ID “Temp” (421), the management information of the record 420a uses the “Req / Rep communication” (422) to obtain the IP address “192.168.8.0. It is necessary to access the node 102 of “15” (423) and acquire data of “4 bytes” (425) with the data type “double” (424) from the memory address “0x0001” (423) of the memory of the node 102 It represents that there is.

  In order to obtain data corresponding to the data ID “Press” (421), the management information of the record 420c is “memory reference” to the memory address “0x0010” of the shared memory corresponding to the shared memory ID “8”. Indicates that it is necessary to acquire data of “4 bytes” with the data type “double”.

  In order to acquire data corresponding to the data ID “Alert” (421), the management information of the record 420d monitors reception of a Pub / Sub message corresponding to the transaction ID “125” by “Pub / Sub communication”. When the Pub / Sub message is received, it indicates that data of “60 bytes” with the data type “char” needs to be extracted from the Pub / Sub message.

  FIG. 8 shows a configuration example of the link table 341.

  The link table 341 has link information necessary for accessing each management information as a record. The management table 103 manages the link table 341.

  The link table 341 includes a management information ID 1706, a node name 411, a version 412, a compatible version 413, and a link destination 431 as field values.

  The management information ID 1706 is information for identifying management information. One management information ID may be given to one management information. When receiving and managing management information, the management application 310 may issue a management information ID 1706 corresponding to the management information. The node name 411, version 412, and compatible version 413 are as described in the property information 410 of FIG.

  The link destination 431 represents the position where the management information corresponding to the management information ID 1706 is stored. The link destination 431 may be a memory address where the management information is stored in the storage unit 250. The management application 310 can access desired management information by referring to the link table 341.

  FIG. 9 shows a configuration example of the cache table 252.

  The cache table 252 temporarily holds management information acquired from the management apparatus 103 by the access middleware 220 of the node 102.

  When acquiring the management information corresponding to the data ID 421 from the management apparatus 103, the access middleware 220 acquires the management information from the cache table 251 if the acquisition target management information is stored in the cache table 252. Thereby, the access middleware 220 can acquire the management information in a shorter time compared to the case where the management information is acquired from the management apparatus 103 through the communication network 101.

  The cache table 252 has a registration order 441, a data ID 421, a data acquisition method 422, a data position 423, a data type 424, and a size 425 as field values.

  The registration order 441 represents the order in which the management information is registered in the cache table 252. For example, the smaller the registration order 441, the older management information may be used. Further, the registration order 441 of the accessed management information may be updated to a new one.

  The data ID 421, the data acquisition method 422, the data position 423, the data type 424, and the size 425 are as described in the management information table 342 in FIG.

  An upper limit may be set for the number of management information that can be stored in the cache table 252. When new management information is stored in the cache table 252 in a state where the upper limit is reached, the oldest management information in the registration order 441 may be deleted.

  FIG. 10 shows a configuration example of the update information table 254.

  The update information table 254 includes information necessary for determining whether each data stored in the shared memory 253 has been updated. The update information table 254 includes a data position 1712, a size 425, and a time stamp 432 as field values.

  The data position 1712 represents a position where data is stored. For example, the data location 1712 is represented by an ID of a shared memory in which data is stored and a memory address in which data on the shared memory indicated by the shared memory ID is stored. The size 425 represents the size of data stored in the data position 1712 (for example, the number of bytes). The time stamp 432 represents the time when the data stored in the data position 1712 is generated or updated.

  FIG. 11 shows a configuration example of the acquisition command 711.

  The acquisition command 711 is a command used when the application 210 acquires data. The application 210 transmits an acquisition command 711 to the access middleware 220 when acquiring data. The access middleware 220 may provide the contents of the acquisition command 711 to the application 210 as an API (Application Programming Interface).

  The acquisition command 711 may include one or more data IDs 421 and an error processing method 1710 as parameters.

  The data ID 421 is a data ID corresponding to the acquisition target data. The data ID 421 may be a file name or a URI (Uniform Resource Identifier). When acquiring a plurality of data with one acquisition command 711, a plurality of data IDs 421 may be included in the acquisition command 711.

  The error processing method 1710 represents a response method when an error occurs in the data acquisition process corresponding to the data ID 421. The error processing method 1710 is used to determine what processing is performed when an error occurs.

  The error processing method 1710 may be configured with, for example, 4 bits of a retransmission flag 752, a temporary acquisition wait flag 753, an acquisition wait flag 754, and a data acquisition flag 755.

  The retransmission flag 752 indicates whether or not the data acquisition request message should be retransmitted when an error occurs in the data acquisition processing by Req / Rep communication. For example, the retransmission flag 752 “1” indicates that the data acquisition request message should be retransmitted. The retransmission flag 752 “0” indicates that it is not necessary to retransmit the data acquisition request message. The number of retransmissions may be set separately by the application 210, the access middleware 220, or the like.

  The temporary acquisition waiting flag 753 indicates whether or not to wait for a predetermined time to update the acquisition target data in the shared memory 253 when an error occurs in the data acquisition processing by the shared memory reference or Pub / Sub communication, or the Pub / Sub Indicates whether or not to wait for a predetermined time for receiving a message. For example, the temporary acquisition wait flag 753 “1” indicates that the update of data to be acquired in the shared memory 253 should be waited for a predetermined time or the reception of a Pub / Sub message should be waited for a predetermined period. The temporary acquisition wait flag 753 “0” indicates that there is no need to wait for the update of the acquisition target data in the shared memory 253 or no need to wait for the reception of the Pub / Sub message. The predetermined period to wait may be set separately by the application 210 or the access middleware 220.

  The acquisition wait flag 754 indicates whether or not to wait until the data to be acquired in the shared memory 253 is updated when an error occurs in the data acquisition processing by the shared memory reference or Pub / Sub communication, or a Pub / Sub message. Indicates whether or not to wait until receiving. For example, the acquisition wait flag 753 “1” indicates that it should wait until the data to be acquired in the shared memory 253 is updated or wait until it receives a Pub / Sub message. The acquisition wait flag 753 “0” indicates that it is not necessary to wait until the data to be acquired in the shared memory 253 is updated, or it is not necessary to wait until a Pub / Sub message is received. Unlike the temporary acquisition waiting flag 753, the acquisition waiting flag 754 does not require a deadline in the waiting period.

  The data acquisition flag 755 indicates whether or not the unupdated data should be acquired when an error indicating that the data has not been updated occurs in the data acquisition process based on the shared memory reference. For example, the data acquisition flag 755 “1” represents that data should be acquired even if the data has not been updated. The data acquisition flag 755 “0” represents that data should not be acquired when the data has not been updated.

  The access middleware 220 can execute appropriate error processing based on the content of the error processing method and the execution result included in the acquisition command result. For example, when the retransmission flag 752 of the error processing method is “1” and a connection error occurs in the Req / Rep communication, the access middleware 220 may try reconnection.

  FIG. 12 shows a configuration example of the acquisition command result 716.

  The acquisition command result 716 is data returned as the execution result of the acquisition command 711. The application 210 transmits an acquisition command 711 to the access middleware 220 and receives an acquisition command result 716 as a response. When providing the content of the acquisition command 711 as an API, the access middleware 220 may provide the content of the acquisition command result 716 as a return value of the API.

  The acquisition command result 716 includes one or more sets of a data ID 421, an execution result 1705, and data 1711 as parameters. That is, the number of sets included in the acquisition command result 716 corresponds to the number of data IDs 421 included in the acquisition command 711.

  Data 1711 is data corresponding to the data ID 421 included in the set. This data 1711 is acquired by the access middleware 220 based on the acquisition command 711.

  The execution result 1705 represents the result of the acquisition process of the data 1711 included in the set. The execution result 1705 may represent a result of the access middleware 220 executing data acquisition processing. The execution result 1706 may include a result code 750 and an acquisition time 751.

  The result code 750 is information (code) for identifying the result of executing the data acquisition process. A specific example of the result code 750 will be described later (see FIG. 13). The acquisition time 751 represents the time when acquisition of the data 1711 is completed.

  FIG. 13 shows an example of the result code 750.

The result code 750 and its meaning may be associated with each other as follows, for example.
The result code “0” means “data acquisition success”.
The result code “1” means “connection error related to Req / Rep communication”.
The result code “2” means “timeout related to Req / Rep communication”.
The result code “3” means “unupdated error related to shared memory reference”.
The result code “4” means “timeout related to Pub / Sub communication”.
The result code “5” means “data ID error”.
The result code “6” means “management information acquisition error”.
The result code “7” means “management information error”.

  FIG. 14 shows a configuration example of the query command 740.

  The query command 740 is used when the access middleware 220 performs data acquisition processing. The access middleware 220 may acquire one data with one query command 740. The query command 740 may include a data acquisition method 422, a data position 342, a size 425, a number limit 742, a time limit 743, and an error processing method 1710 as parameters. The data acquisition method 422, the data position 342, and the size 425 are as described in FIG.

  The number limit 742 represents the upper limit of the number of retransmissions. The deadline 743 represents the upper limit of the data update waiting time. The error processing method 1710 is as described in FIG. The number limit 742 and the time limit 743 may be predetermined set values or may be specified for each query command.

  FIG. 15 shows a configuration example of the query command result 741.

  The query command result 741 is data returned as the execution result of the query command 740. One query command result 741 may be returned for one query command 740. The query command result 741 includes, as parameters, a data acquisition method 422, a data position 342, a size 425, a number limit 742, a time limit 743, an error processing method 1710, data 1711, and an execution result 1705. It's okay. The data acquisition method 422, the data position 342, and the size 425 are as described in FIG. The number limit 742 and the time limit 743 are as described in FIG. The error processing method 1710 is as described in FIG.

  Data 1711 is data corresponding to the query command 740. An execution result 1705 represents a result of executing the query command 740.

  FIG. 16 shows a configuration example of the management information registration request message 511.

  The management information registration request message 511 is a message transmitted when the node 102 requests the management apparatus 103 to register management information. The management information registration request message 511 may include a header 1701, a message type 1703, property information 513, and management information 426 as field values.

  The header 1701 is information necessary for transmitting a message to a transmission destination via the communication network 101. For example, the header 1701 may include the IP address and port number of the transmission source and the transmission destination (destination).

  A message type 1703 represents the type of this message. For example, the message type 1703 includes a management information registration request message 511 and a management information registration response message 512. Management information acquisition request message 712, management information response request message 713, data acquisition request message 714, data acquisition response message 715, management information deletion request message 1611, management information deletion response message 1612 can be distinguished. May be defined.

  The property information 513 is as described in FIG. The management information 426 is as described in FIG. Property information 513 and management information 426 included in the management information registration request message 511 may be registered in the management apparatus 103. The management information 426 may be plural.

  FIG. 17 shows a configuration example of the management information registration response message 512.

  The management information registration response message 512 is a message returned from the management apparatus 103 to the node 102 as a response to the management information registration request message 713. The management information registration response message 512 includes a header 1701, a message type 1703, a registration result 514, and a management information ID 1706. The header 1701 and the message type 1703 are as described in FIG.

  The registration result 514 is information indicating a result (for example, success or failure) of the management apparatus 103 executing a process of registering the management information included in the management information registration request message 511.

  The management information ID 1706 represents the management information ID given to the management information 426 included in the management information registration request message 511 by the management apparatus 103. There may be a plurality of management information IDs 1706.

  FIG. 18 shows a configuration example of the management information acquisition request message 712.

  The management information acquisition request message 712 is a message transmitted when the node 102 requests management information from the management apparatus 103. The management information acquisition request message 712 includes a header 1701, a message type 1703, one or more data IDs 421, and a version 412. The header 1701 and the message type 1703 are as described in FIG.

  The data ID 421 represents the ID of data to be acquired. That is, the access middleware 220 of the node 102 requests management information regarding the data ID 421 included in the management information acquisition request message 712 from the management apparatus 103. When acquiring a plurality of management information, the access middleware 220 may include a plurality of data IDs 421 in the management information acquisition request message 712.

  The version 412 represents the version of the node 102 that transmits the management information acquisition request message 712. That is, the node 102 requests the management information of a version compatible with the version 412 from the management apparatus 103.

  FIG. 19 shows a configuration example of the management information acquisition response message 713.

  The management information acquisition response message 713 is a message returned from the management apparatus 103 to the node 102 as a response to the management information acquisition request message 712. The management information acquisition response message 713 may include, as field values, one or more sets of an execution result 1705, a data ID 421, and management information 426 following the header 1701 and the message type 1703.

  The execution result 1705 represents the result of the acquisition process of the management information 426 included in the set. The data ID 421 represents the data ID related to the management information 426 included in the set. That is, the number of data IDs 421 included in the management information acquisition request message 712 matches the number of sets included in the management information acquisition response message 713 (the number of data IDs 421).

  FIG. 20 shows a configuration example of the data acquisition request message 714.

  The data acquisition request message 714 is a message transmitted when the node 102 requests data from another node.

  The data acquisition request message 714 may include, as field values, a header 1701 and a message type 1703, one or more sets of a memory address 1712 and a size 425, and then an error processing method 1710. . The header 1701 and the message type 1703 are as described in FIG.

  The memory address 1712 represents the memory address of the storage destination of the acquisition target data. The size 425 represents the size of data to be acquired. When requesting a plurality of data, a set of a plurality of memory addresses 1712 and a size 425 may be included in the data acquisition request message 714. The error processing method 1710 is as described in FIG.

  FIG. 21 shows a configuration example of the data acquisition response message 715.

  The data acquisition response message 715 is a message returned from the other node to the node 102 as a response to the data acquisition request message 714. The data acquisition response message 715 may include one or more sets of an execution result 1705, a memory address 1712, a size 425, and data 1711 following the header 1701 and the message type 1703 as field values. The header 1701 and the message type 1703 are as described in FIG.

  The execution result 1705 represents the result of the data acquisition process in another node included in the set. A memory address 1712 represents a memory address where the data 1711 included in the set is stored. The size 425 represents the size of the data 1711 included in the set. Data 1711 is data requested by the data acquisition request message 714. That is, the number of sets included in the data acquisition request message 714 matches the number of sets included in the data acquisition response message 715.

  FIG. 22 shows a configuration example of the Pub / Sub message 1520.

  The Pub / Sub message 1520 is a message in which a certain node 102 broadcasts to a plurality of other nodes. The Pub / Sub message 1520 may have a header 1701, a transaction ID 1521, a size 425, and data 1711 as field values. The header 1701 is as described in FIG. Data 1711 is data carried by the Pub / Sub message 1520, and the size 425 represents the size of the data 1711.

  The transaction ID 1521 is information for identifying the Pub / Sub message 1520. The other node receiving the broadcast / pub message 1520 receives the transaction ID 1521 included in the message 1520 and is waiting for reception ( If the transaction ID matches the subscription (subscription), the Pub / Sub message 1520 is fetched.

  FIG. 23 shows a configuration example of the management information deletion request message 1611.

  The management information deletion request message 1611 is a message transmitted when the node 102 requests the management apparatus 103 to delete management information. The management information deletion request message 1611 may include a header 1701, a message type 1703, and a management information ID 1706. The header 1701 and the message type 1703 are as described in FIG.

  The management information ID 1706 represents the ID of management information to be deleted. That is, when the management apparatus 103 receives the management information deletion request message 1612, the management apparatus 103 performs processing for deleting the management information corresponding to the management information ID 1706.

  FIG. 24 shows a configuration example of the management information deletion response message 1612.

  The management information deletion response message 1612 is a message that the management apparatus 103 returns to the node 102 as a response to the management information deletion request message 1611. The management information deletion response message 1612 may include a header 1701, a message type 1703, and a deletion result 1613. The header 1701 and the message type 1703 are as described in FIG. The deletion result 1613 represents the result (for example, success or failure) of management information deletion processing in the management apparatus 103.

  FIG. 25 is a sequence chart illustrating an example of a process in which the node 102b requests the management apparatus 103 to register management information.

  When the node 102b joins the distributed system 1, the access middleware 220 of the node 102b starts the following processing. For example, the node 102b starts the following process when the node 102b is connected to the communication network 101 and can communicate with the management apparatus 103.

  The management information acquisition unit 223 of the node 102b prepares management information and property information related to the node 102b (S501). Then, the node 102b instructs the message transmission unit 226 to transmit a management information registration request message to the management apparatus 103 (S501).

  The message transmission unit 226 of the node 102b generates a management information registration request message 511 including the management information 426 and property information 513 regarding the node 102b, and transmits the management information registration request message 511 to the management apparatus 103 (S502). .

  Upon receiving the management information registration request message 511 from the node 102b, the management application 310 of the management apparatus 103 executes management information registration processing on the management information 426 and property information 513 included in the registration request message 511 (S600). ). Details of this processing will be described later.

  The management information acquisition unit 223 of the node 102b receives the management information registration response message 512 from the management apparatus 103 (S504). Then, the management information acquisition unit 223 determines the success or failure of the registration process based on the registration result 514 included in the registration response message 512 of the management information. When the registration result 514 is “success”, the management information acquisition unit 223 stores the management information ID 1706 included in the management information registration response message 512. When the registration result 514 is “failure”, the management information acquisition unit 223 notifies the application 210 of “failure”.

  The management information of each node 102 of the distributed system 1 is collected in the management device 103 by each node 102 performing the above-described processing.

  FIG. 26 is a flowchart illustrating an example of management information registration processing in the management apparatus 103. This process corresponds to the process on the management apparatus 103 side in S600 of FIG.

  When receiving the management information registration request message 511 from the node 102b (S601), the management application 310 of the management apparatus 103 extracts the management information 426 and property information 513 from the registration request message 511 (S602).

  The management application 310 determines whether or not the free space in the storage unit 270 is insufficient (S603). This is because management information or the like needs to be stored in the storage unit 270.

  If there is no shortage of free space (S603: NO), the management application 310 determines whether or not there is duplicated property information 513 received from the node 102b in the link table 341 (S604). That is, it is determined whether or not link information (record) overlapping with the node name 411 and the version 412 included in the received property information 513 exists in the link table 341.

  If there is no duplicate in the link table 341 (S604: NO), the management application 310 registers the management information 426 in the management information table 342 (S605). Then, the management application 310 issues a management information ID corresponding to the registered management information.

  Then, the management application 310 has a link having the management information ID, the node name 411, the version 412 and the compatible version 413 included in the property information 513, and a link destination indicating the management information table 342 where the management information 426 is registered. Information is generated and registered in the link table 341 (S606). Also, the management application 310 generates a management information registration response message 512 including a “successful” registration result 514 and a management information ID 1706. Then, the management application 310 proceeds to the process of S610.

  When the free space is insufficient (S603: YES), or when there is a duplicate in the link table 341 (S604: YES), the management application 310 responds to registration of management information including the registration result 514 of “failure”. A message 512 is generated (S608). Then, the management application 310 proceeds to the process of S610.

  In S610, the management application 310 transmits the management information registration response message 512 generated above to the node 102b (S610).

  Further, the management application 310 may broadcast the property information (node name, version, etc.) related to the registered management information to each node 102 on the communication network 101. Each node 102 that has received this broadcast notification may delete from the cache table 251 management information of the same node name as this broadcast notification and a version older than the one notified by broadcast notification ( S611).

  Through the above processing, the management apparatus 103 can return a response message 512 to the management information registration request message 511 to the node 102b.

  FIG. 27 is a sequence chart illustrating an example of processing in which the node 102a acquires data from the node 102b.

  The application 210 of the node 102 a generates an acquisition command 711 including the acquisition target data ID 421 and a desired error processing method 1710. Then, the application 210 passes the acquisition command 711 to the access middleware 220a (S701).

  The access middleware 220a of the node 102a executes management information acquisition processing based on the acquisition command 711 (S800). Details of this processing will be described later.

  The access middleware 220a generates a query command 740 based on the management information acquired by the management information acquisition process (S720). Then, the access middleware 220a executes data acquisition processing based on the query command 740 (S700). Details of this processing will be described later.

  The access middleware 220a generates an acquisition command result 716 based on the query command result 741 obtained by the data acquisition process (S730). Then, the access middleware 220a returns the acquisition command result 716 to the application 210 (S731).

  At this time, the access middleware 220a determines whether or not the query command 740 that is the target of the data acquisition process still remains (S732). If this determination is affirmative (S732: YES), the access middleware 220a executes data acquisition processing (S700) for the remaining query command 740. If this determination is negative (S732: NO), the access middleware 220a ends this process.

  Through the above processing, the application 210 of the node 102a can acquire the data 1711 from the node 102b. The application 210 may pass the next acquisition command 711 to the access middleware 220 before receiving the acquisition command result 716 for the acquisition command 711. That is, the access middleware 220 may process a plurality of acquisition commands 711 in parallel.

  FIG. 28 is a flowchart illustrating an example of management information acquisition processing in the access middleware 220a. This process corresponds to S800 in FIG.

  The access middleware 220a of the node 102a extracts the data ID 421 from the acquisition command 711 passed from the application 210 (S801). The access middleware 220a tries to acquire management information corresponding to the data ID 421 from the cache table 252 (S802). Then, the access middleware 220a determines whether there is management information that could not be acquired from the cache table 252 (S803).

  If there is no management information that could not be acquired from the cache table 252 (S803: NO), the access middleware 220a ends this process.

  If there is management information that could not be acquired from the cache table 252 (S803: YES), the access middleware 220a generates an acquisition request message 712 for management information related to the data ID 421 that could not be acquired. Then, the access middleware 220a transmits the management information acquisition request message 712 to the management apparatus 103 (S804). Then, the access middleware 220a waits for reception of the management information acquisition response message 713 until a timeout occurs (S807: NO, S808: NO).

  When a timeout occurs (S808: YES), the access middleware 220a generates an acquisition command result 716 including an execution result 1705 of the result code 750 “6” (management information acquisition error). Then, the access middleware 220a returns the acquisition command result 716 to the application 210 (S809), and ends this processing.

  When the management information acquisition response message 713 is received before the timeout occurs (S807: YES), the access middleware 220a extracts the management information 426 from the management information acquisition response message 713 (S805). The access middleware 220a registers the extracted management information in the cache table 251 (S806), and ends this process.

  Through the above processing, the access middleware 220a can acquire management information necessary for data acquisition.

  FIG. 29 is a flowchart illustrating an example of processing of the management apparatus 103 that has received the management information acquisition request message 712. This process corresponds to S800 in FIG.

  When receiving the management information acquisition request message 712 from the node 102a (S901), the management application 310 extracts the data ID 421 and the version 412 from the acquisition request message 712.

  The management application 310 searches the link table 341 for management information 426 having a version that is the same as or compatible with the extracted version and having the same data ID as the extracted data ID (S902).

  When such management information can be found (S903: YES), the management application 310 temporarily holds the set management information and the execution result result code “0” (success) as a set. Then (S904), the process proceeds to S906.

  When such management information could not be found (S903: NO), the management application 310 sets the management information “NULL” and the result set result code “5” (data ID error) as a set. The data is temporarily held (S905), and the process proceeds to S906.

  In S906, the management application 310 determines whether there is an unprocessed data ID among all the data IDs 421 received from the node 102a (S906). If there is an unprocessed data ID (S906: YES), the management application 310 returns to S902 and performs the same search as described above for the unprocessed data ID.

  When there is no unprocessed data ID (S906: NO), the management application 310 obtains management information including the data ID 421 received from the node 102a, the temporarily held execution result, and the management information 426. A response message 713 is generated. Then, the management application 310 returns the management information acquisition response message 713 to the node 102a (S907), and ends this processing.

  Through the above processing, the management apparatus 103 can return a response message 713 to the management information acquisition request message 712 to the node 102a.

  FIG. 30 is a flowchart illustrating an example of data acquisition processing in the node 102a. This process corresponds to the process of S700 in FIG.

  The access middleware 220a of the node 102a performs the following processing on the query command 740 generated in S720 of FIG. That is, the access middleware 220a classifies the query commands 740 by the data acquisition method 422 (S1101). Then, the access middleware 220a executes processing corresponding to the data acquisition method. At this time, the access middleware 220a may process a plurality of query commands 740 in parallel.

  The access middleware 220a executes the data acquisition process (S1200) by the Req / Rep communication for the query command 740 whose data acquisition method 422 is “Req / Rep communication”. Details of this processing will be described later.

  The access middleware 220a executes data acquisition processing (S1400) from the shared memory for the query command 740 whose data acquisition method is “reference shared memory”. Details of this processing will be described later.

  The access middleware 220a executes data acquisition processing (S1500) by Pub / Sub communication for the query command 740 whose data acquisition method is “Pub / Sub communication”. Details of this processing will be described later.

  The access middleware 220a generates an execution result of the result code “7” (management information error) for the query command 740 whose data acquisition method is unknown or not supported (S1107).

  The access middleware 220a that has completed the process of S1200, S1400, S1500, or S1107 refers to the execution result of each data acquisition process (S1102).

  The access middleware 220a determines whether an error is included in the execution result (S1103). If no error is included in the execution result (S1103: NO), the access middleware 220a proceeds to the process of S1108.

  When an error is included in the execution result (S1103: YES), the access middleware 220a performs the following error processing. The access middleware 220a compares the result code of the execution result with the error processing method 1710, and determines whether there is a combination that requires the data acquisition process to continue (S1104).

  The combination that requires continuation of the data acquisition process is, for example, a combination in which the result code is “1” (Req / Rep communication: connection error) and the retransmission flag 752 is “1”. Or, the result code is “2” (Req / Rep communication: timeout error) and the retransmission flag 752 is a combination of “1”. Alternatively, the result code is “3” (shared memory reference: unupdate error) and the temporary acquisition waiting flag 753 or the acquisition waiting flag 754 is a combination of “1”. Alternatively, the result code is “4” (Pub / Sub communication: timeout error) and the temporary acquisition waiting flag 753 or the acquisition waiting flag 754 is a combination of “1”.

  Next, when the retransmission flag 752 is “1”, the access middleware 220a sets the waiting time limit 743 if the number of retransmissions does not exceed the number limit 742, and if the temporary acquisition wait flag 753 is “1”. It is determined whether or not it has exceeded (S1105). When these limits are exceeded (S1105: NO), the access middleware 220a proceeds to the process of S1108.

  When these limits are not exceeded (S1105: YES), the access middleware 220a regenerates the query command 740 (S1106). At this time, the number of retransmissions and waiting time may be added. The regenerated query command 740 is added as the remaining query command 740 in S731 in FIG. 27 and re-executed.

  In S1108, the access middleware 220a determines whether there is an unprocessed data acquisition result (S1108). When there is an unprocessed data acquisition result (S1108: YES), the access middleware 220a executes the process from S1102 on the unprocessed data acquisition result. If there is no unprocessed data acquisition result (S1108: NO), the access middleware 220a ends this process.

  With the above processing, the node 102a can acquire data according to the data acquisition method 422 and the error processing method 1710 included in the query command 740.

  FIG. 31 is a flowchart illustrating an example of data acquisition processing by Req / Rep communication in the node 102a. This processing corresponds to the processing of the node 102a on the transmission side of the data acquisition request message 713 in S1200 of FIG.

  The access middleware 220a of the node 102a extracts the IP address and memory address of the other node 102b that is the data acquisition destination from the data position 342 of the query command 740. Then, the access middleware 220a tries to connect to the IP address of the other node 102b (S1201),

  When the connection is successful (S1201: YES), the access middleware 220a transmits a data acquisition request message 714 to the other node 102b (S1202). Then, the access middleware 220a proceeds to the process of S1210.

  When the connection fails (S1201: NO), the access middleware 220a temporarily holds the execution result of the result code “1” (request / response communication: connection error) (S1205). Then, the access middleware proceeds to the process of S1210.

  In S1210, the access middleware 220a determines whether or not there is an unprocessed query command 740 (S1210). When there is an unprocessed query command 740 (S1210: YES), the access middleware 220a executes the processing from S1201 for the unprocessed query command 740. If there is no unprocessed query command 740 (S1210: NO), the access middleware 220a proceeds to the next process of S1212.

  The access middleware 220a of the node 102a waits for reception of the data acquisition response message 715 from the other node 102b until a timeout occurs (S1212: NO, S1218: NO).

  When the data acquisition response message 715 is received (S1212: YES), the access middleware 220a extracts the data 1711 and the execution result 1705 from the data acquisition response message 715. Then, the access middleware 220a generates a query command result 741 including the extracted data 1711 and the execution result 1705, and temporarily stores it (S1215).

  The access middleware 220a determines whether there is an acquisition response message 715 for unreceived data (S1220). If there is an unreceived data acquisition response message 715 (S1220: YES), the access middleware 220a returns to the processing of S1212 and waits for the remaining reception. If there is no unreceived data acquisition response message 712 (S1220: NO), the access middleware 220a ends this process.

  When a timeout occurs in the standby for receiving the data acquisition response message 712 (S1218: YES), the access middleware 220a performs the following processing. That is, the access middleware 220a generates a query command result 741 having an execution result of the result code “2” (request / response communication: timeout error) and temporarily holds it (S1219). Then, the access middleware 220a ends this process.

  The query command result 741 temporarily held in the above is returned to the application 210 in S731 of FIG.

  FIG. 32 is a flowchart illustrating an example of data acquisition processing by Req / Rep communication in the node 102b. This processing corresponds to the processing of the node 102b on the side that receives the data acquisition request message 713 in S1200 of FIG.

  Upon receiving the data acquisition request message 714 from the node 102a (S1301), the access middleware 220b of the node 102b executes the following processing. That is, the access middleware 220b extracts the memory address 1712 and the size 425 from the data acquisition request message 714.

  The access middleware 220b determines whether data exists at the position indicated by the extracted memory address 1712 and size 425 (S1302). If no data exists (S1302: NO), the access middleware 220b temporarily holds an execution result indicating that no data exists (S1320), and proceeds to the process of S1306.

  When the data exists (S1302: YES), the access middleware 220a acquires the data from the position (S1303). Then, the access middleware 220a temporarily holds the execution result including the data ID, the result code “0” (success), and the acquisition time of the current time, and the acquired data (S1304).

  The access middleware 220a determines whether unacquired data exists (S1305). If unacquired data exists (S1305: YES), the access middleware 220a returns to the processing of S1203 and acquires unacquired data.

  When there is no unacquired data (S1305: NO), the access middleware 220a generates a data acquisition response message 715 including the execution result temporarily stored in the above, and returns the message 715 to the node 102a ( S1306). Then, the access middleware 220a ends this process.

  Through the above processing, the node 102 a that has received the data acquisition request message 713 can return an acquisition response message 714 corresponding to the acquisition request message 713.

  In the above processing example, if even one non-existent data is included in the data acquisition request message as a result of the determination in S1302, a data acquisition response message including an execution result indicating that fact is returned. However, the presence / absence of each data in the acquisition request message may be determined, and a data acquisition response message including the determination result regarding each data as an execution result may be returned.

  FIG. 33 is a flowchart illustrating an example of data acquisition processing by reference to the shared memory in the node 102a. This process corresponds to the process of S1400 in FIG.

  The access middleware 220a of the node 102a acquires the data location 423 (shared memory ID and memory address), data type, and size from the management information. Then, the access middleware 220a acquires data from the shared memory 253 using these pieces of information (S1401).

  The access middleware 220a confirms the update time of the acquired data (S1402), and determines whether or not this data has been updated (S1403). For example, the access middleware 220a, when the update time of this acquired data is newer than the time stamp 432 of the record corresponding to this data in the update information table 254 (or when this data is acquired for the first time), You may determine that it has been updated.

  If it is determined that the acquired data has been updated (S1403: YES), the access middleware 220a temporarily holds the acquired data and an execution result 1705 having the current time as the acquisition time 751 (S1408). ). Then, the access middleware 220a proceeds to the process of S1410.

  If it is determined that the acquired data has not been updated (S1403: NO), the access middleware 220a sets the data acquisition flag 755 of the error processing method 1710 to “1” (data should be acquired even if it has not been updated). It is determined whether or not (S1404).

  When the data acquisition flag 755 is “1” (data should be acquired even if not updated) (S1404: YES), the access middleware 220a sets the acquired data and the result code to “3” (shared memory). (Reference: Unupdate error) and the execution result 1705 in which the update time of the acquired data is the acquisition time 751 is temporarily held (S1405, S1408). Then, the access middleware 220a proceeds to the process of S1410.

  When the data acquisition flag 755 is “0” (data should not be acquired if it has not been updated) (S1404: NO), the access middleware 220a sets the data “NULL” and the result code to “3” (shared memory). The execution result 1705 that is referred to as an unupdate error is temporarily held (S1406). Then, the access middleware 220a proceeds to the process of S1410.

  In S1410, the access middleware 220a determines whether unacquired data remains (S1410). When unacquired data remains (S1410: YES), the access middleware 220a returns to the process of S1401 and acquires unacquired data. If unacquired data does not remain (S1410: NO), the access middleware 220a ends this process.

  Through the above processing, the node 102a can acquire data from the shared memory based on the error processing method.

  FIG. 34 is a flowchart illustrating an example of data acquisition processing by Pub / Sub communication in the node 102a. This processing corresponds to the processing of the node 102a on the side of receiving (subscribing) the broadcast notification (Pub / Sub message) in S1500 of FIG.

  When the data acquisition method 422 for management information acquired from the management apparatus 103 is Pub / Sub communication (S1501: YES), the access middleware 220a of the node 102a executes the following processing.

  The access middleware 220a of the node 102a extracts the transaction ID from the data position 423 of the management information. Then, the access middleware 220a starts waiting to receive the Pub / Sub message 1520 having the extracted transaction ID (S1502). Then, the access middleware 220a waits to receive this Pub / Sub message until a timeout occurs (S1503: NO, S1510: NO).

  When the Pub / Sub message 1520 having the extracted transaction ID is received (S1503: YES), the access middleware 220a acquires the data 1711 from the Pub / Sub message 1520 (S1504, S1505). The access middleware 220a temporarily holds the acquired data and an execution result 1705 in which the result code 750 is “0” (success) and the acquisition time 751 is the current time (S1506).

  Then, the access middleware 220a determines whether there is unreceived data (S1520). If unreceived data exists (S1520: YES), the access middleware 220a returns to S1503 and waits for reception again. If there is no unreceived data (S1520: NO), this process ends.

  When a timeout occurs (S1510: YES), the access middleware 220a temporarily holds “NULL” data and an execution result 1705 in which the result code 750 is “4” (Pub / Sub communication: timeout error). (S1511). Then, the access middleware 220a ends this process.

  Through the above processing, the node 102a can acquire the data 1711 from the Pub / Sub message 1520 having the transaction ID to be subscribed.

  FIG. 35 is a sequence chart illustrating an example of processing in which the node 102b issues a management information deletion request to the management apparatus 103.

  The access middleware 220b of the node 102b starts the following processing when the own node 102b leaves the distributed system 1 or the like.

  The access middleware 220b of the node 102b transmits a management information deletion request message 1611 including the management information ID to be deleted to the management apparatus 103 (S1601).

  When receiving the management information deletion request message 1611, the management application 310 of the management apparatus 103 specifies the management information ID 1706 included in the message 1611 and deletes the management information 426 corresponding to the specified management information ID 1706. Further, the management application 310 deletes the link information (record) having the specified management information ID 1706 from the link table 341. Then, the management application 310 transmits a management information deletion response message 1612 including a deletion result 1613 indicating the success or failure of the deletion process to the node 102b (S1700). The node 102b receives the management information deletion response message 1612 (S1603), and ends the process.

  Through the above processing, the management information corresponding to the management information ID requested to be deleted by the node 102b is deleted from the management apparatus 103.

  FIG. 36 is a flowchart showing the deletion process of the management apparatus 103 that has received the management information deletion request message. This processing corresponds to the processing on the management apparatus 103 side in S1700 of FIG.

  When receiving the management information deletion request message 1611, the management application 310 of the management apparatus 103 analyzes the deletion request message 1611 (S1701).

  The management application 310 determines whether or not link information (record) having a management information ID matching the management information ID 1706 included in the management information deletion request message 1611 exists in the link table 341 (S1702, S1703). .

  When there is link information (record) having the management information ID that matches the link table 341 (S1703: YES), the management application 310 deletes the management information stored in the link destination 430 of the link information (record). To do. Also, the management application 310 deletes the link information (record) from the link table 341 (S1704). Then, the management application 310 generates a management information deletion response message 1612 including a deletion result 1613 representing “success” (S1705), and proceeds to the processing of S1707.

  When there is no link information (record) having a management information ID that matches the link table 341 (S1703: NO), the management application 310 generates a management information deletion response message 1612 including a deletion result 1613 indicating “failure”. (S1706), the process proceeds to S1707.

  In S1707, the management application 310 returns the management information deletion response message 1612 generated above to the node 102b (S1707), and ends this processing.

  Through the processing described above, the management apparatus 103 can execute the management information deletion process requested to be deleted from the node 102b and return the result to the node 102b.

  The embodiments described above are examples for explaining the present invention, and are not intended to limit the scope of the present invention only to those embodiments. Those skilled in the art can implement the present invention in various other modes without departing from the gist of the present invention. For example, this embodiment may be expressed as follows.

(Expression 1)
A computer-readable recording medium that records a computer program, and when the computer program is executed by a processor of a node that constitutes a distributed system,
An acquisition command for requesting data acquisition from an application program executed by the processor, including a data identifier and an error processing method indicating a response method when an error occurs in the acquisition processing of data corresponding to the data identifier. , Receive through memory,
Management information related to the data identifier included in the acquisition command is acquired from a predetermined management device via a communication interface, the acquired management information is stored in the memory, the management information includes a data identifier, Including a data acquisition method for data corresponding to the data identifier,
Based on the data acquisition method included in the acquired management information, execute data acquisition processing corresponding to the data identifier,
When an error occurs in the acquisition process, an error process is executed based on the error processing method included in the acquisition command,
A computer-readable recording medium that records a computer program that returns the acquired data to the application program via the memory when the data corresponding to the data identifier is acquired by the acquisition process or the error process.

(Expression 2)
A management device belonging to a distributed system composed of a plurality of nodes,
Managing a plurality of management information including a data identifier for identifying data and a data acquisition method of data corresponding to the data identifier;
When receiving a management information acquisition request message including a data identifier from the node,
Specify management information corresponding to the data identifier included in the management information acquisition request message,
A management apparatus that returns a management information acquisition response message including the identified management information to the node.

(Expression 3)
A node constituting a distributed system,
When receiving an acquisition command for requesting acquisition of data, including a data identifier and an error processing method representing a response method when an error occurs in the acquisition processing of data corresponding to the data identifier, from the application program,
Obtaining management information related to the data identifier included in the acquisition command from the management device,
Based on the data acquisition method included in the acquired management information, execute data acquisition processing corresponding to the data identifier,
When an error occurs in the acquisition process, an error process is executed based on the error processing method included in the acquisition command,
A node that, when data corresponding to the data identifier is acquired by the acquisition process or the error process, returns the acquired data to the application program.

1: Distributed system 101: Communication network 102: Node 103: Management device

Claims (12)

A distributed system including a management device and a plurality of nodes,
The management device has management information including a data identifier for identifying data and a data acquisition method for data corresponding to the data identifier;
The node is
When receiving an acquisition command for requesting acquisition of data, including a data identifier and an error processing method representing a response method when an error occurs in the acquisition processing of data corresponding to the data identifier, from the application program,
Obtaining management information related to the data identifier included in the acquisition command from the management device,
Based on the data acquisition method included in the acquired management information, execute data acquisition processing corresponding to the data identifier,
When an error occurs in the acquisition process, an error process is executed based on the error processing method included in the acquisition command,
A distributed system that, when data corresponding to the data identifier is acquired by the acquisition process or the error process, returns the acquired data to the application program.
In the data acquisition method, a Request / Reply communication method, a Publish / Subscribe communication method, or a shared memory reference method is specified,
The Request / Reply communication method is a method of transmitting a data acquisition request message to another node specified by the management information and waiting for a response from the other node.
The Publish / Subscribe communication method is a method of waiting for reception of a message including a transaction identifier specified by the management information among messages issued from other nodes.
The distributed system according to claim 1, wherein the shared memory reference method is a method of acquiring data from a memory address specified by the management information in a memory space shared by a plurality of nodes.
The distributed system according to claim 2, wherein the error processing method has a setting related to data reacquisition and a setting related to data acquisition standby.
The node is
When the Publish / Subscribe communication method is specified as the data acquisition method and the error processing method is set to wait for data acquisition, a message including the transaction identifier cannot be received within a predetermined time. The distributed system according to claim 3, which waits until the message is received if an error occurs.
The node is
If the shared memory reference method is specified as the data acquisition method and the error processing method is set to wait for data acquisition, the data stored in the memory address of the shared memory is not updated. 4. The distributed system according to claim 3, wherein if an error indicating that the error occurs, the system waits until the data is updated.
The error handling method further includes a setting related to a data update time,
The node is
The data stored in the memory address of the shared memory when the shared memory reference method is specified as the data acquisition method and the error processing method is set not to consider the data update time 4. The distributed system according to claim 3, wherein if an error indicating that the data has not been updated occurs, the data stored in the memory address is returned to the application.
The node is
When a Request / Reply communication method is specified as the data acquisition method and the error processing method is set to re-acquire data, a response to a data request message is sent from the other node within a predetermined time. 4. The distributed system according to claim 3, wherein if an error indicating that the request has not been received has occurred, a data request message is retransmitted to the other node.
The node is
If data corresponding to the data identifier could not be acquired, an error message including a data acquisition method applied to the data acquisition process and information on an error that occurred in the data acquisition method is sent to the application. The distributed system according to any one of claims 1 to 7, which is returned.
The node transmits a management information registration request message to the management device, and the management information registration request message includes a data identifier corresponding to data held by the node,
When the management device receives the management information registration request message from the node, the management device returns a management information registration response message to the management information registration request message to the node. The distributed system according to any one of claims 1 to 7, wherein a management information identifier corresponding to management information related to a data identifier transmitted from a node is included.
The distributed system according to claim 9, wherein the node transmits a management information registration request message to the management device when communication with the management device is enabled.
A method in which each of a plurality of nodes constituting a distributed system obtains data,
When receiving an acquisition command for requesting acquisition of data, including a data identifier and an error processing method representing a response method when an error occurs in the acquisition processing of data corresponding to the data identifier, from the application program,
The management information related to the data identifier included in the acquisition command is acquired from a predetermined management device, the management information includes a data identifier and a data acquisition method of data corresponding to the data identifier,
Based on the data acquisition method included in the acquired management information, execute data acquisition processing corresponding to the data identifier,
When an error occurs in the acquisition process, an error process is executed based on the error processing method included in the acquisition command,
A data acquisition method for returning the acquired data to the application program when the data corresponding to the data identifier is acquired by the acquisition process or the error process.
A computer program executed on each of a plurality of nodes constituting a distributed system, and executed on the nodes,
When receiving an acquisition command for requesting acquisition of data, including a data identifier and an error processing method representing a response method when an error occurs in the acquisition processing of data corresponding to the data identifier, from the application program,
The management information related to the data identifier included in the acquisition command is acquired from a predetermined management device, the management information includes a data identifier and a data acquisition method of data corresponding to the data identifier,
Based on the data acquisition method included in the acquired management information, execute data acquisition processing corresponding to the data identifier,
When an error occurs in the acquisition process, an error process is executed based on the error processing method included in the acquisition command,
When the data corresponding to the data identifier is acquired by the acquisition process or the error process, the computer program returns the acquired data to the application program.

Images