JP2005242813A - Data communication system and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To establish a system capable of managing and using distributed data in large scale. <P>SOLUTION: A plurality of data management apparatuses are constituted of logical hierarchical structures having a plurality of hierarchies. When there is an access request to a data terminal 102 from a client 150, the access requests are sequentially transferred according to order of hierarchy among specified data management apparatuses out of the plurality of data management apparatuses and the access requests are made to reach the data terminal. Also, when access results with respect to the access requests are sent from the data terminal 102, the access results are sequentially transferred according to order of hierarchy among the specific data management apparatuses out of the plurality of data management apparatuses, and the access results are made to reach the client 105. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention mainly relates to a distributed data management system and method for managing distributed data by a computer connected to a network or a plurality of apparatuses to which the technology is applied.

As the scale of data managed by a data management system increases, a client / server configuration has been adopted in which a client that uses data and a server that manages data are separated. For example, in the resource monitoring system disclosed in Japanese Patent Laid-Open No. 2001-344227, resource information is collected from a central management station as a client that collects resource information of the entire system and individual management nodes having data to be monitored. It consists of a management station as a server that notifies the central management station.
JP 2001-344227 A

  Since the conventional distributed data management system is composed of at most two layers of client and server and does not support multi-level distributed data management, the size of distributed data that can be managed by the system is the size of the server that can be handled by the client. There is a problem that the scalability of the system is limited due to the size of distributed data that can be handled by the server.

  The present invention has been made to solve the above-described problems, and an object thereof is to construct a system capable of managing and using large-scale distributed data.

A data communication system according to the present invention includes:
A data holding device for holding data, and a plurality of data management devices for managing the data holding device,
A logical hierarchy is provided in the plurality of data management devices to connect the data holding device and the plurality of data management devices, and the data holding device and the plurality of data management devices have a plurality of hierarchies. A logical hierarchy,
Each of the plurality of data management devices includes:
In a predetermined case, a processing request for requesting a specific process is generated for a data management device and a data holding device belonging to a hierarchy lower than the own device in the hierarchical order, and the generated processing request is automatically generated in the hierarchical order. Sent to the data management device belonging to the hierarchy immediately below the device, and subsequently transfers the processing requests sequentially between the data management device and the data holding device belonging to the hierarchy lower than the own device in the hierarchy order according to the hierarchy order Thus, the data management device and the data holding device belonging to a lower hierarchy than the own device are caused to perform specific processing.

  According to the present invention, since the data holding device and the plurality of data management devices have a hierarchical configuration and processing requests are sequentially transferred according to the hierarchical order, large-scale distributed data can be managed, and if necessary By increasing the number of hierarchies, the size of data that can be managed can be expanded.

Embodiment 1 FIG.
FIG. 1 is a block diagram of a distributed data management system (data communication system) according to the present invention.

  In the figure, reference numerals 100 and 101 denote data, and reference numerals 102 and 103 denote data terminals (data holding apparatuses) that hold the data 100 and 101. Reference numerals 110 to 113 denote data management apparatuses that manage the data terminals 102 and 103. Reference numerals 120 to 123 denote data management devices that manage the lower data management devices including the data terminals 102 and 103 and the data management devices 110 to 113 under the control of the lower data management devices including the data management devices 110 to 113. 130 to 133 are subordinate to lower data management devices including the data management devices 120 to 123, and manage lower data management devices including the data terminals 102 and 103, the data management devices 110 to 113, and the data management devices 120 to 123. Data management device. 140 to 143 are subordinate to lower data management devices including the data management devices 130 to 133, and include the data terminals 102 and 103, the data management devices 110 to 113, the data management devices 120 to 123, and the data management devices 130 to 133. It is a data management device that manages a lower data management device. Reference numerals 150 and 151 denote clients that use distributed data. Reference numerals 160 to 163 denote networks that connect the data management apparatuses 110 to 113, 120 to 123, 130 to 133, 140 to 143, and clients 150 and 151. Reference numerals 170 to 174 denote access requests (processing requests) to the data 100, and reference numerals 180 to 184 denote flows of access results (processing responses) to the data 100.

  In this distributed data management system, data management devices 110 to 113, 120 to 123, 130 to 133, and 140 to 143 having equivalent functions are logically stacked via networks 160 to 163 as shown in the figure. It is characteristic that it takes a configuration. In other words, the distributed data management system according to the present embodiment has a data terminal (data holding device) that holds data and a plurality of data management devices that manage the data terminals, and logically includes a plurality of data management devices. Logical hierarchy configuration in which a data terminal (data holding device) and a plurality of data management devices are connected with a plurality of hierarchies, and the data terminal (data holding device) and the plurality of data management devices have a plurality of hierarchies. It is said.

  Next, the operation of the distributed data management system when the client 150 accesses the data 100 will be described. First, the client 150 issues an access request (processing request) 170 to the data 100 to the lower-level data management device 141 that manages the data terminal 102 under its control. When the data management device 141 receives the access request (processing request) 170, the data management device 141 transfers the access request (processing request) 171 to the lower-level data management device 131 that manages the data terminal 102.

  Thereafter, similarly, access requests (processing requests) 172 and 173 are sequentially transferred in the order of the data management apparatuses 121 and 111. When the data management apparatus 111 receives the access request (processing request) 173, it transfers the access request (processing request) 174 to the data terminal 102 having the data 100.

  When the data management device 111 receives a response to the access request (processing request) to the data 100 from the data terminal 102, that is, the access result (processing response) 180, the data management device 111 transfers the access result (processing response) 181 to the upper data management device 121. To do. Thereafter, similarly, access results (processing responses) 182 and 183 are sequentially transferred in the order of the data management devices 131 and 141.

  Eventually, the access result (processing response) 184 is transferred from the data management device 141 to the client 150 that is the request source of the access request (processing request), and the client sends the access result (processing response) 184 to the data 100. obtain. The client 150 can perform processing using the data 100 based on the access result (processing response) 184 (not shown).

  Although the access from the client 150 to the data 100 has been described above, the access target data is not limited thereto, and all data managed by a hierarchical configuration in which data management apparatuses are connected via a network can be set as access targets. As shown in the figure, the number of data management devices is four, and the lowest data management device manages two data terminals, and each data management device and each data terminal has one. In the case of a distributed data management system having data, a total of 4 × 4 × 4 × 4 × 2 × 1 = 512 data can be managed and accessed. In this case as well, as described above, when an access request (processing request) for data held by any of the data management apparatuses is issued from the client 150, an access request (processing request) between the data management apparatuses. Are sequentially transferred to arrive at an access request (processing request) to the target data management device, and when an access result (processing response) is transmitted from the data management device, the access result between the data management devices (Processing response) can be transferred sequentially, and the access result (processing response) can be made to reach the client 150 that is the request source of the access request (processing request).

  The number of data management devices in each tier, the number of tiers, and the number of data that each data terminal has are not limited to the above. For example, there are 100 data management devices in one tier, and it has a three-tier configuration. If the lowest data management apparatus manages 10 data terminals and each data terminal has 10 data, the total is 100 × 100 × 100 × 10 × 10 = 100,000,000 Individual data can be managed and accessed. In this case as well, as described above, when an access request (processing request) for data held by any of the data management apparatuses is issued from the client 150, an access request (processing request) between the data management apparatuses. Are sequentially transferred to arrive at an access request (processing request) to the target data management device, and when an access result (processing response) is transmitted from the data management device, the access result between the data management devices (Processing response) can be transferred sequentially, and the access result (processing response) can be made to reach the client 150 that is the request source of the access request (processing request).

  As described above, the distributed data management system (data communication system) according to the present embodiment includes a data terminal (data holding device) that holds data and a plurality of data management devices that manage the data terminal. The data terminal is provided with a logical hierarchy, the data terminal is connected to a plurality of data management apparatuses, and the data terminal and the plurality of data management apparatuses are configured in a logical hierarchy having a plurality of hierarchies. When there is an access request (processing request) to the terminal, the access request (processing request) is sequentially transferred between specific data management devices among a plurality of data management devices according to the hierarchical order. ) To the data terminal.

  Further, in the distributed data management system according to the present embodiment, when there is an access result (processing response) for an access request (processing request) from a data terminal, a specific data management device among a plurality of data management devices Then, the access results (process responses) are sequentially transferred according to the hierarchical order, and the access results (process responses) reach the request source of the access request (process request).

  In the distributed data management system according to the present embodiment, the process response is transferred through the same transfer path as the transfer path of the access request (process request).

  As described above, by adopting a hierarchical configuration in which data management apparatuses are connected via a network, large-scale distributed data can be managed and accessed. Moreover, the scale of data that can be managed and accessed can be expanded by increasing the number of hierarchies as necessary. In addition, since each data management device can be configured with devices having equivalent functions, it is not necessary to develop or manage separate devices for each hierarchy, which can reduce the cost of the distributed data management system. it can.

  The configuration of the distributed data management system is not limited to the configuration of FIG. In the figure, an example is shown in which each data management device and its lower level data management device are connected by independent networks. However, for example, they may all be connected to the same network, or a plurality of networks within a hierarchy. It may be constituted by. One or more clients may use the data.

  Moreover, the depth of the hierarchical structure may not be the same for all data, and there may be management across virtual directories. Further, there may be a configuration that serves both as a data management device and a data terminal, and the configuration is not limited to a configuration in which the data terminal is in the lowest layer.

  Further, the connection method between the data management device and the data terminal is not limited. It may be a bus or various networks. In addition, an example in which communication is performed using an access request (processing request) and an access result (processing response) between the data management device and the data terminal as in the case of the data management device is shown. There may be.

  Further, the types and contents of the data 100 and 101 of the data terminals 102 and 103 are not limited, and may be, for example, site data or data distributed over a wide area.

Embodiment 2. FIG.
Next, an example of the internal configuration of the data management apparatus (for example, 131) and its operation will be described with reference to FIG. Reference numerals 131, 161, 162, 171, 172, 182, and 183 are the same as those in FIG. Reference numeral 200 denotes a lower network interface that controls connection to the lower network 161, 201 denotes an upper network interface that controls connection to the upper network 162, 202 denotes a processor that controls data management, and 203 denotes information necessary for data management. A memory 204 is a bus that interconnects the lower network interface 200, the upper network interface 201, the processor 202, and the memory 203.

  The operation of the data management device 131 will be described.

  An access request (processing request) is transferred from the upper data management apparatus 141, and the upper network interface 201 receives the access request (processing request) 171 via the network 162. The upper network interface 201 stores the contents in the memory 203 via the bus 204 and notifies the processor 202 of the contents.

  Upon receiving the notification, the processor 202 reads the contents of the access request (processing request) 171 on the memory 203 via the bus 204 and processes the protocol, thereby making an access request (processing request) 172 to the lower data management apparatus 121. Is transferred to the lower-level network interface 200 via the bus 204 and the transmission of an access request (processing request) 172 is instructed.

  As a result, the lower network interface 200 reads the contents of the access request (processing request) 172 from the memory 203 via the bus 204, and accesses the lower data management apparatus 121 via the lower network 161. Send. The network 161 transfers the access request (processing request) 172 to the data management apparatus 121, and the data management apparatus 121 receives the access request (processing request) 172.

  Next, the operation in transferring the access results (processing responses) 182 and 183 will be described. The access result (processing response) is transferred from the lower data management apparatus 121, and the lower network interface 200 receives the access result (processing response) 182 via the network 161. The lower network interface 200 stores the contents in the memory 203 via the bus 204 and notifies the processor 202 of the contents.

  Upon receiving the notification, the processor 202 reads the contents of the access result (processing response) 182 on the memory 203 via the bus 204 and performs the protocol processing, whereby the access result (processing response) 183 is sent to the host data management device 141. Is transferred to the higher-order network interface 201 via the bus 204 and the transmission of the access result (processing response) 183 is instructed.

  As a result, the upper network interface 201 reads the contents of the access result (processing response) 183 from the memory 203 via the bus 204, and accesses the upper data management apparatus 141 via the upper network 162 to obtain the access result (processing response) 183. Send. The network 162 transfers the access result (processing response) 183 to the data management device 141, and the data management device 141 receives the access result (processing response) 183.

  The above is the basic operation of the data management apparatus (for example, 131). As an extension of this, the access result (processing response) can be cached. For example, after transferring the access result (processing response) 183 corresponding to the access request (processing request) 171, the contents are left as a cache on the memory 203, and the same access target as the access request (processing request) 171 later. When an access request (processing request) of the same type / content is received, instead of transferring the access request (processing request) to the lower-level data management apparatus 121, the contents stored in the cache are accessed as an access result (processing Response) to the upper data management apparatus 141.

  As described above, the access request (processing request) is transferred to the lower data management device and data terminal by returning the access result (processing response) based on the access result (processing response) cached in advance. Access performance can be improved.

  The cached content may be a snapshot (described later).

  Further, the data management apparatus is not limited to the configuration shown in FIG. 2. For example, the lower network interface 200 and the upper network interface 210 may be shared. The configuration of the data management apparatus is not limited to a hardware form, and may be a form such as a program that operates on a computer such as a server or a personal computer. In that case, the program of the data management apparatus may operate on a computer installed shared with other purposes, or the program of the data management apparatus may operate on a computer installed exclusively for data management.

Embodiment 3 FIG.
Next, communication protocols of access requests (processing requests) 171 and 172 and access results (processing responses) 182 and 183 will be described with reference to FIG. Reference numeral 300 denotes a communication protocol for an access request (processing request) or an access result (processing response), and includes information on the following. Reference numeral 301 denotes a destination data management device for an access request (processing request) or access result (processing response), 302 denotes a source data management device for an access request (processing request) or access result (processing response), and 303 denotes an access request (processing). Request) or the requesting client that requested access corresponding to the access result (processing response), 304 is an access identifier that identifies the access, 305 is the access type that indicates the type of access, and 306 is the virtual location of the access target, An access path (virtual access path information) indicating a virtual path for transferring an access request (processing request), 307 indicates an access indicating contents related to an access request (processing request) or an access result (processing response) as necessary. Content.

  The communication protocol 300 is the same for an access request (processing request) 171 received by the data management apparatus 131 via the upper network 162 and an access request (processing request) 172 transmitted via the lower network 161. . The communication protocol 300 is the same for the access result (processing response) 182 received by the data management apparatus 131 via the lower network 161 and the access result (processing response) 183 transmitted via the upper network 162. It is. That is, in each data management device, when transferring an access request (processing request), the data management device of the transfer destination is not changed without changing the data format of the access request (processing request) received from the data management device of the transfer source. The access request (processing request) is transferred to. In addition, each data management device, when transferring the access result (processing response), does not change the data format of the access result (processing response) received from the data management device of the transfer source, and does not change the data format of the transfer destination The access result (processing response) is transferred to.

  Next, protocol processing in which the data management apparatus 131 receives an access request (processing request) 171 and transmits an access request (processing request) 172 when the client 150 requests to read the data 100 will be described. When the data management device 131 receives the access request (processing request) 171 as described above, the processor 202 interprets the communication protocol 300 related to the access request (processing request) 171 and performs protocol processing.

  The communication protocol 300 of the received access request (processing request) 171 includes the following. Information indicating the data management device 131 as the transmission destination data management device 301, information indicating the upper data management device 141 as the transmission source data management device 301, and information indicating the client 150 as the request source client 303 are stored. A unique identifier related to access given by the client 150 as the access identifier 304, information indicating a read as the access type 305, and virtual location information of the distributed data 100 as the access path 306 are stored. The access contents 307 are empty because they are not necessary for reading.

  The processor 202 first confirms that information indicating its own device is stored in the transmission destination data management device 301, and then transmits the transmission source data management device 302, the request source client 303, the access identifier 304, the access type 305, and the access path 306. The information regarding is stored on the memory 203. This information is used later when an access result (processing response) corresponding to this access request (processing request) 171 is processed.

  Next, the virtual location information of the data 100 stored in the access path 306 is interpreted as follows to determine a lower-level data management device to which the access request (processing request) 172 should be transferred. The virtual location information is information indicating a path to the data 100 in a virtual name space shared by the entire data management system.

  For example, a path such as “abc / ghi / xyz / data1” is stored in the access path 306 in the access request (processing request) 171. Here, “/” indicates a delimiter of a directory that is a node in the data management hierarchical structure (the left side is an upper order), and “abc”, “ghi”, and “xyz” delimited by “/” are names of virtual directories. “Data1” on the rightmost (lowest) indicates a virtual name of the data 100 to be accessed.

  The processor 202 subtracts “ghi / xyz / data1” obtained by removing “abc” and “/” indicating the virtual directory managed by the own device from the path indicated by the access request (processing request) 171 as a lower-level data management. It is generated as an access path 306 to be stored in an access request (processing request) 172 to be transferred to the apparatus.

  Further, the destination data management device 301 to be stored in the access request (processing request) 172 is generated based on the virtual directory “ghi” to be managed by the lower-level data management device included in the path (the generation method is In addition, it is assumed that the transmission source data management device 302 is information indicating its own device. As other request source client 303, access identifier 304, access type 305, and access content 307, those included in the received access request (processing request) 171 are copied.

  In this way, the data management device 131 interprets / processes the access request (processing request) 171 received from the host data management device 141, and generates an access request (processing request) 172 using the same protocol as that. , It is transferred to the lower data management device 121. Similarly, the data management apparatus 121 interprets / processes the access request (processing request) 172 and transfers the access request (processing request) 173 to the lower data management apparatus 111.

  Next, protocol processing in which the data management apparatus 131 receives the access result (processing response) 182 and transmits the access result (processing response) 183 when transferring the read result of the data 100 to the client 150 will be described. As described above, when the data management apparatus 131 receives the access result (processing response) 182, the processor 202 interprets the communication protocol 300 related to the access result (processing response) 183 and performs protocol processing.

  The processor 202 first confirms that the information indicating its own device is stored in the transmission destination data management device 301, and then interprets the above access request (processing request) based on the information stored in the access identifier 304. / Identify information stored in the memory 203 during processing. Based on the information of the transmission source data management device in the stored access request (processing request) 171, the transmission destination data management device 301 and the access path 306 in the access result (processing response) 183 to be transferred to the upper level are generated. At the same time, the transmission source data management device 302 is set as information indicating its own device.

  In this way, the data management device 131 interprets / processes the access result (processing response) 182 received from the lower data management device 121, and generates an access result (processing response) 183 according to the same protocol. , It is transferred to the host data management device 141. Similarly, the data management device 141 interprets / processes the access result (processing response) 183 and transfers the access result (processing response) 184 to the upper client 150.

  Thus, in this embodiment, each data management device changes the data format of the access request (processing request) received from the data management device that is the transfer source when transferring the access request (processing request). The access request (processing request) is transferred to the transfer destination data management device, and the access result (processing response) data received from the transfer source data management device when the access result (processing response) is transferred. The access result (processing response) is transferred to the data management apparatus of the transfer destination without changing the format.

  As described above, the communication protocol of the access request (processing request) received by the data management device from the upper data management device and the access request (processing request) transferred to the lower data management device is the same protocol, and the lower data management Since the communication protocol of the access result (processing response) received from the device and the access result (processing response) transferred to the upper data management device is the same protocol, a multi-tier configuration can be achieved by using a plurality of equivalent data management devices. . Thereby, since a distributed data management system can be comprised using the data management apparatus which has an equivalent function, the expense concerning a distributed data management system can be reduced.

  Note that the communication protocol is not limited to the configuration shown in FIG. 3, and may have other configurations, and may include, for example, information related to an application program that accesses data or a user or group thereof. Further, the access request (processing request) and the access result (processing response) may be the same protocol or different protocols. The communication protocol may be implemented on a lower protocol such as TCP / IP which is an industry standard technology.

Embodiment 4 FIG.
A method for associating a virtual path related to access target data included in an access request (processing request) with location information of the data management apparatus and an access control method will be described with reference to FIG. For example, in the data management device 131, the access request (processing request) 172 is transferred to the access request (processing request) 172 based on the virtual directory “ghi” to be managed by the lower data management device included in the access path 306 of the access request (processing request) 171. A case where the transmission destination data management apparatus 301 to be stored is generated will be described as an example.

  In the figure, reference numeral 400 denotes directory management information, which includes the following information. 401 is a virtual directory, 402 is location information of the data management apparatus corresponding to the virtual directory 401, 403 is access control information to the data management apparatus corresponding to the virtual directory 401, and 404 is the access control information 403 from the client 150. Information relating to access, 405 indicates information relating to access from the client 151 in the access control information 403. The directory management information 400 is an example of association information that associates virtual access path information (virtual access path information) included in an access request (processing request) with location information of another data management apparatus.

  Next, the operation will be described. The directory management information 400 is managed by the processor 202, stored on the memory 203, and set in advance as follows. In the virtual directory 401, “.” Stores the name of the current virtual directory, “..” stores the name of the upper virtual directory, and other names of the lower virtual directory. The location information 402 stores the address (for example, IP address) of the data management device corresponding to the virtual directory 401. The access control information 403 stores the access authority (for example, read, write, snapshot permission) for each client with respect to the virtual directory 401.

  For example, when the access path 306 in the access request (processing request) 171 from the upper level is indicated by “abc / ghi / xyz / data1”, the processor 202 uses the directory management information 400 on the memory 203 as the name “ Search for rows with ghi ". As the location information 402 in this row, the address of the lower level data management apparatus 121 corresponding to the virtual directory “ghi” can be detected. This address is set in the transmission destination data management device 301 in the lower access request (processing request) 172.

  Further, the directory management information 400 can be used to control access to the virtual directory. In the access control information 403 in the above row, read, write, and snapshot are permitted as access from the client 150 to the virtual directory “ghi”, and access from the client 151 is prohibited.

  Therefore, in the access request (processing request) 171 from the host received by the data management apparatus 131, if the request source client 303 indicates the client 150 and the access type 305 indicates read, write, or snapshot access, the access is The access request (processing request) 172 is transferred to the lower level as described above.

  On the other hand, when the request source client 303 indicates the client 151, it is determined that access to the virtual directory “ghi” is prohibited, and a lower access request (processing request) 172 is not issued. As an access result (processing response) 183, an error indication is transmitted.

  As described above, the virtual directory included in the access request (processing request) can be associated with the location information of the data management apparatus. As a result, it is possible to identify a lower-level data management apparatus to which an access request (processing request) is to be transferred based on a virtual path indicated by a virtual name space shared by the entire data management system.

  As described above, in this embodiment, each data management device associates the virtual access path information (virtual access path information) included in the access request (processing request) with the location information of other data management devices. Directory management information (association information) to be stored, and when transferring an access request (processing request), virtual access path information (virtual access path information) included in the access request (processing request) and Based on the directory management information (association information), a transfer destination data management device is determined, and an access request (processing request) is transferred to the determined transfer destination data management device.

  In addition, permission or prohibition of access to the virtual directory can be controlled. Accordingly, access requests (processing requests) that are not permitted can be prohibited, and security can be improved.

  Note that the number of virtual directories managed by one data management apparatus is not limited to one. A single data management apparatus may manage a plurality of virtual directories, or a single data management apparatus may manage a plurality of virtual directories. One data management apparatus may have a plurality of directory management information.

  Also, the method of associating the virtual directory with the location information of the data management apparatus and the method of access control are not limited to FIG. For example, identification information in the data management apparatus may be included as location information. Moreover, although the example which has access control information in a client unit was shown, it is not limited to this, For example, the application program unit which processes data on a client, the user unit using it, or a user group unit etc. may be sufficient. In addition, although examples of access control are shown as read, write, and snapshot, they are not limited to this, and there may be other types such as erasure, or fewer types.

  The setting method of the directory management information is not limited. For example, the directory management information may be set to a fixed value, may be set via the network, or may be set by inquiring outside via the network. .

Embodiment 5 FIG.
Next, a case where the data management device causes the data management device and the data holding device belonging to a lower hierarchy than the own device in the hierarchical order to perform specific processing at a specific time point will be described. Specifically, the processing request is transferred between a plurality of lower specific data management devices and all data holding devices and data holding devices, and the data management devices and data belonging to the lower hierarchy than the own apparatus in the hierarchy order A case where the holding device is caused to perform snapshot processing for fixing and copying data held by each holding device will be described with reference to FIG. FIG. 5 shows logical connections and batches of all data management devices and data terminals in a hierarchy under a certain data management device (here, data management device 131) in the configuration of the distributed data management system shown in FIG. It is a figure shown about process control.

  Reference numerals 102, 103, 110-113, 120-123, and 131 are the same as in FIG. Reference numerals 500 to 529 denote data terminals, and reference numerals 530 to 541 denote data management apparatuses that manage data included in the data terminals 500 to 507 and 514 to 529. Reference numerals 550 to 553 denote batch processing control for data and its management information. Here, it is assumed that the data terminals 102, 103, 500 to 529 and their data can change dynamically.

  Next, snapshot processing is collectively performed at a specified time for all data under the virtual directory “abc” managed by the data management apparatus 131 from the client 150 illustrated in FIG. 1 via the data management apparatus 141. The operation will be described with reference to the case where the control is performed as an example.

  Until the client 150 issues an access request (processing request) for instructing batch processing control and reaches the data management device 131 via the data management device 141, the access request (processing request) in the above embodiment is used. The operations are the same as those in 170 and 171. An access request (processing request) 171 related to batch processing control that reaches the data management apparatus 131 corresponds to the batch processing control 550. In the communication protocol 300, a snapshot is specified as the access type 305, “abc / *” indicating all of the virtual directory “abc” and below is specified as the access path 306, and a time when the snapshot is specified as the access content 307. Other items are the same as the access request (processing request) 171.

  When the data management device 131 receives the batch processing control 550, the virtual directory managed by the device 150 from the client 150 is interpreted by interpreting information stored in the request source client 303, the access type 305, the access path 306, and the access content 307. It is recognized that batch processing control for taking a snapshot for all data below “abc” at a specified time is requested.

  The data management apparatus 131 refers to the directory management information 400 to perform lower-level batch processing control 551 based on the location information 402 corresponding to “def”, “ghi”, “jkl”, “mno” of the virtual directory 401. The addresses of the data management devices 120 to 123 to which the message is to be transmitted are specified. Furthermore, it is possible to confirm that snapshot is permitted for the access request (processing request) to the virtual directory from the client 150 based on the access control information 403.

  Next, an access request (processing request) in which the data management devices 120 to 123 are set in the transmission destination data management device 301 is created as the batch processing control 551. In each access request (processing request), the transmission source data management device 302 indicates the data management device 131 and the access path 306 indicates all of the virtual directory and below “def / *”, “ghi / *”, “jkl”. / * "And" mno / * "are set respectively. Other items are the same as the contents of the batch processing control 550.

  The data management device 131 issues access requests (processing requests) related to the batch processing control 551 created as described above to the data management devices 120 to 123, respectively. At the same time, the data management apparatus 131 is set to create a snapshot with the directory management information 400 fixed at the specified time.

  Next, the data management devices 120 to 123 that have received the batch processing control 551 perform the lower level data management devices 110 to 113, 530 in the same manner as the operation of the data management device 131 when the batch processing control 550 is received. The batch processing control 552 is issued to .about.541. At the same time, the data management apparatuses 110 to 113 and 530 to 541 are set to create a snapshot with the directory management information 400 fixed at the specified time.

  Next, the data management apparatuses 110 to 113 and 530 to 541 that have received the batch processing control 552 issue the batch processing control 553 to the lower data terminals 102, 103, and 500 to 529 in the same manner. At the same time, the data terminals 102, 103, and 500 to 529 are set to create a snapshot with the data fixed at the specified time.

  The operation related to the snapshot at the specified time will be described. In the data management devices 131, 120 to 123, 110 to 113, and 530 to 541 that are set to take a snapshot at a specified time by the batch processing control 550 to 552, when the specified time comes, the data management devices 131, 120 to 123, 110 to 113, respectively manage at that time. The directory management information 400 is fixed and copied. This copy is a snapshot of management information. Unpin after copying is completed.

  On the other hand, in the data terminals 102, 103, and 500 to 529 that are set to take a snapshot at the specified time by the batch processing control 553, the data at that time is fixed and copied at the specified time. This copy is a snapshot of the data. Unpin after copying is completed.

  In this way, the snapshot of the data and its management information at the specified time is stored in the distributed data management system while maintaining mutual consistency.

  This will be described with reference to FIG. Reference numerals 600, 610 to 613, and 620 to 635 are management information snapshots. 640 to 671 are data snapshots. Reference numerals 680 to 682 denote distributed snapshots.

  As described above, snapshots 600, 610 to 613, and 620 to 635 of management information are taken in the data management apparatuses 131, 120 to 123, 110 to 113, and 530 to 541 at a certain designated time. At the same time, data snapshots 640-671 are taken at the data terminals 102, 103, 500-529.

  A group of snapshots taken at the same designated time in the distributed data management system is a distributed snapshot, which is 680 at time t1, 681 at time t2, and 681 at time t3. 682. The mutual consistency is maintained between the management information of each of the distributed snapshots 680 to 682 and each snapshot of data.

  As described above, with regard to specific data in the virtual directory and its management information, conventionally, all of the control targets had to be controlled independently, but by providing a function for controlling batch processing, And can be controlled.

  Further, by managing the data and its management information with the distributed snapshot, it is possible to manage and store the data and the management information in a state where consistency between the distributed data and the management information is maintained.

  In other words, in a data management system that manages data of data holding devices by hierarchizing data management devices, it has a function to fix a specific part or all of the data lower than a certain level at a certain time section, thereby managing distributed data and its management Management / accumulation is possible in a state where consistency between information is maintained, so that consistent data can be provided for data analysis at the client, and accuracy can be improved.

  In addition, in a data management system for managing data in a data holding device by hierarchizing the data management device, a large number of distributed data management devices have a collective control function of a certain process with respect to a specific part or all lower than a certain hierarchy. The control for the data holding device can be simplified.

  In addition, although the example of the snapshot has been given as the batch process control process, the process is not limited thereto, and may be, for example, erasure. In addition, the example in which the target of batch processing is all under the virtual directory “abc” has been described, but the present invention is not limited to this. For example, specific data and its management information may be the target of batch processing control. Good.

  Moreover, although the example which controlled the snapshot by batch processing control was demonstrated, it is not limited to it, You may control a snapshot separately about specific data and its management information. Further, although an example of a snapshot at a specific time has been shown, the present invention is not limited to this. For example, a snapshot may be taken when a snapshot is requested or at a specified time interval. Further, the management information fixed by the snapshot is not limited to the above. Further, the use of the result of the snapshot is not limited.

Embodiment 6 FIG.
Next, a method in which data managed by the distributed data management system is handled by a client using the data will be described. FIG. 7 shows a logical configuration related to data management in the distributed data management system. Since 100, 101 and 150 are the same as those in FIG. Reference numeral 700 denotes an application program for processing data, reference numerals 701 and 702 denote file systems for file management, reference numeral 703 denotes a database system for database management, reference numeral 704 denotes a local volume, and reference numerals 705 to 708 denote virtual volumes. The following are the logical components in the virtual volume 706: 709 to 714 and 720 to 735 are virtual directories, and 740 to 769 are data.

  In the client 150, when the application program 700 accesses local data in the client 150, the client program 150 accesses through the file system 701 in the operating system (not shown). The file system 701 creates a local volume 704 as a logical management unit on a local physical device such as a hard disk, manages data, and shows it to the application program 700.

  The above is an operation of handling local data in a normal computer system by a file system. Next, an operation of handling data of a distributed data management system as shown in FIG. 1 by a file system will be described. When the application program 700 accesses data by the distributed data management system, the application program 700 accesses via the file system 702 as described above. The file system 702 shows it to the application as virtual volumes 705 and 706.

  Here, for example, the virtual volume 706 is not directly managed by the file system 702 but distributedly managed by the distributed data management system. However, for the application program 700, a virtual volume 706 having a virtual name space is provided with a hierarchical structure including virtual directories 709 and below and access to data 740 to 769 below the hierarchical structure.

  For example, a case where the application program 700 accesses data in the distributed data management system using a virtual path “vv1 / abc / ghi / xyz / data1” will be described. The virtual directory 709 named “vv1” is managed by the data management device 141, the virtual directory 710 named “abc” below it is managed by the data management device 131, and the virtual directory named “ghi” below it. 712 is managed by the data management device 121, the virtual directory 725 named “xyz” below it is managed by the data management device 111, and the data 100 named “data1” below it is in the data terminal 102.

  When an access request (processing request) 170 storing “vv1 / abc / ghi / xyz / data1” in the access path 306 is issued to the data management apparatus 141 under the control of the file system 702 of the client 150, The data management apparatus 141 transfers an access request (processing request) 171 storing “abc / ghi / xyz / data1” in the access path 306 to the data management apparatus 131. Subsequently, in the same manner as described in the above embodiment, the data management apparatus 131 stores an access request (processing request) 172 in which “ghi / xyz / data1” is stored in the access path 306 with respect to the data management apparatus 121. The data management apparatus 121 transfers an access request (processing request) 173 storing “xyz / data1” in the access path 306 to the data management apparatus 111, and the data management apparatus 111 transmits to the data terminal 102. An access request (processing request) 174 storing “data1” is transferred to the access path 306.

  Next, the data terminal 102 issues an access result (processing response) 180 to the data 100 to the data management apparatus 111. Hereinafter, the access result (processing response) in the reverse order of the path of the access request (processing request). ) Are finally transferred to the client 150 as 181, 182, 183, and 184. For example, when the application program 700 reads the data 100, the access result (processing response) 184 stores the read data of the data 100 in the access content 307 and is provided to the application program 700 via the file system 702. .

  As described above, the data managed by the distributed data management system is handled by the file system in the client that uses the data. Therefore, the application program looks like an ordinary local volume, and the application program is used to access the distributed data. There is no need to fix.

  In the above example, the data managed by the distributed data management system is handled by the file system in the client that uses it. However, the data managed by the distributed data management system depends on the database system in the client that uses it. The structure handled may be sufficient. For example, the database system 703 causes the application program 700 to show virtual volumes 707 and 708 managed by the distributed data management system as a database. Thus, the application program looks like a normal database, and there is no need to modify the application program to access the distributed database.

  In addition, the file system 702 or the database system 703 may handle the virtual volumes 705 to 708 including the snapshots and distributed snapshots described in the above embodiments. As a result, the distributed data can be shown to the user in a state where consistency with the management information is maintained.

Embodiment 7 FIG.
Next, the internal configuration of the data terminal (for example, 102) will be described with reference to FIG. Reference numerals 102, 174 and 180 are the same as those in FIG. Reference numeral 800 denotes a connection with a data management apparatus, reference numeral 801 denotes a data management apparatus interface, reference numeral 802 denotes a processor that performs data control, reference numeral 803 denotes a memory that holds information necessary for data control, and reference numeral 804 denotes a bus. The following is an example of an I / O (input / output) device for data 100: 810 is a control register, 811 is a disk, 812 is a meter, 813 is a camera as a kind of sensor, and 814 is a reader. In addition, a camera is an example of a sensor and is not limited to this.

  When there is an access request (processing request) to any I / O device from the client 150 to the data terminal 102, the data management device 141, the data management device 131, the data management from the client 150, as in the first embodiment. The access request (processing request) is sequentially transferred to the apparatus 121 and the data management apparatus 111, and the data terminal 102 exchanges data with the data management apparatus 111 via the connection 800 with the data management apparatus, and the data terminal 102 The data management device interface 801 receives an access request (processing request) 174 from the connection 800 with the data management device. The data management device interface 801 stores the contents in the memory 803 via the bus 804 and notifies the processor 802 of the contents.

  Upon receiving the notification, the processor 802 reads the contents of the access request (processing request) 174 on the memory 803 via the bus 804 and performs protocol processing, for example, the control register 810, the disk 811, the meter 812, the camera 813, or the reader 814. Access to any of the data is determined and access to them is executed.

  Next, the processor 802 creates contents to be issued as an access result (processing response) 180 on the memory 803 based on the access result, and then accesses the data management device interface 801 via the bus 804 ( Processing response) 180 is instructed.

  As a result, the data management device interface 801 reads the contents of the access result (processing response) 180 from the memory 803 via the bus 804 and accesses the upper data management device 111 via the connection 800 with the data management device ( Processing response) 180 is transmitted. The data management apparatus 111 receives the access result (processing response) 180 via the connection 800 with the data management apparatus. Thereafter, similarly to the first embodiment, access results (processing responses) are sequentially transferred from the data management device 111 to the data management device 121, the data management device 131, and the data management device 141, and finally to the client 150. To reach.

  Thus, in the present embodiment, the data terminal (data holding device) has at least one of a predetermined input device and an output device, and corresponds to at least one of the input device and the output device that the data terminal has. When there is an access request (processing request), the access requests (processing requests) are sequentially transferred according to the hierarchical order between specific data management devices among the plurality of data management devices. An access to at least one of the input device and the output device by the request source is made possible.

  Next, data of the data terminal 102 will be described. The control register 810 is a register that controls the operation of the data terminal 102 and various I / Os. By accessing the data in the control register 810 from the client 150, the data terminal 102 and various I / Os can be controlled and the status can be obtained from a remote location.

  The disk 811 is a medium for storing data in the data terminal 102. By accessing the data on the disk 811 from the client 150, the stored data of the data terminal 102 can be accessed from a remote location. The logical management of the disk 811 may be performed in the data terminal 102, the data management apparatus 111, or the file system 702 in the client 150.

  The meter 812 is, for example, a power meter, a gas meter, or a water meter that measures usage of household power, gas, or water. By accessing the data of the meter 812 from the client 150, the meter can be automatically read from a remote place. In addition, the kind of meter and measurement object are not limited to the above.

  The camera 813 is, for example, a surveillance camera installed in a home or store. Image data is output from the camera. By accessing the data of the camera 813 from the client 150, monitoring from a remote place becomes possible. In addition, a camera is an example of a sensor and is not limited to this.

  The reader 814 is a device that is installed in a store, a warehouse, or the like and reads identification data of a barcode or an electronic tag attached to a product. By accessing the data of the reader 814 from the client 150, identification data can be collected from a remote location. Note that the reader 814 may be capable of writing as well as reading the identification data.

  As described above, by accessing various I / O data included in the data terminal 102, control and information acquisition regarding the data terminal 102 and I / O, access to stored data, automatic meter reading of meters, remote monitoring, Identification data can be collected and used for various purposes.

  The configuration of the data terminal is not limited to the above. For example, the processor 802 may not have an access request (processing request) interpretation function or an access result (processing response) creation function. In that case, the configuration may be such that various I / Os are directly accessed from the data management device interface.

  Further, data held by the data terminal may be managed in the data terminal, may be managed by the data management device, or may be managed by the client.

  Further, the data held by the data terminal is not limited to the above. Further, any combination of I / O such as the above I / O and various sensors may be used.

  Here, the characteristics of the distributed data management system shown in the first to seventh embodiments will be described again below.

  The distributed data management system according to the first embodiment is a distributed data management system including a data terminal having data, a data management device including a computer or a device to which the technology is applied, and a network connecting them. The data terminal and the data management apparatus have a logical hierarchical structure having a plurality of hierarchies, and when there is a processing request, the processing requests are sequentially transferred according to the hierarchical order, and the data management apparatus belonging to the lower hierarchy The data terminal is caused to perform processing corresponding to the processing request.

  The distributed data management system according to the first embodiment is characterized in that the processing response is made to reach the request source of the processing request.

  The distributed data management system according to the first embodiment is characterized in that the processing response is transferred through the same transfer path as the transfer path of the processing request.

  The distributed data management system according to the second embodiment is characterized in that an access result (processing response) is cached in a data management device.

  In the distributed data management system according to the third embodiment, each of the plurality of data management devices transfers the processing request without changing the data format of the processing request received from the transfer source data management device. A processing request is transferred to the previous data management apparatus and data terminal.

  In the distributed data management system according to the third embodiment, each of the plurality of data management devices changes the data format of the processing response received from the transfer source data management device and the data terminal when the processing response is transferred. Instead, the processing response is transferred to the data management apparatus of the transfer destination.

  The distributed data management system according to the fourth embodiment includes virtual identification information (virtual access path information) related to access target data included in a processing request, and location information of a data management apparatus that manages the access target data under the control information. It is characterized by associating.

  The distributed data management system according to the fourth embodiment accesses a data or a data terminal or a data management apparatus for a client that uses data or an application program that processes data on the client or a user or a group of users that uses the application program. An access control function for permitting or prohibiting the access is provided.

  In the distributed data management system according to the fifth embodiment, the data management device sends a specific time (on request or at a specified time or a specified time) to a data management device and a data terminal that belong to a hierarchy lower than the own device in the hierarchical order. Specific processing is performed at intervals).

  In the distributed data management system according to the fifth embodiment, each of a plurality of data management devices is held in a data management device and a data terminal that belong to a lower hierarchy than the own device in a hierarchical order as a specific process. It is characterized in that snapshot processing for copying data is fixed.

  In the distributed data management system according to the fifth embodiment, each of the plurality of data management devices causes a processing request to be transferred between a plurality of lower specific data management devices and data terminals. .

  The distributed data management system according to the sixth embodiment is characterized in that data managed by the distributed data management system is handled by a file system in a client that uses the data.

  The distributed data management system according to the sixth embodiment is characterized in that data managed by the distributed data management system is handled by a database system in a client that uses the data.

  In the distributed data management system according to the sixth embodiment, the file system or the database system manages the result of snapshot processing.

  The distributed data management system according to the seventh embodiment is characterized in that control or information acquisition related to a data terminal and its I / O (input / output) is performed based on data held by the data terminal.

  In the distributed data management system according to the seventh embodiment, the I / O is a meter that indicates a usage amount of electric power, gas, or water.

  In the distributed data management system according to the seventh embodiment, the I / O is a sensor such as a camera for monitoring.

  In the distributed data management system according to the seventh embodiment, the I / O is a reader that reads identification data such as a barcode or an electronic tag.

  The distributed data management method according to the first embodiment is a distributed data management system including a data terminal having data, a data management device including a computer or a device to which the technology is applied, and a network connecting them. The data terminal and the data management apparatus have a logical hierarchical structure having a plurality of hierarchies, and when there is a processing request, the processing requests are sequentially transferred according to the hierarchical order, and the data management apparatus belonging to the lower hierarchy The data terminal is caused to perform processing corresponding to the processing request.

1 is a diagram illustrating a configuration example of a distributed data management system according to a first embodiment. The figure which shows the internal structural example of a data management apparatus. The figure which shows the example of the communication protocol of an access request and an access result. The figure which shows the example of directory management information. The figure shown about the example of batch processing control. The figure shown about the example of a snapshot. A diagram showing a logical configuration example related to data management The figure which shows the internal structural example of a data terminal.

Explanation of symbols

  100 Data, 101 Data, 102 Data Terminal, 103 Data Terminal, 111 Data Management Device, 121 Data Management Device, 131 Data Management Device, 141 Data Management Device, 150 Client, 151 Client 200 Lower Network Interface, 201 Upper Network Interface, 202 Processor 203 memory 204 bus 700 application program 701 file system 702 file system 703 database system 704 local volume 705 virtual volume 706 virtual volume 707 virtual volume 708 virtual volume 801 data management device interface 802 processor, 803 memory, 804 bus, 810 control level Star, 811 disc, 812 meter, 813 camera, 814 reader.

Claims (18)

A data holding device for holding data, and a plurality of data management devices for managing the data holding device,
A logical hierarchy is provided in the plurality of data management devices to connect the data holding device and the plurality of data management devices, and the data holding device and the plurality of data management devices have a plurality of hierarchies. A logical hierarchy,
Each of the plurality of data management devices includes:
In a predetermined case, a processing request for requesting a specific process is generated for a data management device and a data holding device belonging to a hierarchy lower than the own device in the hierarchical order, and the generated processing request is automatically generated in the hierarchical order. Sent to the data management device belonging to the hierarchy immediately below the device, and subsequently transfers the processing requests sequentially between the data management device and the data holding device belonging to the hierarchy lower than the own device in the hierarchy order according to the hierarchy order A data communication system, characterized by causing a data management device and a data holding device belonging to a lower hierarchy than the own device to perform specific processing. Each of the plurality of data management devices includes:
2. The data communication system according to claim 1, wherein the data management apparatus and the data holding apparatus belonging to a hierarchy lower than the own apparatus in the rank order are caused to perform the specific processing at a specific time. Each of the plurality of data management devices includes:
As the specific process, the data management apparatus and the data holding apparatus belonging to the lower hierarchy than the own apparatus in the hierarchy order are caused to perform a snapshot process for fixing and copying the data held by each of them. The data communication system according to claim 1. Each of the plurality of data management devices includes:
The data communication system according to claim 1, wherein a processing request is transferred between a plurality of lower-level specific data management devices and all data holding devices. The data communication system includes:
When there is a processing request for the data management device and the data holding device, the processing request is sequentially transferred in accordance with the hierarchical order between specific data management devices of the plurality of data management devices, and the processing request is transferred to the data The management device and the data holding device, and when there is a processing response to the processing request from the data management device and the data holding device, the hierarchy of the specific data management devices among the plurality of data management devices 2. The data communication system according to claim 1, wherein the processing responses are sequentially transferred according to the order, and the processing responses reach the request source of the processing request. The data communication system includes:
6. The data communication system according to claim 5, wherein the process response is transferred through the same transfer path as the process request transfer path. Each of the plurality of data management devices includes:
6. The data communication system according to claim 5, wherein when the processing response is received, the received processing response is cached. Each of the plurality of data management devices includes:
When transferring a processing request, the processing request is transferred to the destination data management device and the data holding device without changing the data format of the processing request received from the transfer source data management device. Item 4. The data communication system according to Item 1. Each of the plurality of data management devices includes:
When transferring a processing response, the processing response is transferred to the destination data management device without changing the data format of the processing response received from the data management device and the data holding device of the transfer source. Item 6. The data communication system according to Item 5. The processing request to the data management device and the data holding device includes virtual access path information indicating a virtual access path for transferring the processing request to the data management device and the data holding device,
Each of the plurality of data management devices includes:
Stores association information that associates the virtual access path information included in the processing request with the location information of other data management devices and data holding devices, and the virtual access included in the processing request when the processing request is transferred A transfer destination data management device and a data holding device are determined based on path information and the association information, and a processing request is transferred to the determined transfer destination data management device and the data holding device. The data communication system according to claim 1. The data communication system includes:
The request source of the processing request for the data management device and the data holding device can manage the virtual access path information for accessing the data management device and the data holding device in the file system of the processing request request source. The data communication system according to claim 1. The data communication system includes:
The request source of the processing request for the data management device and the data holding device can manage the virtual access path information for accessing the data management device and the data holding device in the database system of the processing request request source. The data communication system according to claim 1. The data holding device has at least one of a predetermined input device and an output device,
When there is a processing request for at least one of the input device and the output device of the data holding device, the processing request is sequentially transferred between specific data management devices of the plurality of data management devices according to the hierarchical order. The data communication system according to claim 1, wherein access to at least one of an input device and an output device by a request source of a processing request is made possible. The data holding device has at least one of a meter and a sensor,
When there is a processing request for at least one of the meter and sensor of the data holding device, the processing request is sequentially transferred according to the hierarchy of the hierarchy among specific data management devices of the plurality of data management devices, The data communication system according to claim 1, wherein access to at least one of a meter and a sensor by a request source of a processing request is enabled. The data holding device has a data reader,
When there is a processing request for a data reader included in the data holding device, the processing request is sequentially transferred between specific data management devices of the plurality of data management devices according to the hierarchical order, and the request source of the processing request The data communication system according to claim 1, wherein access to the data reader is enabled. A data communication method for performing data communication between a data holding device that holds data and a plurality of data management devices that manage the data holding device,
A logical hierarchy is provided in the plurality of data management devices to connect the data holding device and the plurality of data management devices, and the data holding device and the plurality of data management devices have a plurality of hierarchies. A logical hierarchy,
In a predetermined case, the specific data management device generates a processing request for requesting a specific processing to the data management device and the data holding device belonging to a lower hierarchy than the specific data management device in the hierarchy order, The generated processing request is transmitted to the data management apparatus belonging to the hierarchy immediately below the specific data management apparatus in the hierarchy order, and thereafter belongs to the hierarchy lower than the specific data management apparatus in the hierarchy order Processing requests are sequentially transferred between the data management device and the data holding device according to the hierarchical order, and the data management device and the data holding device belonging to a lower hierarchy than the specific data management device perform specific processing. A data communication method characterized by the above. Each of the plurality of data management devices includes:
4. The data communication system according to claim 3, wherein data is managed in units of distributed snapshots in which a group of snapshots at the same point in time are grouped.   The data communication system according to claim 1, wherein the data management device or the data holding device has a function of controlling permission / prohibition of specific processing.

Images