JP2006350470A - Data management device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data management device capable of efficiently distributing resources, securing safety and reliability in data files, and raising access performance. <P>SOLUTION: A data processing system performs access via a network to the plurality of data files which are respectively recorded in a plurality of recording media respectively included in a plurality of nodes. The system comprises: a replicating part 106 for generating the replication files of the data files; a storage destination selecting part 128 for selecting a storage destination node for storing the replication file in response to a prescribed condition; and a storage part 120 for storing the replication files in the selected storage destinations. When the data file is replicated and the replication file is stored in the storage destination, a file for management including path information of the replication file is generated. Then the access is performed to the data file recorded in the recording medium, based on the path information which is included in the file for management in a hierarchical structure having a plurality of directories. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a data management apparatus and method, and more particularly to a data management apparatus and method for managing data recorded in a distributed manner on a plurality of nodes via a network.

  In recent years, with the advancement of computer technology, its performance has improved dramatically. Furthermore, network technology continues to develop, and high-speed and large-capacity communication has become possible over a wide range. Against this background, grid technology has attracted attention as one of the technologies for virtualizing, integrating, and efficiently using a plurality of resources on a network. This grid technology makes it possible to use these resources simply by connecting to the network without making the user aware of the locations and owners of the resources distributed on the wide area network. If grid technology is used, data distributed and stored in many storages connected to the network can be used uniformly.

For example, Patent Document 1 discloses a data management apparatus that makes it possible to access recording media of a plurality of nodes on a network using such grid technology. The data management apparatus described in the document makes it possible to use these data files uniformly without making the user aware of the difference in storage in which a plurality of data files are stored.
JP-A-2005-63214

  By the way, if the data file stored in the recording medium of the node on the network as described above exists only in one place, there is a risk that important information may be completely lost due to a disaster or a failure. Furthermore, when access to certain data files is concentrated, the processing speed and communication speed are reduced. Also, if the access frequency is concentrated on a recording medium or data file, there is a risk that the life of the recording medium will be shortened.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a data management apparatus capable of efficiently distributing resources, ensuring safety and reliability of data files, and improving access performance. Is to provide.

According to the present invention, an access unit that accesses a plurality of data files respectively recorded on a plurality of recording media included in a plurality of nodes via a network;
A generating unit that generates a plurality of management files each including path information for accessing the plurality of data files;
In order to manage the plurality of data files, the management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. A management department to
A duplicating unit for creating a duplicating file of the data file;
A storage destination selection unit for selecting a storage destination node for storing the duplicate file according to a predetermined condition;
A storage unit for storing the duplicate file in the storage destination selected by the storage destination selection unit;
With
The generation unit, when the replication unit replicates the data file, and the storage unit stores the replication file in the storage destination, generates the management file including path information of the replication file,
The access unit accesses the data file recorded on the recording medium based on the path information included in the management file in the hierarchical structure including the plurality of directories. A data management device is provided.

  Here, the data file includes information used by a plurality of resources for calculation processing in response to a calculation request from the client and the calculation processing result. The recording medium includes various media such as a floppy (registered trademark) disk, CD, DVD, MO, magnetic tape, USB memory, and various memory cards, and a storage device such as a hard disk device.

  The path information includes a network address such as an IP (Internet Protocol) address, an address in a recording medium, and an access means for accessing a data file, for example, a protocol such as http (HyperText Transfer Protocol) or ftp (File Transfer Protocol). However, the present invention is not limited to this, and it is sufficient that the position of the data file can be specified. For example, it can be expressed by URL (Uniform Resource Locators) or EPR (WS-Addressing EndPoint Reference).

  The hierarchical structure may be managed by another file. However, the present invention is not limited to this, and information such as the upper and lower directories is added to a part of the management file or directory to logically. It may be formed only.

  According to the present invention, since a copy of a data file can be distributed and stored in a recording medium of a plurality of nodes on a network according to a predetermined condition, it is possible to efficiently secure resource safety and reliability and access performance. Can be improved.

  In the data management device, an attribute information acquisition unit that acquires attribute information of the data file to be replicated by the replication unit, and determines the replication number of the replication file according to a predetermined replication number determination condition based on the attribute information A replication number determination unit.

  Here, the attribute information of the data file is, for example, attributes such as a file size, a generation time, an update time, and an owner. Furthermore, the type of stored information can also be included. In other words, the type of stored information is classified and ranked according to the importance of the file that indicates whether it contains information that would not be lost, personal information, copyright, confidential information, necessary safety indicators, popularity, etc. be able to. The attribute information may be acquired and used every time it is copied, or information acquired and stored periodically or as needed may be used.

  According to this configuration, the number of replicas can be determined according to the attribute of the data file and the type of file storage information. Therefore, it is possible to efficiently ensure the safety and reliability of resources and improve the access performance. Become.

  In the data management apparatus, the storage destination selection unit, based on the performance information, a performance information acquisition unit that inquires and acquires performance information indicating the performance of the recording medium of the plurality of nodes via the network, And a selection unit that selects the storage destination node that satisfies a predetermined storage condition.

  Here, the performance information is, for example, the free capacity of the recording medium of the node, the access performance, the performance of the access CPU, the network installation location with the user, and the like. Such performance information can be acquired, for example, by making an inquiry over a network to a data processing device that performs data processing using the recording medium if the recording medium has free space. The access performance can be acquired by actually accessing the recording medium of the node via the network, measuring the response time. Also, the network location with the user can be acquired by transmitting an inquiry signal from the user's device via the network and measuring the response time. The selection unit selects a recording medium of a node whose performance information satisfies a predetermined condition as a storage destination.

  The condition for the performance information can be a combination of a plurality of conditions or a priority order. Furthermore, the storage destination can be determined on the condition of the attribute of the data file described above. The performance information may be acquired and used every time it is copied, or information acquired and stored periodically or as needed may be used.

  According to this configuration, the storage destination can be selected according to the predetermined storage condition, so that the copy of the data file can be distributed according to the predetermined condition and stored in an appropriate storage destination. It is possible to ensure safety and reliability and improve access performance.

  The data management apparatus further includes a registration unit that registers characteristic information regarding the characteristics of the recording medium of the plurality of nodes, and the storage destination selection unit acquires the characteristic information by inquiring and acquiring the characteristic information from the registration unit And a selection unit that selects the storage destination node that satisfies a predetermined storage condition based on the characteristic information.

  The characteristic information includes the reliability of the recording medium of the node, MTBF (Mean Time Between Failures) of the file system, MTTR (Mean Time To Repair), UPS (Uninterruptable Power Supply) The presence or absence of a device). In addition, geographical and network installation distance intervals, installation site safety, such as installation status, structure, disaster frequency, physical safety, eg security status, cost performance per storage capacity, etc. Can do.

  For example, a node with a geographical distance of more than a predetermined distance, a highly secure installation location, a high structural strength, a low cost, a convenient network installation location with users, etc. It can be a condition.

  The condition for the characteristic information can be a combination of a plurality of conditions or a priority order. Furthermore, the storage destination can be determined on the condition of the attribute of the data file described above. The characteristic information may be acquired and used every time it is copied, or information acquired and stored periodically or as needed may be used.

  According to this configuration, the storage destination can be selected according to the predetermined storage condition, so that the copy of the data file can be distributed according to the predetermined condition and stored in an appropriate storage destination. Secure and reliable and improve access performance.

  In the data management device, a determination unit that determines whether or not a predetermined trigger condition is satisfied, and a control unit that causes the replication unit to generate a copy of the data file when it is determined that the trigger condition is satisfied And can be included.

  Here, the trigger conditions used by the determination unit for determination include, for example, data file storage, update, periodic, maintenance work, events such as disasters, alarms, network and storage failures, etc. Indicates the timing to generate a duplicate file that was triggered. In addition, the usage conditions of storage and network, for example, load on performance, accumulation, short term, access frequency of file unit, disk unit, organization or region unit, change amount of increase / decrease in storage capacity, etc. You can also.

  According to this configuration, the data file can be duplicated at a timing when a predetermined condition is satisfied, the risk of damage or disappearance of the data file can be avoided, and reliability is improved.

  In the data management device, an attribute information presentation unit that presents attribute information of the data file, a file reception unit that receives designation of a data file to be copied, and a copy that receives the number of copies of the data file received by the file reception unit A copy number of the data file received by the file reception unit, and the copy unit can generate a copy file of the number of copies received by the copy number reception unit.

  According to this configuration, there is provided a data management device capable of supporting data management of a replicated file that allows the user to manually specify the number of replicas based on the attribute information of the presented data file.

  In the data management apparatus, an information presentation unit that presents the performance information acquired by the performance information acquisition unit or the characteristic information acquired by the characteristic information acquisition unit, and a storage destination reception unit that receives the storage destination of the duplicate file The storage unit can store the duplicate file in the storage destination received by the storage destination reception unit.

  According to this configuration, there is provided a data management apparatus capable of supporting data management of a replicated file that allows a user to manually specify a storage location of the replicated file based on the presented performance information.

  In the data management apparatus, when the determination unit determines that the trigger condition is satisfied, a notification unit that notifies a user of a message that prompts generation of a copy of the data file, an instruction reception unit that receives a copy instruction, The duplication unit may duplicate the data file in response to the duplication instruction, and the storage unit may store the data file in the storage destination.

  According to this configuration, it is possible to notify the duplication timing of the data file, so that it is possible to efficiently ensure the safety and reliability of resources and improve the access performance at an appropriate timing.

According to the present invention, accessing a plurality of data files respectively recorded on a plurality of recording media respectively included in a plurality of nodes via a network;
Generating a plurality of management files each including path information for accessing the plurality of data files;
In order to manage the plurality of data files, the management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. And steps to
Generating a duplicate file of the data file;
Selecting a storage destination node for storing the duplicate file according to a predetermined condition;
Storing the duplicate file in the storage location selected in the selecting step;
Generating the management file including path information of the duplicate file when the data file is duplicated in the duplicating step and the duplicate file is stored in the storage destination in the storing step;
Accessing the data file recorded on the recording medium based on the path information included in the management file in the hierarchical structure comprising the plurality of directories. A data management method is provided.

  The data management method may include a step of determining whether or not a predetermined trigger condition is satisfied, and a step of generating a copy of the data file when it is determined that the trigger condition is satisfied. it can.

According to the present invention, a data management computer that accesses a plurality of data files respectively recorded on a plurality of recording media included in a plurality of nodes via a network,
Means for accessing the plurality of data files via the network;
Means for generating a plurality of management files each including path information for accessing the plurality of data files;
In order to manage the plurality of data files, the management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. Means to
Means for generating a duplicate file of the data file;
Means for selecting a storage destination node for storing the duplicate file according to a predetermined condition;
Means for storing the duplicate file in the storage location selected by the means for selecting;
Means for generating the management file including path information of the duplicate file when the means for duplicating duplicates the data file and the means for storing stores the duplicate file in the storage destination;
Means for accessing the data file recorded on the recording medium based on the path information included in the management file in the hierarchical structure comprising the plurality of directories;
A program characterized by functioning as is provided.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, there is provided a data management apparatus capable of efficiently distributing resources, ensuring the safety and reliability of data files, and improving access performance.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

  FIG. 1 is a diagram showing a configuration of a data processing system according to an embodiment of the present invention. The present embodiment relates to a parallel computing system that uses grid technology, a plurality of processing devices (nodes) including a recording medium and a CPU are connected via a network, and performs parallel computing processing in conjunction with each other. In particular, the present invention relates to a data management system for managing a plurality of data files respectively recorded on recording media of a plurality of processing apparatuses.

  Here, the data file includes information used by a plurality of resources for calculation processing in response to a calculation request from the client and the calculation processing result. Alternatively, digital data used in the business field or the like is included. In the present embodiment, the recording medium is a storage device such as a hard disk device.

  Further, in the present embodiment, the arithmetic processing processed by the processing device is, for example, digital data archiving including email, backup processing, or science and engineering, for example, CAE (Computer Aided Engineering in the automotive design field) ) And collision analysis, drug screening analysis in pharmaceutical companies, fluid analysis and structural analysis in aircraft design field, credit risk analysis in financial industry, molecular / atom simulation in material development field, particle physics field, astronomy field data analysis These include design, calculation, and analysis related to protein function prediction in the field of bioinformatics, satellites in the earth and planetary science field, and elevation data creation from ground observation data. In addition, processing of digital data stored in a business field or the like can be included.

  As shown in FIG. 1, the data processing system 100 includes a client device 10, a server device 12, a recording unit 14, and a first processing device 16a, a second processing device 16b, a third processing device 16c, which are collectively referred to as a processing device 16. A fourth processing device 16d, a fifth processing device 16e, a sixth processing device 16f, and a network 18 are included. The server device 12 includes an access unit 20, a reception unit 22, a management unit 24, a generation unit 26, a determination unit 28, and an instruction unit 30. The processing device 16 is a first processing unit 40 a that is collectively referred to as a processing unit 40. , Second processing unit 40b, third processing unit 40c, fourth processing unit 40d, fifth processing unit 40e, sixth processing unit 40f, first recording unit 42a, second recording unit 42b, collectively referred to as recording unit 42, A third recording unit 42c, a fourth recording unit 42d, a fifth recording unit 42e, and a sixth recording unit 42f are included. In the present embodiment, each processing device 16 includes a single recording unit 42, but the present invention is not limited to this. Each processing device 16 may include a plurality of recording units 42.

  A plurality of processing devices 16 are connected via the network 18, and in response to a calculation request from the client device 10, the server device 12 performs the requested calculation processing in conjunction with the processing device 16, and the obtained calculation result Is returned to the client device 10.

  The network 18 communicates information signals according to a predetermined protocol. The network 18 is configured by connecting cables for transmitting signals, such as coaxial cables, twisted pair cables, and optical cables, with devices that control signal transmission paths such as routers and exchanges. Although only one network 18 is shown in FIG. 1, the present invention is not limited to this, and the network 18 may be configured by combining a plurality of networks, and the bandwidths of the plurality of networks may be different from each other.

  The client device 10 is a personal computer or the like operated by a user. The client device 10 outputs an instruction such as data file search or calculation processing to the server device 12 described later via the network 18, receives the result from the server device 12, and displays the result on a display (not shown).

  In the present embodiment, the recording unit 42 of the processing device 16 is a storage device such as a hard disk that records data files. Here, the data file “DATA A” is duplicated and recorded in the first recording unit 42a and the sixth recording unit 42f. Further, since the data file “DATA B” has a large data capacity, it is divided into data files “DATA B-1”, “DATA B-2”, and “DATA B-3”. 1 ”is duplicated and recorded in the first recording unit 42a and the fourth recording unit 42d, and the data file“ DATA B-2 ”is duplicated and recorded in the second recording unit 42b and the fifth recording unit 42e. The data file “DATA B-3” is duplicated and recorded in the third recording unit 42c and the sixth recording unit 42f. Further, not only the data file recorded in advance, but also the data file generated by the processing of the processing unit 40 described later is recorded.

  The processing unit 40 of the processing device 16 includes a CPU that executes processing such as calculation and data file search, a RAM (Random Access Memory) that temporarily records data files, and the like. Here, a data file and processing to be subjected to calculation, data file search, and the like are instructed from the server device 12 described later.

  In the server device 12, the access unit 20 is connected to the network 18 and inputs / outputs data and signals for predetermined instructions. The accepting unit 22 accepts data output from the client device 10 or the processing device 16 via the network 18 and the access unit 20. The client device 10 receives instructions such as data file search, calculation processing, and management data file display. From the processing device 16, the path information of the data file recorded in each recording unit 42 is received, or the result according to the instruction output from the server device 12 in advance is received. A specific example of the path information will be described later.

  The generating unit 26 generates a management file including the path information of the data file recorded by the recording unit 42 based on the path information of the data file received by the receiving unit 22. At the time of generation, for example, the name of the management file is determined so as to match the corresponding data file. Here, when data is recorded in a plurality of recording units 42 like the data file “DATA A”, a plurality of path information is described in the management file. Further, a management file associated with the data file in advance may be generated before or at the generation stage of the data file. The generation stage indicates not only the case where the data file is generated but also the stage where the data file is not completed, including before the data file is generated.

  The management unit 24 arranges management files in a predetermined directory, and manages a plurality of directories in a hierarchical structure. Information of a plurality of directories forming a hierarchical structure to be managed and management files are recorded in the recording unit 14. Further, when searching for a predetermined data file according to an instruction from the client device 10, the management unit 24 accesses the recording unit 14 and corresponds to the data file to be searched from information of a plurality of directories forming a hierarchical structure. Search for administrative files.

  Further, when the data file is duplicated as will be described later, the management unit 24 instructs the generation unit 26 to generate a management file for the duplicate file, arranges the management file in a predetermined directory, and Are managed in a hierarchical structure. Information of a plurality of directories and management files forming the hierarchical structure to be managed are recorded in the recording unit 14.

  The determination unit 28 determines the processing unit 40 that should actually execute the process when the plurality of processing units 40 execute the process according to the instruction from the client device 10. Here, if processing that has already been executed is ignored, processing is preferentially assigned to the processing unit 40 included in the same processing device 16 as the recording unit 42 that records the data file to be processed. For example, when the instruction from the client device 10 is a search process of the data file “DATA A”, the determination unit 28 is included in the same first processing device 16a as the first recording unit 42a that records the data file “DATA A”. The process is assigned to the first processing unit 40a. Since the data file “DATA A” is also recorded in the sixth recording unit 42f, the process may be assigned to the sixth processing unit 40f.

  The instruction unit 30 instructs the processing device 16 determined by the determination unit 28 to execute processing, or outputs a processing result directly executed by the management unit 24.

  The configuration of the data processing system 100 can be realized in terms of hardware by a CPU, memory, or other LSI of an arbitrary computer, and can be realized in terms of software by a program having a reservation management function loaded in a memory. Here, the functional blocks realized by the cooperation are depicted. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof. Each figure described below shows functional unit blocks, not hardware unit configurations. Moreover, in each figure, the structure of the part which is not related to the essence of the present invention is omitted.

  In the data processing system 100 of the present embodiment configured as described above, the server device 12 is further connected to the network 18 to which the plurality of processing devices 16 are connected, and the server device 12 is recorded in the plurality of processing devices 16. A management file including path information is generated for each of the plurality of data files. For example, the management file “A” corresponding to the data file “A” includes the network address of the processing device 16 that recorded the data file “A” and the address of the recording area in the recording unit 42 included in the processing device 16. It is described. Further, the server device 12 does not directly manage a plurality of data files, but instead arranges a plurality of management files in a predetermined directory, and manages such a plurality of directories in a hierarchical structure. When a user or program designates a predetermined data file, a management file corresponding to the data file is designated from a directory forming a hierarchical structure. Based on the path information included in the designated management file, the server device 12 accesses the processing device 16 in which the data file is recorded, and designates the data file.

  FIG. 2 shows the configuration of the file system in the server device 12. The hierarchical structure information formed by a plurality of directories recorded by the recording unit 14 is shown in the upper part of FIG. The directory “/ grid” is positioned at the top of the hierarchical structure, and directories “ggf” and “jp” are arranged below the directory “/ grid”. A management file 50a (shown as “DATA A” in the figure) is arranged under the directory “jp”. Further, the management files 50b, 50c, and 50d (in the figure, “DATA B-1”, “DATA B-2”, “DATA B-3”) are subordinate to the directory “ist” that is lower than the directory “ggf”. Are shown). Here, the management files 50b, 50c, and 50d are management files corresponding to the data files “DATA A”, “DATA B-1”, “DATA B-2”, and “DATA B-3” in FIG. 1, respectively. It is.

  The management file 50a describes “ftp://p1.co.jp/a/DATAA” and “http://p6.co.jp/a/DATAA” as path information. Here, the first processing device 16a in FIG. 1 corresponds to “p1.co.jp”, the sixth processing device 16f corresponds to “p6.co.jp”, and the data file “DATA A” corresponds to “DATAA”. To do. The same applies to the management files 50b, 50c, and 50d.

  The lower part of FIG. 2 shows the data file recorded by the recording unit 42 as in the description of FIG. As described above, each protocol and network address is indicated as “ftp://p1.co.jp” in the upper part of each recording unit 42. In each recording unit 42, the address of the recording area is indicated as “/ a / DATAA” and “/ b / DATAB-1”. Data files are distributed and recorded in a plurality of recording units 42, but the recording unit 14 does not directly manage these data files, but manages the management files corresponding to the data files in a hierarchical structure. Therefore, as shown in the figure, the data file can be managed without being aware of the recording unit 42 that recorded the data file.

  When the user or the program instructs the processing for the data file “DATA B-1”, the management unit 24 determines whether the first recording unit 42a or the fourth recording unit 42a is in accordance with the path information of the management file 50b recorded in the recording unit 14. Access “/ b / DATAB-1” of the recording unit 42d. On the other hand, when instructing the processing for all of the data files “DATA B”, the management unit 24 stores the directory “aist” including all the management files 50b, 50c, and 50d corresponding to the data file “DATA B” as the data files. It is regarded as a management file 50e corresponding to “DATA B”. Furthermore, based on the path information described therein, the corresponding recording unit 42 is accessed.

  FIG. 3 is a functional block diagram of the backup unit 60 which is a main configuration of the data processing system 100 according to the embodiment of the present invention. The data processing system 100 of FIG. 1 can further include a backup unit 60 of FIG. In the present embodiment, the backup unit 60 is included in the server device 12, but is not limited thereto.

  The data processing system 100 according to the embodiment of the present invention includes a network (network 18 in FIG. 1) in a plurality of data files respectively recorded in a plurality of recording media (recording unit 42 in FIG. 1) included in a plurality of nodes. ) And a generation unit (generation unit 26 in FIG. 1) that respectively generates a plurality of management files including path information for accessing a plurality of data files. In order to manage a plurality of data files, a management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. A management unit (the management unit 24 in FIG. 1) and a replication unit that generates a duplicate file of the data file (the replication unit in FIG. 3) 06), a storage destination selection unit (storage destination selection unit 128 in FIG. 3) for selecting a storage destination node for storing the duplicate file according to a predetermined condition, and a duplicate file stored in the storage destination selected by the storage destination selection unit A storage unit (storage unit 120 in FIG. 3), and the generation unit includes path information of the replication file when the replication unit replicates the data file and the storage unit stores the replication file in the storage destination. The management file is generated, and the access unit accesses the data file recorded on the recording medium based on the path information included in the management file in a hierarchical structure including a plurality of directories.

  As shown in FIG. 3, the backup unit 60 includes a trigger condition storage unit 102 (shown as “trigger condition” in the figure), a control unit 104, a duplication unit 106, a file information acquisition unit 108, and a file information storage unit. 110 (shown as “file information” in the figure), a number determination condition storage unit 112 (shown as “number decision condition” in the figure), a copy number determination unit 114, a storage unit 120, and a medium information acquisition unit 122, a medium information storage unit 124 (shown as “medium information” in the figure), a storage destination selection condition storage unit 126 (shown as “storage destination selection condition” in the figure), a storage destination selection unit 128, It has.

  The trigger condition storage unit 102 stores a trigger that is a timing at which the duplication unit 106 generates a duplication of the data file. A trigger condition is, for example, creating a replication file that triggers replication when a data file is stored, updated, periodically, when maintenance work occurs, or when an event such as a disaster, alarm, or network or storage failure occurs. Indicates timing. In addition, the usage conditions of storage and network, for example, load on performance, accumulation, short term, access frequency of file unit, disk unit, organization or region unit, change amount of increase / decrease in storage capacity, etc. You can also. Alternatively, whether or not the data file can be accessed is checked periodically or at any time, and the case where the data file cannot be accessed can be used as a trigger. These trigger conditions can be a combination of a plurality of conditions or can be prioritized.

  The control unit 104 accesses the trigger condition storage unit 102 to determine whether or not the trigger condition is satisfied. When determining that the trigger condition is satisfied, the control unit 104 causes the replication unit 106 to generate a copy of the data file. Alternatively, an instruction receiving unit (to be described later) receives an instruction from the user of the client device 10 or the administrator of the server device 12 and notifies the control unit 104 to copy the data file.

  The duplication unit 106 generates a duplication file of the data file. The duplication unit 106 obtains a management file of the duplication source data file from the management unit 24, obtains the storage destination of the data file from the path information included in the management file, and based on the path information, the access unit 20 The data file is accessed via and a replica is generated.

  The file information acquisition unit 108 acquires the storage destination of the replication source data file from the replication unit 106, accesses the storage destination via the access unit 20, and information about the replication source data file (hereinafter referred to as “file information”). To get. Here, the file information includes attribute information of the data file, for example, attributes such as file size, generation time, update time, and owner. Furthermore, the type of stored information can also be included. In other words, the type of stored information is classified and ranked according to the importance of the file that indicates whether it contains information that would not be lost, personal information, copyright, confidential information, necessary safety indicators, popularity, etc. be able to. The file information may be acquired and used every time it is copied, or may be information acquired and stored periodically or as needed.

  The file information can be acquired by the file information acquisition unit 108 accessing the data file recorded in the recording unit 42 of the processing device 16 via the access unit 20.

  The file information storage unit 110 stores information related to the replication source data file acquired by the file information acquisition unit 108. Alternatively, a registration unit (not shown) for registering file information in advance can be included. Then, an administrator of the server device 12 can register file information in the file information storage unit 110 using a registration unit in advance.

  The number determination condition storage unit 112 stores a copy number determination condition used when the copy number determination unit 114 determines the copy number. For example, if the file size is greater than or equal to a predetermined value, the number of replicas is three, and if it is less than the predetermined value, the number of replicas is two. If the value is equal to or higher than a predetermined level, the number of replicas is set to three, and if the level is less than the predetermined level, the number of replicas is set to two. As the condition for determining the number of replicas, a plurality of conditions can be combined or a priority order can be set.

  The number-of-replications determination unit 114 determines the number of copies in accordance with the number-of-replications determination conditions stored in the number determination condition storage unit 112 based on the attribute information of the data file of the replication source stored in the file information storage unit 110. To do. The copy number determination unit 114 can specify the copy number determination condition to be used by a user of the client device 10 or an administrator of the server device 12 as described later. The duplication unit 106 generates duplicate files for the number of duplications determined by the duplication number determination unit 114.

  The storage unit 120 stores the duplicate file generated by the duplication unit 106 in the storage destination selected by the storage destination selection unit 128 via the access unit 20. When the storage unit 120 stores the duplicate file, it notifies the management unit 24 accordingly. In response to the notification from the storage unit 120, the management unit 24 acquires path information of the storage location of the duplicate file, and the generation unit 26 generates a management file and records it in the recording unit 14.

  The medium information acquisition unit 122 acquires information on the performance or characteristics of the plurality of recording units 42 that are storage destinations of data files. The medium information storage unit 124 stores information acquired by the medium information acquisition unit 122.

  The performance information is, for example, the free capacity of the recording unit 42, the access performance, the performance of the access CPU, the network installation location with the user, and the like. These pieces of performance information can be acquired, for example, by making an inquiry via a network to a data processing device that performs data processing using the recording unit 42 if the recording unit 42 has free space. Further, the access performance can be acquired by actually accessing the recording unit 42 via the network 18 and measuring the response time. The network installation location with the user can be acquired by transmitting an inquiry signal from the client apparatus 10 of the user via the network 18 and measuring the response time. The storage destination selection unit 128 selects the recording unit 42 whose performance information satisfies a predetermined condition as the storage destination.

  The characteristic information includes the reliability of the recording unit 42 and the presence / absence of MTBF, MTTR, UPS of the file system. In addition, geographical and network installation distance intervals, installation site safety, such as installation status, structure, disaster frequency, physical safety, eg security status, cost performance per storage capacity, etc. Can do.

  For example, a node with a geographical distance of more than a predetermined distance, a highly secure installation location, a high structural strength, a low cost, a convenient network installation location with users, etc. It can be a condition. These pieces of information can also be ranked for each attribute for each of the plurality of recording units 42. The storage destination selection unit 128 can also select a storage destination using this ranking. In addition, a condition used for selection by the storage destination selection unit 128 can be designated by a user or an administrator.

  The condition for the characteristic information can be a combination of a plurality of conditions or a priority order. Furthermore, the storage destination can be determined on the condition of the attribute of the data file described above. The characteristic information may be acquired and used every time it is copied, or information acquired and stored periodically or as needed may be used.

  The storage location selection condition storage unit 126 stores predetermined storage location selection conditions. The condition for the performance information can be a combination of a plurality of conditions or a priority order, for example. Furthermore, the storage destination can be determined on the condition of the attribute of the data file described above. The performance information may be acquired and used every time it is copied, or information acquired and stored periodically or as needed may be used.

  Also, the condition for the characteristic information can be a combination of a plurality of conditions or can be given a priority order. Furthermore, the storage destination can be determined on the condition of the attribute of the data file described above. The characteristic information may be acquired and used every time it is copied, or information acquired and stored periodically or as needed may be used.

  The storage destination selection unit 128 is based on the information stored in the medium information storage unit 124, and the storage destination recording unit 42 that satisfies a predetermined storage destination selection condition stored in the storage destination selection condition storage unit 126. Select.

  FIG. 4 is a functional block diagram showing a configuration for designating a data file to be copied and its storage destination in the data processing system 100 of the present embodiment.

  Server device 12 further includes a display unit 130, an operation unit 132, a presentation unit 134, and an instruction receiving unit 136. The presentation unit 134 presents the attribute information of the data file stored in the file information storage unit 110 and the performance information and / or characteristic information stored in the medium information storage unit 124. Such information can be displayed on the screen of the display unit 130. Alternatively, it can be displayed on the display unit of the client device 10 on the network 18 via the access unit 20 of FIG. The presentation unit 134 can be realized by a portal site provided by an administrator of the data processing system 100.

  The instruction receiving unit 136 receives an instruction from the operation unit of the client device 10 on the network 18 via the operation unit 132 and the access unit 20 and notifies the control unit 104 of the instruction. The instructions received by the instruction receiving unit 136 are the storage location of the replication source data file, the number of replications of the replication file, the storage location of the replication file, and the like. Alternatively, it is possible to specify a storage location selection condition used when determining a storage location for storing a duplicate file.

  That is, the administrator of the server device 12 and the user of the client device 10 refer to the information presented by the presentation unit 134 and based on the attribute information of the data file and the information on the performance and characteristics of the storage unit recording unit 42. Judgment is made, the number of copies and the storage destination are determined, and an instruction is given via the operation unit. The instruction receiving unit 136 receives the instruction, notifies the control unit 104, and the data file copying and storing process is performed.

  The operation of the data processing system 100 configured as described above will be described below. FIG. 5 is a flowchart showing an example of the operation of the data processing system 100 of the present embodiment. Hereinafter, description will be made with reference to FIGS.

  The control unit 104 determines whether it is the duplication timing according to the trigger condition stored in the trigger condition storage unit 102 (step S11). If it is determined that it is the duplication timing (YES in step S11), the control unit 104 notifies the duplication unit 106, and instructs the file information acquisition unit 108 to obtain file information from the duplication unit 106. Note that the control unit 104 can also notify the duplication unit 106 of the number of duplications and a storage location instruction based on the instruction received by the instruction reception unit 136 of FIG. 4.

  The file information acquisition unit 108 acquires the storage destination of the replication source data file from the replication unit 106, accesses the storage destination via the access unit 20, and acquires information regarding the replication source data file (step S13). The acquired information is stored in the file information storage unit 110. Subsequently, upon receiving a notification from the file information acquisition unit 108, the copy number determination unit 114 stores the copy source data file attribute information stored in the file information storage unit 110 on the number determination condition storage unit 112. The number of copies is determined according to the stored copy number determination conditions (step S15). The determined number of copies is notified to the copy unit 106.

  The file information acquisition by the file information acquisition unit 108 is not necessarily performed, and information already stored in the file information storage unit 110 may be used.

  Subsequently, in response to an instruction from the duplication unit 106, the storage unit 120 instructs the medium information acquisition unit 122 to acquire medium information. The medium information acquisition unit 122 acquires information on the performance or characteristics of the plurality of recording units 42 serving as storage destinations of data files via the access unit 20 (step S17). Or it acquires from the registration part which is not illustrated. The medium information acquired by the medium information acquisition unit 122 is stored in the medium information storage unit 124. The storage destination selection unit 128 receives the notification from the storage destination selection condition storage unit 126, accesses the storage destination selection condition storage unit 126, and selects the storage unit recording unit 42 that satisfies the predetermined storage destination selection condition. (Step S19).

  Subsequently, the duplication unit 106 generates a duplication file having the duplication number determined by the duplication number determination unit 114 (step S21), and the storage unit 120 stores the duplication file in the storage destination selected by the storage destination selection unit 128. (Step S23).

  Subsequently, upon receiving notifications from the duplication unit 106 and the storage unit 120, the management unit 24 acquires path information of the copy file storage destination (step S25). Subsequently, the generation unit 26 generates a management file including path information, generates a management file including the path information of the duplicate file recorded by the recording unit 42, and the management unit 24 manages the management file. Are recorded in a predetermined directory of the recording unit 14, and information on a plurality of directories forming a hierarchical structure to be managed by the plurality of directories is recorded on the recording unit 14 together with a management file (step S27).

  In this manner, data file replicas can be distributed and stored in the recording units 42 of a plurality of nodes on the network 18 in accordance with predetermined conditions, and the management file can be updated. It can be managed by the server device 12. Such a duplicate file is created in order to create a backup of a data file, improve reliability, improve access performance, and the like, as described below.

  Hereinafter, an example of an operation when creating a backup of a data file in the data processing system 100 of the present embodiment will be described.

  For example, the instruction receiving unit 136 receives an instruction to create a copy of a predetermined data file according to an instruction from the user of the client device 10 or the administrator of the server device 12. Here, the instruction receiving unit 136 receives the storage destination of the copy source data file and the number of copies. Here, the instruction receiving unit 136 also receives the storage location of the duplicate file. The storage location of the duplicate file can also be determined by causing the storage location selection unit 128 to automatically select the optimal storage location. The duplicating unit 106 duplicates the data file with the number of duplications received by the instruction receiving unit 136, and the storage unit 120 stores the data file in the storage destination received by the instruction receiving unit 136. Then, the generation unit 26 generates a management file for the duplicate file, and the management unit 24 manages it.

  In another embodiment, the instruction receiving unit 136 receives a copy source data file, and the copy number determination unit 114 sets the copy number to 2 when the file size is a predetermined value or more based on the attribute information of the data file. If the number is less than the predetermined value, the number of copies is determined to be three. The file size can also be obtained by inquiring the processing device 16 including the recording unit 42 in which the data file is recorded at the time of duplication.

  In yet another embodiment, the designation receiving unit 136 accepts designation of the storage destination node, domain name, network address, replication destination node attribute, status, etc. of the duplicate file, A storage location can also be selected. That is, the storage location selection unit 128 selects a storage location from the storage location selection condition storage unit 126 according to the storage location selection condition corresponding to the designation received by the instruction reception unit 136.

  In another embodiment, the storage destination selection unit 128 can automatically select a storage destination according to the storage destination selection conditions stored in the storage destination selection condition storage unit 126. The information on the characteristics and performance of the recording unit 42 stored in the medium information storage unit 124 is used to make a determination according to the conditions stored in the storage destination selection condition storage unit 126. Here, the information stored in the medium information storage unit 124 can be stored by ranking a plurality of nodes according to conditions, for example.

  Next, in the data processing system 100 of this embodiment, an example in the case where a replica is created in order to improve resource reliability will be described.

  For example, when the instruction accepting unit 136 accepts a replication source data file, the replication number determination unit 114 determines the replication number of the replication file according to the MTBF of the storage destination file system. That is, the copy number determination unit 114 determines the copy number to be 2 when the MTBF of the corresponding file system is equal to or greater than a predetermined value, and determines the copy number to be 3 when the MTBF is less than the predetermined value. Thereby, when MTBF is less than a predetermined value, the risk of data file loss and the like can be reduced by increasing the number of replicas.

  In another embodiment, the control unit 104 periodically checks whether or not a duplicate file can be accessed, and if the access cannot be made, instructs the new node to generate a new duplicate file. As a result, it is possible to detect in advance a damaged backup file or an inaccessible state and take measures.

  Furthermore, an example of the operation when creating a duplicate file in order to improve access performance in the data processing system 100 of the present embodiment will be described.

  For example, the copy number determination unit 114 determines the copy number based on the rank of file attributes such as the access frequency and file size of the copy source data file. For example, the copy number determination unit 114 sets the copy number to 3 when the access frequency is equal to or higher than a predetermined rank, and determines the copy number to be 2 when the access frequency is less than the predetermined rank. As a result, the number of copies of data files with high access frequency can be increased, so that the concentration of access to the files can be distributed. As a result, since the access frequency is not concentrated on a recording medium or data file, the processing speed can be improved and the life of the recording medium can be extended.

  In another embodiment, the trigger condition storage unit 102 may issue a duplication instruction periodically or when the access frequency exceeds a threshold value. Further, the storage destination selection unit 128 can automatically select the storage location of the duplicate file. When the duplicate file generated in this way is stored in the storage destination, the management unit 24 causes the generation unit 26 to generate and manage a management file for the duplicate file.

  In another embodiment, the medium information acquisition unit 122 accesses each node via the network 18 to measure the response time, and the storage destination selection unit 128 sets the node whose response time is equal to or less than the threshold as the storage destination. You can choose. As a result, when accessing a data file, a copy can be held in a node with good communication efficiency so that the user does not feel as much stress due to communication traffic as possible.

  As described above, according to the data processing system 100 of the embodiment of the present invention, it is possible to distribute and store a copy of a data file in a plurality of recording units 42 on the network according to a predetermined condition. Well, it is possible to ensure the safety and reliability of resources and improve the access performance.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are the illustrations of this invention, Various structures other than the above are also employable.

  For example, when the control unit 104 determines that the trigger condition stored in the trigger condition storage unit 102 is satisfied, a message prompting generation of a copy of the data file is sent to the user of the client device 10 or the administrator of the server device 12. A notification unit (not shown) for notification may be included. Here, the notification unit displays a message or mark on the portal site of the data processing system 100, transmits a mail including the message to a pre-registered user account, performs voice output, LED or lamp display, and the like. Can be included. Upon receiving the notification, the user or the administrator can issue a copy instruction using the operation unit 132. In response to the copy instruction received by the instruction receiving unit 136, the copy unit 106 copies the data file, The storage unit 120 can store the duplicate file in the storage destination.

  According to this configuration, it is possible to notify the duplication timing of the data file, so that it is possible to efficiently ensure the safety and reliability of resources and improve the access performance at an appropriate timing.

It is a functional block diagram which shows the data processing system which concerns on embodiment of this invention. It is a figure which shows the structure of the file system in the server apparatus of FIG. It is a functional block diagram which shows the backup part which is the principal part structure of the data processing system of this embodiment. In the data processing system of this embodiment, it is a functional block diagram which shows the structure which designates the data file to copy and its storage location. It is a flowchart which shows an example of operation | movement of the data processing system of this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Client apparatus 12 Server apparatus 12 The said server apparatus 14 Recording part 16 Processing apparatus 18 Network 20 Access part 22 Reception part 24 Management part 26 Generation part 28 Determination part 30 Instruction part 40 Processing part 42 Recording part 50 Management file 60 Backup part 100 Data processing system 102 Trigger condition storage unit 104 Control unit 106 Replication unit 108 File information acquisition unit 110 File information storage unit 112 Number determination condition storage unit 114 Replication number determination unit 120 Storage unit 122 Medium information acquisition unit 124 Medium information storage unit 126 Storage Destination selection condition storage unit 128 Storage destination selection unit 130 Display unit 132 Operation unit 134 Presentation unit 136 Instruction reception unit

Claims (11)

An access unit for accessing a plurality of data files respectively recorded on a plurality of recording media included in a plurality of nodes via a network;
A generating unit that generates a plurality of management files each including path information for accessing the plurality of data files;
In order to manage the plurality of data files, the management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. A management department to
A duplicating unit for creating a duplicating file of the data file;
A storage destination selection unit for selecting a storage destination node for storing the duplicate file according to a predetermined condition;
A storage unit for storing the duplicate file in the storage destination selected by the storage destination selection unit;
With
The generation unit, when the replication unit replicates the data file, and the storage unit stores the replication file in the storage destination, generates the management file including path information of the replication file,
The access unit accesses the data file recorded on the recording medium based on the path information included in the management file in the hierarchical structure including the plurality of directories. Data management device. The data management device according to claim 1,
An attribute information acquisition unit for acquiring attribute information of the data file to be replicated by the replication unit;
A data management apparatus, further comprising: a copy number determination unit that determines the copy number of the copy file according to a predetermined copy number determination condition based on the attribute information. In the data management device according to claim 1 or 2,
The storage destination selection unit
A performance information acquisition unit that inquires and acquires performance information indicating the performance of the recording medium of the plurality of nodes via the network;
A selection unit that selects the storage destination node that satisfies a predetermined storage condition based on the performance information;
A data management device comprising: The data management device according to any one of claims 1 to 3,
A registration unit for registering characteristic information related to characteristics of the recording medium of the plurality of nodes;
The storage destination selection unit
A characteristic information acquisition unit that acquires the characteristic information by inquiring of the registration unit;
A selection unit for selecting the storage destination node that satisfies a predetermined storage condition based on the characteristic information;
A data management device comprising: The data management device according to any one of claims 1 to 4,
A determination unit for determining whether or not a predetermined trigger condition is satisfied;
When it is determined that the trigger condition is satisfied, a controller that causes the replica to generate a copy of the data file;
A data management device comprising: The data management device according to any one of claims 1 to 5,
An attribute information presentation unit for presenting attribute information of the data file;
A file reception unit that accepts specification of a data file to be replicated;
A copy number receiving unit for receiving the number of copies of the data file received by the file receiving unit;
Including
The duplication unit generates duplicate files of the data file accepted by the file accepting unit for the number of duplications accepted by the duplication number accepting unit. The data management device according to any one of claims 3 to 6,
An information presentation unit for presenting the performance information acquired by the performance information acquisition unit or the characteristic information acquired by the characteristic information acquisition unit;
A storage location receiving unit that receives the storage location of the duplicate file;
And the storage unit stores the duplicate file in the storage destination received by the storage destination reception unit. The data management device according to any one of claims 5 to 7,
When the determination unit determines that the trigger condition is satisfied, a notification unit that notifies a user of a message that prompts generation of a copy of the data file;
An instruction receiving unit for receiving a copy instruction;
Including
The data management apparatus, wherein the duplication unit duplicates the data file in response to the duplication instruction, and the storage unit stores the data file in the storage destination. Accessing a plurality of data files respectively recorded on a plurality of recording media included in a plurality of nodes via a network;
Generating a plurality of management files each including path information for accessing the plurality of data files;
In order to manage the plurality of data files, the management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. And steps to
Generating a duplicate file of the data file;
Selecting a storage destination node for storing the duplicate file according to a predetermined condition;
Storing the duplicate file in the storage location selected in the selecting step;
Generating the management file including path information of the duplicate file when the data file is duplicated in the duplicating step and the duplicate file is stored in the storage destination in the storing step;
Accessing the data file recorded on the recording medium based on the path information included in the management file in the hierarchical structure comprising the plurality of directories. Data management method. The data management method according to claim 9,
Determining whether a predetermined trigger condition is satisfied;
Generating a copy of the data file when it is determined that the trigger condition is satisfied;
A data management method comprising: To a data management computer that accesses a plurality of data files respectively recorded on a plurality of recording media included in a plurality of nodes via a network,
Means for accessing the plurality of data files via the network;
Means for generating a plurality of management files each including path information for accessing the plurality of data files;
In order to manage the plurality of data files, the management file corresponding to each of the plurality of data files is arranged in a predetermined directory, and the plurality of directories including the directory are logically managed by a hierarchical structure. Means to
Means for generating a duplicate file of the data file;
Means for selecting a storage destination node for storing the duplicate file according to a predetermined condition;
Means for storing the duplicate file in the storage location selected by the means for selecting;
Means for generating the management file including path information of the duplicate file when the means for duplicating duplicates the data file and the means for storing stores the duplicate file in the storage destination;
Means for accessing the data file recorded on the recording medium based on the path information included in the management file in the hierarchical structure comprising the plurality of directories;
A program characterized by functioning as

Images