WO2014203397A1 - Système informatique, procédé de gestion de métadonnées et programme - Google Patents

Système informatique, procédé de gestion de métadonnées et programme Download PDF

Info

Publication number
WO2014203397A1
WO2014203397A1 PCT/JP2013/067099 JP2013067099W WO2014203397A1 WO 2014203397 A1 WO2014203397 A1 WO 2014203397A1 JP 2013067099 W JP2013067099 W JP 2013067099W WO 2014203397 A1 WO2014203397 A1 WO 2014203397A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
metadata
information
stored
movement
Prior art date
Application number
PCT/JP2013/067099
Other languages
English (en)
Japanese (ja)
Inventor
陽介 石井
児玉　昇司
Original Assignee
株式会社日立製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立製作所 filed Critical 株式会社日立製作所
Priority to PCT/JP2013/067099 priority Critical patent/WO2014203397A1/fr
Publication of WO2014203397A1 publication Critical patent/WO2014203397A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Definitions

  • the present invention relates to a computer system using hierarchical storage.
  • the search server analyzes the text of the data stored in the computer system and generates a search index in advance.
  • the user can transmit a search query for searching the acquisition target data to the search server and access the acquisition target data based on the search result output from the search server.
  • the search server extracts data including a combination of the metadata name and metadata value included in the search target data, and generates a search index for the data extracted in advance.
  • a user can acquire a search result by designating search conditions regarding the metadata name and metadata value to the search server.
  • the search service as described above is an important service for the user, and its use is considered to be further expanded.
  • the metadata search can be used not only for simple data search but also for data analysis by executing statistical processing or the like on data that matches the search condition.
  • Patent Document 1 discloses a storage system that virtually reduces the management cost per capacity by combining an expensive and high-performance upper tier storage and an inexpensive and low-performance lower tier storage. That is, a hierarchical storage technology is described.
  • the hierarchical control server stores files in appropriate storage based on the usage status of the stored files.
  • the tiered storage it is possible to construct a system configuration in which the upper tier storage is installed at each site used by the user and the lower tier storage is installed in the data center. By constructing the system having the above-described system configuration, it is possible to collect data generated at each site on the data center side using the function of the hierarchical storage.
  • the size of the metadata is relatively small.
  • the tendency of the metadata size to become smaller than the data size becomes significant.
  • a typical example of the invention disclosed in the present application is as follows. That is, a computer system comprising a plurality of computers, each of the plurality of computers having a processor, a main storage device, a secondary storage device, and an interface for communicating with other computers, and at least one or more
  • the computer includes a buffer for temporarily storing a file received from an external device connected via the interface, a data storage area for storing the file, a file management unit for managing the file, and access to the file
  • a file access control unit that controls the metadata
  • a metadata generation unit that acquires metadata by analyzing the file and generates a metadata file using the acquired metadata.
  • the generation unit acquires operating information indicating an operating state of the computer that has received the file, Whether to generate the metadata file is determined based on the acquired operation information, and if it is determined to generate the metadata file, the metadata is acquired from the file stored in the buffer And generating the metadata file based on the acquired metadata, storing the generated metadata file in the data storage area, and storing the generated metadata file in the metadata storage area. Information for accessing the file is added.
  • file metadata can be retained according to the operation information of the computer that has received the file. Further, since information for accessing the file is added to the metadata file, the file and the metadata file can be managed in association with each other.
  • FIG. 1 is an explanatory diagram showing an outline of processing of the computer system according to the first embodiment of this invention.
  • the computer system includes a front-end file server 1, a back-end file server A (2-1), a back-end file server B (2-2), a data source 4, and a client 5.
  • Each device is connected via a network 6 so that data communication is possible.
  • the back-end file server A (2-1) and the back-end file server B (2-2) are not distinguished, they are also referred to as the back-end file server 2.
  • the front-end file server 1 and the back-end file server 2 are not distinguished, they are also described as file servers.
  • the front-end file server 1 provides a file storage service and a file sharing service.
  • the front end file server 1, the back end file server A (2-1), and the back end file server B (2-2) constitute a hierarchical storage.
  • the front-end file server 1 is a higher-level storage and provides a virtual large-capacity file server to the user.
  • the back-end file server 2 is a lower-level storage, and stores a part of data stored in the front-end file server 1.
  • Tier0 is set as the management hierarchy of the front-end file server 1
  • Tier1 is set as the management hierarchy of the back-end file server A (2-1)
  • the back-end file server B (2- It is assumed that “Tier2” is set as the management hierarchy of 2).
  • the front-end file server 1 determines that the back-end file server A (2-1) is based on a predetermined policy such as file access frequency or elapsed time after file update. Move the file.
  • a predetermined policy such as file access frequency or elapsed time after file update.
  • Move the file When a file access request for the file is received from the user after the file is moved, the front-end file server 1 acquires the file from the back-end file server A (2-1), and responds with the file acquired by the user. Send as.
  • the hierarchical storage configuration may be a multi-stage configuration.
  • the back end file server B (2-2) may be included.
  • the front-end file server 1 moves the file to the back-end file server A (2-1)
  • the back-end file server A (2 It may be controlled to move a part of the file stored in -1) to the back-end file server B (2-2).
  • the front-end file server 1 controls to select the back-end file server B (2-2) as the file transfer destination. May be.
  • Data source 4 is a device that generates data to be stored.
  • the data source 4 for example, an electronic medical record server that registers electronic medical record information and manages the electronic medical record information, and a monitoring server that accumulates video data for monitoring recorded using a monitoring camera can be considered.
  • the front-end file server 1 stores various data generated by the data source 4 as file format data.
  • the front-end file server 1 manages data so that a plurality of clients 5 can access the data and share the data.
  • the front-end file server 1 also stores metadata extracted from stored data and manages it as sharable data.
  • metadata include a patient ID for patient identification included in the electronic medical record data, identification information of a person shown in the moving image specified based on a person image included in the moving image data, and the like.
  • the computer system of this embodiment may include a search server (not shown).
  • the search server generates a search index by indexing the file and metadata in order to search for data and metadata extracted from the data.
  • the function of the search server may be realized using the front-end file server 1 or the back-end file server 2, or may be realized using another server (not shown).
  • the front end file server 1 and the data source 4 may be constructed in the user side base, and the back end file server 2 may be constructed in the data center.
  • data and metadata extracted from the data can be managed using a hierarchical storage.
  • the present invention implements control so that metadata can be reliably transferred between file servers for the reasons described above.
  • the present invention is not limited to file storage but can be applied to block storage. Further, the present invention is not limited to a file server, but can be applied to a storage system that stores files.
  • the storage system includes a controller having a CPU and a memory, an I / F, and an auxiliary storage device.
  • the front-end file server 1 receives a file to be stored from the data source 4.
  • the file to be stored is also referred to as an actual data file 180.
  • the front end file server 1 temporarily stores the actual data file 180 in the data reception buffer 160 on the memory.
  • the front-end file server 1 is not limited to a mode of receiving a file from an external device, but may be a mode of acquiring the actual data file 180 by transmitting a file acquisition request to the external device.
  • step S1 the metadata generation unit 120 (see FIG. 2) of the front-end file server 1 extracts metadata from the actual data file 180 stored in the data reception buffer 160.
  • the metadata generation unit 120 stores the extracted metadata in the data storage area 170 on the auxiliary storage device 13 as file format data.
  • the metadata generation unit 120 also stores the actual data file 180 in the data storage area 170.
  • file format metadata is also referred to as a metadata file 181.
  • data including at least one of the actual data file 180 and the metadata file 181 is described as a file.
  • the metadata generation unit 120 stores the correspondence between the actual data file 180 and the metadata file 181 in the file correspondence management table 152, and stores information on the storage locations of the actual data file 180 and the metadata file 181 in the storage file management table 154. To store. In addition, the metadata generation unit 120 stores, in the operation management table 153, information related to the remaining capacity of the data storage area 170 that has changed with the storage of the file in the data storage area 170.
  • step S2 the file movement control unit 130 (see FIG. 2) of the front-end file server 1 selects a movement candidate file from the files stored in the data storage area 170. Specifically, the following processing is executed.
  • the file movement control unit 130 acquires the operation information of the front end file server 1 from the operation management table 153.
  • the file movement control unit 130 refers to the movement condition management table 151, collates the acquired operation information with the registered movement condition, and determines the processing content of the movement process.
  • the file movement control unit 130 refers to the storage file management table 154 based on the determined processing contents of the movement process, and selects a file to be moved from among the files stored in the data storage area 170.
  • the file movement control unit 130 stores the information of the selected file in the file movement management table 155.
  • the file migration control unit 130 refers to the file correspondence management table 152 in order to distinguish whether the migration target file is the actual data file 180 or the metadata file 181. In addition, the file migration control unit 130 acquires the identification information of the file server to which the file is migrated from the hierarchical storage configuration management table 150.
  • step S3 the file movement control unit 130 of the front end file server 1 executes a file movement process.
  • the file migration control unit 130 refers to the file migration management table 155 and sequentially migrates the migration target file to the migration destination file server.
  • the file movement control unit 130 may delete the actual data file 180 and the metadata file 181 stored in the data storage area 170 after moving the file, or store the actual data file 180 and the metadata file 181. It may be replaced with a stub file containing link information regarding the destination.
  • the file movement control unit 130 reflects information related to file movement in the stored file management table 154. In addition, the file movement control unit 130 reflects information on the remaining capacity of the data storage area 170 associated with file deletion or replacement with a stub file in the operation management table 153.
  • FIG. 1 shows the state after the file is moved.
  • the file 280-1 indicates that the actual data file 180 and the metadata file 181 are stored in the backend file server A (2-1) as one file.
  • the file 280-2 is stored in the backend file server A (2-1) as a file including only the metadata file 181.
  • the file 280-3 is stored as a file including only actual data in the backend file server B (2- 2) indicates that it is stored. That is, the actual data file 180 and the metadata file 181 are stored in different back-end file servers 2.
  • the metadata file 181 included in the file 280-2 includes storage location information indicating the storage location of actual data. For example, information on the file 280-3 including the corresponding actual data file 180 is included.
  • FIG. 2 is a block diagram illustrating a configuration example of the computer system according to the first embodiment of this invention.
  • the computer system includes a front-end file server 1, a plurality of back-end file servers 2, a data source 4, and a client 5, and is connected to each other via a network 6 so that data communication is possible.
  • the network 6 may be a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like. Moreover, it is not limited to the network connection type, and may be either wired or wireless. Moreover, the structure which connects via several different networks as needed may be sufficient.
  • the front-end file server 1, the data source 4, and the client 5 are connected to each other via a LAN, and each of the plurality of back-end file servers 2 is connected to another LAN having a different subnet, etc.
  • a configuration in which two LANs are connected via a LAN is conceivable.
  • each file server can be applied with cascade connection or ring connection, and data stored in the front-end file server 1 and the plurality of back-end file servers 2 can be accessed using one file path. It has become.
  • the present invention is not limited to this, and an independent file path may be used for each file server.
  • the front end file server 1 provides the client 5 with a file storage service and a file sharing service.
  • the plurality of back-end file servers 2 store files stored in the front-end file server 1.
  • the data source 4 is a device that generates a file to be stored.
  • the client 5 is a computer that the user operates to acquire data or search for data.
  • the front end file server 1 stores the file generated by the data source 4 in its own storage area or the back end file server 2. Further, the front-end file server 1 responds with a file including at least one of the actual data file 180 and the metadata file 181 in response to the file access request transmitted from the client 5.
  • each component is one is described for simplicity, but a plurality of components may exist.
  • the front-end file server 1 and the plurality of back-end file servers 2 constituting the tiered storage can add another file storage or file server in a hierarchy, and a plurality of file storages or It is also possible to set up a file server.
  • front-end file server 1 is intended for storing data generated by the data source 4, the present invention is not limited to this.
  • the front-end file server 1 may store data generated by another device or may store data generated in the front-end file server 1.
  • the front-end file server 1 uses a general-purpose computer and includes a CPU 10, a memory 11, an I / F 12, and an auxiliary storage device 13.
  • CPU 10 executes a program stored in the memory 11.
  • the functions of the front-end file server 1 can be realized by the CPU 10 executing the program.
  • the CPU 10 executes a program stored in the memory 11.
  • the memory 11 stores a program executed by the CPU 10 and information necessary for executing the program.
  • the program and information stored in the memory 11 will be described later.
  • the I / F 12 controls data communication with an external device.
  • the auxiliary storage device 13 is a device that provides a large-capacity storage area.
  • a RAID may be configured using a plurality of auxiliary storage devices 13.
  • the auxiliary storage device 13 is used as the data storage area 170.
  • the memory 11 stores programs for realizing the file management unit 100, the file access control unit 110, the metadata generation unit 120, and the file movement control unit 130.
  • the memory 11 also stores a tiered storage configuration management table 150, a migration condition management table 151, a file correspondence management table 152, an operation management table 153, a storage file management table 154, and a file migration management table 155.
  • the memory 11 also has a storage area for realizing the data reception buffer 160.
  • Hierarchical storage configuration management table 150 migration condition management table 151, file correspondence management table 152, operation management table 153, storage file management table 154, and file migration management table 155 will be described later with reference to FIGS.
  • the file management unit 100 provides a file system used for managing files. Note that the file management unit 100 may provide a database instead of the file system.
  • the file access control unit 110 receives a file access request transmitted from an external device such as the client 5 and controls access to the file.
  • the file access control unit 110 reads the file from the back-end file server 2 and returns the read file to the client 5.
  • the file access control unit 110 is read from the storage device to the back-end file server 2. The file is returned to the client 5.
  • the metadata generation unit 120 executes a metadata extraction process for generating the metadata file 181 from the actual data file 180. Details of the metadata extraction processing will be described later with reference to FIG.
  • the metadata generation unit 120 includes a plurality of functional units such as an actual data acquisition unit 121, a metadata extraction unit 122, and a metadata file generation unit 123.
  • the actual data acquisition unit 121 reads the actual data file 180 temporarily stored in the data reception buffer 160 as appropriate.
  • the actual data acquisition unit 121 executes a filtering process for acquiring a file for extracting metadata from the actual data file 180 based on the conditions and processing contents registered in the movement condition management table 151. Details of the filtering process will be described later with reference to FIG.
  • the metadata extraction unit 122 extracts metadata from the acquired actual data file 180. Specifically, the metadata extraction unit 122 specifies the data type of the actual data file 180, and executes a metadata extraction process to extract metadata based on the specified data type.
  • the metadata extraction unit 122 may specify the data type of the actual data file 180 based on the extension name given to the file name of the actual data file 180. Further, the metadata extraction unit 122 may analyze the actual data file 180 and specify the data type of the actual data file 180 based on the analysis result. Therefore, the metadata extraction unit 122 holds a data type and a metadata extraction module (not shown) corresponding to the data type.
  • the metadata extraction unit 122 mainly extracts the following two types of information as file metadata.
  • the metadata extraction unit 122 extracts information given as file attribute information from the file. Specifically, the file size, file owner, file creation date and time, file last update date and time, and the like are extracted.
  • the metadata extraction unit 122 analyzes the contents of the file and extracts information acquired as a result of the analysis. Specifically, when a building is reflected in the image of the image file, feature information such as the shape and color of the building is extracted. When the location of the building is identifiable, the metadata extraction unit 122 also extracts feature information such as the name of the building and the location.
  • the metadata file generation unit 123 generates file format data, that is, a metadata file 181 from the extracted metadata. Specifically, the following processing is executed.
  • the metadata file generation unit 123 newly generates a metadata file 181 including the extracted metadata, and stores the corresponding actual data file 180 and metadata file 181 in the data storage area 170. At this time, the metadata file generation unit 123 stores the correspondence relationship between the actual data file 180 and the metadata file 181 in the file correspondence management table 152.
  • the metadata file generation unit 123 stores information regarding the storage locations of the actual data file 180 and the metadata file 181 in the storage file management table 154.
  • the metadata file generation unit 123 stores, in the operation management table 153, the remaining capacity of the data storage area 170 that has changed along with the storage of the file in the data storage area 170, that is, information on the remaining capacity of the auxiliary storage device 13, and the like.
  • the file movement control unit 130 selects a movement candidate file from the files stored in the data storage area 170, and controls the movement of the selected file.
  • the file movement control unit 130 includes a plurality of functional units such as an information acquisition unit 131, a determination unit 132, and a file movement unit 133.
  • the information acquisition unit 131 acquires information from each management table used for selecting a movement candidate file. Specifically, the information acquisition unit 131 acquires information from the hierarchical storage configuration management table 150, the migration condition management table 151, the file correspondence management table 152, the operation management table 153, the storage file management table 154, and the file migration management table 155. To do.
  • the determination part 132 determines the processing content of a movement process based on the information acquired from each management table. Specifically, the following processing is executed.
  • the determination unit 132 compares the condition registered in the movement condition management table 151 with the information acquired from the operation management table 153, and searches for a matching condition.
  • the determination unit 132 specifies an action corresponding to the matched condition, and determines the specified action as the processing content of the movement process.
  • the determination unit 132 selects a movement candidate file based on the processing content of the determined movement process, and stores information on the selected file in the file movement management table 155.
  • the determination unit 132 refers to the storage file management table 154 based on the processing content of the determined transfer process, and selects a transfer candidate file from the files stored in the data storage area 170.
  • the determination unit 132 stores information on the selected movement candidate file in the file movement management table 155.
  • the determination unit 132 refers to the file correspondence management table 152 in order to distinguish whether the migration candidate file is the actual data file 180 or the metadata file 181. In addition, the determination unit 132 appropriately acquires the identification information of the file server that is the migration destination file from the hierarchical storage configuration management table 150.
  • the file mover 133 moves the move candidate file to a predetermined move destination file server. Specifically, the file mover 133 refers to the file move management table 155 and sequentially moves the move candidate files to the move destination file server.
  • the back-end file server 2 is a general-purpose computer and includes a CPU 20, a memory 21, an I / F 22, and an auxiliary storage device 23.
  • CPU 20 executes a program stored in memory 21.
  • the functions of the back-end file server 2 can be realized by the CPU 20 executing the program.
  • a process is described with a program as a subject, it indicates that the CPU 20 is executing the program.
  • the memory 21 stores a program executed by the CPU 20 and information necessary for executing the program.
  • the program and information stored in the memory 21 will be described later.
  • the I / F 22 controls data communication with an external device.
  • the auxiliary storage device 23 is a device that provides a large-capacity storage area.
  • the auxiliary storage device 23 may be, for example, an HDD or an SSD.
  • a RAID may be configured using a plurality of auxiliary storage devices 23.
  • the memory 21 stores programs for realizing the file management unit 201 and the file access control unit 202.
  • the memory 21 also stores a back-end storage file management table 250.
  • the file management unit 201 and the file access control unit 202 are the same as the file management unit 100 and the file access control unit 110 included in the front-end file server 1, description thereof is omitted.
  • the back end storage file management table 250 will be described later.
  • each back-end file server 2 may be different.
  • the data source 4 is a general-purpose computer and includes a CPU 40, a memory 41, an I / F 42, and an auxiliary storage device 43.
  • CPU 40 executes a program stored in memory 41.
  • the function of the data source 4 can be realized by the CPU 40 executing the program.
  • a process is described with a program as a subject, it indicates that the CPU 40 is executing the program.
  • the memory 41 stores a program executed by the CPU 40 and information necessary for executing the program.
  • the program and information stored in the memory 41 will be described later.
  • the I / F 42 controls data communication with an external device.
  • the auxiliary storage device 43 is a device that provides a large-capacity storage area.
  • an HDD and an SSD can be considered.
  • the memory 11 stores programs for realizing the data management unit 401 and the data providing unit 402.
  • the data management unit 401 manages data generated and held in the data source 4. Specifically, a file system function or a database function is provided, and control is performed so that the function can be used.
  • the data providing unit 402 provides various data managed by the data management unit 401 to the outside of the data source 4.
  • processing for storing data stored in the data source 4 as a file in the front-end file server 1 or processing for providing the file is executed in response to a request from the front-end file server 1 or the like.
  • the data source 4 may be realized by a device other than a general-purpose computer.
  • a device other than a general-purpose computer.
  • an apparatus in which a function for managing the recording data is incorporated into a camera apparatus that records a moving image can be considered.
  • the client 5 is a general-purpose computer and includes a CPU 50, a memory 51, an I / F 52, and an auxiliary storage device 53.
  • CPU 50 executes a program stored in memory 51.
  • the functions of the client 5 can be realized by the CPU 50 executing the program.
  • the CPU 50 executes a program stored in memory 51.
  • the functions of the client 5 can be realized by the CPU 50 executing the program.
  • the CPU 50 executes a program stored in memory 51.
  • the functions of the client 5 can be realized by the CPU 50 executing the program.
  • the memory 51 stores a program executed by the CPU 50 and information necessary for executing the program.
  • the program and information stored in the memory 51 will be described later.
  • the I / F 52 controls data communication with an external device.
  • the auxiliary storage device 53 is a device that provides a large-capacity storage area.
  • an HDD and an SSD can be considered.
  • the memory 51 stores programs for realizing the file management unit 501, the file access client control unit 502, and the file share management unit 503.
  • file management unit 201 is the same as the file management unit 100 included in the front-end file server 1, description thereof is omitted.
  • the file access client control unit 502 transmits a file access request to the front-end file server 1 based on the file access request input from the user or the like, and provides the acquired file to the request source.
  • the file access client control unit 502 acquires the actual data file 180 and the metadata file 181.
  • the transmission destination of the file access request is not limited to the front end file server 1, and the access request may be transmitted to the back end file server 2.
  • the file sharing management unit 503 provides a file sharing function used for accessing a file shared by each file server from the client 5.
  • FIG. 3 is an explanatory diagram illustrating a configuration example of the hierarchical storage configuration management table 150 according to the first embodiment of this invention.
  • the hierarchical storage configuration management table 150 manages information related to the configuration of the file server that configures the hierarchical storage.
  • information about the front-end file server 1 and the back-end file server 2 is stored.
  • the tier storage configuration management table 150 includes a tier name 1501, storage identification information 1502, media type information 1503, and access protocol 1504.
  • the columns included in the hierarchical storage configuration management table 150 are not limited to those described above.
  • the tier name 1501 stores information indicating the management tier of the file server in the tier storage. For example, “Tier0” is stored in the record corresponding to the front-end file server 1, and “Tier1” is stored in the record corresponding to the back-end file server A (2-1).
  • Storage identification information 1502 stores file server identification information.
  • the storage identification information 1502 stores the IP address or host name of the file server.
  • the media type information 1503 stores information indicating the type of recording medium used by the file server for storing data. For example, information such as “SSD”, “HDD”, or “magnetic tape” is stored.
  • the media type information 1503 may store more detailed information. For example, when data is stored in the HDD, information regarding a storage interface such as a serial number, SAS or SATA, or a vendor name may be stored. When data is stored in the tape device, tape cartridge identification information, device vendor name, and the like may be stored.
  • the access protocol 1504 stores information related to the access protocol used when outputting an access request to the file server. For example, identification information such as HTTP (Hyper Text Transfer Protocol), NFS (Network File System), or LTFS (Linear Tape File System) is stored.
  • HTTP Hyper Text Transfer Protocol
  • NFS Network File System
  • LTFS Linear Tape File System
  • the access protocol 1504 may store more detailed information. For example, when an authentication process is required when accessing the file server, information for identifying an authentication method in the authentication process may be stored.
  • FIG. 4 is an explanatory diagram illustrating a configuration example of the movement condition management table 151 according to the first embodiment of this invention.
  • the movement condition management table 151 manages the selection condition of the movement candidate file and the action for the movement candidate file in association with each other. Specifically, the movement condition management table 151 includes a condition management number 1511, a determination order 1512, a condition 1513, and an action 1514. The columns included in the movement condition management table 151 are not limited to those described above.
  • the condition management number 1511 stores a number for managing an algorithm for selecting a migration candidate file. Records having the same number stored in the condition management number 1511 indicate the same algorithm. Further, when a different number is stored in the condition management number 1511, it indicates a different algorithm. As described above, in this embodiment, a plurality of algorithms can be registered.
  • the determination order 1512 stores the determination order in the same algorithm.
  • the determination unit 132 determines the conditions in ascending order of the determination order 1512 values.
  • the condition 1513 stores a conditional expression for determination.
  • a conditional expression that can be determined by using information acquired from an operation management table 153 described later is stored.
  • the condition 1513 may store a conditional expression obtained by combining a plurality of conditional expressions using an AND condition or an OR condition.
  • a conditional expression using information not included in the operation management table 153 may be stored.
  • the action 1514 stores the content of the process executed when the condition 1513 is met. Specifically, information about the movement destination of the actual data file 180 or the movement destination of the metadata file 181 can be set. In addition, the processing contents considering the system state can be stored as follows.
  • the following processing contents are stored in case the remaining capacity of the data reception buffer 160 decreases.
  • the filtering process is executed on the data stored in the data reception buffer 160, and then the metadata extraction process is executed.
  • the trigger for executing the metadata extraction process is not clear, such as when additional data is stored in the log file and movie file by executing the process described above, it is compulsory when the specified conditions are met.
  • the actual data can be extracted as a file and the metadata extraction process can be executed.
  • the following processing contents are stored in case the storage capacity of the front-end file server 1 is not enough.
  • the process of moving the moved real data file 180 to the front-end file server 1 or the deleted real data A process of acquiring the file 180 from the data source 4 is executed.
  • the condition 1513 stores the condition “HDD usage rate” ⁇ 30% ”, and the action 1514 records the processing content“ set “front-end file server” as “actual data transfer destination” ”. May be registered in the movement condition management table 151.
  • the actual data file 180 When the size occupied by the actual data file 180 with respect to the total capacity of the HDD in the data storage area 170 is larger than a predetermined threshold, the actual data file 180 is transferred to a file server different from the transfer destination of the metadata file 181.
  • condition 1513 “HDD usage rate” ⁇ 80% AND “actual data file total size ratio” ⁇ 75% ”is stored, and action 1514 is“ backend to “metadata transfer destination”. It is only necessary to register in the migration condition management table 151 the processing contents of “setting file server A” and “setting“ back-end file server B ”as“ actual data transfer destination ””.
  • the above-described process is a process for quickly securing a storage area for storing the metadata file 181 stored in the data storage area 170.
  • actual data file total size ratio indicates the ratio of the total size of all actual data files 180 stored in the data storage area 170 to the total capacity of the HDD. For example, if the total capacity of the HDD is 10 GB and the total size of the actual data files 180 is 8 GB, the actual data file total size ratio is 80%.
  • the total size of the actual data file 180 may be managed as management information.
  • the operation management table 153 includes the actual data file total size 1533.
  • FIG. 5 is an explanatory diagram showing a configuration example of the file correspondence management table 152 according to the first embodiment of the present invention.
  • the file correspondence management table 152 manages the correspondence between the actual data file 180 stored in the data storage area 170 and the metadata file 181. Specifically, the file correspondence management table 152 includes an actual data file name 1521, a metadata file name 1522, an actual data file last update date / time 1523, and a metadata file last update date / time 1524.
  • the columns included in the file correspondence management table 152 are not limited to those described above.
  • the actual data file name 1521 stores the file name of the actual data file 180.
  • the actual data file name 1521 stores the file name when the actual data file 180 is stored in the front end file server 1.
  • the metadata file name 1522 stores the file name of the metadata file 181.
  • the metadata file name 1522 stores a file name given to the metadata file 181 when the metadata file 181 is generated.
  • the file name of the metadata file 181 given by the metadata generation unit 120 is converted based on a predetermined rule.
  • the file name conversion rule may be registered by a system administrator or the like.
  • the actual data file last update date and time 1523 stores the last update date and time held as the file attribute of the actual data file 180.
  • the metadata file last update date and time 1524 stores the last update date and time held as the file attribute of the metadata file 181.
  • the date and time stored in the metadata file last update date and time 1524 is the date and time when the metadata generation unit 120 extracts the metadata from the actual data file 180 and generates the metadata file 181 from the extracted metadata. .
  • FIG. 6 is an explanatory diagram illustrating a configuration example of the operation management table 153 according to the first embodiment of this invention.
  • the operation management table 153 manages information indicating the operation state of each component in the front-end file server 1. Information indicating the operating state is collected periodically or when necessary, and the collected information is stored in the operation management table 153.
  • the operation management table 153 includes a storage area usage rate 1531, a storage area total capacity 1532, a real data file total size 1533, a metadata file total size 1534, a data reception buffer memory usage size 1535, and a last update date 1536.
  • the storage area usage rate 1531 stores the usage rate value of the storage medium used as the data storage area 170.
  • the value stored in the storage area usage rate 1531 is calculated by dividing the size of all files stored in the data storage area 170 by the size of the data storage area 170.
  • the storage area total capacity 1532 stores a value indicating the overall size of the storage medium used as the data storage area 170.
  • the real data file total size 1533 stores the total size of all real data files 180 stored in the data storage area 170.
  • the metadata file total size 1534 stores the total size of all metadata files 181 stored in the data storage area 170.
  • the ratio of the actual data file 180 or the ratio of the metadata file 181 to all the files stored in the data storage area 170 can be determined. Can be calculated.
  • the data reception buffer memory use size 1535 stores the total size of all actual data files 180 stored in the data reception buffer 160. Since the actual data file 180 stored in the data reception buffer 160 is stored in the data storage area 170 after a predetermined process is executed, it is useful information for grasping the operation status of the data storage area 170. Become one.
  • the last update date / time 1536 stores the date / time when the record was registered or the value of the record was last updated.
  • the columns included in the operation management table 153 are not limited to those described above.
  • the CPU usage rate, network performance (throughput or response time), HDD I / O performance (throughput or response time), cache hit rate, and the like may be managed.
  • the front-end file server 1 does not need to hold the operation management table 153 itself, and may acquire each information individually at a predetermined timing. For example, regarding the storage area usage rate 1531, the front-end file server 1 does not periodically execute a command for acquiring information, but executes the command when necessary to acquire information. May be.
  • FIG. 7 is an explanatory diagram illustrating a configuration example of the storage file management table 154 according to the first embodiment of this invention.
  • the stored file management table 154 manages information related to files stored in the file server constituting the hierarchical storage.
  • the storage file management table 154 includes a file name 1541, a file storage hierarchy 1542, a file path 1543, and a last update date / time 1544.
  • the columns included in the storage file management table 154 are not limited to those described above.
  • the file name 1541 stores the access path name of the file stored in the tiered storage.
  • the access path name is used when accessing a file stored in the hierarchical storage. For example, when the user requests access to a file stored in the tiered storage, the access path name stored in the file name 1541 is used.
  • the file storage hierarchy 1542 stores the identification information of the file server where the file is actually stored.
  • the information stored in the file storage tier 1542 is the same information as the tier name 1501 in the tier storage configuration management table 150.
  • the file storage hierarchy 1542 may store identification information of each file server that stores the file.
  • the identification information of the data source 4 that is the generation source of the actual data file is stored in the file storage hierarchy 1542 of the record corresponding to the actual data file 180.
  • the front-end file server 1 transmits an acquisition request for the actual data file to the data source 4, and requests the acquired actual data file. Provide to the original.
  • the file path 1543 stores the path name of the file in the file server.
  • the file path 1543 may store the path name of each file server that stores the file.
  • the path name in the device may be stored.
  • the last update date / time 1544 stores the date / time when the record was registered or the date / time when the value of the record was last updated.
  • the front-end file server 1 When the front-end file server 1 receives the file access request, the front-end file server 1 can specify the file server to which the file access request is transferred based on the stored file management table 154.
  • FIG. 8 is an explanatory diagram illustrating a configuration example of the file movement management table 155 according to the first embodiment of this invention.
  • the file movement management table 155 manages the movement candidate files extracted by the file movement control unit 130 and the progress status of the movement process. Specifically, the file move management table 155 includes an actual data file name 1551, a metadata file name 1552, an actual data file move state 1553, a metadata file move state 1554, an actual data file move destination 1555, and a metadata file move destination. 1556 and last update date and time 1557.
  • the columns included in the storage file management table 154 are not limited to those described above.
  • the real data file name 1551 stores the access path name of the migration candidate real data file 180.
  • the information stored in the actual data file name 1551 may be the same as the actual data file name 1521 in the file correspondence management table 152.
  • the metadata file name 1552 stores the access path name of the migration candidate metadata file 181.
  • the information stored in the metadata file name 1552 may be the same as the metadata file name 1522 of the file correspondence management table 152.
  • the real data file movement status 1553 stores information indicating the progress status of the migration processing of the migration candidate real data file 180. For example, when “not moved” is stored in the actual data file movement state 1553, this indicates that the actual data file 180 has not been moved, and when “moved” is stored in the actual data file movement state 1553, Indicates that the data file 180 has been stored.
  • the metadata file movement status 1554 stores information indicating the progress status of the migration processing of the migration candidate metadata file 181.
  • the information stored in the metadata file move state 1554 is the same as that stored in the actual data file move state 1553.
  • the real data file migration destination 1555 stores identification information of the file server that is the migration destination of the migration candidate real data file 180.
  • the information stored in the actual data file migration destination 1555 may be the same information as the tier name 1501 of the tier storage configuration management table 150.
  • the metadata file migration destination 1556 stores identification information of the file server that is the migration destination of the migration candidate metadata file 181.
  • the information stored in the metadata file migration destination 1556 may be the same information as the tier name 1501 of the tier storage configuration management table 150.
  • the last update date and time 1557 stores the date and time when the record was registered or the date and time when the value of the record was last updated.
  • the actual data file 180 associated with the metadata file 181 does not need to be migrated, so the actual data file migration state 1553 and the actual data file migration destination 1555 are displayed. Stores the value of “Null”. Thus, it can be seen that the actual data file 180 is not a movement candidate.
  • FIG. 9 is an explanatory diagram illustrating a configuration example of the back-end storage file management table 250 according to the first embodiment of this invention.
  • the backend storage file management table 250 manages files moved from the frontend file server 1 to the backend file server 2 in the hierarchical storage. Specifically, the back-end storage file management table 250 includes a file path 2501, a metadata file storage hierarchy 2502, an actual data file storage hierarchy 2503, a metadata file last update date and time 2504, and an actual data file last update date and time 2505. .
  • the columns included in the back-end storage file management table 250 are not limited to those described above.
  • the file path 2501 stores identification information for specifying the file moved to the back-end file server 2.
  • the file path 2501 stores a file path that can be uniquely identified in the hierarchical storage. For example, the same information as the file path 1543 is stored.
  • the metadata file storage hierarchy 2502 stores information for specifying a file server in which the metadata file is stored.
  • the metadata file storage hierarchy 2502 stores information indicating the management hierarchy of the file server. For example, the same information as the hierarchy name 1501 is stored.
  • the actual data file storage hierarchy 2503 stores information for specifying a file server in which the actual data file is stored. Specifically, the same information as the metadata file storage hierarchy 2502 is stored.
  • the metadata file last update date and time 2504 stores a new value in the metadata file storage hierarchy 2502 of the record or the date and time when the value was updated.
  • the actual data file last update date and time 2505 stores a new value in the actual data file storage hierarchy 2503 of the record or the date and time when the value was updated.
  • the back-end file server 2 uses the back-end storage file management table 250, so that after the file is moved to the back-end file server 2, the actual data file 180 or the metadata file 181 included in the file is further stored.
  • the movement destination of the actual data file 180 or the metadata file 181 can be specified.
  • the back-end storage file management table 250 is not always necessary, and the storage location of the file may be specified using only the storage file management table 154 held by the front-end file server 1.
  • the back-end file server 2 when a file is moved between the back-end file servers 2, the back-end file server 2 notifies the front-end file server 1 of information regarding the movement destination of the file.
  • the front-end file server 1 reflects the notified information on the storage file management table 154.
  • 10A and 10B are flowcharts for explaining the metadata extraction processing in the embodiment of the present invention.
  • the front-end file server 1 extracts metadata from the actual data file 180, and generates a metadata file 181 using the extracted metadata. Further, the front end file server 1 stores the generated metadata file 181 in the data storage area 170. Further, the front-end file server 1 determines whether to store the actual data file 180 in the data storage area 170 according to the operating state.
  • the metadata generation unit 120 starts the metadata extraction process periodically when the actual data file 180 is received or when an instruction from a system administrator or the like is received.
  • the metadata generation unit 120 acquires information from the operation management table 153 and the movement condition management table 151 (step S101).
  • the metadata generation unit 120 acquires the operation information from the operation management table 153 in order to grasp the operation state of the front-end file server 1, but the present invention is not limited to this.
  • the metadata generation unit 120 executes management commands and the like provided by the file system used by the front-end file server 1 to operate the total storage area capacity, the storage area usage rate, the memory usage size, and the like. Information may be acquired.
  • the metadata generation unit 120 determines whether or not the condition 1513 of the movement condition management table 151 is satisfied based on the acquired information (step S103). Specifically, the following processing is executed.
  • the metadata generation unit 120 compares the operation information acquired from the operation management table 153 with the conditions of the movement condition management table 151, and searches for a condition 1513 that matches the acquired operation state. At this time, the metadata generation unit 120 searches all the conditions 1513 that match the acquired operating state.
  • the metadata generation unit 120 executes processing in ascending order of the condition management number 1511. In addition, when there are a plurality of determination items for one determination algorithm, the metadata generation unit 120 further performs processing in ascending order of the determination order 1512.
  • the metadata generation unit 120 determines that the condition 1513 of the movement condition management table 151 is satisfied.
  • the metadata generation unit 120 refers to the action 1514 of the record corresponding to the condition 1513 that matches the operating condition, and temporarily holds the processing content of the process to be executed.
  • the metadata generation unit 120 may delete the retained processing content after the metadata extraction processing is completed.
  • the metadata generation unit 120 ends the process.
  • step S103 determines whether to move the file stored in the data storage area 170 (step S105).
  • the metadata generation unit 120 determines whether or not the processing content that is temporarily stored includes processing content that instructs execution of the file move processing.
  • step S105 If it is determined not to move the file stored in the data storage area 170 (No in step S105), the metadata generation unit 120 proceeds to step S109.
  • step S105 When it is determined that the file stored in the data storage area 170 is to be moved (Yes in step S105), the metadata generation unit 120 instructs the file movement control unit 130 to execute the movement candidate file selection process (step S107). ), The process proceeds to step S109.
  • the metadata generation unit 120 includes the processing content related to the file move processing from the temporarily stored processing content in the execution instruction.
  • the file movement control unit 130 executes a movement candidate file selection process described later.
  • the metadata generation unit 120 determines whether to generate the metadata file 181 (step S109).
  • the metadata generation unit 120 determines whether or not the processing content that is temporarily stored includes processing content that instructs generation of the metadata file 181.
  • the metadata generation unit 120 ends the process.
  • the metadata generation unit 120 determines whether the processing has been completed for all the actual data files 180 stored in the data reception buffer 160. (Step S111).
  • the metadata generation unit 120 instructs the actual data acquisition unit 121 to acquire the actual data file 180, and determines whether or not the actual data file 180 has been acquired. When the actual data file 180 is not acquired, the metadata generation unit 120 determines that the processing has been completed for all the actual data files 180 stored in the data reception buffer 160.
  • the metadata generation unit 120 ends the processing.
  • the metadata generation unit 120 selects one arbitrary actual data file 180 from the data reception buffer 160. (Step S113).
  • the metadata generation unit 120 instructs the actual data acquisition unit 121 to select the actual data file 180.
  • the actual data acquisition unit 121 selects one arbitrary actual data file 180 from the data reception buffer 160.
  • the metadata generation unit 120 determines whether or not the selected actual data file 180 is closed (step S115).
  • the actual data acquisition unit 121 determines whether or not the selected actual data file 180 is in a closed state in the file system used by the front-end file server 1.
  • the closed state indicates a state in which all data requested to be written are reflected in the file.
  • step S115 If it is determined that the selected actual data file 180 is closed (step S115 is Yes), the metadata generation unit 120 proceeds to step S123.
  • the metadata generation unit 120 determines whether the size of the selected actual data file 180 is larger than a predetermined threshold value. Is determined (step S117).
  • the actual data acquisition unit 121 determines whether the size of the selected actual data file 180 is larger than a predetermined threshold value.
  • the size of the actual data file 180 can be obtained from attribute information provided by the file system.
  • the predetermined threshold value is set in advance by the system administrator. The system administrator can change the predetermined threshold value as appropriate.
  • step S117 the actual data file 180 having a size larger than a predetermined threshold is selected from the actual data files stored in the data reception buffer 160, and the filtering process is executed on the actual data file 180. It can. Further, this reduces the amount of use of the data reception buffer 160, so that the data reception buffer 160 can be used effectively.
  • step S117 If it is determined that the size of the selected actual data file 180 is larger than the predetermined threshold value (Yes in step S117), the metadata generation unit 120 proceeds to step S121.
  • the metadata generation unit 120 determines the predetermined size after the selected actual data file 180 is generated. It is determined whether or not the period has elapsed (step S119).
  • the actual data acquisition unit 121 determines whether or not a predetermined period has elapsed since the selected actual data file 180 was generated.
  • the generation date and time of the actual data file 180 can be obtained from attribute information provided by the file system.
  • the predetermined period is set in advance by the system administrator. The system administrator can change the predetermined period as appropriate.
  • step S119 the actual data file 180 that has been generated for a predetermined period is selected from the actual data files 180 stored in the data reception buffer 160, and the actual data file 180 is filtered. Processing can be executed. Further, this reduces the amount of use of the data reception buffer 160, so that the data reception buffer 160 can be used effectively.
  • the metadata generation unit 120 returns to step S111 and executes the same processing. . This is because the selected actual data file 180 is not a metadata extraction target. Therefore, at the present time, the processing after step S121 is not executed on the selected actual data file 180.
  • step S119 When it is determined that a predetermined period has elapsed since the selected actual data file 180 was generated (Yes in step S119), the metadata generation unit 120 proceeds to step S121.
  • Step S117 is Yes or Step S119 is Yes
  • the metadata generation unit 120 performs a filtering process on the selected actual data file 180 (Step S121). Specifically, the following processing is executed.
  • the actual data acquisition unit 121 instructs the file access control unit 110 to suspend the file access request for the selected actual data file 180.
  • the actual data acquisition unit 121 instructs the file access control unit 110 to execute an update process on the selected actual data file 180. As a result, data that has not been updated is reflected in the selected actual data file 180.
  • the actual data acquisition unit 121 changes (renames) the file name of the selected actual data file 180 to the file name for metadata extraction. Furthermore, the actual data acquisition unit 121 newly generates a file with the file name before renaming. At this time, data is not included in the generated file.
  • the actual data acquisition unit 121 instructs the file access control unit 110 to resume the file access request for the selected actual data file 180.
  • the size of the selected actual data file 180 in the data reception buffer 160 becomes zero, and the data stored in the actual data file 180 can be acquired as a separate file.
  • the actual data acquisition unit 121 outputs the acquired file to the metadata extraction unit 122 and instructs the extraction of metadata.
  • the front-end file server 1 can extract metadata even when data is continuously added to an open file by executing the filtering process as described above.
  • a usage mode in which data is continuously added to an open file there are a case where log data is sequentially stored in a log file, a case where recorded data is stored sequentially, and the like.
  • step S115 is Yes or the processing of step S121 is executed
  • the metadata generation unit 120 determines a metadata extraction method (step S123).
  • the metadata extraction unit 122 determines the data type of the selected actual data file 180, and determines a metadata extraction method based on the determination result. For example, the metadata extraction unit 122 determines a metadata extraction module (not shown) for extracting metadata. In this case, the metadata extraction unit 122 executes the following processing using the determined metadata extraction module.
  • a method for determining the data type for example, a method for determining the data type based on the extension name given to the file name of the selected actual data file 180, the contents of the actual data file 180 are read and read.
  • a method of determining the data type based on the content that has been performed is conceivable.
  • the metadata generation unit 120 extracts metadata of the selected actual data file 180, generates a metadata file 181 using the extracted metadata, and stores the generated metadata file 181 in the data storage area 170.
  • Store step S125. Specifically, the following processing is executed.
  • the metadata extraction unit 122 extracts metadata from the selected actual data file 180 using the metadata extraction module determined in step S113.
  • the metadata extraction unit 122 outputs the extracted metadata to the metadata file generation unit 123.
  • the metadata file generation unit 123 generates a metadata file 181 from the input metadata, and stores the generated metadata file 181 in the data storage area 170.
  • the metadata generation unit 120 determines whether or not the selected actual data file 180 is stored in the data storage area 170 (step S127).
  • the metadata generation unit 120 determines whether or not the processing content that is temporarily stored includes processing content that instructs to store the actual data file 180 in the data storage area.
  • Step S127 the metadata generation unit 120 stores the selected actual data file 180 in the data storage area 170. (Step S129), the process proceeds to Step S133.
  • the metadata generation unit 120 stores the storage location information of the actual data file 180 selected as the metadata file 181. Is added (step S131), and the process proceeds to step S133.
  • the storage location information of the actual data file 180 includes information on the data source 4 from which the actual data file 180 can be acquired or a file server in which the actual data file 180 is stored. That is, information for accessing the actual data file 180 is added to the metadata file 181.
  • the storage location information of the actual data file 180 may be designated in advance by the system administrator.
  • the metadata file to which the storage destination information is added will be described later with reference to FIG.
  • step S129 or step S131 After execution of the process of step S129 or step S131, the metadata generation unit 120 updates each management table (step S133), and then returns to step S111. Specifically, the following processing is executed.
  • the metadata generation unit 120 stores information on the selected actual data file 180 and the generated metadata file 181 in the file correspondence management table 152.
  • a record in which the file name of the actual data file 180 and the file name of the metadata file 181 are associated with each other is stored.
  • the metadata generation unit 120 updates the storage file management table 154. Specifically, the metadata generation unit 120 stores the file name of the file in the file name 1541 of the file stored in the data storage area 170, and the identification information of the front-end file server 1 in the file storage hierarchy 1542. Store. Also, the metadata generation unit 120 stores “Null” in the file path 1543 because the file has not been moved to another file server at this time.
  • the metadata generation unit 120 updates the operation management table 153. Specifically, the metadata generation unit 120 acquires the memory size of the data reception buffer 160, the usage rate of the data storage area 170, and the like, and stores the acquired various types of information in the operation management table 153.
  • the metadata generation unit 120 specifies the process to be executed based on the operating state of the front-end file server 1, but the present invention is not limited to this.
  • the metadata generation unit 120 may instruct the determination unit 132 to execute the determination process.
  • the determination unit 132 executes a determination process described later, and outputs a determination result to the metadata generation unit 120. Therefore, the process of step S101 and step S103 can be omitted.
  • the metadata generation unit 120 may determine whether or not the record of the selected actual data file 180 is registered with reference to the file movement management table 155. When the record of the selected actual data file 180 is registered in the file movement management table 155, the metadata generation unit 120 determines that the selected actual data file 180 is not stored in the data storage area 170.
  • the metadata file 181 is generated when there is no room in the data reception buffer 160, but the present invention is not limited to this.
  • the metadata file 181 may be always generated without depending on the operating state of the front-end file server 1.
  • the metadata generation unit 120 adds the storage location information to the metadata file 181 when the actual data file 180 is deleted, but the present invention is not limited to this.
  • the storage location information may be added to all the metadata files 181.
  • the process of step S109, step S127, and step S129 is abbreviate
  • FIG. 11 is a flowchart for explaining the selection process of the movement candidate file according to the first embodiment of the present invention.
  • the front end file server 1 selects a move candidate file to be moved to the back end file server 2 from the actual data file 180 and the metadata file 181 stored in the data storage area 170. Further, the front-end file server 1 stores information on the file selected as the migration candidate file in the file migration management table 155.
  • the file movement control unit 130 executes the movement candidate file selection process periodically when an execution instruction is received from the metadata generation unit 120 or when an instruction from a system administrator or the like is received.
  • the file migration control unit 130 acquires information from the hierarchical storage configuration management table 150, the operation management table 153, and the migration condition management table 151 (step S201).
  • the file movement control unit 130 instructs the information acquisition unit 131 to acquire information.
  • the information acquisition unit 131 acquires necessary information from the hierarchical storage configuration management table 150, the operation management table 153, and the migration condition management table 151.
  • the information acquisition unit 131 acquires the operation information from the operation management table 153 in order to grasp the operation state of the front end file server 1, but the present invention is not limited to this.
  • the information acquisition unit 131 executes a management command or the like provided by the file system used by the front-end file server 1, and operates information such as the total capacity of the storage area, the storage area usage rate, and the memory usage size May be obtained.
  • the file movement control unit 130 determines the content of the movement process based on the acquired information (step S203). Specifically, the following processing is executed.
  • the file movement control unit 130 instructs the determination unit 132 to determine the processing content of the movement process.
  • the determination unit 132 compares the operation information acquired from the operation management table 153 with the conditions of the movement condition management table 151 and determines whether or not the operation state matches the condition 1513 of the movement condition management table 151.
  • the determination unit 132 executes processing in ascending order of the condition management number 1511.
  • the determination unit 132 further performs processing in ascending order of the determination order 1512.
  • the determination unit 132 changes the action 1514 of the record corresponding to the matching condition 1513 to the processing content of the movement process. Determine as.
  • the execution instruction When execution is instructed from the metadata generation unit 120, the execution instruction includes the processing contents of the movement process, and thus the processing in step S201 and step S203 may be omitted.
  • the file movement control unit 130 instructs the metadata generation unit 120 to execute filtering processing and metadata extraction processing (step S205). At this time, if the actual data file 180 to be processed is specified in the action 1514, the file movement control unit 130 instructs the metadata generation unit 120 to execute the process on the actual data file 180 to be processed. If the actual data file 180 to be processed is not specified in the action 1514, the file movement control unit 130 instructs the metadata generation unit 120 to execute processing for all the actual data files 180.
  • the metadata generation unit 120 executes the processing from step S115 to step S121 on the specified actual data file 180.
  • step S205 is executed when the action 1514 stores information indicating that the filtering process and the metadata extraction process are to be executed. Therefore, when the above-described information is not stored in the action 1514, the process of step S205 is omitted.
  • the file movement control unit 130 determines whether or not the movement candidate file selection process has been completed (step S207).
  • the file movement control unit 130 determines whether or not the processing has been completed for all of the movement contents determined in step S203. When it is determined that the processing has been completed for all the movement processing contents, the file movement control unit 130 determines that the movement candidate file selection processing has been completed.
  • the file movement control unit 130 selects a movement candidate file based on the processing content of the determined movement process.
  • the file migration control unit 130 selects a migration candidate file for each migration process content. That is, the processing from step S211 to step S219 is executed sequentially.
  • the movement candidate file when a movement candidate file can be selected in parallel for processing contents of a plurality of movement processes, the movement candidate file may be selected in parallel.
  • step S207 If it is determined that the migration candidate file selection process has been completed (Yes in step S207), the file migration control unit 130 ends the process.
  • the file movement control unit 130 selects a movement candidate file based on the determined content of the movement process (step S207). S209 to step S219).
  • the file migration control unit 130 selects a migration candidate file for each process content of the migration process. That is, the processing from step S209 to step S219 is sequentially executed.
  • the file movement control unit 130 may select a movement candidate file in parallel for the processing contents of a plurality of movement processes.
  • step S209 to step S219 will be described.
  • the file movement control unit 130 acquires information from the stored file management table 154 (step S209).
  • the file movement control unit 130 instructs the information acquisition unit 131 to acquire information.
  • the information acquisition unit 131 acquires information from the stored file management table 154. At this time, the information acquisition unit 131 may acquire information on all files stored in the storage file management table 154, or may acquire only information on files stored in the front-end file server 1. .
  • the file movement control unit 130 executes the following processing for the acquired file.
  • a file corresponding to the information acquired from the stored file management table 154 in step S209 is also described as a target file.
  • the file movement control unit 130 determines whether or not processing for all target files has been completed (step S211).
  • step S211 If it is determined that the processing for all the target files has been completed (Yes in step S211), the file movement control unit 130 returns to step S204 and executes the same processing.
  • the file movement control unit 130 selects one arbitrary target file from the plurality of target files (step S213). .
  • the file movement control unit 130 identifies the type of the selected target file (step S215). That is, it is determined whether the selected target file is the actual data file 180 or the metadata file 181.
  • the file movement control unit 130 instructs the information acquisition unit 131 to acquire information from the file correspondence management table 152.
  • the file movement control unit 130 refers to the actual data file name 1521 and the metadata file name 1522 in the file correspondence management table 152 and searches for a file name that matches the file name of the selected target file.
  • the file movement control unit 130 determines that the selected target file is the actual data file 180. If the selected file name matches the file name stored in the metadata file name 1522, the file movement control unit 130 determines that the selected target file is the metadata file 181.
  • the file movement control unit 130 acquires the file name of the related metadata file 181 from the metadata file name 1522 of the corresponding record in the file correspondence management table 152. To do.
  • the file movement control unit 130 acquires the file name of the related actual data file 180 from the actual data file name 1521 of the corresponding record in the file correspondence management table 152. To do.
  • the file movement control unit 130 determines whether or not the target file matches the movement condition (step S217).
  • the file movement control unit 130 instructs the determination unit 132 to execute processing.
  • the determination unit 132 further narrows down the target file based on the processing content of the movement processing determined in step S203, that is, the action 1514.
  • a file having a specific extension in the file name or a file owned by a specific user is excluded from the movement candidates. Is stored.
  • the determination unit 132 executes a movement candidate file narrowing process based on the information stored in the action 1514.
  • step S217 the file movement control unit 130 returns to step S211 and executes the same processing.
  • the file movement control unit 130 stores the information of the target file in the file movement management table 155 (step S219), and returns to step S211. Specifically, the following processing is executed.
  • the file movement control unit 130 adds a new record to the file movement management table 155.
  • the file movement control unit 130 stores the access path name in each of the actual data file name 1551 and the metadata file name 1552 of the added record.
  • the file movement control unit 130 stores “not moved” in the actual data file movement state 1553 and the metadata file movement state 1554 of the added record.
  • the file movement control unit 130 stores information on the migration destination in the actual data file migration destination 1555 and the metadata file migration destination 1556 based on the processing content of the migration processing determined in step S203. If the actual data file 180 is not a migration candidate file, “Null” is stored in the actual data file migration state 1553 and the actual data file migration destination 1555.
  • FIG. 12 is a flowchart for explaining the file move process according to the first embodiment of the present invention.
  • the front-end file server 1 moves the movement candidate file based on the file movement management table 155.
  • the file movement control unit 130 starts the file movement process periodically or when receiving an instruction from a system administrator or the like. Further, the file movement control unit 130 may start the file movement process after the movement candidate file selection process is completed. The execution cycle is preferably longer than the move candidate file selection process.
  • the file movement control unit 130 acquires information from the file movement management table 155 (step S301).
  • the file movement control unit 130 instructs the information acquisition unit 131 to acquire information from the file movement management table 155.
  • the information acquisition unit 131 acquires a record from the file movement management table 155.
  • the information acquisition unit 131 may acquire all the records included in the file movement management table 155 or may acquire a predetermined number of records.
  • the file movement control unit 130 determines whether or not processing has been completed for all the records acquired from the file movement management table 155 (step S303).
  • step S303 If it is determined that the processing has been completed for all the records acquired from the file movement management table 155 (Yes in step S303), the file movement control unit 130 ends the processing.
  • the file movement control unit 130 selects any record from the plurality of acquired records. Is selected (step S305).
  • the file movement control unit 130 instructs the file movement unit 133 to execute processing described below.
  • the selected record is also described as a target record.
  • the file moving unit 133 determines whether or not the metadata file 181 corresponding to the target record has been moved (step S307).
  • the file mover 133 determines whether or not the metadata file move state 1554 of the target record is “not moved”. When it is determined that the metadata file movement state 1554 of the target record is “not moved”, the file moving unit 133 determines that the metadata file 181 corresponding to the target record has not been moved.
  • step S307 When it is determined that the metadata file 181 corresponding to the target record has been moved (Yes in step S307), the file moving unit 133 proceeds to step S319.
  • the file moving unit 133 determines whether or not the actual data file 180 corresponding to the target record is a movement candidate file. Is determined (step S309).
  • the file moving unit 133 determines whether or not the actual data file name 1551 of the target record is “Null”. If the actual data file name 1551 of the target record is “Null”, the file migration unit 133 determines that the actual data file 180 corresponding to the target record is not a migration candidate file.
  • step S309 When it is determined that the actual data file 180 corresponding to the target record is a movement candidate file (Yes in step S309), the file moving unit 133 proceeds to step S317.
  • the file moving unit 133 stores the actual data file 180 in the metadata file 181 corresponding to the target record. Previous information is added (step S311).
  • the storage destination information of the actual data file 180 information stored in the actual data file name 1551 of the file movement management table 155 can be considered. Further, if necessary, information regarding the storage location in the external device such as the data source 4 may be added. The metadata file to which the storage destination information is added will be described later with reference to FIG.
  • the file moving unit 133 determines whether or not to delete the actual data file 180 corresponding to the target record from the data storage area 170 (step S313).
  • the file moving unit 133 determines whether or not the processing content for deleting the actual data file 180 is included in the processing content of the moving processing determined in step S202.
  • step S313 If it is determined not to delete the actual data file 180 corresponding to the target record (No in step S313), the file moving unit 133 proceeds to step S317.
  • the file moving unit 133 deletes the actual data file 180 from the data storage area 170 (step S315).
  • the file moving unit 133 may delete the actual data files 180 sequentially, or may list the actual data files 180 to be deleted and delete them at the end of this process.
  • step S313 is No, or the process of step S315 is executed, the file moving unit 133 moves the metadata file 181 corresponding to the target record to a predetermined file server (step S319).
  • the file moving unit 133 moves the metadata file 181 to the file server corresponding to the metadata file moving destination 1556 of the target record.
  • the metadata file 181 stored in the data storage area 170 may be deleted or replaced with a stub file as it moves.
  • the file moving unit 133 determines whether or not the actual data file 180 corresponding to the target record has been moved (step S319).
  • the file moving unit 133 determines whether or not the actual data file moving state 1553 of the target record is “not moved”. When the actual data file movement state 1553 of the target record is “not moved”, the file moving unit 133 determines that the actual data file 180 corresponding to the target record has not been moved.
  • Step S319 When it is determined that the actual data file 180 corresponding to the target record has been moved (Yes in Step S319), the file moving unit 133 proceeds to Step S323.
  • the file moving unit 133 moves the actual data file 180 to a predetermined file server (step S321).
  • the file moving unit 133 moves the actual data file 180 to the file server corresponding to the actual data file destination 1555 of the target record.
  • the actual data file 180 stored in the data storage area 170 may be deleted or replaced with a stub file as it moves.
  • the file moving unit 133 updates the storage file management table 154 and the operation management table 153 (step S323), and then returns to step S303. Specifically, the following processing is executed.
  • the file moving unit 133 stores the information on the moved destination of the moved metadata file 181 and the moved actual data file 180 in the file storage hierarchy 1542 and the file path 1543 of the corresponding record in the stored file management table 154.
  • the file moving unit 133 updates the operation management table 153 with the updated contents such as the storage area usage rate that have changed with the deletion of the actual data file 180 and the metadata file 181 from the data storage area 170 or the replacement with the stub file. Store.
  • FIG. 13 is a flowchart for explaining file access processing according to the first embodiment of the present invention.
  • the front-end file server 1 provides a file designated by the client 5 operated by the user or the like.
  • the file access control unit 110 receives a file access request (step S401).
  • the file access request has a format conforming to the file access protocol provided by the front-end file server 1 and includes the following two pieces of information.
  • the first information is information for specifying a file to be accessed. Specifically, this is information specifying the file name of the actual data file 180. For example, the same information as the file name 1541 of the stored file management table 154 is included.
  • the second information is information that specifies the actual file to be accessed. For example, information specifying whether the file to be accessed is only the actual data file 180, only the metadata file 181, or the actual data file 180 and the metadata file 181 is included. Hereinafter, this information is also referred to as access designation information.
  • the number of files to be accessed in the file access request may be plural.
  • the file access control unit 110 determines whether the metadata file 181 is included in the access target file (step S403).
  • the file access control unit 110 refers to the access designation information included in the file access request, and determines whether or not the metadata file 181 is designated as an access target file. For example, when the access specification information includes only the metadata file 181 or information specifying either the actual data file 180 or the metadata file 181 as the access target file, the file access control unit 110 may It is determined that the metadata file 181 is included in this file.
  • step S403 If it is determined that the metadata file 181 is not included in the access target file (No in step S403), the file access control unit 110 proceeds to step S409.
  • the file access control unit 110 determines the access target meta data based on the file correspondence management table 152 and the stored file management table 154. Information indicating the storage location of the data file 181 is acquired (step S405). Specifically, the following processing is executed.
  • the file access control unit 110 refers to the file correspondence management table 152 and searches for a record in which the actual data file name 1521 matches the file name of the actual data file 180 included in the file access request.
  • the file access control unit 110 acquires the value of the metadata file name 1522 of the retrieved record as the file name of the metadata file 181 to be accessed.
  • the file access control unit 110 refers to the stored file management table 154 and searches for a record that matches the file name of the metadata file 181 from which the file name 1541 is acquired.
  • the file access control unit 110 acquires the values of the file storage hierarchy 1542 and the file path 1543 of the retrieved record.
  • the file access control unit 110 acquires the access target metadata file 181 based on the acquired information (step S407). Specifically, the following processing is executed.
  • the file access control unit 110 determines whether the metadata file 181 is stored in the front-end file server 1 based on the value of the file storage hierarchy 1542. When it is determined that the metadata file 181 is stored in the front-end file server 1, the file access control unit 110 acquires the metadata file 181 based on the value information of the file path 1543.
  • the file access control unit 110 acquires the metadata file 181 from the back-end file server 2. For example, the file access control unit 110 transmits a read request including the value of the file path 1543 to the back-end file server 2.
  • the file access control unit 110 determines whether or not the actual data file 180 is included in the file to be accessed (step S409).
  • the file access control unit 110 refers to the access designation information included in the file access request and determines whether or not the actual data file 180 is designated as an access target file. For example, when the access designation information includes only the actual data file 180 or information designating either the actual data file 180 or the metadata file 181 as the access target file, the file access control unit 110 may It is determined that the actual data file 180 is included in this file.
  • step S409 the file access control unit 110 proceeds to step S415.
  • the file access control unit 110 When it is determined that the actual data file 180 is included in the file to be accessed (Yes in step S409), the file access control unit 110 indicates the storage destination of the actual data file 180 to be accessed from the storage file management table 154. Is acquired (step S411).
  • the file access control unit 110 searches for a record in which the file name 1541 matches the file name of the actual data file 180 included in the file access request.
  • the file access control unit 110 acquires the values of the file storage hierarchy 1542 and the file path 1543 of the retrieved record.
  • the file access control unit 110 acquires the actual data file 180 to be accessed based on the acquired information (step S413). Specifically, the following processing is executed.
  • the file access control unit 110 determines whether or not the actual data file 180 is stored in the front-end file server 1 based on the value of the file storage hierarchy 1542. When it is determined that the actual data file 180 is stored in the front-end file server 1, the file access control unit 110 acquires the actual data file 180 to be accessed based on the value of the file storage hierarchy 1542.
  • the file access control unit 110 acquires the actual data file 180 from the back-end file server 2. For example, the file access control unit 110 transmits a read request including the value of the file path 1543 to the back-end file server 2.
  • the file access control unit 110 transmits the acquired file to the requesting client 5 (step S415), and ends the process.
  • the file access control unit 110 responds to the requesting client 5 to that effect.
  • FIG. 14 is an explanatory diagram illustrating an example of a data configuration of the metadata file 181 according to the first embodiment of this invention.
  • FIG. 14 shows a data structure 156 that is the content of the metadata included in the metadata file 181.
  • the data structure 156 includes a metadata type 1561, a metadata name 1562, and a metadata value 1563. Note that the columns included in the data structure 156 are not limited to those described above.
  • the metadata type 1561 stores information indicating the type of metadata stored in the metadata file 181.
  • FIG. 14 shows three types of metadata as an example.
  • Metadata derived from file attribute information corresponds to the metadata acquired from the attribute information of the actual data file 180.
  • “Metadata derived from actual data” corresponds to metadata extracted by the metadata extraction unit 122 analyzing the actual data file 180.
  • “metadata derived from hierarchical storage” corresponds to metadata for managing the actual data file 180 corresponding to the metadata file 181 in the hierarchical storage.
  • metadata for managing the actual data file 180 corresponding to the metadata file 181 in the hierarchical storage for example, the identification information of the file server in which the actual data file 180 is stored, Information such as file path can be considered.
  • Methodadata derived from tiered storage corresponds to the storage location information of the actual data file added in step S131 or step S311.
  • the metadata name 1562 stores metadata identification information.
  • the identification information of the metadata for example, an identification name or an identification number for specifying the metadata can be considered.
  • the metadata value 1563 stores a metadata value. One or more values are stored for the metadata name 1562. In this embodiment, the metadata value 1563 corresponding to the storage destination information of the actual data file stores a URL for accessing the actual data file 180.
  • the data structure 156 of the metadata file 181 has been described as tabular data.
  • the present invention is not limited to this, for example, the CSV (Comma Separated Values) format or the XML format. It may be used.
  • FIG. 15 is an explanatory diagram showing an example of the file list screen 190 used in the first embodiment of the present invention.
  • the file list screen 190 is a GUI used to refer to file storage information in the tiered storage.
  • the system administrator or the like operates the client 5 or the like to access the front-end file server 1
  • the storage location of the file stored in the tiered storage and the storage information of the moved file are displayed on the file list screen 190. Is done.
  • a file list display field 191 In the file list screen 190, a file list display field 191, a filter condition input field 192, a sort condition input field 193, and a refresh button 194 are displayed.
  • the file list display field 191 displays information on files stored in the tiered storage.
  • the file list display column 191 includes an actual data file name 1911, a metadata file name 1912, an actual data file storage state 1913, a metadata file storage state 1914, an actual data file storage destination URL 1915, and a metadata file storage destination URL 1916. .
  • the file name of the actual data file 180 stored in the hierarchical storage is displayed.
  • the file name of the metadata file 181 stored in the hierarchical storage is displayed.
  • the storage state of the actual data file 180 is displayed. For example, when the actual data file 180 is stored in the hierarchical storage, the actual data file storage state 1913 is displayed as “stored”, and when the actual data file 180 is deleted from the hierarchical storage, the actual data file storage state 1913 is “Deleted” is displayed.
  • the storage state of the metadata file 181 is displayed.
  • the information displayed in the metadata file storage state 1914 is the same as that in the actual data file storage state 1913.
  • information for accessing the actual data file 180 is displayed. For example, information combining the tier name in the tier storage and the file path of the file server corresponding to the tier is displayed.
  • Metadata file storage destination URL 1916 information for accessing the metadata file 181 is displayed.
  • the information displayed in the metadata file storage location URL 1916 is the same as the actual data file storage location URL 1915.
  • the filter condition input field 192 is a display field for inputting a condition for filtering a file to be displayed.
  • the sort condition input field 193 is a display field for inputting a condition for designating the order of records to be displayed.
  • the refresh button 194 is an operation button for updating the display contents of the file list display field 191 and the like to the latest state.
  • the storage information of the file in the tiered storage is displayed as the GUI screen, the present invention is not limited to this.
  • a command or a program method for acquiring file storage information may be provided, and a CLI or API may be provided.
  • the file server configuring the hierarchical storage can reliably hold the metadata (metadata file 181) of the actual data file 180 in the hierarchical storage. For example, even if the free space of the data storage area 170 is small and it is difficult for the front-end file server 1 to store a new file, the metadata of the file can be reliably acquired.
  • the first embodiment it is possible to reliably move the metadata from the upper tier storage (file server) to the lower tier storage (file server).
  • a large amount of data is stored at one time, or a large amount of data is moved between file servers, etc.
  • the selection of the movement candidate file and the movement of the file can be controlled.
  • the present invention can also be applied to a system that handles data in units of blocks.
  • the physical storage location of the block may be specified using the block address, block length, and the like.
  • the front-end file server 1 when a file is moved between file servers that constitute a hierarchical storage, the front-end file server 1 that is the movement source of the file designates the file path of the file server that is the movement destination. Further, after the file is moved, the front-end file server 1 manages the file and the file path in the destination file server in association with each other. Thus, when a file access request for the file is received after the file is moved, the front-end file server 1 can read the file from the destination file server and respond to the read file to the request source.
  • a form of a computer system in which an identification number and an identification name unique to the entire computer system are assigned to a file to be managed and the file is accessed using the assigned information can be considered.
  • an arbitrary file server or dedicated server assigns and manages an identification number and an identification name.
  • Example 2 describes an example in which the present invention is applied to a computer system in which file identification numbers and identification names are managed in a unified manner.
  • the second embodiment will be described with reference to FIGS. 16 to 18 focusing on differences from the first embodiment.
  • FIG. 16 is a block diagram illustrating a configuration example of the computer system according to the second embodiment of this invention.
  • a name space management server 7 is newly included in the computer system.
  • the namespace management server 7 performs data communication with other devices via the network 6.
  • the configurations of the front-end file server 1, the back-end file server 2, the data source 4, and the client 5 are the same as those in the first embodiment, and thus detailed configurations are omitted.
  • the namespace management server 7 centrally manages the identification numbers and identification names of files stored in the tiered storage.
  • the file path is determined based on the file identification number and the identification name assigned by the namespace management server 7.
  • the name space management server 7 uses a general-purpose computer and includes a CPU 70, a memory 71, an I / F 72, and an auxiliary storage device 73.
  • CPU 70 executes a program stored in memory 71.
  • the functions of the namespace management server 7 can be realized by the CPU 70 executing the program. In the following description, when a process is described with a program as the subject, it indicates that the CPU 70 is executing the program.
  • the memory 71 stores a program executed by the CPU 70 and information necessary for executing the program.
  • the program and information stored in the memory 71 will be described later.
  • the I / F 72 controls data communication with an external device.
  • the auxiliary storage device 73 is a device used to provide a large capacity storage area.
  • As the auxiliary storage device 73 for example, an HDD and an SSD can be considered.
  • a RAID may be configured using a plurality of auxiliary storage devices 73.
  • the memory 71 stores a program for realizing the namespace management unit 701 and a namespace management table 750.
  • the name space management table 750 will be described later with reference to FIG.
  • the name space management unit 701 assigns a unique identification number and identification name to a file stored in the tier storage. Information about the assigned identification number and identification name is stored in a namespace management table 750 described later. In the following description, a unique identification number and identification name assigned to a file are also referred to as a global name.
  • the namespace management unit 701 When assigning a global name to a file, the namespace management unit 701 also assigns identification information of the file server in which the file is stored and a file path indicating a storage location in the file server.
  • the front-end file server 1 may specify identification information of a file server that is a file storage destination candidate and a file path candidate when requesting the assignment of a global name.
  • the namespace management unit 701 determines whether or not duplication or inconsistency with an existing file occurs based on the designated content, and if no duplication or inconsistency occurs, the designated content is Used to give file server identification information and file path.
  • the namespace management unit 701 receives a name resolution request including the assigned global name, and responds with a file storage location or access path associated with the global name. Note that the name space management unit 701 also receives an update request for information regarding the storage location and access path of the file.
  • the namespace management server 7 has been described as a configuration independent of the file server, but the present invention is not limited to this.
  • One file server may have a function of the namespace management server 7.
  • achieves the function which the name space management server 7 has cooperated by a some file server may be sufficient.
  • the tiered storage configuration management table 150 Since the tiered storage configuration management table 150, the migration condition management table 151, the file correspondence management table 152, the operation management table 153, the storage file management table 154, and the file migration management table 155 are the same as those in the first embodiment, a description will be given. Omitted. Further, the metadata extraction process is the same as that of the first embodiment, and thus the description thereof is omitted.
  • FIG. 17 is an explanatory diagram showing a configuration example of the name space management table 750 according to the second embodiment of the present invention.
  • the namespace management table 750 manages the correspondence between the global name managed by the namespace management unit 701 and information related to the file to which the global name is assigned. Specifically, the namespace management table 750 includes a global name 7501, a local host name 7502, and a local path name 7503. Note that the columns included in the namespace management table 750 are not limited to those described above.
  • the global name 7501 stores the global name assigned by the namespace management unit 701.
  • the global name is assigned so as not to be duplicated with other global names.
  • the local host name 7502 stores identification information of a file server that stores a file to which a global name corresponding to the global name 7501 is assigned.
  • the local host name 7502 stores the host name or IP address of the file server.
  • the local path name 7503 stores a file path indicating the storage location of the file assigned the global name in the file server.
  • the global name 7501 and the local path name 7503 may include a wild card representing an arbitrary character string. As a result, information on all files existing in a specific hierarchy can be managed by one record.
  • FIG. 18A and FIG. 18B are flowcharts for explaining file movement processing according to the second embodiment of the present invention.
  • step S307 If the determination result of step S307 is Yes, the process of step S351 is newly executed.
  • the file movement control unit 130 requests the namespace management server 7 to add the storage location information of the metadata file 181 (step S351).
  • the file moving unit 133 requests the namespace management server 7 to add storage destination information including identification information of the back-end file server 2 that is a candidate for the storage destination of the metadata file 181.
  • the file moving unit 133 acquires a global name, a local host name, and a local path name as a response from the namespace management server 7.
  • the local host name and local path name correspond to the storage location information. Thereafter, the file moving unit 133 proceeds to step S309.
  • step S353 is newly executed.
  • the file movement control unit 130 requests the namespace management server 7 to allocate storage location information of the actual data file 180 (step S353).
  • the file moving unit 133 requests the namespace management server 7 to allocate storage location information including identification information of the back-end file server 2 that is a candidate for the storage location of the actual data file 180. .
  • the file moving unit 133 acquires a global name, a local host name, and a local path name as a response from the namespace management server 7. Thereafter, the file moving unit 133 proceeds to step S321.
  • step S351 and step S353 The information acquired in step S351 and step S353 is used when the file is moved, and is stored in the storage file management table 154.
  • FIG. 19 is a flowchart illustrating file access processing according to the second embodiment of the present invention.
  • step S451 After the process of step S401 is executed, the process of step S451 is newly executed.
  • the file access control unit 110 requests the name space management server 7 for name resolution (step S451). Specifically, the following processing is executed.
  • the file access control unit 110 analyzes the access path included in the received file access request and determines whether name resolution is necessary.
  • the file access control unit 110 determines whether or not the access path included in the request is a global name given by the namespace management server 7.
  • the file access control unit 110 determines that name resolution is necessary.
  • the file access control unit 110 does not make a request for name resolution and proceeds to step S403.
  • the file access control unit 110 determines that name resolution is necessary because it is necessary to specify the storage destination of the file to be accessed. Further, the file access control unit 110 requests the name space management server 7 to perform name resolution including the access path included in the received file access request, that is, the global name.
  • the namespace management server 7 acquires the values of the local host name 7502 and local path name 7503 associated with the global name included in the request from the namespace management table 750, and uses the acquired values as the local host name and the local host name. Reply as a path name.
  • the file access control unit 110 receives the local host name and local path name, and then proceeds to step S402.
  • the information acquired in step S451 is used when acquiring the access target file.
  • the present invention can be applied to a computer system in which files stored in a tiered storage are managed in an integrated manner.
  • the migration candidate file is selected based on the operation information of the front-end file server 1. That is, the operation information of the file server from which the file is moved is used.
  • a more appropriate transfer candidate file can be selected by considering the operation information of the back-end file server 2 to which the file is moved and the operation information of other components included in the computer system. As a result, the computer resources available in the computer system can be used more effectively.
  • the third embodiment is different from the first embodiment in that the migration candidate file is selected based on the operation information of each file server constituting the tiered storage.
  • the third embodiment will be described with a focus on differences from the first embodiment, with reference to FIGS. 20 to 22.
  • FIG. 20 is an explanatory diagram showing a configuration example of the operation management table 153 according to the third embodiment of the present invention.
  • the operation management table 153 includes a machine name 1537.
  • the machine name 1537 stores identification information of a machine from which operation information is acquired.
  • the same machine name 1537 as the tier name 1501 and storage identification information 1502 of the tier storage configuration management table 150 may be used.
  • the machine for which the operation information is acquired may be a component that is included in the computer system such as a file server or a storage system and that can acquire the operation information.
  • the operation information for each machine can be grasped by associating and managing the operation information of a plurality of machines.
  • 21A and 21B are explanatory diagrams illustrating a configuration example of the movement condition management table 151 according to the third embodiment of the present invention.
  • the description will focus on differences from the movement condition management table 151 in the first embodiment shown in FIG.
  • the contents stored in the condition 1513 and the action 1514 are different.
  • condition 1513 information for specifying a machine that is a target of an index used for determination is added.
  • the condition “Local HDD usage rate is 80% or more” is stored in the condition 1513 of the record whose condition management number 1511 is “1” and the determination order 1512 is “1”. That is, the front-end file server 1 is designated as an index target used for determination.
  • condition 1513 of the record with the condition management number 1511 being “1” and the determination order 1512 being “2” is set with the conditions “Local HDD usage rate is 80% or more” and “Tier2 HDD usage rate is 80% or more”. Is done. Further, the condition 1513 of the record with the condition management number 1511 being “1” and the determination order 1512 being “3” is “Local HDD usage rate is 80% or more” and “Tier2 HDD usage rate is less than 80%”. A condition is set.
  • the content of the action 1514 of the record in which the condition management number 1511 is “1” and the determination order 1512 is “3” is the same as that in the first embodiment.
  • the contents different from those in the first embodiment are also stored in the action 1514 of the record in which the condition management number 1511 is “1” and the determination order 1512 is “2”. Specifically, an action “move a plurality of files stored in the back-end file server B to the back-end file server C until the HDD usage rate of the server becomes less than 60% as pre-file move processing” 1514 is added.
  • the movement candidate file selection process is changed to realize the action 1514 described above. Specific changes will be described later.
  • the HDD capacity of the destination file server can be increased by moving the file stored in the file server to another file server. You can move the file at a later stage.
  • back-end file server C has the same configuration as the back-end file server 2 and is a lower-level file than the back-end file server B (2-2).
  • FIGS. 22A and 22B are flowcharts for explaining the movement candidate file selection processing according to the third embodiment of the present invention.
  • step S251 After the process of step S205, the process of step S251 is newly executed.
  • the file movement control unit 130 determines whether or not to execute the file movement pre-processing (step S251).
  • the determination unit 132 determines whether to execute the file movement pre-processing based on the processing content of the movement processing determined in step S203.
  • step S205 If it is determined not to execute the file move pre-processing, the file move control unit 130 proceeds to step S205.
  • the file movement control unit 130 executes the file movement pre-processing (step S253).
  • the file movement control unit 130 refers to the processing contents of the movement processing determined in step S203, confirms the processing contents of the file movement pre-processing, and executes the file movement pre-processing according to the processing contents. .
  • front-end file server 1 may perform the file movement pre-processing as the main body, or may request the back-end file server 2 to be processed to execute the process.
  • the front-end file server 1 reads the file from the back-end file server B (2-2), and stores the read file in the back-end file server C. Further, the front-end file server 1 deletes the file read from the back-end file server B (2-2). Further, the front-end file server 1 stores the file movement result in each management table.
  • step S253 After the process of step S253 is complete
  • a migration candidate file can be selected in consideration of operation information of components other than the front-end file server 1. Further, by utilizing various operation information such as a file server at the destination, metadata transfer control can be further enhanced.
  • the various software illustrated in the present embodiment can be stored in various recording media (for example, non-temporary storage media) such as electromagnetic, electronic, and optical, and through a communication network such as the Internet. It can be downloaded to a computer.
  • recording media for example, non-temporary storage media
  • a communication network such as the Internet. It can be downloaded to a computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un système informatique qui comporte une pluralité d'ordinateurs, dont chacun comprend un tampon pour stocker temporairement un fichier reçu à partir d'un dispositif externe et une zone de stockage de données pour stocker le fichier, et comprend en outre une unité de gestion de fichier, une unité de contrôle d'accès à un fichier et une unité de génération de métadonnées, l'unité de génération de métadonnées obtenant des informations de fonctionnement indiquant l'état de fonctionnement d'un ordinateur lorsque l'ordinateur a reçu un fichier, déterminant, sur la base des informations de fonctionnement, s'il faut ou non générer un fichier de métadonnées, et s'il est déterminé qu'un fichier de métadonnées devrait être généré, alors l'unité de génération de métadonnées obtenant des métadonnées à partir du fichier, stocké dans le tampon, générant un fichier de métadonnées sur la base des métadonnées obtenues, stockant le fichier de métadonnées dans la zone de stockage de données, et ajoutant des informations pour accéder au fichier au fichier de métadonnées stocké dans la zone de stockage de données.
PCT/JP2013/067099 2013-06-21 2013-06-21 Système informatique, procédé de gestion de métadonnées et programme WO2014203397A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2013/067099 WO2014203397A1 (fr) 2013-06-21 2013-06-21 Système informatique, procédé de gestion de métadonnées et programme

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2013/067099 WO2014203397A1 (fr) 2013-06-21 2013-06-21 Système informatique, procédé de gestion de métadonnées et programme

Publications (1)

Publication Number Publication Date
WO2014203397A1 true WO2014203397A1 (fr) 2014-12-24

Family

ID=52104159

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/067099 WO2014203397A1 (fr) 2013-06-21 2013-06-21 Système informatique, procédé de gestion de métadonnées et programme

Country Status (1)

Country Link
WO (1) WO2014203397A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017123140A (ja) * 2016-01-04 2017-07-13 ベイジン バイドゥ ネットコム サイエンス アンド テクノロジー カンパニー リミテッド オブジェクト記憶システムにおけるオブジェクトデータの更新方法及び更新装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002318809A (ja) * 2001-04-19 2002-10-31 Nec Corp インデックス構築型検索エンジン及びインデックス構築方法並びにプログラム
JP2007079774A (ja) * 2005-09-13 2007-03-29 Hitachi Ltd ファイルシステムの構築方法
JP2007140919A (ja) * 2005-11-18 2007-06-07 Hitachi Ltd 記憶システム及びデータ移動方法
JP2011244043A (ja) * 2010-05-14 2011-12-01 Mitsubishi Electric Corp 映像記録再生装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002318809A (ja) * 2001-04-19 2002-10-31 Nec Corp インデックス構築型検索エンジン及びインデックス構築方法並びにプログラム
JP2007079774A (ja) * 2005-09-13 2007-03-29 Hitachi Ltd ファイルシステムの構築方法
JP2007140919A (ja) * 2005-11-18 2007-06-07 Hitachi Ltd 記憶システム及びデータ移動方法
JP2011244043A (ja) * 2010-05-14 2011-12-01 Mitsubishi Electric Corp 映像記録再生装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017123140A (ja) * 2016-01-04 2017-07-13 ベイジン バイドゥ ネットコム サイエンス アンド テクノロジー カンパニー リミテッド オブジェクト記憶システムにおけるオブジェクトデータの更新方法及び更新装置
US10380103B2 (en) 2016-01-04 2019-08-13 Beijing Baidu Netcom Science And Technology Co., Ltd. Object data updating method and apparatus in an object storage system

Similar Documents

Publication Publication Date Title
US11645183B1 (en) User interface for correlation of virtual machine information and storage information
JP6697392B2 (ja) 半構造データスキーマのトランスペアレントディスカバリ
US8874627B2 (en) Enumerating metadata in file system directories
JP5895099B2 (ja) 移行先ファイルサーバ及びファイルシステム移行方法
JP6093773B2 (ja) キャッシュベースのキー・バリューストアのマッピングおよび複製
US8972405B1 (en) Storage resource management information modeling in a cloud processing environment
JP4686305B2 (ja) ストレージ管理システムおよびその方法
CN110447017A (zh) 数据存储装备监视器中基于规则的修改
US20150134796A1 (en) Dynamic partitioning techniques for data streams
CN110291518A (zh) 合并树无用单元指标
CN110383261A (zh) 用于多流存储装置的流选择
WO2015000502A1 (fr) Procédé et appareil pour la virtualisation d'un système de fichiers, système de stockage de données pour la virtualisation d'un système de fichiers, et serveur de fichiers à utiliser dans un système de stockage de données
JP2014535107A5 (fr)
CN108140040A (zh) 存储器中数据库的选择性数据压缩
JP2009064120A (ja) 検索システム
WO2016016944A1 (fr) Système de gestion de base de données et procédé de gestion de base de données
US20160070745A1 (en) Index suspension prior to database update
US10789234B2 (en) Method and apparatus for storing data
JP6002832B2 (ja) 計算機システム、データ管理方法及びプログラムを格納する記録媒体
WO2013186891A1 (fr) Système de gestion d'état d'avancement, et procédé de gestion d'état d'avancement
WO2014203397A1 (fr) Système informatique, procédé de gestion de métadonnées et programme
US20180165380A1 (en) Data processing system and data processing method
JP6193491B2 (ja) 計算機システム
JP2013246773A (ja) 計算機システム及びデータ管理方法
JP6585283B2 (ja) データ処理システム及びデータ処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13887537

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13887537

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP