US20080065587A1 - Data management system and data management method - Google Patents
Data management system and data management method Download PDFInfo
- Publication number
- US20080065587A1 US20080065587A1 US11/605,596 US60559606A US2008065587A1 US 20080065587 A1 US20080065587 A1 US 20080065587A1 US 60559606 A US60559606 A US 60559606A US 2008065587 A1 US2008065587 A1 US 2008065587A1
- Authority
- US
- United States
- Prior art keywords
- file system
- file
- storage
- storage apparatus
- transfer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/119—Details of migration of file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/1658—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
- G06F11/1662—Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit the resynchronized component or unit being a persistent storage device
Definitions
- the present invention relates to a data management system and data management method appropriate for application in data management systems for searching files based on requests from, for example, client apparatus.
- a client apparatus In the related art, computer systems are implemented where a client apparatus, host apparatus and storage apparatus are connected to a network so that files managed by the client apparatus are transmitted and received via the host apparatus and the network, and the files are stored on a hard disc within the storage apparatus.
- file systems sets of files that are not used often are transferred from so-called high-cost hard disc drives within the storage apparatus to so-called low cost disc drives.
- file search apparatus are connected to a network.
- the client apparatus searches for the requested file, and the file is read out from the storage apparatus to the client apparatus.
- a search engine for the file search apparatus is usually developed and put into the form of a product by a dedicated vendor, while software for transferring files of the storage apparatus is developed and put into the form of a product by a storage apparatus vendor.
- the means that the interface connecting the file search apparatus and the storage apparatus is not standardized.
- the administrator of the storage apparatus refers to the transfer history, and takes into consideration an operation where the search index table is updated every transferred file system, but there are cases where several hundred thousand files are stored in one file system of the storage apparatus.
- the search index table is made in file units, it is not practical for the administrator of the storage apparatus to update the search index table.
- the present invention therefore proposes a data management system and data management method with highly reliable data management.
- a data management system with a host apparatus receiving requests from client apparatus, first storage apparatus having a first volume for storing a file system composed of a set of files, and a second storage apparatus having a second volume for storing the file system sent from the first storage apparatus comprises a file system transfer designation section for transferring the file system matching with file system migration rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system migration rules, a file system storage information manager for updating storage information of the file system in accordance with transfer of the file system by the file system transfer designation section, and transmitting the updated file system storage information, and a search information manager for updating search information for searching the files based on a file search request from the client apparatus using the file system storage information sent by the file system storage information manager.
- a data management system with a host apparatus receiving requests from client apparatus, first storage apparatus having a first volume for storing a file system composed of a set of files, and a second storage apparatus having a second volume for storing the file system sent from the first storage apparatus comprises a first step of transferring the file system matching with file system migration rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system migration rules, a second step of updating storage information for the file system in accordance with transfer of the file system of the first step, and transmitting the stored information of the updated file system, and a third step of updating the search information for searching the files based on file search requests from the client apparatus using file system storage information sent in the second step.
- FIG. 1 is a block view showing an outline configuration for a data management system of an embodiment
- FIG. 2 is a block view showing an outline configuration for host apparatus
- FIG. 3 is a block view showing an outline configuration for file system transfer apparatus
- FIG. 4 is a block view showing an outline configuration for the first storage apparatus and the second storage apparatus
- FIG. 5 is conceptual view illustrating an index table
- FIG. 6 is a conceptual view illustrating a reverse index table
- FIG. 7 is a conceptual view illustrating a file system transfer management table
- FIG. 8 is a conceptual view illustrating an outline of an operation for a data management system
- FIG. 9 is a flowchart showing an index generation procedure
- FIG. 10 is a flowchart showing a file system transfer start determination processing procedure
- FIG. 11 is a flowchart showing a file system transfer processing procedure
- FIG. 12 is a flowchart showing an index update management processing procedure
- FIG. 13 is a flowchart showing a file system search handling procedure
- FIG. 14 is a flowchart showing a file system read processing procedure
- FIG. 15 is a flowchart showing a file system mount preparation processing procedure
- FIG. 16 is a block view showing an outline configuration for a data management system of a further embodiment.
- FIG. 1 shows a configuration for a data management system 1 of an embodiment of the present invention.
- the data management system 1 is comprised of client apparatus 2 , host apparatus 3 and file system transfer apparatus 4 connected via a LAN (Local Area Network) 5 , and the host apparatus 3 , file system transfer apparatus 4 , first storage apparatus 6 and second storage apparatus 7 connected via an SAN (Storage Area Network) 8 .
- LAN Local Area Network
- SAN Storage Area Network
- the client apparatus 2 is computer apparatus equipped with information processing resources such as a CPU (Central Processing Unit) and memory etc., such as, for example, a personal computer etc.
- the client apparatus 2 are connected to information input apparatus (not shown) such as a keyboard, switches and pointing apparatus, and microphone etc., and also have information output apparatus (not shown) such as monitor displays and speakers etc.
- information input apparatus such as a keyboard, switches and pointing apparatus, and microphone etc.
- information output apparatus not shown
- the client apparatus 2 sends a file read-out request to read out a file to the host apparatus 3 based on a file search request for searching files resulting from designation using, for example, keywords etc. and the search results.
- FIG. 2 shows a configuration for the host apparatus 3 .
- the host apparatus 3 is configured from an Ethernet (registered trademark) controller 11 connected to LAN 5 for controlling Ethernet (registered trademark) communication, a fiber channel controller 12 for controlling fiber channel communication, an input/output controller 13 for controlling information input apparatus and information output apparatus, a hard disc drive controller 14 for controlling a hard disc drive section 19 , a CPU 15 for performing overall control of the host apparatus 3 , and a memory 16 for storing various programs etc, with each element of this configuration described above being connected via a bridge chip 17 for controlling transfer of files etc.
- Ethernet registered trademark
- LAN 5 for controlling Ethernet (registered trademark) communication
- a fiber channel controller 12 for controlling fiber channel communication
- an input/output controller 13 for controlling information input apparatus and information output apparatus
- a hard disc drive controller 14 for controlling a hard disc drive section 19
- a CPU 15 for performing overall control of the host apparatus 3
- a memory 16 for storing various programs etc, with each element of this configuration described above being connected via
- the hard disc drive section 19 equipped with a number of hard disc drives 18 is connected to the hard disc drive controller 14 .
- some or all of predetermined hard disc drives 18 are used as an index database 20 for storing various tables etc. used in file searching etc.
- An index table 21 for storing location information etc. every file, a reverse index table 22 for storing file information etc. containing the keywords every keyword, and a file system transfer management table 23 for storing information etc. for before and after transfer for file systems for between logical volumes of storage apparatus described later are stored in the index database 20 .
- a detailed description of the index table 21 , reverse index table 22 and file system transfer management table 23 is given in the following.
- a file system is a set of files. In this event, a large number of files are contained in one file system.
- a file is digital data and a file system that is this set is also digital data.
- a logical volume is equipment for storing the file system.
- a one to one configuration is adopted where one file system is stored in one logical volume, but the present invention is by no means limited in this respect, and may also be applied to a configuration where a single file system spans a number of logical volumes, or where a single logical volume stores a number of file systems.
- an application program 31 that is a program for executing various processing is also stored in the memory 16 of the host apparatus 3 .
- the application program 31 is comprised of an agent 21 that is a program executing file search processing and file reading processing, an HTTP/XML stack 33 for executing control based on HTTP/XML (HyperText Transfer Protocol/eXtensible Markup Language), a TCP/IP stack 34 for executing control based on TCP/IP (Transmission Control Protocol/Internet Protocol), a file system mounting program 35 that is a program for managing file systems mounted/un-mounted at the host apparatus 3 , a file system management program 36 that is a program for managing file systems, an Ethernet (registered trademark) driver 37 that is a driver for controlling the Ethernet (registered trademark) controller 11 , and a fiber channel driver 38 that is a driver for controlling the fiber channel controller 12 .
- “mount” refers to recognizing the file system at the host 3 and puts the host apparatus 3 into a state enabling the execution of read/write processing.
- the agent 32 is comprised of a file search processing program 41 , a file reading processing program 42 , an indexing program 43 , an index update management program 44 , a synchronization management program 45 , and a database management program 46 .
- a file search processing program 41 a file searching processing program 41 , a file reading processing program 42 , an indexing program 43 , an index update management program 44 , a synchronization management program 45 , and a database management program 46 .
- the hardware configuration of this host apparatus 3 may be implemented through various forms of general purpose computers (personal computers), workstations, and mainframes, etc.
- the index table 21 , reverse index table 22 and file system transfer management table 23 may be stored in the memory 16 .
- FIG. 3 shows a configuration for the file system transfer apparatus 4 .
- the file system transfer apparatus 4 adopts the same configuration as for the host apparatus 3 , with the exception that the various programs and information etc. stored in the memory 56 and the hard disc drive section 59 are different. Namely, the file system transfer apparatus 4 is implemented using a general purpose computer (personal computer), etc.
- the file system transfer apparatus 4 is comprised of an Ethernet (registered trademark) controller 51 , a fiber channel controller 52 , an input/output controller 53 , a hard disc drive controller 54 , a CPU 55 , and a memory 56 connected via a bridge 57 .
- Ethernet registered trademark
- a hard disc drive section 59 equipped with a plurality of hard disc drives 58 is connected to the hard disc controller 54 .
- migration rules 61 and a file system transfer management table 62 are stored in a predetermined hard disc drive 58 .
- Fixed rules for transferring a file system such as, for example, “in the event that any files of the file system have not been subjected to reading or writing processing for a predetermined period, transfer this file system from the first storage apparatus 6 to the second storage apparatus 7 ”, and “in the event that the final update time where the frequency of read/write processing during the preceding three months is low at 20% of the file systems mounted at the host apparatus 3 is checked and one month or more has passed since the final update time, this file system is transferred from the first storage apparatus 6 to the second storage apparatus 7 ”, are described in the migration rules 61 .
- the configuration of the file system transfer management table 62 is the same as the file system transfer management table 23 described above.
- a file system transfer manager 71 that is a program for executing file system transfer processing etc., a HTTP/XML stack 72 , a TCP/IP stack 73 , a file system management program 74 , an Ethernet (registered trademark) driver 75 and a fiber channel driver 76 are stored in the memory 16 of the file system transfer apparatus 4 .
- the file system management program 74 differs from the file system mounting program 35 in managing the file systems that are mounted/un-mounted at all of the host apparatus 3 .
- the file system transfer manager 71 is comprised of a file system transfer initiating program 81 , a synchronization management program 82 , a file system transfer notification program 83 , a file system transfer program 84 , and a file system mounting preparation program 85 .
- a file system transfer initiating program 81 a file system transfer initiating program 81
- a synchronization management program 82 a file system transfer notification program 83
- a file system transfer program 84 a file system mounting preparation program 85 .
- the hardware configuration of this file system transfer apparatus 4 may be implemented by personal computers etc. in various forms.
- the migration rules 61 and the file system transfer management table 62 may also be stored in the memory 56 .
- FIG. 4 shows a configuration for the first storage apparatus 6 and the second storage apparatus 7 .
- the first storage apparatus 6 is comprised of storage controller 91 A and a storage section 92 A.
- the storage controller 91 A is configured in such a manner that a front end controller 101 A for controlling fiber channel communication connected to SAN 8 , CPU 102 A for overall control of the first storage apparatus 6 , a back end controller 106 A connected to the cache memory 104 A for temporarily storing transmitted and received files and storage section 92 A, for controlling communication with the storage section 92 A, are connected with each element of the configuration described above, and are connected via data transfer controller 103 A for controlling the transfer of data.
- a local memory 105 A for storing programs and tables necessary for controlling the first storage apparatus 6 and information etc. is connected to the CPU 102 A.
- the storage section 92 A is equipped with a plurality of hard disc drives 111 A.
- the CPU 102 A is capable of controlling the plurality of hard disc drives 111 A using RAID levels (for example, RAID0, 1, 5 etc.) defined in the so-called RAID system.
- the RAID system manages a plurality of hard disc drives 111 A as a single RAID group.
- a plurality of logical volumes 112 A are then defined in access units by the host apparatus 3 on the RAID group.
- a LUN Logical Unit Number
- the hard disc drive 111 A is, for example, a storage device such as an FC (Fibre Channel) disc drive, a SATA (Serial Advanced Technology Attachment) disc drive, a PATA (Parallel Advanced Technology Attachment) disc drive, a FATA (Fibre Attached Technology Adapted) disc drive, an SAS (Serial Attached SCSI) disc drive, or a SCSI (Small Computer System Interface) disc drive, etc.
- FC Fibre Channel
- SATA Serial Advanced Technology Attachment
- PATA Parallel Advanced Technology Attachment
- FATA Fibre Attached Technology Adapted
- SAS Serial Attached SCSI
- SCSI Serial Computer System Interface
- the second storage apparatus 7 has the same configuration as the first storage apparatus 6 described above, with each element of the configuration of the second storage apparatus 7 being given the same numerals for the same portions as the configuration elements of the first storage apparatus 6 , with suffix “A” being substituted with suffix “B”.
- the disc drive 111 A of the first storage system 6 is configured from, for example, a so-called high-cost hard disc drive such as a SCSI (Small Computer System Interface) disc drive etc.
- the drive 111 B of the second storage system 7 is configured from a so-called low-cost hard disc drive such as, for example, a SATA (Serial AT Attachment) disc drive or optical disc drive, etc.
- FIG. 5 is a view showing a configuration for the index table 21 .
- the index table 21 stores an index record number 121 for identifying the index record, a host apparatus identifier 122 for identifying the host apparatus 3 that made the file, a file system identifier 123 for identifying the file system for the logical volume storing the file, a path name 124 for a path indicating the location of the file system, a file name 125 for the file, attributes 126 comprised of a file update date and access priority control information etc., and a keyword list 127 composed of keywords for the file, every index record.
- the host apparatus identifier 122 is to take into consideration the case where a plurality of host apparatus exist, where a host apparatus 3 mounted to after a logical volume goes temporarily offline (described later) and goes online once again (described later) is different to the host apparatus 3 that first made the file.
- FIG. 6 is a view showing a configuration for the reverse index table 22 .
- the reverse index table 22 is a table adopting, for example, a B-Tree (Balanced tree) algorithm, that is capable of searching using an index tree (index tree) of this structure at high-speed.
- This reverse index table 22 is capable of searching every keyword 131 .
- the reverse index table 22 stores an index record number 132 and attributes 133 for a file having this keyword 131 every reverse record for the keyword 131 .
- the index record number 132 of the reverse index table 22 corresponds to the index record number 121 of the index table 21 .
- the reverse index table 22 is then capable of searching the index record number 121 of the index table 21 from the keyword 131 .
- FIG. 7 shows a configuration for the file system transfer management table 23 .
- the file system transfer management table 23 stores a file system identifier 141 , a file system state 142 indicating the storage state of the file system, an original host apparatus identifier 143 for identifying the host apparatus 3 that made the file first, an original storage apparatus identifier 144 for identifying the storage apparatus that stored the file system before transfer, an original logical volume identifier 145 for identifying the logical volume that stored the file system before transfer, a current host apparatus identifier 146 for identifying a host apparatus 3 mounting a file system currently containing the file, a current storage apparatus identifier 147 for identifying the storage apparatus currently storing the file system, and an original logical volume identifier 148 for identifying the logical volume currently storing the file system, every mapping record.
- file system transfer information 149 It is necessary to update the file system state 142 , the original host apparatus identifier 143 , the original storage apparatus identifier 144 , the original logical volume identifier 145 , the current host apparatus identifier 146 , the current storage apparatus identifier 147 , and the original logical volume identifier 148 as a result of transfer of the file system. Further, there are cases where the file system state 142 , the original host apparatus identifier 143 , the original storage apparatus identifier 144 , the original logical volume identifier 145 , the current host apparatus identifier 146 , the current storage apparatus identifier 147 , and the original logical volume identifier 148 are collectively referred to as file system transfer information 149 .
- “primary storage” is described for the case where the file system is stored in the first storage apparatus 6
- “secondary storage” is described for the case where the file system is stored in the second storage apparatus 6
- “write enable” is described in the case where the file system can be written to
- “write inhibit” is described for the case where it is not possible to write to the file system
- “online” is described for the case where the file system is mounted at the host apparatus 3
- “offline” is described for the case where the file system is not mounted at the host apparatus 3 .
- a sequential description is given of a state transition for the storage state of the file system. Further, “online” indicates the state where the file system is mounted at the host apparatus 3 , and offline indicates the state where the file system is not mounted at the host apparatus 3 .
- the file system transfer management table 62 stores a file system identifier 151 , a file system state 152 , an original host identifier 153 , an original storage apparatus identifier 154 , an original logical volume identifier 155 , a current host apparatus identifier 156 , a current storage apparatus identifier 157 , and an original logical volume identifier 158 , every mapping record.
- file system transfer information 159 there are cases where the file system state 152 , the original host identifier 153 , the original storage apparatus identifier 154 , the original logical volume identifier 155 , the current host apparatus identifier 156 , the current storage apparatus identifier 157 , and the original logical volume identifier 158 are collectively referred to as file system transfer information 159 .
- FIG. 8 shows an outline of the operation of the data management system 1 .
- the file system is transferred from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 based on the file system transfer management table 61 (A 1 ), the file system transfer update information 159 of the file system transfer management table 62 of the file system transfer management apparatus 4 updated in accordance with the transfer of the file system is sent to the host apparatus 3 (A 2 ), and the file system transfer information 159 of the file system transfer management table 23 of the index database 20 of the host apparatus 3 is updated (A 3 ).
- the host apparatus 3 stores the file in the logical volume 112 A of the first storage apparatus 6 by sending the file sent from the client apparatus 2 to the first storage apparatus 6 .
- the host apparatus 3 reads out and analyzes a file sent from the client apparatus 2 , or made or updated at the host apparatus 3 from the first storage apparatus 6 by periodically executing the agent 32 , and makes location information 128 , attributes 126 , or a keyword list 127 .
- the host apparatus 3 updates the index table 21 and the reverse index table 22 of the index database 20 using the produced location information 128 , attributes 126 , and keyword list 127 .
- the file system transfer apparatus 4 transfers the file system 161 from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 by executing the file system transfer manager 71 at a predetermined timing, and sending a file system transfer request for the file system 161 to the first storage apparatus 6 and the second storage apparatus 7 at a predetermined period where reading and writing processing is not taking place in accordance with the migration rules 61 .
- the file system transfer apparatus 4 updates the file system transfer information 159 of the file system transfer management table 62 .
- the file system transfer apparatus 4 then sends the file system transfer information 159 of the updated file system transfer management table 62 to the host apparatus 3 together with the file system transfer notification.
- the host apparatus 3 then updates the file system transfer information 149 of the file system transfer management table 23 of the index database 20 using the file system transfer update information 159 of the transmitted file system transfer management table 62 .
- FIG. 9 is a flowchart showing a specific processing procedure of the CPU 15 of the host apparatus 3 relating to the index generation occurring at this host apparatus 3 .
- the CPU 15 of the host apparatus 3 waits in a standby mode for an activation start time of the index generation to elapse in accordance with the index generation procedure RT 1 shown in FIG. 9 by executing the index generation program 43 making each type of index table.
- the activation start time is set to a fixed time interval that is comparatively short that is, for example a predetermined certain time of one day (once every 24 hours), etc.
- the CPU 15 of the host apparatus 3 checks the file systems mounted at the host apparatus 3 by executing the file system management program 36 , and selects one file system of the file systems (S 2 ). The CPU 15 of the host apparatus 3 then selects one file of the selected file system (S 3 ). The CPU 15 of the host apparatus 3 then checks whether the selected file is a file that is newly added or whether or not the file is an updated file (S 4 ).
- the CPU 15 of the host apparatus 3 proceeds to step S 7 .
- the CPU 15 of the host apparatus 3 reads out the file and analyzes the content of the file (S 5 ).
- the CPU 15 of the host apparatus 3 then produces the location information 128 , attributes 126 and keyword list 127 based on the file analysis results by executing the database management program 46 , and updates the index table 21 and/or the reverse index table 22 of the index database 20 using the produced location information 128 , attributes 126 , and keyword list 127 (S 6 ).
- the CPU 15 of the host apparatus 3 then checks whether or not all of the files of the selected file system have been selected (S 7 ). In the event that all of the files for the selected file system have not been selected (S 7 : NO), the CPU 15 of the host apparatus 3 selects the next file of the selected file system (S 8 ). After this, step S 4 for checking whether or not the selected file is a newly added file or an updated file is returned to, and the same processing is then repeated (S 4 to S 8 ).
- the CPU 15 of the host apparatus 3 checks whether or not all of the file systems mounted at the host apparatus 3 have been selected (S 9 ). In the event that all of the file systems mounted at the host apparatus have not been selected (S 9 : NO), the CPU 15 of the host apparatus 3 selects the next file system (S 10 ). After this, step S 3 for selecting one file of the selected file systems is returned to and the same processing is repeated (S 3 to S 10 ).
- FIG. 10 is a flowchart showing a detailed processing procedure of the CPU 55 of the file system transfer apparatus 4 relating to the file system transfer start determination processing occurring at this file system transfer apparatus 4 .
- the CPU 55 of the file system transfer apparatus 4 waits in standby mode for the activation start time of the file system transfer start determination process to elapse in accordance with file system transfer start determination processing procedure RT 2 shown in FIG. 10 by executing the file system transfer start determination processing program 81 for determining whether or not transfer of the file system has started (S 11 ).
- the activation start time is set to a fixed time interval that is comparatively long that is, for example, a predetermined certain time of a certain day of one month.
- the CPU 55 of the file system transfer apparatus 4 checks the file systems mounted at a predetermined host apparatus 3 by executing the file system management program 74 , and selects one file system of the file systems (S 12 ). The CPU 55 of the file system transfer apparatus 4 then checks whether or not the selected file system matches with the migration rules 61 .
- the CPU 55 of the file system transfer apparatus 4 proceeds to step S 14
- the CPU 55 of the file system transfer apparatus 4 executes file system transfer notification processing (RT 3 ) by executing the file system transfer notification program 83 .
- FIG. 11 is a flowchart showing a detailed processing procedure of the CPU 55 of the file system transfer apparatus 4 relating to the file system transfer processing occurring at this file system transfer apparatus 4 .
- the CPU 55 of the file system transfer apparatus 4 transfers the selected file system from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 in accordance with the file system transfer processing procedure RT 3 shown in FIG. 11 by executing the file system transfer processing procedure 84 for transferring the file systems (S 21 ).
- the CPU 55 of the file system transfer apparatus 4 sends the file system transfer request for the selected file system to the first storage apparatus 6 and the second storage apparatus 7 .
- the CPU 102 A of the first storage apparatus 6 sends a file system stored in the logical volume 112 A corresponding to the file system transfer request to the second storage apparatus 7 .
- the CPU 102 B of the second storage apparatus 7 stores the file system sent from the first storage apparatus 6 in the logical volume 112 B.
- the second storage apparatus 7 sends the file system transfer completion notification to the file system transfer apparatus 4 and the first storage system 6 .
- the first storage apparatus 6 deletes the transmitted file system from the logical volume 112 A.
- the CPU 55 of the file system transfer apparatus 4 updates the file system transfer management table 62 (S 22 ).
- the CPU 55 of the file system transfer apparatus 4 changes the file system state 152 from “primary storage”, “write enabled”, and “online” to “secondary storage”, “write inhibit”, and “online”, changes the original storage apparatus identifier 154 and the original logical volume identifier 155 to the identifiers for the first storage apparatus 6 and the logical volume 112 A stored by the transferred file system, and changes the current storage apparatus identifier 157 and the current logical volume identifier 158 to the identifiers for the second storage apparatus 7 and the logical volume 112 B storing the transferred file system.
- the CPU 55 of the file system transfer apparatus 4 then makes a file system transfer notification by executing the synchronization management program 82 that is a program controlling communication and synchronization with other apparatus such as the host apparatus 3 , and sends the file system transfer notification and the file system transfer information 159 for the updated file transfer management table 62 to the host apparatus 3 corresponding to the current host apparatus identifier 156 of the mapping record (S 23 ).
- the CPU 55 of the file system transfer apparatus 4 then ends the file system transfer processing procedure RT 3 (S 24 ) after this, and proceeds to step S 14 of the file system transfer start determination processing procedure RT 2 shown in FIG. 10 .
- the CPU 55 of the file system transfer apparatus 4 then checks whether or not all of the file systems mounted on the host apparatus 3 have been selected (S 14 ). In the event that all of the file systems mounted at the host apparatus 3 are not selected (S 14 : NO), the CPU 55 of the file system transfer apparatus 4 selects the next file system (S 10 ). After this, step S 13 for checking whether or not the selected file system matches with the migration rules 61 is returned to, and the same processing is then repeated (S 13 to S 15 , RT 3 ).
- FIG. 12 is a flowchart showing a specific processing procedure of the CPU 15 of the host apparatus 3 relating to the index update management processing occurring at this host apparatus 3 .
- the CPU 15 of the host apparatus 3 waits in a standby mode to receive file system transfer notification and file system transfer information 159 of the system transfer management table 62 from the file system transfer apparatus 4 in accordance with the index update management processing procedure RT 4 shown in FIG. 12 by executing the index update management processing program 44 for managing and updating the various index tables (S 31 ).
- the CPU 15 of the host apparatus 3 updates the file system transfer management table 23 of the index database 20 in the same way as updating the file system transfer management table 62 by executing the database management program 46 managing the index database 20 using the file system transfer information 159 of the file system transfer management table 62 (S 32 ).
- the CPU 15 of the host apparatus 3 then returns to a standby mode awaiting the receipt of file system transfer notification and file system transfer information 159 for the system transfer management table 62 from the file system transfer apparatus 4 (S 31 ), and the same process is then repeated after this (S 31 and S 32 ).
- FIG. 13 is a flowchart showing a specific processing procedure of the CPU 15 of the host apparatus 3 relating to the index update management processing occurring at this host apparatus 3 .
- the CPU 15 of the host apparatus 3 goes into standby mode awaiting the receipt of a file transfer request from the client apparatus 2 in accordance with the file search processing procedure RT 5 shown in FIG. 13 by executing the file search processing program 41 for searching files (S 41 ).
- Conditions for searching for files such as, for example, “files containing both keyword “budget” and keyword “IT” and searching of all of the files made during the past three years” etc. are described in the file search request.
- the CPU 15 of the host apparatus 3 Upon receiving the file search request from the client apparatus 2 (S 41 ; YES), the CPU 15 of the host apparatus 3 searches the index table 21 , reverse index table 22 and file system transfer management table 23 of the index database 20 by executing the database management program 46 , and sends the file search request to another host apparatus 3 by executing the synchronization management program 45 (S 42 ).
- the CPU 15 of the host apparatus 3 obtains the index record number 132 of the keywords 131 by searching for keywords 131 of the reverse index table 22 .
- the CPU 15 of the host apparatus 3 obtains the file system identifier 123 of the index record number 121 from this index record number 121 .
- the CPU 15 of the host apparatus 3 the obtains the current storage apparatus identifier 147 and the logical volume identifier 148 from this file system identifier 141 because the file system identifier 123 corresponds to the file system identifier 141 .
- the CPU 15 of the host apparatus 3 is capable of obtaining the file name 125 and file system identifier 123 of the attributes 126 by searching the attributes 126 of the index table 21 . As in the case described above, the CPU 15 of the host apparatus 3 then obtains the current storage apparatus identifier 147 and the logical volume identifier 148 from this file system identifier 141 because the file system identifier 123 corresponds to the file system identifier 141 .
- the CPU 15 of the host apparatus 3 is therefore able to recognize which logical volume of which storage apparatus the file system corresponding to the file search request of the client apparatus 2 is currently stored in by searching the index table 21 , reverse index table 22 , and file system transfer management table 23 of the index database 20 as described above.
- the synchronization management program 45 is executed, and the file search request is received from the host apparatus 3 , as in the case described above, by executing the database management program 46 , the CPU 15 of the other host apparatus 3 searches the index table 21 , the reverse index table 22 and the file system transfer management table 23 of the index database 20 , and sends the file name 125 , current storage apparatus identifier 147 and current logical volume identifier 148 subject to the file search request from the client apparatus 2 to the host apparatus 3 that received the file search request as a file search results report.
- the CPU 15 of the host apparatus 3 ends searching of the index table 21 , reverse index table 22 and file system transfer management table 23 of the index database 20 , and awaits the receipt of the file search results report from all of the other host apparatus 3 in standby mode (S 43 ).
- the CPU 15 of the host apparatus 3 makes a file search results synopsis list corresponding to the file search request from the client apparatus 2 based on the file search results at the host apparatus 3 and the file search results report from all of the other host apparatus 3 , and sends the file search results synopsis list to the client apparatus 2 (S 44 ).
- the CPU 15 of the host apparatus 3 then returns to standby mode where receipt of a file search request from the client apparatus 2 is again awaited (S 41 ), and then repeats the same processing (S 41 to S 44 ).
- FIG. 14 is a flowchart showing a specific processing procedure of the CPU 15 of the host apparatus 3 relating to the file read processing occurring at this host apparatus 3 .
- the CPU 15 of the host apparatus 3 goes into standby mode awaiting the receipt of a file read request from the client apparatus 2 in accordance with the file read processing procedure RT 6 shown in FIG. 14 by executing the file reading processing program 42 for reading files (S 51 ).
- the client apparatus 2 then refers to the file search results synopsis list sent from the host apparatus 3 , and in the event that a file the user of the client apparatus 2 wishes to read exists, a file read request for the file is sent to the host apparatus 3 .
- the CPU 15 of the host apparatus 3 refers to the file search results at the host apparatus 3 and the current storage apparatus identifier 147 and the current logical volume identifier 148 in the file search results report from the all of the other host apparatus 3 , and checks whether or not the file corresponding to the file read request is stored in the second storage apparatus 7 (S 52 ).
- the CPU 15 of the host apparatus 3 proceeds to step S 57 .
- the CPU 15 of the host apparatus 3 sends a mount request for the file system of the file corresponding to the file read request to the file system transfer apparatus 4 by executing the synchronization management program 45 (S 53 ).
- the CPU 15 of the host apparatus 3 awaits the receipt of a mount preparation completion notification from the file system transfer apparatus 4 in standby mode (S 54 ).
- the synchronization management program 82 is executed at the file system transfer apparatus 4
- the synchronization management program 45 is executed, and mount preparation completion notification and file system transfer information 159 of the system transfer management table 62 is received (S 54 : YES)
- the CPU 15 of the host apparatus 3 refers to the current storage apparatus identifier 157 and the current logical volume identifier 158 at the file system transfer information 159 of the system transfer management table 62 , and mounts the file system corresponding to the file read request at the host apparatus 3 (S 55 ).
- the CPU 15 of the host apparatus 3 updates the file system transfer management table 23 (S 56 ) in the same way as the updating occurring at the file system transfer management table 62 described in the following by executing the database management program 46 and using the file system transfer information 159 for the file system transfer management table 62 .
- the CPU 15 of the host apparatus 3 then reads out a file corresponding to the file read request from the file system (S 57 ).
- the CPU 15 of the host apparatus 3 sends the file to the client apparatus 2 that transmitted the file read request (S 58 ).
- the CPU 15 of the host apparatus 3 then returns to standby mode where receipt of a file read request from the client apparatus 2 is again awaited (S 51 ), and then repeats the same processing (S 51 to S 58 ).
- FIG. 15 is a flowchart showing a detailed processing procedure of the CPU 55 of the file system transfer apparatus 4 relating to the file system mount preparation processing occurring at this file system transfer apparatus 4 .
- the CPU 55 of the file system transfer apparatus 4 awaits the receipt of a mount request from the host apparatus 3 in standby mode (S 61 ).
- the file system of the file corresponding to the file read request checks whether or not direct mounting from the second storage apparatus 7 to the host apparatus 3 is possible (S 62 ).
- the CPU 55 of the file system transfer apparatus 4 has a table (not shown) within memory 56 describing in advance whether direct mounting of host apparatus 3 such as, for example, storage apparatus of a VDL (Virtual Disk Library) or SATA disc drive etc. can be directly mounted, or whether direct mounting is possible at host apparatus 3 such as a tape library etc. for, for example, all of the second storage apparatus 7 that are connected, and checks whether or not direct mounting is possible in accordance with the table.
- host apparatus 3 such as, for example, storage apparatus of a VDL (Virtual Disk Library) or SATA disc drive etc.
- the CPU 55 of the file system transfer apparatus 4 proceeds to step 64 .
- the CPU 55 of the file system transfer apparatus 4 transfers the file system for the file corresponding to the file read request from the logical volume 112 B of the second storage apparatus 7 to the logical volume 112 A of the first storage apparatus 6 (S 63 ).
- the CPU 55 of the file system transfer apparatus 4 sends the file system transfer request of the file system of the file corresponding to the file read request to the first storage apparatus 6 and the second storage apparatus 7 .
- the CPU 102 B of the second storage apparatus 6 sends a file system stored in the logical volume 112 B corresponding to the file system transfer request to the first storage apparatus 6 .
- the CPU 102 A of the first storage apparatus 6 then stores the file system sent from the second storage apparatus 7 in the logical volume 112 A.
- the first storage apparatus 6 sends the file system transfer completion report to the file system transfer apparatus 4 and the first storage system 6 .
- the CPU 55 of the file system transfer apparatus 4 updates the file system transfer management table 62 (S 64 ).
- the CPU 55 of the file system transfer apparatus 4 changes the file system state 152 from “secondary storage”, “write inhibit” and “offline” to “secondary storage”, “write permit”, and “online” at the mapping record corresponding to the file system identifier 151 of the file transfer management table 62 of the file system.
- the CPU 55 of the file system transfer apparatus 4 changes the file system state 152 from “secondary storage”, “write inhibit”, and “offline” to “primary storage”, “write inhibit”, and “online” at the mapping record corresponding to the file system identifier 151 of the file transfer management table 62 of the file system, changes the original storage apparatus identifier 154 and the original logical volume identifier 155 to the identifiers of the second storage apparatus 7 and the logical volume 112 B that stored the transferred file, and changes the current storage apparatus identifier 157 and current logical volume identifier 158 to the identifiers of the first storage apparatus 6 and the logical volume 112 A that stored the transferred file system.
- the CPU 55 of the file system transfer apparatus 4 makes file system mount preparation completion notification by executing the synchronization management program 82 , and sends the file system mount preparation completion notification and the file system transfer information 159 for the updated file transfer management table 62 to the host apparatus 3 corresponding to the current host apparatus identifier 156 of the mapping code (S 65 ).
- the CPU 55 of the file system transfer apparatus 4 returns to a standby mode so as to await the receipt of a mount request from the host apparatus 3 (S 61 ), and the same processing is then repeated thereafter (S 61 to 565 ).
- the CPU 55 of the file system transfer apparatus 4 can also operate so as to duplicate the file system to the logical volume 112 A of the first storage apparatus 6 .
- the CPU 55 of the file system transfer apparatus 4 stores a table (not shown) housing a file system identifier 151 for the duplicated file system in memory 56 , and stores a file system identifier 153 for the duplicated file system in the table when the file system is duplicated at the logical volume 112 A of the first storage apparatus 6 .
- the file system is already stored in the logical volume 112 B of the second storage apparatus 7 , and the same results as for transferring the system file from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 can be obtained by deleting the file system from the logical volume 112 A of the first storage apparatus 6 .
- the file system selected as described above is transferred from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 (S 21 ), the table storing the file system identifier 151 of the duplicated file system is searched, and in the event that the file system identifier 151 for the file system selected in the manner described above is stored on the table, the CPU 55 of the file system transfer apparatus 4 deletes the file system from the logical volume 112 A of the first storage apparatus 6 , transfer from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 can be completed, and the speed of file system transfer processing can be increased dramatically.
- the CPU 55 of the file system transfer apparatus 4 changes the file system state 152 from “primary storage”, “write enabled”, and “online” to “secondary storage”, “write inhibited”, and “online”, and switches over the original storage apparatus identifier 154 and the original logical volume identifier 155 with the current storage apparatus identifier 157 and the current logical volume identifier 158 .
- the file system is transferred from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 based on the file system transfer management table 61 , the file system transfer update information 159 of the file system transfer management table 62 of the file system transfer management apparatus 4 updated in accordance with the transfer of the file system is sent to the host apparatus 3 , and the file system transfer information 159 of the file system transfer management table 23 of the index database 20 of the host apparatus 3 is updated (A 3 ).
- the data management system 1 when a file search request is received from the client apparatus 2 , the state where the file system management table 23 is not updated is effectively prevented before it happens, it is possible to search for the file corresponding to the file search request, and as a result, it is possible to read the file requested by the client apparatus 2 . It is therefore possible to provide a data management system with highly convenient file management and low operating costs, having both high availability and reliability.
- the file system transfer apparatus 4 it is determined whether or not the file system for the file corresponding to the file read request is capable of being mounted directly from the second storage apparatus 7 to the host apparatus 3 .
- the file system for the file corresponding to the file read request is transferred from the logical volume 112 B of the second storage apparatus 7 to the logical volume 112 A of the first storage apparatus 6 so as to make mounting possible. This means that it is possible to prevent a situation where a file of a file system cannot be read even in the event of transfer from the first storage apparatus 6 to the second storage apparatus 7 and it is possible to read the file requested by the client apparatus 2 .
- the data management system 1 at the file system transfer apparatus 4 , by duplicating the file system of the file corresponding to the file read request from the logical volume 112 B of the second storage apparatus 7 to the logical volume 112 A of the first storage apparatus 6 , when the file system is then again transferred from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 , it is possible to obtain the same results as for transferring from the logical volume 112 A of the first storage apparatus 6 to the logical volume 112 B of the second storage apparatus 7 by deleting the file system from the logical volume 112 A of the first storage apparatus 6 .
- each host apparatus 3 implements an index producing function and an index updating management function
- an index server 171 may be provided, with the agent 32 then being divided into an agent 172 having a file search function and a file read function and an agent 173 having an index producing function and an index update management function.
- An index producing function and index update management function implemented with each host apparatus 3 distributed may then be implemented focused on the index server 171 .
- the file system transfer apparatus 4 implements each of the functions relating to the file system transfer described above, but the present invention is by no means limited in this respect, and, for example, as shown in FIG. 16 , it is also possible for the first storage apparatus 6 to have the function relating to transfer of the file systems in the possession of the file system transfer apparatus 4 , and to implement each of the functions relating to file system transfer the file system transfer apparatus 4 is in possession of at the first storage apparatus 6 .
- the present invention is therefore broadly applicable to data management systems that search files based on requests from client apparatus and read out the files from client apparatus.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A file system transfer designation section for transferring the file system matching with file system transfer rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system transfer rules, a file system storage information manager for updating storage information of the file system in accordance with transfer of the file system by the file system transfer designation section, and transmitting the updated file system storage information, and a search information manager for updating search information for searching the files based on a file search request from the client apparatus using the file system storage information sent by the file system storage information manager are provided.
Description
- This application relates to and claims priority from Japanese Patent Application No. 2006-247896, filed on Sep. 13, 2006, the entire disclosure of which is incorporated herein by reference.
- The present invention relates to a data management system and data management method appropriate for application in data management systems for searching files based on requests from, for example, client apparatus.
- In the related art, computer systems are implemented where a client apparatus, host apparatus and storage apparatus are connected to a network so that files managed by the client apparatus are transmitted and received via the host apparatus and the network, and the files are stored on a hard disc within the storage apparatus.
- In this kind of computer system, for example, file systems (sets of files) that are not used often are transferred from so-called high-cost hard disc drives within the storage apparatus to so-called low cost disc drives.
- Further, with this kind of computer system, for example, file search apparatus are connected to a network. When there is then a file read request instructing the reading of a predetermined file from the client apparatus, the client apparatus searches for the requested file, and the file is read out from the storage apparatus to the client apparatus.
- However, a search engine for the file search apparatus is usually developed and put into the form of a product by a dedicated vendor, while software for transferring files of the storage apparatus is developed and put into the form of a product by a storage apparatus vendor. The means that the interface connecting the file search apparatus and the storage apparatus is not standardized.
- Because of this, at the search engine for the file search apparatus, after making a search index table constituting an index in the searching of a file, at the storage apparatus, in the event of transferring the file system, it is no longer possible to search files of the file system based on the search index table and an error therefore results. Therefore, with this kind of computer system, it is necessary to re-make the search index table in order to search files based on the search index table.
- In theory, the administrator of the storage apparatus refers to the transfer history, and takes into consideration an operation where the search index table is updated every transferred file system, but there are cases where several hundred thousand files are stored in one file system of the storage apparatus. As the search index table is made in file units, it is not practical for the administrator of the storage apparatus to update the search index table.
- As a result, a file management system is proposed in Japanese Patent Laid-open Publication No. 2003-280950 where not used information for file storage locations, metadata for the files and access priority information is managed, the stored files are accessed and rewriting of the files is carried out, where data access are exclusively controlled and the storage locations of files are changed based on storage location information and access priority information for files contained in the metadata.
- However, in recent years from the point of view of compliance (act adherence) etc., it is necessary even for files that are used infrequently to be held for fixed periods. File systems for files that are used infrequently are therefore transferred to other storage apparatus and saved.
- In the event that this kind of file system is transferred to another storage apparatus, or in the event that a file system is transferred to another storage apparatus the host apparatus is incapable of mounting, it is not possible to update the storage location information for the file system, files of the file system cannot be searched, and as a result it is not possible to read files requested by the client apparatus.
- In order to take the aforementioned points into consideration, the present invention therefore proposes a data management system and data management method with highly reliable data management.
- In order to resolve these problems, in one aspect of the present invention, a data management system with a host apparatus receiving requests from client apparatus, first storage apparatus having a first volume for storing a file system composed of a set of files, and a second storage apparatus having a second volume for storing the file system sent from the first storage apparatus comprises a file system transfer designation section for transferring the file system matching with file system migration rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system migration rules, a file system storage information manager for updating storage information of the file system in accordance with transfer of the file system by the file system transfer designation section, and transmitting the updated file system storage information, and a search information manager for updating search information for searching the files based on a file search request from the client apparatus using the file system storage information sent by the file system storage information manager.
- Further, in an aspect of the present invention, a data management system with a host apparatus receiving requests from client apparatus, first storage apparatus having a first volume for storing a file system composed of a set of files, and a second storage apparatus having a second volume for storing the file system sent from the first storage apparatus comprises a first step of transferring the file system matching with file system migration rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system migration rules, a second step of updating storage information for the file system in accordance with transfer of the file system of the first step, and transmitting the stored information of the updated file system, and a third step of updating the search information for searching the files based on file search requests from the client apparatus using file system storage information sent in the second step.
- Therefore, when a file search request is received from the client apparatus, a state where search information is not updated is effectively prevented before it happens, it is possible to search for a file corresponding to the file search request, and as a result, it is possible to read the file requested by the client apparatus.
- According to the present invention, it is possible to implement a data management system and data management method where the reliability of file management is high.
- Other aspects and advantages of the invention will be apparent from the following description and the appended claims.
-
FIG. 1 is a block view showing an outline configuration for a data management system of an embodiment; -
FIG. 2 is a block view showing an outline configuration for host apparatus; -
FIG. 3 is a block view showing an outline configuration for file system transfer apparatus; -
FIG. 4 is a block view showing an outline configuration for the first storage apparatus and the second storage apparatus; -
FIG. 5 is conceptual view illustrating an index table; -
FIG. 6 is a conceptual view illustrating a reverse index table; -
FIG. 7 is a conceptual view illustrating a file system transfer management table; -
FIG. 8 is a conceptual view illustrating an outline of an operation for a data management system; -
FIG. 9 is a flowchart showing an index generation procedure; -
FIG. 10 is a flowchart showing a file system transfer start determination processing procedure; -
FIG. 11 is a flowchart showing a file system transfer processing procedure; -
FIG. 12 is a flowchart showing an index update management processing procedure; -
FIG. 13 is a flowchart showing a file system search handling procedure; -
FIG. 14 is a flowchart showing a file system read processing procedure; -
FIG. 15 is a flowchart showing a file system mount preparation processing procedure; and -
FIG. 16 is a block view showing an outline configuration for a data management system of a further embodiment. - The following is a description with reference to the drawings below of a first embodiment of the present invention.
-
FIG. 1 shows a configuration for adata management system 1 of an embodiment of the present invention. Thedata management system 1 is comprised ofclient apparatus 2,host apparatus 3 and filesystem transfer apparatus 4 connected via a LAN (Local Area Network) 5, and thehost apparatus 3, filesystem transfer apparatus 4,first storage apparatus 6 andsecond storage apparatus 7 connected via an SAN (Storage Area Network) 8. - The
client apparatus 2 is computer apparatus equipped with information processing resources such as a CPU (Central Processing Unit) and memory etc., such as, for example, a personal computer etc. Theclient apparatus 2 are connected to information input apparatus (not shown) such as a keyboard, switches and pointing apparatus, and microphone etc., and also have information output apparatus (not shown) such as monitor displays and speakers etc. In this event, theclient apparatus 2 sends a file read-out request to read out a file to thehost apparatus 3 based on a file search request for searching files resulting from designation using, for example, keywords etc. and the search results. -
FIG. 2 shows a configuration for thehost apparatus 3. Thehost apparatus 3 is configured from an Ethernet (registered trademark)controller 11 connected toLAN 5 for controlling Ethernet (registered trademark) communication, afiber channel controller 12 for controlling fiber channel communication, an input/output controller 13 for controlling information input apparatus and information output apparatus, a harddisc drive controller 14 for controlling a hard disc drive section 19, aCPU 15 for performing overall control of thehost apparatus 3, and amemory 16 for storing various programs etc, with each element of this configuration described above being connected via abridge chip 17 for controlling transfer of files etc. - Further, the hard disc drive section 19 equipped with a number of
hard disc drives 18 is connected to the harddisc drive controller 14. In this event, some or all of predeterminedhard disc drives 18 are used as anindex database 20 for storing various tables etc. used in file searching etc. An index table 21 for storing location information etc. every file, a reverse index table 22 for storing file information etc. containing the keywords every keyword, and a file system transfer management table 23 for storing information etc. for before and after transfer for file systems for between logical volumes of storage apparatus described later are stored in theindex database 20. A detailed description of the index table 21, reverse index table 22 and file system transfer management table 23 is given in the following. - Here, a file system is a set of files. In this event, a large number of files are contained in one file system. A file is digital data and a file system that is this set is also digital data. A logical volume is equipment for storing the file system. In this embodiment, a one to one configuration is adopted where one file system is stored in one logical volume, but the present invention is by no means limited in this respect, and may also be applied to a configuration where a single file system spans a number of logical volumes, or where a single logical volume stores a number of file systems.
- Further, an
application program 31 that is a program for executing various processing is also stored in thememory 16 of thehost apparatus 3. Theapplication program 31 is comprised of anagent 21 that is a program executing file search processing and file reading processing, an HTTP/XMLstack 33 for executing control based on HTTP/XML (HyperText Transfer Protocol/eXtensible Markup Language), a TCP/IP stack 34 for executing control based on TCP/IP (Transmission Control Protocol/Internet Protocol), a filesystem mounting program 35 that is a program for managing file systems mounted/un-mounted at thehost apparatus 3, a filesystem management program 36 that is a program for managing file systems, an Ethernet (registered trademark)driver 37 that is a driver for controlling the Ethernet (registered trademark)controller 11, and afiber channel driver 38 that is a driver for controlling thefiber channel controller 12. Here, “mount” refers to recognizing the file system at thehost 3 and puts thehost apparatus 3 into a state enabling the execution of read/write processing. - The
agent 32 is comprised of a filesearch processing program 41, a filereading processing program 42, anindexing program 43, an indexupdate management program 44, asynchronization management program 45, and adatabase management program 46. A detailed description of each type of program is given in the using the flowchart in the following. - The hardware configuration of this
host apparatus 3 may be implemented through various forms of general purpose computers (personal computers), workstations, and mainframes, etc. The index table 21, reverse index table 22 and file system transfer management table 23 may be stored in thememory 16. -
FIG. 3 shows a configuration for the filesystem transfer apparatus 4. The filesystem transfer apparatus 4 adopts the same configuration as for thehost apparatus 3, with the exception that the various programs and information etc. stored in thememory 56 and the harddisc drive section 59 are different. Namely, the filesystem transfer apparatus 4 is implemented using a general purpose computer (personal computer), etc. The filesystem transfer apparatus 4 is comprised of an Ethernet (registered trademark)controller 51, afiber channel controller 52, an input/output controller 53, a harddisc drive controller 54, aCPU 55, and amemory 56 connected via abridge 57. - A hard
disc drive section 59 equipped with a plurality of hard disc drives 58 is connected to thehard disc controller 54. In this event, migration rules 61 and a file system transfer management table 62 are stored in a predeterminedhard disc drive 58. - Fixed rules for transferring a file system, such as, for example, “in the event that any files of the file system have not been subjected to reading or writing processing for a predetermined period, transfer this file system from the
first storage apparatus 6 to thesecond storage apparatus 7”, and “in the event that the final update time where the frequency of read/write processing during the preceding three months is low at 20% of the file systems mounted at thehost apparatus 3 is checked and one month or more has passed since the final update time, this file system is transferred from thefirst storage apparatus 6 to thesecond storage apparatus 7”, are described in the migration rules 61. The configuration of the file system transfer management table 62 is the same as the file system transfer management table 23 described above. - Further, a file
system transfer manager 71 that is a program for executing file system transfer processing etc., a HTTP/XML stack 72, a TCP/IP stack 73, a filesystem management program 74, an Ethernet (registered trademark)driver 75 and afiber channel driver 76 are stored in thememory 16 of the filesystem transfer apparatus 4. The filesystem management program 74 differs from the filesystem mounting program 35 in managing the file systems that are mounted/un-mounted at all of thehost apparatus 3. - The file
system transfer manager 71 is comprised of a file systemtransfer initiating program 81, asynchronization management program 82, a file systemtransfer notification program 83, a filesystem transfer program 84, and a file system mountingpreparation program 85. A detailed description of each type of program is given in the using the flowchart in the following. - The hardware configuration of this file
system transfer apparatus 4 may be implemented by personal computers etc. in various forms. The migration rules 61 and the file system transfer management table 62 may also be stored in thememory 56. -
FIG. 4 shows a configuration for thefirst storage apparatus 6 and thesecond storage apparatus 7. Thefirst storage apparatus 6 is comprised of storage controller 91A and astorage section 92A. The storage controller 91A is configured in such a manner that afront end controller 101A for controlling fiber channel communication connected toSAN 8,CPU 102A for overall control of thefirst storage apparatus 6, aback end controller 106A connected to thecache memory 104A for temporarily storing transmitted and received files andstorage section 92A, for controlling communication with thestorage section 92A, are connected with each element of the configuration described above, and are connected viadata transfer controller 103A for controlling the transfer of data. Further, alocal memory 105A for storing programs and tables necessary for controlling thefirst storage apparatus 6 and information etc. is connected to theCPU 102A. Moreover, thestorage section 92A is equipped with a plurality ofhard disc drives 111A. - The
CPU 102A is capable of controlling the plurality ofhard disc drives 111A using RAID levels (for example, RAID0, 1, 5 etc.) defined in the so-called RAID system. The RAID system manages a plurality ofhard disc drives 111A as a single RAID group. A plurality oflogical volumes 112A are then defined in access units by thehost apparatus 3 on the RAID group. A LUN (Logical Unit Number) is allocated to the respectivelogical volumes 112A. - The
hard disc drive 111A is, for example, a storage device such as an FC (Fibre Channel) disc drive, a SATA (Serial Advanced Technology Attachment) disc drive, a PATA (Parallel Advanced Technology Attachment) disc drive, a FATA (Fibre Attached Technology Adapted) disc drive, an SAS (Serial Attached SCSI) disc drive, or a SCSI (Small Computer System Interface) disc drive, etc. - The
second storage apparatus 7 has the same configuration as thefirst storage apparatus 6 described above, with each element of the configuration of thesecond storage apparatus 7 being given the same numerals for the same portions as the configuration elements of thefirst storage apparatus 6, with suffix “A” being substituted with suffix “B”. At thedata management system 1, from the point of view that thesecond storage system 7 is a storage system for archiving use, thedisc drive 111A of thefirst storage system 6 is configured from, for example, a so-called high-cost hard disc drive such as a SCSI (Small Computer System Interface) disc drive etc., and thedrive 111B of thesecond storage system 7 is configured from a so-called low-cost hard disc drive such as, for example, a SATA (Serial AT Attachment) disc drive or optical disc drive, etc. - Next, a description is given of the configuration of the index table 21, reverse index table 22, and file system transfer management table 23.
-
FIG. 5 is a view showing a configuration for the index table 21. The index table 21 stores anindex record number 121 for identifying the index record, ahost apparatus identifier 122 for identifying thehost apparatus 3 that made the file, afile system identifier 123 for identifying the file system for the logical volume storing the file, apath name 124 for a path indicating the location of the file system, afile name 125 for the file, attributes 126 comprised of a file update date and access priority control information etc., and akeyword list 127 composed of keywords for the file, every index record. There are also cases where thehost apparatus identifier 122,file system identifier 123,path name 124 andfile name 125 are collectively referred to aslocation information 128. Thehost apparatus identifier 122 is to take into consideration the case where a plurality of host apparatus exist, where ahost apparatus 3 mounted to after a logical volume goes temporarily offline (described later) and goes online once again (described later) is different to thehost apparatus 3 that first made the file. -
FIG. 6 is a view showing a configuration for the reverse index table 22. The reverse index table 22 is a table adopting, for example, a B-Tree (Balanced tree) algorithm, that is capable of searching using an index tree (index tree) of this structure at high-speed. This reverse index table 22 is capable of searching everykeyword 131. The reverse index table 22 stores anindex record number 132 and attributes 133 for a file having thiskeyword 131 every reverse record for thekeyword 131. In this event, theindex record number 132 of the reverse index table 22 corresponds to theindex record number 121 of the index table 21. The reverse index table 22 is then capable of searching theindex record number 121 of the index table 21 from thekeyword 131. -
FIG. 7 shows a configuration for the file system transfer management table 23. The file system transfer management table 23 stores a file system identifier 141, a file system state 142 indicating the storage state of the file system, an original host apparatus identifier 143 for identifying thehost apparatus 3 that made the file first, an original storage apparatus identifier 144 for identifying the storage apparatus that stored the file system before transfer, an original logical volume identifier 145 for identifying the logical volume that stored the file system before transfer, a current host apparatus identifier 146 for identifying ahost apparatus 3 mounting a file system currently containing the file, a current storage apparatus identifier 147 for identifying the storage apparatus currently storing the file system, and an original logical volume identifier 148 for identifying the logical volume currently storing the file system, every mapping record. - It is necessary to update the file system state 142, the original host apparatus identifier 143, the original storage apparatus identifier 144, the original logical volume identifier 145, the current host apparatus identifier 146, the current storage apparatus identifier 147, and the original logical volume identifier 148 as a result of transfer of the file system. Further, there are cases where the file system state 142, the original host apparatus identifier 143, the original storage apparatus identifier 144, the original logical volume identifier 145, the current host apparatus identifier 146, the current storage apparatus identifier 147, and the original logical volume identifier 148 are collectively referred to as file system transfer information 149.
- At the file system state 142, for example, “primary storage” is described for the case where the file system is stored in the
first storage apparatus 6, “secondary storage” is described for the case where the file system is stored in thesecond storage apparatus 6, “write enable” is described in the case where the file system can be written to, “write inhibit” is described for the case where it is not possible to write to the file system, “online” is described for the case where the file system is mounted at thehost apparatus 3, and “offline” is described for the case where the file system is not mounted at thehost apparatus 3. A sequential description is given of a state transition for the storage state of the file system. Further, “online” indicates the state where the file system is mounted at thehost apparatus 3, and offline indicates the state where the file system is not mounted at thehost apparatus 3. - Further, the file system transfer management table 62 stores a file system identifier 151, a file system state 152, an original host identifier 153, an original storage apparatus identifier 154, an original logical volume identifier 155, a current host apparatus identifier 156, a current
storage apparatus identifier 157, and an original logical volume identifier 158, every mapping record. Moreover, there are cases where the file system state 152, the original host identifier 153, the original storage apparatus identifier 154, the original logical volume identifier 155, the current host apparatus identifier 156, the currentstorage apparatus identifier 157, and the original logical volume identifier 158 are collectively referred to as filesystem transfer information 159. -
FIG. 8 shows an outline of the operation of thedata management system 1. With thedata management system 1, at the filesystem transfer apparatus 4, the file system is transferred from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 based on the file system transfer management table 61 (A1), the file systemtransfer update information 159 of the file system transfer management table 62 of the file systemtransfer management apparatus 4 updated in accordance with the transfer of the file system is sent to the host apparatus 3 (A2), and the filesystem transfer information 159 of the file system transfer management table 23 of theindex database 20 of thehost apparatus 3 is updated (A3). - Specifically, describing an outline of an operation of the
data management system 1, thehost apparatus 3 stores the file in thelogical volume 112A of thefirst storage apparatus 6 by sending the file sent from theclient apparatus 2 to thefirst storage apparatus 6. Thehost apparatus 3 reads out and analyzes a file sent from theclient apparatus 2, or made or updated at thehost apparatus 3 from thefirst storage apparatus 6 by periodically executing theagent 32, and makeslocation information 128, attributes 126, or akeyword list 127. Thehost apparatus 3 updates the index table 21 and the reverse index table 22 of theindex database 20 using the producedlocation information 128, attributes 126, andkeyword list 127. - Here, the file
system transfer apparatus 4 transfers thefile system 161 from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 by executing the filesystem transfer manager 71 at a predetermined timing, and sending a file system transfer request for thefile system 161 to thefirst storage apparatus 6 and thesecond storage apparatus 7 at a predetermined period where reading and writing processing is not taking place in accordance with the migration rules 61. - When transfer of the
file system 161 is complete, the filesystem transfer apparatus 4 updates the filesystem transfer information 159 of the file system transfer management table 62. The filesystem transfer apparatus 4 then sends the filesystem transfer information 159 of the updated file system transfer management table 62 to thehost apparatus 3 together with the file system transfer notification. - The
host apparatus 3 then updates the file system transfer information 149 of the file system transfer management table 23 of theindex database 20 using the file systemtransfer update information 159 of the transmitted file system transfer management table 62. - As a result, at the
data management system 1, when a file search request is received from theclient apparatus 2, the state where the file system management table 23 is not updated is effectively prevented before it happens, it is possible to search for the file corresponding to the file search request, and as a result, it is possible to read the file requested by theclient apparatus 2. - Next, a description is given of index generation occurring at the
host apparatus 3.FIG. 9 is a flowchart showing a specific processing procedure of theCPU 15 of thehost apparatus 3 relating to the index generation occurring at thishost apparatus 3. - When the
host apparatus 3 is started up, theCPU 15 of thehost apparatus 3 waits in a standby mode for an activation start time of the index generation to elapse in accordance with the index generation procedure RT1 shown inFIG. 9 by executing theindex generation program 43 making each type of index table. The activation start time is set to a fixed time interval that is comparatively short that is, for example a predetermined certain time of one day (once every 24 hours), etc. - When the activation start time for the index generation elapses (S1: YES), the
CPU 15 of thehost apparatus 3 checks the file systems mounted at thehost apparatus 3 by executing the filesystem management program 36, and selects one file system of the file systems (S2). TheCPU 15 of thehost apparatus 3 then selects one file of the selected file system (S3). TheCPU 15 of thehost apparatus 3 then checks whether the selected file is a file that is newly added or whether or not the file is an updated file (S4). - In the event that the selected file is not a newly added file and is not an updated file (S4: NO), the
CPU 15 of thehost apparatus 3 proceeds to step S7. On the other hand, in the event that the selected file is a newly added file or an updated file (S4: NO), theCPU 15 of thehost apparatus 3 reads out the file and analyzes the content of the file (S5). - The
CPU 15 of thehost apparatus 3 then produces thelocation information 128, attributes 126 andkeyword list 127 based on the file analysis results by executing thedatabase management program 46, and updates the index table 21 and/or the reverse index table 22 of theindex database 20 using the producedlocation information 128, attributes 126, and keyword list 127 (S6). - The
CPU 15 of thehost apparatus 3 then checks whether or not all of the files of the selected file system have been selected (S7). In the event that all of the files for the selected file system have not been selected (S7: NO), theCPU 15 of thehost apparatus 3 selects the next file of the selected file system (S8). After this, step S4 for checking whether or not the selected file is a newly added file or an updated file is returned to, and the same processing is then repeated (S4 to S8). - On the other hand, in the event that all of the files for the selected file system are selected (S7: YES), the
CPU 15 of thehost apparatus 3 checks whether or not all of the file systems mounted at thehost apparatus 3 have been selected (S9). In the event that all of the file systems mounted at the host apparatus have not been selected (S9: NO), theCPU 15 of thehost apparatus 3 selects the next file system (S10). After this, step S3 for selecting one file of the selected file systems is returned to and the same processing is repeated (S3 to S10). - On the other hand, in the event that all of the file systems mounted at the
host apparatus 3 have been selected (S9: YES), theCPU 15 of thehost apparatus 3 returns to a standby mode awaiting the elapsing of the activation start time for the index generation (S1), and after this, the same processing is repeated (S1 to S10). - Next a description is given of file system transfer start determination processing occurring at the file
system transfer apparatus 4.FIG. 10 is a flowchart showing a detailed processing procedure of theCPU 55 of the filesystem transfer apparatus 4 relating to the file system transfer start determination processing occurring at this filesystem transfer apparatus 4. - When the file
system transfer apparatus 4 starts up, theCPU 55 of the filesystem transfer apparatus 4 waits in standby mode for the activation start time of the file system transfer start determination process to elapse in accordance with file system transfer start determination processing procedure RT2 shown inFIG. 10 by executing the file system transfer startdetermination processing program 81 for determining whether or not transfer of the file system has started (S11). The activation start time is set to a fixed time interval that is comparatively long that is, for example, a predetermined certain time of a certain day of one month. - When the activation start time for the index generation elapses (S11; YES), the
CPU 55 of the filesystem transfer apparatus 4 checks the file systems mounted at apredetermined host apparatus 3 by executing the filesystem management program 74, and selects one file system of the file systems (S12). TheCPU 55 of the filesystem transfer apparatus 4 then checks whether or not the selected file system matches with the migration rules 61. - In the event that the selected file system does not match with the migration rules 61 (S13: NO), the
CPU 55 of the filesystem transfer apparatus 4 proceeds to step S14 On the other hand, in the event that the selected file system matches with the migration rules 61 (S13: YES), theCPU 55 of the filesystem transfer apparatus 4 executes file system transfer notification processing (RT3) by executing the file systemtransfer notification program 83. - Here, a description is given of file system transfer processing occurring at the file
system transfer apparatus 4.FIG. 11 is a flowchart showing a detailed processing procedure of theCPU 55 of the filesystem transfer apparatus 4 relating to the file system transfer processing occurring at this filesystem transfer apparatus 4. - When the selected file system matches with the migration rules 61, the
CPU 55 of the filesystem transfer apparatus 4 transfers the selected file system from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 in accordance with the file system transfer processing procedure RT3 shown inFIG. 11 by executing the file systemtransfer processing procedure 84 for transferring the file systems (S21). - Specifically, the
CPU 55 of the filesystem transfer apparatus 4 sends the file system transfer request for the selected file system to thefirst storage apparatus 6 and thesecond storage apparatus 7. When a file system transfer request transmitted from the filesystem transfer apparatus 4 is received, theCPU 102A of thefirst storage apparatus 6 sends a file system stored in thelogical volume 112A corresponding to the file system transfer request to thesecond storage apparatus 7. TheCPU 102B of thesecond storage apparatus 7 stores the file system sent from thefirst storage apparatus 6 in thelogical volume 112B. When transfer of the file system is complete, thesecond storage apparatus 7 sends the file system transfer completion notification to the filesystem transfer apparatus 4 and thefirst storage system 6. When a file system transfer completion notification is received from thesecond storage apparatus 7, thefirst storage apparatus 6 deletes the transmitted file system from thelogical volume 112A. - When file system transfer completion notification is received from the
second storage apparatus 7, theCPU 55 of the filesystem transfer apparatus 4 updates the file system transfer management table 62 (S22). - Specifically, at the mapping record corresponding to the file system identifier 151 of the file transfer management table 62 of the transferred file system, for example, the
CPU 55 of the filesystem transfer apparatus 4 changes the file system state 152 from “primary storage”, “write enabled”, and “online” to “secondary storage”, “write inhibit”, and “online”, changes the original storage apparatus identifier 154 and the original logical volume identifier 155 to the identifiers for thefirst storage apparatus 6 and thelogical volume 112A stored by the transferred file system, and changes the currentstorage apparatus identifier 157 and the current logical volume identifier 158 to the identifiers for thesecond storage apparatus 7 and thelogical volume 112B storing the transferred file system. - The
CPU 55 of the filesystem transfer apparatus 4 then makes a file system transfer notification by executing thesynchronization management program 82 that is a program controlling communication and synchronization with other apparatus such as thehost apparatus 3, and sends the file system transfer notification and the filesystem transfer information 159 for the updated file transfer management table 62 to thehost apparatus 3 corresponding to the current host apparatus identifier 156 of the mapping record (S23). - The
CPU 55 of the filesystem transfer apparatus 4 then ends the file system transfer processing procedure RT3 (S24) after this, and proceeds to step S14 of the file system transfer start determination processing procedure RT2 shown inFIG. 10 . - The
CPU 55 of the filesystem transfer apparatus 4 then checks whether or not all of the file systems mounted on thehost apparatus 3 have been selected (S14). In the event that all of the file systems mounted at thehost apparatus 3 are not selected (S14: NO), theCPU 55 of the filesystem transfer apparatus 4 selects the next file system (S10). After this, step S13 for checking whether or not the selected file system matches with the migration rules 61 is returned to, and the same processing is then repeated (S13 to S15, RT3). - On the other hand, in the event that all of the file systems mounted at the
host apparatus 3 have been selected (S14: YES), theCPU 15 of thehost apparatus 3 returns to a standby mode awaiting the elapsing of the activation start time for the file system start determination processing (S11), and after this, the same processing is repeated (S11 to S15). - Next, a description is given of index update management processing occurring at the
host apparatus 3.FIG. 12 is a flowchart showing a specific processing procedure of theCPU 15 of thehost apparatus 3 relating to the index update management processing occurring at thishost apparatus 3. - When the
host apparatus 3 starts up, theCPU 15 of thehost apparatus 3 waits in a standby mode to receive file system transfer notification and filesystem transfer information 159 of the system transfer management table 62 from the filesystem transfer apparatus 4 in accordance with the index update management processing procedure RT4 shown inFIG. 12 by executing the index updatemanagement processing program 44 for managing and updating the various index tables (S31). - When the
synchronization management program 82 is executed at the filesystem transfer apparatus 4, when thesynchronization management program 45 is then executed and file system transfer notification and filesystem transfer information 159 for the system transfer management table 62 is then received from the file system transfer apparatus 4 (S31: YES), theCPU 15 of thehost apparatus 3 updates the file system transfer management table 23 of theindex database 20 in the same way as updating the file system transfer management table 62 by executing thedatabase management program 46 managing theindex database 20 using the filesystem transfer information 159 of the file system transfer management table 62 (S32). - In this event, it is possible to perform update processing for the
index database 20 at a substantially higher speed at theCPU 15 of thehost apparatus 3 without updating the index records of the index table 21 constituted by an enormous number for managing each file by putting the index table to one side, managing the transfer of file systems at mapping records of the file system transfer management table 23 of a small total number of files, and managing logical volumes where these files are stored. - The
CPU 15 of thehost apparatus 3 then returns to a standby mode awaiting the receipt of file system transfer notification and filesystem transfer information 159 for the system transfer management table 62 from the file system transfer apparatus 4 (S31), and the same process is then repeated after this (S31 and S32). - Next, a description is given of file search processing occurring at the
host apparatus 3.FIG. 13 is a flowchart showing a specific processing procedure of theCPU 15 of thehost apparatus 3 relating to the index update management processing occurring at thishost apparatus 3. - When the
host apparatus 3 is started up, theCPU 15 of thehost apparatus 3 goes into standby mode awaiting the receipt of a file transfer request from theclient apparatus 2 in accordance with the file search processing procedure RT5 shown inFIG. 13 by executing the filesearch processing program 41 for searching files (S41). Conditions for searching for files such as, for example, “files containing both keyword “budget” and keyword “IT” and searching of all of the files made during the past three years” etc. are described in the file search request. - Upon receiving the file search request from the client apparatus 2 (S41; YES), the
CPU 15 of thehost apparatus 3 searches the index table 21, reverse index table 22 and file system transfer management table 23 of theindex database 20 by executing thedatabase management program 46, and sends the file search request to anotherhost apparatus 3 by executing the synchronization management program 45 (S42). - Specifically, in the event that the files are searched using the keywords, the
CPU 15 of thehost apparatus 3 obtains theindex record number 132 of thekeywords 131 by searching forkeywords 131 of the reverse index table 22. Next, because theindex record number 132 corresponds to theindex record number 121, theCPU 15 of thehost apparatus 3 obtains thefile system identifier 123 of theindex record number 121 from thisindex record number 121. TheCPU 15 of thehost apparatus 3 the obtains the current storage apparatus identifier 147 and the logical volume identifier 148 from this file system identifier 141 because thefile system identifier 123 corresponds to the file system identifier 141. - Further, in the case of searching files using these attributes, the
CPU 15 of thehost apparatus 3 is capable of obtaining thefile name 125 andfile system identifier 123 of theattributes 126 by searching theattributes 126 of the index table 21. As in the case described above, theCPU 15 of thehost apparatus 3 then obtains the current storage apparatus identifier 147 and the logical volume identifier 148 from this file system identifier 141 because thefile system identifier 123 corresponds to the file system identifier 141. - The
CPU 15 of thehost apparatus 3 is therefore able to recognize which logical volume of which storage apparatus the file system corresponding to the file search request of theclient apparatus 2 is currently stored in by searching the index table 21, reverse index table 22, and file system transfer management table 23 of theindex database 20 as described above. - When the
synchronization management program 82 is executed at thehost apparatus 3 receiving a file search request from theclient apparatus 2, thesynchronization management program 45 is executed, and the file search request is received from thehost apparatus 3, as in the case described above, by executing thedatabase management program 46, theCPU 15 of theother host apparatus 3 searches the index table 21, the reverse index table 22 and the file system transfer management table 23 of theindex database 20, and sends thefile name 125, current storage apparatus identifier 147 and current logical volume identifier 148 subject to the file search request from theclient apparatus 2 to thehost apparatus 3 that received the file search request as a file search results report. - Continuing on, the
CPU 15 of thehost apparatus 3 ends searching of the index table 21, reverse index table 22 and file system transfer management table 23 of theindex database 20, and awaits the receipt of the file search results report from all of theother host apparatus 3 in standby mode (S43). - When searching of the index table 21, reverse index table 22, and file system transfer management table 23 of the
index database 20 is complete and file search results report is received from all of the other host apparatus 3 (S43: YES), theCPU 15 of thehost apparatus 3 makes a file search results synopsis list corresponding to the file search request from theclient apparatus 2 based on the file search results at thehost apparatus 3 and the file search results report from all of theother host apparatus 3, and sends the file search results synopsis list to the client apparatus 2 (S44). - The
CPU 15 of thehost apparatus 3 then returns to standby mode where receipt of a file search request from theclient apparatus 2 is again awaited (S41), and then repeats the same processing (S41 to S44). - Next, a description is given of file read processing occurring at the
host apparatus 3.FIG. 14 is a flowchart showing a specific processing procedure of theCPU 15 of thehost apparatus 3 relating to the file read processing occurring at thishost apparatus 3. - When the
host apparatus 3 is started up, theCPU 15 of thehost apparatus 3 goes into standby mode awaiting the receipt of a file read request from theclient apparatus 2 in accordance with the file read processing procedure RT6 shown inFIG. 14 by executing the filereading processing program 42 for reading files (S51). Theclient apparatus 2 then refers to the file search results synopsis list sent from thehost apparatus 3, and in the event that a file the user of theclient apparatus 2 wishes to read exists, a file read request for the file is sent to thehost apparatus 3. - When a file read request is received from the client apparatus 2 (S51: YES), the
CPU 15 of thehost apparatus 3 refers to the file search results at thehost apparatus 3 and the current storage apparatus identifier 147 and the current logical volume identifier 148 in the file search results report from the all of theother host apparatus 3, and checks whether or not the file corresponding to the file read request is stored in the second storage apparatus 7 (S52). - In the event that a file corresponding to the file read request is not stored in the second storage apparatus 7 (S52: NO), the
CPU 15 of thehost apparatus 3 proceeds to step S57. On the other hand, in the event that the file corresponding to the file read request is stored in the second storage apparatus 7 (S52: YES), theCPU 15 of thehost apparatus 3 sends a mount request for the file system of the file corresponding to the file read request to the filesystem transfer apparatus 4 by executing the synchronization management program 45 (S53). - Continuing on, the
CPU 15 of thehost apparatus 3 awaits the receipt of a mount preparation completion notification from the filesystem transfer apparatus 4 in standby mode (S54). When thesynchronization management program 82 is executed at the filesystem transfer apparatus 4, thesynchronization management program 45 is executed, and mount preparation completion notification and filesystem transfer information 159 of the system transfer management table 62 is received (S54: YES), by executing the filesystem mounting program 35, theCPU 15 of thehost apparatus 3 refers to the currentstorage apparatus identifier 157 and the current logical volume identifier 158 at the filesystem transfer information 159 of the system transfer management table 62, and mounts the file system corresponding to the file read request at the host apparatus 3 (S55). - The
CPU 15 of thehost apparatus 3 updates the file system transfer management table 23 (S56) in the same way as the updating occurring at the file system transfer management table 62 described in the following by executing thedatabase management program 46 and using the filesystem transfer information 159 for the file system transfer management table 62. - The
CPU 15 of thehost apparatus 3 then reads out a file corresponding to the file read request from the file system (S57). TheCPU 15 of thehost apparatus 3 sends the file to theclient apparatus 2 that transmitted the file read request (S58). TheCPU 15 of thehost apparatus 3 then returns to standby mode where receipt of a file read request from theclient apparatus 2 is again awaited (S51), and then repeats the same processing (S51 to S58). - Next a description is given of file system mount preparation processing occurring at the file
system transfer apparatus 4.FIG. 15 is a flowchart showing a detailed processing procedure of theCPU 55 of the filesystem transfer apparatus 4 relating to the file system mount preparation processing occurring at this filesystem transfer apparatus 4. - The
CPU 55 of the filesystem transfer apparatus 4 awaits the receipt of a mount request from thehost apparatus 3 in standby mode (S61). When theCPU 55 of the filesystem transfer apparatus 4 receives a mount request from the host apparatus 3 (S61: YES), the file system of the file corresponding to the file read request checks whether or not direct mounting from thesecond storage apparatus 7 to thehost apparatus 3 is possible (S62). - Specifically, the
CPU 55 of the filesystem transfer apparatus 4 has a table (not shown) withinmemory 56 describing in advance whether direct mounting ofhost apparatus 3 such as, for example, storage apparatus of a VDL (Virtual Disk Library) or SATA disc drive etc. can be directly mounted, or whether direct mounting is possible athost apparatus 3 such as a tape library etc. for, for example, all of thesecond storage apparatus 7 that are connected, and checks whether or not direct mounting is possible in accordance with the table. - In the event that it is possible for the file system corresponding to the file read request to be directly mounted at the
host apparatus 3 from the second storage apparatus 7 (S62: YES), theCPU 55 of the filesystem transfer apparatus 4 proceeds to step 64. On the other hand, in the event that it is not possible to directly mount the file system for the file corresponding to the file read request from thesecond storage apparatus 7 to the host apparatus 3 (S62: NO), theCPU 55 of the filesystem transfer apparatus 4 transfers the file system for the file corresponding to the file read request from thelogical volume 112B of thesecond storage apparatus 7 to thelogical volume 112A of the first storage apparatus 6 (S63). - Specifically, the
CPU 55 of the filesystem transfer apparatus 4 sends the file system transfer request of the file system of the file corresponding to the file read request to thefirst storage apparatus 6 and thesecond storage apparatus 7. When a file system transfer request transmitted from the filesystem transfer apparatus 4 is received, theCPU 102B of thesecond storage apparatus 6 sends a file system stored in thelogical volume 112B corresponding to the file system transfer request to thefirst storage apparatus 6. TheCPU 102A of thefirst storage apparatus 6 then stores the file system sent from thesecond storage apparatus 7 in thelogical volume 112A. When transfer of the file system is complete, thefirst storage apparatus 6 sends the file system transfer completion report to the filesystem transfer apparatus 4 and thefirst storage system 6. - When file system transfer completion notification is received from the
first storage apparatus 6, theCPU 55 of the filesystem transfer apparatus 4 updates the file system transfer management table 62 (S64). - Specifically, in the event that, for example, it is possible to directly mount the file system for the file corresponding to the file read request from the
second storage apparatus 7 to thehost apparatus 3, theCPU 55 of the filesystem transfer apparatus 4 changes the file system state 152 from “secondary storage”, “write inhibit” and “offline” to “secondary storage”, “write permit”, and “online” at the mapping record corresponding to the file system identifier 151 of the file transfer management table 62 of the file system. - Further, in the event that, for example, the file system of the file corresponding to the file read request cannot be directly mounted from the
second storage apparatus 7 to thehost apparatus 3, theCPU 55 of the filesystem transfer apparatus 4 changes the file system state 152 from “secondary storage”, “write inhibit”, and “offline” to “primary storage”, “write inhibit”, and “online” at the mapping record corresponding to the file system identifier 151 of the file transfer management table 62 of the file system, changes the original storage apparatus identifier 154 and the original logical volume identifier 155 to the identifiers of thesecond storage apparatus 7 and thelogical volume 112B that stored the transferred file, and changes the currentstorage apparatus identifier 157 and current logical volume identifier 158 to the identifiers of thefirst storage apparatus 6 and thelogical volume 112A that stored the transferred file system. - The
CPU 55 of the filesystem transfer apparatus 4 makes file system mount preparation completion notification by executing thesynchronization management program 82, and sends the file system mount preparation completion notification and the filesystem transfer information 159 for the updated file transfer management table 62 to thehost apparatus 3 corresponding to the current host apparatus identifier 156 of the mapping code (S65). - After this, the
CPU 55 of the filesystem transfer apparatus 4 returns to a standby mode so as to await the receipt of a mount request from the host apparatus 3 (S61), and the same processing is then repeated thereafter (S61 to 565). - Rather than transferring the file system corresponding to the file read request from the
logical volume 112B of thesecond storage apparatus 7 to thelogical volume 112A of the first storage apparatus 6 (S63), theCPU 55 of the filesystem transfer apparatus 4 can also operate so as to duplicate the file system to thelogical volume 112A of thefirst storage apparatus 6. - In this case, the
CPU 55 of the filesystem transfer apparatus 4, for example, stores a table (not shown) housing a file system identifier 151 for the duplicated file system inmemory 56, and stores a file system identifier 153 for the duplicated file system in the table when the file system is duplicated at thelogical volume 112A of thefirst storage apparatus 6. - After this, in the event that the
CPU 55 of the filesystem transfer apparatus 4 transfers the files again from thelogical volume 112A to thelogical volume 112B of thesecond storage apparatus 7 in accordance with the file system transfer processing procedure RT3 shown inFIG. 11 described above (S21), the file system is already stored in thelogical volume 112B of thesecond storage apparatus 7, and the same results as for transferring the system file from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 can be obtained by deleting the file system from thelogical volume 112A of thefirst storage apparatus 6. - Namely, the file system selected as described above is transferred from the
logical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of the second storage apparatus 7 (S21), the table storing the file system identifier 151 of the duplicated file system is searched, and in the event that the file system identifier 151 for the file system selected in the manner described above is stored on the table, theCPU 55 of the filesystem transfer apparatus 4 deletes the file system from thelogical volume 112A of thefirst storage apparatus 6, transfer from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 can be completed, and the speed of file system transfer processing can be increased dramatically. - At this time, at the mapping record corresponding to the file system identifier 151 of the file transfer management table 62 of the transferred file system, for example, the
CPU 55 of the filesystem transfer apparatus 4 changes the file system state 152 from “primary storage”, “write enabled”, and “online” to “secondary storage”, “write inhibited”, and “online”, and switches over the original storage apparatus identifier 154 and the original logical volume identifier 155 with the currentstorage apparatus identifier 157 and the current logical volume identifier 158. - In this way, at the
data management system 1, at the filesystem transfer apparatus 4, the file system is transferred from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 based on the file system transfer management table 61, the file systemtransfer update information 159 of the file system transfer management table 62 of the file systemtransfer management apparatus 4 updated in accordance with the transfer of the file system is sent to thehost apparatus 3, and the filesystem transfer information 159 of the file system transfer management table 23 of theindex database 20 of thehost apparatus 3 is updated (A3). - Therefore, at the
data management system 1, when a file search request is received from theclient apparatus 2, the state where the file system management table 23 is not updated is effectively prevented before it happens, it is possible to search for the file corresponding to the file search request, and as a result, it is possible to read the file requested by theclient apparatus 2. It is therefore possible to provide a data management system with highly convenient file management and low operating costs, having both high availability and reliability. - Further, with the
data management system 1, at the filesystem transfer apparatus 4, it is determined whether or not the file system for the file corresponding to the file read request is capable of being mounted directly from thesecond storage apparatus 7 to thehost apparatus 3. In the event that direct mounting is not possible, the file system for the file corresponding to the file read request is transferred from thelogical volume 112B of thesecond storage apparatus 7 to thelogical volume 112A of thefirst storage apparatus 6 so as to make mounting possible. This means that it is possible to prevent a situation where a file of a file system cannot be read even in the event of transfer from thefirst storage apparatus 6 to thesecond storage apparatus 7 and it is possible to read the file requested by theclient apparatus 2. - Further, with the
data management system 1, at the filesystem transfer apparatus 4, by duplicating the file system of the file corresponding to the file read request from thelogical volume 112B of thesecond storage apparatus 7 to thelogical volume 112A of thefirst storage apparatus 6, when the file system is then again transferred from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7, it is possible to obtain the same results as for transferring from thelogical volume 112A of thefirst storage apparatus 6 to thelogical volume 112B of thesecond storage apparatus 7 by deleting the file system from thelogical volume 112A of thefirst storage apparatus 6. - In this embodiment, a description is given of the case where each
host apparatus 3 implements an index producing function and an index updating management function, but the present invention is by no means limited in this respect, and, for example, as shown inFIG. 16 , anindex server 171 may be provided, with theagent 32 then being divided into anagent 172 having a file search function and a file read function and anagent 173 having an index producing function and an index update management function. An index producing function and index update management function implemented with eachhost apparatus 3 distributed may then be implemented focused on theindex server 171. - Further, in this embodiment, a description is given of the case where the file
system transfer apparatus 4 is provided, and the filesystem transfer apparatus 4 implements each of the functions relating to the file system transfer described above, but the present invention is by no means limited in this respect, and, for example, as shown inFIG. 16 , it is also possible for thefirst storage apparatus 6 to have the function relating to transfer of the file systems in the possession of the filesystem transfer apparatus 4, and to implement each of the functions relating to file system transfer the filesystem transfer apparatus 4 is in possession of at thefirst storage apparatus 6. - The present invention is therefore broadly applicable to data management systems that search files based on requests from client apparatus and read out the files from client apparatus.
- While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as disclosed herein. Accordingly, the scope of the invention should be limited only by the attached claims.
Claims (16)
1. A data management system with a host apparatus receiving requests from client apparatus, a first storage apparatus having a first volume for storing a file system composed of a set of files, and a second storage apparatus having a second volume for storing the file system sent from the first storage apparatus, comprising:
a file system transfer designation section for transferring the file system matching with file system migration rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system migration rules;
a file system storage information manager for updating storage information of the file system in accordance with transfer of the file system by the file system transfer designation section, and transmitting the updated file system storage information; and
a search information manager for updating search information for searching the files based on a file search request from the client apparatus using the file system storage information sent by the file system storage information manager.
2. The data management system according to claim 1 , further comprising:
a mount determination section for determining whether or not a file system for a file corresponding to a file read request from the client apparatus is capable of being directly mounted from the second storage apparatus to the host apparatus,
wherein the file system transfer section:
transfers the file system from the second volume of the second storage apparatus to the first volume of the first storage apparatus that is capable of mounting the file system at the host apparatus in the event that the mount determination section determines that the file system can not be directly mounted.
3. The data management system according to claim 2 , the host apparatus further comprising:
a mount management section for managing mounting and un-mounting of the file system;
and the mount management section;
directly mounting the file system for the file corresponding to the file read request from the client apparatus from the second storage apparatus in the event that the mount determination section determines that direct mounting of the file system is possible.
4. The data management system according to claim 2 , wherein the file system transfer section:
duplicates the file system from the second volume of the second storage apparatus to the first volume of the first storage apparatus that is capable of mounting the file system at the host apparatus in the event that the mount determination section determines that the file system can not be directly mounted.
5. The data management system according to claim 1 , wherein the search information comprises;
information containing host apparatus, file system, attributes and a keyword list every file, information containing file and attributes every keyword, and information containing the host apparatus, storage apparatus and volume for before transfer, the host apparatus, storage apparatus and volume for after transfer, and the storage state of the file system, every file system.
6. The data management system according to claim 1 , the storage information further comprising;
information containing the host apparatus, storage apparatus and volume for before transfer, the host apparatus, storage apparatus and volume for after transfer, and the storage state of the file system, every file system.
7. The data management system according to claim 1 , wherein the file system transfer designation section and the file system storage information manager
are provided at the file system transfer apparatus,
and the search information manager comprises:
a plurality of the host apparatus.
8. The data management system according to claim 1 , wherein the search information manager is provided at
an index server, and
the file system transfer designation section and the file system storage information manager
are provided at the first storage apparatus.
9. A data management method for a data management system with a host apparatus receiving requests from client apparatus, first storage apparatus having a first volume for storing a file system composed of a set of files, and a second storage apparatus having a second volume for storing the file system sent from the first storage apparatus, comprising:
a first step of transferring the file system matching with file system migration rules from the first volume of the first storage apparatus to the second volume of the second storage apparatus based on the first file system migration rules;
a second step of updating storage information for the file system in accordance with transfer of the file system of the first step, and transmitting the stored information of the updated file system; and
a third step of updating the search information for searching the files based on file search requests from the client apparatus using file system storage information sent in the second step.
10. The data management method according to claim 9 , further comprising:
a fourth step of determining whether or not a file system for a file corresponding to a file read request from the client apparatus is capable of being directly mounted from the second storage apparatus to the host apparatus; and
a fifth step of transferring the file system from the second volume of the second storage apparatus to the first volume of the first storage apparatus that is capable of mounting the file system at the host apparatus, in the event that it is determined in the fourth step that the file system cannot be directly mounted.
11. The data management method according to claim 10 , further comprising:
a sixth step of managing mounting and un-mounting of the file system;
wherein, in the sixth step,
a file system for a file corresponding to a file read request from the client apparatus is directly mounted from the second storage apparatus in the event that it is determined that the file system can be directly mounted in the fourth step.
12. The data management method according to claim 10 , wherein in the fifth step: the file system is transferred from the second volume of the second storage apparatus to the first volume of the first storage apparatus that is capable of mounting the file system at the host apparatus, in the event that it is determined in the fourth step that the file system cannot be directly mounted.
13. The data management method according to claim 9 , wherein the search information comprises:
information containing host apparatus, file system, attributes and a keyword list every file, information containing file and attributes every keyword, and information containing the host apparatus, storage apparatus and volume for before transfer, the host apparatus, storage apparatus and volume for after transfer, and the storage state of the file system, every file system.
14. The data management method according to claim 9 , the storage information further comprising:
information containing the host apparatus, storage apparatus and volume for before transfer, the host apparatus, storage apparatus and volume for after transfer, and the storage state of the file system, every file system.
15. The data management method according to claim 9 , wherein the first step and the second step:
are carried out at the file system transfer apparatus,
and the third step:
is carried out at the corresponding host apparatus of the plurality of host apparatus.
16. The data management method according to claim 9 , wherein the first step and the second step:
are carried out at the first storage apparatus,
and the third step:
is carried out focusing on the index server.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2006-247896 | 2006-09-13 | ||
JP2006247896A JP4939152B2 (en) | 2006-09-13 | 2006-09-13 | Data management system and data management method |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080065587A1 true US20080065587A1 (en) | 2008-03-13 |
Family
ID=39170980
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/604,684 Expired - Fee Related US7730071B2 (en) | 2006-09-13 | 2006-11-28 | Data management system and data management method |
US11/605,596 Abandoned US20080065587A1 (en) | 2006-09-13 | 2006-11-28 | Data management system and data management method |
US12/775,549 Expired - Fee Related US8364645B2 (en) | 2006-09-13 | 2010-05-07 | Data management system and data management method |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/604,684 Expired - Fee Related US7730071B2 (en) | 2006-09-13 | 2006-11-28 | Data management system and data management method |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/775,549 Expired - Fee Related US8364645B2 (en) | 2006-09-13 | 2010-05-07 | Data management system and data management method |
Country Status (3)
Country | Link |
---|---|
US (3) | US7730071B2 (en) |
EP (1) | EP1942421A1 (en) |
JP (1) | JP4939152B2 (en) |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100049768A1 (en) * | 2006-07-20 | 2010-02-25 | Robert James C | Automatic management of digital archives, in particular of audio and/or video files |
EP2383667A1 (en) * | 2010-04-23 | 2011-11-02 | Alexander Vitas Ceponkus | Mobile wireless communications device to perform local and remote search and associated methods |
CN102411546A (en) * | 2011-10-18 | 2012-04-11 | 深圳市共进电子股份有限公司 | Method for implementing connected communication of 3G data cards in at least two operation modes |
US20120254257A1 (en) * | 2011-03-31 | 2012-10-04 | Emc Corporation | Resource efficient scale-out file systems |
US20130117225A1 (en) * | 2011-11-03 | 2013-05-09 | Michael W. Dalton | Distributed storage medium management for heterogeneous storage media in high availability clusters |
US20140106672A1 (en) * | 2012-10-04 | 2014-04-17 | Lg Electronics Inc. | Mobile terminal, home appliance and method for operating the same |
US20150032785A1 (en) * | 2013-07-25 | 2015-01-29 | Fujitsu Limited | Non-transitory computer-readable media storing file management program, file management apparatus, and file management method |
US20160019263A1 (en) * | 2012-12-28 | 2016-01-21 | Facebook, Inc. | Saved queries in a social networking system |
US9619474B2 (en) | 2011-03-31 | 2017-04-11 | EMC IP Holding Company LLC | Time-based data partitioning |
US11144251B2 (en) * | 2018-10-17 | 2021-10-12 | International Business Machines Corporation | Providing a global unique identifier for a storage volume |
Families Citing this family (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080228900A1 (en) * | 2007-03-14 | 2008-09-18 | Disney Enterprises, Inc. | Method and system for facilitating the transfer of a computer file |
KR101451562B1 (en) * | 2007-08-10 | 2014-10-24 | 삼성전자주식회사 | Method and apparatus for data storage in mobile communication system |
US20090187551A1 (en) * | 2008-01-17 | 2009-07-23 | Oracle International Corporation | Search results when searching for records of a business object |
US8745079B2 (en) | 2008-07-29 | 2014-06-03 | Oracle International Corporation | Reducing lag time when searching a repository using a keyword search |
US20100036858A1 (en) * | 2008-08-06 | 2010-02-11 | Microsoft Corporation | Meta file system - transparently managing storage using multiple file systems |
US8015343B2 (en) * | 2008-08-08 | 2011-09-06 | Amazon Technologies, Inc. | Providing executing programs with reliable access to non-local block data storage |
US8239389B2 (en) * | 2008-09-29 | 2012-08-07 | International Business Machines Corporation | Persisting external index data in a database |
WO2010050288A1 (en) * | 2008-10-30 | 2010-05-06 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Server system, server device, program, and method |
US20100121828A1 (en) * | 2008-11-11 | 2010-05-13 | You Wang | Resource constraint aware network file system |
US10496608B2 (en) * | 2009-10-28 | 2019-12-03 | Sandisk Il Ltd. | Synchronizing changes in a file system which are initiated by a storage device and a host device |
US20120124009A1 (en) * | 2010-11-12 | 2012-05-17 | Internation Business Machines Corporation | Automatic expiration of data in file systems under certain scenarios |
US9213709B2 (en) * | 2012-08-08 | 2015-12-15 | Amazon Technologies, Inc. | Archival data identification |
US9767098B2 (en) | 2012-08-08 | 2017-09-19 | Amazon Technologies, Inc. | Archival data storage system |
US9251097B1 (en) | 2011-03-22 | 2016-02-02 | Amazon Technologies, Inc. | Redundant key management |
US9563681B1 (en) | 2012-08-08 | 2017-02-07 | Amazon Technologies, Inc. | Archival data flow management |
US8577934B2 (en) * | 2011-03-24 | 2013-11-05 | Spectra Logic Corporation | System and method for a storage system response with data migration |
WO2012137262A1 (en) * | 2011-04-08 | 2012-10-11 | Hitachi, Ltd. | Information processing system and data processing method |
US20130246553A1 (en) * | 2012-03-16 | 2013-09-19 | Empire Technology Development Llc | Data migration |
US10120579B1 (en) | 2012-08-08 | 2018-11-06 | Amazon Technologies, Inc. | Data storage management for sequentially written media |
US9652487B1 (en) | 2012-08-08 | 2017-05-16 | Amazon Technologies, Inc. | Programmable checksum calculations on data storage devices |
US9904788B2 (en) | 2012-08-08 | 2018-02-27 | Amazon Technologies, Inc. | Redundant key management |
US8959067B1 (en) | 2012-08-08 | 2015-02-17 | Amazon Technologies, Inc. | Data storage inventory indexing |
US9354683B2 (en) | 2012-08-08 | 2016-05-31 | Amazon Technologies, Inc. | Data storage power management |
US9779035B1 (en) | 2012-08-08 | 2017-10-03 | Amazon Technologies, Inc. | Log-based data storage on sequentially written media |
US9225675B2 (en) | 2012-08-08 | 2015-12-29 | Amazon Technologies, Inc. | Data storage application programming interface |
US8805793B2 (en) | 2012-08-08 | 2014-08-12 | Amazon Technologies, Inc. | Data storage integrity validation |
US9830111B1 (en) | 2012-08-08 | 2017-11-28 | Amazon Technologies, Inc. | Data storage space management |
US10169348B2 (en) * | 2012-08-23 | 2019-01-01 | Red Hat, Inc. | Using a file path to determine file locality for applications |
US10558581B1 (en) | 2013-02-19 | 2020-02-11 | Amazon Technologies, Inc. | Systems and techniques for data recovery in a keymapless data storage system |
JP2014229166A (en) | 2013-05-24 | 2014-12-08 | 富士通株式会社 | Communication system, communication method, business management system, management device, program, relay device, and data position management device |
US9449018B1 (en) * | 2013-11-25 | 2016-09-20 | Google Inc. | File operation task optimization |
US11386060B1 (en) | 2015-09-23 | 2022-07-12 | Amazon Technologies, Inc. | Techniques for verifiably processing data in distributed computing systems |
WO2019061132A1 (en) * | 2017-09-28 | 2019-04-04 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration, and application thereof |
CN107943908A (en) * | 2017-11-17 | 2018-04-20 | 郑州云海信息技术有限公司 | A kind of document acquisition methods and device |
US11561935B2 (en) | 2021-06-17 | 2023-01-24 | Netapp, Inc. | Methods for ensuring correctness of file system analytics and devices thereof |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05313971A (en) * | 1992-05-13 | 1993-11-26 | Mitsubishi Electric Corp | Key word managing system in relational data base |
US5495607A (en) * | 1993-11-15 | 1996-02-27 | Conner Peripherals, Inc. | Network management system having virtual catalog overview of files distributively stored across network domain |
US5566331A (en) * | 1994-01-24 | 1996-10-15 | University Corporation For Atmospheric Research | Mass storage system for file-systems |
US5537585A (en) | 1994-02-25 | 1996-07-16 | Avail Systems Corporation | Data storage management for network interconnected processors |
US6353878B1 (en) * | 1998-08-13 | 2002-03-05 | Emc Corporation | Remote control of backup media in a secondary storage subsystem through access to a primary storage subsystem |
US6668264B1 (en) * | 2001-04-03 | 2003-12-23 | Network Appliance, Inc. | Resynchronization of a target volume with a source volume |
WO2003025795A1 (en) * | 2001-08-31 | 2003-03-27 | Arkivio, Inc. | Techniques for restoring data based on contents and attributes of the data |
JP4229626B2 (en) * | 2002-03-26 | 2009-02-25 | 富士通株式会社 | File management system |
JP4183443B2 (en) | 2002-05-27 | 2008-11-19 | 株式会社日立製作所 | Data relocation method and apparatus |
JP4124331B2 (en) * | 2002-09-17 | 2008-07-23 | 株式会社日立製作所 | Virtual volume creation and management method for DBMS |
EP1570357A1 (en) * | 2002-12-02 | 2005-09-07 | Arkivio, Inc. | Data recovery techniques in storage systems |
US7590667B2 (en) * | 2003-01-30 | 2009-09-15 | Hitachi, Ltd. | File replication method for distributed file systems |
US7346664B2 (en) | 2003-04-24 | 2008-03-18 | Neopath Networks, Inc. | Transparent file migration using namespace replication |
US7546324B2 (en) * | 2003-11-13 | 2009-06-09 | Commvault Systems, Inc. | Systems and methods for performing storage operations using network attached storage |
JP2005165485A (en) * | 2003-12-01 | 2005-06-23 | Sony Corp | File management device, storage management system, system management method, program, and recording medium |
US8127095B1 (en) * | 2003-12-31 | 2012-02-28 | Symantec Operating Corporation | Restore mechanism for a multi-class file system |
JP2005276017A (en) * | 2004-03-26 | 2005-10-06 | Hitachi Ltd | Storage system |
US7441096B2 (en) * | 2004-07-07 | 2008-10-21 | Hitachi, Ltd. | Hierarchical storage management system |
JP2006247896A (en) | 2005-03-08 | 2006-09-21 | Sumitomo Bakelite Co Ltd | Plastic laminating method, joining device and plastic product manufactured using plastic laminating method |
-
2006
- 2006-09-13 JP JP2006247896A patent/JP4939152B2/en not_active Expired - Fee Related
- 2006-11-28 US US11/604,684 patent/US7730071B2/en not_active Expired - Fee Related
- 2006-11-28 US US11/605,596 patent/US20080065587A1/en not_active Abandoned
-
2007
- 2007-04-05 EP EP07251515A patent/EP1942421A1/en not_active Withdrawn
-
2010
- 2010-05-07 US US12/775,549 patent/US8364645B2/en not_active Expired - Fee Related
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100049768A1 (en) * | 2006-07-20 | 2010-02-25 | Robert James C | Automatic management of digital archives, in particular of audio and/or video files |
US9031965B2 (en) * | 2006-07-20 | 2015-05-12 | S.I. SV. EL. S.p.A. | Automatic management of digital archives, in particular of audio and/or video files |
EP2383667A1 (en) * | 2010-04-23 | 2011-11-02 | Alexander Vitas Ceponkus | Mobile wireless communications device to perform local and remote search and associated methods |
US8744415B2 (en) | 2010-04-23 | 2014-06-03 | Blackberry Limited | Mobile wireless communications device to perform local and remote search and associated methods |
US20120254257A1 (en) * | 2011-03-31 | 2012-10-04 | Emc Corporation | Resource efficient scale-out file systems |
US10664453B1 (en) | 2011-03-31 | 2020-05-26 | EMC IP Holding Company LLC | Time-based data partitioning |
CN103562914A (en) * | 2011-03-31 | 2014-02-05 | 伊姆西公司 | Resource efficient scale-out file systems |
US9916258B2 (en) * | 2011-03-31 | 2018-03-13 | EMC IP Holding Company LLC | Resource efficient scale-out file systems |
US9619474B2 (en) | 2011-03-31 | 2017-04-11 | EMC IP Holding Company LLC | Time-based data partitioning |
CN102411546A (en) * | 2011-10-18 | 2012-04-11 | 深圳市共进电子股份有限公司 | Method for implementing connected communication of 3G data cards in at least two operation modes |
US9063939B2 (en) * | 2011-11-03 | 2015-06-23 | Zettaset, Inc. | Distributed storage medium management for heterogeneous storage media in high availability clusters |
US20130117225A1 (en) * | 2011-11-03 | 2013-05-09 | Michael W. Dalton | Distributed storage medium management for heterogeneous storage media in high availability clusters |
US9407330B2 (en) * | 2012-10-04 | 2016-08-02 | Lg Electronics Inc. | Mobile terminal, home appliance and method for operating the same |
US20140106672A1 (en) * | 2012-10-04 | 2014-04-17 | Lg Electronics Inc. | Mobile terminal, home appliance and method for operating the same |
US20160019263A1 (en) * | 2012-12-28 | 2016-01-21 | Facebook, Inc. | Saved queries in a social networking system |
US9846726B2 (en) * | 2012-12-28 | 2017-12-19 | Facebook, Inc. | Saved queries in a social networking system |
US20150032785A1 (en) * | 2013-07-25 | 2015-01-29 | Fujitsu Limited | Non-transitory computer-readable media storing file management program, file management apparatus, and file management method |
US9690796B2 (en) * | 2013-07-25 | 2017-06-27 | Fujitsu Limited | Non-transitory computer-readable media storing file management program, file management apparatus, and file management method |
US11144251B2 (en) * | 2018-10-17 | 2021-10-12 | International Business Machines Corporation | Providing a global unique identifier for a storage volume |
US11797177B2 (en) | 2018-10-17 | 2023-10-24 | International Business Machines Corporation | Providing a global unique identifier for a storage volume |
Also Published As
Publication number | Publication date |
---|---|
JP4939152B2 (en) | 2012-05-23 |
US8364645B2 (en) | 2013-01-29 |
EP1942421A1 (en) | 2008-07-09 |
US20080065707A1 (en) | 2008-03-13 |
JP2008071035A (en) | 2008-03-27 |
US7730071B2 (en) | 2010-06-01 |
US20100241656A1 (en) | 2010-09-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8364645B2 (en) | Data management system and data management method | |
US20200012568A1 (en) | Scalable log-based continuous data protection for distributed databases | |
US7974950B2 (en) | Applying a policy criteria to files in a backup image | |
US10853182B1 (en) | Scalable log-based secondary indexes for non-relational databases | |
US8078819B2 (en) | Arrangements for managing metadata of an integrated logical unit including differing types of storage media | |
US20200019516A1 (en) | Primary Data Storage System with Staged Deduplication | |
US7631151B2 (en) | Systems and methods for classifying and transferring information in a storage network | |
US7366846B2 (en) | Redirection of storage access requests | |
JP4799936B2 (en) | Conditional snapshot acquisition method and system | |
US20070038822A1 (en) | Copying storage units and related metadata to storage | |
US20090132621A1 (en) | Selecting storage location for file storage based on storage longevity and speed | |
US7860909B2 (en) | Search engine system using snapshot function of storage system | |
US20090070879A1 (en) | Computer system and method for scanning computer virus | |
CN103597440A (en) | Method for creating clone file, and file system adopting the same | |
CN111356996A (en) | Versioning verification of data transfers between heterogeneous data stores | |
EP1636690B1 (en) | Managing a relationship between one target volume and one source volume | |
US7882086B1 (en) | Method and system for portset data management | |
US10452644B2 (en) | Computer system, method for verifying data, and computer | |
US10133505B1 (en) | Cooperative host and data storage system services for compression and encryption | |
US20070174360A1 (en) | Storage system embedding database | |
US20060059188A1 (en) | Operation environment associating data migration method | |
US7516133B2 (en) | Method and apparatus for file replication with a common format | |
US20080244055A1 (en) | Computer that manages devices | |
US7831623B2 (en) | Method, system, and article of manufacture for storing device information | |
US20070124366A1 (en) | Storage control method for managing access environment enabling host to access data |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HITACHI, LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:IWASAKI, MASAAKI;KUMAZAWA, KIYOTAKE;REEL/FRAME:018650/0471 Effective date: 20061113 |
|
STCB | Information on status: application discontinuation |
Free format text: EXPRESSLY ABANDONED -- DURING EXAMINATION |