WO2009145928A1 - Ranking and prioritizing point in time snapshots - Google Patents
Ranking and prioritizing point in time snapshots Download PDFInfo
- Publication number
- WO2009145928A1 WO2009145928A1 PCT/US2009/030293 US2009030293W WO2009145928A1 WO 2009145928 A1 WO2009145928 A1 WO 2009145928A1 US 2009030293 W US2009030293 W US 2009030293W WO 2009145928 A1 WO2009145928 A1 WO 2009145928A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- snapshots
- data
- snapshot
- storage
- manager
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/84—Using snapshots, i.e. a logical point-in-time copy of the data
Definitions
- This invention relates to the field of computerized data storage systems management. More particularly, this invention relates to snapshot management for storage area networks.
- SANs Storage area networks
- Storage area networks are networks of computer data storage devices that are attached to servers in such a way that they appear to each and every server to be a locally attached storage device, instead of a remotely attached storage device.
- the topography of a SAN is such that data travels according to one or more extremely high bandwidth interface, such as Inf ⁇ niband, Fiber Channel, Ethernet, or SAS, and protocols such as SCSI.
- SANs are typically managed by a remote storage manager program, such as SANtricity, provided by LSI Corporation of Milpitas, California.
- Storage manager software such as SANtricity optionally offer a volume snapshot feature that provides a point-in-time image of a SAN storage volume.
- the snapshot is the logical equivalent of a complete physical copy, but is created much more quickly and requires less disk space. Snapshot volumes appear and function as standard storage volumes. They are host-addressable and can be read, written to, or copied.
- Volume snapshot enables nonproduction, backup, or analysis servers to access an up-to-date copy of production data while the production data remains online and user-accessible.
- Volume snapshot is designed for users whose data availability cannot be disrupted for routine management functions.
- Volume snapshot supports round-the- clock processing as it stages data for operations such as backup, data mining/analysis, and work distribution. In other words, some operations that do not tolerate continuous data access, such as a backup operation, can be performed on snapshot data instead of live data. Thus, the process of taking a snapshot is not the same as merely making a backup.
- snapshot volumes have the same high-availability characteristics as standard storage volumes, such as RAID protection and redundant path failover.
- a storage area network system having a data storage means for storing computer data, a storage manager routine running on a client, the storage manager routine having functional elements for directing snapshots to be taken of the computer data on the data storage means, and a snapshot ranking manager for determining characteristics of the snapshots, and for selectively deleting given ones of the snapshots based at least in part on the characteristics of the snapshots.
- the characteristics of the snapshots might include the type of application that uses the data in the logical volume from which the snapshots were taken, or mission critical aspects of the data.
- the embodiments of the present invention provide a system in which the snapshots are ranked and prioritized based on the importance and critical aspect of any changes that have been made, and automatically destroyed in real-time when one or more of the snapshots are no longer needed, to conserve disk space and enhance performance and storage space utilization.
- the system maintains the snapshots until they are needed, and then destroys them once any necessary backups and later point- in-time snapshots have been acquired.
- the system internally generates snapshots in real-time on specific deterministic intervals, such as user-selectable intervals, upon new writes being propagated to a volume.
- the snapshots are then ranked by their order of importance, which is derived by the degree of change in data content from the previous snapshot
- Ranking snapshots according to their relative importance is performed according to criteria input to or generated by the snapshot manager program, which in one embodiment is implemented in the controller firmware for the SAN. In one embodiment, the mechanism and the implementation is accomplished in the manner next described.
- the user selects the time interval for the creation of the snapshots. If no interval is selected, then a default setting is implemented, which creates snapshots upon every minute or upon occurrence of a read or write operation to a volume within the storage array. The snapshots are compared to each other (a previous snapshot is compared to the next snapshot), and a given snapshot is retained if there are data content changes that meet or exceed the specified criteria.
- any change to the data content is sufficient for a snapshot to be retained.
- the criteria might include a minimum amount of the data that must change, or certain types of changes, or certain data that changes.
- the comparison mechanism of some embodiments is similar to the 'diff command used in UNIX operating systems. If there are no differences in the data content (or the snapshots otherwise do not meet the criteria), then the snapshot that was created during the earlier point-in-time is destroyed, and the snapshot that was created during the later point-in-time is retained.
- a ranking mechanism is employed to rank them by relative importance based on the criteria provided to, resident in, or generated by the snapshot manager, which in some embodiments includes the past state of the application when the snapshots were created.
- the snapshot manager then provides location specific information and faster access to the higher ranked snapshots.
- Lower ranked snapshots are eventually destroyed based on criteria such as a user-selectable snapshot purging interval for the lower ranked snapshots.
- the number of snapshots that are to be kept at any given point in time is a user-selectable number. This number only applies to snapshots that have data content differences, as taken from the same logical volume. All of the snapshots that are retained have priority ranking that in some embodiments is based on the type of application that uses the logical volume, and mission critical aspects of the data. For example, if the snapshot is taken from a logical volume that is hosting a mirror volume's repository, this snapshot is ranked higher than a snapshot taken from a logical volume that is being used as a mere file sharing volume. All ranked snapshots also have location specific information in the realm of storage array for a shorter seek time. In some embodiments, those snapshots with lower priority rankings might be deleted if storage space falls below a defined amount, and higher priority snapshots need to be saved.
- the system provides many benefits and advantages. For example, the system provides faster access to higher ranked snapshots, while lower ranked snapshots can be destroyed to conserve disk space and system performance. Prior point-in-time snapshots can be automatically destroyed when there is no data content change when compared to later point-in-time snapshots, or when other criteria are met.
- the system also provides for user-selectable snapshot generation and purging intervals.
- the embodiments of the system can guarantee data integrity because of the continuous snapshot generation in real-time, which in some embodiments is based on new reads or writes to the volumes.
- the system 10 depicts a portion of a network, including an I/O attached server 12 that directs data storage and retrieval in a SAN, connected to a data storage system 20, such as by network 14, such as a Fiberchannel, Ethernet, or SAS connection.
- a storage manager client computer 16 is also attached to the network and the storage system 20, such as by an Ethernet network connection 18.
- the data storage system 20 is functionally represented by the data volume 24, which is connected to the data storage system 20, such as by network 22, such as a Fiberchannel, Ethernet, or SAS connection. Under the control of the storage manager 16, a snapshot group of snapshots 26a, 26b, and 26c is taken of the data volume 24 at desired intervals.
- a snapshot ranking manager 30 is also connected to the data storage system 20, such as by an Ethernet connection 28, and receives snapshot ranking information from the storage manager 16.
- the snapshot ranking manager interrogates the snapshots 26, to determine the characteristics of each, such as degree of difference between the data sets represented in each snapshot 26. If any difference exists, or if a desired level or type of difference exists, then the snapshot ranking manager 30 can delete or otherwise destroy older snapshots 26c, for example, and only retain newer snapshots 26a, for example.
- snapshot ranking manager 30 By using the snapshot ranking manager 30 in this manner, only the snapshots 26 that are deemed sufficiently important to the system users or administrators are retained, and the storage space used for the deleted snapshots 26 is released to the storage system, and is available for new data to overwrite. Alternately, if the snapshots 26 are written in a dedicated location of the data storage system, then the space can be released for additional snapshots 26 to be generated.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN200980120559.5A CN102047234B (en) | 2008-05-31 | 2009-01-07 | Ranking and prioritizing point in time snapshots |
JP2011511657A JP5107462B2 (en) | 2008-05-31 | 2009-01-07 | A system for ranking and prioritizing points in time snapshots |
KR1020107026869A KR101206556B1 (en) | 2008-05-31 | 2009-01-07 | Ranking and prioritizing point in time snapshots |
EP09755291.3A EP2286344A4 (en) | 2008-05-31 | 2009-01-07 | Ranking and prioritizing point in time snapshots |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/131,049 | 2008-05-31 | ||
US12/131,049 US8171246B2 (en) | 2008-05-31 | 2008-05-31 | Ranking and prioritizing point in time snapshots |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2009145928A1 true WO2009145928A1 (en) | 2009-12-03 |
Family
ID=41377453
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2009/030293 WO2009145928A1 (en) | 2008-05-31 | 2009-01-07 | Ranking and prioritizing point in time snapshots |
Country Status (7)
Country | Link |
---|---|
US (1) | US8171246B2 (en) |
EP (1) | EP2286344A4 (en) |
JP (1) | JP5107462B2 (en) |
KR (1) | KR101206556B1 (en) |
CN (1) | CN102047234B (en) |
TW (1) | TWI421712B (en) |
WO (1) | WO2009145928A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2450784A3 (en) * | 2010-11-08 | 2013-07-10 | LSI Corporation | Latency reduction associated with a response to a request in a storage system |
EP2965207A4 (en) * | 2013-03-06 | 2016-10-26 | Dell Products Lp | System and method for managing storage system snapshots |
Families Citing this family (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102096613B (en) * | 2010-12-30 | 2012-09-05 | 上海爱数软件有限公司 | Method and device for generating snapshot |
JP2013073557A (en) * | 2011-09-29 | 2013-04-22 | Hitachi Solutions Ltd | Information search system, search server and program |
US9116811B1 (en) * | 2012-06-30 | 2015-08-25 | Emc Corporation | System and method for cache management |
CN103914389B (en) * | 2012-12-31 | 2018-06-15 | 伊姆西公司 | For managing the method and apparatus of storage space |
US9864760B1 (en) * | 2013-12-05 | 2018-01-09 | EMC IP Holding Company LLC | Method and system for concurrently backing up data streams based on backup time estimates |
US9665307B1 (en) * | 2013-12-19 | 2017-05-30 | EMC IP Holding Company LLC | Incremental continuous data protection |
US10140303B1 (en) * | 2014-08-22 | 2018-11-27 | Nexgen Storage, Inc. | Application aware snapshots |
US9092376B1 (en) | 2014-08-29 | 2015-07-28 | Nimble Storage, Inc. | Methods and systems for ordering virtual machine snapshots |
US9778990B2 (en) | 2014-10-08 | 2017-10-03 | Hewlett Packard Enterprise Development Lp | Methods and systems for concurrently taking snapshots of a plurality of virtual machines |
US9727252B2 (en) | 2014-11-13 | 2017-08-08 | Hewlett Packard Enterprise Development Lp | Methods and systems for optimal snapshot distribution within a protection schedule |
US11080242B1 (en) * | 2016-03-30 | 2021-08-03 | EMC IP Holding Company LLC | Multi copy journal consolidation |
US11010064B2 (en) * | 2017-02-15 | 2021-05-18 | Amazon Technologies, Inc. | Data system with flush views |
CN107577798B (en) * | 2017-09-20 | 2021-01-22 | 苏州浪潮智能科技有限公司 | Snapshot creating method and device and computer readable storage medium |
CN107918650A (en) * | 2017-11-14 | 2018-04-17 | 郑州云海信息技术有限公司 | A kind of snapshot creation method, device, system and computer-readable recording medium |
CN111124745B (en) * | 2018-10-30 | 2023-08-18 | 伊姆西Ip控股有限责任公司 | Method, apparatus, and computer readable storage medium for managing a storage system |
US11151090B2 (en) * | 2019-10-17 | 2021-10-19 | EMC IP Holding Company LLC | Snapshot management based on data churn delta |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040250033A1 (en) * | 2002-10-07 | 2004-12-09 | Anand Prahlad | System and method for managing stored data |
US20060069861A1 (en) * | 2004-09-28 | 2006-03-30 | Takashi Amano | Method and apparatus for storage pooling and provisioning for journal based strorage and recovery |
US20060265568A1 (en) * | 2003-05-16 | 2006-11-23 | Burton David A | Methods and systems of cache memory management and snapshot operations |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020073175A1 (en) * | 2000-12-11 | 2002-06-13 | Dekoning Rodney A. | Updating world wide web pages in a storage area network environment |
DE60232165D1 (en) * | 2001-09-28 | 2009-06-10 | Commvault Systems Inc | SYSTEM AND METHOD FOR PRODUCING AND MANAGING FAST RECOVERY VOLUME |
US6999999B2 (en) * | 2001-12-28 | 2006-02-14 | Hewlett-Packard Development Company, L.P. | System and method for securing fiber channel drive access in a partitioned data library |
US7475098B2 (en) * | 2002-03-19 | 2009-01-06 | Network Appliance, Inc. | System and method for managing a plurality of snapshots |
US7379954B2 (en) * | 2003-07-08 | 2008-05-27 | Pillar Data Systems, Inc. | Management of file system snapshots |
CN100385853C (en) * | 2004-04-01 | 2008-04-30 | 顺昱科技股份有限公司 | Device for integrating storage interface and network function and operation method thereof |
US20060004818A1 (en) * | 2004-07-01 | 2006-01-05 | Claudatos Christopher H | Efficient information management |
US7493349B2 (en) * | 2004-07-09 | 2009-02-17 | International Business Machines Corporation | Method and system for performing a backup by querying a backup infrastructure |
US7305530B2 (en) * | 2004-11-02 | 2007-12-04 | Hewlett-Packard Development Company, L.P. | Copy operations in storage networks |
JP2006302015A (en) * | 2005-04-21 | 2006-11-02 | Hitachi Ltd | Storage system and data management method |
US7933936B2 (en) * | 2005-06-10 | 2011-04-26 | Network Appliance, Inc. | Method and system for automatic management of storage space |
JP4799936B2 (en) * | 2005-07-11 | 2011-10-26 | 株式会社日立製作所 | Conditional snapshot acquisition method and system |
TW200723102A (en) * | 2005-12-02 | 2007-06-16 | Inventec Corp | Snapshot management method for a logical diskette manager |
JP4796854B2 (en) * | 2006-01-30 | 2011-10-19 | 株式会社日立製作所 | Measures against data overflow of intermediate volume in differential remote copy |
US8095751B2 (en) * | 2006-02-28 | 2012-01-10 | International Business Machines Corporation | Managing set of target storage volumes for snapshot and tape backups |
US7882071B2 (en) * | 2006-08-18 | 2011-02-01 | Isilon Systems, Inc. | Systems and methods for a snapshot of data |
-
2008
- 2008-05-31 US US12/131,049 patent/US8171246B2/en not_active Expired - Fee Related
-
2009
- 2009-01-07 WO PCT/US2009/030293 patent/WO2009145928A1/en active Application Filing
- 2009-01-07 CN CN200980120559.5A patent/CN102047234B/en not_active Expired - Fee Related
- 2009-01-07 JP JP2011511657A patent/JP5107462B2/en not_active Expired - Fee Related
- 2009-01-07 EP EP09755291.3A patent/EP2286344A4/en not_active Withdrawn
- 2009-01-07 KR KR1020107026869A patent/KR101206556B1/en not_active IP Right Cessation
- 2009-01-21 TW TW098102353A patent/TWI421712B/en not_active IP Right Cessation
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040250033A1 (en) * | 2002-10-07 | 2004-12-09 | Anand Prahlad | System and method for managing stored data |
US20060265568A1 (en) * | 2003-05-16 | 2006-11-23 | Burton David A | Methods and systems of cache memory management and snapshot operations |
US20060069861A1 (en) * | 2004-09-28 | 2006-03-30 | Takashi Amano | Method and apparatus for storage pooling and provisioning for journal based strorage and recovery |
Non-Patent Citations (1)
Title |
---|
See also references of EP2286344A4 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2450784A3 (en) * | 2010-11-08 | 2013-07-10 | LSI Corporation | Latency reduction associated with a response to a request in a storage system |
EP2965207A4 (en) * | 2013-03-06 | 2016-10-26 | Dell Products Lp | System and method for managing storage system snapshots |
US10346079B2 (en) | 2013-03-06 | 2019-07-09 | Dell Products, L.P. | System and method for managing storage system snapshots |
Also Published As
Publication number | Publication date |
---|---|
US8171246B2 (en) | 2012-05-01 |
KR101206556B1 (en) | 2012-11-29 |
TW200951745A (en) | 2009-12-16 |
CN102047234A (en) | 2011-05-04 |
KR20110022577A (en) | 2011-03-07 |
EP2286344A1 (en) | 2011-02-23 |
CN102047234B (en) | 2015-04-29 |
US20090300303A1 (en) | 2009-12-03 |
TWI421712B (en) | 2014-01-01 |
JP5107462B2 (en) | 2012-12-26 |
JP2011523746A (en) | 2011-08-18 |
EP2286344A4 (en) | 2014-01-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8171246B2 (en) | Ranking and prioritizing point in time snapshots | |
US7975115B2 (en) | Method and apparatus for separating snapshot preserved and write data | |
EP1959346B1 (en) | Methods and apparatus for adjusting a journal area for continuous data protection | |
EP1922624B1 (en) | Snapshot restore method and apparatus | |
US8204858B2 (en) | Snapshot reset method and apparatus | |
US7783850B2 (en) | Method and apparatus for master volume access during volume copy | |
EP2391968B1 (en) | System and method for secure and reliable multi-cloud data replication | |
US10528520B2 (en) | Snapshot management using heatmaps in a large capacity disk environment | |
US7831565B2 (en) | Deletion of rollback snapshot partition | |
US8001345B2 (en) | Automatic triggering of backing store re-initialization | |
US7783603B2 (en) | Backing store re-initialization method and apparatus | |
US7185048B2 (en) | Backup processing method | |
US20060106893A1 (en) | Incremental backup operations in storage networks | |
EP2407884A2 (en) | Data duplication operations in storage networks | |
US20090216973A1 (en) | Computer system, storage subsystem, and data management method | |
US20070061540A1 (en) | Data storage system using segmentable virtual volumes | |
KR100819022B1 (en) | Managing a relationship between one target volume and one source volume | |
US11151090B2 (en) | Snapshot management based on data churn delta |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200980120559.5 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09755291 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011511657 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2009755291 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 20107026869 Country of ref document: KR Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |