KR20230097184A - Asynchronous cross-region block volume replication - Google Patents

Abstract

Techniques for cross-region replication of block volume data are provided. The techniques include the operation of a computer system creating a first snapshot of a block volume at a first logical time at a first geographic location, the block volume comprising a plurality of partitions, storing an object in a second geographic region. sending first snapshot data corresponding to the first snapshot to the system, creating a second snapshot of the block volume at a second logical time at a first geographic location, and creating a plurality of deltas; each delta of the plurality of deltas corresponds to one of the plurality of partitions; transmitting a plurality of delta data sets corresponding to the plurality of deltas to an object storage system in a second geographic region; Creating a checkpoint, in part by aggregating object metadata associated with a first snapshot and a plurality of deltas, receiving a restore request to create a restore volume, and creating a restore volume from the checkpoint. Includes methods for implementing operations that include.

Description

Asynchronous cross-region block volume replication

CROSS REFERENCES TO RELATED APPLICATIONS

This application claims priority to and benefit from US Provisional Patent Application Serial No. 17/091,635, entitled "ASYNCHRONOUS CROSS-REGION BLOCK VOLUME REPLICATION", filed on November 06, 2020. The contents of US Provisional Patent Application Serial No. 17/091,635 are incorporated herein by reference in their entirety for all purposes.

Cloud-based platforms provide scalable and flexible computing resources to users. These cloud-based platforms, also referred to as infrastructure as a service (IaaS), provide cloud solutions for customers' data, such as solutions for authoring transformations, loading data, and presenting data. can provide an entire suite of . In some cases, customer data may be stored in object storage and/or block volume storage within a distributed storage system (eg, cloud storage).

Techniques (eg, methods, systems, non-transitory computer-readable media storing code or instructions executable by one or more processors) are provided for asynchronous cross-region block volume replication.

In one embodiment, the method includes creating, by a computer system, a first snapshot of a block volume at a first logical time at a first geographic location, the block volume comprising a plurality of partitions. . The method includes sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic region. The method includes creating, by a computer system, a second snapshot of the block volume at a first geographic location and at a second logical time. The method includes generating, by a computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one partition of the plurality of partitions. The method includes sending, by a computer system, a plurality of delta data sets corresponding to the plurality of deltas to an object storage system in a second geographic region. The method includes generating, by a computer system, a checkpoint by aggregating, at least in part, object metadata associated with the first snapshot and the plurality of deltas. The method includes receiving, by a computer system, a restore request to create a restore volume. The method also includes creating, by the computer system, a restore volume from the checkpoint.

In one variation, generating the plurality of deltas includes generating a comparison between the second snapshot and the first snapshot. Generating the plurality of deltas may include determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot based on the comparison. there is. A plurality of deltas may describe modified data for a plurality of partitions. Creating the first snapshot may include suspending input/output operations for a plurality of partitions corresponding to logical time, generating a plurality of block images describing volume data in the plurality of partitions. and enabling input/output operations on a plurality of partitions. A restore request may be or include a failover request. The method may further include enabling the recovery volume to be created in the second geographic region, and enabling input/output operations using the recovery volume in the second geographic region. The restore request may be or include a failback request, the method comprising: creating a restore volume in the second geographic region, cloning the restore volume so that the failback volume is at least partially in the first geographic region. region, and restoring the first snapshot data in the first geographic region. Transmitting the plurality of delta data sets includes creating a plurality of chunk objects from the plurality of delta data sets, transmitting the plurality of deltas, and transmitting the plurality of chunk objects to an object storage system. steps may be included. A checkpoint may contain a manifest of object metadata. Object metadata may include chunk pointers corresponding to a plurality of chunk objects in the object storage system. Aggregating the object metadata may include updating the manifest to reflect the plurality of differences between the plurality of delta data sets and the first snapshot data.

In certain embodiments, a computer system includes one or more processes and memory in communication with the one or more processors, the memory configured to store computer-executable instructions that execute the one or more processors. Causes the processors to perform one or more of the steps of the method or variations thereof described above.

In certain embodiments, a computer-readable storage medium stores computer-executable instructions that, when executed, cause one or more processors of a computer system to perform one or more steps of a method described above or variations thereof.

1 illustrates an example system for asynchronous cross-region block volume replication in accordance with one or more embodiments.
2 illustrates an example technique for asynchronous block volume replication in accordance with one or more embodiments.
3 illustrates an example technique for restoring a block volume system with asynchronous replication in accordance with one or more embodiments.
4 illustrates an example technique for aggregating block volume replication metadata in accordance with one or more embodiments.
5 illustrates an example technique for creating a failover volume from a standby volume in accordance with one or more embodiments.
6 illustrates an example technique for resizing a standby volume in accordance with one or more embodiments.
7 illustrates an example flow for creating a recovery volume in accordance with one or more embodiments.
8 illustrates an example flow for creating a failover volume in accordance with one or more embodiments.
9 illustrates an example flow for creating a failback volume in accordance with one or more embodiments.
10 illustrates an example flow for restoring a block volume system using a standby system in accordance with one or more embodiments.
11 illustrates an example flow for resizing a block volume system and a standby system in accordance with one or more embodiments.
12 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system according to at least one embodiment.
13 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system according to at least one embodiment.
14 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system according to at least one embodiment.
15 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system according to at least one embodiment.
16 is a block diagram illustrating an exemplary computer system according to at least one embodiment.
In the accompanying drawings, similar components and/or features may have the same reference labels. Additionally, various elements of the same type may be distinguished by a second label and a dash distinguishing similar elements following the reference label. When a first reference label is used in the specification, the description is applicable to any one of similar elements having the same first reference label regardless of the second reference label.

In the following description, various embodiments will be described. For purposes of explanation, specific configurations and details are set forth to provide a thorough understanding of the embodiments. However, it will also be apparent to those skilled in the art that the embodiments may be practiced without these specific details. Additionally, well-known features may be omitted or simplified in order not to obscure the described embodiment.

Cloud-based platforms provide scalable and flexible computing resources to users. These cloud-based platforms, also referred to as infrastructure as a service (IaaS), are a suite of cloud solutions for a customer's data, such as solutions for authoring transformations, loading data, and presenting data. suites can be provided. In some cases, customer data may be stored in object storage and/or block volume storage within a distributed storage system (eg, cloud storage). Customer data may be stored, for example, in data centers located in geographic areas as part of a global distributed storage system. A data center may be selected based, at least in part, on one or more performance metrics including latency, input-output operations per second (IOPS), throughput, cost, reliability, and the like. In some cases, the optimal location of the data center may correspond to the customer's location (eg, when the customer generates significant volumes of internal data). In some cases, the optimal location may correspond to the location of the customer's clients and/or users (eg, when the customer operates a content delivery network).

Customer data is placed on a backup system (also referred to as an availability domain, or “AD”) in a different geographic area or data center to be recovered following a failure by designating the backup system as the primary system (referred to as “failover”). can be copied. Failover systems can be characterized by a number of metrics including recovery point objective (RPO), recovery time objective (RTO) and disaster recovery (DR). Typically, RPO describes how much data may be lost during a failover, so the goal of a backup system may be to minimize the RPO. Typically, creating a backup copy involves performing a backup in one region, waiting for the backup to be uploaded to the cloud system, then initiating a cross-region copy operation, and in the destination region This entails waiting for the backup to complete. Significant metadata overhead can be associated with each backup, which can increase computational demands and resource cost at high repetition rates. For this reason, RPOs are typically available on the order of hours for backup systems (where time represents the amount of data given the average data transfer rate).

Similarly, RTO can be an important metric of a failover system as it indicates the length of time involved to restore data after failover. For a backup restored from a different geographic area, input-output operations may exhibit elevated latency compared to typical conditions until the volume is fully restored. Moreover, recovery times and latency of on-demand data operations can increase significantly in response to a spike in system load, for example, corresponding to data traffic movement in response to a region-wide outage. Latency may be further raised while the distributed data storage system also implements backup restoration from cross-region backup copies.

Because customer data may be valuable, customers may, for example, have one or more data redundancy and infrastructure configured to increase storage reliability (eg, power supply augmentation, surge protection, etc.) It can be expected that data will be protected by the approaches. Data redundancy can take many forms, including, but not limited to, replicating customer data to local storage media (eg, physical storage media), distributed storage systems, and/or different distributed storage systems. In some cases, customer data may be copied all at once, referred to as a backup. This approach can involve significant latency in system input-output operations while backups occur. For example, the storage system may freeze all read and write operations while the backup is in progress to ensure that a complete duplicate of customer data is preserved. In such cases, additional buffer capacity may be used to temporarily store new data in the distributed storage system and/or customer data requests may be deferred.

In contrast to full-volume backups, customer data can be replicated asynchronously, as described in detail with reference to the figures below. For example, customer data is continuously (eg, backed up) at least in part by creating objects that describe incremental changes to sub-units of a data storage system (eg, partitions in block volume systems). may be replicated (in intervals of minutes, as opposed to hours that may be typical in systems). In the following paragraphs, the object describing the incremental change is also referred to as a "delta". As described in detail with reference to FIG. 1, a delta can account for one or more changes of customer data between two asynchronous records. The record of the state of user data in a distributed storage system is also referred to as a "snapshot", and the state of the data in a single logical time as opposed to a single chronological time, as described in detail with reference to the figures below. can indicate In some cases, deltas may be created, and deltas may be stored and/or transmitted/replicated as the deltas are created. As such, asynchronous replication may allow customer data to be replicated by sending deltas rather than backups.

For at least these reasons, the techniques described herein exhibit one or more advantages over data redundancy approaches that rely on backup customer data. For example, creating multiple deltas to replicate changes to the data since the snapshot can allow the data replication process to be broken down in chronological time, and can measure latency, IOPS, and/or other performance metrics. Reduce the impact of data duplication.

Approaches that rely on backing up the entire volume are better suited for cases where a system failure occurs when an available backup does not reflect valuable changes to customer data (for example, when an existing backup fails long after it was made; or if significant changes have occurred to customer data since the last backup). Implementing asynchronous replication may also improve retention of customer data as well as improve overall system performance. As such, asynchronous replication approaches can provide improved RPO, RTO, and DR over backup systems.

As an illustrative example, a block volume system may be implemented in a distributed storage system having multiple data centers in multiple geographic regions around the world. Customer data stored in a block volume system in one of the data centers may be replicated to a second data center to prevent data loss in case the first data center experiences a catastrophic failure. Customer data can be replicated by creating both snapshots and deltas, where snapshots can provide holistic replication of customer data across multiple partitions, where a delta follows a preceding snapshot. may allow incremental tracking of changes to customer data for a single partition that Once a delta is created, the delta, along with the customer data it describes, can be transferred to a second data center, which is stored in the second data center to restore customer data in the event of a failure in the first data center. can be used by the system. While a delta is being created for one partition, other partitions may remain available for input-output operations, enabling the storage system in the first data center to maintain a desired level of performance. can In the event of a failure in the first data center, the second data center may take over the role of the first data center (referred to as a "failover") and/or customer data may be restored to the first data center to resume operations. (referred to as "failback").

1 illustrates an example system 100 for asynchronous cross-region block volume replication in accordance with one or more embodiments. As described above, system 100 may enable data redundancy by replicating customer data from a source system to a destination system to provide customer data during disaster recovery. In some embodiments, first data center 120 (eg, source system) may store data in block volume system 122 . In some embodiments, first data center 120 may store data in an object storage system rather than a block volume system. Data may be generated and/or provided by users of the first data center 120 . The first data center 120 may correspond to one or more operating criteria, such as, for example, performance metrics (eg, latency, IOPS, throughput, cost, reliability, etc.) and/or a user's location. It may be located in a first geographic area that may be proximate (eg, area A). In some cases, the possibility of disruptions in the normal operation of the first data center 120 (eg, power failure, data corruption, distributed denial of service attack (DDOS), natural disaster, etc.) may pose a risk of data loss. To potentially reduce the risk of data loss, the data may be replicated to the second data center 130 (eg, destination system). In some embodiments, the second data center 130 may be located in a different geographic area than the location of the first data center 120, which reduces the risk of data loss posed by natural disasters or infrastructure failures. can reduce risk. Similarly, storing data in a second data center 130 separate from the first data center 120 may disrupt input-output operations targeting the first data center and/or cause data loss. It can reduce potential vulnerability to possible malicious actions (eg, DDOS, data corruption, etc.). As described in more detail below, the second data center 130 may include (e.g., based on customer configuration and/or preferences) a standby volume, object storage, and/or store data in data store 132 in other storage formats.

In some embodiments, data stored in first data center 120 may be replicated to second data center 130 via an asynchronous replication 140 system. As described above, asynchronous replication may represent reduced interruption of normal input-output processes when restoring operation (e.g., RTO) in the first data center 120. It can provide technical advantages over periodic backup replication in that it can reduce the extent of data loss caused by outages (eg, RPO). In some embodiments, asynchronous replication 140 is incrementally, rather than as a coherent backup image created at a specific chronological time, which can be periodically replaced with a new backup image ( 120 to the second data sensor 130 to generate and transmit the copied data. Instead, as described in more detail with reference to FIG. 2 below, asynchronous replication 140 may, for example, copy data and/or records of changes to data stored in first data center 120 . By generating and transmitting data stored in the second data center 130 may be dynamically updated. In some embodiments, customers and/or users of the block volume system 122 may have subsequent availability when the configuration volumes are created and/or after the block volume system 122 is already operating as a distributed data storage system. As an option, asynchronous replication 140 can be configured. In some embodiments, configuring asynchronous replication 140 may include designating a destination area (eg, a second geographic area, “region B”). In some embodiments, configuring asynchronous replication 140 may include specifying a destination AD within the destination area, for example when the replicated data is stored on the standby volume system.

In some embodiments, asynchronous replication 140 may include a snapshot creation 142 sub-system that may create snapshots of data stored in block volume system 122 . In some cases, a snapshot can describe the instant state of user data stored in block volume system 122 at a particular logical time, as described in more detail below. In contrast to the backup replication approach, asynchronous replication 140 includes a delta creation 144 sub-system by which changes made to data in the block volume system 122 since the snapshot can be verified. can do. Changes can consequently be expressed as deltas, which can be used to update the snapshot rather than replace it entirely. Snapshot creation 142 may be, for example, minute-by-minute (eg, 1-10 minutes, 10-20 minutes, 20-30 minutes, etc.) and/or hourly (eg, 1-10 minutes, etc.). time) can create new snapshots periodically.

In some embodiments, snapshot creation 142 may proceed with a two-phase commit protocol involving one or more partitions of block volume system 122 . The two-phase commit may include creating a partition image after snapshot creation 142 receives a commitment from the block volume system that input-output operations for the partition have been suspended. After the partition image is complete, the partition can be released to resume input-output operations. In some embodiments, a two-phase commit may be applied to the block volume system 122 as a whole so that all read and write operations to the entire block volume system may be blocked during the time the snapshot is created. In some embodiments, snapshots may be created on the order of milliseconds (eg, 1-15 msec, 5-10 msec, etc.).

In some embodiments, creating snapshot 142 may include creating an image of data stored in block volume system 122 on a partition by partition basis. If block volume system 122 stores data in one or more partitions, snapshot creation 142 may schedule creation of a partition image such that snapshot creation does not disrupt input-output operations of one partition. there is. Snapshot creation 142 may create and compile images of all partitions included in the block volume system while potentially avoiding interruption of normal input-output operations of partitions other than the one being imaged.

A snapshot may record an image of data at a particular logical time, where the logical time represents an implementation of snapshot creation 142 . For example, a first snapshot by create snapshot 142 may be identified as a first logical time, the chronological time at which one or more partitions of block volume system 122 are processed by create snapshot 142 . can be generated for the length of Similarly, snapshot creation 142 may occur when a chronological time period, which may be identified as a second logical time, has expired (e.g., 1-10 minutes), corresponding to a second iteration of the snapshot creation 142 subsystem. , etc.), a second snapshot of the block volume system 122 may be created.

In some embodiments, delta generation 144, at least in part, enables asynchronous replication 140 to transmit new or modified data to second data center 130 without transmitting unchanged data. By doing so, asynchronous replication 140 can be made possible. In some cases, a delta can account for changes in data stored in a partition of block volume system 122 between the first and second snapshots. For example, delta generation 144 may compare the second snapshot with the first snapshot, and identify blocks that have been added, removed, modified, and the like. In this way, deltas can also account for data removed between two consecutive snapshots.

In some cases, a delta may be a logical structure containing a unique identifier as well as a list of memory pointers. A memory pointer may describe memory locations within the first data center 120 and/or the second data center 130 for data that has changed between two successive snapshots. As described in more detail with reference to FIGS. 2-6 below, implementing asynchronous replication 140 does not transfer the full snapshot, but rather the delta for the partition to the second data center 130. may include doing

In some embodiments, data store 132 in second data center 130 may store replicated data as an object store. As such, asynchronous replication 140 may include a data transformation 146 sub-system for transforming data from first data center 120 into chunked objects. In some embodiments, data from block volume system 122 may be consolidated into 4 MB chunk objects. In some embodiments, chunk objects may implement a format similar to that used for system backup operations, in which asynchronous replication 140 is integrated into existing distributed storage systems that utilize backup and/or tiered upload techniques. can make things possible

In some embodiments, asynchronous replication 140 may create a checkpoint after deltas corresponding to partitions of block volume system 122 have been created. In some embodiments, the checkpoint generation 148 sub-system checks by aggregating the deltas and applying the aggregated changes to a previously created checkpoint, as described in more detail with reference to FIG. 4 below. points can be created.

When the first data center experiences a failure (eg, severe failure) and restoration of the system is desired by, for example, users and/or customers of the block volume system 122, a failover/failback request is requested. (150) may be provided for asynchronous replication (140). Failover/failback request 150 may be in the form of a restore request that causes asynchronous replication 140 to create a restore volume using replicated data stored in the second geographic region. In some embodiments, asynchronous replication 140 may implement one or more restore operations in response to receiving failover/failback request 150 . In some embodiments, failover/failback request 150 may include a request by a service external to the distributed storage system and/or a request by a user of the distributed storage system. In some cases, such as when data store 132 is a replica (e.g., a standby volume) of block volume system 122, a failover request may result in data store 132 being a block volume It may indicate that the role of system 122 should be assumed. For example, the data store may handle input-output operations involving user data. In some embodiments, the failover/failback request 150 causes the first data center 120 to receive the replicated data as described above and resume operations prior to the failure that caused the failover request. It may indicate that it should be configured.

2 illustrates an example technique 200 for asynchronous block volume replication in accordance with one or more embodiments. As described above, asynchronous replication may enable user data to be transferred from a source system to a destination system in increments rather than as a single backup image. In some embodiments, block volume system 122 may store data that may be received from and presented to a user via input-output (I/O) operations 210 . Thus, received data may be stored in one or more constituent partitions 220 of block volume system 122 . Asynchronous replication systems (e.g., asynchronous replication 140 of FIG. 1) replicate user data from block volume system 122 to data store 132 by one or more processes described below. can be sent to

In some embodiments, user data may be replicated by creating a first snapshot 230 (eg, operation 250 ). As described in detail with reference to FIG. 1 , a snapshot may be a record of data stored in block volume system 122 . Snapshot creation of the first snapshot 130 (eg, snapshot creation 142 of FIG. 1 ) may occur at a first logical time 260 . The logical time described in more detail with reference to FIG. 1 is, for example, when snapshot creation proceeds on a partition-by-partition basis, each of the constituent partitions 220 of the block volume system 122 uses a two-phase commit protocol. An iteration of the snapshot creation operation (eg, operation 250) to be implemented may be described. In such cases, first logical time 260 may correspond to a period of time during which the snapshot is being created.

In some cases, first snapshot 230 is the very first implementation of an asynchronous copy of user data stored in block volume system 122 . In such cases, the entire data set represented by first snapshot 230 may be transferred to data store 132 . In this way, first snapshot 130 provides at least the knowledge that records of user data and metadata (e.g., a manifest as described below) can be transferred from a source system to a destination system (e.g., a manifest). For example, operation 252) may be similar to a backup.

In some embodiments, the first snapshot may be a disk image of block volume system 122 and/or constituent partitions 220 . In some embodiments, first snapshot 230 may include multiple block images 232 such that the data described by first snapshot 230 is at least partially block images 232 may be asynchronously transmitted to the data store 132 by transmitting individually and/or in groups. As described in more detail with reference to FIG. 3 , the block images 232 are chunk objects to be stored in the data store 132 when the data store 132 is an object storage system, for example. can be converted

In some embodiments, asynchronous replication may include creating a second snapshot 240 at a second logical time 262 (eg, operation 254 ). In some embodiments, the second logical time 262 corresponds to a time period following the first logical time 260 as described in more detail with reference to FIG. 1 above. For example, the second snapshot 240 may take a few minutes (eg, 1-15 minutes, 5-10 minutes, etc.) after the creation of the first snapshot 230 to perform the above-described 2- It can be created by a step-commit protocol. In some embodiments, second snapshot 240 may include one or more changes to data stored in block volume system 122 relative to first snapshot 230 .

Rather than directly transferring the second snapshot 240, the technique 200 involves creating deltas 242 corresponding to constituent partitions of the block volume system 122 (e.g., operation 256 )) may be included. As described in more detail with reference to FIG. 1 above, delta generation (eg, delta generation 144 of FIG. 1 ) is data included in deltas 242 for each partition 220 . First, to generate a list of modified blocks (identified by points) paired to a reference chunk identifier of the destination system (e.g., data store 132), which may constitute at least a portion of This may include comparing the state of data reflected in the snapshot 230 and the state of data reflected in the second snapshot 240 . In some embodiments, deltas may also include a unique delta identifier. In some embodiments, the unique delta identifier may correspond to the identifier of second snapshot 240 . In this way, deltas 242 can reflect changes to data that can be tracked up to a second logical time 262, for example, as an approach to providing I/O history.

Once generated, deltas 242 may be transmitted to a destination system, eg, data store 132 (eg, operation 258). In some embodiments, deltas 242 may be transmitted along with any new data added to block volume system 122 . Data removed from the block volume system 122 between first logical time 260 and second logical time 262 does not transfer any data (eg, chunk objects) from the source system to the destination system. and may be reflected by the metadata included in delta 242.

3 illustrates an example technique 300 for restoring a block volume system with asynchronous replication in accordance with one or more embodiments. As part of the asynchronous replication system described with reference to FIGS. 1 and 2 above, the destination system, for example, the second data center 130 is a source system (for example, the first data center of FIG. 1 ( 120)) may receive and maintain a copy of the data stored. In some embodiments, second data center 130 may be located in a different geographic area than the geographic area of the source system (eg, area B rather than area A). The second data center 130 may include a data storage unit 132 capable of storing replicated data as one or more data objects 310 in the object storage unit.

In some embodiments, data store 132 can operate as an object storage system based at least in part on the configuration of an asynchronous replication system, such that asynchronous replication stores replicated data as chunk objects rather than blocks. can be configured. That is, in some embodiments, the second data center 130 may include a standby volume to enable direct application of deltas to the standby volume. In embodiments where data store 132 includes a standby volume, asynchronous replication may be completed without converting blocks to objects (eg, data conversion 146 ).

In some embodiments, asynchronous replication may include creating a first checkpoint 320 (eg, operation 350 ). A checkpoint may, at least in part, generate a manifest (e.g., metadata) describing identifiers and locations of chunk objects stored in the data store 132, as described in more detail with reference to FIG. It may differ from a snapshot in that it can be included. For example, if first checkpoint 320 corresponds to the first time the checkpoint was created as part of an asynchronous replication, then the first checkpoint is a chunk pointer to each of the chunk objects that make up the replicated data. may contain a list of

As part of the asynchronous replication, the second data center 130 will receive the deltas 330 and corresponding data from the source system (eg, operation 352). As described above, deltas are changes to data stored in the source system resulting from input-output operations (I/O operations 210 in FIG. 2) between the first and second snapshots. may include metadata describing them. Corresponding data may be received as chunk objects (eg, 4 MB chunk objects).

Deltas 330 may be applied to first checkpoint 320 as part of generating second checkpoint 340 (eg, operation 354 ). Generating the second checkpoint may include identifying one or more chunk objects identified in the deltas 330 and applying the indicated modifications indicated by the deltas. For example, one or more conversions to chunk data may be indicated by first delta 330-1 of deltas 330, where first delta 330-1 is (e.g., chunk pointer ) indicates the location in the object store of the data store 132 of the referenced data for a given partition. In some embodiments, rather than directly modifying the referenced data, the asynchronous replication is performed as part of generating the second checkpoint 340 as described in more detail with reference to FIG. 4 below. It may include updating the first checkpoint 320 to reflect the changes indicated by the 1 delta 330-1.

In some embodiments, the second data center 130 (e.g., the destination system) is an asynchronous replication system (e.g., asynchronous replication system 140 of FIG. 1 via operation 356). It may be the subject of a failover/failback request that may be received from a user (eg, failover/failback request 150 of FIG. 1). A failover/failback request may be in response to a failure (eg, natural disaster) affecting the source system (eg, first data center 120 of FIG. 1 ). The failover/failback request may include parameters guiding how the block volume system is to be restored from the replicated data and the second checkpoint, as described in more detail with reference to FIGS. 7-10 below. there is.

Implementing failover/failback may include creating a failover/failback volume, also referred to as a restore volume (eg, operation 358). As described above, the failover system may be hosted on the destination system, while the failback system may be hosted on the source system. In some embodiments, such as when a failback request is received, second checkpoint 340 transfers data stored in objects 310 to blocks within the source system (eg, block volume system 122). can be used to map to The process of creating a failback volume is described in more detail with reference to FIG. 5 below. Similarly, failover is a block volume in the second data center 130 to assume the role of the first data center 120 with respect to input-output operations until the first data center 120 recovers from a failure. This may include creating a system. Creating a failover volume in the second data center 130 may be performed on objects 310 by, for example, using a second checkpoint 340 as described in more detail with reference to FIG. 4 below. It may include mapping stored data to blocks. In some embodiments, restoring block volume data includes, but is not limited to, creating a new block volume in the destination region, implementing a failover in the destination region, or cross-region replication to a new volume in the source region. and performing a failover in the source zone.

In some embodiments, such as when the second data center 130 maintains a standby volume for storing replicated data, the standby volume contains data from any deltas not transferred from the source system at the time of failure. It may already contain all changes from the source volume except changes. As such, failover may be immediately available with relatively little lost data (e.g., RPO for snapshot creation times of approximately 1-15 minutes), as described in more detail with reference to FIG. 5 below. (e.g. exhibiting a low RTO). In contrast, a backup system that implements synchronous replication of full backup images can result in an RPO of around several hours.

In some embodiments, rather than creating a failover/failback volume, asynchronous replication may include reverse replication. In this case, the destination system may be designated as the source system, while the previous source system may be re-designated as the destination system's. In this way, input-output operations where data is written to and read from data store 132 can occur in the second data center 130, and snapshot creation and delta creation can also occur in the second data center. . Correspondingly, the replicated data may be transmitted to the first data center 120 .

4 illustrates an example technique 400 for aggregating block volume replication metadata in accordance with one or more embodiments. Asynchronous replication can include multiple iterations of one or more configuration operations (eg, create snapshots, create deltas, etc.) over a period of time over which data can be replicated. In such cases, updating the system (eg, second data center 130 of FIGS. 1-3 ) on purpose corresponds to the replicated data reflecting the changes introduced by each subsequent iteration. It may include maintaining an updated record of memory locations.

In some embodiments, asynchronous replication generates second checkpoint 340 through one or more operations on deltas (eg, deltas 330 in FIG. 3 ) and first checkpoint 320 . may include doing For example, when multiple deltas from the source system are received at the destination system after the creation of the first checkpoint 320, the correct record of memory locations does not refer only to the first checkpoint ( 320) can be better represented by changes indicated by deltas.

In some embodiments, first checkpoint 320 may include manifest 420 . Manifest 420 may correspond to a partition of the source system (eg, to guide failover/failback operations), and as such, multiple manifests may correspond to a source system when the source system includes multiple partitions. Asynchronous replication of data from Manifest 420 includes chunk pointers 422 (e.g., chunk pointers 1-N, where 'N' is an integer referencing the number of chunk pointers 422 included in manifest 420). Chunk pointers 422 may describe memory locations associated with replicated data (eg, stored as chunk objects in data store 132) First checkpoint 320 Subsequent to generating , the destination system may receive a number of deltas, e.g., as generated by iterating snapshot creation and delta creation In this way, the second checkpoint 340 Generating may include aggregating the deltas (eg, operation 450 ) For example, aggregating the deltas may include a first checkpoint 320 to verify the snapshot from which the deltas were created. ) In this way, the checkpoint reference determines whether the received deltas (which also include an identifier referencing the snapshot) have already been reflected in the manifest 420. can make it possible

In some embodiments, the deltas can be combined with the first checkpoint to provide an updated manifest 440 . Combining the deltas may include modifying the first chunk pointer 422-1 in the manifest as indicated by the delta (e.g., the pointer to the location of the data is duplicated described by the delta). may change in response to transformations on the data). In this way, by applying all transformations indicated in all received deltas to the manifest, the updated manifest 440 will ensure that the failover/failback volume is the same if the previous manifest (e.g., manifest 420) is used. It can be made possible with a more potentially reduced RPO. Similarly, for each iteration of the asynchronous replication, newly created deltas may be created and sent to the destination system, where they may be aggregated with the most recent manifest (eg, updated manifest).

5 illustrates an example technique 500 for creating a failback volume from a standby volume in accordance with one or more embodiments. As described above, a failback request (eg, failover/failback request 150 of FIG. 1 ) is part of a failover at the source system, eg, first data center 120 . can be received as In contrast to a failover volume or reverse replication hosted on the destination system, a failback volume has the same reasons that cause the first data center 120 to be chosen to host user data (e.g., performance metrics, stability, etc.).

In some embodiments, the destination system, eg, second data center 130 , may clone standby volume 560 to replicate data from source volume 522 . Standby volume 560 may include all deltas received from the source volume, so that standby volume may reflect the current state of source volume 522 . For example, first delta 540 may be generated from source volume 522 (eg, by operation 512), and first delta 540 may be generated (eg, by operation 514). by) can be applied directly to the standby volume 560. In some embodiments, applying the delta to standby volume 560 copies the referenced data (eg, modified blocks) from the source volume and performs an input-output operation (eg, modified blocks) of the referenced data. eg, a user-write operation) to the standby volume in a similar way. Additionally, applying the delta may include updating the manifest as described with reference to FIG. 4 . Although Figure 4 describes the manifest in terms of chunk pointers, the manifest can similarly describe memory locations within a standby volume where replicated data can be stored.

In some embodiments, updating standby volume 560 may include applying the deltas in separate operations such that the deltas may not be partially applied to standby volume 560 . For example, first snapshot 530 of standby volume 560 provides a failback location if a failover/failback request is received during an operation to apply the delta (eg, operation 514 ). can be created to provide In this way, rather than waiting for the delta to be applied, system recovery can implement the first snapshot 530, which can provide an improved RTO at the cost of an incremental increase to the RPO. In some embodiments, first snapshot 530 may be deleted once first delta 540 is applied to standby volume 560 . Consequently, when the second delta 550 (or deltas) is generated (eg, operation 516 ), the second snapshot 532 may apply the second delta (eg, operation 516 ). It can be created in standby volume 560 prior to operation 520 (eg, operation 518). After fully applying the second delta, the second snapshot 550 may also be deleted.

In some embodiments, standby volume 560 may be used to create a failover volume or failback volume in response to receiving a failover/failback request (eg, operation 522 ). In some embodiments, restoring data may include transferring the data back to the source zone by enabling cross-region replication for the failover volume. In some embodiments, when the request is a failback request, standby volume 560 may be cloned and restored in first data center 120 (eg, operation 524). Cloning the standby volume 560 may include creating a failback volume 562 in the first data center 120 and restoring the data (e.g., sometimes "hydrating the clone"). )"). In some embodiments, the restored data may include the modifications indicated by second delta 550 if the second delta was fully applied at the time the failback request was implemented. Implementing failover recovery using cloning could potentially enable cross-region replication to continue on an existing standby volume. Advantageously, clone volumes can be available with low or no latency, allowing negligible effect on RTO.

6 illustrates an example technique 600 for resizing a standby volume in accordance with one or more embodiments. Normal operation of the source system, such as the source volume 522 in the first data center 120, (e.g., adding partitions, removing partitions, or changing the size of one or more partitions of the source volume 522) resizing the source volume 522 (to adjust, etc.). In some embodiments, because deltas are created for one partition, resizing source volume 522 may affect the mapping of source volume deltas 522 to standby volume 560. . As such, the standby volume 560 may, after implementing one or more approaches to limit potential errors in delta application introduced by resizing the source volume 522, It can be scaled to account for resizing.

In some embodiments, one or more deltas 620 may be generated from source volume 522 as described in more detail with reference to FIGS. 1-2 above (eg, operation 610 ). ). Delta 620 may be applied to standby volume 560 as described above with reference to FIG. 5 (eg, operation 612). In some cases, resizing the source volume 522 may include receiving a resize request from a user or from another system (eg, operation 614 ). To potentially limit the impact of resizing on the RPO, the last delta 630 can be generated after receiving a resize request and before resizing the source volume 522 (e.g. , action 616). In some embodiments, creating the last delta 630 includes creating a new snapshot of the source volume 522, creating the deltas based at least in part on the new snapshot, and creating the resulting deltas. It may include applying deltas. In this way, standby volume 560 can describe the state of the source volume immediately prior to implementation of the resize request by resizing source volume 522 (e.g., operation 618). . Applying the last delta (e.g., operation 620) can be separated from resizing the source volume 522, so that once the last delta 630 is created, the source volume may be resized before it can be applied to the standby volume 560 . In some embodiments, standby volume 560 may be resized in a corresponding manner following application of the last delta (eg, operation 622 ).

7 illustrates an example flow 700 for creating a recovery volume in accordance with one or more embodiments. The operations of flow 700 may be implemented as hardware circuitry and/or stored as computer-readable instructions on a non-transitory computer-readable medium of a computer system, such as asynchronous replication system 140 of FIG. 1 . can As implemented, instructions represent modules containing code or circuitry executable by processor(s) of a computer system. Execution of these instructions configures the computer system to perform certain operations described herein. Each piece of circuitry or code in conjunction with the processor performs a separate operation(s). Although the actions are illustrated in a specific order, it should be understood that the specific order is not essential and that one or more actions may be omitted, skipped, and/or rearranged.

In one example, flow 700 includes act 702 where the computer system creates a snapshot of a block volume in a first geographic region. As described in detail with reference to FIGS. 1-2, a block volume (eg, block volume 122 of FIG. 1 ) is dependent on one or more performance metrics associated with input/output operations, stability, cost, and the like. may be hosted at a first data center (eg, first data center 120 of FIG. 1 ) within a first geographic region, which may be determined based at least in part on the basis of creating a first snapshot (eg, For example, operation 250 of FIG. 2 may include generating one or more block images (eg, block images 232 of FIG. 2 ) of one or more blocks that make up the block volume. Creating the snapshot may include implementing a two-phase commit protocol, whereby input-output operations (eg, I/O operations 210 of FIG. 2 ) are blocked for a partition. Images can be suspended for that partition of a block volume during the time they are being created. After creation of a snapshot for a block volume system or for a partition, the computer system can resume input-output operations to the block volume.

In one example, flow 700 includes an act 704 of the computer system sending first snapshot data corresponding to the first snapshot to a second storage system in a second geographic region. In some embodiments, the snapshot data may include block images, such as when the first snapshot is the first created snapshot of a block volume or when a previously created snapshot is not available. As such, the block images can be transmitted to the system (eg, data store 132 of FIG. 1). In some implementations, the block images can be converted to chunk objects and sent as snapshot data to the second storage system (eg, operation 252 of FIG. 2 ). In some cases, chunk objects are transmitted along with metadata describing the snapshot data including, for example, a manifest listing chunk pointers identifying locations of the snapshot data in memory.

In one example, flow 700 includes an act 706 of the computer system creating a second snapshot of a block volume in the first geographic region. As described in more detail with reference to FIG. 2 , the second snapshot (eg, second snapshot 240 of FIG. 2 ) is taken at a first logical time (eg, second snapshot 240 of FIG. 2 ) at which the first snapshot was created. . The second snapshot may include information on modifications to data stored in the block volume system (eg, read-write operations on data) occurring after the first snapshot is created.

In one example, flow 700 includes an act 708 where the computer system generates a plurality of deltas. Rather than sending the second snapshot directly to the second storage system, a plurality of deltas are used to determine the creation of the second snapshot (eg, the second logical time) and the creation of the first snapshot (eg, the second snapshot). 1 logical time) to the data stored in the block volume between Generating the deltas may include comparing block images of the partition in the second snapshot to corresponding images of the partition in the first snapshot, and identifying one or more modifications to data in the partition. As described in more detail with reference to FIGS. 1-2 , deltas include, but are not limited to, a delta identifier corresponding to a snapshot, one or more block identifiers describing data within a modified block volume, and/or block metadata comprising a corresponding number of object identifiers describing the locations of the data identified by the identifiers.

In one example, flow 700 includes an act 710 of the computer system transmitting data for a plurality of deltas. As described in detail with reference to FIG. 2, transmitting the data may include creating a copy of the data from the block volume system to the second storage system. This may include creating chunk objects and copying the chunk objects to a second storage system (eg, the destination system), such as when replicated data is stored in an object store. The data may be accompanied by a plurality of deltas, which may provide metadata describing the correspondence between blocks and chunk objects of the block volume system on a partition-by-partition basis.

In one example, flow 700 includes act 712 where the computer system creates a checkpoint in a second geographic area. As described in more detail with reference to FIGS. 3-4, a checkpoint (e.g., checkpoint 320 in FIG. 3) is a manifest describing a plurality of chunk pointers (e.g., manifest in FIG. 4). (420)). In some embodiments, creating a checkpoint is an updated manifest describing a second plurality of chunk pointers that reflect modifications included in the plurality of deltas as applied to the checkpoint (e.g., in FIG. 4 ). Aggregating the deltas with previously created checkpoints to create an updated checkpoint (e.g., second checkpoint 340 in FIG. 3) that may include an updated manifest 420 ( For example, operation 450 of FIG. 4 ).

In one example, flow 700 includes an act 714 of the computer system receiving a restore request. A restore request (eg, failover/failback request 150 of FIG. 1 ) may include a user request to create a restore volume, also referred to as a failover volume, at the second geographic location. In some embodiments, the restore request can include a user request to create a restore volume, also referred to as a failback volume, at the first geographic location. In some embodiments, the restore request may include a request to reverse the asynchronous replication, thereby retrieving the second storage system (eg, data store 132 of FIG. 1 ) for asynchronous replication. Specify the source system as the source system and the block volume system (eg, block volume system 122 of FIG. 1) as the destination system. In some embodiments, the restore request may be automatically generated (eg, without user interaction) as part of the configuration of the asynchronous replication. For example, an asynchronous replication system can be configured to automatically generate restore requests in response to failures occurring on the source system.

In one example, flow 700 includes an act 716 of the computer system creating a restore volume. Creating a restore volume may, for example, refer to an updated manifest (eg, updated manifest 440 of FIG. 4 ), as described with reference to FIGS. 3 and 5 , stored in the object store. It may include creating a block volume by mapping duplicate data from chunk objects to blocks within the block volume. In this way, the restore volume can be created with a lower RPO than can be achieved by the backup system. Similarly, the RTO of a system restore operation, which may reflect the time between receiving a restore request and resuming normal input/output operations, may depend on whether the restore request is a failover or failback request, which is It will be described in more detail with reference to FIGS. 8 and 9 below.

8 illustrates an example flow 800 for creating a failover volume in accordance with one or more embodiments. The operations of flow 800 may be implemented as hardware circuitry and/or stored as computer-readable instructions on a non-transitory computer-readable medium of a computer system, such as asynchronous replication system 140 of FIG. 1 . can As implemented, instructions represent modules containing code or circuitry executable by processor(s) of a computer system. Execution of these instructions configures the computer system to perform certain operations described herein. Each piece of circuitry or code in conjunction with the processor performs a separate operation(s). Although the actions are illustrated in a specific order, it should be understood that the specific order is not essential and that one or more actions may be omitted, skipped, and/or rearranged.

In one example, flow 800 includes an act 802 of the computer system receiving a restore request that is a failover request. As described in more detail with reference to FIGS. 1-6 above, a failover request is a block volume system (eg, a cloud storage) that can form part of a distributed storage system (eg, cloud storage). may be received by asynchronous replication (eg, asynchronous replication 140 of FIG. 1 ) after a disaster or other failure affecting block volume system 122 of FIG. 1 . In some embodiments, the failover request may include a request to create a failover volume that reproduces the block volume system in a second geographic area potentially unaffected by the failure affecting the block volume system. there is.

In one example, flow 800 includes an act 804 of the computer system creating a failover volume in the second geographic region. In some cases, a failover volume may allow resume of input-output operations (eg, I/O operations 210 of FIG. 2 ) before a failure affecting the block volume system is resolved. there is. In these cases, RTO can be improved by creating a failover volume over a failback volume. As described above with reference to FIG. 7 , creating a failover volume using replicated data stored in the object storage unit is a memory of the replicated data, which is updated with recently aggregated deltas generated during asynchronous replication. creating the failover volume with reference to an updated manifest describing the locations within (eg, updated manifest 440 of FIG. 4 ).

In one example, flow 800 includes act 806 where the computer system hydrates the failover volume. As described in more detail above, hydrating a failover volume may transfer the data described in the updated manifest to an object store (e.g., as being in chunk objects as described above with reference to Figure 1). equal) to a block that can be addressed by read/write operations of the block volume system (eg, I/O operations 210 of FIG. 2). In some embodiments, partitions of a block volume system may be preserved in a failover volume.

In one example, flow 800 includes act 808 where the computer system initiates input-output operations. Following "hydration" where the replicated data from the block volume system is restored and mapped to the failover volume, the failover volume can initiate input-output operations and assume the role of the block volume system while the failure can be resolved. can

9 illustrates an example flow 900 for creating a failback volume in accordance with one or more embodiments. The operations of flow 900 may be implemented as hardware circuitry and/or stored as computer-readable instructions on a non-transitory computer-readable medium of a computer system, such as asynchronous replication system 140 of FIG. 1 . can As implemented, instructions represent modules containing code or circuitry executable by processor(s) of a computer system. Execution of these instructions configures the computer system to perform certain operations described herein. Each piece of circuitry or code in conjunction with the processor performs a separate operation(s). Although the actions are illustrated in a specific order, it should be understood that the specific order is not essential and that one or more actions may be omitted, skipped, and/or rearranged.

In one example, flow 900 includes an act 902 of the computer system receiving a restore request that is a failback request. In some embodiments, the failure that caused the restoration request may not be of indeterminate duration. For example, a disturbance may be resolvable within a predictable length of time, such as when a disturbance is caused by temporary power or network outages that can be easily overcome. As such, a failback request may represent a desirable alternative for creating a failover volume (eg, from a distributed storage performance perspective) and a potentially limited difference in terms of the RTO for the failover volume.

In one example, flow 900 includes an act 904 of the computer system creating a standby volume in the second geographic region. In some embodiments, the standby volume is capable of failover, except that the standby volume is not designed to perform input-output operations in a second geographic area (eg, second data center 130 in FIG. 1 ). It can include a block volume system similar to a volume. For example, a standby volume may be invisible to a user of a distributed storage system and/or configured to be unsuitable for input-output operations. In some embodiments, the standby volume may reproduce the structure of the block volume system (eg, with respect to partitioning, size, etc.) and the mapping of blocks to replicated data stored as chunk objects.

In one example, flow 900 includes an act 906 of the computer system cloning the standby volume in the first geographic region. As described in more detail with reference to FIG. 5 above, cloning the standby volume (eg, operation 524 of FIG. 5 ) at least partially uses the structure of the standby volume to create a failback volume. and restoring the block volume system to the first geographic region by creating. For example, the standby volume can describe the structure and mappings of the replicated data (eg, with reference to an updated manifest), such that the failback volume allows the data, once replicated, to be transferred back to the first geographic area from the second geographic area. Once copied to a geographic area, it can be configured to be restored with new mappings.

In one example, flow 900 includes an act 908 of the computer system hydrating a failback volume in a first geographic area. In some embodiments, hydrating the failback volume is replicated to a first geographic area (eg, first data center 120 of FIG. 1 ) so that the failback volume can resume input-output operations. This may include restoring data. Restoring the duplicated data may include copying the data from the second geographic area to the first geographic area. Restoring replicated data also includes data from mappings from asynchronous replication iterations (e.g., those created by delta application) to new mappings corresponding to new data locations within the failback convex volume system. may include mapping back to .

In one example, flow 900 includes act 910 where the computer system initiates input-output operations in a first geographic area. Input-output operations describe accessing, storing, and modifying data using a failback volume. As part of system recovery, a low RTO and low RPO can improve system performance. For at least this reason, in some embodiments, input-output operations may be started while data recovery of operation 908 is in progress. For example, read-write operations may begin before the copied data is completely copied from the second geographic area back to the first geographic area. In some embodiments, copy over and remapping can be fully completed before input-output operations begin on the failback volume.

10 illustrates an example flow 1000 for creating a failback volume in accordance with one or more embodiments. The operations of flow 1000 may be stored as computer-readable instructions on a non-transitory computer-readable medium of a computer system, such as asynchronous replication system 140 of FIG. 1, and/or implemented as hardware circuitry. can As implemented, instructions represent modules containing code or circuitry executable by processor(s) of a computer system. Execution of these instructions configures the computer system to perform certain operations described herein. Each piece of circuitry or code in conjunction with the processor performs a separate operation(s). Although the actions are illustrated in a specific order, it should be understood that the specific order is not essential and that one or more actions may be omitted, skipped, and/or rearranged.

As described in more detail with reference to FIG. 6 , in some embodiments the asynchronous replication may include a standby volume at the second geographic location, in addition to or alternatively to the object store of replicated data. there is. Rather than converting the data described by the deltas into chunk objects (e.g., as described with reference to data transformation 146 of FIG. 1), the standby volume is a block from the block volume system to the standby volume. This may include direct transmission of data. In some embodiments, the standby volume may not be visible to the user of the block volume system (e.g., may not be available for input-output operations), but RTO in case of an outage affecting the block volume system. and keep block data updated to potentially minimize RPO. In some embodiments, the standby volume can interact with the object store in the destination area, for example by caching data from deltas while also implementing checkpoint operations on the object store data.

In one example, flow 1000 includes an act 1002 of the computer system generating a delta from a block volume system in a first geographic region. A delta represents modifications to data stored in a block volume system between a first snapshot at a first logical time and a second snapshot at a second logical time, as described in more detail with reference to FIG. 2 . can

In one example, flow 1000 includes an act 1004 of the computer system creating a snapshot of the standby volume system in the second geographic region. In some embodiments, creating a snapshot of the standby volume may precede updating the standby volume. As described in detail with reference to FIG. 1 , creating a snapshot may include creating a plurality of block images of blocks constituting a standby volume in units of partitions. Once assembled, these images can describe the state of the data in the standby volume at the logical time corresponding to the creation of the snapshot (eg, as described with reference to the two-phase commit protocols above). Snapshots can provide a fallback location for an asynchronous replication system to use when restoring a block volume system. For example, if a restore request is received while the standby volume is being updated with new deltas, asynchronous replication can use the snapshot to create the restore volume.

In one example, flow 1000 includes act 1006 where the computer system applies the delta to the standby volume system. In contrast to the approach described with reference to data replication using an object store, the standby volume can be updated by applying the deltas directly. For example, as described in more detail with reference to FIG. 2, deltas include metadata (e.g., block identifiers) that describe locations within memory of a block volume system for data that is copied as a result of modifications. can include In this way, modifications can be made to the standby volume by applying the indicated modifications from delta.

In one example, flow 1000 includes an act 1008 of the computer system receiving a failback request. As described above, a failback request may be received after a failure or other disruption of operation in the block volume system, so that the standby system may be used to create a failback system after the disruption is resolved.

In one example, flow 1000 includes act 1010 where the computer system clones the standby volume. As described in more detail with reference to FIG. 5, cloning the standby volume (eg, operation 524 of FIG. 5) may (eg, with respect to size, partitions, etc.) Creating a failback volume (eg, failback volume 562 in FIG. 5 ) in a first geographic area (eg, first data center 120 in FIG. 1 ) that can reproduce the structure of can include

In one example, flow 1000 includes an act 1012 of the computer system restoring the replicated data to a failback volume in the first geographic region. As described above, restoring replicated data involves copying block data from the second geographic area to the first geographic area (also referred to as “hydrating”) and copying the block data to a failback volume. may include mapping. In some implementations, input-output operations can be initiated on the failback volume once the replicated data is restored and/or while restoring the replicated data.

11 illustrates an example flow 1100 for creating a failback volume in accordance with one or more embodiments. The operations of flow 1100 may be implemented as hardware circuitry and/or stored as computer-readable instructions on a non-transitory computer-readable medium of a computer system, such as asynchronous replication system 140 of FIG. 1 . can As implemented, instructions represent modules containing code or circuitry executable by processor(s) of a computer system. Execution of these instructions configures the computer system to perform certain operations described herein. Each piece of circuitry or code in conjunction with the processor performs a separate operation(s). Although the actions are illustrated in a specific order, it should be understood that the specific order is not essential and that one or more actions may be omitted, skipped, and/or rearranged.

In one example, flow 1100 includes an act 1102 of the computer system receiving a resize request in a first geographic area. For example, during operation of the asynchronous replication system between iterations of snapshot creation or during delta creation, the block volume system in the first geographic region may receive a resizing request. Because a resize request may involve adding and/or removing one or more partitions, or changing the size of one or more partitions of a block volume system, the deltas created before resizing are the same as those of the standby volume after it has been resized. may not be compatible with

In one example, flow 1100 includes an act 1104 of the computer system generating a final delta in a first geographic area. In some embodiments, the asynchronous replication system may implement a modified replication protocol in response to receiving a request to resize the block volume system. For example, rather than resizing the block volume system immediately upon receipt of a resize request, another snapshot may be created, and one or more final deltas may be created. This approach may enable the replicated data to reflect the updated state of the block volume system prior to resizing, which in turn may reduce the RPO of the restore volume created from the replicated data.

In one example, flow 1100 includes an act 1106 of the computer system applying the last delta to the standby volume in the second geographic region. As described above, applying the last delta may include modifying the standby volume to reflect modifications to the block volume system described by the one or more last deltas generated in operation 1104 .

In one example, flow 1100 includes an act 1108 of the computer system resizing a block volume in a first geographic region. Following operation 1104, a resize request may be applied to the block volume system.

In one example, flow 1100 includes an act 1110 of the computer system resizing the standby volume in response to the resizing request. Similarly, following operation 1106, the standby volume may also be resized in a manner responsive to the resize request. Because asynchronous replication can create partition-specific deltas, the standby volume can be structured to reproduce the structure of a block volume system. For example, a standby volume can contain the same number of partitions of the same size as a block volume system.

As mentioned above, infrastructure as a service (IaaS) is one specific type of cloud computing. IaaS may be configured to provide virtualized computing resources over a public network (eg, the Internet). In the IaaS model, a cloud computing provider provides infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), distribution software, platform virtualization (e.g., hypervisor layer) , or similar). In some cases, the IaaS provider may also supply various services (eg, billing, monitoring, logging, security, load balancing and clustering, etc.) that accompany these infrastructure components. Thus, because these services can be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

In some instances, an IaaS customer can access resources and services over a wide area network (WAN), such as the Internet, and can use the services of a cloud provider to install the rest of the application stack. . For example, a user creates virtual machines (VMs), installs operating systems (OSs) in each VM, deploys middleware such as databases, and provides information about workloads and backups. It creates storage buckets and can even log in to the IaaS platform to install enterprise software within that VM. Customers can then use the provider's services to perform various functions including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, and the like.

In most cases, the cloud computing model will require the involvement of a cloud provider. A cloud provider may, but need not be, a third party service that specializes in providing (eg, offering, leasing, selling) IaaS. An entity may also choose to become its own provider of infrastructure services by deploying a private cloud.

In some examples, IaaS deployment is the process of deploying a new application or a new version of an application to a prepared application server or similar. This may also include the process of preparing the server (eg, installing libraries, daemons, etc.). It is usually managed by the cloud provider below the hypervisor layer (eg, servers, storage, network hardware, and virtualization). Thus, the customer can be responsible for handling (OS), middleware, and/or application deployment (eg, on self-service virtual machines (eg, that can be launched on demand) or the like).

In some examples, IaaS provisioning can refer to obtaining computers or virtual hosts for use, and even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may have to be performed first.

In some cases, there are two different issues with IaaS provisioning. First, there's the initial problem of having to provision an initial set of infrastructure before anything goes live. Second, there is the problem of evolving the existing infrastructure (eg, adding new services, changing services, removing services, etc.) once everything is provisioned. In some cases, these two The problems can be solved by enabling the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (eg, required components and how they interact) may be defined by one or more configuration files. Thus, the overall topology of an infrastructure (eg, which resources depend on which, and how they each work together) can be described declaratively. In some cases, once the topology is defined, a workflow can be created that creates and/or manages the different components described in the configuration files.

In some examples, an infrastructure may have multiple interconnected elements. For example, there may be one or more virtual private clouds (VPCs), also known as core networks (eg, configurable and/or potentially on-demand pools of shared computing resources). In some examples, there may also be one or more security group rules provisioned to define one or more Virtual Machines (VMs) and how the security of the network will be set up. Other infrastructure elements such as load balancers, databases, or the like may also be provisioned. The infrastructure may evolve incrementally as more and more infrastructure elements are desired and/or added.

In some cases, continuous deployment techniques may be used to enable deployment of infrastructure code across a variety of virtualized computing environments. Additionally, the described technologies may enable infrastructure management within these environments. In some examples, service teams may write code that is desired to be distributed to one or more but often many different production environments (eg, across a variety of different geographic locations, sometimes across the world). However, in some examples, the infrastructure on which the code will be deployed must first be set up. In some cases, provisioning can be done manually, a provisioning tool can be used to provision resources and/or a deployment tool can be used to deploy code once the infrastructure is provisioned.

12 is a block diagram 1200 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service operators 1202 can be communicatively coupled to a secure host subnet 1208 and secure host tenancy 1204 , which can include a virtual cloud network (VCN) 1206 . . In some examples, service operators 1202 run software, such as Microsoft Windows Mobile® and/or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, and the like, and use the Internet, Portable handheld devices (e.g., iPhone®, cellular telephones, iPad®, using one or more client computing devices, which may be computing tablets, personal digital assistants (PDAs)) or wearable devices (eg, Google Glass® head mounted displays); There may be. Alternatively, the client computing devices may be general purpose personal computers including, for example, personal computers and/or laptop computers running various versions of the Microsoft Windows®, Apple Macintosh®, and/or Linux operating systems. Client computing devices are workstations running any of a variety of commercially-available UNIX® or UNIX-like operating systems, including, but not limited to, various GNU/Linux operating systems such as, for example, Google Chrome OS. could be computers. Alternatively or additionally, the client computing devices can be any other electronic device capable of communicating over a network and/or the Internet that can access VCN 1206, such as a thin-client computer, an Internet- capable gaming system (eg, a Microsoft Xbox gaming console with or without a Kinect® gesture input device), and/or a personal messaging device.

VCN 1206 is a local peering gateway (LPG) that can be communicatively coupled to secure shell (SSH) VCN 1212 via LPG 1210 included in SSH VCN 1212. (1210). SSH VCN 1212 may include an SSH subnet 1214, and SSH VCN 1212 may be communicatively coupled to control plane VCN 1216 via LPG 1210 included in control plane VCN 1216. can Additionally, SSH VCN 1212 can be communicatively coupled to data plane VCN 1218 via LPG 1210 . Control plane VCN 1216 and data plane VCN 1218 may be included in service tenancy 1219, which may be owned and/or operated by an IaaS provider.

The control plane VCN 1216 may include a control plane demilitarized zone (DMZ) layer 1220 that serves as a perimeter network (eg, portions of the corporate network between the corporate intranet and external networks). . DMZ-based servers can have limited responsibilities and can help contain security breaches. Additionally, DMZ layer 1220 includes control plane app layer 1224, which may include one or more load balancer (LB) subnet(s) 1222, app subnet(s) 1226, database ;DB) control plane data layer 1228, which may include subnet(s) 1230 (eg, frontend DB subnet(s) and/or backend DB subnet(s)). The LB subnet(s) 1222 included in the control plane DMZ layer 1220 is an internet gateway that can be included in the app subnet(s) 1226 included in the control plane app layer 1224 and the control plane VCN 1216 1234, the app subnet(s) 1226 may include DB subnet(s) 1230 included in the control plane data layer 1228, services gateway 1236 and network address translation. (network address translation; NAT) gateway 1238. The control plane VCN 1216 may include a service gateway 1236 and a NAT gateway 1238 .

Control plane VCN 1216 may include data plane mirror app layer 1240 , which may include app subnet(s) 1226 . The app subnet(s) 1226 included in the data plane mirror app layer 1240 can include a virtual network interface controller (VNIC) 1242 on which compute instances 1244 can run. The compute instance 1244 will communicatively couple the app subnet(s) 1226 of the data plane mirror app layer 1240 to the app subnet(s) 1226 that can be included in the data plane app layer 1246. can

The data plane VCN 1218 may include a data plane App layer 1246 , a data plane DMZ layer 1248 , and a data plane data layer 1250 . Data plane DMZ layer 1248 includes LB subnet(s) that can be communicatively coupled to App subnet(s) 1226 of data plane App layer 1246 and Internet gateway 1234 of data plane VCN 1218. ) 1222 may be included. App subnet(s) 1226 can be communicatively coupled to service gateway 1236 of data plane VCN 1218 and NAT gateway 1238 of data plane VCN 1218 . Data plane data layer 1250 can also include DB subnet(s) 1230 that can be communicatively coupled to App subnet(s) 1226 of data plane App layer 1246 .

Internet gateways 1234 of control plane VCN 1216 and data plane VCN 1218 can be communicatively coupled to metadata management service 1252 , which can be communicatively coupled to public Internet 1254 . Public Internet 1254 can be communicatively coupled to NAT gateway 1238 of control plane VCN 1216 and data plane VCN 1218 . The service gateway 1236 of the control plane VCN 1216 and data plane VCN 1218 can be communicatively coupled to cloud services 1256 .

In some examples, service gateway 1236 of control plane VCN 1216 or data plane VCN 1218 provides an application programming interface (API) to cloud services 1256 that do not go over public internet 1254. You can make calls. API calls from service gateway 1236 to cloud services 1256 can be one-way: service gateway 1236 can make calls to cloud services 1256, and cloud services 1256 can make calls to cloud services 1256. The requested data may be transmitted to the gateway 1236. However, cloud services 1256 may not initiate API calls to service gateway 1236 .

In some examples, secure host tenancy 1204 can be directly linked to an otherwise isolated service tenancy 1219 . Secure host subnet 1208 may communicate with SSH subnet 1214 via LPG 1210, which may enable two-way communication over an otherwise isolated system. Connecting the secure host subnet 1208 to the SSH subnet 1214 can provide secure host subnet 1208 access to other entities within the service tenancy 1219 .

Control plane VCN 1216 may enable users of service tenancy 1219 to set up or otherwise provision desired resources. Desired resources provisioned in the control plane VCN 1216 may be distributed or otherwise used in the data plane VCN 1218. In some examples, the control plane VCN 1216 can be isolated from the data plane VCN 1218, and the data plane mirror app layer 1240 of the control plane VCN 1216 includes the data plane mirror app layer 1240 and data plane VCN 1216. It can communicate with data plane app layer 1246 of data plane VCN 1218 via VNICs 1242 , which can be included in plane app layer 1246 .

In some examples, users or customers of the system may send requests, e.g., create, read, update, or delete (create, read, update, or delete (CRUD) operations. The metadata management service 1252 can communicate the request to the control plane VCN 1216 via the internet gateway 1234 . The request may be received by the LB subnet(s) 1222 included in the control plane DMZ layer 1220. LB subnet(s) 1222 may determine that the request is valid, and in response to such determination, LB subnet(s) 1222 may send the request to the app subnet(s) included in control plane app layer 1224. It can be sent to (1226). If the request is verified and requires a call to the public internet 1254, the call to the public internet 1254 can be sent to the NAT gateway 1238, which can make the call to the public internet 1254. Memory that may be desired to be stored by request may be stored in DB subnet(s) 1230 .

In some examples, data plane mirror app layer 1240 can enable direct communication between control plane VCN 1216 and data plane VCN 1218 . For example, configuration changes, updates, or other suitable modifications may be desired to be applied to resources included in data plane VCN 1218. Via VNIC 1242, control plane VCN 1216 can communicate directly with resources included in data plane VCN 1218, thereby making changes to the configuration on resources included in data plane VCN 1218. , updates or other appropriate modifications.

In some embodiments, control plane VCN 1216 and data plane VCN 1218 may be included in service tenancy 1219 . In this case, a user or customer of the system may not own or operate the control plane VCN 1216 or the data plane VCN 1218. Instead, an IaaS provider may own or operate a control plane VCN 1216 and a data plane VCN 1218 , both of which may be included in service tenancy 1219 . Such an embodiment may enable isolation of networks that may prevent users or customers from interacting with other users' or other customers' resources. Further, this embodiment may enable users or customers of the system to store databases privately without having to rely on the public internet 1254, which may not have the desired level of security for storage.

In other embodiments, LB subnet(s) 1222 included in control plane VCN 1216 may be configured to receive signals from service gateway 1236 . In this embodiment, the control plane VCN 1216 and data plane VCN 1218 may be configured to be called by customers of the IaaS provider without calling the public internet 1254. Because the database(s) used by customers can be controlled by the IaaS provider and stored on a service tenancy 1219 that can be isolated from the public Internet 1254, customers of the IaaS provider may use this embodiment. you can hope

13 is a block diagram 1300 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operators 1302 (e.g., service operators 1202 of FIG. 12) may include a virtual cloud network (VCN) 1306 (e.g., VCN 1206 of FIG. 12) and a secure host subnet ( 1308) (e.g., secure host subnet 1208 of FIG. 12) communicatively coupling to secure host tenancy 1304 (e.g., secure host tenancy 1204 of FIG. 12). It can be. The VCN 1306 is configured to communicate with a secure shell (SSH) VCN 1312 (eg, the SSH VCN 1212 of FIG. 12) through the LPG 1310 included in the SSH VCN 1312. and a local peering gateway (LPG) 1310 (eg, LPG 1210 in FIG. 12 ) to which it may be coupled. SSH VCN 1312 can include SSH subnet 1314 (eg, SSH subnet 1214 in FIG. 12 ), and SSH VCN 1312 can include LPG 1310 included in control plane VCN 1316 may be communicatively coupled to control plane VCN 1316 (eg, control plane VCN 1216 of FIG. 12 ) via Control plane VCN 1316 can be included in service tenancy 1319 (eg, service tenancy 1219 of FIG. 12 ), and data plane VCN 1318 (eg, data plane VCN of FIG. 12 ). 1218) may be included in customer tenancy 1321, which may be owned or operated by users or customers of the system.

The control plane VCN 1316 includes a control plane DMZ layer 1320 (eg, LB subnet(s) 1322 (eg, LB subnet(s) 1222 in FIG. 12 )). Control plane app layer 1324 (which may include control plane DMZ layer 1220 of FIG. 12), app subnet(s) 1326 (e.g., app subnet(s) 1226 of FIG. 12) 12), database (DB) subnet(s) 1330 (eg, similar to DB subnet(s) 1230 in FIG. 12). control plane data layer 1328 (eg, control plane data layer 1228 of FIG. 12 ) that can be The LB subnet(s) 1322 included in the control plane DMZ layer 1320 is an internet gateway that can be included in the app subnet(s) 1326 included in the control plane app layer 1324 and the control plane VCN 1316 1334 (e.g., Internet gateway 1234 of FIG. 12), and app subnet(s) 1326 can be communicatively coupled to DB subnet(s) included in control plane data layer 1328. 1330, to communicate with a service gateway 1336 (e.g., the service gateway of FIG. 12) and a network address translation (NAT) gateway 1338 (e.g., the NAT gateway 1238 of FIG. 12). can be combined The control plane VCN 1316 may include a service gateway 1336 and a NAT gateway 1338 .

Control plane VCN 1316 may include data plane mirror app layer 1340 (eg, data plane mirror app layer 1240 in FIG. 12 ), which may include app subnet(s) 1326 . . The app subnet(s) 1326 included in the data plane mirror app layer 1340 are virtual network interface controllers that can run compute instances 1344 (eg, similar to compute instances 1244 in FIG. 12 ). (VNIC) 1342 (eg, VNIC 1242). Compute instance 1344 is connected to data plane app layer 1346 via VNIC 1342 included in data plane mirror app layer 1340 and VNIC 1342 included in data plane app layer 1346 (e.g. , may enable communication between the app subnet(s) 1326 that may be included in the data plane app layer 1246 of FIG. 12 and the app subnet(s) 1326 of the data plane mirror app layer 1340. there is.

An internet gateway 1334 included in the control plane VCN 1316 is a metadata management service 1352 that can be communicatively coupled to the public internet 1354 (e.g., public internet 1254 in FIG. 12). (eg, metadata management service 1252 of FIG. 12). Public Internet 1354 may be communicatively coupled to NAT gateway 1338 of control plane VCN 1316 . A service gateway 1336 included in control plane VCN 1316 can be communicatively coupled to cloud services 1356 (eg, cloud services 1256 in FIG. 12 ).

In some examples, data plane VCN 1318 may be included in customer tenancy 1321 . In this case, the IaaS provider may provide a control plane VCN 1316 for each customer, and the IaaS provider may provide, for each customer, a unique compute instance 1344 included in the service tenancy 1319. can be set up. Each compute instance 1344 can facilitate communication between the control plane VCN 1316 included in the service tenancy 1319 and the data plane VCN 1318 included in the customer tenancy 1321 . Compute instance 1344 enables resources provisioned in control plane VCN 1316 included in service tenancy 1319 to be deployed or otherwise used in data plane VCN 1318 included in customer tenancy 1321 can do

In other examples, a customer of an IaaS provider may have databases that live in customer tenancy 1321 . In this example, control plane VCN 1316 can include data plane mirror app layer 1340 , which can include app subnet(s) 1326 . Data plane mirror app layer 1340 may reside in data plane VCN 1318 , but data plane mirror app layer 1340 may not reside in data plane VCN 1318 . That is, the data plane mirror app layer 1340 may have access to the customer tenancy 1321, but the data plane mirror app layer 1340 may not exist in the data plane VCN 1318 or the IaaS provider's May not be owned or operated by the customer. The data plane mirror app layer 1340 may be configured to make calls to the data plane VCN 1318, but may not be configured to make calls to entities included in the control plane VCN 1316. A customer may wish to distribute or otherwise use resources in the data plane VCN 1318 provisioned in the control plane VCN 1316, and the data plane mirror app layer 1340 may wish to distribute or otherwise use the customer's desired distribution or other resources of the customer's resources. use can be made possible.

In some embodiments, a customer of an IaaS provider may apply filters to the data plane VCN 1318. In this embodiment, the customer can determine what the data plane VCN 1318 can access, and the customer can restrict access to the public internet 1354 from the data plane VCN 1318. The IaaS provider may not be able to apply filters or otherwise control the data plane VCN 1318's access to any external networks or databases. Applying filters and controls by the customer on the data plane VCN 1318 included in the customer tenancy 1321 can help isolate the data plane VCN 1318 from other customers and the public Internet 1354 .

In some embodiments, cloud services 1356 are used to access services that may not exist on the public Internet 1354, on the control plane VCN 1316, or on the data plane VCN 1318. Can be called by service gateway 1336. The connection between cloud services 1356 and control plane VCN 1316 or data plane VCN 1318 may not be live or may not be continuous. Cloud services 1356 may exist on different networks owned or operated by the IaaS provider. Cloud services 1356 can be configured to receive calls from service gateway 1336 and not to receive calls from public internet 1354 . Some cloud services 1356 may be isolated from other cloud services 1356, and control plane VCN 1316 may be isolated from cloud services 1356, which may not be in the same region as control plane VCN 1316. can be isolated. For example, the control plane VCN 1316 may be located in “region 1” and the cloud service “distribution 12” may be located in region 1 and “region 2”. If a call to distribution 12 is made by service gateway 1336 included in control plane VCN 1316 located in area 1, the call may be sent to distribution 12 in area 1. In this example, distribution 12 in region 1 or control plane VCN 1316 may be communicatively coupled or otherwise not in communication with distribution 12 in region 2.

14 is a block diagram 1400 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operators 1402 (eg, service operators 1202 of FIG. 12 ) operate on a virtual cloud network (VCN) 1406 (eg, VCN 1206 of FIG. 12 ) and secure host subnets ( 1408) (e.g., secure host subnet 1208 of FIG. 12) communicatively coupling to secure host tenancy 1404 (e.g., secure host tenancy 1204 of FIG. 12). It can be. VCN 1406 is an LPG 1410 that can be communicatively coupled to SSH VCN 1412 (e.g., SSH VCN 1212 in FIG. 12) via LPG 1410 included in SSH VCN 1412. ) (eg, LPG 1210 of FIG. 12). SSH VCN 1412 can include SSH subnet 1414 (eg, SSH subnet 1214 in FIG. 12 ), and SSH VCN 1412 can include LPG 1410 contained in control plane VCN 1416 to the control plane VCN 1416 (e.g., the control plane VCN 1216 of FIG. 12) via the data plane VCN 1418 (e.g., , may be communicatively coupled to the data plane VCN 1218 of FIG. 12. Control plane VCN 1416 and data plane VCN 1418 may be included in service tenancy 1419 (eg, service tenancy 1219 in FIG. 12 ).

Control plane VCN 1416 includes control plane DMZ layer 1420 (which may include load balancer (LB) subnet(s) 1422 (e.g., LB subnet(s) 1222 in FIG. 9)). Controls that may include, for example, control plane DMZ layer 1220 of FIG. 12), app subnet(s) 1426 (e.g., similar to app subnet(s) 1226 of FIG. 12) control plane app layer 1424 (e.g., control plane app layer 1224 in FIG. 12), control plane data layer 1428 (e.g., FIG. of the control plane data layer 1428). The LB subnet(s) 1422 included in the control plane DMZ layer 1420 is an internet gateway that can be included in the app subnet(s) 1426 included in the control plane app layer 1424 and the control plane VCN 1416 1434 (e.g., Internet gateway 1234 of FIG. 12), and app subnet(s) 1426 can be communicatively coupled to DB subnet(s) included in control plane data layer 1428. 1430, to communicate with a service gateway 1436 (e.g., the service gateway of FIG. 12) and a network address translation (NAT) gateway 1438 (e.g., the NAT gateway 1238 of FIG. 12). can be combined The control plane VCN 1416 may include a service gateway 1436 and a NAT gateway 1438 .

Data plane VCN 1418 includes data plane app layer 1446 (e.g., data plane app layer 1246 in FIG. 12), data plane DMZ layer 1448 (e.g., data plane DMZ layer in FIG. 12) 1248), and a data plane data layer 1450 (eg, data plane data layer 1250 of FIG. 12). The data plane DMZ layer 1448 is included in the trusted app subnet(s) 1460 and untrusted app subnet(s) 1462 of the data plane app layer 1446, and the data plane VCN 1418. LB subnet(s) 1422 that can be communicatively coupled to the Internet gateway 1434 of the Internet Gateway 1434. Trusted app subnet(s) 1460 include service gateway 1436 included in data plane VCN 1418, NAT gateway 1438 included in data plane VCN 1418, and data plane data layer 1450 can be communicatively coupled to the DB subnet(s) 1430 included in. Untrusted app subnet(s) 1462 are communicatively coupled to service gateway 1436 included in data plane VCN 1418 and DB subnet(s) 1430 included in data plane data layer 1450. It can be. Data plane data layer 1450 can include DB subnet(s) 1430 that can be communicatively coupled to service gateways 1436 included in data plane VCN 1418 .

The untrusted app subnet(s) 1462 are one or more primary VNICs 1464(1) that can be communicatively coupled to tenant virtual machines (VMs) 1466(1)-(N). )-(N)). Each tenant VM 1466(1)-(N) has separate container egress VCNs 1468(1)-(N), which may be included in respective customer tenancies 1470(1)-(N). may be communicatively coupled to individual app subnets 1467(1)-(N), which may be included in Individual secondary VNICs 1472(1)-(N) include untrusted app subnet(s) 1462 contained in data plane VCN 1418 and container egress VCNs 1468(1)-(N) ) may enable communication between app subnets included in the subnet. Each container egress VCN 1468(1)-(N) has a NAT gateway 1438 that can be communicatively coupled to public internet 1454 (e.g., public internet 1254 in FIG. 12). can include

Internet gateway 1434, included in control plane VCN 1416 and included in data plane VCN 1418, provides metadata management services 1452 (e.g., It can be communicatively coupled to the metadata management system 1252 of FIG. 12 . Public Internet 1454 is included in control plane VCN 1416 and can be communicatively coupled to NAT gateway 1438 included in data plane VCN 1418 . A service gateway 1436 included in control plane VCN 1416 and included in data plane VCN 1418 can be communicatively coupled to cloud services 1456 .

In some embodiments, data plane VCN 1418 may be integrated with customer tenancies 1470 . Such integration may be useful or desirable for the IaaS provider's customers in some cases, such as where they may wish to assist when executing code. A customer may provide code to be executed, which may be disruptive or may communicate with other customer resources or cause other undesirable effects. In response, the IaaS provider may decide whether to execute code given to the IaaS provider by the customer.

In some examples, a customer of the IaaS provider may grant temporary network access to the IaaS provider and may request functionality to be attached to the data plane layer app 1446 . Code to execute a function may be executed in VMs 1466(1)-(N), and the code may not be configured to run anywhere on data plane VCN 1418. Each VM 1466(1)-(N) may be associated with one customer tenancy 1470. Individual containers 1471(1)-(N) included in VMs 1466(1)-(N) may be configured to execute code. In this case, there may be double isolation (e.g., containers 1471(1)-(N) executing code, where containers 1471(1)-(N) are untrusted apps). At least one VM (which may be included in 1466(1)-(N)) included in subnet(s) 1462, which may contain incorrect or otherwise undesirable code that may compromise the IaaS provider's network or It can help prevent compromising your customer's network. Containers 1471(1)-(N) can be communicatively coupled to customer tenancy 1470 and can be configured to transmit data to or receive data from customer tenancy 1470. . Containers 1471(1)-(N) may not be configured to transmit data to or receive data from any other entity within data plane VCN 1418. Upon completion of code execution, the IaaS provider may kill or otherwise discard the containers 1471(1)-(N).

In some embodiments, Trusted App subnet(s) 1460 may execute code that may be owned or operated by an IaaS provider. In this embodiment, trusted app subnet(s) 1460 can be communicatively coupled to DB subnet(s) 1430 and will be configured to execute CRUD operations on DB subnet(s) 1430. can Untrusted app subnet(s) 1462 can be communicatively coupled to DB subnet(s) 1430, but in this embodiment, untrusted app subnet(s) are DB subnet(s) 1430 ) to execute read operations. Containers 1471(1)-(N), which can be included in each customer's VM 1466(1)-(N) and can execute code from the customer, communicate with the DB subnet(s) 1430. Possibly uncoupled.

In other embodiments, control plane VCN 1416 and data plane VCN 1418 may not be directly communicatively coupled. In such an embodiment, there may not be direct communication between the control plane VCN 1416 and the data plane VCN 1418. However, communication may occur indirectly through at least one method. An LPG 1410 may be established by an IaaS provider that may enable communication between the control plane VCN 1416 and the data plane VCN 1418. In another example, control plane VCN 1416 or data plane VCN 1418 can make calls to cloud services 1456 via service gateway 1436 . For example, a call from control plane VCN 1416 to cloud services 1456 may include a request for a service that can communicate with data plane VCN 1418 .

15 is a block diagram 1500 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operators 1502 (e.g., service operators 1202 of FIG. 12) may include a virtual cloud network (VCN) 1506 (e.g., VCN 1206 of FIG. 12) and secure host subnets ( 1508) (e.g., secure host subnet 1208 of FIG. 12) communicatively coupling to secure host tenancy 1504 (e.g., secure host tenancy 1204 of FIG. 12) It can be. VCN 1506 is an LPG 1510 that can be communicatively coupled to SSH VCN 1512 (e.g., SSH VCN 1212 in FIG. 12) via LPG 1510 included in SSH VCN 1512. ) (eg, LPG 1210 of FIG. 12). SSH VCN 1512 can include SSH subnet 1514 (eg, SSH subnet 1214 in FIG. 12 ), and SSH VCN 1512 can include LPG 1510 included in control plane VCN 1516 to the control plane VCN 1516 (e.g., the control plane VCN 1216 of FIG. 12) via the data plane VCN 1518 (e.g., , can be communicatively coupled to the data plane VCN 1218 of FIG. Control plane VCN 1516 and data plane VCN 1518 may be included in service tenancy 1519 (eg, service tenancy 1219 in FIG. 12 ).

Control plane VCN 1516 includes control plane DMZ layer 1520 (e.g., LB subnet(s) 1522 (e.g., LB subnet(s) 1222 in FIG. 12)). Control plane app layer 1524 (which may include control plane DMZ layer 1220 of FIG. 12 ), app subnet(s) 1526 (e.g., app subnet(s) 1226 of FIG. 12 ) Control plane data layer, which may include, for example, control plane app layer 1224 in FIG. 12), DB subnet(s) 1530 (eg, DB subnet(s) 1430 in FIG. 14) 1528 (eg, control plane data layer 1228 in FIG. 12). The LB subnet(s) 1522 included in the control plane DMZ layer 1520 are Internet gateways that can be included in the App subnet(s) 1526 included in the control plane App layer 1524 and the control plane VCN 1516 1534 (e.g., Internet gateway 1234 of FIG. 12), and app subnet(s) 1526 can be communicatively coupled to DB subnet(s) included in control plane data layer 1528. 1530, to communicate with a service gateway 1536 (e.g., the service gateway of FIG. 12) and a network address translation (NAT) gateway 1538 (e.g., the NAT gateway 1238 of FIG. 12). can be combined Control plane VCN 1516 may include service gateway 1536 and NAT gateway 1538 .

Data plane VCN 1518 includes data plane app layer 1546 (e.g., data plane app layer 1246 in FIG. 12), data plane DMZ layer 1548 (e.g., data plane DMZ layer in FIG. 12) 1248), and a data plane data layer 1550 (eg, data plane data layer 1250 of FIG. 12). The data plane DMZ layer 1548 connects the trusted app subnet(s) 1560 (e.g., the trusted app subnet(s) 1460 of FIG. 14) of the data plane app layer 1546. Communicate with untrusted app subnet(s) 1562 (e.g., untrusted app subnet(s) 1462 in FIG. 14), and internet gateway 1534 included in data plane VCN 1518. LB subnet(s) 1522 that can be combined. Trusted app subnet(s) 1560 include service gateway 1536 included in data plane VCN 1518, NAT gateway 1538 included in data plane VCN 1518, and data plane data layer 1550 can be communicatively coupled to the DB subnet(s) 1530 included in Untrusted app subnet(s) 1562 are communicatively coupled to service gateway 1536 included in data plane VCN 1518 and DB subnet(s) 1530 included in data plane data layer 1550. It can be. Data plane data layer 1550 can include DB subnet(s) 1530 that can be communicatively coupled to service gateways 1536 included in data plane VCN 1518 .

Untrusted app subnet(s) 1562 will be communicatively coupled to tenant virtual machines (VMs) 1566(1)-(N) residing within untrusted app subnet(s) 1562. primary VNICs 1564(1)-(N) that can be Each tenant VM 1566(1)-(N) can run code in a separate container 1567(1)-(N), and the data plane app layer 1546 can be included in the container exit VCN 1568. ), which can be communicatively coupled to the app subnet 1526. Individual secondary VNICs 1572(1)-(N) are provided between the untrusted app subnet(s) 1562 contained in data plane VCN 1518 and the app subnet contained in container egress VCNs 1568. can enable communication. The container egress VCN can include a NAT gateway 1538 that can be communicatively coupled to public internet 1554 (eg, public internet 1254 in FIG. 12 ).

Internet gateway 1534, included in control plane VCN 1516 and included in data plane VCN 1518, provides metadata management services 1552 (e.g., It can be communicatively coupled to the metadata management system 1252 of FIG. 12 . Public Internet 1554 is included in control plane VCN 1516 and can be communicatively coupled to NAT gateway 1538 included in data plane VCN 1518 . A service gateway 1536 included in control plane VCN 1516 and included in data plane VCN 1518 can be communicatively coupled to cloud services 1556 .

In some examples, the pattern illustrated by the architecture of block diagram 1500 of FIG. 15 can be considered an exception to the pattern illustrated by the architecture of block diagram 1400 of FIG. This may be desirable for customers of IaaS providers when they are unable to communicate over the Internet (e.g., in an isolated area). Individual containers 1567(1)-(N) contained in VMs 1566(1)-(N) for each customer may be accessed in real time by the customer. Containers 1567(1)-(N) are individual secondary VNICs 1572 included in app subnet(s) 1526 of data panel app layer 1546, which may be included in container egress VCN 1568. It can be configured to make calls to (1)-(N)). Secondary VNICs 1572(1)-(N) can send calls to NAT gateway 1538, which can send calls to public internet 1554. In this example, containers 1567(1)-(N) that may be accessed in real time by the customer may be isolated from control plane VCN 1516 and other entities included in data plane VCN 1518. can be isolated from Containers 1567(1)-(N) may also be isolated from other customers' resources.

In other examples, a customer may use containers 1567(1)-(N) to invoke cloud services 1556. In this example, a customer may execute code in containers 1567(1)-(N) requesting service from cloud services 1556. Containers 1567(1)-(N) are routed to secondary VNICs 1572(1)-(N) that can send requests to a NAT gateway that can send requests to the public internet 1554. You can send these requests. Public Internet 1554 may send a request to LB subset(s) 1522 included in control plane VCN 1516 via Internet gateway 1534 . In response to determining that the request is valid, the LB subnet(s) will send a request to the app subnet(s) 1526, which can send requests to cloud services 1556 via the service gateway 1536. can

It should be understood that the IaaS architectures 1200, 1300, 1400, and 1500 shown in the figures may have other components than those shown. Additionally, the embodiments shown in the figures are just some examples of cloud infrastructure systems that may incorporate an embodiment of the present invention. In some other embodiments, IaaS systems may have more or fewer components than shown in the drawings, may combine two or more components, or may have a different configuration or arrangement of components. can have

In certain embodiments, the IaaS systems described herein are self-service, subscription-based, elastically scalable, reliable, highly available, and , a suite of applications, middleware, and database service offerings that are delivered to customers in a secure manner. One example of such an IaaS system is Oracle Cloud Infrastructure (OCI) provided by the assignee.

16 illustrates an example computer system 1600 that may be implemented in various embodiments of the present disclosure. System 1600 can be used to implement any of the computer systems described above. As shown in the figure, computer system 1600 includes a processing unit 1604 that communicates with a plurality of peripheral subsystems via a bus subsystem 1602. These peripheral subsystems may include processing acceleration unit 1606 , I/O subsystem 1608 , storage subsystem 1618 and communications subsystem 1624 . Storage subsystem 1618 includes tangible computer-readable storage media 1622 and system memory 1610 .

Bus subsystem 1602 provides a mechanism for enabling the various components and subsystems of computer system 1600 to communicate with each other as intended. Although bus subsystem 1602 is shown schematically as a single bus, alternative embodiments of the bus subsystem may use multiple buses. Bus subsystem 1602 can be any of several types of bus structures, including a memory bus or memory controller using any of a variety of bus architectures, a peripheral bus, and a local bus. For example, these architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Council Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, which may be implemented as a Mezzanine bus manufactured to the IEEE P1386.1 standard.

Processing unit 1604, which may be implemented as one or more integrated circuits (eg, a conventional microprocessor or microcontroller) controls the operation of computer system 1600. One or more processors may be included in processing unit 1604 . These processors may include single core or multicore processors. In certain embodiments, processing unit 1604 may be implemented as one or more independent processing units 1632 and/or 1634 with single or multicore processors included within each processing unit. In other embodiments, processing unit 1604 can also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

In various embodiments, processing unit 1604 can execute various programs in response to program code, and can maintain simultaneous multiple execution of programs or processes. At any given point in time, all or part of the program code to be executed may reside within processor(s) 1604 and/or within storage subsystem 1618 . With proper programming, processor(s) 1604 can provide the various functions described above. The computer system 1600 may additionally include a processing acceleration unit 1606, which may include a digital signal processor (DSP), special-purpose processor, and/or the like.

I/O subsystem 1608 can include user interface input devices and user interface output devices. User interface input devices include keyboard, pointing devices such as mouse or trackball, touchpad or touch screen integrated into the display, scroll wheel, click wheel, dial, button, switch, keypad, audio input devices with voice command recognition systems. , microphones, and other types of input devices. User interface input devices enable users to control and interact with an input device, such as a Microsoft Xbox® 360 game controller, through a natural user interface using, for example, gestures and spoken commands. motion sensing and/or gesture recognition devices such as a Microsoft Kinect® motion sensor that User interface input devices may also detect eye activity (eg, 'blinking' while taking a picture and/or selecting a menu) from users and send eye gestures to the input device (eg, For example, eye gesture recognition devices such as a Google Glass® eye blink detector that translates as input into Google Glass®. Additionally, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (eg, Siri® navigator) via voice commands.

User interface input devices also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphics tablets, and audio/visual devices such as speakers, digital cameras, digital camcorders. fields, portable media players, webcams, image scanners, fingerprint scanners, barcode reader 3D scanners, 3D printers, laser rangefinders, and eye tracking devices. Additionally, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, medical ultrasound devices. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, and the like.

User interface output devices may include a display subsystem, indicators, or non-visual displays such as audio output devices, and the like. The display subsystem can be a flat-panel device such as using a cathode ray tube (CRT), liquid crystal display (LCD) or plasma display, projection device, touch screen, and the like. In general, use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information from computer system 1600 to a user or other computer. For example, user interface output devices include, but are not limited to, various display devices such as monitors, printers, speakers, headphones, car navigation systems that visually convey text, graphics and audio/video information. , plotters, audio output devices, and modems.

Computer system 1600 may include a storage subsystem 1618 that includes software elements shown as being currently located within system memory 1610 . System memory 1610 may store executable program instructions that are loadable onto processing unit 1604 as well as data generated during execution of such programs.

Depending on the configuration and type of computer system 1600, system memory 1610 may be volatile (such as random access memory (RAM)) and/or (read-only memory; ROM), flash memory, etc.) may be non-volatile. RAM typically contains data and/or program modules that are currently being executed and operated upon by processing unit 1604 and/or are immediately accessible to it. In some implementations, system memory 1610 may include multiple different types of memories, such as static random access memory (SRAM) or dynamic random access memory (DRAM). can In some implementations, a basic input/output system (BIOS) containing basic routines that help transfer information between elements within computer system 1600, such as during startup, typically resides in ROM. can be stored By way of example and not limitation, system memory 1610 may also include client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), and the like. Illustrates application programs 1612 , program data 1614 , and operating system 1616 . By way of example, operating system 1616 may include various versions of Microsoft Windows®, Apple Macintosh®, and/or Linux operating systems, including but not limited to various GNU/Linux operating systems, Google Chrome® OS, and the like. ) various commercially-available UNIX® or UNIX-like operating systems and/or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 16 OS, and Palm® OS operating systems. .

Storage subsystem 1618 may also provide tangible computer-readable storage media for storing basic programming and data constructs that provide functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by a processor, provides the functionality described above may be stored within storage subsystem 1618 . These software modules or instructions may be executed by processing unit 1604 . Storage subsystem 1618 can also provide storage for storing data used in accordance with the present disclosure.

Storage subsystem 1600 may also include computer-readable storage media reader 1620 , which may additionally be coupled to computer-readable storage media 1622 . Along with and optionally in combination with system memory 1610, computer-readable storage medium 1622 may temporarily and/or more permanently contain, store, transmit, and retrieve computer-readable information. Remote, local, fixed, and/or removable storage devices may be generically referred to in addition to storage media for storage.

A computer-readable storage medium 1622 containing code or portions of code may also include, but is not limited to, volatile and non-volatile, removable media implemented in any method or technology for storage and/or transmission of information. and any suitable media known or used in the art including storage media and communication media, such as non-removable media. It is a tangible computer-readable storage medium such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk; DVD), or optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer readable media. It can also include non-tangible computer-readable media, such as data signals, data transmissions, or any other medium that can be used to transmit desired information and that can be accessed by computing system 1600. there is.

By way of example, computer-readable storage medium 1622 may include a hard disk drive that reads from or writes to non-removable nonvolatile magnetic media, a magnetic disk drive that reads from or writes to removable nonvolatile magnetic disks, and an optical disc drive that reads from or writes to removable non-volatile optical discs such as CD ROM, DVD, and Blu-Ray® discs, or other optical media. Computer-readable storage medium 1622 includes, but is not limited to, Zip® drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD discs, digital video tape, and the like. Computer-readable storage medium 1622 may also include non-volatile memory based solid-state drives (SSDs) such as flash-memory based SSDs, enterprise flash drives, solid state ROM, and the like. , volatile memory based SSDs such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM and flash memory based SSDs. can include Disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computer system 1600.

Communications subsystem 1624 provides an interface to other computer systems and networks. Communications subsystem 1624 serves as an interface for transmitting data from computer system 1600 to and receiving data from other systems. For example, communications subsystem 1624 can enable computer system 1600 to connect to one or more devices over the Internet. In some embodiments, communication subsystem 1624 may be configured (e.g., cellular telephone technology, advanced data network technology such as 3G, 4G or enhanced data rates for global evolution (EDGE), WiFi (IEEE 802.11 family of standards) radio frequency (RF) transceiver components for access to wireless voice and/or data networks, satellite positioning systems (using other mobile communication technologies, or any combination thereof) global positioning system (GPS) receiver components, and/or other components. In some embodiments, communication subsystem 1624 can provide a wired network connection (eg, Ethernet) in addition to or instead of a wireless interface.

In some embodiments, communication subsystem 1624 also includes structured and/or unstructured data feeds 1626, event streams 1628 on behalf of one or more users who may use computer system 1600. ), event updates 1630, and the like.

By way of example, communication subsystem 1624 may send real-time data feeds 1626 from users of social networks and/or other communication services such as Twitter® feeds, Facebook® updates, web feeds such as rich site Rich Site Summary (RSS) feeds, and/or real-time updates from one or more third party information sources.

Additionally, communication subsystem 1624 may also include event streams 1628 of real-time events and/or event updates 1630 that may be unlimited or continuous, with no explicit end in nature. may be configured to receive data in the form of continuous data streams that may be Examples of applications that generate continuous data include, for example, sensor data applications, financial tickers, network performance measurement tools (eg network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, and the like.

Communications subsystem 1624 also includes structured and/or unstructured data feeds 1626, event streams 1628, event updates 1630, and the like coupled to computer system 1600. It can be configured to output to one or more databases that can communicate with one or more streaming data source computers.

Computer system 1600 includes a handheld portable device (eg, iPhone® cellular phone, iPad® computing tablet, PDA), wearable device (eg, Google Glass® head-worn display), PC, workstation, It may be any of a variety of types including mainframes, kiosks, server racks, or any other data processing system.

Due to the ever-changing nature of computers and networks, the description of computer system 1600 shown in the figure is intended as a specific example only. Many other configurations with more or fewer components than the system shown in the figures are possible. For example, customized hardware may also be used and/or certain elements may be implemented in hardware, firmware, software (including applets), or combinations thereof. Additionally, connectivity to other computing devices, such as network input/output devices, may be used. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and/or methods for implementing the various embodiments.

Although particular embodiments of this disclosure have been described, various modifications, alternatives, alternative configurations, and equivalents are also encompassed within the scope of this disclosure. Embodiments of the present disclosure are not limited to operating within precise particular data processing environments, but are free to operate within a plurality of data processing environments. Additionally, although embodiments of the present disclosure have been described using a specific series of transactions and steps, it will be apparent to those skilled in the art that the scope of the present disclosure is not limited to the described series of transactions and steps. Various features and aspects of the embodiments described above may be used individually or together.

Additionally, although embodiments of the present disclosure have been described using a specific combination of hardware and software, it should be understood that other combinations of hardware and software are also within the scope of the present disclosure. Embodiments of the present disclosure may be implemented using only hardware, or only software, or combinations thereof. The various processes described herein may be implemented on the same processor or different processors in any combination. Thus, where components or modules are described as being configured to perform particular operations, such configuration may include programmable electronic circuits (eg, by designing electronic circuits to perform the operations) to perform the operations. microprocessors), or any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional techniques for inter-process communication, different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times. can be used

Accordingly, the present specification and drawings are to be regarded in an illustrative rather than a restrictive sense. However, it will be apparent that additions, substitutions, deletions, and other modifications and changes may be made thereto without departing from the broad spirit and scope as set forth in the claims. Thus, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

Use of the terms "a and an" and "the" and similar referents in the context of describing disclosed embodiments (particularly in the context of the claims that follow) may be used herein, as otherwise indicated or in context. should be construed to cover both the singular and the plural unless clearly contradicted by The terms “comprising,” “having,” “comprising,” and “including” are to be interpreted as open-ended terms (ie, “including but not limited to”) unless otherwise stated. The term "linked to" should be construed as partially or wholly contained within, attached to, or bonded with something, even if there are any inclusions. Recitation of ranges of values herein, unless otherwise indicated herein, are merely intended to serve as a shorthand method of referring individually to each individual value falling within the range, each individual value as if it were individually referred to herein. incorporated into the specification as such. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. Any and all examples or use of exemplary language (eg, “such as”) provided herein is merely intended to better illuminate embodiments of the present disclosure, and unless otherwise claimed, the present disclosure do not limit the scope of No language in the specification should be construed as indicating any non-claimed element essential to the practice of the disclosure.

Disjunctive language, such as the phrase "at least one of X, Y, or Z", unless specifically stated otherwise, means that an item, term, etc. is one of X, Y, or Z, or a combination thereof (e.g., X , Y, and/or Z). Accordingly, such disjunctive language is generally not intended to imply that at least one of X, at least one of Y, or at least one of Z is required for each specific embodiment to be present.

Preferred embodiments of the present disclosure are described herein, including the best known mode for carrying out the present disclosure. Variations of these preferred embodiments will become apparent to those skilled in the art upon reading the above description. Skilled artisans should be able to employ such variations as appropriate, and the present disclosure may be practiced otherwise than specifically described herein. Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Also, any combination of the elements described above in all possible variations thereof is thus encompassed by this disclosure unless otherwise indicated herein.

All references, including publications, patent applications and patents, cited herein are individually and specifically indicated to be incorporated by reference, and each reference is hereby incorporated by reference to the same extent as if it were set forth herein in its entirety. included in

Although aspects of the present disclosure have been described in the above specification with reference to specific embodiments of the present disclosure, those skilled in the art will recognize that the present disclosure is not limited thereto. The various features and aspects of the disclosure described above may be used individually or together. Additionally, the embodiments may be used in any number of environments and applications beyond those described herein without departing from the broad spirit and scope of this disclosure. Accordingly, the present specification and drawings are to be regarded as illustrative rather than restrictive.

The following clauses describe embodiments of the disclosed implementation:

Clause 1: As a method,

creating, by a computer system, a first snapshot of a block volume at a first geographic location at a first logical time, the block volume comprising a plurality of partitions;

sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic area;

creating, by the computer system, a second snapshot of the block volume at the first geographic location and at a second logical time;

generating, by the computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one of the plurality of partitions;

sending, by the computer system, a plurality of delta data sets corresponding to the plurality of deltas to the object storage system in the second geographic region;

generating, by the computer system, a checkpoint at least in part by aggregating object metadata associated with the first snapshot and the plurality of deltas;

receiving, by the computer system, a restore request to create a restore volume; and

creating, by the computer system, the recovery volume from the checkpoint.

Clause 2: The method of clause 1, wherein generating the plurality of deltas comprises:

generating a comparison between the second snapshot and the first snapshot;

based on the comparison, determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot; and

generating the plurality of deltas, the plurality of deltas describing modified data for the plurality of partitions.

Clause 3: The method according to clause 1, wherein creating the first snapshot comprises:

corresponding to a logical time, suspending input/output operations for the plurality of partitions;

generating a plurality of block images describing volume data in the plurality of partitions; and

and enabling input/output operations to the plurality of partitions.

Clause 4: The method according to clause 1, wherein the restore request is a failover request, the method comprising:

enabling the recovery volume to be created in the second geographic region; and

and enabling input/output operations using the recovery volume in the second geographic area.

Clause 5: The method according to clause 1, wherein the restore request is a failback request, and the method comprises:

creating the recovery volume in the second geographic region;

enabling a failback volume to be created in the first geographic region; and

restoring the first snapshot data in the first geographic region.

Clause 6: The method of clause 1, wherein transmitting the plurality of delta data sets comprises:

generating a plurality of chunk objects from the plurality of delta data sets;

transmitting the plurality of deltas; and

and sending the plurality of chunk objects to the object storage system.

Article 7: For the purpose of Article 6,

the checkpoint includes a manifest of the object metadata;

wherein the object metadata includes chunk pointers corresponding to the plurality of chunk objects in the object storage system.

Clause 8: The clause 7, wherein aggregating the object metadata comprises updating the manifest to reflect a plurality of differences between the plurality of delta data sets and the first snapshot data. , method.

Article 9: As a computer system,

one or more processors;

a memory in communication with the one or more processors, the memory configured to store computer-executable instructions, wherein executing the computer-executable instructions causes the one or more processors to perform the steps of: The steps are

creating, by a computer system, a first snapshot of a block volume at a first geographic location at a first logical time, the block volume comprising a plurality of partitions;

sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic area;

creating, by the computer system, a second snapshot of the block volume at the first geographic location and at a second logical time;

generating, by the computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one of the plurality of partitions;

sending, by the computer system, a plurality of delta data sets corresponding to the plurality of deltas to the object storage system in the second geographic region;

generating, by the computer system, a checkpoint at least in part by aggregating object metadata associated with the first snapshot and the plurality of deltas;

receiving, by the computer system, a restore request to create a restore volume; and

creating, by the computer system, the restore volume from the checkpoint.

Clause 10: The method of clause 9, wherein generating the plurality of deltas comprises:

generating a comparison between the second snapshot and the first snapshot;

based on the comparison, determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot; and

generating the plurality of deltas, the plurality of deltas describing modified data for the plurality of partitions.

Clause 11: The method according to clause 9, wherein creating the first snapshot comprises:

corresponding to a logical time, suspending input/output operations for the plurality of partitions;

generating a plurality of block images describing volume data in the plurality of partitions; and

enabling input/output operations to the plurality of partitions.

Clause 12: The method according to clause 9, wherein the restore request is a failover request, the method comprising:

enabling the recovery volume to be created in the second geographic region; and

enabling input/output operations using the recovery volume in the second geographic region.

Clause 13: The method according to clause 9, wherein the restore request is a failback request, the method comprising:

creating the recovery volume in the second geographic region;

enabling a failback volume to be created in the first geographic region; and

restoring the first snapshot data in the first geographic region.

Clause 14: The method of clause 9, wherein transmitting the plurality of delta data sets comprises:

generating a plurality of chunk objects from the plurality of delta data sets;

transmitting the plurality of deltas; and

and transferring the plurality of chunk objects to the object storage system.

Article 15; The method according to clause 14, wherein the checkpoint includes a manifest of the object metadata,

wherein the object metadata includes chunk pointers corresponding to the plurality of chunk objects in the object storage system.

Clause 16: The clause 16 of clause 15, wherein aggregating the object metadata comprises updating the manifest to reflect a plurality of differences between the plurality of delta data sets and the first snapshot data. , computer system.

Clause 17: A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors of a computer system to perform the steps of:

creating, by a computer system, a first snapshot of a block volume at a first geographic location at a first logical time, the block volume comprising a plurality of partitions;

sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic area;

creating, by the computer system, a second snapshot of the block volume at the first geographic location and at a second logical time;

generating, by the computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one of the plurality of partitions;

sending, by the computer system, a plurality of delta data sets corresponding to the plurality of deltas to the object storage system in the second geographic region;

generating, by the computer system, a checkpoint at least in part by aggregating object metadata associated with the first snapshot and the plurality of deltas;

receiving, by the computer system, a restore request to create a restore volume; and

creating, by the computer system, the restore volume from the checkpoint.

Clause 18: The method of clause 17, wherein generating the plurality of deltas comprises:

generating a comparison between the second snapshot and the first snapshot;

based on the comparison, determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot; and

generating the plurality of deltas, the plurality of deltas describing modified data for the plurality of partitions.

Clause 19: The method of clause 17, wherein transmitting the plurality of delta data sets comprises:

generating a plurality of chunk objects from the plurality of delta data sets;

transmitting the plurality of deltas; and

and transmitting the plurality of chunk objects to the object storage system.

Article 20: For the purposes of Article 19,

the checkpoint includes a manifest of the object metadata;

wherein the object metadata includes chunk pointers corresponding to the plurality of chunk objects in the object storage system.

Claims (20)

As a method,
creating, by a computer system, a first snapshot of a block volume at a first geographic location at a first logical time, the block volume comprising a plurality of partitions;
sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic area;
creating, by the computer system, a second snapshot of the block volume at the first geographic location and at a second logical time;
generating, by the computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one of the plurality of partitions;
sending, by the computer system, a plurality of delta data sets corresponding to the plurality of deltas to the object storage system in the second geographic region;
generating, by the computer system, a checkpoint at least in part by aggregating object metadata associated with the first snapshot and the plurality of deltas;
receiving, by the computer system, a restore request to create a restore volume; and
creating, by the computer system, the recovery volume from the checkpoint.
The method of claim 1,
Generating the plurality of deltas,
generating a comparison between the second snapshot and the first snapshot;
based on the comparison, determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot; and
generating the plurality of deltas, the plurality of deltas describing modified data for the plurality of partitions.
The method of claim 1,
Creating the first snapshot,
corresponding to a logical time, suspending input/output operations for the plurality of partitions;
generating a plurality of block images describing volume data in the plurality of partitions; and
and enabling input/output operations to the plurality of partitions.
The method of claim 1,
The restoration request is a failover request, and the method comprises:
enabling the recovery volume to be created in the second geographic region; and
and enabling input/output operations using the recovery volume in the second geographic area.
The method of claim 1,
The restoration request is a failback request, and the method comprises:
creating the recovery volume in the second geographic region;
enabling a failback volume to be created in the first geographic region; and
restoring the first snapshot data in the first geographic region.
The method of claim 1,
Transmitting the plurality of delta data sets comprises:
generating a plurality of chunk objects from the plurality of delta data sets;
transmitting the plurality of deltas; and
and sending the plurality of chunk objects to the object storage system.
The method of claim 6,
the checkpoint includes a manifest of the object metadata;
wherein the object metadata includes chunk pointers corresponding to the plurality of chunk objects in the object storage system.
The method of claim 7,
wherein aggregating the object metadata comprises updating the manifest to reflect a plurality of differences between the plurality of delta data sets and the first snapshot data.
As a computer system,
one or more processors;
a memory in communication with the one or more processors, the memory configured to store computer-executable instructions, wherein executing the computer-executable instructions causes the one or more processors to perform the steps of: The steps are
creating, by a computer system, a first snapshot of a block volume at a first geographic location at a first logical time, the block volume comprising a plurality of partitions;
sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic area;
creating, by the computer system, a second snapshot of the block volume at the first geographic location and at a second logical time;
generating, by the computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one of the plurality of partitions;
sending, by the computer system, a plurality of delta data sets corresponding to the plurality of deltas to the object storage system in the second geographic region;
generating, by the computer system, a checkpoint at least in part by aggregating object metadata associated with the first snapshot and the plurality of deltas;
receiving, by the computer system, a restore request to create a restore volume; and
creating, by the computer system, the restore volume from the checkpoint.
The method of claim 9,
Generating the plurality of deltas,
generating a comparison between the second snapshot and the first snapshot;
based on the comparison, determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot; and
generating the plurality of deltas, the plurality of deltas describing modified data for the plurality of partitions.
The method of claim 9,
Creating the first snapshot,
corresponding to a logical time, suspending input/output operations for the plurality of partitions;
generating a plurality of block images describing volume data in the plurality of partitions; and
enabling input/output operations to the plurality of partitions.
The method of claim 9,
The restoration request is a failover request, and the method comprises:
enabling the recovery volume to be created in the second geographic region; and
enabling input/output operations using the recovery volume in the second geographic region.
The method of claim 9,
The restoration request is a failback request, and the method comprises:
creating the recovery volume in the second geographic region;
enabling a failback volume to be created in the first geographic region; and
restoring the first snapshot data in the first geographic region.
The method of claim 9,
Transmitting the plurality of delta data sets comprises:
generating a plurality of chunk objects from the plurality of delta data sets;
transmitting the plurality of deltas; and
and transferring the plurality of chunk objects to the object storage system.
The method of claim 14,
the checkpoint includes a manifest of the object metadata;
wherein the object metadata includes chunk pointers corresponding to the plurality of chunk objects in the object storage system.
The method of claim 15
wherein aggregating the object metadata comprises updating the manifest to reflect a plurality of differences between the plurality of delta data sets and the first snapshot data.
A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors of a computer system to perform the steps of:
creating, by a computer system, a first snapshot of a block volume at a first geographic location at a first logical time, the block volume comprising a plurality of partitions;
sending, by the computer system, first snapshot data corresponding to the first snapshot to an object storage system in a second geographic area;
creating, by the computer system, a second snapshot of the block volume at the first geographic location and at a second logical time;
generating, by the computer system, a plurality of deltas, each delta of the plurality of deltas corresponding to one of the plurality of partitions;
sending, by the computer system, a plurality of delta data sets corresponding to the plurality of deltas to the object storage system in the second geographic region;
generating, by the computer system, a checkpoint at least in part by aggregating object metadata associated with the first snapshot and the plurality of deltas;
receiving, by the computer system, a restore request to create a restore volume; and
creating, by the computer system, the restore volume from the checkpoint.
The method of claim 17
Generating the plurality of deltas,
generating a comparison between the second snapshot and the first snapshot;
based on the comparison, determining modified data corresponding to changes between the first snapshot data and second snapshot data corresponding to the second snapshot; and
generating the plurality of deltas, the plurality of deltas describing modified data for the plurality of partitions.
The method of claim 17
Transmitting the plurality of delta data sets comprises:
generating a plurality of chunk objects from the plurality of delta data sets;
transmitting the plurality of deltas; and
and transmitting the plurality of chunk objects to the object storage system.
The method of claim 19
the checkpoint includes a manifest of the object metadata;
wherein the object metadata includes chunk pointers corresponding to the plurality of chunk objects in the object storage system.

Images