US10379777B2 - Method for performing replication control in storage system with aid of relationship tree within database, and associated apparatus - Google Patents
Method for performing replication control in storage system with aid of relationship tree within database, and associated apparatus Download PDFInfo
- Publication number
- US10379777B2 US10379777B2 US15/627,404 US201715627404A US10379777B2 US 10379777 B2 US10379777 B2 US 10379777B2 US 201715627404 A US201715627404 A US 201715627404A US 10379777 B2 US10379777 B2 US 10379777B2
- Authority
- US
- United States
- Prior art keywords
- snapshot
- storage server
- node
- replicated
- protection target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/065—Replication mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Definitions
- the present invention relates to storage management, and more particularly, to a method for performing replication control in a storage system, and an associated apparatus.
- the storage system may automatically perform unnecessary data transmission for replication, and the high bandwidth usage in a network may degrade the overall performance of the storage system.
- the storage system may automatically transmit unnecessary data for replication, and the associated storage space available for further replication may decrease rapidly.
- a novel method and associated architecture are required for enhancing the overall performance of the storage system.
- One of the objects of the present invention is to provide a method for performing replication control in a storage system, and an associated apparatus, to solve the problems which exist in the related arts.
- Another of the objects of the present invention is to provide a method for performing replication control in a storage system, and an associated apparatus, to guarantee the overall performance of the storage system.
- a method for performing replication control in a storage system may include a first storage server and a second storage server.
- the method may include: recording relationships between a protection target in the first storage server and a plurality of snapshots of the protection target in a database of the first storage server and updating the database during recording the relationships, wherein the database comprises a relationship tree, and data structure of the relationship tree comprises snapshot nodes respectively corresponding to the plurality of snapshots and a protection target node corresponding to the protection target; selecting a non-replicated snapshot that has not been replicated from the first storage server to the second storage server, wherein the plurality of snapshots comprises the non-replicated snapshot, and the non-replicated snapshot is created in the first storage server at a first time point; selecting a replicated snapshot that has been replicated from the first storage server to the second storage server as reference of replication difference calculation according to the relationship tree within the database, wherein the plurality of snapshots comprises the replicated snapshot, and the replicated snapshot
- an apparatus for performing replication control in a storage system may include a first storage server and a second storage server.
- the apparatus may include a processing circuit that is positioned in the first storage server.
- the processing circuit may be arranged to control operations of the first storage server, and controlling the operations of the first storage server may include: recording relationships between a protection target in the first storage server and a plurality of snapshots of the protection target in a database of the first storage server and updating the database during recording the relationships, wherein the database comprises a relationship tree, and data structure of the relationship tree comprises snapshot nodes respectively corresponding to the plurality of snapshots and a protection target node corresponding to the protection target; selecting a non-replicated snapshot that has not been replicated from the first storage server to the second storage server, wherein the plurality of snapshots comprises the non-replicated snapshot, and the non-replicated snapshot is created in the first storage server at a first time point; selecting a replicated snapshot that has been replicated from the first storage server
- the method and associated apparatus of the present invention may solve problems existing in the related arts without introducing unwanted side effects, or in a way that is less likely to introduce a side effect.
- the method and associated apparatus of the present invention can properly select a snapshot as reference of replication difference calculation according to the relationship tree within the database, to reduce bandwidth consumption and save the storage space of the storage system (more particularly, the storage space of the second storage server).
- the method and associated apparatus of the present invention can monitor at least one user behavior of a user of the first storage server to update the data structure of the relationship tree according to the aforementioned at least one user behavior, to guarantee the overall performance of the storage system.
- FIG. 1 is a diagram of a storage system according to an embodiment of the present invention.
- FIG. 2 illustrates some replication operations of the storage system shown in FIG. 1 according to an embodiment of the present invention.
- FIG. 3 is a flowchart of a method for performing replication control in a storage system according to an embodiment of the present invention.
- FIG. 4 illustrates some snapshots and corresponding replication versions in the storage system shown in FIG. 1 according to an embodiment of the present invention.
- FIG. 5 illustrates some snapshots and corresponding replication versions in the storage system shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 6 illustrates some snapshots and corresponding replication versions in the storage system shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 7 illustrates transition of the data structure of the relationship tree within the database shown in FIG. 1 according to an embodiment of the present invention.
- FIG. 8 illustrates transition of the data structure of the relationship tree within the database shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 9 illustrates transition of the data structure of the relationship tree within the database shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 10 illustrates transition of the data structure of the relationship tree within the database shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 11 illustrates transition of the data structure of the relationship tree within the database shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 12 illustrates transition of the data structure of the relationship tree within the database shown in FIG. 1 according to another embodiment of the present invention.
- FIG. 13 illustrates a reference selection control scheme of the method shown in FIG. 3 according to an embodiment of the present invention.
- FIG. 14 illustrates some snapshots and corresponding replication versions in the storage system shown in FIG. 1 according to another embodiment of the present invention.
- Embodiments of the present invention provide a method for performing replication control in a storage system, and provide an associated apparatus.
- the storage system may include a first storage server (e.g. a source storage server, such as a server utilized as a source of replication) and a second storage server (e.g. a destination storage server, such as a server utilized as a destination of replication).
- the apparatus may control the storage system to properly replicate at least one snapshot (e.g. one or more snapshots) from the first storage server to the second storage server without introducing the related art problems.
- FIG. 1 is a diagram of a storage system 100 according to an embodiment of the present invention.
- the storage system 100 may be taken as an example of the aforementioned storage system.
- the storage system 100 may include storage servers 110 and 120 , which may be taken as examples of the first storage server (e.g. the source storage server) and the second storage server (e.g. the destination storage server), respectively.
- the storage server 110 may include a processing circuit 111 (e.g. at least one processor, and associated circuits such as a chipset, a random access memory (RAM), etc.), a network interface circuit 112 , a storage interface circuit 113 , and at least one storage device (e.g.
- the storage device 114 may include a database 110 D.
- One or more program modules may run on the processing circuit 111 (more particularly, the processor thereof) to control operations of the storage server 110 , for example, according to the method.
- the storage server 120 may include a processing circuit 121 (e.g. at least one processor, and associated circuits such as a chipset, a RAM, etc.), a network interface circuit 122 , a storage interface circuit 123 , and at least one storage device (e.g.
- one or more storage devices such as HDDs and/or SSDs which may be collectively referred to as the storage device 124 .
- One or more program modules may run on the processing circuit 121 (more particularly, the processor thereof) to control operations of the storage server 120 .
- a user of the storage system 100 may access the storage system 100 through a client device 10 , and the client device 10 may include a processing circuit 11 (e.g. at least one processor, and associated circuits such as a chipset, a RAM, etc.) and a network interface circuit 12 .
- the client device 10 and the storage servers 110 and 120 may be coupled to each other through at least one network, in which wired and/or wireless network connections may be utilized.
- both of the storage servers 110 and 120 may be implemented as network attached storage (NAS) devices, and the client device 10 may be any of a multifunctional mobile phone, a laptop computer, a tablet, etc.
- the user may access at least one active storage server within the storage system 100 (e.g. one or both of the storage servers 110 and 120 ) after logging in to the active storage server through a browser running on the processing circuit 11 (more particularly, the processor thereof).
- the apparatus may include at least one portion (e.g. a portion or all) of the aforementioned storage system such as the storage system 100 .
- the apparatus may include a portion of the storage server 110 , and more particularly, may include the processing circuit 111 .
- the apparatus may include the whole of the storage server 110 .
- the apparatus may include the whole of the storage system 100 .
- FIG. 2 illustrates some replication operations of the storage system 100 shown in FIG. 1 according to an embodiment of the present invention.
- the storage devices SD( 1 ), SD( 2 ), etc. e.g. HDDs and/or SSDs
- the storage devices DD( 1 ), DD( 2 ), etc. may be taken as examples of the aforementioned at least one storage device within the storage device 124 shown in FIG. 1
- the storage server 110 may allow the user to assign some objects in the file system of the storage server 110 as protection targets, respectively.
- the processing circuit 111 may control the storage server 110 to generate snapshots of a specific protection target (e.g. any of the protection targets) with respect to time, and allow the user to restore to any snapshot within the snapshots (e.g. a snapshot corresponding to a specific time point).
- the processing circuit 111 may control the storage server 110 to trigger replication operations related to the snapshots of the specific protection target, to replicate the snapshots to the storage server 120 .
- the storage server 110 may transmit snapshot data and/or snapshot difference data to the storage server 120 , and the processing circuit 121 may generate replication versions of the snapshots in the storage server 120 .
- the storage server 120 may be utilized as a replacement or backup device of the storage server 110 when needed.
- the replication versions in the storage server 120 may be regarded as snapshots of the aforementioned copy of the object (i.e. the copy in the storage server 120 ), so the copy of the object may be protected by the replication versions in the storage server 120 .
- some examples of the protection targets and the associated snapshots are illustrated in the upper half of FIG. 2
- some examples of the associated replication versions are illustrated in the lower half of FIG. 2 .
- the snapshots may be generated in at least one Btrfs region of the storage device 114 (e.g. Btrfs regions in the storage devices SD( 1 ), SD( 2 ), etc., respectively), and the replication versions may be generated in at least one Btrfs region of the storage device 124 (e.g. Btrfs regions in the DD( 1 ), DD( 2 ), etc., respectively), but the present invention is not limited thereto.
- FIG. 3 is a flowchart of a method 300 for performing replication control in a storage system according to an embodiment of the present invention.
- the method 300 may be taken as an example of the method mentioned above.
- the method 300 may be applied to the storage system 100 , and may be applied to the storage server 110 and the processing circuit 111 therein.
- the first storage server e.g. the source storage server
- the second storage server e.g. the destination storage server
- the processing circuit 111 may record relationships between a protection target (e.g. the specific protection target) in the storage server 110 and a plurality of snapshots of the protection target in the database 110 D of the storage server 110 and update the database 110 D during recording the relationships.
- a protection target e.g. the specific protection target
- the database 110 D may include a relationship tree
- the data structure of the relationship tree may include snapshot nodes respectively corresponding to the plurality of snapshots and a protection target node corresponding to the protection target.
- the protection target node may be a leaf node, and may be a child node of one of other nodes in the data structure of the relationship tree.
- One of the snapshot nodes may be set as an uppermost node in the data structure of the relationship tree, and therefore may be regarded as a root node.
- the processing circuit 111 may update the data structure of the relationship tree within the database 110 D during recording the relationships, but the present invention is not limited thereto.
- the processing circuit 111 may select a non-replicated snapshot that has not been replicated from the storage server 110 to the storage server 120 .
- the plurality of snapshots may include the non-replicated snapshot, and the non-replicated snapshot may be created in the storage server 110 at a first time point.
- the processing circuit 111 may select a replicated snapshot that has been replicated from the storage server 110 to the storage server 120 as reference of replication difference calculation according to the relationship tree within the database 110 D.
- the plurality of snapshots may include the replicated snapshot, and the replicated snapshot may be created in the storage server 110 at a second time point prior to the first time point.
- the processing circuit 111 may generate snapshot difference data between the non-replicated snapshot and the replicated snapshot.
- the snapshot difference data may indicate any difference between the non-replicated snapshot and the replicated snapshot.
- the snapshot difference data may indicate that no difference between the non-replicated snapshot and the replicated snapshot exists.
- Step 350 the processing circuit 111 may transmit the snapshot difference data from the storage server 110 to the storage server 120 , to replicate the non-replicated snapshot from the storage server 110 to the storage server 120 .
- Step 310 may be entered to allow similar operations of Step 310 through to Step 350 to be performed with regard to the latest replicated/non-replicated statuses of the snapshots, but the present invention is not limited thereto.
- the latest replicated/non-replicated statuses of the snapshots as the number of snapshots in the storage server 110 may increase as time goes by, there may be a latest non-replicated snapshot, which may be regarded as the non-replicated snapshot mentioned above during performing the similar operations of Step 310 through to Step 350 .
- FIG. 4 illustrates some snapshots and corresponding replication versions in the storage system 100 shown in FIG. 1 according to an embodiment of the present invention.
- the protection target may be a folder (e.g. a shared folder), and may originally include files A and B therein before the time point t 11 .
- the snapshots Snapshot(t 11 ), Snapshot(t 12 ), and Snapshot(t 13 ) may be generated at the time points t 11 , t 12 , and t 13 , respectively.
- the snapshot Snapshot(t 11 ) may also include the files A and B.
- the user may delete the file B in the protection target, so the snapshot Snapshot(t 12 ) may include the file A, rather than the file B.
- the user may trigger a restore operation to restore to the snapshot Snapshot(t 11 ), so the snapshot Snapshot(t 13 ) may include the files A and B.
- the processing circuit 111 may update the data structure of the relationship tree within the database 110 D in response to operations of changing the protection target, such as the operation of deleting the file B and the operation of restoring to the snapshot Snapshot(t 11 ).
- the snapshots Snapshot′(t 11 ), Snapshot′(t 12 ), and Snapshot′(t 13 ) in the storage server 120 may be the replication versions of the snapshots Snapshot(t 11 ), Snapshot(t 12 ), and Snapshot(t 13 ) in the storage server 110 , respectively.
- the processing circuit 111 may determine that the file B in the snapshot Snapshot(t 13 ) and the file B in the snapshot Snapshot(t 11 ) are the same file. As a result, in a situation where the snapshot Snapshot(t 13 ) is selected as the non-replicated snapshot mentioned in Step 320 , the processing circuit 111 may select the snapshot Snapshot(t 11 ) (rather than the snapshot Snapshot(t 12 )) as the reference of replication difference calculation mentioned in Step 330 .
- the processing circuit 111 when replicating the snapshot Snapshot(t 13 ) from the storage server 110 to the storage server 120 , the processing circuit 111 does not need to transmit the file content data of the file B, since the processing circuit 111 knows that the file B in the snapshot Snapshot(t 13 ) and the file B in the snapshot Snapshot(t 11 ) are the same file.
- the X-shape notation depicted with dashed lines may indicate that the processing circuit 111 will not treat the file B in the snapshot Snapshot(t 13 ) and the file B in the snapshot Snapshot(t 11 ) as different files (which may be illustrated with different types/styles of shading patterns), and that a full backup of the file B will not occur when replicating the snapshot Snapshot(t 13 ) from the storage server 110 to the storage server 120 is performed for generating the snapshot Snapshot′(t 13 ). Therefore, the present invention method (e.g. the method 300 ) and the associated apparatus (e.g. the storage server 110 or the processing circuit 111 therein) can prevent the related art problems.
- the present invention method e.g. the method 300
- the associated apparatus e.g. the storage server 110 or the processing circuit 111 therein
- the processing circuit 111 may monitor at least one user behavior of a user of the storage server 110 (e.g. the user of the storage system 100 ) to update the data structure of the relationship tree according to the aforementioned at least one user behavior.
- the user behavior may include, but are not limited to: a cloning operation triggered by the user of the storage server 110 .
- the data structure of the relationship tree may further include another protection target node, and the other protection target node may indicate another protection target (e.g. a protection target TB) that is generated by performing the cloning operation in the storage server 110 .
- FIG. 5 illustrates some snapshots and corresponding replication versions in the storage system 100 shown in FIG. 1 according to another embodiment of the present invention.
- the protection target TA may be a folder (e.g. a shared folder), and may originally include the files A and B therein before the time point t 20 .
- the user may trigger the cloning operation to clone the protection target TA to the protection target TB (labeled “Clone to new target TB” in FIG. 5 , for brevity).
- the snapshot SnapshotTA(t 20 ) of the protection target TA and the snapshot SnapshotTB(t 21 ) of the protection target TB may be generated at the time points t 20 and t 21 , respectively.
- the snapshot SnapshotTA(t 20 ) may also include the files A and B.
- the user may trigger the cloning operation, so the snapshot SnapshotTB(t 21 ) may include the files A and B.
- the processing circuit 111 may update the data structure of the relationship tree within the database 110 D in response to the cloning operation.
- the snapshots SnapshotTA′(t 20 ) and SnapshotTB′(t 21 ) in the storage server 120 may be the replication versions of the snapshots SnapshotTA(t 20 ) and SnapshotTB(t 21 ) in the storage server 110 , respectively.
- the processing circuit 111 may determine that the file A in the snapshot SnapshotTB(t 21 ) and the file A in the snapshot SnapshotTA(t 20 ) are the same file and that the file B in the snapshot SnapshotTB(t 21 ) and the file B in the snapshot SnapshotTA(t 20 ) are the same file.
- the processing circuit 111 when replicating the snapshot SnapshotTB(t 21 ) from the storage server 110 to the storage server 120 , the processing circuit 111 does not need to transmit the file content data of each of the files A and B, since the processing circuit 111 knows that the file A in the snapshot SnapshotTB(t 21 ) and the file A in the snapshot SnapshotTA(t 20 ) are the same file and that the file B in the snapshot SnapshotTB(t 21 ) and the file B in the snapshot SnapshotTA(t 20 ) are the same file.
- the X-shape notation depicted with dashed lines may indicate that the processing circuit 111 will not treat the file A in the snapshot SnapshotTB(t 21 ) and the file A in the snapshot SnapshotTA(t 20 ) as different files (which may be illustrated with different types/styles of shading patterns) and will not treat the file B in the snapshot SnapshotTB(t 21 ) and the file B in the snapshot SnapshotTA(t 20 ) as different files (which may be illustrated with different types/styles of shading patterns), and that a full backup of each of the files A and B will not occur when replicating the snapshot SnapshotTB(t 21 ) from the storage server 110 to the storage server 120 is performed for generating the snapshot SnapshotTB′(t 21 ). Therefore, the present invention method (e.g. the method 300 ) and the associated apparatus (e.g. the storage server 110 or the processing circuit 111 therein) can prevent the related art problems.
- the present invention method e.g. the method 300
- the associated apparatus e.g. the storage server
- FIG. 6 illustrates some snapshots and corresponding replication versions in the storage system 100 shown in FIG. 1 according to another embodiment of the present invention.
- the protection target may be a folder (e.g. a shared folder), and may originally include the files A and B therein before the time point t 31 .
- the snapshots Snapshot(t 31 ), Snapshot(t 32 ), Snapshot(t 33 ), and Snapshot(t 34 ) may be generated at the time points t 31 , t 32 , t 33 , and t 34 , respectively.
- the snapshot Snapshot(t 31 ) may also include the files A and B.
- the user may delete the file B and create a file C in the protection target, so the snapshot Snapshot(t 32 ) may include the files A and C, rather than the file B.
- the user may delete the file C in the protection target, so the snapshot Snapshot(t 33 ) may include the file A, rather than the file C.
- the user may trigger at least one operation (e.g. one or more operations, such as one or a combination of operations of restore, recovery, copy, move, etc.) to obtain the files B and C in the protection target, so the snapshot Snapshot(t 34 ) may include the files A, B, and C.
- the processing circuit 111 may update the data structure of the relationship tree within the database 110 D in response to operations of changing the protection target.
- the snapshots Snapshot′(t 31 ), Snapshot′(t 32 ), Snapshot′(t 33 ), and Snapshot′(t 34 ) in the storage server 120 may be the replication versions of the snapshots Snapshot(t 31 ), Snapshot(t 32 ), Snapshot(t 33 ), and Snapshot(t 34 ) in the storage server 110 , respectively.
- the processing circuit 111 may determine that the file B in the snapshot Snapshot(t 34 ) and the file B in the snapshot Snapshot(t 31 ) are the same file and that the file C in the snapshot Snapshot(t 34 ) and the file C in the snapshot Snapshot(t 32 ) are the same file.
- the processing circuit 111 when replicating the snapshot Snapshot(t 34 ) from the storage server 110 to the storage server 120 , the processing circuit 111 does not need to transmit the file content data of each of the files B and C, since the processing circuit 111 knows that the file B in the snapshot Snapshot(t 34 ) and the file B in the snapshot Snapshot(t 31 ) are the same file and that the file C in the snapshot Snapshot(t 34 ) and the file C in the snapshot Snapshot(t 32 ) are the same file.
- the X-shape notation depicted with dashed lines may indicate that the processing circuit 111 will not treat the file B in the snapshot Snapshot(t 34 ) and the file B in the snapshot Snapshot(t 31 ) as different files (which may be illustrated with different types/styles of shading patterns) and will not treat the file C in the snapshot Snapshot(t 34 ) and the file C in the snapshot Snapshot(t 32 ) as different files (which may be illustrated with different types/styles of shading patterns), and that a full backup of each of the files B and C will not occur when replicating the snapshot Snapshot(t 34 ) from the storage server 110 to the storage server 120 is performed for generating the snapshot Snapshot′(t 34 ). Therefore, the present invention method (e.g. the method 300 ) and the associated apparatus (e.g. the storage server 110 or the processing circuit 111 therein) can prevent the related art problems.
- the present invention method e.g. the method 300
- the associated apparatus e.g. the storage server 110 or the processing circuit
- the data structure of the relationship tree within the database 110 D may be illustrated with one or more nodes (e.g. one or more protection target nodes, one or more snapshot nodes, and/or one or more clone point nodes).
- the protection target node(s) may be labeled with the notation “RW” (which stands for Read/Write) and the snapshot node(s) may be labeled with the notation “RO” (which stands for Read-Only), but the present invention is not limited thereto.
- the protection target node(s) being labeled with the notation “RW” may indicate that, when having the RW authority, the user can read and can write (e.g. change, edit, etc.) the protection target(s) corresponding to the protection target node(s).
- the snapshot node(s) being labeled with the notation “RO” may indicate that, when having the RO authority, the user can read the snapshot(s) corresponding to the snapshot node(s), rather than writing (e.g. changing, editing, etc.) the snapshot(s) corresponding to the snapshot node(s).
- FIG. 7 illustrates transition of the data structure of the relationship tree within the database 110 D shown in FIG. 1 according to an embodiment of the present invention.
- the user may apply a setting to the storage server 110 to assign the object as the protection target (labeled “Create protection target” in FIG. 7 , for better comprehension), and the processing circuit 111 may generate the protection target node (labeled “Protection target (RW)” around the left-hand side of FIG. 7 ) in the data structure of the relationship tree.
- the replicated snapshot e.g.
- a snapshot Snap(t 1 )) is created at the time point t 1
- the processing circuit 111 may add a snapshot node (labeled “Snap(t 1 ) (RO)” around the center of FIG. 7 ) corresponding to the replicated snapshot above the protection target node in the data structure of the relationship tree.
- the non-replicated snapshot e.g. a snapshot Snap(t 2 )
- the processing circuit 111 may add a snapshot node (labeled “Snap(t 2 ) (RO)” around the right-hand side of FIG.
- the parent node of the protection target node is changed from the snapshot node (labeled “Snap(t 1 ) (RO)” around the center of FIG. 7 ) corresponding to the replicated snapshot to the snapshot node (labeled “Snap(t 2 ) (RO)” around the right-hand side of FIG. 7 ) corresponding to the non-replicated snapshot.
- FIG. 8 illustrates transition of the data structure of the relationship tree within the database 110 D shown in FIG. 1 according to another embodiment of the present invention.
- the data structure of the relationship tree around the left-hand side of FIG. 8 may be the same as that around the right-hand side of FIG. 7 .
- the processing circuit 111 may remove the snapshot node (labeled “Snap(t 2 ) (RO)” around the left-hand side of FIG. 8 ) corresponding to the non-replicated snapshot from the data structure of the relationship tree to make the snapshot node (labeled “Snap(t 1 ) (RO)” around the right-hand side of FIG.
- the protection target node (labeled “Protection target (RW)” around the right-hand side of FIG. 8 ) be connected to each other in the data structure of the relationship tree.
- the parent node of the protection target node is changed from the snapshot node (labeled “Snap(t 2 ) (RO)” around the left-hand side of FIG. 8 ) corresponding to the non-replicated snapshot to the snapshot node (labeled “Snap(t 1 ) (RO)” around the right-hand side of FIG. 8 ) corresponding to the replicated snapshot, but the present invention is not limited thereto.
- the processing circuit 111 may remove the node corresponding to a deleted snapshot (e.g. the snapshot Snap(t 2 )) to make the child node of the removed node be connected to the parent node of the removed node.
- a deleted snapshot e.g. the snapshot Snap(t 2 )
- FIG. 9 illustrates transition of the data structure of the relationship tree within the database 110 D shown in FIG. 1 according to another embodiment of the present invention.
- the user may trigger an operation such as a restore operation, and, in response to the triggered operation, the processing circuit 111 may update the data structure of the relationship tree to make it become that shown around the left-hand side of FIG. 9 .
- the processing circuit 111 may remove the snapshot node (labeled “Snap(t 1 ) (RO)” around the left-hand side of FIG.
- the processing circuit 111 may remove the node corresponding to a deleted snapshot (e.g. the snapshot Snap(t 1 )) to make a child node (e.g. the node corresponding to the snapshot Snap(t 2 )) of the removed node replace the removed node.
- a deleted snapshot e.g. the snapshot Snap(t 1 )
- a child node e.g. the node corresponding to the snapshot Snap(t 2 )
- FIG. 10 illustrates transition of the data structure of the relationship tree within the database 110 D shown in FIG. 1 according to another embodiment of the present invention.
- the data structure of the relationship tree around the left-hand side of FIG. 10 may be the same as that around the right-hand side of FIG. 7 .
- the processing circuit 111 may move the protection target node (labeled “Protection target (RW)” around the left-hand side of FIG. 10 ) upward to make the protection target node (labeled “Protection target (RW)” around the right-hand side of FIG.
- the parent node of the protection target node is changed from the snapshot node (labeled “Snap(t 2 ) (RO)” around the left-hand side of FIG. 10 ) corresponding to the non-replicated snapshot to the snapshot node (labeled “Snap(t 1 ) (RO)” around the right-hand side of FIG. 10 ) corresponding to the replicated snapshot, but the present invention is not limited thereto.
- the processing circuit 111 may move the protection target node (labeled “Protection target (RW)” around the left-hand side of FIG. 10 ) to make it become a child node of the snapshot node (labeled “Snap(t 1 ) (RO)” around the right-hand side of FIG. 10 ) corresponding to the replicated snapshot.
- RW Protection target
- Snap(t 1 ) (RO) child node of the snapshot node
- the aforementioned operation of restoring to the replicated snapshot may be the restore operation mentioned in the embodiment shown in FIG. 9 .
- the processing circuit 111 may perform the associated operations of the embodiment shown in FIG. 9 .
- similar descriptions for these embodiments are not repeated in detail here.
- FIG. 11 illustrates transition of the data structure of the relationship tree within the database 110 D shown in FIG. 1 according to another embodiment of the present invention.
- the protection target mentioned in Step 310 may be the protection target TA.
- the node labeled “Protection target TA (RW)” around the left-hand side of FIG. 11 may be taken as an example of the protection target node, and the node labeled “Protection target TB (RW)” around the right-hand side of FIG. 11 may be taken as an example of the other protection target node.
- the data structure of the relationship tree around the left-hand side of FIG. 11 may be the same as that around the right-hand side of FIG. 7 .
- the processing circuit 111 may add the other protection target node (labeled “Protection target TB (RW)” around the right-hand side of FIG. 11 ) below the snapshot node (labeled “Snap(t 1 ) (RO)” around the right-hand side of FIG. 11 ) corresponding to the replicated snapshot in the data structure of the relationship tree.
- the other protection target node becomes a child node of the snapshot node (labeled “Snap(t 1 ) (RO)” around the right-hand side of FIG. 11 ) corresponding to the replicated snapshot.
- FIG. 12 illustrates transition of the data structure of the relationship tree within the database 110 D shown in FIG. 1 according to another embodiment of the present invention.
- the protection target mentioned in Step 310 may be the protection target TA.
- the node labeled “Protection target TA (RW)” around the left-hand side of FIG. 12 may be taken as an example of the protection target node, and the node labeled “Protection target TB (RW)” around the right-hand side of FIG. 12 may be taken as an example of the other protection target node.
- the data structure of the relationship tree around the left-hand side of FIG. 12 may be the same as that around the right-hand side of FIG. 8 .
- the processing circuit 111 may insert a clone point node (labeled “Clone point” in FIG. 12 , since the clone point node may be simply referred to as the clone point) between the protection target node (labeled “Protection target TA (RW)”) and the parent node of the protection target node (e.g.
- the node labeled “Snap(t 1 ) (RO)” is originally the parent node of the node labeled “Protection target TA (RW)” as shown in the left-hand side of FIG. 12 ) and add the other protection target node (labeled “Protection target TB (RW)” around the right-hand side of FIG. 12 ) below the clone point node in the data structure of the relationship tree.
- the data structure of the relationship tree may include the clone point node, and the clone point node may be a node inserted into the data structure of the relationship tree to indicate the cloning operation triggered by the user of the storage server 110 .
- the other protection target node (labeled “Protection target TB (RW)” around the right-hand side of FIG. 12 ) may correspond to the other protection target such as the protection target TB, and both of the protection target node (labeled “Protection target TA (RW)” around the right-hand side of FIG. 12 ) and the other protection target node (labeled “Protection target TB (RW)” around the right-hand side of FIG. 12 ) may become child nodes of the clone point node.
- the processing circuit 111 may record at least one relationship between the other protection target (e.g. the protection target TB) in the storage server 110 and at least one portion (e.g. a portion or all) of the plurality of snapshots in the database 110 D, and may update the data structure of the relationship tree within the database 110 D during recording the aforementioned at least one relationship.
- the data structure of the relationship tree may include the other protection target node (e.g. the node labeled “Protection target TB (RW)” around the right-hand side of any of FIGS. 11-12 ) to indicate the other protection target (e.g. the protection target TB).
- FIG. 13 illustrates a reference selection control scheme of the method 300 shown in FIG. 3 according to an embodiment of the present invention.
- the processing circuit 111 may generate some additional snapshots Snap(t 3 ) and Snap(t 4 ) of the protection target in the storage server 110 , and the snapshots Snap(t 3 ) and Snap(t 4 ) may be generated at time points t 3 and t 4 , respectively, where t 1 ⁇ t 2 ⁇ t 3 ⁇ t 4 .
- the processing circuit 111 may update the data structure of the relationship tree to make it become that shown in FIG. 13 .
- the nodes respectively labeled “Snap(t 1 ) (RO)”, “Snap(t 2 ) (RO)”, “Snap(t 3 ) (RO)”, and “Snap(t 4 ) (RO)” in the data structure of the relationship tree correspond to the snapshots Snap(t 1 ), Snap(t 2 ), Snap(t 3 ), and Snap(t 4 ) of the protection target, respectively.
- the snapshot Snap(t 4 ) may be taken as an example of the latest non-replicated snapshot mentioned above at a specific time point after the time point t 4 .
- the reference of replication difference calculation for the non-replicated snapshot e.g.
- the processing circuit 111 may search for at least one closest snapshot node (e.g. one or more closest snapshot nodes) along the direction toward the root (or the root node) in the data structure of the relationship tree.
- the aforementioned at least one closest snapshot node may include at least one node (e.g. one or more nodes) that is closest to the snapshot node corresponding to the non-replicated snapshot (e.g. the latest non-replicated snapshot such as the snapshot Snap(t 4 )) in the data structure of the relationship tree, and the reference of replication difference calculation may include at least one snapshot corresponding to the aforementioned at least one closest snapshot node.
- the direction toward the root is the same as the direction toward its parent node such as the snapshot node corresponding to the snapshot Snap(t 3 ) (e.g. the upward direction), and the processing circuit 111 may find the aforementioned at least one closest snapshot node along the direction toward the parent node.
- the aforementioned at least one closest snapshot node may include the parent node such as the snapshot node corresponding to the snapshot Snap(t 3 ), but the present invention is not limited thereto.
- the number of snapshot nodes within the aforementioned at least one closest snapshot node may be greater than one, and, in addition to the parent node such as the snapshot node corresponding to the snapshot Snap(t 3 ), the aforementioned at least one closest snapshot node may further include one or more other snapshot nodes searchable in the data structure of the relationship tree (e.g. the snapshot node corresponding to the snapshot Snap(t 1 ), the snapshot node corresponding to the snapshot Snap(t 2 ), etc.), no matter whether one or more clone point nodes exist or not.
- FIG. 14 illustrates some snapshots and corresponding replication versions in the storage system 100 shown in FIG. 1 according to another embodiment of the present invention.
- the protection target TA may be a folder (e.g. a shared folder), and may originally include the files A, B, and C therein before the time point t 40 .
- the user may trigger some cloning operations to clone the protection target TA to create some other protection targets such as the protection targets TB and TC in the storage server 110 .
- the snapshot SnapshotTA(t 40 ) of the protection target TA, the snapshot SnapshotTB(t 41 ) of the protection target TB, and the snapshot SnapshotTC(t 42 ) of the protection target TC may be generated at the time points t 40 , t 41 , and t 42 , respectively, where t 40 ⁇ t 41 ⁇ t 42 .
- the snapshot SnapshotTA(t 40 ) may also include the files A, B, and C.
- each of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) may also include the files A, B, and C.
- the processing circuit 111 may update the data structure of the relationship tree within the database 110 D in response to the cloning operations. For example, after some replication operations are performed (labeled “Replication” in FIG. 14 , for brevity), the snapshots SnapshotTA′(t 40 ), SnapshotTB′(t 41 ), and SnapshotTC′(t 42 ) are generated in the storage server 120 , in which the snapshots SnapshotTA′(t 40 ), SnapshotTB′(t 41 ), and SnapshotTC′(t 42 ) may be the replication versions of the snapshots SnapshotTA(t 40 ), SnapshotTB(t 41 ), and SnapshotTC(t 42 ) in the storage server 110 , respectively.
- Replication replication operations
- the processing circuit 111 may determine that the file A in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) and the file A in the snapshot SnapshotTA(t 40 ) are the same file and that the file B in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) and the file B in the snapshot SnapshotTA(t 40 ) are the same file and that the file C in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) and the file C in the snapshot SnapshotTA(t 40 ) are the same file.
- the processing circuit 111 when replicating the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) from the storage server 110 to the storage server 120 , the processing circuit 111 does not need to transmit the file content data of each of the files A, B, and C, since the processing circuit 111 knows that the file A in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC (t 42 ) and the file A in the snapshot SnapshotTA(t 40 ) are the same file and that the file B in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) and the file B in the snapshot SnapshotTA(t 40 ) are the same file and that the file C in any of the snapshots SnapshotTB (t 41 ) and SnapshotTC(t 42 ) and the file C in the snapshot SnapshotTA(t 40 ) are the same file.
- the X-shape notation depicted with dashed lines may indicate that the processing circuit 111 will not treat the file A in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) and the file A in the snapshot SnapshotTA(t 40 ) as different files (which may be illustrated with different types/styles of shading patterns) and will not treat the file B in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC(t 42 ) and the file B in the snapshot SnapshotTA(t 40 ) as different files (which may be illustrated with different types/styles of shading patterns) and will not treat the file C in any of the snapshots SnapshotTB(t 41 ) and SnapshotTC (t 42 ) and the file C in the snapshot SnapshotTA(t 40 ) as different files (which may be illustrated with different types/styles of shading patterns), and that a full backup of each of the files A, B, and C will not occur when the operations of replicating the snapshots SnapshotTB(t 41 ) and SnapshotTC
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (19)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/627,404 US10379777B2 (en) | 2017-06-19 | 2017-06-19 | Method for performing replication control in storage system with aid of relationship tree within database, and associated apparatus |
CN201711079687.2A CN109144774A (en) | 2017-06-19 | 2017-11-06 | For carrying out the method and device of duplication control within the storage system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/627,404 US10379777B2 (en) | 2017-06-19 | 2017-06-19 | Method for performing replication control in storage system with aid of relationship tree within database, and associated apparatus |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180364912A1 US20180364912A1 (en) | 2018-12-20 |
US10379777B2 true US10379777B2 (en) | 2019-08-13 |
Family
ID=64656844
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/627,404 Expired - Fee Related US10379777B2 (en) | 2017-06-19 | 2017-06-19 | Method for performing replication control in storage system with aid of relationship tree within database, and associated apparatus |
Country Status (2)
Country | Link |
---|---|
US (1) | US10379777B2 (en) |
CN (1) | CN109144774A (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190155936A1 (en) * | 2017-11-22 | 2019-05-23 | Rubrik, Inc. | Replication Catch-up Strategy |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7404051B2 (en) * | 2005-04-18 | 2008-07-22 | Hitachi, Ltd. | Method for replicating snapshot volumes between storage systems |
US7567991B2 (en) * | 2003-06-25 | 2009-07-28 | Emc Corporation | Replication of snapshot using a file system copy differential |
US20100082921A1 (en) * | 2008-10-01 | 2010-04-01 | Thompson Robert D | Systems And Methods For Data Snapshot Replication |
US20150066857A1 (en) * | 2013-09-03 | 2015-03-05 | Tintri Inc. | Replication of snapshots and clones |
US20170249222A1 (en) * | 2016-02-26 | 2017-08-31 | Netapp Inc. | Granular consistency group replication |
US20170300247A1 (en) * | 2016-04-14 | 2017-10-19 | Seagate Technology Llc | Intelligent snapshot tree replication |
-
2017
- 2017-06-19 US US15/627,404 patent/US10379777B2/en not_active Expired - Fee Related
- 2017-11-06 CN CN201711079687.2A patent/CN109144774A/en not_active Withdrawn
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7567991B2 (en) * | 2003-06-25 | 2009-07-28 | Emc Corporation | Replication of snapshot using a file system copy differential |
US7404051B2 (en) * | 2005-04-18 | 2008-07-22 | Hitachi, Ltd. | Method for replicating snapshot volumes between storage systems |
US20100082921A1 (en) * | 2008-10-01 | 2010-04-01 | Thompson Robert D | Systems And Methods For Data Snapshot Replication |
US20150066857A1 (en) * | 2013-09-03 | 2015-03-05 | Tintri Inc. | Replication of snapshots and clones |
US20170249222A1 (en) * | 2016-02-26 | 2017-08-31 | Netapp Inc. | Granular consistency group replication |
US20170300247A1 (en) * | 2016-04-14 | 2017-10-19 | Seagate Technology Llc | Intelligent snapshot tree replication |
Also Published As
Publication number | Publication date |
---|---|
CN109144774A (en) | 2019-01-04 |
US20180364912A1 (en) | 2018-12-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108376109B (en) | Apparatus and method for copying volume of source array to target array, storage medium | |
US10169163B2 (en) | Managing backup operations from a client system to a primary server and secondary server | |
US20190018738A1 (en) | Method for performing replication control in a storage system with aid of characteristic information of snapshot, and associated apparatus | |
US8832028B2 (en) | Database cloning | |
US9367598B2 (en) | Merging an out of synchronization indicator and a change recording indicator in response to a failure in consistency group formation | |
US9934104B2 (en) | Metadata generation for incremental backup | |
US10789131B2 (en) | Transportable backups for pluggable database relocation | |
CN103354923B (en) | A kind of data re-establishing method, device and system | |
US20150213100A1 (en) | Data synchronization method and system | |
WO2016060675A1 (en) | Method and apparatus for managing data recovery of distributed storage system | |
US9483367B1 (en) | Data recovery in distributed storage environments | |
EP3451173A1 (en) | Restoring a database using a fully hydrated backup | |
US10120595B2 (en) | Optimizing backup of whitelisted files | |
JP6028641B2 (en) | Information processing system, control program for information processing apparatus, and control method for information processing system | |
US9672113B1 (en) | Data recovery from multiple data backup technologies | |
US10628298B1 (en) | Resumable garbage collection | |
CN109753381B (en) | Continuous data protection method based on object storage | |
US9880776B1 (en) | Content-driven data protection method for multiple storage devices | |
US20180095833A1 (en) | Restoration of content of a volume | |
US10379777B2 (en) | Method for performing replication control in storage system with aid of relationship tree within database, and associated apparatus | |
CN104572339A (en) | Data backup restoring system and method based on distributed file system | |
US9037762B2 (en) | Balancing data distribution in a fault-tolerant storage system based on the movements of the replicated copies of data | |
CN115955488B (en) | Distributed storage copy cross-machine room placement method and device based on copy redundancy | |
JP2008097156A (en) | Storage control device, storage control method, and storage control program | |
US10884874B1 (en) | Federated restore of availability group database replicas |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SYNOLOGY INCORPORATED, TAIWAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KE, KUN-CING;REEL/FRAME:042752/0322 Effective date: 20170615 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING TC RESP, ISSUE FEE PAYMENT RECEIVED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING TC RESP, ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20230813 |