CN113849352A - Business data storage exception processing method and device and server - Google Patents
Business data storage exception processing method and device and server Download PDFInfo
- Publication number
- CN113849352A CN113849352A CN202111137683.1A CN202111137683A CN113849352A CN 113849352 A CN113849352 A CN 113849352A CN 202111137683 A CN202111137683 A CN 202111137683A CN 113849352 A CN113849352 A CN 113849352A
- Authority
- CN
- China
- Prior art keywords
- bucket
- service
- storage
- data
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
- G06F11/2023—Failover techniques
- G06F11/2033—Failover techniques switching over of hardware resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
- G06F11/2094—Redundant storage or storage space
Abstract
The invention provides a method, a device and a server for processing business data storage exception, which are used for requesting storage service from a source storage bucket corresponding to a business data operation instruction based on an acquired business data operation instruction; if the storage service corresponding to the source storage bucket is detected to be unavailable, setting the service domain name of another storage bucket which is in a main-standby relationship with the source storage bucket as a main analysis record, and responding to the service data operation instruction through the storage service of the storage bucket corresponding to the main analysis record; if the storage service of the source storage bucket is available, copying the data operation corresponding to the business data operation instruction from the storage bucket corresponding to the main analytic record to the source storage bucket; and setting the service domain name corresponding to the source bucket as a main analytic record. According to the method, when the source bucket service fails, the standby bucket response service is quickly switched, and when the storage service of the source bucket is available, the source bucket response service is switched, so that the service response speed is ensured, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of cloud storage, in particular to a method and a device for processing business data storage exception and a server.
Background
With the rapid development of information technology, the data volume begins to increase explosively, and the data value is higher and higher, so people need to perform backup disaster recovery protection on data so as to timely and effectively recover the data when a fault occurs, and avoid loss.
Particularly for cloud-stored data, in order to ensure the use of the data by a user, a disaster recovery backup scheme is generally adopted, and for the cloud-stored data, a cross-machine room full-volume data backup disaster recovery scheme, a cross-region data full-volume backup disaster recovery scheme, a two-place three-center data backup disaster recovery scheme, and the like are generally adopted. In these disaster recovery backup schemes, when an abnormal data service storage condition occurs, a standby remote storage space is usually directly used to continue service data service, but the remote storage space is usually far from the service side, and the response is slow, which affects the user experience.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus and a server for handling exception of service data storage, so as to alleviate the above technical problem.
In a first aspect, an embodiment of the present invention provides a method for processing an exception in service data storage, where the method includes: requesting storage service from a source storage bucket corresponding to the business data operation instruction based on the acquired business data operation instruction; if the storage service corresponding to the source storage bucket is detected to be unavailable, setting a service domain name corresponding to another storage bucket which is in a main-standby relationship with the source storage bucket as a main analysis record, and responding to a service data operation instruction through the storage service of the storage bucket corresponding to the main analysis record; if the storage service of the source storage bucket is detected to be available, copying the data operation corresponding to the business data operation instruction from the storage bucket corresponding to the main analytic record to the source storage bucket; and setting the service domain name corresponding to the source storage bucket as a main analytic record.
Preferably, in a possible implementation manner, before requesting storage service from a source bucket corresponding to a service data operation instruction based on the obtained service data operation instruction, the method further includes: acquiring a preset service requirement; the service requirement comprises a plurality of sub-requirements corresponding to the set area; for each sub-demand, creating a bucket corresponding to the set area; establishing a main-standby relation among a plurality of storage buckets based on business requirements; the main and standby relationship comprises: aiming at each sub-requirement, a bucket corresponding to the sub-requirement is a source bucket of the sub-requirement, and a bucket except the source bucket is a standby bucket of the source bucket; and configuring service read-write logic and cross-region synchronization logic for the bucket of each set region so as to realize data backup of the bucket of each set region.
Preferably, in a possible embodiment, the step of creating a bucket corresponding to the set area comprises: adding a service domain name based on a set area corresponding to the sub-requirement; the service domain name carries an identifier of a set area; and creating a storage bucket corresponding to each service domain name.
Preferably, in one possible implementation, the bucket of sub-requirements comprises a service domain name; based on the service requirement, the step of establishing the main-standby relationship among a plurality of storage buckets comprises the following steps: for each bucket, adding the service domain name of the standby bucket of the bucket as an alias record of the service domain name of the bucket to establish a primary-standby relationship between the buckets.
Preferably, in a possible implementation, the service read-write logic includes a data write logic and a data read logic; the step of configuring service read-write logic for the bucket created by each region comprises the following steps: respectively configuring data writing logic and data reading logic for the buckets established for each region; wherein the data write logic comprises: setting prefix information of a file corresponding to the written data according to an uploading area corresponding to the data writing operation, and storing the file containing the written data based on the prefix information; the data read logic includes: and acquiring a file corresponding to the data reading operation, and setting a reading path of the data reading operation as a path corresponding to the area written in the file.
Preferably, in a possible embodiment, the method further comprises: and if the storage service corresponding to the source bucket is available, responding to the business data operation instruction through the storage service of the source bucket, and synchronizing the data operation corresponding to the business data operation instruction into a standby bucket of the source bucket according to the pre-configured cross-region synchronization logic so as to keep the object data in the standby bucket consistent with the object data in the source bucket.
Preferably, in a possible implementation, the step of configuring the cross-region synchronization logic for the bucket created for each region includes: acquiring a preset synchronous operation range; wherein the synchronization operation range includes at least one of: adding, modifying and deleting; the synchronization logic of the buckets created for each zone is set based on the synchronization operation range, respectively, so that the synchronization operation of the source bucket and the standby bucket of the source bucket is kept synchronized.
Preferably, in a possible embodiment, the method further comprises: and if the copying operation range comprises the adding operation, configuring a life cycle for the object data corresponding to the adding operation.
In a second aspect, an embodiment of the present invention further provides a device for processing a service data storage exception, where the device includes: the service request module is used for requesting storage service to a source storage bucket corresponding to the business data operation instruction based on the acquired business data operation instruction; a first main parsing record setting module, configured to set, if it is detected that a storage service corresponding to a source bucket is unavailable, a service domain name corresponding to another bucket, which is in a primary-standby relationship with the source bucket, as a main parsing record, so as to respond to a service data operation instruction through the storage service of the bucket corresponding to the main parsing record; the data operation copying module is used for copying the data operation corresponding to the business data operation instruction from the storage bucket corresponding to the main analytic record to the source storage bucket if the storage service of the source storage bucket is detected to be available; and the second main analysis record setting module is used for setting the service domain name corresponding to the source storage bucket as a main analysis record.
In a third aspect, an embodiment of the present invention further provides a server, including a processor and a memory, where the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method in the first aspect.
In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of the first aspect.
The embodiment of the invention has the following beneficial effects:
according to the method, the device and the server for processing the storage exception of the service data, which are provided by the embodiment of the invention, based on the acquired service data operation instruction, a storage service is requested from a source storage bucket corresponding to the service data operation instruction; if the storage service corresponding to the source storage bucket is detected to be unavailable, setting a service domain name corresponding to another storage bucket which is in a main-standby relationship with the source storage bucket as a main analysis record, and responding to a service data operation instruction through the storage service of the storage bucket corresponding to the main analysis record; if the storage service of the source storage bucket is available, copying the data operation corresponding to the business data operation instruction from the storage bucket corresponding to the main analytic record to the source storage bucket; and setting the service domain name corresponding to the source bucket as a main analytic record. According to the method, when the source bucket service fails, the standby bucket response service is quickly switched, and when the storage service of the source bucket is available, the source bucket response service is switched, so that the service response speed is ensured, and the user experience is improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a method for processing an exception of a service data storage according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a backup of service data according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a service data storage exception handling apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, for a disaster recovery solution for cloud storage data, a two-place three-center full backup is generally adopted, and a main backup is generally adopted, so that the cost is extremely high under the effect of obtaining system safety, and a main backup is generally adopted, and meanwhile, a self-built high-availability disaster recovery architecture needs to be manually used for carrying out full data backup, so that not only is the workload large, but also the maintenance cost is high, and the backup efficiency is low.
Based on this, the service data storage exception handling method, the service data storage exception handling device and the server provided by the embodiment of the invention can effectively alleviate the technical problem.
To facilitate understanding of the embodiment, a detailed description is first given to a method for handling exception handling of service data storage disclosed in the embodiment of the present invention.
In a possible implementation manner, an embodiment of the present invention provides a method for processing a service data storage exception, and in particular, the method for processing a service data storage exception, which is provided by the embodiment of the present invention, is generally used for processing cloud storage data when an exception occurs. Specifically, as shown in fig. 1, a flowchart of a method for processing an exception in a service data storage includes the following steps:
and step S102, based on the acquired service data operation instruction, requesting storage service from a source storage bucket corresponding to the service data operation instruction.
In actual use, the service data operation instruction in step S102 is a service data operation instruction that is sent by a user at a service side after logging in a console of a public cloud through a client for access, and generally, the user needs to input a legal login account at the client to authenticate a legal identity of the user, and when the authentication passes, the user can access object data in a storage bucket in the public cloud only after confirming that the user is a legal user, and specifically, the service data operation instruction at least includes one of the following operations: and performing addition, modification, deletion and other operations on the object data in the source storage bucket.
The service data operation instruction usually carries certain regional information, for example, an IP address of a client of a user belongs to a beijing area, and a bucket set in beijing is usually used as a source bucket of a corresponding service operation in an initial setting. When the nearby response is carried out, the response speed is high, and the user experience is good.
Step S104, if the storage service corresponding to the source storage bucket is detected to be unavailable, setting the service domain name corresponding to another storage bucket which is in a master-slave relationship with the source storage bucket as a master analysis record, so as to respond to the service data operation instruction through the storage service of the storage bucket corresponding to the master analysis record.
When in actual use, the source storage bucket and the standby storage bucket are public cloud storage buckets arranged in different areas, and a data cross-area synchronization function can be set for the source storage bucket and the standby storage bucket during initial setting. By means of public cloud object storage and cross-region synchronization, the number of files in a source storage bucket and the number of files in a target storage bucket are finally consistent with the content of the files, and the structures are mutually standby and active.
When a system problem occurs, if the object storage service of the shanghai area is unavailable, the shanghai related records in the parsing records will be invalid, and the service domain name of beijing will be the main parsing record, that is, all the file writing and reading will be performed from the bucket buckek 1 of beijing, so that the writing of the files can be guaranteed not to stop, at this time, all new files at the beginning of BJPrefix-are produced, and at the same time, the data at the beginning of SHPrefix-stored in the bucket1 copied by cross-area can be read, and the operation of reading and writing of the service is almost lossless.
Similarly, when the public cloud object storage in the beijing region is unavailable, the records related to beijing in the analysis records will be invalid, the service domain name in shanghai will become the main analysis record, that is, all the file writing and reading will be read and written from the shanghai storage bucket buckek 2, the write files can be guaranteed not to stop, at this time, all new files at the beginning of SHPrefix-will be produced, and simultaneously, the data at the beginning of BJPrefix-stored in the bucket2, which is copied by cross-region, can be read. And step S106, if the storage service of the source bucket is detected to be available, copying the data operation corresponding to the business data operation instruction from the bucket corresponding to the main analytic record to the source bucket.
Data operations, such as deletion, addition of a portion of data, may have been performed on data in a spare bucket of a source bucket during a time period when storage services of the source bucket are unavailable. In order to keep the data in the source bucket consistent with the data in the spare bucket, the data operation in the spare bucket needs to be copied into the source bucket.
Step S108, setting the service domain name corresponding to the source bucket as a master resolution record.
Therefore, the method, the device and the server for processing the business data storage exception provided by the embodiment of the invention request a storage service from a source storage bucket corresponding to a business data operation instruction based on the obtained business data operation instruction; if the storage service corresponding to the source storage bucket is detected to be unavailable, setting a service domain name corresponding to another storage bucket which is in a main-standby relationship with the source storage bucket as a main analysis record, and responding to a service data operation instruction through the storage service of the storage bucket corresponding to the main analysis record; if the storage service of the source storage bucket is available, copying the data operation corresponding to the business data operation instruction from the storage bucket corresponding to the main analytic record to the source storage bucket; and setting the service domain name corresponding to the source bucket as a main analytic record. According to the method, when the source bucket service fails, the standby bucket response service is quickly switched, and when the storage service of the source bucket is available, the source bucket response service is switched, so that the service response speed is ensured, and the user experience is improved.
In the above method, the source bucket in the embodiment of the present invention has a preconfigured data bucket that is in a primary-standby relationship with the source bucket. When two storage buckets are in a primary-backup relationship with each other, when a user accesses any one data storage bucket in the primary-backup relationship with each other, the data storage bucket can be regarded as a source storage bucket, and the other data storage bucket in the primary-backup relationship with the data storage bucket is a standby storage bucket.
In practical use, the public cloud object storage is distributed storage facing the internet, supports reading, writing and management of files at any time and place through an Http/Http protocol, supports a standard Rest API, and provides an infinite storage space for a client through a flat storage architecture, so that the object storage is a storage mode with high reliability, high availability, low cost and infinite expansion, is suitable for massive unstructured data storage, and can support elastic expansion of resources.
Further, in order to achieve convenient synchronization of data, cross-region synchronization functions are provided for public cloud object storage. The data buckets in the embodiment of the present invention also borrow this function, specifically, bucket cross-region synchronization is a rule configuration for the buckets, and a synchronization logic rule of the bucket needs to be configured, so that the buckets after configuring the rule can automatically and asynchronously copy the incremental object in different buckets. After the cross-region synchronization logic is enabled, the object store will exactly copy the object contents (such as object metadata and version ID) in the source bucket into the target bucket, and it can be realized that the synchronized object copy and the source bucket have completely consistent attribute information.
In addition, the access operation to the object data in the source bucket, such as the operation of adding an object, deleting an object, etc., will also be synchronized to the target bucket, further, when configuring the cross-region synchronization logic of the source bucket to the target bucket, the operation type of synchronization may also be configured, for example, whether to add, change synchronization, or add, delete, change synchronization, i.e., the addition, modification, deletion, etc., included in the above access operation in the embodiment of the present invention, may be configured in advance according to the actual usage requirement, and these operations cover all API operation types, so that it is possible to implement that two buckets are mirror images of each other, i.e., the object data in the target bucket and the object data in the source bucket are kept consistent, further, for the file generated synchronously in the target bucket, the number of synchronization times may also be recorded, so that the file can only be synchronized once, the synchronization behavior cannot be continuously triggered, so that the phenomenon of circular synchronization of data cannot occur, and the object data in the source bucket and the object data in the target bucket are ensured to be highly consistent.
Further, the data buckets in the primary-backup relationship and the cross-region synchronization logic between the source bucket and the target bucket are usually configured in advance, so as to facilitate data backup between the source bucket and the target bucket. Specifically, for the configuration process of the primary-standby relationship and the cross-regional synchronization logic, a user may complete the configuration process in a console provided by a domain name service provider and a public cloud service provider, and the configuration process generally includes the following steps:
(1) acquiring a preset service requirement; the service requirement comprises a plurality of sub-requirements corresponding to the set area; for each sub-demand, a bucket corresponding to the set region is created.
(2) Establishing storage buckets of corresponding areas based on business requirements, and establishing a main-standby relation between the storage buckets; the main and standby relationship comprises: for each sub-requirement, the bucket corresponding to the sub-requirement is a source bucket of the sub-requirement, and the buckets except the source bucket are standby buckets of the source bucket.
(3) And configuring service read-write logic and cross-region synchronization logic for the buckets created for each region so as to realize data backup of the buckets of each region.
In the above (2), when the primary-standby relationship between the buckets is established, it is usually completed by using a console provided by a domain name service provider, and specifically, when the primary-standby relationship between the buckets is established based on the service requirement in the above (2), the service domain name needs to be added based on the set region corresponding to the sub-requirement; the service domain name carries an identifier of a set area; creating a storage bucket corresponding to each service domain name; for each bucket, adding the service domain name of the standby bucket of the bucket as an alias record of the service domain name of the bucket to establish a primary-standby relationship between the buckets.
For example, the service requirement of the user is mainly described in beijing and shanghai, so when the service domain name is added based on the area corresponding to the service requirement, the service domain name in beijing may be set as: com, and simultaneously creating a Beijing storage bucket which is bucket 1; similarly, for the area in Shanghai, its service domain name may be set as: com, and meanwhile, the storage bucket corresponding to the service domain name created in the area is bucket 2;
further, with the created storage bucket, a CNAME analysis record needs to be further added to establish a primary-backup relationship between the storage buckets, specifically, the CNAME analysis record is usually performed on a domain name management platform provided by a domain name service provider, and also taking the service deployed in beijing and shanghai as an example, the service domain name of beijing is: com, the corresponding added CNAME analytic record is 1:
mainly, buckettl.ks3-cn-beijing.ksyun.com (beijing);
com (Shanghai), buckett 2. ks3-cn-shanghaiksyun;
that is, for the beijing area, the storage bucket created in the beijing area is usually a primary storage bucket, and other areas, such as the storage bucket created in the shanghai area, are standby storage buckets, and by adding the CNAME analysis record, the primary-standby relationship between the storage buckets can be established, and similarly, for the shanghai area, the service domain name in the shanghai area may be: com, the added CNAME record was parsed as:
mainly, buckett 2.ks3-cn-shanghai. ksyun. com (shanghai);
prepare, buckett 1. ks3-cn-beijingksiyun. com (Beijing).
Further, the service read-write logic configured for the bucket created for each region in (3) above generally includes data write logic and data read logic; the service read-write logic is configured to help realize own service at the user side. Specifically, the step of configuring the service read-write logic for the bucket created for each region in (3) above includes: respectively configuring data writing logic and data reading logic for the buckets established for each region; wherein the data write logic comprises: setting prefix information of a file corresponding to write-in data according to an uploading area corresponding to data write-in operation, and storing the file containing the write-in data based on the prefix information; the data read logic includes: and acquiring a file corresponding to the data reading operation, and setting a reading path of the data reading operation as a path corresponding to the area written in the file.
In practical use, the data writing logic corresponds to an operation of uploading a file, which may also be referred to as a writing operation, and specifically, when a user writes data, the user can write the data into the storage buckets in two regions of Beijing and Shanghai when executing the data writing operation, then, the region written with the original data is recorded through the database, and the prefix information of the file corresponding to the written data is set according to the data uploading region, such as adding fixed prefix information to the file and the like, wherein, the file 1.txt is uploaded from Beijing, the file name is the BJPrefix-1.txt according to the BJPrefix-file name, and similarly, the file actively uploaded in Shanghai region can be started according to SHPrefix-.
Further, the data reading logic corresponds to an operation of downloading a file, which may also be referred to as a read operation, and specifically, when a data reading operation is performed, the file corresponding to the data reading operation is usually obtained, and a read path of the data reading operation is set to a path corresponding to an area in which the file is written, so that the file can be preferentially downloaded to an area in which the file is actively written when the file is read.
Further, if the storage service corresponding to the source bucket is available, the storage service of the source bucket may directly respond to the service data operation instruction, and synchronize the data operation corresponding to the service data operation instruction into the standby bucket of the source bucket according to the pre-configured cross-region synchronization logic, so that the object data in the standby bucket is consistent with the object data in the source bucket.
Further, in configuring the cross-region synchronization logic in (3), a user generally completes a configuration process on a console of a public cloud object storage, specifically, the configuration process includes: acquiring a preset synchronous operation range; wherein the synchronization operation range includes at least one of: adding, modifying and deleting, and respectively setting the synchronization logic of the buckets created by each region based on the synchronization operation range so as to keep the synchronization operation of the source bucket and the standby bucket of the source bucket synchronous.
For example, for the buckets in beijing and shanghai regions, when the cross-region synchronization logic is configured, the data of the Bucket1 in beijing region may be configured to be synchronized to the Bucket2, and the synchronization operation range is to increase, delete, and change, that is, the write and read operations of all source buckets are synchronized. Meanwhile, the data of Bucket2 configured in shanghai region is synchronized to Bucket1, and the synchronization operation range is increased, deleted or changed, and at this time, when Bucket2 in shanghai is used as all active buckets, the write and read operations are synchronized.
Further, in view of further saving the storage cost of the public cloud, if the synchronization operation range includes an addition operation, a life cycle may be configured for the object data corresponding to the addition operation. For example, life cycle management is configured for each of the Bucket1 and Bucket2, specifically, for Bucket1, a policy for expiration deletion for 90 days is set, for example, for a file whose matching file prefix satisfies BJPrefix-start, and for Bucket2, a policy for expiration deletion for 90 days is set, so that files in the Bucket are automatically deleted after 90 days from the date of uploading. Therefore, the above-mentioned manner of configuring the lifecycle can save the storage cost, and for the services without cost pressure, the configuration manner of the lifecycle management rule can be omitted, that is, all files are retained, and the configuration may be specifically set according to the actual use situation, which is not limited in the embodiment of the present invention.
Further, for convenience of understanding, fig. 2 shows a schematic view of backup of service data, and it is also described by taking an example that service requirements of users are mainly in beijing and shanghai, that is, public clouds of service requirements are respectively established in beijing and shanghai areas, and a user can respectively log in the public clouds of each area by a client with a legal identity, so as to implement storage and backup of service data. Specifically, the public cloud of the beijing area includes bucket1, and a corresponding CNAME resolution record, and further includes normal data of the area, that is, object data directly written in the bucket of the area by a user, where the object data includes prefix information of the area, and the like. Further, the area also includes backup data backed up from the shanghai area, that is, backup data backed up from bucket2 of the shanghai area through cross-region synchronization logic, and likewise, the public cloud of the shanghai area includes bucket2 and a corresponding CNAME parse record, and also includes normal data in bucket2 and backup data backed up from bucket1 of the beijing area through cross-region synchronization logic, and meanwhile, bucket1 and bucket2 are also configured with a life cycle, for example, a policy of 90-day overdue deletion, and the like.
To sum up, the backup schematic diagram of the service data shown in fig. 2 in the embodiment of the present invention has the following advantages:
(1) the automatic expansion storage service based on public cloud object storage is high in quality and low in price, can realize the real-time copying of data in different places, is beneficial to realizing the cross-region disaster recovery backup of a data center and the hierarchical management or automatic overdue deletion of the data, and reduces the cost.
(2) The storage buckets across the regions are mutually main and standby, when a certain storage bucket service fails, the writing and reading of the service of the storage bucket can be quickly switched, the service side does not sense the failure of the underlying data storage cluster, the service can easily realize disaster tolerance across cities, and emergency switching when the failure occurs can be freely carried out.
Further, on the basis of the foregoing embodiments, an embodiment of the present invention further provides a service data storage exception handling apparatus, as shown in fig. 3, a schematic structural diagram of the service data storage exception handling apparatus, including:
the service request module 30 is configured to request a storage service from a source bucket corresponding to the service data operation instruction based on the acquired service data operation instruction;
a first main parsing record setting module 32, configured to set, if it is detected that a storage service corresponding to a source bucket is unavailable, a service domain name corresponding to another bucket, which is in a master-slave relationship with the source bucket, as a main parsing record, so as to respond to a service data operation instruction through the storage service of the bucket corresponding to the main parsing record;
the data operation copying module 34 is configured to copy a data operation corresponding to the service data operation instruction from the bucket corresponding to the master analytic record to the source bucket if it is detected that the storage service of the source bucket is available;
the second main parsing record setting module 36 is configured to set the service domain name corresponding to the source bucket as the main parsing record, and the service data storage exception handling apparatus provided in the embodiment of the present invention has the same technical features as the service data storage exception handling method provided in the foregoing embodiment, so that the same technical problems can be solved, and the same technical effects can be achieved.
Further, an embodiment of the present invention further provides a server, including a processor and a memory, where the memory stores computer-executable instructions capable of being executed by the processor, and the processor executes the computer-executable instructions to implement the method illustrated in fig. 1.
Further, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method described in fig. 1.
Further, an embodiment of the present invention further provides a schematic structural diagram of a server, as shown in fig. 4, which is the schematic structural diagram of the server, where the server includes a processor 41 and a memory 40, the memory 40 stores a computer executable instruction that can be executed by the processor 41, and the processor 41 executes the computer executable instruction to implement the above-mentioned method for processing the business data storage exception.
In the embodiment shown in fig. 4, the server further comprises a bus 42 and a communication interface 43, wherein the processor 41, the communication interface 43 and the memory 40 are connected by the bus 42.
The Memory 40 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 43 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, etc. may be used. The bus 42 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 42 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 4, but that does not indicate only one bus or one type of bus.
The processor 41 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 41. The Processor 41 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory, and the processor 41 reads information in the memory and completes the service data storage exception handling method of the foregoing embodiment in combination with hardware thereof.
The method, the apparatus, and the computer program product of the server for processing the storage exception of the service data provided in the embodiments of the present invention include a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiments, and specific implementations may refer to the method embodiments and are not described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood in specific cases for those skilled in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that the following embodiments are merely illustrative of the present invention, and not restrictive, and the scope of the present invention is not limited thereto: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (11)
1. A method for processing exception of service data storage is characterized by comprising the following steps:
requesting storage service from a source storage bucket corresponding to the business data operation instruction based on the acquired business data operation instruction;
if the storage service corresponding to the source storage bucket is detected to be unavailable, setting a service domain name corresponding to another storage bucket which is in a master-slave relationship with the source storage bucket as a master analysis record, and responding to the service data operation instruction through the storage service of the storage bucket corresponding to the master analysis record;
if the storage service of the source storage bucket is detected to be available, copying the data operation corresponding to the business data operation instruction from the storage bucket corresponding to the main analytic record to the source storage bucket;
and setting the service domain name corresponding to the source storage bucket as a main analytic record.
2. The method according to claim 1, wherein before requesting storage service from a source bucket corresponding to the service data operation instruction based on the obtained service data operation instruction, the method further comprises:
acquiring a preset service requirement; the service requirement comprises a plurality of sub-requirements corresponding to the set area;
for each sub-demand, creating a bucket corresponding to the set area;
establishing a main-standby relation among a plurality of storage buckets based on the service requirement; the main-standby relationship comprises: for each sub-requirement, a bucket corresponding to the sub-requirement is a source bucket of the sub-requirement, and a bucket except the source bucket is a standby bucket of the source bucket;
and configuring service read-write logic and cross-region synchronization logic for the storage bucket of each set region so as to realize data backup of the storage bucket of each set region.
3. The method of claim 2, wherein the step of creating a bucket corresponding to a set region comprises:
adding a service domain name based on a set area corresponding to the sub-requirement; wherein, the service domain name carries the identifier of the set region;
and creating a storage bucket corresponding to each service domain name.
4. The method of claim 2, wherein the bucket of sub-requirements comprises a service domain name;
based on the service requirement, the step of establishing the main-standby relationship among a plurality of storage buckets comprises the following steps:
and aiming at each storage bucket, adding the service domain name of the standby storage bucket of the storage bucket into an alias record of the service domain name of the storage bucket so as to establish a main-standby relationship between the storage buckets.
5. The method of claim 2, wherein the business read-write logic comprises data write logic and data read logic;
configuring service read-write logic for the buckets created for each region, including:
respectively configuring data writing logic and data reading logic for the buckets created for each region;
wherein the data write logic comprises: setting prefix information of a file corresponding to write-in data according to an uploading area corresponding to data write-in operation, and storing the file containing the write-in data based on the prefix information;
the data read logic comprises: and acquiring a file corresponding to the data reading operation, and setting a reading path of the data reading operation as a path corresponding to the area written in the file.
6. The method of claim 1, further comprising:
responding to the business data operation command through the storage service of the source bucket if the storage service corresponding to the source bucket is available,
and synchronizing the data operation corresponding to the service data operation instruction to a spare bucket of the source bucket according to a pre-configured cross-region synchronization logic so as to keep the object data in the spare bucket consistent with the object data in the source bucket.
7. The method of claim 2, wherein the step of configuring the bucket cross-region synchronization logic for each region creation comprises:
acquiring a preset synchronous operation range; wherein the synchronization operation range includes at least one of: adding, modifying and deleting;
setting the synchronization logic of the buckets created by each zone based on the synchronization operation range respectively so as to keep the synchronization operation of the source bucket and the standby bucket of the source bucket synchronous.
8. The method of claim 7, further comprising:
and if the synchronous operation range comprises an adding operation, configuring a life cycle for the object data corresponding to the adding operation.
9. A device for handling exception of service data storage, comprising:
the service request module is used for requesting storage service from a source storage bucket corresponding to the business data operation instruction based on the acquired business data operation instruction;
a first main parsing record setting module, configured to set, if it is detected that a storage service corresponding to the source bucket is unavailable, a service domain name corresponding to another bucket, which is in a master-slave relationship with the source bucket, as a main parsing record, so as to respond to the service data operation instruction through the storage service of the bucket corresponding to the main parsing record;
a data operation copying module, configured to copy, if it is detected that the storage service of the source bucket is available, a data operation corresponding to the service data operation instruction from a bucket corresponding to the master analytic record to the source bucket;
and the second main analysis record setting module is used for setting the service domain name corresponding to the source storage bucket as a main analysis record.
10. A server comprising a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 8.
11. A computer-readable storage medium having computer-executable instructions stored thereon which, when invoked and executed by a processor, cause the processor to implement the method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111137683.1A CN113849352A (en) | 2021-09-27 | 2021-09-27 | Business data storage exception processing method and device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111137683.1A CN113849352A (en) | 2021-09-27 | 2021-09-27 | Business data storage exception processing method and device and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113849352A true CN113849352A (en) | 2021-12-28 |
Family
ID=78980189
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111137683.1A Pending CN113849352A (en) | 2021-09-27 | 2021-09-27 | Business data storage exception processing method and device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113849352A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396296A (en) * | 2022-08-18 | 2022-11-25 | 中电金信软件有限公司 | Service processing method and device, electronic equipment and computer readable storage medium |
-
2021
- 2021-09-27 CN CN202111137683.1A patent/CN113849352A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115396296A (en) * | 2022-08-18 | 2022-11-25 | 中电金信软件有限公司 | Service processing method and device, electronic equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10977277B2 (en) | Systems and methods for database zone sharding and API integration | |
| US10997211B2 (en) | Systems and methods for database zone sharding and API integration | |
| US9906598B1 (en) | Distributed data storage controller | |
| JP6404907B2 (en) | Efficient read replica | |
| US10140118B2 (en) | Application data synchronization method and apparatus | |
| US8918392B1 (en) | Data storage mapping and management | |
| CN110377395B (en) | Pod migration method in Kubernetes cluster | |
| CN107734026B (en) | Method, device and equipment for designing network additional storage cluster | |
| CN102594849B (en) | Data backup and recovery method and device, virtual machine snapshot deleting and rollback method and device | |
| US11314444B1 (en) | Environment-sensitive distributed data management | |
| US20150213100A1 (en) | Data synchronization method and system | |
| US8930364B1 (en) | Intelligent data integration | |
| EP3899710A1 (en) | Cross cluster replication | |
| US20140019495A1 (en) | Processing a file system operation in a distributed file system | |
| CN111475483B (en) | Database migration method and device and computing equipment | |
| CN111078667B (en) | Data migration method and related device | |
| US9733869B1 (en) | Provisioning a slave for data storage using metadata with updated references | |
| US10452680B1 (en) | Catch-up replication with log peer | |
| CN112394881B (en) | Cloud host migration method and electronic equipment | |
| CN111221678A (en) | Hbase data backup/recovery system, method and device and electronic equipment | |
| CN112883119B (en) | Data synchronization method and device, computer equipment and computer readable storage medium | |
| CN111459749A (en) | Prometous-based private cloud monitoring method and device, computer equipment and storage medium | |
| US11144407B1 (en) | Synchronous database geo-mirroring using delayed visibility write operations | |
| CN113010549A (en) | Data processing method based on remote multi-active system, related equipment and storage medium | |
| CN113157487A (en) | Data recovery method and apparatus thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |