CN115756959A - Data backup method and device and electronic equipment - Google Patents

Data backup method and device and electronic equipment Download PDF

Info

Publication number
CN115756959A
CN115756959A CN202211426965.8A CN202211426965A CN115756959A CN 115756959 A CN115756959 A CN 115756959A CN 202211426965 A CN202211426965 A CN 202211426965A CN 115756959 A CN115756959 A CN 115756959A
Authority
CN
China
Prior art keywords
data
storage
target log
module
newly written
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211426965.8A
Other languages
Chinese (zh)
Inventor
江文龙
牛小龙
罗心
周明伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Dahua Technology Co Ltd
Original Assignee
Zhejiang Dahua Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Dahua Technology Co Ltd filed Critical Zhejiang Dahua Technology Co Ltd
Priority to CN202211426965.8A priority Critical patent/CN115756959A/en
Publication of CN115756959A publication Critical patent/CN115756959A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The method comprises the steps of firstly obtaining storage information corresponding to newly written data in a preset cache space, then generating index information corresponding to the newly written data from the storage information, converting the index information into a target log, and finally pushing the target log to a data backup end so that the data backup end can obtain the newly written data from the preset cache space according to the target log. By the method, the index information corresponding to the newly written data is converted into the target log, and the target log is pushed to the data backup end, so that the data backup end can obtain the newly written data corresponding to the target log in the preset cache region by analyzing the target log, further real-time backup of the data is realized, and the data backup efficiency is improved.

Description

Data backup method and device and electronic equipment
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a data backup method and apparatus, and an electronic device.
Background
Data backup is an off-line protection method of data, and is to prevent a system from losing data due to an operation error or a system failure, and further copy all or part of a data set from a hard disk or an array of an application host to other storage media. In the traditional data backup process, a built-in or external tape unit is mainly adopted to directly read data stored in a data storage node disk so as to realize data backup, but the speed of reading the data by the method is slow, so that the data backup is long, and when new data is written, the new data cannot be backed up in real time, so that the backup data is lost.
Disclosure of Invention
The application provides a data backup method, a data backup device and electronic equipment, and can solve the problems that newly written data cannot be backed up in real time, and the speed of reading the newly written data is slow, so that the time consumption for backing up the newly written data is long.
In a first aspect, the present application provides a data backup method, including:
acquiring storage information corresponding to newly written data in a preset cache space, wherein the storage information at least comprises a storage name, a storage length and a storage address;
generating index information corresponding to the newly written data from the storage information, and converting the index information into a target log, wherein the target log can be identified by any data backup terminal;
and pushing the target log to the data backup end so that the data backup end obtains the newly written data from the preset cache space according to the target log.
By the method, the index information corresponding to the newly written data is converted into the target log, and the target log is pushed to the data backup end, so that the data backup end can obtain the newly written data corresponding to the target log in the preset cache region by analyzing the target log, further real-time backup of the data is realized, and the data backup efficiency is improved.
In a possible design, before the obtaining of the storage information corresponding to the new write data in the preset storage space, the method further includes:
partitioning the newly written data, and naming the partitions to obtain each data block;
partitioning a preset cache space, and numbering the partitions to obtain each cache area;
and storing each data block into each cache region according to the name corresponding to each data block and the number corresponding to each cache region.
By the method, the newly written data are stored to the preset cache space in a blocking mode, and the writing rate and the transmission rate of the data can be improved.
In one possible design, the converting the index information into the target log includes:
determining an index type corresponding to the index information;
inquiring a log conversion template corresponding to the index type;
and converting the index information into the target log according to the log conversion template.
By the method, the index information corresponding to the newly written data is converted into the target log which can be identified by any data backup terminal, and the newly written data can be acquired based on the target log.
In one possible design, the pushing the target log to the data backup side includes:
receiving a target log subscription request, and determining a data backup terminal sending the subscription request;
searching address information corresponding to the data backup terminal in a database;
and pushing the target log to the data backup terminal through the address information.
By the method, the address information of the data backup terminal is stored in the database, so that when the target log is pushed to the data backup terminal, the address information corresponding to the data backup terminal can be directly searched in the database, and the target log is pushed to the data backup terminal through the address information.
In a possible design, after the obtaining the new write data from the preset buffer space, the method further includes:
detecting whether the new written data is completely backed up;
responding to the completion of the backup of the new written data, and detecting whether the new written data is stored in a preset storage space;
and if so, deleting the target log, and deleting the newly written data in the preset cache space.
By the method, when the backup of the newly written data is completed and the newly written data is stored in the preset storage space, the target log in the log management module and the newly written data corresponding to the target log in the preset cache space can be deleted, so that the preset cache space is released, and the cache pressure is reduced.
In a possible design, after the determining, by calculation, the cache regions corresponding to the respective data blocks, further includes:
reading each data block from each cache region;
segmenting each data block to obtain each data slice;
determining each storage node for storing each data slice;
and storing each data slice to each storage node according to the number information and the space size information respectively corresponding to each storage node.
By the method, each data slice is written to different data storage nodes, and when any node fails, the system can still read and write data normally.
In a second aspect, the present application provides a data backup apparatus, the apparatus comprising:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring storage information corresponding to newly written data in a preset cache space, and the storage information at least comprises a storage name, a storage length and a storage address;
the conversion module is used for generating index information corresponding to the newly written data from the storage information and converting the index information into a target log, wherein the target log can be identified by any data backup terminal;
and the pushing module is used for pushing the target log to the data backup end so that the data backup end obtains the newly written data from the preset cache space according to the target log.
In one possible design, the apparatus further includes:
the blocking module is used for blocking the newly written data and naming the blocks to obtain each data block;
the partition module is used for partitioning a preset cache space and numbering the partitions to obtain each cache area;
and the first storage module is used for storing each data block to each cache region according to the name corresponding to each data block and the number corresponding to each cache region.
In one possible design, the conversion module is specifically configured to:
determining an index type corresponding to the index information;
inquiring a log conversion template corresponding to the index type;
and converting the index information into the target log according to the log conversion template.
In a possible design, the push module is specifically configured to:
receiving a target log subscription request, and determining a data backup terminal sending the subscription request;
searching address information corresponding to the data backup terminal in a database;
and pushing the target log to the data backup terminal through the address information.
In one possible design, the apparatus further includes:
the first detection module is used for detecting whether the new written data is completely backed up;
the second detection module is used for responding to the completion of the backup of the new written data and detecting whether the new written data is stored in a preset storage space;
and the deleting module is used for deleting the target log if the newly written data is stored in a preset storage space, and deleting the newly written data in the preset cache space.
In one possible design, the apparatus further includes:
the reading module is used for reading each data block from each cache region;
the fragmentation module is used for fragmenting each data block to obtain each data fragment;
the determining module is used for determining each storage node for storing each data slice;
and the second storage module is used for storing each data slice to each storage node according to the number information and the space size information which are respectively corresponding to each storage node.
In a third aspect, the present application provides an electronic device, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the data backup method when executing the computer program stored in the memory.
In a fourth aspect, the present application provides a computer-readable storage medium having stored therein a computer program which, when executed by a processor, implements the data backup method steps described above.
According to the data backup method, the index information corresponding to the newly written data is converted into the target log, and the target log is pushed to the data backup end, so that the data backup end can obtain the newly written data corresponding to the target log in the preset cache area by analyzing the target log, real-time backup of the data is achieved, and the data backup efficiency is improved.
For each of the second to fourth aspects and possible technical effects of each aspect, reference is made to the above description of the possible technical effects of the first aspect or various possible schemes of the first aspect, and repeated description is omitted here.
Drawings
Fig. 1 is a flowchart of a data backup method provided in the present application;
FIG. 2 is a schematic diagram of a data storage method provided in the present application;
fig. 3 is a second schematic diagram of a data storage method provided in the present application;
fig. 4 is a schematic view of an application scenario of a data backup method provided in the present application;
fig. 5 is a schematic structural diagram of a data backup device provided in the present application;
fig. 6 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, the present application will be further described in detail with reference to the accompanying drawings. The particular methods of operation in the method embodiments may also be applied to apparatus embodiments or system embodiments. It should be noted that "a plurality" is understood as "at least two" in the description of the present application. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, and A and B exist, and B exists alone. A is connected with B and can represent: a and B are directly connected and A and B are connected through C. In addition, in the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not intended to indicate or imply relative importance nor order to be construed.
The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
In the current data backup process, a built-in or external tape unit is mainly used for directly reading data stored in a data storage node disk so as to realize data backup, but the speed for reading the data is slow, so that the data backup is long, and when new data is written, the new data cannot be backed up in real time, so that the backup data is lost.
In order to solve the above problem, in the data backup method provided in the embodiment of the present application, index information corresponding to newly written data is converted into a target log, and the target log is pushed to a data backup end, so that the data backup end can obtain the newly written data corresponding to the target log in a preset cache region by analyzing the target log, thereby implementing real-time backup of the data and improving the efficiency of data backup. The method and the device in the embodiment of the application are based on the same technical concept, and because the principles of the problems solved by the method and the device are similar, the device and the embodiment of the method can be mutually referred, and repeated parts are not repeated.
In order to make the objects, technical solutions and advantages of the present application more clear, the present application is further described in detail below with reference to the accompanying drawings.
As shown in fig. 1, a flowchart of a data backup method provided in the present application specifically includes the following steps:
s11, acquiring storage information corresponding to the newly written data in a preset cache space;
s12, generating index information corresponding to newly written data from the stored information, and converting the index information into a target log;
and S13, pushing the target log to a data backup end so that the data backup end obtains newly written data from a preset cache space according to the target log.
In the embodiment of the present application, when new write data is generated, the new write data is first partitioned into blocks, the size of each block is defaulted to 64M, and certainly, the new write data may also be set according to an actual service scenario, and the blocks are named, for example, according to a format of "data name + current time + sequence number" to obtain each data block. Meanwhile, partitioning a preset cache space, wherein the preset cache space is composed of solid state disks with a certain proportion of capacity, the characteristic that the solid state disks can read data quickly is utilized, the data reading and writing speed is improved, the partitions are numbered to obtain each cache region, and then hash operation is carried out on the names respectively corresponding to the data blocks and the numbers respectively corresponding to the cache regions to determine the cache regions respectively corresponding to the data blocks.
As shown in fig. 2, when new write data is generated, the new write data is partitioned and the partitions are named to obtain N data blocks, where N represents an integer greater than or equal to 1. Meanwhile, the preset cache space is partitioned, the partitions are numbered to obtain N cache regions, then, hash operations are respectively performed on the names corresponding to the N data blocks and the numbers corresponding to the N cache regions, and it can be determined that the cache region corresponding to the data block 1 is the cache region 1, the cache region corresponding to the data block 2 is the cache region 2, the cache region corresponding to the data block 3 is the cache region 3, and the cache region corresponding to the data block N is the cache region N.
And after the cache regions corresponding to the data blocks are determined, storing the data blocks into the cache regions. Specifically, a cache segment for storing a data block is applied from a cache region, address information corresponding to the cache segment in the cache region is obtained, when the data block is written into the cache segment, the cache segment is divided into two parts, the first part is used for storing index information of the data block, the space size can be 4 kbytes, the second part is used for storing the data block, and storage information corresponding to the data block in the cache segment is obtained according to the address information corresponding to the cache segment in the cache region, wherein the storage information at least comprises a storage name, a storage length and a storage address.
Further, the storage information is generated into index information corresponding to the data block and stored in the first part of the cache segment, then the index information is sent to the log management module through the client, the log management module queries the log conversion template corresponding to the index type after determining the index type corresponding to the index information, wherein each index type corresponds to a different log conversion template, and the service end to which each log template is applied is different, for example, the log template 1 corresponding to the index type 1 can be identified by the data backup end, and the log template 2 corresponding to the index type 2 cannot be identified by the data backup end. And then, converting the index information into a target log according to a log conversion template, wherein the target log can be identified by any data backup terminal.
As shown in table 1, the format of the target log is mainly composed of a name DataName of the data block, a storage address offset corresponding to the data block, a length len of the data block, and a number index of a cache region corresponding to the data block.
Figure BDA0003942661520000091
TABLE 1 target Log Format
After the target log is obtained, the target log is pushed to a data backup end, so that the data backup end obtains newly written data from a preset cache space according to the target log. Specifically, before data backup, a target log subscription request is received, a data backup end sending the subscription request is determined, and then address information of the data backup end is stored in a database, so that when the target log is pushed to the data backup end, the address information corresponding to the data backup end can be directly searched in the database, and the target log is pushed to the data backup end through the address information.
By the method, the index information corresponding to the newly written data is converted into the target log which can be identified by any data backup terminal, and the newly written data can be acquired based on the target log.
After receiving the target log, the data backup terminal analyzes the target log through the log analysis module and sends an analysis result to the data reading module, and the data reading module carries the analysis result and obtains new write-in data corresponding to the target log from a preset cache space. And then, the data backup terminal calls a data writing module, stores the acquired new written data into a local storage for backup, and meanwhile, the subscription request receiving terminal detects whether the new written data corresponding to the target log is completely backed up.
Responding to the completion of the backup of the new written data corresponding to the target log, detecting whether the new written data corresponding to the target log is stored in a preset storage space, if so, deleting the target log in the log management module, deleting the new written data corresponding to the target log in the preset cache space by a cache data deleting thread, and releasing the preset cache space; if not, the receiving end of the subscription request is waited to store the new written data in the preset cache space to the preset storage space.
By the method, the newly written data are acquired based on the target log, so that the newly written data are backed up in real time, and the data backup efficiency is improved.
In the embodiment of the application, not only can the newly written data in the preset cache space be backed up in real time, but also the newly written data can be stored in the preset storage space, so that the data cannot be lost, wherein the preset storage space is composed of a plurality of data storage nodes.
Specifically, the preset storage space is stored in a Metadata management server (MDS) for registration, and the MDS allocates a serial number to each data storage node in the preset storage space, so that different data can be stored in the corresponding data storage node. Then, the data reading module reads each data block from each thread in the cache area, and in order to improve reliability of the data block, the data block may be partitioned according to a preset method to obtain each data slice, for example, the size of the read data block is 4M, and the data block is partitioned into 4 data slices with a size of 1M and 1 check data slice with a size of 1M according to the redundancy rule of "4+1". Further, the data reading module thread acquires the number information and the space size information of each storage node in the preset storage space from the MDS, and stores each data slice to each storage node according to the number information and the space size information corresponding to each storage node.
As shown in fig. 3, the MDS of the data reading module thread acquires the number information and the space size information of each storage node in the preset storage space, and according to the number information and the space size information corresponding to each storage node, the data slice 1 is stored in the storage node 1, the data slice 2 is stored in the storage node 2, the data slice 3 is stored in the storage node 3, the data slice 4 is stored in the storage node 4, and the data slice 5 is stored in the storage node 5.
By the method, each data slice is written to different data storage nodes, and when any node fails, the system can still read and write data normally.
Further, the subscription request receiving end updates a second state corresponding to the target log into new write data and stores the new write data into a preset storage space, then detects whether a first state corresponding to the target log is updated into the new write data to complete backup, if so, deletes the target log, deletes the new write data corresponding to the target log in the preset cache space by the cache data deleting thread, and releases the preset cache space; if not, waiting for the data backup end to acquire new write-in data corresponding to the target log.
By the data backup method, the index information corresponding to the newly written data is converted into the target log, and the target log is pushed to the data backup end, so that the data backup end can obtain the newly written data corresponding to the target log in the preset cache region by analyzing the target log, further real-time backup of the data is realized, and the data backup efficiency is improved. To explain the data backup method provided by the present application in more detail, a specific application scenario is described below, where the application scenario mainly includes interactions among 2 clusters, where a cluster a is a terminal receiving a subscription request, and a cluster B is a data backup terminal, as shown in fig. 4.
1. The cluster A and the cluster B establish subscription service, the cluster B sends a target log subscription request to the cluster A, and the cluster A stores the address information of the cluster B into a memory and/or a database.
2. When the client stores the newly written data into the cache layer, firstly, the cache layer is partitioned and the partitions are numbered to obtain each cache region, meanwhile, the newly written data are partitioned and named to obtain each data block, then, hash operation is carried out on naming information of the data blocks by using a hash algorithm to calculate the cache region corresponding to each data block, each data block is stored into each cache region, after the data blocks are successfully stored, storage information corresponding to the data blocks in the cache regions is obtained, index information corresponding to the data blocks is generated by the storage information, finally, the cache layer returns the index information to the client, and the client transmits the index information to the log management module.
3. The log management module converts the index information into a target log and adds the target log into a write management queue writeOpQueue, wherein a data structure in the writeOpQueue records two states corresponding to the target log, the first state represents whether newly written data are stored in a storage layer, the second state represents whether the newly written data are read by a cluster B and are successfully backed, and further, the log management module transmits the target log to a subscription service and pushes the target log to the cluster B by inquiring address information of the cluster B.
4. After receiving the target log, the cluster B sends the target log to a log analysis module, the log analysis module analyzes the target log and sends an analysis result to a data reading module, the data reading module carries the analysis result to a cache layer of the cluster A to read newly written data, and after the newly written data is read, the data writing module is called to store the newly written data in a local storage of the cluster B for backup.
5. After the data backup is successful, updating a second state corresponding to a target log in a cluster A, writeOpQueue, to be 1, wherein the second state is 1, which indicates that newly written data has been read by a cluster B and the backup is successful, then, judging whether a first state corresponding to the target log is 1, wherein the first state is 1, which indicates that the newly written data is stored in a storage layer, if so, deleting the target log from the writeOpQueue, adding the target log into a delete management queue, deleting the newly written data corresponding to the target log in a cache layer by a delete cache data thread, and releasing the space of the cache layer; if not, waiting for the cluster A to store the newly written data in the cache layer to the storage layer.
5. When newly written data in a cache layer is stored in a storage layer, firstly, the storage layer is registered with an MDS, the MDS allocates a number for each data storage node in the storage layer, then, a data reading module reads each data block from each cache region and fragments each data block according to a preset method to obtain each data fragment, and further, the data reading module reads the number information and the space size information of each storage node from the MDS, and stores each data fragment to each storage node according to the number information and the space size information which correspond to each storage node.
6. After new write in the cache layer is stored in the storage layer, updating a first state corresponding to a target log in the writeOpQueue to 1, then, judging whether a second state corresponding to the target log is 1, if so, deleting the target log from the writeOpQueue and adding the target log into the deleteOpQueue, and if so, traversing the target log in the deleteOpQueue at regular time by a cache data deleting thread, deleting new write data corresponding to the target log in the cache layer and releasing the space of the cache layer; if not, waiting for the cluster B to read new write data corresponding to the target log.
Based on the same inventive concept, an embodiment of the present application further provides a data backup apparatus, as shown in fig. 5, which is a schematic structural diagram of the data backup apparatus in the present application, and the apparatus includes:
the obtaining module 51 is configured to obtain storage information corresponding to the newly written data in a preset cache space, where the storage information at least includes a storage name, a storage length, and a storage address;
a conversion module 52, configured to generate, from the storage information, index information corresponding to the newly written data, and convert the index information into a target log, where the target log can be identified by any data backup terminal;
the pushing module 53 is configured to push the target log to the data backup end, so that the data backup end obtains the newly written data from the preset cache space according to the target log.
In one possible design, the apparatus further includes:
the blocking module is used for blocking the newly written data and naming the blocks to obtain each data block;
the partition module is used for partitioning a preset cache space and numbering the partitions to obtain each cache area;
and the first storage module is used for storing each data block to each cache region according to the name corresponding to each data block and the number corresponding to each cache region.
In one possible design, the conversion module 52 is specifically configured to:
determining an index type corresponding to the index information;
inquiring a log conversion template corresponding to the index type;
and converting the index information into the target log according to the log conversion template.
In a possible design, the pushing module 53 is specifically configured to:
receiving a target log subscription request, and determining a data backup terminal sending the subscription request;
searching address information corresponding to the data backup terminal in a database;
and pushing the target log to the data backup terminal through the address information.
In one possible design, the apparatus further includes:
the first detection module is used for detecting whether the new written data is completely backed up;
the second detection module is used for responding to the completion of the backup of the new written data and detecting whether the new written data is stored in a preset storage space;
and the deleting module is used for deleting the target log if the newly written data is stored in a preset storage space, and deleting the newly written data in the preset cache space.
In one possible design, the apparatus further includes:
the reading module is used for reading each data block from each cache region;
the fragmentation module is used for fragmenting each data block to obtain each data fragment;
the determining module is used for determining each storage node for storing each data slice;
and the second storage module is used for storing each data slice to each storage node according to the number information and the space size information which are respectively corresponding to each storage node.
Based on the data backup device, the index information corresponding to the newly written data is converted into the target log, and the target log is pushed to the data backup end, so that the data backup end can obtain the newly written data corresponding to the target log in the preset cache region by analyzing the target log, thereby realizing real-time backup of the data and improving the efficiency of data backup.
Based on the same inventive concept, an embodiment of the present application further provides an electronic device, where the electronic device may implement the function of the foregoing data backup apparatus, and with reference to fig. 6, the electronic device includes:
at least one processor 61, and a memory 62 connected to the at least one processor 61, in this embodiment, a specific connection medium between the processor 61 and the memory 62 is not limited in this application, and fig. 6 illustrates an example in which the processor 61 and the memory 62 are connected through a bus 60. The bus 60 is shown in fig. 6 by a thick line, and the connection between other components is merely illustrative and not intended to be limiting. The bus 60 may be divided into an address bus, a data bus, a control bus, etc., and is shown with only one thick line in fig. 6 for ease of illustration, but does not represent only one bus or type of bus. Alternatively, the processor 61 may also be referred to as a controller, without limitation to name a few.
In the embodiment of the present application, the memory 62 stores instructions executable by the at least one processor 61, and the at least one processor 61 may execute the data backup method discussed above by executing the instructions stored in the memory 62. The processor 61 may implement the functions of the various modules in the apparatus shown in fig. 5.
The processor 61 is a control center of the apparatus, and may connect various parts of the entire control device by using various interfaces and lines, and perform various functions of the apparatus and process data by operating or executing instructions stored in the memory 62 and calling data stored in the memory 62, thereby performing overall monitoring of the apparatus.
In one possible design, processor 61 may include one or more processing units, and processor 61 may integrate an application processor, which primarily handles operating systems, user interfaces, application programs, and the like, and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 61. In some embodiments, the processor 61 and the memory 62 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.
The processor 61 may be a general-purpose processor, such as a Central Processing Unit (CPU), digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, that implements or performs the methods, steps, and logic blocks disclosed in embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the data backup method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
The memory 62, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 62 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and the like. The memory 62 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 62 in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.
The processor 61 is programmed to solidify the codes corresponding to the data backup method described in the foregoing embodiment into the chip, so that the chip can execute the steps of the data backup method of the embodiment shown in fig. 1 when running. How to program the processor 61 is well known to those skilled in the art and will not be described in detail herein.
Based on the same inventive concept, embodiments of the present application further provide a storage medium storing computer instructions, which, when executed on a computer, cause the computer to perform the data backup method discussed above.
In some possible embodiments, the aspects of the data backup method provided by the present application may also be implemented in the form of a program product comprising program code for causing the control apparatus to perform the steps of the data backup method according to various exemplary embodiments of the present application described above in this specification when the program product is run on a device.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (14)

1. A method for backing up data, the method comprising:
acquiring storage information corresponding to newly written data in a preset cache space, wherein the storage information at least comprises a storage name, a storage length and a storage address;
generating index information corresponding to the newly written data from the storage information, and converting the index information into a target log, wherein the target log can be identified by any data backup terminal;
and pushing the target log to the data backup terminal so that the data backup terminal acquires the newly written data from the preset cache space according to the target log.
2. The method of claim 1, wherein before the obtaining of the storage information corresponding to the new write data in the preset storage space, the method further comprises:
partitioning the newly written data, and naming the partitions to obtain each data block;
partitioning a preset cache space, and numbering the partitions to obtain each cache area;
and storing each data block to each cache region according to the name corresponding to each data block and the number corresponding to each cache region.
3. The method of claim 1, wherein said converting the index information into a target log comprises:
determining an index type corresponding to the index information;
inquiring a log conversion template corresponding to the index type;
and converting the index information into the target log according to the log conversion template.
4. The method of claim 1, wherein the pushing the target log to the data backup site comprises:
receiving a target log subscription request, and determining a data backup terminal sending the subscription request;
searching address information corresponding to the data backup terminal in a database;
and pushing the target log to the data backup terminal through the address information.
5. The method of claim 1, wherein after the obtaining the new write data from the predetermined buffer space, further comprising:
detecting whether the new written data is completely backed up;
responding to the completion of the backup of the new written data, and detecting whether the new written data is stored in a preset storage space;
and if so, deleting the target log, and deleting the newly written data in the preset cache space.
6. The method as claimed in claim 2, wherein after said determining, by calculation, the buffer areas corresponding to the respective data blocks, further comprises:
reading each data block from each cache region;
segmenting each data block to obtain each data slice;
determining each storage node for storing each data slice;
and storing each data slice to each storage node according to the number information and the space size information respectively corresponding to each storage node.
7. A data backup apparatus, characterized in that the apparatus comprises:
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring storage information corresponding to newly written data in a preset cache space, and the storage information at least comprises a storage name, a storage length and a storage address;
the conversion module is used for generating index information corresponding to the newly written data from the storage information and converting the index information into a target log, wherein the target log can be identified by any data backup terminal;
and the pushing module is used for pushing the target log to the data backup end so that the data backup end obtains the newly written data from the preset cache space according to the target log.
8. The apparatus of claim 7, wherein the apparatus further comprises:
the blocking module is used for blocking the newly written data and naming the blocks to obtain each data block;
the partition module is used for partitioning a preset cache space and numbering the partitions to obtain each cache area;
and the first storage module is used for storing each data block to each cache region according to the name corresponding to each data block and the number corresponding to each cache region.
9. The apparatus of claim 7, wherein the conversion module is specifically configured to:
determining an index type corresponding to the index information;
inquiring a log conversion template corresponding to the index type;
and converting the index information into the target log according to the log conversion template.
10. The apparatus of claim 7, wherein the push module is specifically configured to:
receiving a target log subscription request, and determining a data backup end for sending the subscription request;
searching address information corresponding to the data backup terminal in a database;
and pushing the target log to the data backup terminal through the address information.
11. The apparatus of claim 7, wherein the apparatus further comprises:
the first detection module is used for detecting whether the new written data is completely backed up;
the second detection module is used for responding to the completion of the backup of the new written data and detecting whether the new written data is stored in a preset storage space;
and the deleting module is used for deleting the target log if the newly written data is stored in a preset storage space, and deleting the newly written data in the preset cache space.
12. The apparatus of claim 8, wherein the apparatus further comprises:
the reading module is used for reading each data block from each cache region;
the fragmentation module is used for fragmenting each data block to obtain each data fragment;
the determining module is used for determining each storage node for storing each data slice;
and the second storage module is used for storing each data slice to each storage node according to the number information and the space size information which are respectively corresponding to each storage node.
13. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1-6 when executing the computer program stored on the memory.
14. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1-6.
CN202211426965.8A 2022-11-14 2022-11-14 Data backup method and device and electronic equipment Pending CN115756959A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211426965.8A CN115756959A (en) 2022-11-14 2022-11-14 Data backup method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211426965.8A CN115756959A (en) 2022-11-14 2022-11-14 Data backup method and device and electronic equipment

Publications (1)

Publication Number Publication Date
CN115756959A true CN115756959A (en) 2023-03-07

Family

ID=85371446

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211426965.8A Pending CN115756959A (en) 2022-11-14 2022-11-14 Data backup method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN115756959A (en)

Similar Documents

Publication Publication Date Title
US11474972B2 (en) Metadata query method and apparatus
US7293145B1 (en) System and method for data transfer using a recoverable data pipe
US11461027B2 (en) Deduplication-aware load balancing in distributed storage systems
US7058783B2 (en) Method and mechanism for on-line data compression and in-place updates
US10936228B2 (en) Providing data deduplication in a data storage system with parallelized computation of crypto-digests for blocks of host I/O data
US20160350302A1 (en) Dynamically splitting a range of a node in a distributed hash table
CN108733306B (en) File merging method and device
EP3876106A1 (en) File storage method and deletion method, server, and storage medium
WO2017020576A1 (en) Method and apparatus for file compaction in key-value storage system
CN111061752B (en) Data processing method and device and electronic equipment
KR20180008774A (en) Method and system for implementing indexing when saving files
CN111381779A (en) Data processing method, device, equipment and storage medium
CN111177143A (en) Key value data storage method and device, storage medium and electronic equipment
CN114936188A (en) Data processing method and device, electronic equipment and storage medium
CN109388651B (en) Data processing method and device
CN113253932B (en) Read-write control method and system for distributed storage system
US20240070120A1 (en) Data processing method and apparatus
US11210236B2 (en) Managing global counters using local delta counters
CN111694806A (en) Transaction log caching method, device, equipment and storage medium
CN115756959A (en) Data backup method and device and electronic equipment
CN108021562B (en) Disk storage method and device applied to distributed file system and distributed file system
CN111782622B (en) Log processing method, device, server and storage medium
CN115904211A (en) Storage system, data processing method and related equipment
CN116868173A (en) Reducing the impact of network latency during recovery operations
CN115793957A (en) Method and device for writing data and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination