CN111880967A - File backup method, device, medium and electronic equipment in cloud scene - Google Patents

File backup method, device, medium and electronic equipment in cloud scene Download PDF

Info

Publication number
CN111880967A
CN111880967A CN202010739622.1A CN202010739622A CN111880967A CN 111880967 A CN111880967 A CN 111880967A CN 202010739622 A CN202010739622 A CN 202010739622A CN 111880967 A CN111880967 A CN 111880967A
Authority
CN
China
Prior art keywords
file
backed
list
attribute information
backup
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010739622.1A
Other languages
Chinese (zh)
Inventor
赵宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202010739622.1A priority Critical patent/CN111880967A/en
Priority to PCT/CN2020/117777 priority patent/WO2021139224A1/en
Publication of CN111880967A publication Critical patent/CN111880967A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • G06F11/1451Management of the data involved in backup or backup restore by selection of backup contents
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • G06F16/137Hash-based
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The disclosure relates to the technical field of cloud, and discloses a file backup method, device, medium and electronic equipment in a cloud scene. The method comprises the following steps: acquiring a backup file directory from a backup service database; acquiring a first file attribute information list corresponding to the backup file directory in a cloud server; determining a list of files to be backed up by comparing a second file attribute information list with the first file attribute information list in the backup service database; and reading the file to be backed up corresponding to the file list to be backed up from the cloud server based on the file list to be backed up so as to backup the file to be backed up. In the method, the second file attribute information list and the first file attribute information list are compared in the backup service database, so that the memory comparison can be reduced, the utilization rate of the memory is saved, and the safety of the file attribute information is also ensured.

Description

File backup method, device, medium and electronic equipment in cloud scene
Technical Field
The present disclosure relates to the field of cloud technologies, and in particular, to a method, an apparatus, a medium, and an electronic device for file backup in a cloud scenario.
Background
At present, in order to improve the reliability and the security of cloud resources, the demand for performing backup service on the cloud resources is increasing, and in order to avoid resource data loss in a cloud host due to hardware failure, a cloud backup technology is becoming more and more important.
In the prior art, cloud backup in a cloud scene is mainly performed by acquiring file data in a cloud host and then comparing the file data with backed-up file data, so as to determine the file data to be backed up.
However, how to reduce the memory occupation in the file backup process to ensure the reliability and security of the file backup is an urgent technical problem to be solved.
Disclosure of Invention
The present disclosure aims to provide a file backup method and apparatus in a cloud scenario, a computer-readable storage medium, and an electronic device, so as to at least reduce the occupation of a memory in a file backup process and ensure the reliability and security of the file backup.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to an aspect of the embodiments of the present disclosure, a file backup method in a cloud scenario is provided, where the method includes: acquiring a backup file directory from a backup service database; acquiring a first file attribute information list corresponding to the backup file directory in a cloud server; determining a list of files to be backed up by comparing a second file attribute information list with the first file attribute information list in the backup service database; and reading the file to be backed up corresponding to the file list to be backed up from the cloud server based on the file list to be backed up so as to backup the file to be backed up.
According to an aspect of the embodiments of the present disclosure, there is provided a file backup apparatus in a cloud scenario, the apparatus including: a first acquisition unit configured to acquire a backup file directory from a backup service database; a second obtaining unit, configured to obtain, in the cloud server, a first file attribute information list corresponding to the backup file directory; a determining unit, configured to determine a list of files to be backed up by comparing a second file attribute information list with the first file attribute information list in the backup service database; and the reading unit is used for reading the file to be backed up corresponding to the file list to be backed up from the cloud server based on the file list to be backed up so as to backup the file to be backed up.
According to an aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, the computer program including executable instructions that, when executed by a processor, implement the file backup method in the cloud scenario as described in the above embodiments.
According to an aspect of an embodiment of the present disclosure, there is provided an electronic device including: one or more processors; a memory for storing executable instructions of the processor, which when executed by the one or more processors, cause the one or more processors to implement the file backup method in the cloud scenario as described in the above embodiments.
In the technical solutions of some embodiments of the present disclosure, a first file attribute information list is obtained in a cloud server through a backup file directory obtained from a backup service database, and a second file attribute information list is compared with the first file attribute information list in the backup service database, so that a to-be-backed-up file list can be determined, and a corresponding to-be-backed-up file is read from the cloud server according to the to-be-backed-up file list. The file list to be backed up is obtained by comparing the second file attribute information list with the first file attribute information list in the backup service database, so that the memory comparison can be reduced, the occupation of the memory in the file backup process is reduced, the utilization rate of the memory is further saved, and the reliability and the safety of the file in the backup process are ensured.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The above and other features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
Fig. 1 is a schematic view illustrating an application scenario of a file backup method in a cloud scenario according to an embodiment of the present disclosure;
FIG. 2 shows a flowchart of a file backup method in a cloud scenario according to one embodiment of the present disclosure;
fig. 3 is a detailed flowchart illustrating obtaining a first file attribute information list corresponding to the backup file directory in a cloud server according to an embodiment of the present disclosure;
FIG. 4 illustrates a flowchart of a method before sending a file backup request to the cloud server, according to one embodiment of the present disclosure;
FIG. 5 illustrates a detailed flow diagram for determining a list of files to be backed up according to one embodiment of the present disclosure;
FIG. 6 illustrates a detailed flow diagram for determining a list of files to be backed up according to one embodiment of the present disclosure;
fig. 7 is a flowchart illustrating details of reading a file to be backed up corresponding to the file list to be backed up from a cloud server according to an embodiment of the present disclosure;
FIG. 8 is a detailed flowchart illustrating the determination of at least two sub-lists of files to be backed up from the list of files to be backed up according to one embodiment of the present disclosure;
fig. 9 illustrates a block diagram of a file backup apparatus in a cloud scenario according to an embodiment of the present disclosure;
FIG. 10 illustrates a computer-readable storage medium implementing a file backup method in a cloud scenario according to one embodiment of the present disclosure;
fig. 11 shows an example block diagram of an electronic device implementing a file backup method in a cloud scenario according to an embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It is also noted that the terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the objects so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than those illustrated or described herein.
First, the present disclosure briefly explains an application scenario of a file backup method in a cloud scenario.
In an embodiment of the present application, the file backup method in the cloud scenario may be implemented in the scenario shown in fig. 1, and referring to fig. 1, an application scenario diagram of the file backup method in the cloud scenario according to an embodiment of the present disclosure is shown.
In fig. 1, an application scenario in which a user backs up a file in a cloud scenario is shown. Specifically, the file backup method in the cloud scenario may be implemented based on the backup server 103 shown in the figure, and of course, the backup server shown in the figure may also be replaced by an electronic device with a touch screen, such as a tablet computer and a notebook computer.
It should be further noted that the file backup method in the cloud scenario may also be executed by a cloud server having a cloud computing function. Specifically, the cloud computing (cloud computing) is a computing mode, which distributes computing tasks on a resource pool formed by a large number of computers, so that various application systems can acquire computing power, storage space and information services as required. The network that provides the resources is referred to as the "cloud". Resources in the cloud can be infinitely expanded to users, and can be acquired at any time, used as required and expanded at any time. The cloud computing resource pool mainly comprises computing equipment (which is a virtualization machine and comprises an operating system), storage equipment and network equipment.
In the scenario illustrated in fig. 1, a user may send a file backup request to the backup service database 102 through the user terminal 101, and a file backup task is immediately generated in the backup service database 102, where it is to be noted that information such as file backup time, file backup times, file backup frequency, and a file directory of a file to be backed up is recorded in the file backup task.
The backup server 103 may obtain a backup file directory in a file backup task from the backup service database 102, obtain a first file attribute information list corresponding to the backup file directory in the cloud server 104, determine a file list to be backed up by comparing a second file attribute information list with the first file attribute information list in the backup service database 102, and finally read a file to be backed up corresponding to the file list to be backed up from the cloud server 104 based on the file list to be backed up to backup the file to be backed up. It should be noted that the second file attribute information list includes file attribute information of backed-up files, which is stored in the backup service data 102.
The following describes details of implementation of the technical solution of the embodiment of the present application in detail with reference to fig. 2:
referring to fig. 2, a flow diagram illustrating file backup in a cloud scenario is shown, according to one embodiment of the present disclosure. The file backup method in the cloud scenario may be executed by an entity device or a virtual device having a computing processing function. As shown in fig. 2, the file backup method in the cloud scenario at least includes steps 210 to 270:
in step 210, a directory of backup files is retrieved from a backup service database.
In the application, the backup service database may be created in advance and is mainly used for storing a file backup task initiated by a user, wherein the file backup task includes the backup time, the backup frequency of a file and a file directory of the file to be backed up. For example, a user a initiates a file backup task for backing up picture files, where the backup time is 00 minutes 00 seconds at 7 months, 1 day, 00 hour in 2020, the backup frequency is 8 times, the backup frequency is once per week, and the file directories where the picture files to be backed up are "directory 1, directory 2, directory 3" in the cloud server.
In addition, the backup service database may further store a file backup record of a user in history, where the file backup record includes a second file attribute information list (i.e. a file attribute information list of files that have been backed up in history), where the second file attribute information list includes file attribute information of at least one second file, and it should be understood by those skilled in the art that one second file corresponds to one second file attribute information.
It should be noted that, as shown in table 1, the file attribute information of each file is an attribute information list, and in the present application, the file attribute information of each file may include one or more of attribute information such as a file name, a file path, a file size, and a file modification time.
File numbering File name File path File size File modification time
Document 1 Picture 1 Route 1 2153KB 2020/7/1
Document 2 Picture 2 Route 2 2312KB 2020/7/2
…… …… …… …… ……
File n Picture n Path n 3278KB 2020/7/3
TABLE 1
It should be noted that, if the user has not performed file backup historically, the backup service database does not include the second file attribute information list.
In step 210, a backup file directory, that is, a file directory corresponding to a file to be backed up, is mainly obtained from the backup service database.
In the application, the storage position of the file which needs to be backed up by the user can be found in the cloud server storing the user file through the backup file directory.
With continued reference to fig. 2, in step 230, a first file attribute information list corresponding to the backup file directory is obtained in the cloud server.
In the application, at least one original file of a user is generally stored in a cloud server, each original file corresponds to one piece of file attribute information, and similarly, the file attribute information corresponding to each original file may also be one or more pieces of attribute information including a file name, a file path, a file size, a file modification time, and the like.
In the application, the original file corresponding to the backup file directory is a file object that the user needs to backup, and therefore, a first file attribute information list corresponding to the backup file directory is obtained in the cloud server, that is, the file attribute information list corresponding to the file that the user needs to backup.
In an embodiment of the present application, the first file attribute information list corresponding to the backup file directory may be acquired in the cloud server at one time.
In an embodiment of the present application, the obtaining of the first file attribute information list corresponding to the backup file directory in the cloud server may also be performed according to the steps shown in fig. 3.
Referring to fig. 3, a detailed flowchart of obtaining the first file attribute information list corresponding to the backup file directory in the cloud server according to an embodiment of the present disclosure is shown. Specifically, the method comprises steps 231 to 232:
in step 231, a file backup request is sent to the cloud server based on the backup file directory.
In step 232, the first file attribute information sent by the cloud server is received in batches, so as to obtain the first file attribute information list.
In the application, after receiving a file backup request, the cloud server may scan in its own memory according to the backup file directory, so as to determine an original file (i.e., a first file) that a user needs to backup, and further determine first file attribute information to obtain a first file attribute information list.
After obtaining the first file attribute information list, the cloud service may send the first file attribute information in the first file attribute information list in batches, where each batch sends a part of the first file attribute information in the first file attribute information list, for example, 1000 pieces of the first file attribute information are sent in each batch.
Therefore, in this embodiment, a file backup request may be sent to the cloud server based on the backup file directory, and then the first file attribute information sent by the cloud server is received in batches to obtain the first file attribute information list. The method has the advantages that the first file attribute information sent by the cloud server is received in batches, so that the situation of network congestion caused by receiving the first file attribute information with overlarge quantity at one time can be prevented, and the stability of file backup is further ensured.
In this embodiment, before sending the file backup request to the cloud server based on the backup file directory, the steps shown in fig. 4 may also be performed.
Referring to fig. 4, a flowchart of a method before sending a file backup request to the cloud server is shown, according to one embodiment of the present disclosure. The method specifically comprises the following steps 221 to 222:
in step 221, a first list hash value corresponding to the backup file directory is obtained in the cloud server, where the first list hash value is a hash value mapped by the first file attribute information list.
In step 222, when the first list hash value is not consistent with a second list hash value in the backup service database, a file backup request is sent to the cloud server based on the backup file directory, where the second list hash value is a hash value mapped by the second file attribute information list.
Specifically, in this application, before sending a file backup request to the cloud server, a request may be made to the cloud server to obtain a first list hash value mapped by a first file attribute information list corresponding to the backup file directory, and then a hash value of a second file attribute information list in the backup service database is calculated by using a hash algorithm to obtain a second list hash value.
In this application, it should be noted that the manner in which the second list hash value is determined needs to be consistent with the manner in which the first list hash value is determined.
For example, if the first list hash value obtained from the cloud server is obtained based on the MD5 algorithm, the second list hash value is obtained by hashing the second file attribute information list in the backup service database through the MD5 algorithm.
For example, if the first list hash value obtained from the cloud server is obtained based on the SHA-1 algorithm, the second list hash value is obtained by hashing the second file attribute information list in the backup service database through the SHA-1 algorithm.
In view of the one-way unique characteristic of the hash algorithm, after the second list hash value is obtained, the second list hash value may be compared with the first list hash value, if the second list hash value is not consistent with the first list hash value, it is stated that the file stored in the cloud server has changed relative to the file which has been backed up historically, and needs to be backed up, so that a file backup request is sent to the cloud server, and if the second list hash value is consistent with the first list hash value, it is stated that the file stored in the cloud server has not changed relative to the file which has been backed up historically, and does not need to be backed up, so that the file backup request does not need to be sent to the cloud server.
By performing steps 221 to 222, it is beneficial to send a file backup request to the cloud server only when a file stored in the cloud server changes, so that network resources can be saved.
With continued reference to fig. 2, in step 250, a list of files to be backed up is determined by comparing the second list of file attribute information with the first list of file attribute information in the backup service database.
In this application, the first file attribute information list includes at least one piece of first file attribute information, and the second file attribute information list includes at least one piece of second file attribute information.
In an embodiment of the present application, determining a list of files to be backed up by comparing the second file attribute information list with the first file attribute information list in the backup service database may be performed according to the steps shown in fig. 5.
Referring to FIG. 5, a detailed flow diagram for determining a list of files to be backed up is shown, according to one embodiment of the present disclosure. The method specifically comprises steps 251 to 252:
in step 251, the at least one first file attribute information and the at least one second file attribute information are compared in the backup service database, where the first file attribute information is information representing the first file identity and the associated parameter, and the second file attribute information is information representing the second file identity and the associated parameter.
In step 252, when the first file attribute information is different from any one of the second file attribute information, the first file corresponding to the first file attribute information is determined as a file to be backed up, so as to obtain a list of files to be backed up.
In this application, since the combination of the file attribute information such as the file name, the file path, the file size, and the file modification time of each file has uniqueness, the first file attribute information may be used to identify the identity of the first file corresponding to the first file attribute information, and the second file attribute information may be used to identify the identity of the second file corresponding to the second file attribute information. Further, comparing the at least one first file attribute information with the at least one second file attribute information in the backup service database may be to compare whether the file name, the file path, the file size, the file modification time, and other attribute information in the first file attribute information and the at least one second file attribute information are the same. Specifically, for example, for "file 1" of the first file in the cloud server, the file attribute information is: "Picture 1 (file name), path 1 (file path), 3000KB (file size), 2020/6/5 (file modification time)", and for "file 1" of the second file, the file attribute information is: "Picture 1 (file name), path 1 (file path), 2000KB (file size), 2020/6/1 (file modification time)", it can be seen that the file sizes and the file modification times of the two are different, which means that "file 1" in the cloud server has changed relative to "file 1" that has been backed up historically, and therefore "file 1" in the cloud server needs to be determined as a file to be backed up to perform the backup again.
For example, for "file 2" of the first file in the cloud server, the file attribute information is: "Picture 2 (file name), path 2 (file path), 1000KB (file size), 2020/6/6 (file modification time)", and there is no file attribute information of "file 2" in the second file attribute information, which means that "file 2" in the cloud server is a newly added file, and therefore "file 2" in the cloud server needs to be determined as a file to be backed up in order to be backed up again.
In an embodiment of the present application, determining a list of files to be backed up by comparing the second file attribute information list with the first file attribute information list in the backup service database may also be performed according to the steps shown in fig. 6.
Referring to FIG. 6, a detailed flow diagram for determining a list of files to be backed up is shown, according to one embodiment of the present disclosure. Specifically, the method comprises steps 253 to 256:
in step 253, hash values mapped by the at least one first file attribute information are respectively determined, so as to obtain at least one first attribute hash value.
In step 254, hash values mapped by the at least one second file attribute information are respectively determined, so as to obtain at least one second attribute hash value.
In step 255, the at least one first attribute hash value and the at least one second attribute hash value are compared in the backup service database.
In step 256, when the first attribute hash value is different from any one of the second attribute hash values, determining the first file corresponding to the first attribute hash value as a file to be backed up, and obtaining a list of files to be backed up.
In this application, the backup service database may record a hash value mapped by the second file attribute information of each second file, that is, a second attribute hash value. Furthermore, the hash value mapped by the first file attribute information of each first file, that is, the first attribute hash value, may also be determined by a hash algorithm. Due to the one-way uniqueness of the hash values, the file list to be backed up may be determined only by comparing whether the at least one first attribute hash value and the at least one second attribute hash value are the same in the backup service database.
Specifically, for example, for "file 1" of a first file in the cloud server, the first attribute hash value is "hash value 1", and there is no "hash value 1" in the second attribute hash value, which indicates that "file 1" in the cloud server is a new file or a modified file, so that "file 1" in the cloud server needs to be determined as a file to be backed up, so as to perform backup again.
This has the advantage that, since only one hash value is compared without comparing all file attribute information, the computer resources required to be invoked during the comparison process can be reduced when comparing the second list of file attribute information with the first list of file attribute information.
In addition, in the two embodiments, the second file attribute information list and the first file attribute information list are compared in the backup service database, and the memory does not need to be occupied to compare the second file attribute information list and the first file attribute information list, so that the memory occupation of the file in the backup process can be reduced, the memory utilization rate is further saved, and the reliability and the safety of the file in the backup process are ensured.
With reference to fig. 2, in step 270, based on the to-be-backed-up file list, the to-be-backed-up file corresponding to the to-be-backed-up file list is read from the cloud server, so as to backup the to-be-backed-up file.
In an embodiment of the present application, based on the to-be-backed-up file list, reading the to-be-backed-up file corresponding to the to-be-backed-up file list from the cloud server, which may be performed according to the steps shown in fig. 7.
Referring to fig. 7, a detailed flowchart of reading a file to be backed up corresponding to the file list to be backed up from the cloud server according to an embodiment of the present disclosure is shown. Specifically, the method comprises steps 271 to 274:
in step 271, at least two sub-lists of files to be backed up are determined from the list of files to be backed up.
In a specific implementation of an embodiment, determining at least two to-be-backed file sub-lists from the to-be-backed file list may be performed according to the steps shown in fig. 8.
Referring to fig. 8, a detailed flowchart for determining at least two sub-lists of files to be backed up from the list of files to be backed up according to an embodiment of the present disclosure is shown. Specifically, the method comprises steps 2711 to 2712:
in step 2711, the number of files to be backed up in the list of files to be backed up and the file size of each file to be backed up are determined.
In step 2712, the files to be backed up are grouped according to the number of the files to be backed up and the file size of each file to be backed up, so as to obtain at least two sub-lists of the files to be backed up.
In the method, the files to be backed up can be grouped according to the number of the files to be backed up and the size of each file to be backed up and a balanced grouping principle.
Specifically, for example, the number of files to be backed up in the file list to be backed up is 5, and the file sizes thereof are "1: 1000KB of file to be backed up", "2: 1000KB of file to be backed up", "3: 1000KB of file to be backed up", "4: 2000KB of file to be backed up", and "5: 1000KB of file to be backed up", respectively. When the files to be backed up are grouped, if the files to be backed up are divided into two groups, the total file size of the file to be backed up 1, the file to be backed up 2 and the file to be backed up 3 is 3000KB, and the total file size of the file to be backed up 4 and the file to be backed up 5 is 3000KB, so that the file to be backed up 1, the file to be backed up 2 and the file to be backed up 3 can be divided into one group to obtain a file to be backed up sublist 1, and the file to be backed up 4 and the file to be backed up 5 are divided into one group to obtain a file to be backed up sublist.
In the present application, according to the number of the files to be backed up and the file size of each file to be backed up, grouping the files to be backed up according to the principle of balanced grouping has the advantage that it can be ensured that the file size difference between the files to be backed up corresponding to each sub-list of the files to be backed up is not large.
With continued reference to fig. 7, in step 272, file reading tasks to be backed up are respectively created in one-to-one correspondence with the respective sub-lists of files to be backed up.
Specifically, for example, a to-be-backed-up file reading task 1 may be created for the to-be-backed-up file sub-list 1, and a to-be-backed-up file reading task 2 may be created for the to-be-backed-up file sub-list 2.
In step 273, each of the file reading tasks to be backed up is distributed to at least two file reading threads.
In step 274, the at least two file reading threads are concurrently executed to read the file to be backed up corresponding to the file list to be backed up in the cloud server.
In the above step 274, the network traffic in the file backup process may be controlled through two dimensions, i.e. the number of the concurrently executed read threads is controlled according to the bandwidth that can be provided in the network, and if the total traffic required when all the file read threads are concurrently executed is greater than the bandwidth that can be provided in the network, then part of the concurrently executed file read threads are controlled, so as to ensure that the bandwidth that can be provided in the network is fully utilized, and the queuing process is performed for the remaining part of the file read threads.
In the above step 274, during the multi-threaded file reading process, the operation condition of each thread may also be detected, and when there is a thread error, the file backup task corresponding to the error thread may be distributed to other threads.
In this embodiment, the file to be backed up in the cloud server is backed up by executing at least two file reading threads concurrently, so that the file backup time can be shortened, and the file backup efficiency can be improved.
In the application, after all the target backup files are backed up, the backed-up files can be checked to determine whether the files which are not backed up exist in the cloud server.
In summary, in the technical solutions of some embodiments of the present disclosure, a first file attribute information list is obtained in a cloud server through a backup file directory obtained from a backup service database, and a second file attribute information list is compared with the first file attribute information list in the backup service database, so that a to-be-backed file list can be determined, and a corresponding to-be-backed file is read from the cloud server according to the to-be-backed file list. The file list to be backed up is obtained by comparing the second file attribute information list with the first file attribute information list in the backup service database, so that the memory comparison can be reduced, the occupation of the memory in the file backup process is reduced, the utilization rate of the memory is further saved, and the reliability and the safety of the file in the backup process are ensured.
The following describes an embodiment of an apparatus of the present disclosure, which may be used to execute a file backup method in a cloud scenario in the foregoing embodiment of the present disclosure. For details that are not disclosed in the embodiments of the apparatus of the present disclosure, refer to the embodiments of the file backup method in the cloud scenario described above in the present disclosure.
Fig. 9 illustrates a block diagram of a file backup apparatus in a cloud scenario according to an embodiment of the present disclosure.
Referring to fig. 9, a file backup apparatus 900 in a cloud scenario according to an embodiment of the present disclosure includes: a first acquisition unit 901, a second acquisition unit 902, a determination unit 903, and a reading unit 904.
A first obtaining unit 901, configured to obtain a backup file directory from a backup service database; a second obtaining unit 902, configured to obtain, in the cloud server, a first file attribute information list corresponding to the backup file directory; a determining unit 903, configured to determine a to-be-backed-up file list by comparing the second file attribute information list with the first file attribute information list in the backup service database; the reading unit 904 is configured to read a file to be backed up corresponding to the file list to be backed up from the cloud server based on the file list to be backed up, so as to backup the file to be backed up.
In some embodiments of the present disclosure, based on the foregoing scheme, the first obtaining unit 901 includes: a sending unit, configured to send a file backup request to the cloud server based on the backup file directory; the receiving unit is used for receiving the first file attribute information sent by the cloud server in batches to obtain the first file attribute information list.
In some embodiments of the present disclosure, based on the foregoing solution, the apparatus further includes a third obtaining unit, configured to obtain, in the cloud server, a first list hash value corresponding to the backup file directory before sending the file backup request to the cloud server based on the backup file directory, where the first list hash value is a hash value mapped by the first file attribute information list; the transmitting unit is configured to: and when the first list hash value is inconsistent with a second list hash value in the backup service database, sending a file backup request to the cloud server based on the backup file directory, wherein the second list hash value is a hash value mapped by the second file attribute information list.
In some embodiments of the present disclosure, based on the foregoing scheme, the first file attribute information list includes at least one first file attribute information, the second file attribute information list includes at least one second file attribute information, and the determining unit 903 is configured to: comparing the at least one first file attribute information with the at least one second file attribute information in the backup service database, wherein the first file attribute information is information representing the first file identity and the associated parameters, and the second file attribute information is information representing the second file identity and the associated parameters; and when the first file attribute information is different from any one of the second file attribute information, determining the first file corresponding to the first file attribute information as a file to be backed up, and obtaining a list of the file to be backed up.
In some embodiments of the present disclosure, based on the foregoing scheme, the first file attribute information list includes at least one first file attribute information, the second file attribute information list includes at least one second file attribute information, and the determining unit 903 is configured to: respectively determining hash values mapped by the at least one first file attribute information to obtain at least one first attribute hash value; respectively determining hash values mapped by the at least one second file attribute information to obtain at least one second attribute hash value; comparing, in the backup service database, the at least one first attribute hash value and the at least one second attribute hash value; and when the first attribute hash value is different from any one of the second attribute hash values, determining a first file corresponding to the first attribute hash value as a file to be backed up, and obtaining a file list to be backed up.
In some embodiments of the present disclosure, based on the foregoing scheme, the reading unit 904 is configured to: determining at least two sub-lists of files to be backed up from the list of files to be backed up; respectively creating a file reading task to be backed up which corresponds to each file sub-list to be backed up one by one; distributing each file reading task to be backed up to at least two file reading threads; and concurrently executing the at least two file reading threads to read the files to be backed up corresponding to the file list to be backed up in the cloud server.
In some embodiments of the present disclosure, based on the foregoing scheme, the reading unit 904 is configured to: determining the number of files to be backed up in the file list to be backed up and the size of each file to be backed up; grouping the files to be backed up according to the number of the files to be backed up and the file size of each file to be backed up to obtain at least two sub-lists of the files to be backed up.
It should be noted that although in the above detailed description, a file backup method in a cloud scenario and several units of a file backup apparatus in a cloud scenario are mentioned, such division is not mandatory. Indeed, two or more of the units and functions described above may be embodied in one unit, according to embodiments of the present disclosure. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units. The components displayed as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the elements can be selected according to actual needs to achieve the purpose of the disclosed solution. One of ordinary skill in the art can understand and implement it without inventive effort.
As another aspect, the present disclosure also provides a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the disclosure described in the "exemplary methods" section above of this specification, when the program product is run on the terminal device.
Referring to fig. 10, a program product 1000 for implementing the above method according to an embodiment of the present disclosure is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
As another aspect, the present disclosure also provides an electronic device capable of implementing the above method.
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 1100 according to this embodiment of the disclosure is described below with reference to fig. 11. The electronic device 1100 shown in fig. 11 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present disclosure.
As shown in fig. 11, electronic device 1100 is embodied in the form of a general purpose computing device. The components of the electronic device 1100 may include, but are not limited to: the at least one processing unit 1110, the at least one memory unit 1120, and a bus 1130 that couples various system components including the memory unit 1120 and the processing unit 1110.
Wherein the storage unit stores program code that can be executed by the processing unit 1110 to cause the processing unit 1110 to perform the steps according to various exemplary embodiments of the present disclosure described in the section "example methods" above in this specification.
The storage unit 1120 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM)1121 and/or a cache memory unit 1122, and may further include a read-only memory unit (ROM) 1123.
The storage unit 1120 may also include a program/utility 1124 having a set (at least one) of program modules 1125, such program modules 1125 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Bus 1130 may be representative of one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 1100 may also communicate with one or more external devices 1200 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 1100, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 1100 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 1150. Also, the electronic device 1100 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 1160. As shown, the network adapter 1160 communicates with the other modules of the electronic device 1100 over the bus 1130. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Furthermore, the above-described figures are merely schematic illustrations of processes included in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It is to be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (10)

1. A file backup method in a cloud scene is characterized by comprising the following steps:
acquiring a backup file directory from a backup service database;
acquiring a first file attribute information list corresponding to the backup file directory in a cloud server;
determining a list of files to be backed up by comparing a second file attribute information list with the first file attribute information list in the backup service database;
and reading the file to be backed up corresponding to the file list to be backed up from the cloud server based on the file list to be backed up so as to backup the file to be backed up.
2. The method according to claim 1, wherein the obtaining a first file attribute information list corresponding to the backup file directory in the cloud server includes:
sending a file backup request to the cloud server based on the backup file directory;
receiving first file attribute information sent by a cloud server in batches to obtain a first file attribute information list.
3. The method of claim 2, wherein prior to sending a file backup request to the cloud server based on the backup file directory, the method further comprises:
acquiring a first list hash value corresponding to the backup file directory in a cloud server, wherein the first list hash value is a hash value mapped by the first file attribute information list;
and when the first list hash value is inconsistent with a second list hash value in the backup service database, sending a file backup request to the cloud server based on the backup file directory, wherein the second list hash value is a hash value mapped by the second file attribute information list.
4. The method according to claim 1, wherein the first file attribute information list includes at least one first file attribute information, the second file attribute information list includes at least one second file attribute information, and the determining the list of files to be backed up by comparing the second file attribute information list with the first file attribute information list in the backup service database comprises:
comparing the at least one first file attribute information with the at least one second file attribute information in the backup service database, wherein the first file attribute information is information representing the first file identity and the associated parameters, and the second file attribute information is information representing the second file identity and the associated parameters;
and when the first file attribute information is different from any one of the second file attribute information, determining the first file corresponding to the first file attribute information as a file to be backed up, and obtaining a list of the file to be backed up.
5. The method according to claim 1, wherein the first file attribute information list includes at least one first file attribute information, the second file attribute information list includes at least one second file attribute information, and the determining the list of files to be backed up by comparing the second file attribute information list with the first file attribute information list in the backup service database comprises:
respectively determining hash values mapped by the at least one first file attribute information to obtain at least one first attribute hash value;
respectively determining hash values mapped by the at least one second file attribute information to obtain at least one second attribute hash value;
comparing, in the backup service database, the at least one first attribute hash value and the at least one second attribute hash value;
and when the first attribute hash value is different from any one of the second attribute hash values, determining a first file corresponding to the first attribute hash value as a file to be backed up, and obtaining a file list to be backed up.
6. The method according to claim 1, wherein reading the file to be backed up corresponding to the file list to be backed up from a cloud server based on the file list to be backed up comprises:
determining at least two sub-lists of files to be backed up from the list of files to be backed up;
respectively creating a file reading task to be backed up which corresponds to each file sub-list to be backed up one by one;
distributing each file reading task to be backed up to at least two file reading threads;
and concurrently executing the at least two file reading threads to read the files to be backed up corresponding to the file list to be backed up in the cloud server.
7. The method according to claim 6, wherein the determining at least two sub-lists of files to be backed up from the list of files to be backed up comprises:
determining the number of files to be backed up in the file list to be backed up and the size of each file to be backed up;
grouping the files to be backed up according to the number of the files to be backed up and the file size of each file to be backed up to obtain at least two sub-lists of the files to be backed up.
8. An apparatus for backing up a file in a cloud scenario, the apparatus comprising:
a first acquisition unit configured to acquire a backup file directory from a backup service database;
a second obtaining unit, configured to obtain, in the cloud server, a first file attribute information list corresponding to the backup file directory;
a determining unit, configured to determine a list of files to be backed up by comparing a second file attribute information list with the first file attribute information list in the backup service database;
and the reading unit is used for reading the file to be backed up corresponding to the file list to be backed up from the cloud server based on the file list to be backed up so as to backup the file to be backed up.
9. A computer-readable program medium, characterized in that it stores computer program instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 7.
10. A video-based account login electronic device, the electronic device comprising:
a processor;
a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1 to 7.
CN202010739622.1A 2020-07-28 2020-07-28 File backup method, device, medium and electronic equipment in cloud scene Pending CN111880967A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010739622.1A CN111880967A (en) 2020-07-28 2020-07-28 File backup method, device, medium and electronic equipment in cloud scene
PCT/CN2020/117777 WO2021139224A1 (en) 2020-07-28 2020-09-25 Method and apparatus for file backup in cloud scenario, and medium and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010739622.1A CN111880967A (en) 2020-07-28 2020-07-28 File backup method, device, medium and electronic equipment in cloud scene

Publications (1)

Publication Number Publication Date
CN111880967A true CN111880967A (en) 2020-11-03

Family

ID=73200370

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010739622.1A Pending CN111880967A (en) 2020-07-28 2020-07-28 File backup method, device, medium and electronic equipment in cloud scene

Country Status (2)

Country Link
CN (1) CN111880967A (en)
WO (1) WO2021139224A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112835747A (en) * 2021-02-24 2021-05-25 上海泰宇信息技术股份有限公司 Remote backup method without client
CN112882866A (en) * 2021-02-24 2021-06-01 上海泰宇信息技术股份有限公司 Backup method suitable for massive files
CN114422509A (en) * 2022-04-01 2022-04-29 天津联想协同科技有限公司 Automatic file backup method and device, network disk and storage medium
CN115269276A (en) * 2022-08-10 2022-11-01 广州鼎甲计算机科技有限公司 Incremental backup method and device, computer equipment and storage medium thereof
CN115794491A (en) * 2022-11-30 2023-03-14 广州市保伦电子有限公司 Database file backup method, device, equipment and storage medium
CN116319762A (en) * 2023-05-18 2023-06-23 华夏卓越(天津)科技有限公司 File batch copying method and device in local area network, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182294A (en) * 2013-05-23 2014-12-03 华为技术有限公司 Method and device for backing up and recovering file
CN104410683A (en) * 2014-11-21 2015-03-11 四川神琥科技有限公司 Data backup method
CN107145409A (en) * 2017-05-31 2017-09-08 广州鼎甲计算机科技有限公司 A kind of method of file multichannel backup
CN111104258A (en) * 2019-12-23 2020-05-05 北京金山云网络技术有限公司 MongoDB database backup method and device and electronic equipment

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104252397A (en) * 2013-06-27 2014-12-31 广州中国科学院软件应用技术研究所 Realizing method for supporting automatic backup of cloud platform user file
JP6376869B2 (en) * 2014-07-10 2018-08-22 キヤノン株式会社 Data synchronization system, control method thereof, authorization server, and program thereof
CN104503864A (en) * 2014-11-20 2015-04-08 北京世纪高蓝科技有限公司 Method and device for file backup based on local area network
CN106131123B (en) * 2016-06-21 2019-03-29 福建星网视易信息系统有限公司 The remote resource synchronous method and system of three-tier architecture

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104182294A (en) * 2013-05-23 2014-12-03 华为技术有限公司 Method and device for backing up and recovering file
CN104410683A (en) * 2014-11-21 2015-03-11 四川神琥科技有限公司 Data backup method
CN107145409A (en) * 2017-05-31 2017-09-08 广州鼎甲计算机科技有限公司 A kind of method of file multichannel backup
CN111104258A (en) * 2019-12-23 2020-05-05 北京金山云网络技术有限公司 MongoDB database backup method and device and electronic equipment

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112835747A (en) * 2021-02-24 2021-05-25 上海泰宇信息技术股份有限公司 Remote backup method without client
CN112882866A (en) * 2021-02-24 2021-06-01 上海泰宇信息技术股份有限公司 Backup method suitable for massive files
CN112882866B (en) * 2021-02-24 2023-12-15 上海泰宇信息技术股份有限公司 Backup method suitable for mass files
CN114422509A (en) * 2022-04-01 2022-04-29 天津联想协同科技有限公司 Automatic file backup method and device, network disk and storage medium
CN114422509B (en) * 2022-04-01 2022-07-26 天津联想协同科技有限公司 Automatic file backup method and device, network disk and storage medium
CN115269276A (en) * 2022-08-10 2022-11-01 广州鼎甲计算机科技有限公司 Incremental backup method and device, computer equipment and storage medium thereof
CN115794491A (en) * 2022-11-30 2023-03-14 广州市保伦电子有限公司 Database file backup method, device, equipment and storage medium
CN116319762A (en) * 2023-05-18 2023-06-23 华夏卓越(天津)科技有限公司 File batch copying method and device in local area network, electronic equipment and storage medium
CN116319762B (en) * 2023-05-18 2023-08-04 华夏卓越(天津)科技有限公司 File batch copying method and device in local area network, electronic equipment and storage medium

Also Published As

Publication number Publication date
WO2021139224A1 (en) 2021-07-15

Similar Documents

Publication Publication Date Title
CN111880967A (en) File backup method, device, medium and electronic equipment in cloud scene
US9495197B2 (en) Reliable and scalable image transfer for data centers with low connectivity using redundancy detection
JP6865219B2 (en) Event batch processing, output sequencing, and log-based state storage in continuous query processing
US9122642B2 (en) Hybrid data backup in a networked computing environment
US20170140315A1 (en) Managing incident tickets in a cloud managed service environment
US9760441B2 (en) Restoration of consistent regions within a streaming environment
US11372549B2 (en) Reclaiming free space in a storage system
CN111858146B (en) Method, apparatus and computer program product for recovering data
CN111966631A (en) Mirror image file generation method, system, equipment and medium capable of being rapidly distributed
CN111338834B (en) Data storage method and device
CN115220876A (en) Virtual resource creating method, device, program product, medium and electronic equipment
CN113051102A (en) File backup method, device, system, storage medium and computer equipment
US11243979B1 (en) Asynchronous propagation of database events
CN112738249A (en) File uploading method, device, equipment and storage medium based on quantitative transaction
CN112181662A (en) Task scheduling method and device, electronic equipment and storage medium
CN113448770A (en) Method, electronic device and computer program product for recovering data
CN113590703B (en) ES data importing method and device, electronic equipment and readable storage medium
CN113076175B (en) Memory sharing method and device for virtual machine
US20150160959A1 (en) Managing virtual machine images in a distributed computing environment
US10091298B2 (en) Enhancing performance of data storage in a dispersed storage network
US11263094B2 (en) Recovering dynamic system data
CN113760469A (en) Distributed computing method and device
US9489271B1 (en) User interface for restoring databases
US10528400B2 (en) Detecting deadlock in a cluster environment using big data analytics
US11275770B2 (en) Parallelization of node's fault tolerent record linkage using smart indexing and hierarchical clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination