CN105824723A - Method and system for backup of data of public cloud storage account - Google Patents
Method and system for backup of data of public cloud storage account Download PDFInfo
- Publication number
- CN105824723A CN105824723A CN201610158148.7A CN201610158148A CN105824723A CN 105824723 A CN105824723 A CN 105824723A CN 201610158148 A CN201610158148 A CN 201610158148A CN 105824723 A CN105824723 A CN 105824723A
- Authority
- CN
- China
- Prior art keywords
- cloud storage
- data
- publicly
- account
- storage system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Abstract
The invention discloses a method and system for backup of data of a public cloud storage account. The method comprises the following steps: (S1) binding an account of a personal private cloud storage system with the public cloud storage account and a storage device with a wifi function; (S2) carrying out first-time traversing on contents and filenames belonging to the public cloud storage account by the personal private cloud storage system, and exporting data on the contents and the filenames after the traversing is completed, and saving the data in a database; (S3) generating the data in the database into a plurality of tasks by the personal private cloud storage system, and allocating the tasks to the storage device with the wifi function; (4) after the storage device with the wifi function accesses to the Internet, automatically starting to download task data from the public cloud storage account according to the contents and the filenames in a current task. According to the method and the system, the data can be intelligently and automatically backed to the storage device with the wifi function of a user from the public cloud storage account, the safety of the data is guaranteed, manual operations of the user are not required in a whole process, and the efficiency of data backup is high.
Description
Technical field
The present invention relates to a kind of method and system that the data of publicly-owned cloud storage account are backed up, belong to data security arts.
Background technology
Data backup, refers to occur that operational error or the system failure cause loss of data for anti-locking system, and all or part of data acquisition system copies to the process of other storage medium from the hard disk of applied host machine or array.
Cloud storage is in cloud computing (cloudcomputing) conceptive extension and a new concept developed, refer to by functions such as cluster application, grid or distributed file systems, storage device various types of in network is gathered collaborative work by application software, the common system that data storage and Operational Visit function are externally provided.
In cloud storage system (the most publicly-owned cloud storage account, such as Baidu's Dropbox, 360 cloud disks, Kingsoft fast disk etc.) in, the data volume that can store is the biggest, thus much user shares file (such as photo, document etc.) to clear up mobile phone EMS memory with good friend simultaneously, often the documentum privatum of individual is sent and store to cloud storage system.But meanwhile, the most gradually expose new problem: such as cloud account is stolen, then the individual privacy of user is likely to will be compromised.Therefore, when using cloud storage, Information Security problem becomes the main misgivings of user.Additionally, after some vital document needed for user is uploaded to cloud storage system, may because of some former thus result in examination & verification not by so that be deleted, thus bring certain loss to user, cause bad Consumer's Experience simultaneously, thus the data in cloud storage system are backed up in time extremely important with regard to show.
The memory space of portable hard drive is the biggest, has for individual subscriber, for storing the file that backup is important.If user wants to be stored in its portable hard drive file useful in cloud storage system (include file that user oneself is uploaded in cloud storage system and other people be uploaded to cloud storage system and be shared with the file of user) by mobile phone, then need to first pass through computer to be downloaded from cloud storage system by file, portable hard drive of then restoring.If additionally, cloud storage system there is file update, needs also remain for user and go artificial cognition download to be stored in its people's portable hard drive.Above-mentioned mode of operation complexity is loaded down with trivial details, needs manual operation, it is impossible to meet the intelligentized requirement of people.
Summary of the invention
It is an object of the invention to, it is provided that a kind of method and system backing up the data of publicly-owned cloud storage account, it can effectively solve problems of the prior art, it is achieved the intelligent backup of data, it is ensured that the safety of data.
For solving above-mentioned technical problem, the present invention adopts the following technical scheme that: a kind of method backing up the data of publicly-owned cloud storage account, comprises the following steps:
S1, binds the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
S2, the catalogue under publicly-owned cloud storage account and filename are traveled through by individual private possession cloud storage system first, after having traveled through, catalogue and file name data are derived and are preserved to data base；
S3, individual private possession cloud storage system is by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function；
S4, after the storage device networking with wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account.
Preferably, step S1 specifically includes:
S11, accesses the web management interface of the storage device with wifi function, and utilizes the account of individual private possession cloud storage system and password to log in；After logging in successfully, it is somebody's turn to do the account i.e. with individual private possession cloud storage system of the storage device with wifi function and establishes binding relationship；
S12, initiates the bind request to publicly-owned cloud storage account on the web management interface of the storage device with wifi function；Obtain to access by the API of publicly-owned cloud storage system and authorize, obtain one and authorize key；
S13, by the title of publicly-owned cloud storage system, authorizes key and the device numbering with the storage device of wifi function to be uploaded to individual private possession cloud storage system；Individual private possession cloud storage system creates a binding relationship record, stores the relation between publicly-owned cloud storage account and the storage device with wifi function.
Realize the account of individual private possession cloud storage system being bound with publicly-owned cloud storage account and the storage device with wifi function, such that it is able to realize carrying out efficient, safe and stable communication between storage device and the publicly-owned cloud storage system between individual private possession cloud storage system and storage device and publicly-owned cloud storage system with wifi function and with wifi function by said method.
Preferably, step S2 specifically includes: the catalogue under the publicly-owned cloud storage account of appointment and filename are traveled through by individual private possession cloud storage system first, generate the structure of arrays (i.e. catalogue and the structure of file further comprises last modification time, file size, MD5 value, whether have been downloaded into the information such as storage device with wifi function) of METADATA；After having traveled through, by described filename according to N number of be one group, after being divided into multiple groups, export as the data (it is all supported by multiple computer language) of json form, and preserve to data base, such that it is able to optimize individual's cloud storage system management to backup tasks, it is simple to the storage device transmission tasks with wifi function.
It is furthermore preferred that during traversal, using catalogue or file name as index value, using file size, last modification time, MD5 value, whether be the information such as the catalogue data block as structure type；Wherein, comprising member a: fetched in described structure, for indicating whether to be downloaded to the storage device with wifi function of user, the default value of fetched is 0, such that it is able to be easy to generate METADATA, simultaneously facilitate the download performance of labelling task；So after the storage device with wifi function is networked next time again, being not fully complete of task can be directly downloaded, and need not start anew all to download, alleviate pressure and the data transmission efficiency of storage device with wifi function.
In the aforesaid method that the data of publicly-owned cloud storage account are backed up, step S3 specifically includes: the data in data base are generated multiple task groups according to packet by individual private possession cloud storage system, and every one task groups of sub-distribution gives the storage device with wifi function, storage device with wifi function downloads a part of mission bit stream every time, rather than whole mission bit streams, such that it is able to the performance of readily available new task, more new task and the pressure of reduction individual's cloud storage platform.
The aforesaid method that the data of publicly-owned cloud storage account are backed up, step S4 specifically includes: after the storage device networking with wifi function, logger task content and the numbering of packet, and from publicly-owned cloud storage account, automatically start downloading task data according to the catalogue in current task and filename；Often complete the download of all or part of file downloading or completing a task groups of a file, then to individual private possession cloud storage system more new task performance；Individual private possession cloud storage system is according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1 (wherein, each file corresponding a fetched value)；If not finding the file record of correspondence, then return false signal to the storage device with wifi function, after receiving false signal with the storage device of wifi function, abandon the follow-up work of current group that obtained and reacquire first task groups being not fully complete；Often complete the download of a task groups with the storage device of wifi function after, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.Such that it is able to update list according to data to back up desired data accurately from publicly-owned cloud storage account, improve the efficiency of data backup and the operational efficiency of whole system.
In said method, after the described storage device with wifi function is networked, it is firstly connected to individual private possession cloud storage system, downloads data backup task；Wherein, all of task all according to N number of be one group, all contain in every group task publicly-owned cloud storage system account information and need from this publicly-owned cloud storage system backup file name.
In said method, during performing task groups, often complete the download/backup of a file, can be to individual private possession cloud storage system more new task performance, such that it is able to be clearly visible that the percentage ratio of tasks carrying by individual private possession cloud storage system.Or selection is after all or part of file download/backup completing a task groups, whole task groups batch reports performance, such that it is able to download efficiency when improving particularly high-volume small documents download.In addition, the unmatched situation of task list if there is the file in the storage device task list with wifi function Yu individual private possession cloud storage system, then it is as the criterion with the task list of drop cloud, therefore when not finding respective file record, abandon the task groups being not fully complete, reacquire first task groups being not fully complete.
In the aforesaid method that the data of publicly-owned cloud storage account are backed up, step S3 also includes: individual private possession cloud storage system periodically carries out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename, and (interval time supplementing traversal can adjust, give tacit consent to 24 hours once), and compare with catalogue and the list of file names of local cache, find out new file, generate the list of download/synchronous task, such that it is able to only to publicly-owned cloud storage account update file carry out automatic, back up in time, further increase the efficiency of automatically updating data.
Preferably, catalogue under publicly-owned cloud storage account and filename are periodically carried out supplementing traversal by described individual private possession cloud storage system, and compare with catalogue and the list of file names of local cache, find out new file, generate the list of download/synchronous task and specifically include:
First, individual private possession cloud storage system loads the json data of bibliographic structure from data base according to the order being grouped, and preserves this structure of arrays in the internal memory of program, and by named for this array: old；(element comprised in described array can be retrieved by filename, each element contains a data block, saves the information relevant with METADATA such as the MD5 value corresponding to file name, file size, last modification time in data block)；
Secondly, carry out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename according to the mode traveled through first, it is thus achieved that the structure of arrays of up-to-date directory traversal result, and by named for this array: new；(its structure is identical with old, same, and the element comprised in this array can be retrieved by filename, but the fetched value that in this data structure, each element comprises is 0)；
Again, from old array, take out an element, new array is searched the element identical with this element files name, without finding, then abandons this element in this old array；If have found the element of correspondence, then the contrast MD5 value of two elements or file size and last modification time are (when publicly-owned cloud storage system supports MD5 value, then by contrasting MD5 value；When publicly-owned cloud storage system does not support MD5 value, then replaced the comparison of MD5 value by documents length and last modification time；After once downloading/backup to the storage device with wifi function due to the file of cloud disk, user can delete the file of correspondence from cloud disk, is then stored in more new file to cloud disk；Therefore will appearance under the circumstances when publicly-owned cloud storage account files is scanned by individual private possession cloud storage system)；If identical, then the fetched value of element in old array is copied in new array the element of correspondence；Repeat this operation, until all of element all takes out in old array；
Finally, after supplementary traversal completes, the data of new array are exported as from the internal memory of program the data of json form, and preserves to data base with packet mode.
In said method, after upper once scanning result is generated as multiple task groups, partial task has been had to have been completed (fetched is equal to 1) in task groups, if these tasks are already at the state completed, and file corresponding in cloud disk is not deleted, so it is necessary to arrange the state of task for complete (arranging fetched is 1) according to old task list in up-to-date cloud disc file scanning result.Specifically, by said method, such that it is able to catalogue and the file structure of up-to-date publicly-owned cloud storage are merged with the old data of storage in data base.If in old data there is not (such as being deleted by user) in the filename of record in current publicly-owned cloud storage, then will delete these records in merging process；If there being file of the same name in new and old record, also to judge the most substantial change of file according to the value of MD5 or file size and last modification time, if file creates change, just it is labeled as not downloading, the download state of task is the most just set according to the state of reporter's record.For the filename existed in only new record, then it is directly disposed as not downloading.
A kind of system that the data of publicly-owned cloud storage account are backed up, including:
Binding module, for binding the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
Spider module, travels through the catalogue under publicly-owned cloud storage account and filename first for individual private possession cloud storage system；
First data derive and memory module, after traversal completes, catalogue and file name data are derived and are preserved to data base；
Task generates and distribution module, by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function for individual private possession cloud storage system；
Automatically download module, after networking with the storage device of wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account.
Preferably, described binding module also includes:
Login module, for accessing the web management interface of the storage device with wifi function, and utilizes the account of individual private possession cloud storage system and password to log in；After logging in successfully, it is somebody's turn to do the account i.e. with individual private possession cloud storage system of the storage device with wifi function and establishes binding relationship；
Bind request sending module, for initiating the bind request to publicly-owned cloud storage account on the web management interface of the storage device with wifi function；
Access and authorize acquisition module, authorize for obtaining to access by the API of publicly-owned cloud storage system, obtain one and authorize key；
Transmission module in information, for being uploaded to individual private possession cloud storage system by the title of publicly-owned cloud storage system, mandate key and the device numbering with the storage device of wifi function；
Record creation module, creates a binding relationship record for individual private possession cloud storage system, stores the relation between publicly-owned cloud storage account and the storage device with wifi function.
Preferably, described spider module also includes:
Structure of arrays generation module, travels through first the catalogue under the publicly-owned cloud storage account of appointment and filename for individual private possession cloud storage system, generates the structure of arrays of METADATA；
First data derive and memory module also includes:
Packet module, after traversal completes, by described filename according to N number of be one group, be divided into multiple groups.
In the aforesaid system backing up the data of publicly-owned cloud storage account, described automatic download module also includes:
Logging modle, after networking with the storage device of wifi function, logger task content and the numbering of packet；
Task completes more new module, for often completing the download of all or part of file downloading or completing a task groups of a file with the storage device of wifi function, then to individual private possession cloud storage system more new task performance；
File record is searched and arranges module, for individual private possession cloud storage system according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function；
Task is abandoned and reacquires module, after receiving false signal with the storage device of wifi function, abandons the follow-up work of current group that obtained and reacquires first task groups being not fully complete；
The automatic request module of task, after the download often completing a task groups with the storage device of wifi function, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
In the aforesaid system backing up the data of publicly-owned cloud storage account, described spider module also includes:
Supplement traversal and contrast module, periodically carry out the catalogue under publicly-owned cloud storage account and filename for individual private possession cloud storage system supplementing traversal；And compare with catalogue and the list of file names of local cache, find out new file.
Preferably, described supplementary traversal and contrast module also include:
Data load and memory module: load the json data of bibliographic structure for individual private possession cloud storage system from data base according to the order being grouped, preserve this structure of arrays in the internal memory of program, and by named for this array: old；
Supplement spider module: for carrying out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename according to the mode traveled through first, it is thus achieved that the structure of arrays of up-to-date directory traversal result, and by named for this array: new；
Data Comparison module: for taking out an element from old array, search the element identical with this element files name in new array, without finding, then abandon this element in this old array；If have found the element of correspondence, then the contrast MD5 value of two elements or file size and last modification time are (when publicly-owned cloud storage system supports MD5 value, then by contrasting MD5 value；When publicly-owned cloud storage system does not support MD5 value, then replaced the comparison of MD5 value by documents length and last modification time)；If identical, then the fetched value of element in old array is copied in new array the element of correspondence；Repeat this operation, until all of element all takes out in old array；
Second data derive and memory module, are used for supplementing after traversal completes, the data of new array export as from the internal memory of program the data of json form, and preserves to data base with packet mode.
Compared with prior art, the present invention is by binding the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；Then utilize individual private possession cloud storage system that the catalogue under publicly-owned cloud storage account and filename are traveled through first, after having traveled through, data are derived and preserves to data base；Individual private possession cloud storage system is by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function；After storage device networking with wifi function, automatically begin to downloading task data, it is achieved thereby that data are from the intelligent automatic storage device with wifi function backing up to user of publicly-owned cloud storage account, ensure that the safety of data, whole process is all without user's manual operations, and data backup efficiency is higher.In addition, in the present invention, the catalogue under publicly-owned cloud storage account and filename are also periodically carried out supplementing traversal by individual private possession cloud storage system, find out new file, generate the list of download/synchronous task, such that it is able to only to publicly-owned cloud storage account update file carry out automatic, back up in time, further increase the efficiency of automatically updating data.
Accompanying drawing explanation
Fig. 1 is the workflow diagram of a kind of embodiment of the present invention；
Fig. 2 is account and the BaiduYun account of individual private possession cloud storage system and the storage device with wifi function carries out the method flow diagram bound；
Fig. 3 is that the storage device with wifi function performs backup tasks and individual private possession cloud storage system (drop cloud) finds there is new file in publicly-owned cloud storage and generates the flow chart of backup tasks.
The present invention is further illustrated with detailed description of the invention below in conjunction with the accompanying drawings.
Detailed description of the invention
Embodiments of the invention 1: a kind of method that the data of publicly-owned cloud storage account are backed up, as it is shown in figure 1, comprise the following steps:
S1, binds the account of the individual private possession cloud storage system storage device (as with the storage device of wifi function) with publicly-owned cloud storage account and with wifi function；Specifically include:
S11, accesses the web management interface of the storage device with wifi function, and utilizes the account of individual private possession cloud storage system and password to log in；After logging in successfully, it is somebody's turn to do the account i.e. with individual private possession cloud storage system of the storage device with wifi function and establishes binding relationship；
S12, initiates the bind request to publicly-owned cloud storage account on the web management interface of the storage device with wifi function；Obtain to access by the API of publicly-owned cloud storage system and authorize, obtain one and authorize key；
S13, by the title of publicly-owned cloud storage system, authorizes key and the device numbering with the storage device of wifi function to be uploaded to individual private possession cloud storage system；Individual private possession cloud storage system creates a binding relationship record, stores the relation between publicly-owned cloud storage account and the storage device with wifi function；
S2, the catalogue under publicly-owned cloud storage account and filename are traveled through by individual private possession cloud storage system first, after having traveled through, catalogue and file name data are derived and are preserved to data base；Specifically include: the catalogue under the publicly-owned cloud storage account of appointment and filename are traveled through by individual private possession cloud storage system first, generate the structure of arrays of METADATA；After having traveled through, by described filename according to N number of be one group, after being divided into multiple groups, export as the data of json form, and preserve to data base；Wherein, during traversal, using catalogue or file name as index value, using file size, last modification time, MD5 value, whether be the information such as the catalogue data block as structure type；Wherein, comprising member a: fetched in described structure, for indicating whether to be downloaded to the storage device with wifi function of user, the default value of fetched is 0；
S3, individual private possession cloud storage system is by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function；Specifically include: the data in data base are generated multiple task groups according to packet by individual private possession cloud storage system, and every one task groups of sub-distribution gives the storage device with wifi function；
Also include: individual private possession cloud storage system periodically carries out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename, and (interval time supplementing traversal can adjust, give tacit consent to 24 hours once), and compare with catalogue and the list of file names of local cache, find out new file, generate the list of download/synchronous task；Wherein, described individual private possession cloud storage system periodically carries out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename, and compares with catalogue and the list of file names of local cache, finds out new file, generates the list of download/synchronous task and specifically includes:
First, individual private possession cloud storage system loads the json data of bibliographic structure from data base according to the order being grouped, and preserves this structure of arrays in the internal memory of program, and by named for this array: old；(element comprised in described array can be retrieved by filename, each element contains a data block, saves the information relevant with METADATA such as the MD5 value corresponding to file name, file size, last modification time in data block)；
Secondly, carry out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename according to the mode traveled through first, it is thus achieved that the structure of arrays of up-to-date directory traversal result, and by named for this array: new；(its structure is identical with old, same, and the element comprised in this array can be retrieved by filename, but the fetched value that in this data structure, each element comprises is 0)；
Again, from old array, take out an element, new array is searched the element identical with this element files name, without finding, then abandons this element in this old array；If have found the element of correspondence, then the contrast MD5 value of two elements or file size and last modification time are (when publicly-owned cloud storage system supports MD5 value, then by contrasting MD5 value；When publicly-owned cloud storage system does not support MD5 value, then replaced the comparison of MD5 value by documents length and last modification time)；If identical, then the fetched value of element in old array is copied in new array the element of correspondence；Repeat this operation, until all of element all takes out in old array；
Finally, after supplementary traversal completes, the data of new array are exported as from the internal memory of program the data of json form, and preserves to data base with packet mode.
S4, after the storage device networking with wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account；Specifically include: after the storage device networking with wifi function, logger task content and the numbering of packet, and from publicly-owned cloud storage account, automatically start downloading task data according to the catalogue in current task and filename；Often complete the download of all or part of file downloading or completing a task groups of a file, then to individual private possession cloud storage system more new task performance；Individual private possession cloud storage system, according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function, after receiving false signal with the storage device of wifi function, abandon the follow-up work of current group that obtained and reacquire first task groups being not fully complete；Often complete the download of a task groups with the storage device of wifi function after, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
A kind of system that the data of publicly-owned cloud storage account are backed up, including:
Binding module, for binding the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
Spider module, travels through the catalogue under publicly-owned cloud storage account and filename first for individual private possession cloud storage system；
First data derive and memory module, after traversal completes, catalogue and file name data are derived and are preserved to data base；
Task generates and distribution module, by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function for individual private possession cloud storage system；
Automatically download module, after networking with the storage device of wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account.
Described binding module also includes:
Login module, for accessing the web management interface of the storage device with wifi function, and utilizes the account of individual private possession cloud storage system and password to log in；After logging in successfully, it is somebody's turn to do the account i.e. with individual private possession cloud storage system of the storage device with wifi function and establishes binding relationship；
Bind request sending module, for initiating the bind request to publicly-owned cloud storage account on the web management interface of the storage device with wifi function；
Access and authorize acquisition module, authorize for obtaining to access by the API of publicly-owned cloud storage system, obtain one and authorize key；
Transmission module in information, for being uploaded to individual private possession cloud storage system by the title of publicly-owned cloud storage system, mandate key and the device numbering with the storage device of wifi function；
Record creation module, creates a binding relationship record for individual private possession cloud storage system, stores the relation between publicly-owned cloud storage account and the storage device with wifi function.
Described spider module also includes:
Structure of arrays generation module, travels through first the catalogue under the publicly-owned cloud storage account of appointment and filename for individual private possession cloud storage system, generates the structure of arrays of METADATA；
First data derive and memory module also includes:
Packet module, after traversal completes, by described filename according to N number of be one group, be divided into multiple groups.
Described automatic download module also includes:
Logging modle, after networking with the storage device of wifi function, logger task content and the numbering of packet；
Task completes more new module, for often completing the download of all or part of file downloading or completing a task groups of a file with the storage device of wifi function, then to individual private possession cloud storage system more new task performance；
File record is searched and arranges module, for individual private possession cloud storage system according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function；
Task is abandoned and reacquires module, after receiving false signal with the storage device of wifi function, abandons the follow-up work of current group that obtained and reacquires first task groups being not fully complete；
The automatic request module of task, after the download often completing a task groups with the storage device of wifi function, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
Described spider module also includes:
Supplement traversal and contrast module, periodically carry out the catalogue under publicly-owned cloud storage account and filename for individual private possession cloud storage system supplementing traversal；And compare with catalogue and the list of file names of local cache, find out new file.
Described supplementary traversal and contrast module also include:
Data load and memory module: load the json data of bibliographic structure for individual private possession cloud storage system from data base according to the order being grouped, preserve this structure of arrays in the internal memory of program, and by named for this array: old；
Supplement spider module: for carrying out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename according to the mode traveled through first, it is thus achieved that the structure of arrays of up-to-date directory traversal result, and by named for this array: new；
Data Comparison module: for taking out an element from old array, search the element identical with this element files name in new array, without finding, then abandon this element in this old array；If have found the element of correspondence, then the contrast MD5 value of two elements or file size and last modification time are (when publicly-owned cloud storage system supports MD5 value, then by contrasting MD5 value；When publicly-owned cloud storage system does not support MD5 value, then replaced the comparison of MD5 value by documents length and last modification time)；If identical, then the fetched value of element in old array is copied in new array the element of correspondence；Repeat this operation, until all of element all takes out in old array；
Second data derive and memory module, are used for supplementing after traversal completes, the data of new array export as from the internal memory of program the data of json form, and preserves to data base with packet mode.
Embodiment 2: a kind of method that the data of publicly-owned cloud storage account are backed up, as it is shown in figure 1, comprise the following steps:
S1, binds the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
S2, the catalogue under publicly-owned cloud storage account and filename are traveled through by individual private possession cloud storage system first, after having traveled through, catalogue and file name data are derived and are preserved to data base；Specifically include: the catalogue under the publicly-owned cloud storage account of appointment and filename are traveled through by individual private possession cloud storage system first, generate the structure of arrays of METADATA；After having traveled through, by described filename according to N number of be one group, after being divided into multiple groups, export as the data of json form, and preserve to data base；Wherein, during traversal, using catalogue or file name as index value, using file size, last modification time, MD5 value, whether be the information such as the catalogue data block as structure type；Wherein, comprising member a: fetched in described structure, for indicating whether to be downloaded to the storage device with wifi function of user, the default value of fetched is 0；
S3, individual private possession cloud storage system is by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function；Specifically include: the data in data base are generated multiple task groups according to packet by individual private possession cloud storage system, and every one task groups of sub-distribution gives the storage device with wifi function；
S4, after the storage device networking with wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account；Specifically include: after the storage device networking with wifi function, logger task content and the numbering of packet, and from publicly-owned cloud storage account, automatically start downloading task data according to the catalogue in current task and filename；Often complete the download of all or part of file downloading or completing a task groups of a file, then to individual private possession cloud storage system more new task performance；Individual private possession cloud storage system, according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function, after receiving false signal with the storage device of wifi function, abandon the follow-up work of current group that obtained and reacquire first task groups being not fully complete；Often complete the download of a task groups with the storage device of wifi function after, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
A kind of system that the data of publicly-owned cloud storage account are backed up, including:
Binding module, for binding the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
Spider module, travels through the catalogue under publicly-owned cloud storage account and filename first for individual private possession cloud storage system；
First data derive and memory module, after traversal completes, catalogue and file name data are derived and are preserved to data base；
Task generates and distribution module, by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function for individual private possession cloud storage system；
Automatically download module, after networking with the storage device of wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account.
Described spider module also includes:
Structure of arrays generation module, travels through first the catalogue under the publicly-owned cloud storage account of appointment and filename for individual private possession cloud storage system, generates the structure of arrays of METADATA；
First data derive and memory module also includes:
Packet module, after traversal completes, by described filename according to N number of be one group, be divided into multiple groups.
Described automatic download module also includes:
Logging modle, after networking with the storage device of wifi function, logger task content and the numbering of packet；
Task completes more new module, for often completing the download of all or part of file downloading or completing a task groups of a file with the storage device of wifi function, then to individual private possession cloud storage system more new task performance；
File record is searched and arranges module, for individual private possession cloud storage system according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function；
Task is abandoned and reacquires module, after receiving false signal with the storage device of wifi function, abandons the follow-up work of current group that obtained and reacquires first task groups being not fully complete；
The automatic request module of task, after the download often completing a task groups with the storage device of wifi function, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
Fig. 2 is to use the method for the present invention to realize the method flow diagram that the account of individual private possession cloud storage system and BaiduYun account and the storage device with wifi function carry out binding.Wherein, described " management platform " i.e. individual private possession cloud storage system；It is verified establishment synchronization container to carry out in individual private possession cloud storage system.Fig. 3 is that in the present invention, storage device with wifi function performs backup tasks and individual private possession cloud storage system (drop cloud) and finds there is new file in publicly-owned cloud storage and generates the flow chart of backup tasks.
Claims (14)
1. the method that the data of publicly-owned cloud storage account are backed up, it is characterised in that comprise the following steps:
S1, binds the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
S2, the catalogue under publicly-owned cloud storage account and filename are traveled through by individual private possession cloud storage system first, after having traveled through, catalogue and file name data are derived and are preserved to data base；
S3, individual private possession cloud storage system is by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function；
S4, after the storage device networking with wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account.
The method that the data of publicly-owned cloud storage account are backed up the most according to claim 1, it is characterised in that step S1 specifically includes:
S11, accesses the web management interface of the storage device with wifi function, and utilizes the account of individual private possession cloud storage system and password to log in；After logging in successfully, it is somebody's turn to do the account i.e. with individual private possession cloud storage system of the storage device with wifi function and establishes binding relationship；
S12, initiates the bind request to publicly-owned cloud storage account on the web management interface of the storage device with wifi function；Obtain to access by the API of publicly-owned cloud storage system and authorize, obtain one and authorize key；
S13, by the title of publicly-owned cloud storage system, authorizes key and the device numbering with the storage device of wifi function to be uploaded to individual private possession cloud storage system；Individual private possession cloud storage system creates a binding relationship record, stores the relation between publicly-owned cloud storage account and the storage device with wifi function.
The method that the data of publicly-owned cloud storage account are backed up the most according to claim 1, it is characterized in that, step S2 specifically includes: the catalogue under the publicly-owned cloud storage account of appointment and filename are traveled through by individual private possession cloud storage system first, generate the structure of arrays of METADATA；After having traveled through, by described filename according to N number of be one group, after being divided into multiple groups, export as the data of json form, and preserve to data base.
The method backing up the data of publicly-owned cloud storage account the most according to claim 3, it is characterised in that during traversal, using catalogue or file name as index value；Wherein, comprising member a: fetched in described structure, for indicating whether to be downloaded to the storage device with wifi function of user, the default value of fetched is 0.
The method that the data of publicly-owned cloud storage account are backed up the most according to claim 4, it is characterized in that, step S3 specifically includes: the data in data base are generated multiple task groups according to packet by individual private possession cloud storage system, and every one task groups of sub-distribution gives the storage device with wifi function.
The method that the data of publicly-owned cloud storage account are backed up the most according to claim 5, it is characterized in that, step S4 specifically includes: after the storage device networking with wifi function, logger task content and the numbering of packet, and from publicly-owned cloud storage account, automatically start downloading task data according to the catalogue in current task and filename；Often complete the download of all or part of file downloading or completing a task groups of a file, then to individual private possession cloud storage system more new task performance；Individual private possession cloud storage system, according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function, after receiving false signal with the storage device of wifi function, abandon the follow-up work of current group that obtained and reacquire first task groups being not fully complete；Often complete the download of a task groups with the storage device of wifi function after, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
7. the method backed up according to the data to publicly-owned cloud storage account described in any one of claim 1～6, it is characterized in that, step S3 also includes: the catalogue under publicly-owned cloud storage account and filename are periodically carried out supplementing traversal by individual private possession cloud storage system, and compare with catalogue and the list of file names of local cache, find out new file, generate the list of download/synchronous task.
The method that the data of publicly-owned cloud storage account are backed up the most according to claim 7, it is characterized in that, catalogue under publicly-owned cloud storage account and filename are periodically carried out supplementing traversal by described individual private possession cloud storage system, and compare with catalogue and the list of file names of local cache, find out new file, generate the list of download/synchronous task and specifically include:
First, individual private possession cloud storage system loads the json data of bibliographic structure from data base according to the order being grouped, and preserves this structure of arrays in the internal memory of program, and by named for this array: old；
Secondly, carry out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename according to the mode traveled through first, it is thus achieved that the structure of arrays of up-to-date directory traversal result, and by named for this array: new；
Again, from old array, take out an element, new array is searched the element identical with this element files name, without finding, then abandons this element in this old array；If have found the element of correspondence, then the contrast MD5 value of two elements or file size and last modification time；If identical, then the fetched value of element in old array is copied in new array the element of correspondence；Repeat this operation, until all of element all takes out in old array；
Finally, after supplementary traversal completes, the data of new array are exported as from the internal memory of program the data of json form, and preserves to data base with packet mode.
9. the system that the data of publicly-owned cloud storage account are backed up, it is characterised in that including:
Binding module, for binding the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi function；
Spider module, travels through the catalogue under publicly-owned cloud storage account and filename first for individual private possession cloud storage system；
First data derive and memory module, after traversal completes, catalogue and file name data are derived and are preserved to data base；
Task generates and distribution module, by multiple for the data genaration in data base tasks and distributes to the storage device with wifi function for individual private possession cloud storage system；
Automatically download module, after networking with the storage device of wifi function, starts downloading task data automatically according to the catalogue in current task and filename from publicly-owned cloud storage account.
The system that the data of publicly-owned cloud storage account are backed up the most according to claim 9, it is characterised in that described binding module also includes:
Login module, for accessing the web management interface of the storage device with wifi function, and utilizes the account of individual private possession cloud storage system and password to log in；After logging in successfully, it is somebody's turn to do the account i.e. with individual private possession cloud storage system of the storage device with wifi function and establishes binding relationship；
Bind request sending module, for initiating the bind request to publicly-owned cloud storage account on the web management interface of the storage device with wifi function；
Access and authorize acquisition module, authorize for obtaining to access by the API of publicly-owned cloud storage system, obtain one and authorize key；
Transmission module in information, for being uploaded to individual private possession cloud storage system by the title of publicly-owned cloud storage system, mandate key and the device numbering with the storage device of wifi function；
Record creation module, creates a binding relationship record for individual private possession cloud storage system, stores the relation between publicly-owned cloud storage account and the storage device with wifi function.
11. systems that the data of publicly-owned cloud storage account are backed up according to claim 9, it is characterised in that described spider module also includes:
Structure of arrays generation module, travels through first the catalogue under the publicly-owned cloud storage account of appointment and filename for individual private possession cloud storage system, generates the structure of arrays of METADATA；
First data derive and memory module also includes:
Packet module, after traversal completes, by described filename according to N number of be one group, be divided into multiple groups.
12. systems that the data of publicly-owned cloud storage account are backed up according to claim 11, it is characterised in that described automatic download module also includes:
Logging modle, after networking with the storage device of wifi function, logger task content and the numbering of packet；
Task completes more new module, for often completing the download of all or part of file downloading or completing a task groups of a file with the storage device of wifi function, then to individual private possession cloud storage system more new task performance；
File record is searched and arranges module, for individual private possession cloud storage system according to packet numbering and file name locating file record in a packet, if finding the file record of correspondence, then arranges fetched=1；If not finding the file record of correspondence, then return false signal to the storage device with wifi function；
Task is abandoned and reacquires module, after receiving false signal with the storage device of wifi function, abandons the follow-up work of current group that obtained and reacquires first task groups being not fully complete；
The automatic request module of task, after the download often completing a task groups with the storage device of wifi function, the request next one is not fully complete task groups, until completing last to be not fully complete task groups automatically.
13. systems backed up according to the data to publicly-owned cloud storage account described in any one of claim 9～12, it is characterised in that described spider module also includes:
Supplement traversal and contrast module, periodically carry out the catalogue under publicly-owned cloud storage account and filename for individual private possession cloud storage system supplementing traversal；And compare with catalogue and the list of file names of local cache, find out new file.
14. systems that the data of publicly-owned cloud storage account are backed up according to claim 13, it is characterised in that described supplementary traversal and contrast module also include:
Data load and memory module: load the json data of bibliographic structure for individual private possession cloud storage system from data base according to the order being grouped, preserve this structure of arrays in the internal memory of program, and by named for this array: old；
Supplement spider module: for carrying out supplementing traversal to the catalogue under publicly-owned cloud storage account and filename according to the mode traveled through first, it is thus achieved that the structure of arrays of up-to-date directory traversal result, and by named for this array: new；
Data Comparison module: for taking out an element from old array, search the element identical with this element files name in new array, without finding, then abandon this element in this old array；If have found the element of correspondence, then the contrast MD5 value of two elements or file size and last modification time；If identical, then the fetched value of element in old array is copied in new array the element of correspondence；Repeat this operation, until all of element all takes out in old array；
Second data derive and memory module, are used for supplementing after traversal completes, the data of new array export as from the internal memory of program the data of json form, and preserves to data base with packet mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610158148.7A CN105824723B (en) | 2016-03-18 | 2016-03-18 | The method and system that a kind of data to publicly-owned cloud storage account are backed up |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610158148.7A CN105824723B (en) | 2016-03-18 | 2016-03-18 | The method and system that a kind of data to publicly-owned cloud storage account are backed up |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105824723A true CN105824723A (en) | 2016-08-03 |
| CN105824723B CN105824723B (en) | 2018-09-18 |
Family
ID=56524023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610158148.7A Active CN105824723B (en) | 2016-03-18 | 2016-03-18 | The method and system that a kind of data to publicly-owned cloud storage account are backed up |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105824723B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331066A (en) * | 2016-08-15 | 2017-01-11 | 努比亚技术有限公司 | Electronic device and information processing method |
| CN106941415A (en) * | 2017-01-20 | 2017-07-11 | 深圳市先河系统技术有限公司 | Use the method and private clound equipment, terminal device of private clound equipment |
| CN107277152A (en) * | 2017-06-29 | 2017-10-20 | 郑州云海信息技术有限公司 | A kind of public cloud data safety backup system |
| CN107678882A (en) * | 2017-09-14 | 2018-02-09 | 北京春鸿科技有限公司 | A kind of method and system of data backup |
| CN108038028A (en) * | 2017-12-13 | 2018-05-15 | 江苏神州信源系统工程有限公司 | File backup method, device and file recovery method, device |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN112286928A (en) * | 2019-09-16 | 2021-01-29 | 重庆傲雄在线信息技术有限公司 | Chain type storage system |
| CN113450063A (en) * | 2020-09-07 | 2021-09-28 | 北京新氧科技有限公司 | Task information processing method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102377827A (en) * | 2011-12-13 | 2012-03-14 | 方正国际软件有限公司 | Multilevel cloud storage system and storage method thereof |
| US20120203742A1 (en) * | 2011-02-08 | 2012-08-09 | International Business Machines Corporation | Remote data protection in a networked storage computing environment |
| CN103067406A (en) * | 2013-01-14 | 2013-04-24 | 暨南大学 | Access control system and access control method between public cloud and private cloud |
| CN105025084A (en) * | 2015-06-10 | 2015-11-04 | 国网智能电网研究院 | A cloud storage system based on synchronization agents and mixed storage |
-
2016
- 2016-03-18 CN CN201610158148.7A patent/CN105824723B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120203742A1 (en) * | 2011-02-08 | 2012-08-09 | International Business Machines Corporation | Remote data protection in a networked storage computing environment |
| CN102377827A (en) * | 2011-12-13 | 2012-03-14 | 方正国际软件有限公司 | Multilevel cloud storage system and storage method thereof |
| CN103067406A (en) * | 2013-01-14 | 2013-04-24 | 暨南大学 | Access control system and access control method between public cloud and private cloud |
| CN105025084A (en) * | 2015-06-10 | 2015-11-04 | 国网智能电网研究院 | A cloud storage system based on synchronization agents and mixed storage |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331066A (en) * | 2016-08-15 | 2017-01-11 | 努比亚技术有限公司 | Electronic device and information processing method |
| CN106331066B (en) * | 2016-08-15 | 2019-06-21 | 努比亚技术有限公司 | Electronic equipment and information processing method |
| CN106941415A (en) * | 2017-01-20 | 2017-07-11 | 深圳市先河系统技术有限公司 | Use the method and private clound equipment, terminal device of private clound equipment |
| CN107277152A (en) * | 2017-06-29 | 2017-10-20 | 郑州云海信息技术有限公司 | A kind of public cloud data safety backup system |
| CN107678882A (en) * | 2017-09-14 | 2018-02-09 | 北京春鸿科技有限公司 | A kind of method and system of data backup |
| CN108038028A (en) * | 2017-12-13 | 2018-05-15 | 江苏神州信源系统工程有限公司 | File backup method, device and file recovery method, device |
| CN108038028B (en) * | 2017-12-13 | 2021-03-23 | 北信源系统集成有限公司 | File backup method and device and file restoration method and device |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN112286928A (en) * | 2019-09-16 | 2021-01-29 | 重庆傲雄在线信息技术有限公司 | Chain type storage system |
| CN112286928B (en) * | 2019-09-16 | 2023-11-28 | 重庆傲雄在线信息技术有限公司 | Chain type storage system |
| CN113450063A (en) * | 2020-09-07 | 2021-09-28 | 北京新氧科技有限公司 | Task information processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105824723B (en) | 2018-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105824723A (en) | Method and system for backup of data of public cloud storage account | |
| CN111324360B (en) | Container mirror image construction method and system for edge calculation | |
| CN103116618B (en) | Based on Telefile mirror method and the system of the lasting buffer memory of client | |
| CN106156359B (en) | A kind of data synchronization updating method under cloud computing platform | |
| US10001913B2 (en) | Shared workspaces with selective content item synchronization | |
| US8417746B1 (en) | File system management with enhanced searchability | |
| US11914585B2 (en) | Servicing queries of a hybrid event index | |
| US9576071B2 (en) | Graph-based data models for partitioned data | |
| CN103002027B (en) | Data-storage system and the method for tree directory structure is realized based on key-value pair system | |
| US9697258B2 (en) | Supporting enhanced content searches in an online content-management system | |
| US9565232B2 (en) | Importing content items | |
| CN105930233B (en) | To specifying the data of public cloud account to carry out the method and system of automated back-up respectively | |
| CN103282899B (en) | The storage method of data, access method and device in file system | |
| WO2018201583A1 (en) | File management method and system, electronic device, and medium | |
| CN110399425B (en) | Intelligent network disk micro-service system | |
| CN106874383A (en) | A kind of decoupling location mode of metadata of distributed type file system | |
| US7991767B2 (en) | Method for providing a shared search index in a peer to peer network | |
| US20080208870A1 (en) | Managing files on multiple computing devices | |
| US20170193039A1 (en) | Servicing queries of an event log | |
| CN103595730A (en) | Ciphertext cloud storage method and system | |
| CN105978981B (en) | The linear expansion method and system of data backup based on publicly-owned cloud storage account | |
| CN105956491B (en) | The data of publicly-owned cloud storage system are carried out with the method and system of more account backups | |
| JP2012098934A (en) | Document management system, method for controlling document management system and program | |
| US8977662B1 (en) | Storing data objects from a flat namespace in a hierarchical directory structured file system | |
| WO2018118287A1 (en) | Method and system for maintaining and searching index records |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100190 room 10-08, building Haidian District, Haidian street, Beijing, 38 Applicant after: Beijing Spring Technology Co., Ltd. Address before: 100190 room 10-08, building Haidian District, Haidian street, Beijing, 38 Applicant before: Beijing Chunhong Enterprise Management Consulting Co., Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20220314 Granted publication date: 20180918 |
|
| PP01 | Preservation of patent right |