CN105824723B - The method and system that a kind of data to publicly-owned cloud storage account are backed up - Google Patents
The method and system that a kind of data to publicly-owned cloud storage account are backed up Download PDFInfo
- Publication number
- CN105824723B CN105824723B CN201610158148.7A CN201610158148A CN105824723B CN 105824723 B CN105824723 B CN 105824723B CN 201610158148 A CN201610158148 A CN 201610158148A CN 105824723 B CN105824723 B CN 105824723B
- Authority
- CN
- China
- Prior art keywords
- cloud storage
- publicly
- data
- account
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of method and system that the data to publicly-owned cloud storage account are backed up, and the described method comprises the following steps：S1, by the account of individual private possession cloud storage system with publicly-owned cloud storage account and the storage device with wifi functions is bound；S2, individual private possession cloud storage system under publicly-owned cloud storage account catalogue and filename traversed for the first time, after the completion of traversals, catalogue and file name data are exported and preserved into database；Data in database are generated multiple tasks and distribute to the storage device with wifi functions by S3, individual private possession cloud storage system；S4, with wifi functions storage device networking after, according in current task catalogue and filename automatically since publicly-owned cloud storage account downloading task data.The present invention realizes data from the intelligent automatic storage device with wifi functions for backing up to user of publicly-owned cloud storage account, ensure that the safety of data, and for overall process without user's manual operations, data backup is efficient.
Description
Technical field
The present invention relates to a kind of method and system that the data to publicly-owned cloud storage account are backed up, and belong to data safety
Field.
Background technology
Data backup, refer to for anti-locking system there is operation error or the system failure leads to loss of data, and will all or
Partial data set copies to the process of other storage mediums from the hard disk or array of applied host machine.
Cloud storage be in the conceptive extension of cloud computing (cloud computing) and a new concept developed,
Refer to being set various types of storage in network by functions such as cluster application, grid or distributed file systems
It is standby that collaborative work is gathered by application software, the common system that data storage and Operational Visit function is externally provided.
In cloud storage system (i.e. publicly-owned cloud storage account, such as Baidu's Dropbox, 360 cloud disks, Kingsoft fast disk), Ke Yicun
The data volume of storage is very big, thus many users share file (such as photo, document with good friend simultaneously in order to clear up mobile phone EMS memory
Deng), often personal documentum privatum is sent in cloud storage system and is stored.But it exposes at the same time, and gradually
New problem：For example cloud account is stolen, then the individual privacy of user is likely to be leaked.When therefore, using cloud storage,
Data safety sex chromosome mosaicism becomes the main misgivings of user.In addition, certain vital documents needed for user are uploaded to cloud storage system
Afterwards, it may cause the audit fails to be deleted in turn because of certain reasons, to bring certain loss to user, simultaneously
Bad user experience is caused, thus backup in time is carried out to the data in cloud storage system and just shows extremely important.
The memory space of mobile hard disk is generally also bigger, possesses for individual subscriber, and important file is backed up for storing.
If user wants that (including user oneself is uploaded to by mobile phone the file in cloud storage system by file useful in cloud storage system
And other people are uploaded to cloud storage system and are shared with the file of user) it is stored in its mobile hard disk, then it needs to first pass through computer
File is downloaded from cloud storage system, is then restored again into mobile hard disk.If in addition, there is file update in cloud storage system,
User is needed also remain for go manual identified and download to be stored in its people's mobile hard disk.Above-mentioned mode of operation complexity is cumbersome, needs
Manual operation, cannot be satisfied the intelligentized requirement of people.
Invention content
The object of the present invention is to provide a kind of method and system that the data to publicly-owned cloud storage account are backed up,
It can effectively solve problems of the prior art, realize the intelligent backup of data, ensure that the safety of data.
In order to solve the above technical problems, the present invention adopts the following technical scheme that：A kind of number to publicly-owned cloud storage account
According to the method backed up, include the following steps：
S1, by the account of individual private possession cloud storage system and publicly-owned cloud storage account and storage device with wifi functions
It is bound；
S2, individual private possession cloud storage system under publicly-owned cloud storage account catalogue and filename traversed for the first time, time
After the completion of going through, catalogue and file name data are exported and preserved into database；
Data in database are generated multiple tasks and distributed to wifi functions by S3, individual private possession cloud storage system
Storage device；
S4, with wifi functions storage device networking after, according in current task catalogue and filename automatically from public affairs
There are beginning downloading task data in cloud storage account.
Preferably, step S1 is specifically included：
S11 accesses the web management interface of the storage device with wifi functions, and utilizes individual private possession cloud storage system
Account and password logged in；After logging in successfully, should with wifi functions storage device i.e. and individual private possession cloud storage system
The account of system establishes binding relationship；
S12 initiates the binding to publicly-owned cloud storage account on the web management interface of the storage device with wifi functions
Request；Access mandate is obtained by the API of publicly-owned cloud storage system, obtains a mandate key；
S13 by the title of publicly-owned cloud storage system, authorizes key and the device numbering of storage device with wifi functions
It is uploaded to individual private possession cloud storage system；Individual private possession cloud storage system creates a binding relationship record, and storage public cloud is deposited
Store up the relationship between account and storage device with wifi functions.
It is realized the account of individual private possession cloud storage system and publicly-owned cloud storage account by the above method and carries wifi
The storage device of function is bound, so as to the storage device realized individual private possession cloud storage system with carry wifi functions
Efficient, peace is carried out between publicly-owned cloud storage system and between the storage device with wifi functions and publicly-owned cloud storage system
Entirely, the communication stablized.
Preferably, step S2 is specifically included：Individual private possession cloud storage system is to specifying the catalogue under publicly-owned cloud storage account
And filename is traversed for the first time, (i.e. the structure of catalogue and file further comprises last the structure of arrays of generation META DATA
Whether modification time file size, MD5 values, has been downloaded into the information such as storage device with wifi functions)；It has traversed
Cheng Hou, by the filename according to it is N number of be one group, after being divided into multiple groups, export as data (a variety of calculating of json formats
Machine language all supports it), and preserve into database, so as to optimize personal cloud storage system to backup tasks
Management, convenient for the storage device transmission tasks with wifi functions.
It is furthermore preferred that when traversal, using catalogue or file name as index value, with file size, last modification time,
Whether MD5 values are data block of the information such as catalogue as structure type；Wherein, include a member in the structure：
Fetched, the storage device with wifi functions for indicating whether to be downloaded to user, the default value of fetched
It is 0, so as to be convenient for generating META DATA, while convenient for the download performance of label task；In this way when with wifi work(
After the storage device of energy is networked next time again, unfinished task can be directly downloaded, and without all downloading from the beginning, mitigate
The pressure and data transmission efficiency of storage device with wifi functions.
In the method that data above-mentioned to publicly-owned cloud storage account are backed up, step S3 is specifically included：Individual private possession
Data in database are generated multiple tasks group by cloud storage system according to grouping, and are carried per one task groups of sub-distribution
The storage device of wifi functions, the storage device with wifi functions download a part of mission bit stream every time, rather than whole
Mission bit stream, so as to readily available new task, the performance of more new task and the pressure for reducing personal cloud storage platform
Power.
The method that data above-mentioned to publicly-owned cloud storage account are backed up, step S4 are specifically included：With wifi work(
Can storage device networking after, the number of logger task content and grouping, and according in current task catalogue and filename from
Downloading task data moving since publicly-owned cloud storage account；Often complete one task groups of download or completion of a file
The download of all or part of file, then to individual private possession cloud storage system more new task performance；Individual private possession cloud storage system
According to packet numbering and file name, locating file records system in a packet, if finding corresponding file record, is arranged
Fetched=1 (wherein, each file has corresponded to a fetched value)；If not finding corresponding file record, return
False signals to the storage device with wifi functions is returned to put after the storage device with wifi functions receives false signals
It abandons the follow-up work of the current group obtained and reacquires first unfinished task groups；With depositing for wifi functions
After storage equipment often completes the download of a task groups, next unfinished task groups are asked automatically, until completing the last one not
Complete task groups.So as to accurately back up required data from publicly-owned cloud storage account according to data update list, improve
The efficiency of data backup and the operational efficiency of whole system.
In the above method, after the storage device networking with wifi functions, it is firstly connected to individual private possession cloud and deposits
Storage system, downloading data backup tasks；Wherein, all tasks all in accordance with it is N number of be one group, it is publicly-owned per containing in group task
The file name that the account information and needs of cloud storage system are backed up from this publicly-owned cloud storage system.
In the above method, during executing task groups, download/backup of a file is often completed, it can be to a
People's private cloud storage system update task performance, so as to clearly see task by individual private possession cloud storage system
The percentage of execution.Or it selects after all or part of file download/backup for completing a task groups, entire task batching
Amount reports performance, so as to improve download efficiency when especially high-volume small documents download.In addition, if there is band
There are the unmatched feelings of task list of the file and individual private possession cloud storage system in the storage device task list of wifi functions
Condition, then be subject to the task list of drop cloud, therefore when not finding respective file record, abandons unfinished task groups, weight
Newly obtain first unfinished task groups.
In the method that data above-mentioned to publicly-owned cloud storage account are backed up, step S3 further includes：Individual private possession cloud
Storage system periodically under publicly-owned cloud storage account catalogue and filename carry out supplement traversal (supplement traversal interval time can
With adjustment, 24 hours are given tacit consent to once), and be compared with the catalogue of local cache and list of file names, new file is found out, is generated
Download/synchronous task list, so as to only carry out automatic, backup in time to the newer file of publicly-owned cloud storage account, into one
Step improves the efficiency of automatically updating data.
Preferably, the individual private possession cloud storage system periodically under publicly-owned cloud storage account catalogue and filename into
Row supplement traversal, and be compared with the catalogue of local cache and list of file names, new file is found out, download/synchronization is generated and appoints
Business list specifically includes：
First, individual private possession cloud storage system loads the json numbers of bibliographic structure from database according to the sequence of grouping
According to preserving the structure of arrays in the memory of program, and the array is named as：old；(the element for including in the array
It can be retrieved by filename, each element contains a data block, is saved in data block and file name
Corresponding MD5 values, file size, last modification time etc. and the relevant information of METADATA)；
Secondly, in the way of traversing for the first time under publicly-owned cloud storage account catalogue and filename carry out supplement traversal,
The structure of arrays of newest directory traversal result is obtained, and the array is named as：new；(its structure is identical as old, equally
, the element for including in the array can be retrieved by filename, but each element includes in the data structure
0) fetched values are all；
Again, an element is taken out from old arrays, and element identical with the element files name is searched in new arrays,
It is if it is not found, then abandon the element in this old array；If having found corresponding element, two elements of comparison
MD5 values or file size and last modification time are (when publicly-owned cloud storage system supports MD5 values, then by comparing MD5 values；When
When publicly-owned cloud storage system does not support MD5 values, then the ratio of MD5 values is replaced by documents length and last modification time
It is right；Since the file of cloud disk is after downloading/backuping to the storage device with wifi functions, user can delete pair from cloud disk
Then the file answered is stored in more new files to cloud disk；Therefore in individual private possession cloud storage system to publicly-owned cloud storage account
File will be under the circumstances when being scanned appearance)；If identical, the fetched values of element in old arrays are answered
It makes to corresponding element in new arrays；This operation is repeated, until element all in old arrays all takes out；
Finally, after the completion of supplement traversal, the data of new arrays are exported as to the data of json formats from the memory of program,
And it is preserved into database with packet mode.
In the above method, after upper primary scanning result is generated as multiple tasks group, there is part in task groups
Task has been completed (fetched is equal to 1), if these tasks have been in the state completed, and it is corresponding in cloud disk
File is not deleted, then it is necessary to task is arranged according to old task list in newest cloud disc file scanning result
State is to be completed (setting fetched is 1).Specifically, by the above method, so as to newest publicly-owned cloud storage
Catalogue and file structure merged with the old data stored in database.If the filename recorded in old data exists
(such as being deleted by user) is not present in current publicly-owned cloud storage, then these will be deleted in merging process
Record；If there is file of the same name in new and old record, also to be sentenced according to the value of MD5 or file size and last modification time
The whether substantial variation of disconnected file, if file produces variation, just labeled as not downloading, otherwise just according to the shape of reporter's record
State is arranged the download state of task.For there was only existing filename in new record, then it is directly disposed as not downloading.
The system that a kind of data to publicly-owned cloud storage account are backed up, including：
Binding module, for by the account of individual private possession cloud storage system and publicly-owned cloud storage account and with wifi functions
Storage device bound；
Spider module, for individual private possession cloud storage system to the catalogue and filename progress head under publicly-owned cloud storage account
Secondary traversal；
First data export and memory module, for traverse after the completion of, catalogue and file name data are exported and preserve to
In database；
Task generates and distribution module, and the data in database are generated multiple tasks for individual private possession cloud storage system
And distribute to the storage device with wifi functions；
Automatic download module, for wifi functions storage device networking after, according in current task catalogue and
Filename automatically since publicly-owned cloud storage account downloading task data.
Preferably, the binding module further includes：
Login module, the web management interface for accessing the storage device with wifi functions, and utilize individual private possession cloud
The account and password of storage system are logged in；After logging in successfully, should with wifi functions storage device i.e. and individual private possession
The account of cloud storage system establishes binding relationship；
Bind request sending module, for being initiated to public affairs on the web management interface of the storage device with wifi functions
There is the bind request of cloud storage account；
Access mandate acquisition module obtains a mandate for obtaining access mandate by the API of publicly-owned cloud storage system
key；
Information uploading module, for by the title of publicly-owned cloud storage system, authorize key and storage with wifi functions to set
Standby device numbering is uploaded to individual private possession cloud storage system；
Creation module is recorded, creates a binding relationship record for individual private possession cloud storage system, storage public cloud is deposited
Store up the relationship between account and storage device with wifi functions.
Preferably, the spider module further includes：
Structure of arrays generation module, for individual private possession cloud storage system to specify publicly-owned cloud storage account under catalogue and
Filename is traversed for the first time, generates the structure of arrays of META DATA；
First data export and memory module further includes：
Packet module, for traverse after the completion of, by the filename according to it is N number of be one group, be divided into multiple groups.
In the system that data above-mentioned to publicly-owned cloud storage account are backed up, the automatic download module also wraps
It includes：
Logging modle, after the storage device networking with wifi functions, the number of logger task content and grouping；
Task completes update module, and the download of file or complete is often completed for the storage device with wifi functions
At the download of all or part of file of a task groups, then to individual private possession cloud storage system more new task performance；
File record is searched and setup module, is existed according to packet numbering and file name for individual private possession cloud storage system
Locating file records in grouping, if finding corresponding file record, fetched=1 is arranged；If not finding corresponding file note
Record then returns to false signals to the storage device for carrying wifi functions；
Module is abandoned and reacquired to task, after receiving false signals for the storage device with wifi functions, abandons
The follow-up work of the current group obtained simultaneously reacquires first unfinished task groups；
The automatic request module of task, after the download that a task groups are often completed for the storage device with wifi functions,
Automatically next unfinished task groups are asked, until completing the last one does not complete task groups.
In the system that data above-mentioned to publicly-owned cloud storage account are backed up, the spider module further includes：
Supplement traversal and contrast module, for individual private possession cloud storage system periodically to the catalogue under publicly-owned cloud storage account
And filename carries out supplement traversal；And be compared with the catalogue of local cache and list of file names, find out new file.
Preferably, supplement traversal and contrast module further include：
Data load and memory module：It is loaded according to the sequence of grouping from database for individual private possession cloud storage system
The json data of bibliographic structure preserve the structure of arrays in the memory of program, and the array are named as：old；
Supplement spider module：For in the way of traversing for the first time under publicly-owned cloud storage account catalogue and filename into
Row supplement traversal, obtains the structure of arrays of newest directory traversal result, and the array is named as：new；
Data comparison module：For taking out an element from old arrays, searched and the element files in new arrays
The identical element of name, then abandons the element in this old array if it is not found,；It is right if having found corresponding element
(when publicly-owned cloud storage system supports MD5 values, then pass through than the MD5 values or file size of two elements and last modification time
Compare MD5 values；When publicly-owned cloud storage system does not support MD5 values, then by documents length and last modification time come generation
For the comparison of MD5 values)；If identical, the fetched values of element in old arrays are copied to corresponding member in new arrays
Element；This operation is repeated, until element all in old arrays all takes out；
Second data export and memory module, after the completion of supplementing traversal, by the data of new arrays from the memory of program
The data of json formats are exported as, and are preserved into database with packet mode.
Compared with prior art, the present invention by by the account of individual private possession cloud storage system and publicly-owned cloud storage account and
Storage device with wifi functions is bound；Then utilize individual private possession cloud storage system under publicly-owned cloud storage account
Catalogue and filename are traversed for the first time, after the completion of traversal, data are exported and are preserved into database；Individual private possession cloud storage
Data in database are generated multiple tasks and distribute to the storage device with wifi functions by system；With wifi functions
After storage device networking, downloading task data are automatically begun to, it is intelligent automatic from publicly-owned cloud storage account to realize data
The storage device with wifi functions for backing up to user ensure that the safety of data, and it is manual that whole process is not necessarily to user
Operation, data backup efficiency are higher.In addition, individual private possession cloud storage system is also periodically under publicly-owned cloud storage account in the present invention
Catalogue and filename carry out supplement traversal, find out new file, download/synchronous task list generated, so as to only to publicly-owned
The newer file of cloud storage account carries out automatic, backup in time, further improves the efficiency of automatically updating data.
Description of the drawings
Fig. 1 is a kind of work flow diagram of embodiment of the present invention；
Fig. 2 is the account and Baidu Yun accounts and the storage device with wifi functions of individual private possession cloud storage system
The method flow diagram bound；
Fig. 3 is that the storage device with wifi functions executes backup tasks and individual private possession cloud storage system (drop cloud)
It was found that having new file in publicly-owned cloud storage and generating the flow chart of backup tasks.
The present invention is further illustrated with reference to the accompanying drawings and detailed description.
Specific implementation mode
The embodiment of the present invention 1：A method of the data of publicly-owned cloud storage account are backed up, as shown in Figure 1, packet
Include following steps：
S1, by the account of individual private possession cloud storage system and publicly-owned cloud storage account and storage device with wifi functions
(storage device as carried wifi functions) is bound；It specifically includes：
S11 accesses the web management interface of the storage device with wifi functions, and utilizes individual private possession cloud storage system
Account and password logged in；After logging in successfully, should with wifi functions storage device i.e. and individual private possession cloud storage system
The account of system establishes binding relationship；
S12 initiates the binding to publicly-owned cloud storage account on the web management interface of the storage device with wifi functions
Request；Access mandate is obtained by the API of publicly-owned cloud storage system, obtains a mandate key；
S13 by the title of publicly-owned cloud storage system, authorizes key and the device numbering of storage device with wifi functions
It is uploaded to individual private possession cloud storage system；Individual private possession cloud storage system creates a binding relationship record, and storage public cloud is deposited
Store up the relationship between account and storage device with wifi functions；
S2, individual private possession cloud storage system under publicly-owned cloud storage account catalogue and filename traversed for the first time, time
After the completion of going through, catalogue and file name data are exported and preserved into database；It specifically includes：Individual private possession cloud storage system pair
It specifies catalogue and filename under publicly-owned cloud storage account to be traversed for the first time, generates the structure of arrays of META DATA；It has traversed
Cheng Hou, by the filename according to it is N number of be one group, after being divided into multiple groups, export as the data of json formats, and preserve extremely
In database；Wherein, when traversal, using catalogue or file name as index value, with file size, last modification time, MD5
Whether value is data block of the information such as catalogue as structure type；Wherein, include a member in the structure：
Fetched, the storage device with wifi functions for indicating whether to be downloaded to user, the default value of fetched
It is 0；
Data in database are generated multiple tasks and distributed to wifi functions by S3, individual private possession cloud storage system
Storage device；It specifically includes：Data in database are generated multiple tasks group by individual private possession cloud storage system according to grouping,
And give the storage device with wifi functions per one task groups of sub-distribution；
Further include：Individual private possession cloud storage system periodically under publicly-owned cloud storage account catalogue and filename supplement
Traversal (interval time of supplement traversal can adjust, and give tacit consent to 24 hours once), and ranked with the catalogue of local cache and file
Table is compared, and finds out new file, generates download/synchronous task list；Wherein, the individual private possession cloud storage system is regular
To under publicly-owned cloud storage account catalogue and filename carry out supplement traversal, and with the catalogue of local cache and list of file names into
Row compares, and finds out new file, generates download/synchronous task list and specifically includes：
First, individual private possession cloud storage system loads the json numbers of bibliographic structure from database according to the sequence of grouping
According to preserving the structure of arrays in the memory of program, and the array is named as：old；(the element for including in the array
It can be retrieved by filename, each element contains a data block, is saved in data block and file name
Corresponding MD5 values, file size, last modification time etc. and the relevant information of METADATA)；
Secondly, in the way of traversing for the first time under publicly-owned cloud storage account catalogue and filename carry out supplement traversal,
The structure of arrays of newest directory traversal result is obtained, and the array is named as：new；(its structure is identical as old, equally
, the element for including in the array can be retrieved by filename, but each element includes in the data structure
0) fetched values are all；
Again, an element is taken out from old arrays, and element identical with the element files name is searched in new arrays,
It is if it is not found, then abandon the element in this old array；If having found corresponding element, two elements of comparison
MD5 values or file size and last modification time are (when publicly-owned cloud storage system supports MD5 values, then by comparing MD5 values；When
When publicly-owned cloud storage system does not support MD5 values, then the ratio of MD5 values is replaced by documents length and last modification time
It is right)；If identical, the fetched values of element in old arrays are copied to corresponding element in new arrays；Repeat this behaviour
Make, until element all in old arrays all takes out；
Finally, after the completion of supplement traversal, the data of new arrays are exported as to the data of json formats from the memory of program,
And it is preserved into database with packet mode.
S4, with wifi functions storage device networking after, according in current task catalogue and filename automatically from public affairs
There are beginning downloading task data in cloud storage account；It specifically includes：After storage device networking with wifi functions, logger task
The number of content and grouping, and according in current task catalogue and filename automatically since publicly-owned cloud storage account download
Task data；The download for often completing all or part of file of one task groups of download or completion of a file, then to a
People's private cloud storage system update task performance；Individual private possession cloud storage system is dividing according to packet numbering and file name
Locating file records in group, if finding corresponding file record, fetched=1 is arranged；If not finding corresponding file note
Record then returns to false signals to the storage device for carrying wifi functions, and the storage device with wifi functions receives false letters
After number, abandons the follow-up work of the current group obtained and reacquire first unfinished task groups；With wifi
After the storage device of function often completes the download of a task groups, next unfinished task groups are asked automatically, until completing most
The latter does not complete task groups.
The system that a kind of data to publicly-owned cloud storage account are backed up, including：
Binding module, for by the account of individual private possession cloud storage system and publicly-owned cloud storage account and with wifi functions
Storage device bound；
Spider module, for individual private possession cloud storage system to the catalogue and filename progress head under publicly-owned cloud storage account
Secondary traversal；
First data export and memory module, for traverse after the completion of, catalogue and file name data are exported and preserve to
In database；
Task generates and distribution module, and the data in database are generated multiple tasks for individual private possession cloud storage system
And distribute to the storage device with wifi functions；
Automatic download module, for wifi functions storage device networking after, according in current task catalogue and
Filename automatically since publicly-owned cloud storage account downloading task data.
The binding module further includes：
Login module, the web management interface for accessing the storage device with wifi functions, and utilize individual private possession cloud
The account and password of storage system are logged in；After logging in successfully, should with wifi functions storage device i.e. and individual private possession
The account of cloud storage system establishes binding relationship；
Bind request sending module, for being initiated to public affairs on the web management interface of the storage device with wifi functions
There is the bind request of cloud storage account；
Access mandate acquisition module obtains a mandate for obtaining access mandate by the API of publicly-owned cloud storage system
key；
Information uploading module, for by the title of publicly-owned cloud storage system, authorize key and storage with wifi functions to set
Standby device numbering is uploaded to individual private possession cloud storage system；
Creation module is recorded, creates a binding relationship record for individual private possession cloud storage system, storage public cloud is deposited
Store up the relationship between account and storage device with wifi functions.
The spider module further includes：
Structure of arrays generation module, for individual private possession cloud storage system to specify publicly-owned cloud storage account under catalogue and
Filename is traversed for the first time, generates the structure of arrays of METADATA；
First data export and memory module further includes：
Packet module, for traverse after the completion of, by the filename according to it is N number of be one group, be divided into multiple groups.
The automatic download module further includes：
Logging modle, after the storage device networking with wifi functions, the number of logger task content and grouping；
Task completes update module, and the download of file or complete is often completed for the storage device with wifi functions
At the download of all or part of file of a task groups, then to individual private possession cloud storage system more new task performance；
File record is searched and setup module, is existed according to packet numbering and file name for individual private possession cloud storage system
Locating file records in grouping, if finding corresponding file record, fetched=1 is arranged；If not finding corresponding file note
Record then returns to false signals to the storage device for carrying wifi functions；
Module is abandoned and reacquired to task, after receiving false signals for the storage device with wifi functions, abandons
The follow-up work of the current group obtained simultaneously reacquires first unfinished task groups；
The automatic request module of task, after the download that a task groups are often completed for the storage device with wifi functions,
Automatically next unfinished task groups are asked, until completing the last one does not complete task groups.
The spider module further includes：
Supplement traversal and contrast module, for individual private possession cloud storage system periodically to the catalogue under publicly-owned cloud storage account
And filename carries out supplement traversal；And be compared with the catalogue of local cache and list of file names, find out new file.
The supplement traversal and contrast module further include：
Data load and memory module：It is loaded according to the sequence of grouping from database for individual private possession cloud storage system
The json data of bibliographic structure preserve the structure of arrays in the memory of program, and the array are named as：old；
Supplement spider module：For in the way of traversing for the first time under publicly-owned cloud storage account catalogue and filename into
Row supplement traversal, obtains the structure of arrays of newest directory traversal result, and the array is named as：new；
Data comparison module：For taking out an element from old arrays, searched and the element files in new arrays
The identical element of name, then abandons the element in this old array if it is not found,；It is right if having found corresponding element
(when publicly-owned cloud storage system supports MD5 values, then pass through than the MD5 values or file size of two elements and last modification time
Compare MD5 values；When publicly-owned cloud storage system does not support MD5 values, then by documents length and last modification time come generation
For the comparison of MD5 values)；If identical, the fetched values of element in old arrays are copied to corresponding member in new arrays
Element；This operation is repeated, until element all in old arrays all takes out；
Second data export and memory module, after the completion of supplementing traversal, by the data of new arrays from the memory of program
The data of json formats are exported as, and are preserved into database with packet mode.
Embodiment 2：A method of the data of publicly-owned cloud storage account are backed up, as shown in Figure 1, including following step
Suddenly：
S1, by the account of individual private possession cloud storage system and publicly-owned cloud storage account and storage device with wifi functions
It is bound；
S2, individual private possession cloud storage system under publicly-owned cloud storage account catalogue and filename traversed for the first time, time
After the completion of going through, catalogue and file name data are exported and preserved into database；It specifically includes：Individual private possession cloud storage system pair
It specifies catalogue and filename under publicly-owned cloud storage account to be traversed for the first time, generates the structure of arrays of METADATA；It has traversed
Cheng Hou, by the filename according to it is N number of be one group, after being divided into multiple groups, export as the data of json formats, and preserve extremely
In database；Wherein, when traversal, using catalogue or file name as index value, with file size, last modification time, MD5
Whether value is data block of the information such as catalogue as structure type；Wherein, include a member in the structure：
Fetched, the storage device with wifi functions for indicating whether to be downloaded to user, the default value of fetched
It is 0；
Data in database are generated multiple tasks and distributed to wifi functions by S3, individual private possession cloud storage system
Storage device；It specifically includes：Data in database are generated multiple tasks group by individual private possession cloud storage system according to grouping,
And give the storage device with wifi functions per one task groups of sub-distribution；
S4, with wifi functions storage device networking after, according in current task catalogue and filename automatically from public affairs
There are beginning downloading task data in cloud storage account；It specifically includes：After storage device networking with wifi functions, logger task
The number of content and grouping, and according in current task catalogue and filename automatically since publicly-owned cloud storage account download
Task data；The download for often completing all or part of file of one task groups of download or completion of a file, then to a
People's private cloud storage system update task performance；Individual private possession cloud storage system is dividing according to packet numbering and file name
Locating file records in group, if finding corresponding file record, fetched=1 is arranged；If not finding corresponding file note
Record then returns to false signals to the storage device for carrying wifi functions, and the storage device with wifi functions receives false letters
After number, abandons the follow-up work of the current group obtained and reacquire first unfinished task groups；With wifi
After the storage device of function often completes the download of a task groups, next unfinished task groups are asked automatically, until completing most
The latter does not complete task groups.
The system that a kind of data to publicly-owned cloud storage account are backed up, including：
Binding module, for by the account of individual private possession cloud storage system and publicly-owned cloud storage account and with wifi functions
Storage device bound；
Spider module, for individual private possession cloud storage system to the catalogue and filename progress head under publicly-owned cloud storage account
Secondary traversal；
First data export and memory module, for traverse after the completion of, catalogue and file name data are exported and preserve to
In database；
Task generates and distribution module, and the data in database are generated multiple tasks for individual private possession cloud storage system
And distribute to the storage device with wifi functions；
Automatic download module, for wifi functions storage device networking after, according in current task catalogue and
Filename automatically since publicly-owned cloud storage account downloading task data.
The spider module further includes：
Structure of arrays generation module, for individual private possession cloud storage system to specify publicly-owned cloud storage account under catalogue and
Filename is traversed for the first time, generates the structure of arrays of METADATA；
First data export and memory module further includes：
Packet module, for traverse after the completion of, by the filename according to it is N number of be one group, be divided into multiple groups.
The automatic download module further includes：
Logging modle, after the storage device networking with wifi functions, the number of logger task content and grouping；
Task completes update module, and the download of file or complete is often completed for the storage device with wifi functions
At the download of all or part of file of a task groups, then to individual private possession cloud storage system more new task performance；
File record is searched and setup module, is existed according to packet numbering and file name for individual private possession cloud storage system
Locating file records in grouping, if finding corresponding file record, fetched=1 is arranged；If not finding corresponding file note
Record then returns to false signals to the storage device for carrying wifi functions；
Module is abandoned and reacquired to task, after receiving false signals for the storage device with wifi functions, abandons
The follow-up work of the current group obtained simultaneously reacquires first unfinished task groups；
The automatic request module of task, after the download that a task groups are often completed for the storage device with wifi functions,
Automatically next unfinished task groups are asked, until completing the last one does not complete task groups.
Fig. 2 is the account and Baidu Yun accounts and band that method using the present invention realizes individual private possession cloud storage system
There is the method flow diagram that the storage device of wifi functions is bound.Wherein, described " management platform " the i.e. individual private possession cloud is deposited
Storage system；The synchronous container of establishment is verified to carry out in individual private possession cloud storage system.Fig. 3 is carried in the present invention
The storage device of wifi functions executes backup tasks and individual private possession cloud storage system (drop cloud) is found in publicly-owned cloud storage
There is new file and generates the flow chart of backup tasks.
Claims (16)
1. a kind of method that data to publicly-owned cloud storage account are backed up, which is characterized in that include the following steps：
S1 carries out the account of individual private possession cloud storage system and publicly-owned cloud storage account and the storage device with wifi functions
Binding；
S2, individual private possession cloud storage system under publicly-owned cloud storage account catalogue and filename traversed for the first time, traversed
Catalogue and file name data are exported and are preserved into database by Cheng Hou；
Data in database are generated multiple tasks and distribute to depositing with wifi functions by S3, individual private possession cloud storage system
Store up equipment；
S4, with wifi functions storage device networking after, according in current task catalogue and filename automatically from public cloud
It stores and starts downloading task data in account.
2. the method that the data according to claim 1 to publicly-owned cloud storage account are backed up, which is characterized in that step
S1 is specifically included：
S11 accesses the web management interface of the storage device with wifi functions, and utilizes the account of individual private possession cloud storage system
Number and password logged in；After logging in successfully, should with wifi functions storage device i.e. and individual private possession cloud storage system
Account establishes binding relationship；
S12 initiates to ask the binding of publicly-owned cloud storage account on the web management interface of the storage device with wifi functions
It asks；Access mandate is obtained by the API of publicly-owned cloud storage system, obtains a mandate key；
S13 by the title of publicly-owned cloud storage system, authorizes key and the device numbering of storage device with wifi functions to upload
To individual private possession cloud storage system；Individual private possession cloud storage system creates a binding relationship record, stores publicly-owned cloud storage account
Relationship between family and storage device with wifi functions.
3. the method that the data according to claim 1 to publicly-owned cloud storage account are backed up, which is characterized in that step
S2 is specifically included：Individual private possession cloud storage system is to specifying catalogue under publicly-owned cloud storage account and filename to carry out for the first time time
It goes through, generates the structure of arrays of META DATA；After the completion of traversal, by the filename according to it is N number of be one group, be divided into multiple
After group, the data of json formats are exported as, and preserve into database.
4. the method that the data according to claim 3 to publicly-owned cloud storage account are backed up, which is characterized in that traversal
When, using catalogue or file name as index value, with file size, last modification time, MD5 values, whether be these letters of catalogue
Cease the data block as structure type；Wherein, include a member in the structure：Fetched, for indicating whether
The storage device with wifi functions of user is had been downloaded into, the default value of fetched is 0.
5. the method that the data according to claim 4 to publicly-owned cloud storage account are backed up, which is characterized in that step
S3 is specifically included：Data in database are generated multiple tasks group by individual private possession cloud storage system according to grouping, and every time
One task groups of distribution give the storage device with wifi functions.
6. the method that the data according to claim 5 to publicly-owned cloud storage account are backed up, which is characterized in that step
S4 is specifically included：After storage device networking with wifi functions, the number of logger task content and grouping, and according to as predecessor
Catalogue and filename in business automatically since publicly-owned cloud storage account downloading task data；Often complete the download of a file
Or complete the download of all or part of file of a task groups, then complete feelings to individual private possession cloud storage system more new task
Condition；According to packet numbering and file name, locating file records individual private possession cloud storage system in a packet, if finding corresponding
Fetched=1 is then arranged in file record；If not finding corresponding file record, false signals are returned to wifi work(
The storage device of energy, after the storage device with wifi functions receives false signals, after abandoning the current group obtained
Continuous task simultaneously reacquires first unfinished task groups；Storage device with wifi functions often completes task groups
After download, next unfinished task groups are asked automatically, until completing the last one does not complete task groups.
7. the method that the data according to claim 1 or 2 to publicly-owned cloud storage account are backed up, which is characterized in that
Step S3 further includes：Individual private possession cloud storage system periodically under publicly-owned cloud storage account catalogue and filename carry out supplement time
It goes through, and is compared with the catalogue of local cache and list of file names, find out new file, generate download/synchronous task list.
8. according to the method that claim 3~6 any one of them backs up the data of publicly-owned cloud storage account, feature
It is, step S3 further includes：Individual private possession cloud storage system periodically under publicly-owned cloud storage account catalogue and filename carry out
Supplement traversal, and be compared with the catalogue of local cache and list of file names, new file is found out, download/synchronous task is generated
List.
9. the method that the data according to claim 8 to publicly-owned cloud storage account are backed up, which is characterized in that described
Individual private possession cloud storage system periodically under publicly-owned cloud storage account catalogue and filename carry out supplement traversal, and with local
The catalogue and list of file names of caching are compared, and find out new file, generate download/synchronous task list and specifically include：
First, individual private possession cloud storage system loads the json data of bibliographic structure from database according to the sequence of grouping,
The structure of arrays is preserved in the memory of program, and the array is named as：old；
Secondly, in the way of traversing for the first time under publicly-owned cloud storage account catalogue and filename carry out supplement traversal, obtain
The structure of arrays of newest directory traversal result, and the array is named as：new；
Again, an element is taken out from old arrays, and element identical with the element files name is searched in new arrays, if
It does not find, then abandons the element in this old array；If having found corresponding element, the MD5 values of two elements are compared
Or file size and last modification time；If identical, the fetched values of element in old arrays are copied in new arrays
Corresponding element；This operation is repeated, until element all in old arrays all takes out；
Finally, after the completion of supplement traversal, the data of new arrays export as to the data of json formats from the memory of program, and with
Packet mode is preserved into database.
10. the system that a kind of data to publicly-owned cloud storage account are backed up, which is characterized in that including：
Binding module, for by the account of individual private possession cloud storage system and publicly-owned cloud storage account and depositing with wifi functions
Storage equipment is bound；
Spider module, for individual private possession cloud storage system under publicly-owned cloud storage account catalogue and filename carry out for the first time time
It goes through；
First data export and memory module, after the completion of traversing, catalogue and file name data are exported and preserved to data
In library；Task generates and distribution module, and the data in database are generated multiple tasks simultaneously for individual private possession cloud storage system
Distribute to the storage device with wifi functions；
Automatic download module, after the storage device networking with wifi functions, according to the catalogue and file in current task
Name automatically since publicly-owned cloud storage account downloading task data.
11. the system that the data according to claim 10 to publicly-owned cloud storage account are backed up, which is characterized in that institute
The binding module stated further includes：
Login module, the web management interface for accessing the storage device with wifi functions, and utilize individual private possession cloud storage
The account and password of system are logged in；After logging in successfully, the storage device that should carry wifi functions is deposited with individual private possession cloud
The account of storage system establishes binding relationship；
Bind request sending module, for being initiated to public cloud on the web management interface of the storage device with wifi functions
Store the bind request of account；
Access mandate acquisition module obtains a mandate key for obtaining access mandate by the API of publicly-owned cloud storage system；
Information uploading module, for by the title of publicly-owned cloud storage system, authorize key and storage device with wifi functions
Device numbering is uploaded to individual private possession cloud storage system；
Creation module is recorded, a binding relationship record is created for individual private possession cloud storage system, stores publicly-owned cloud storage account
Relationship between family and storage device with wifi functions.
12. the system that the data according to claim 10 to publicly-owned cloud storage account are backed up, which is characterized in that institute
The spider module stated further includes：
Structure of arrays generation module, for individual private possession cloud storage system to specifying catalogue and file under publicly-owned cloud storage account
Name is traversed for the first time, generates the structure of arrays of META DATA；
First data export and memory module further includes：
Packet module, for traverse after the completion of, by the filename according to it is N number of be one group, be divided into multiple groups.
13. the system that the data according to claim 12 to publicly-owned cloud storage account are backed up, which is characterized in that institute
The automatic download module stated further includes：
Logging modle, after the storage device networking with wifi functions, the number of logger task content and grouping；
Task completes update module, often completes the download of file for the storage device with wifi functions or completes one
The download of all or part of file of a task groups, then to individual private possession cloud storage system more new task performance；
File record is searched and setup module, is being grouped according to packet numbering and file name for individual private possession cloud storage system
Middle locating file record, if finding corresponding file record, is arranged fetched=1；If not finding corresponding file record,
False signals are then returned to the storage device for carrying wifi functions；
Module is abandoned and reacquired to task, after receiving false signals for the storage device with wifi functions, abandons
The follow-up work of the current group of acquisition simultaneously reacquires first unfinished task groups；
The automatic request module of task, after the download that a task groups are often completed for the storage device with wifi functions, automatically
Next unfinished task groups are asked, until completing the last one does not complete task groups.
14. the system that the data to publicly-owned cloud storage account according to claim 10 or 11 are backed up, feature exist
In the spider module further includes：
Supplement traversal and contrast module, for individual private possession cloud storage system periodically to the catalogue and text under publicly-owned cloud storage account
Part name carries out supplement traversal；And be compared with the catalogue of local cache and list of file names, find out new file.
15. the system backed up according to claim 12 or 13 to the data of publicly-owned cloud storage account, feature exist
In the spider module further includes：
Supplement traversal and contrast module, for individual private possession cloud storage system periodically to the catalogue and text under publicly-owned cloud storage account
Part name carries out supplement traversal；And be compared with the catalogue of local cache and list of file names, find out new file.
16. the system that the data according to claim 15 to publicly-owned cloud storage account are backed up, which is characterized in that institute
The supplement stated traverses and contrast module further includes：
Data load and memory module：For individual private possession cloud storage system catalogue is loaded according to the sequence of grouping from database
The json data of structure preserve the structure of arrays in the memory of program, and the array are named as：old；
Supplement spider module：For in the way of traversing for the first time under publicly-owned cloud storage account catalogue and filename mend
Traversal is filled, obtains the structure of arrays of newest directory traversal result, and the array is named as：new；
Data comparison module：For taking out an element from old arrays, searched and element files famous prime minister in new arrays
Same element, then abandons the element in this old array if it is not found,；If having found corresponding element, two are compared
The MD5 values or file size of a element and last modification time；If identical, the fetched values of element in old arrays are answered
It makes to corresponding element in new arrays；This operation is repeated, until element all in old arrays all takes out；
Second data export and memory module, and after the completion of supplementing traversal, the data of new arrays are exported from the memory of program
For the data of json formats, and preserved into database with packet mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610158148.7A CN105824723B (en) | 2016-03-18 | 2016-03-18 | The method and system that a kind of data to publicly-owned cloud storage account are backed up |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610158148.7A CN105824723B (en) | 2016-03-18 | 2016-03-18 | The method and system that a kind of data to publicly-owned cloud storage account are backed up |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105824723A CN105824723A (en) | 2016-08-03 |
| CN105824723B true CN105824723B (en) | 2018-09-18 |
Family
ID=56524023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610158148.7A Active CN105824723B (en) | 2016-03-18 | 2016-03-18 | The method and system that a kind of data to publicly-owned cloud storage account are backed up |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105824723B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106331066B (en) * | 2016-08-15 | 2019-06-21 | 努比亚技术有限公司 | Electronic equipment and information processing method |
| CN106941415A (en) * | 2017-01-20 | 2017-07-11 | 深圳市先河系统技术有限公司 | Use the method and private clound equipment, terminal device of private clound equipment |
| CN107277152A (en) * | 2017-06-29 | 2017-10-20 | 郑州云海信息技术有限公司 | A kind of public cloud data safety backup system |
| CN107678882A (en) * | 2017-09-14 | 2018-02-09 | 北京春鸿科技有限公司 | A kind of method and system of data backup |
| CN108038028B (en) * | 2017-12-13 | 2021-03-23 | 北信源系统集成有限公司 | File backup method and device and file restoration method and device |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN112286928B (en) * | 2019-09-16 | 2023-11-28 | 重庆傲雄在线信息技术有限公司 | Chain type storage system |
| CN113450063A (en) * | 2020-09-07 | 2021-09-28 | 北京新氧科技有限公司 | Task information processing method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102377827A (en) * | 2011-12-13 | 2012-03-14 | 方正国际软件有限公司 | Multilevel cloud storage system and storage method thereof |
| CN103067406A (en) * | 2013-01-14 | 2013-04-24 | 暨南大学 | Access control system and access control method between public cloud and private cloud |
| CN105025084A (en) * | 2015-06-10 | 2015-11-04 | 国网智能电网研究院 | A cloud storage system based on synchronization agents and mixed storage |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8676763B2 (en) * | 2011-02-08 | 2014-03-18 | International Business Machines Corporation | Remote data protection in a networked storage computing environment |
-
2016
- 2016-03-18 CN CN201610158148.7A patent/CN105824723B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102377827A (en) * | 2011-12-13 | 2012-03-14 | 方正国际软件有限公司 | Multilevel cloud storage system and storage method thereof |
| CN103067406A (en) * | 2013-01-14 | 2013-04-24 | 暨南大学 | Access control system and access control method between public cloud and private cloud |
| CN105025084A (en) * | 2015-06-10 | 2015-11-04 | 国网智能电网研究院 | A cloud storage system based on synchronization agents and mixed storage |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105824723A (en) | 2016-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105824723B (en) | The method and system that a kind of data to publicly-owned cloud storage account are backed up | |
| US11816126B2 (en) | Large scale unstructured database systems | |
| US11928029B2 (en) | Backup of partitioned database tables | |
| US10873629B2 (en) | System and method of implementing an object storage infrastructure for cloud-based services | |
| US10387402B2 (en) | System and method for conditionally updating an item with attribute granularity | |
| CN106156359B (en) | A kind of data synchronization updating method under cloud computing platform | |
| CN106294585B (en) | A kind of storage method under cloud computing platform | |
| US20180322017A1 (en) | Restoring partitioned database tables from backup | |
| US11609697B2 (en) | System and method for providing a committed throughput level in a data store | |
| CN105930233B (en) | To specifying the data of public cloud account to carry out the method and system of automated back-up respectively | |
| US20170228417A1 (en) | Verification of database table partitions during backup | |
| US11914585B2 (en) | Servicing queries of a hybrid event index | |
| CN105956491B (en) | The data of publicly-owned cloud storage system are carried out with the method and system of more account backups | |
| CN104462185B (en) | A kind of digital library's cloud storage system based on mixed structure | |
| CN105978981B (en) | The linear expansion method and system of data backup based on publicly-owned cloud storage account | |
| US20170193039A1 (en) | Servicing queries of an event log | |
| CN103002027A (en) | System and method for data storage on basis of key-value pair system tree-shaped directory achieving structure | |
| CN103180842A (en) | Cloud computing system and data synchronization method therefor | |
| US10579597B1 (en) | Data-tiering service with multiple cold tier quality of service levels | |
| CN102664914A (en) | IS/DFS-Image distributed file storage query system | |
| CN109542861A (en) | File management method, device and system | |
| CN103049574A (en) | Key value system and key value method for implementation of dynamic duplicates of documents | |
| CN114610680A (en) | Method, device and equipment for managing metadata of distributed file system and storage medium | |
| CN107408239B (en) | Architecture for managing mass data in communication application through multiple mailboxes | |
| CN115563073A (en) | Method and device for data processing of distributed metadata and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100190 room 10-08, building Haidian District, Haidian street, Beijing, 38 Applicant after: Beijing Spring Technology Co., Ltd. Address before: 100190 room 10-08, building Haidian District, Haidian street, Beijing, 38 Applicant before: Beijing Chunhong Enterprise Management Consulting Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20220314 Granted publication date: 20180918 |