CN106294627A - Data managing method and data server - Google Patents
Data managing method and data server Download PDFInfo
- Publication number
- CN106294627A CN106294627A CN201610624761.3A CN201610624761A CN106294627A CN 106294627 A CN106294627 A CN 106294627A CN 201610624761 A CN201610624761 A CN 201610624761A CN 106294627 A CN106294627 A CN 106294627A
- Authority
- CN
- China
- Prior art keywords
- data file
- file
- data
- stored
- memory node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/1827—Management specifically adapted to NAS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the present invention provides a kind of data managing method and data server, and the method includes: receiving client and send data storage request, data file to be stored is stored by data storage request for instruction；Judge whether memory node exists the storage data file identical with data file to be stored；The most then obtain the current absolute path having stored data file in memory node, current absolute path is defined as the absolute path of file to be stored；If it is not, then obtain multiple subdata files of data file to be stored, determine the memory node that each subdata file is corresponding, and each subdata file is stored to corresponding memory node.In the efficiency improving data storage.
Description
Technical field
The present embodiments relate to communication technical field, particularly relate to a kind of data managing method and data server.
Background technology
At present, for the ease of storing the data on terminal unit, the data in terminal unit can be deposited by user
Storage is to data server.
In the prior art, when terminal unit carries out data storage, data to be stored are generally sent extremely by terminal unit
Data server, data server selects a memory node in multiple memory nodes (such as, computer), and by number to be stored
According to sending to this memory node, so that data to be stored are stored by this memory node.But, if at data server to depositing
Storage node sends during data to be stored, if data to be stored fail correct transmission to memory node, then needs to treat
Total data in storage data retransmits, and causes data storage efficiency low.
Summary of the invention
The embodiment of the present invention provides a kind of data managing method and data server, for improving the efficiency of data storage.
First aspect, the embodiment of the present invention provides a kind of data managing method, including:
Receiving client and send data storage request, data file to be stored is entered by described data storage request for instruction
Row storage；
Judge whether memory node exists the storage data file identical with described data file to be stored；
Store the data file current absolute path in described memory node described in the most then obtaining, worked as described
Front absolute path is defined as the absolute path of described file to be stored；
If it is not, then obtain multiple subdata files of described data file to be stored, determine each described subdata file pair
The memory node answered, and each described subdata file is stored to corresponding memory node.
In a kind of possible embodiment, described judge whether memory node exists and described data file to be stored
Identical storage data file, including:
Obtain each of storage in the MD5 value of described data file to be stored and described memory node and store data literary composition
The MD5 value of part；
In MD5 value according to described data file to be stored and described memory node, each of storage has stored data literary composition
The MD5 value of part, it is judged that whether there is the storage data file identical with described data file to be stored in memory node.
In alternatively possible embodiment, multiple subdata files of the described data file to be stored of described acquisition,
Including:
Obtain attribute information and the size of described data file to be stored of described memory node；
Attribute information according to described memory node and the size of described data file to be stored, to described data to be stored
File carries out dividing processing, obtains the plurality of subdata file.
In alternatively possible embodiment, the attribute information of described memory node includes data in described memory node
The size of block；Accordingly, according to attribute information and the size of described data file to be stored of described memory node, treat described
Storage data file carries out dividing processing, obtains the plurality of subdata file, including:
According to the size of described data block, described data file to be stored is carried out dividing processing, obtain N number of subdata literary composition
, wherein, in described N number of subdata file, at least there is the size size equal to described data block of N-1 data file in part.
In alternatively possible embodiment, the attribute information of described memory node includes data in described memory node
The size of block and the number of memory node；Accordingly, according to attribute information and the described data to be stored literary composition of described memory node
The size of part, carries out dividing processing to described data file to be stored, obtains the plurality of subdata file, including:
Size according to described data block and the size of described data file to be stored, determine described data file to be stored
The the first data block number included；
Number according to described memory node and described first data block number, determine that each described memory node is corresponding
Two data block numbers；
The second data block number corresponding according to each described memory node and the size of described data block, deposit described waiting
Storage data file carries out dividing processing, obtains M sub-data file, and wherein, described M is equal to the number of described memory node.
In alternatively possible embodiment, described each described subdata file is stored to corresponding memory node
Afterwards, also include:
Obtain each described subdata file absolute path in corresponding memory node；
According to each described subdata file absolute path in corresponding memory node, generate described data to be stored literary composition
The absolute path of part.
In alternatively possible embodiment, described each described subdata file is stored to corresponding memory node
Afterwards, also include:
The subdata file that store in each described memory node, described data file is corresponding is merged process,
To verification data file；
Judge that the MD5 value of described data file to be stored is the most identical with the MD5 value of described inspection data file；
The most then the client to described user place sends and uploads successfully corresponding message；
The most described data file is heavily deposited process.
In alternatively possible embodiment, described method also includes:
Receiving the data download request that client sends, described data download request is for indicating data file to be downloaded
It is downloaded；
According to the absolute path of described data file to be downloaded, described memory node obtains described data to be downloaded literary composition
Multiple subdata files of part；
Multiple subdata files of described data file to be downloaded are merged process, obtains described data to be downloaded literary composition
Part；
Described data file to be downloaded is sent to described client.
In alternatively possible embodiment, described method also includes:
Receiving the Data Migration request that client sends, the request of described Data Migration includes the mark of data file to be migrated
Know and target relative path；
Ask according to described Data Migration, the relative path of described data file to be migrated is updated to described target relative
Path.
Second aspect, the embodiment of the present invention provides a kind of data server, including receiver module, judge module, first true
Cover half block, the first acquisition module, second determine module and memory module, wherein,
Described receiver module is used for, and receives client and sends data storage request, and described data storage request is used for indicating
Data file to be stored is stored；
Whether described judge module is used for, it is judged that exist in memory node and described identical the depositing of data file to be stored
Storage data file；
Described first determine module for, in described judge module judges described memory node exist to be stored with described
During the identical storage data file of data file, stored described in acquisition data file in described memory node current absolutely
To path, and described current absolute path is defined as the absolute path of described file to be stored；
Described first acquisition module is used for, and does not exist and deposit with described waiting in described judge module judges described memory node
During the storage identical storage data file of data file, obtain multiple subdata files of described data file to be stored；Accordingly
, described second determine module for determining the memory node that each described subdata file is corresponding, described memory module for will
Each described subdata file stores to corresponding memory node.
In a kind of possible embodiment, described judge module specifically for:
Obtain each of storage in the MD5 value of described data file to be stored and described memory node and store data literary composition
The MD5 value of part；
In MD5 value according to described data file to be stored and described memory node, each of storage has stored data literary composition
The MD5 value of part, it is judged that whether there is the storage data file identical with described data file to be stored in memory node.
In alternatively possible embodiment, described first acquisition module includes acquiring unit and cutting unit, wherein,
Described acquiring unit is used for, and obtains the attribute information of described memory node and the big of described data file to be stored
Little；
Described cutting unit is used for, big according to the attribute information of described memory node and described data file to be stored
Little, described data file to be stored is carried out dividing processing, obtains the plurality of subdata file.
In alternatively possible embodiment, the attribute information of described memory node includes data in described memory node
The size of block；Accordingly, described cutting unit specifically for:
According to the size of described data block, described data file to be stored is carried out dividing processing, obtain N number of subdata literary composition
, wherein, in described N number of subdata file, at least there is the size size equal to described data block of N-1 data file in part.
In alternatively possible embodiment, the attribute information of described memory node includes data in described memory node
The size of block and the number of memory node；Accordingly, described cutting unit specifically for:
Size according to described data block and the size of described data file to be stored, determine described data file to be stored
The the first data block number included；
Number according to described memory node and described first data block number, determine that each described memory node is corresponding
Two data block numbers；
The second data block number corresponding according to each described memory node and the size of described data block, deposit described waiting
Storage data file carries out dividing processing, obtains M sub-data file, and wherein, described M is equal to the number of described memory node.
In alternatively possible embodiment, described data server also includes the second acquisition module and generation module,
Wherein,
Described second acquisition module is used for, and stores each described subdata file to corresponding storage in described memory module
After in node, obtain each described subdata file absolute path in corresponding memory node；
Described generation module is used for, according to each described subdata file absolute path in corresponding memory node, raw
Become the absolute path of described data file to be stored.
In alternatively possible embodiment, described data server also includes merging module and sending module, wherein,
Described merging module is used for, and stores each described subdata file to corresponding memory node in described memory module
After in, the subdata file that store, described data file is corresponding is merged process, obtain in each described memory node
Verification data file；
Described judge module is additionally operable to, it is judged that the MD5 value of described data file to be stored and described inspection data file
MD5 value is the most identical；
Described sending module is used for, and judges the MD5 value of described data file to be stored and described inspection at described judge module
Test the MD5 value of data file identical time, the client to described user place sends and uploads successfully corresponding message；
Described memory module is additionally operable to, and judges the MD5 value of described data file to be stored and described at described judge module
When the MD5 value of inspection data file differs, described data file is heavily deposited process.
In alternatively possible embodiment, described data server also includes the 3rd acquisition module, wherein,
Described receiver module is additionally operable to, and receives the data download request that client sends, and described data download request is used for
Data file to be downloaded is downloaded by instruction；
Described 3rd acquisition module is used for, according to the absolute path of described data file to be downloaded, at described memory node
Multiple subdata files of the described data file to be downloaded of middle acquisition；
Described merging module is additionally operable to, and multiple subdata files of described data file to be downloaded are merged process,
Obtain described data file to be downloaded；
Described sending module is additionally operable to, and sends described data file to be downloaded to described client.
In alternatively possible embodiment, described data server also includes transferring module, wherein,
Described receiver module is additionally operable to, and receives the Data Migration request that client sends, wraps in the request of described Data Migration
Include mark and the target relative path of data file to be migrated；
Described transferring module is used for, and asks according to described Data Migration, by the relative path of described data file to be migrated
It is updated to described target relative path.
The data managing method of embodiment of the present invention offer and data server, receive client at data server and send out
After sending data storage request, if memory node exists the storage data file identical with data file to be stored, then obtain
Take the current absolute path storing data file in memory node, current absolute path is defined as the exhausted of file to be stored
To path, and then avoid storing the data file of repetition in memory node.If memory node not existing and data to be stored
The storage data file that file is identical, then data server obtains multiple subdata files of data file to be stored, determines
The memory node that each subdata file is corresponding, and each subdata file is stored to corresponding memory node, so, not only can
The enough speed improved to memory node transmission data to be stored, if in data transmission procedure, each subdata file fails correctly
Transmission to corresponding memory node, then have only to this subdata file is retransmitted, it is not necessary to other subdata literary composition
Part retransmits, and then improves the efficiency of data storage.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
In having technology to describe, the required accompanying drawing used is briefly described, it should be apparent that, the accompanying drawing in describing below is this
Some bright embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to
Other accompanying drawing is obtained according to these accompanying drawings.
The application scenarios schematic diagram of the data managing method that Fig. 1 provides for the present invention；
The schematic flow sheet of the data managing method that Fig. 2 provides for the present invention；
The schematic flow sheet of the acquisition subdata document method that Fig. 3 provides for the present invention；
The schematic flow sheet of the data file downloading method that Fig. 4 provides for the present invention；
The schematic flow sheet of the data migration method that Fig. 5 provides for the present invention；
The structural representation one of the data server that Fig. 6 provides for the present invention；
The structural representation two of the data server that Fig. 7 provides for the present invention.
Detailed description of the invention
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
The a part of embodiment of the present invention rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under not making creative work premise, broadly falls into the scope of protection of the invention.
The application scenarios schematic diagram of the data managing method that Fig. 1 provides for the present invention, refers to Fig. 1, including multiple clients
End (being designated as 101-1 to 101-N respectively), data server 102 and multiple memory node (being designated as 103-1 to 103-M respectively).
Client can be the terminal unit such as mobile phone, computer.Data file can be uploaded to data, services by client 101 by user
Device 102, the data file of client upload is stored in memory node by data server 102.When user needs to obtain data
During file, the data file that user can have been stored to data server 102 acquisition request by client, data server
102 obtain corresponding data file in memory node, and send to client.Below, by specific embodiment to this Shen
Technical scheme shown in please is described in detail.
It should be noted that these specific embodiments can be combined with each other, for same or analogous concept below
Or process may repeat no more in certain embodiments.
The schematic flow sheet of the data managing method that Fig. 2 provides for the present invention, refers to Fig. 2, and the method may include that
S201, reception client send data storage request, and data storage request is for indicating data file to be stored
Store；
S202, judge whether memory node exists the storage data file identical with data file to be stored；
The most then perform S203；
If it is not, then perform S204；
S203, acquisition have stored the data file current absolute path in memory node, current absolute path are determined
Absolute path for file to be stored；
S204, obtain multiple subdata files of data file to be stored, determine the storage joint that each subdata file is corresponding
Point, and each subdata file is stored to corresponding memory node.
When user needs request data server to store data file, user can be logged in by individual's account
Data server, and select under individual's account the store path of data file, this store path can be under individual account
It is stored in file, it is also possible under individual's account, create new file for user.Optionally, the data file under individual's account
Privately owned file and Shared Folders can be included, privately owned file stores the private data file of user, Shared Folders
Middle storage common data file.User uploads to data server needs the data file of storage, receives at data server
After data file to be stored, the mark (or icon) of data file is stored in the store path that user selects by server,
And data file is stored in memory node.For the ease of being described below, by data file mark (or icon) exist
Path in the file that user selects is referred to as relative path, and data file path in memory node is referred to as absolute road
Footpath.
In actual application, when user needs to store data file, user passes through client to data
Server sends data storage request, and data file to be stored is deposited by this data storage request for request data server
Storage.Optionally, data file to be stored can be carried in data storage request.
After data server receives the data storage request that client sends, data server judges memory node
In whether there is the storage data file identical with data file to be stored.Optionally, data server can be by as follows
Feasible implementation judges whether to exist in memory node the storage data file identical with data file to be stored, specifically
: in the MD5 value of data server acquisition data file to be stored and memory node, each of storage has stored data file
5th edition Message Digest 5 (Message Digest Algorithm is called for short MD5) value, according to data file to be stored
Each MD5 value having stored data file of storage in MD5 value and memory node, it is judged that whether exist in memory node and treat
The storage data file that storage data file is identical.Wherein, when content difference in data file, the MD5 of data file
Value is also different, therefore, can judge whether include in memory node and treat accurately according to the MD5 value of data file to be stored
The storage data file that storage data file is identical.
If memory node includes the storage data file identical with data file to be stored, then data server obtains
Store the data file current absolute path in memory node, current absolute path has been defined as the absolute of file to be stored
Path.This way it is possible to avoid store the data file of repetition in memory node.
If memory node does not include the storage data file identical with data file to be stored, then data server pair
Data file to be stored is split, and obtains multiple subdata files of data file to be stored, determines each subdata literary composition respectively
The memory node that part is corresponding, and each subdata file is stored to corresponding memory node.Further, data server is also
Obtain each subdata file absolute path in corresponding memory node, according to each subdata file at corresponding memory node
In absolute path, generate the absolute path of data file to be stored.
Exemplary, when user needs data server to store data file 1, user passes through client to number
Data file 1 is sent according to server.
After data server receives data file 1, data server obtains the MD5 value and respectively of data file 1
The MD5 value of the data file stored in memory node, and store according in the MD5 value of data file 1 and each memory node
The MD5 value of data file, it is judged that whether memory node includes the storage data literary composition identical with data file 1 to be stored
Part.
If memory node includes the storage data file (data file 2) identical with data file 1, then data, services
Device obtains the absolute path of data file 2, and the absolute path of data file 2 is defined as the absolute path of data file 1.This
Sample, when user obtains data file 1 to data server requests, data server can be according to the absolute road of data file 1
Footpath (absolute path of data file 2) acquires data file 2, owing to this data file 2 is identical with data file 1, then counts
Can be to the user feedback data file 2 identical with data file 1 according to server.
If not including the storage data file identical with data file 1 in memory node, then data server is by data
File 1 is divided into multiple subdata file, it is assumed that data file 1 is divided into subdata file 1, subdata literary composition by data server
Part 2 and subdata file 3, then subdata file 1 can be stored in memory node 1, by subdata file 2 by data server
It is stored in memory node 2, subdata file is stored in memory node 3.Further, data server is according to each subnumber
According to file absolute path in corresponding memory node, generate the absolute path of data file to be stored.
The data managing method that the embodiment of the present invention provides, receiving client transmission data storage at data server please
After asking, if memory node exists the storage data file identical with data file to be stored, then obtain and stored data
The file current absolute path in memory node, is defined as the absolute path of file to be stored by current absolute path, and then
Avoid storing the data file of repetition in memory node.If memory node does not exist identical with data file to be stored
Storage data file, then data server obtains multiple subdata files of data file to be stored, determines each subdata file
Corresponding memory node, and each subdata file is stored to corresponding memory node, so, it is not only able to improve to storage
Node sends the speed of data to be stored, if in data transmission procedure, each subdata file fails correct transmission to accordingly
Memory node, then have only to this subdata file is retransmitted, it is not necessary to other subdata file is retransmitted, enters
And improve the efficiency of data storage.
On the basis of embodiment illustrated in fig. 2, optionally, data server can be obtained by the most feasible implementation
Take multiple subdata files (S204 in embodiment illustrated in fig. 2) of data file to be stored, concrete, shown in Figure 3
Embodiment.
The schematic flow sheet of the acquisition subdata document method that Fig. 3 provides for the present invention, refers to Fig. 3, and the method is permissible
Including:
S301, the attribute information obtaining memory node and the size of data file to be stored；
S302, according to the attribute information of memory node and the size of data file to be stored, data file to be stored is entered
Row dividing processing, obtains multiple subdata file.
In actual application, when data server needs to obtain multiple subdata files of data file to be stored
Time, data server obtains attribute information and the size of data file to be stored of each memory node, optionally, memory node
Attribute information can include the residual capacity etc. of the size of data block, memory node in memory node.
Data server is according to the attribute information of memory node and the size of data file to be stored, to data to be stored literary composition
Part carries out dividing processing, obtains multiple subdata file.Optionally, the attribute of the memory node acquired when data server
During information difference, the process that data, services carries out dividing processing to data to be stored also differs, concrete, including the following two kinds
Feasible implementation:
A kind of feasible implementation: the attribute information of memory node includes the size of data block in memory node.
In this kind of feasible implementation, data file to be stored, according to the size of data block, is entered by data server
Row dividing processing, obtains N number of subdata file, wherein, at least there is the size of N-1 data file in N number of subdata file
Size equal to data block.
Accordingly, after data server acquires N number of subdata file, data server can be according to storage joint
The number of point, by each subdata file uniform distribution to each memory node.
Optionally, data server passes through equation below one, data file to be stored is carried out dividing processing, obtains N number of
Subdata file:
Wherein, D is the size of data file to be stored, and d is the size of data block,For rightRound up.
Exemplary, it is assumed that the number of memory node is 4, and in memory node, the size of data block is 68MB, it is further assumed that
The size of data file to be stored is 960MB, then data server is according to above-mentioned formula one, determines the subdata file that obtains
Number is as follows:
After data file to be stored is split by data server, obtain 15 sub-data files, wherein, 14 sons
The size of data file is 68MB, and the size of 1 sub-data file is 960-68*14=8MB.
Accordingly, the distribution that each subdata file is divided equally, according to the number of memory node, is deposited by data server to each
Storage node, such as, the corresponding relation of each subdata file and each memory node can be as shown in table 1:
Table 1
| The mark of memory node | The mark of subdata file |
| Memory node 1 | Subdata file 1-subdata file 4 |
| Memory node 2 | Subdata file 5-subdata file 8 |
| Memory node 3 | Subdata file 9-subdata file 12 |
| Memory node 4 | Subdata file 13-subdata file 15 |
In above process, data server is according to the size of the data block in memory node, to data file to be stored
Splitting, the size of the subdata file obtained is identical with the size of data block so that enter data block in memory node
During row storage, improve the resource utilization to memory node.
Another kind of feasible implementation: the attribute information of memory node includes in memory node the size of data block and deposits
The number of storage node；
In this kind of feasible implementation, data server is big according to the size of data block and data file to be stored
Little, determine the first data block number that data file to be stored includes, according to number and first data block of memory node
Number, determines the second data block number that each memory node is corresponding, the second data block number corresponding according to each memory node and number
According to the size of block, data file to be stored being carried out dividing processing, obtain M sub-data file, wherein, M is equal to memory node
Number.
Exemplary, it is assumed that the number of memory node is 4, and these 4 memory nodes are designated as memory node 1-storage respectively
Node 4, in memory node, the size of data block is 68MB, it is further assumed that the size of data file to be stored is 960MB.
Data server is first according to size (68MB) and the size (960MB) of data file to be stored of data block, really
The first data block number that fixed data file to be stored includes.Optionally, number to be stored can be determined by above-mentioned formula one
15 data blocks are included according to file.
Data server, in the number (4) according to memory node and the first data block number (15), determines each storage joint
The second data block number that point is corresponding, concrete, the second data block number that 3 memory nodes are corresponding is 4,1 storage joint
The number of the second data block that point is corresponding is 3.
Data server is according to the second data block number corresponding to each memory node and the size of data block, to number to be stored
Carrying out dividing processing according to file, concrete, data file segmentation to be stored is 4 data files by data server, wherein 3
The size of data file is 4*68=272MB, and the size of 1 data file is 3*68=207MB.
In above process, the data server size according to the data block in memory node and the number of memory node,
Data file to be stored is split, the integral multiple that size is data block size of the subdata file obtained, the most permissible
Improve the resource utilization to memory node, decrease the segmentation times to data file to be stored, and then improve to be stored
The segmentation efficiency of data file.
On the basis of any one embodiment above-mentioned, each subdata file stored to corresponding at data server and depositing
After in storage node, the subdata file that store in each memory node, data file is corresponding can also be entered by data server
Row merging treatment, obtains verifying data file, it is judged that the MD5 value of data file to be stored and the MD5 value of inspection data file are
No identical, the most then the client to user place sends and uploads successfully corresponding message, the most then data file carries out weight
Deposit process.
On the basis of any one embodiment above-mentioned, the data file stored can also be downloaded by user, under
Face, is described in detail to the downloading process of data file by the embodiment shown in Fig. 4.
The schematic flow sheet of the data file downloading method that Fig. 4 provides for the present invention, refers to Fig. 4, and the method can be wrapped
Include:
The data download request that S401, reception client send, data download request is civilian to data to be downloaded for instruction
Part is downloaded；
S402, absolute path according to data file to be downloaded, obtain data file to be downloaded many in memory node
Individual sub-data file；
S403, multiple subdata files to data file to be downloaded merge process, obtain data file to be downloaded；
S404, to client send data file to be downloaded.
In the embodiment shown in fig. 4, when user needs to be downloaded the data file stored, user can be
Client, by individual's account logon data server, selects the data file stored under individual's account to be downloaded operation.
After data server receives the data download request that client sends, data server obtains number to be downloaded
According to the absolute path of file, the absolute path of this data file to be downloaded is used for indicating data file to be downloaded in memory node
Store path.Data server, according to the absolute path of data file to be downloaded, obtains data to be downloaded in memory node
Multiple subdata files of file.
After data server acquires multiple subdata file, data server is many to data file to be downloaded
Individual sub-data file merges process, obtains data file to be downloaded.Optionally, data server can be according to to be downloaded
Segmentation order when data file is split, merges process to each subdata file.
After data server acquires data file to be downloaded, data server sends number to be downloaded to client
According to file.
In above process, during data server obtains data file to be downloaded from memory node, data
Server can send data acquisition request to multiple memory nodes simultaneously, to realize obtaining from multiple memory nodes treating simultaneously
Download multiple subdata files that data file is corresponding, and then improve the speed obtaining data file to be downloaded.Further, if
One sub-data file is sent unsuccessfully by memory node, then memory node has only to carry out this data file retransmitting i.e.
Can, it is not necessary to other subdata file is retransmitted, and then improve the efficiency of data storage.
On the basis of any one embodiment above-mentioned, the data file stored can also be migrated by user, under
Face, is described in detail to the transition process of data file by the embodiment shown in Fig. 5.
The schematic flow sheet of the data migration method that Fig. 5 provides for the present invention, refers to Fig. 5, and the method may include that
The Data Migration request that S501, reception client send, Data Migration request includes data file to be migrated
Mark and target relative path；
S502, according to Data Migration ask, the relative path of data file to be migrated is updated to target relative path.
In the embodiment shown in fig. 5, the data file stored can be migrated by user, i.e. by individual subscriber
Another file in data file migration in account next one file.When user needs to enter the data file stored
When row migrates, user can pass through individual's account logon data server in client, and by the data in a data file
File migration is to another data file.
After data server receives Data Migration request, data server obtains the number to be migrated in migration request
According to mark and the target relative path of file, this target relative path is the path of destination folder corresponding to data to be migrated.
Data server is asked according to Data Migration, and the relative path of data file to be migrated is updated to target relative path.
In above process, after data file is migrated by user, it is not necessary to change the actual storage of data file
Position, it is only necessary to the relative path of data file is updated, and then improve the effect that data file is migrated
Rate.
On the basis of any one embodiment above-mentioned, the data file stored can also be shared by user, tool
Body, user can select data file to be shared, and selects user corresponding to this data file to be shared.Data server is then
According to shared data file and corresponding user, generate data file and the corresponding relation of user, so that corresponding user possesses
Check, download the authority of this data file.
The structural representation one of the data server that Fig. 6 provides for the present invention, refers to Fig. 6, and this data server is permissible
Including: receiver module 601, judge module 602, first determine module the 603, first acquisition module 604, second determine module 605
And memory module 606, wherein,
Receiver module 601 is used for, and receives client and sends data storage request, and data storage request is treated for instruction and deposited
Storage data file stores；
Judge module 602 is used for, it is judged that whether there is the storage number identical with data file to be stored in memory node
According to file；
First determine module 603 for, judge module 602 judge in memory node exist and data file phase to be stored
With storage data file time, obtain and stored the data file current absolute path in memory node, and will be the most exhausted
Path is defined as the absolute path of file to be stored；
First acquisition module 604 is used for, and judges not exist in memory node and data file to be stored at judge module 602
During identical storage data file, obtain multiple subdata files of data file to be stored；Accordingly, second determines module
605 for determining the memory node that each subdata file is corresponding, and memory module 606 is for storing each subdata file to correspondence
Memory node in.
The data server that the embodiment of the present invention provides can perform the technical scheme shown in said method embodiment, in fact
Existing principle and beneficial effect are similar to, and the most no longer repeat.
In a kind of possible embodiment, it is judged that module 602 specifically for:
Obtain each MD5 having stored data file of storage in the MD5 value of data file to be stored and memory node
Value；
Each MD5 having stored data file of storage in MD5 value according to data file to be stored and memory node
Value, it is judged that whether there is the storage data file identical with data file to be stored in memory node.
The structural representation two of the data server that Fig. 7 provides for the present invention, on the basis of embodiment illustrated in fig. 6, please
Seeing Fig. 7, the first acquisition module 604 includes acquiring unit 6041 and cutting unit 6042, wherein,
Acquiring unit 6041 is used for, and obtains attribute information and the size of data file to be stored of memory node；
Cutting unit 6042 is used for, and according to attribute information and the size of data file to be stored of memory node, treats and deposits
Storage data file carries out dividing processing, obtains multiple subdata file.
In a kind of possible embodiment, when the attribute information of memory node includes the size of data block in memory node
Time, accordingly, cutting unit 6042 specifically for:
According to the size of data block, data file to be stored is carried out dividing processing, obtain N number of subdata file, wherein,
N number of subdata file at least exists the size size equal to data block of N-1 data file.
In alternatively possible embodiment, when the attribute information of memory node includes the big of data block in memory node
During the number of little and memory node, accordingly, cutting unit 6042 specifically for:
Size according to data block and the size of data file to be stored, determine that data file to be stored includes first
Data block number；
Number according to memory node and the first data block number, determine the second data block that each memory node is corresponding
Number；
The second data block number corresponding according to each memory node and the size of data block, enter data file to be stored
Row dividing processing, obtains M sub-data file, and wherein, M is equal to the number of memory node.
Further, this data server also includes the second acquisition module 607 and generation module 608, wherein,
Second acquisition module 607 is used for, and in memory module, each subdata file is stored to corresponding memory node it
After, obtain each subdata file absolute path in corresponding memory node；
Generation module 608 is used for, and according to each subdata file absolute path in corresponding memory node, generates and waits to deposit
The absolute path of storage data file.
Further, this data server also includes merging module 609 and sending module 610, wherein,
Merge module 609 to be used for, after each subdata file is stored to corresponding memory node by memory module, right
Subdata file that store in each memory node, that data file is corresponding merges process, obtains verifying data file；
Judge module 602 is additionally operable to, it is judged that whether the MD5 value of the MD5 value of data file to be stored and inspection data file
Identical；
Sending module 610 is used for, and judges MD5 value and the inspection data file of data file to be stored at judge module 602
MD5 value identical time, client to user place sends and uploads successfully corresponding message；
Memory module 606 is additionally operable to, and judges MD5 value and the inspection data literary composition of data file to be stored at judge module 602
When the MD5 value of part differs, data file is heavily deposited process.
Further, this data server also includes the 3rd acquisition module 611, wherein,
Receiver module 601 is additionally operable to, and receives the data download request that client sends, and it is right that data download request is used for indicating
Data file to be downloaded is downloaded；
3rd acquisition module 611 is used for, and according to the absolute path of data file to be downloaded, obtains under treating in memory node
Carry multiple subdata files of data file；
Merge module 609 to be additionally operable to, multiple subdata files of data file to be downloaded are merged process, is treated
Download data file；
Sending module 610 is additionally operable to, and sends data file to be downloaded to client.
Further, this data server also includes transferring module 612, wherein,
Receiver module 601 is additionally operable to, and receives the Data Migration request that client sends, and Data Migration request includes waiting to move
Move mark and the target relative path of data file；
Transferring module 612 is used for, and asks according to Data Migration, the relative path of data file to be migrated is updated to target
Relative path.
The data server that the embodiment of the present invention provides can perform the technical scheme shown in said method embodiment, in fact
Existing principle and beneficial effect are similar to, and the most no longer repeat.
One of ordinary skill in the art will appreciate that: all or part of step realizing above-mentioned each method embodiment can be led to
The hardware crossing programmed instruction relevant completes.Aforesaid program can be stored in a computer read/write memory medium.This journey
Sequence upon execution, performs to include the step of above-mentioned each method embodiment；And aforesaid storage medium includes: ROM, RAM, magnetic disc or
The various media that can store program code such as person's CD.
Last it is noted that various embodiments above is only in order to illustrate technical scheme, it is not intended to limit；To the greatest extent
The present invention has been described in detail by pipe with reference to foregoing embodiments, it will be understood by those within the art that: it depends on
So the technical scheme described in foregoing embodiments can be modified, or the most some or all of technical characteristic is entered
Row equivalent；And these amendments or replacement, do not make the essence of appropriate technical solution depart from various embodiments of the present invention technology
The scope of scheme.
Claims (18)
1. a data managing method, it is characterised in that including:
Receiving client and send data storage request, data file to be stored is deposited by described data storage request for instruction
Storage；
Judge whether memory node exists the storage data file identical with described data file to be stored；
The data file current absolute path in described memory node has been stored, by described current exhausted described in the most then obtaining
Path is defined as the absolute path of described file to be stored；
If it is not, then obtain multiple subdata files of described data file to be stored, determine that each described subdata file is corresponding
Memory node, and each described subdata file is stored to corresponding memory node.
Method the most according to claim 1, it is characterised in that described judge whether memory node exists deposit with described waiting
The storage data file that storage data file is identical, including:
Obtain each of storage in the MD5 value of described data file to be stored and described memory node and store data file
MD5 value；
In MD5 value according to described data file to be stored and described memory node, each of storage has stored data file
MD5 value, it is judged that whether there is the storage data file identical with described data file to be stored in memory node.
Method the most according to claim 1, it is characterised in that multiple subnumbers of the described data file to be stored of described acquisition
According to file, including:
Obtain attribute information and the size of described data file to be stored of described memory node；
Attribute information according to described memory node and the size of described data file to be stored, to described data file to be stored
Carry out dividing processing, obtain the plurality of subdata file.
Method the most according to claim 3, it is characterised in that the attribute information of described memory node includes described storage joint
The size of data block in point；Accordingly, according to attribute information and the size of described data file to be stored of described memory node,
Described data file to be stored is carried out dividing processing, obtains the plurality of subdata file, including:
According to the size of described data block, described data file to be stored is carried out dividing processing, obtains N number of subdata file,
Wherein, described N number of subdata file at least exists the size size equal to described data block of N-1 data file.
Method the most according to claim 3, it is characterised in that the attribute information of described memory node includes described storage joint
The size of data block and the number of memory node in point；Accordingly, attribute information and described waiting according to described memory node are deposited
The size of storage data file, carries out dividing processing to described data file to be stored, obtains the plurality of subdata file, bag
Include:
Size according to described data block and the size of described data file to be stored, determine in described data file to be stored and wrap
The the first data block number included；
Number according to described memory node and described first data block number, determine the second number that each described memory node is corresponding
According to block number；
The second data block number corresponding according to each described memory node and the size of described data block, to described number to be stored
Carrying out dividing processing according to file, obtain M sub-data file, wherein, described M is equal to the number of described memory node.
6. according to the method described in any one of claim 1-5, it is characterised in that described each described subdata file is stored to
After in corresponding memory node, also include:
Obtain each described subdata file absolute path in corresponding memory node；
According to each described subdata file absolute path in corresponding memory node, generate described data file to be stored
Absolute path.
7. according to the method described in any one of claim 1-5, it is characterised in that described each described subdata file is stored to
After in corresponding memory node, also include:
The subdata file that store in each described memory node, described data file is corresponding is merged process, obtains school
Test data file；
Judge that the MD5 value of described data file to be stored is the most identical with the MD5 value of described inspection data file；
The most then the client to described user place sends and uploads successfully corresponding message；
The most described data file is heavily deposited process.
8. according to the method described in any one of claim 1-5, it is characterised in that described method also includes:
Receiving the data download request that client sends, data file to be downloaded is carried out by described data download request for instruction
Download；
According to the absolute path of described data file to be downloaded, described memory node obtains described data file to be downloaded
Multiple subdata files；
Multiple subdata files of described data file to be downloaded are merged process, obtains described data file to be downloaded；
Described data file to be downloaded is sent to described client.
9. according to the method described in any one of claim 1-5, it is characterised in that described method also includes:
Receive client send Data Migration request, described Data Migration request include data file to be migrated mark and
Target relative path；
Ask according to described Data Migration, the relative path of described data file to be migrated is updated to described target and satisfies the need mutually
Footpath.
10. a data server, it is characterised in that include receiver module, judge module, first determine module, first obtain
Module, second determine module and memory module, wherein,
Described receiver module is used for, and receives client and sends data storage request, and described data storage request is treated for instruction
Storage data file stores；
Described judge module is used for, it is judged that whether there is the storage number identical with described data file to be stored in memory node
According to file；
Described first determine module for, in described judge module judges described memory node exist with described data to be stored
During the identical storage data file of file, described in acquisition, store the data file current definitely road in described memory node
Footpath, and described current absolute path is defined as the absolute path of described file to be stored；
Described first acquisition module is used for, and does not exist and described number to be stored in described judge module judges described memory node
During according to the storage data file that file is identical, obtain multiple subdata files of described data file to be stored；Accordingly, institute
State second determine module for determining the memory node that each described subdata file is corresponding, described memory module for will each described in
Subdata file stores to corresponding memory node.
11. data servers according to claim 10, it is characterised in that described judge module specifically for:
Obtain each of storage in the MD5 value of described data file to be stored and described memory node and store data file
MD5 value；
In MD5 value according to described data file to be stored and described memory node, each of storage has stored data file
MD5 value, it is judged that whether there is the storage data file identical with described data file to be stored in memory node.
12. data servers according to claim 10, it is characterised in that described first acquisition module includes acquiring unit
And cutting unit, wherein,
Described acquiring unit is used for, and obtains attribute information and the size of described data file to be stored of described memory node；
Described cutting unit is used for, according to attribute information and the size of described data file to be stored of described memory node, right
Described data file to be stored carries out dividing processing, obtains the plurality of subdata file.
13. data servers according to claim 12, it is characterised in that the attribute information of described memory node includes institute
State the size of data block in memory node；Accordingly, described cutting unit specifically for:
According to the size of described data block, described data file to be stored is carried out dividing processing, obtains N number of subdata file,
Wherein, described N number of subdata file at least exists the size size equal to described data block of N-1 data file.
14. data servers according to claim 12, it is characterised in that the attribute information of described memory node includes institute
State the size of data block in memory node and the number of memory node；Accordingly, described cutting unit specifically for:
Size according to described data block and the size of described data file to be stored, determine in described data file to be stored and wrap
The the first data block number included；
Number according to described memory node and described first data block number, determine the second number that each described memory node is corresponding
According to block number；
The second data block number corresponding according to each described memory node and the size of described data block, to described number to be stored
Carrying out dividing processing according to file, obtain M sub-data file, wherein, described M is equal to the number of described memory node.
15. according to the data server described in any one of claim 10-14, it is characterised in that described data server also wraps
Include the second acquisition module and generation module, wherein,
Described second acquisition module is used for, and stores each described subdata file to corresponding memory node in described memory module
After in, obtain each described subdata file absolute path in corresponding memory node；
Described generation module is used for, and according to each described subdata file absolute path in corresponding memory node, generates institute
State the absolute path of data file to be stored.
16. according to the data server described in any one of claim 10-14, it is characterised in that described data server also wraps
Include and merge module and sending module, wherein,
Described merging module is used for, and in described memory module, each described subdata file is stored to corresponding memory node it
After, the subdata file that store in each described memory node, described data file is corresponding is merged process, is verified
Data file；
Described judge module is additionally operable to, it is judged that the MD5 value of described data file to be stored and the MD5 value of described inspection data file
The most identical；
Described sending module is used for, and judges the MD5 value of described data file to be stored and described check number at described judge module
According to the MD5 value of file identical time, client to described user place sends and uploads successfully corresponding message；
Described memory module is additionally operable to, and judges the MD5 value of described data file to be stored and described inspection at described judge module
When the MD5 value of data file differs, described data file is heavily deposited process.
17. data servers according to claim 16, it is characterised in that described data server also includes the 3rd acquisition
Module, wherein,
Described receiver module is additionally operable to, and receives the data download request that client sends, and described data download request is used for indicating
Data file to be downloaded is downloaded；
Described 3rd acquisition module is used for, and according to the absolute path of described data file to be downloaded, obtains in described memory node
Take multiple subdata files of described data file to be downloaded；
Described merging module is additionally operable to, and multiple subdata files of described data file to be downloaded is merged process, obtains
Described data file to be downloaded；
Described sending module is additionally operable to, and sends described data file to be downloaded to described client.
18. according to the data server described in any one of claim 10-14, it is characterised in that described data server also wraps
Include transferring module, wherein,
Described receiver module is additionally operable to, and receives the Data Migration request that client sends, and the request of described Data Migration includes treating
Migrate mark and the target relative path of data file；
Described transferring module is used for, and asks according to described Data Migration, is updated by the relative path of described data file to be migrated
For described target relative path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610624761.3A CN106294627A (en) | 2016-07-28 | 2016-07-28 | Data managing method and data server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610624761.3A CN106294627A (en) | 2016-07-28 | 2016-07-28 | Data managing method and data server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106294627A true CN106294627A (en) | 2017-01-04 |
Family
ID=57664353
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610624761.3A Pending CN106294627A (en) | 2016-07-28 | 2016-07-28 | Data managing method and data server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106294627A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020093501A1 (en) * | 2018-11-08 | 2020-05-14 | 网宿科技股份有限公司 | File storage method and deletion method, server, and storage medium |
| CN111866169A (en) * | 2018-01-23 | 2020-10-30 | 创新先进技术有限公司 | Service updating method, device and system |
| CN112231619A (en) * | 2020-10-15 | 2021-01-15 | 北京三快在线科技有限公司 | Conversion method, conversion device, electronic equipment and storage medium |
| CN112835529A (en) * | 2021-02-23 | 2021-05-25 | 深圳市得一微电子有限责任公司 | Data storage method and device, electronic equipment and storage medium |
| CN113254402A (en) * | 2021-05-18 | 2021-08-13 | 福州理工学院 | Shared file management method and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110295803A1 (en) * | 2010-05-27 | 2011-12-01 | Fujitsu Limited | Database system, method, and recording medium of program |
| CN103138871A (en) * | 2011-11-23 | 2013-06-05 | 毕书清 | Server data processing system and method of application program of mobile communication system |
| CN103577503A (en) * | 2012-08-10 | 2014-02-12 | 鸿富锦精密工业（深圳）有限公司 | Cloud file storage system and method |
| CN103634407A (en) * | 2013-12-11 | 2014-03-12 | 北京荣之联科技股份有限公司 | Method and device for breakpoint resuming of cloud storage files |
| CN104378422A (en) * | 2014-11-13 | 2015-02-25 | 中国联合网络通信有限公司保定市分公司 | Mobile terminal and file transmission method and system thereof |
| CN104679905A (en) * | 2015-03-20 | 2015-06-03 | 四川特伦特科技股份有限公司 | High-speed storage system based on cloud storage |
| CN104967619A (en) * | 2015-06-17 | 2015-10-07 | 深圳市腾讯计算机系统有限公司 | File pushing method, device and system |
-
2016
- 2016-07-28 CN CN201610624761.3A patent/CN106294627A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110295803A1 (en) * | 2010-05-27 | 2011-12-01 | Fujitsu Limited | Database system, method, and recording medium of program |
| CN103138871A (en) * | 2011-11-23 | 2013-06-05 | 毕书清 | Server data processing system and method of application program of mobile communication system |
| CN103577503A (en) * | 2012-08-10 | 2014-02-12 | 鸿富锦精密工业（深圳）有限公司 | Cloud file storage system and method |
| CN103634407A (en) * | 2013-12-11 | 2014-03-12 | 北京荣之联科技股份有限公司 | Method and device for breakpoint resuming of cloud storage files |
| CN104378422A (en) * | 2014-11-13 | 2015-02-25 | 中国联合网络通信有限公司保定市分公司 | Mobile terminal and file transmission method and system thereof |
| CN104679905A (en) * | 2015-03-20 | 2015-06-03 | 四川特伦特科技股份有限公司 | High-speed storage system based on cloud storage |
| CN104967619A (en) * | 2015-06-17 | 2015-10-07 | 深圳市腾讯计算机系统有限公司 | File pushing method, device and system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111866169A (en) * | 2018-01-23 | 2020-10-30 | 创新先进技术有限公司 | Service updating method, device and system |
| CN111866169B (en) * | 2018-01-23 | 2023-08-18 | 创新先进技术有限公司 | Service updating method, device and system |
| WO2020093501A1 (en) * | 2018-11-08 | 2020-05-14 | 网宿科技股份有限公司 | File storage method and deletion method, server, and storage medium |
| CN112231619A (en) * | 2020-10-15 | 2021-01-15 | 北京三快在线科技有限公司 | Conversion method, conversion device, electronic equipment and storage medium |
| CN112835529A (en) * | 2021-02-23 | 2021-05-25 | 深圳市得一微电子有限责任公司 | Data storage method and device, electronic equipment and storage medium |
| CN113254402A (en) * | 2021-05-18 | 2021-08-13 | 福州理工学院 | Shared file management method and storage medium |
| CN113254402B (en) * | 2021-05-18 | 2022-08-30 | 福州理工学院 | Shared file management method and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106294627A (en) | Data managing method and data server | |
| CN104580104B (en) | The method, apparatus and system of authentication | |
| CN108933805A (en) | A kind of document transmission method and system | |
| CN110995513A (en) | Data sending and receiving method in Internet of things system, Internet of things equipment and platform | |
| KR20120087233A (en) | Method for tile-map service using image fusion, and system for the same | |
| US20180013610A1 (en) | File delivery method, apparatus and system | |
| CN109413202B (en) | System and method for sorting block chain transaction information | |
| CN103634361A (en) | Method and apparatus for downloading file | |
| CN105812456A (en) | File transmission method and device | |
| CN102624916A (en) | Method, node manager and system for load balancing in cloud computing system | |
| CN105408871A (en) | Method and apparatus for providing redundant data access | |
| CN106487768A (en) | A kind of file sharing method and device | |
| CN107395559A (en) | Data processing method and equipment based on redis | |
| CN106991008A (en) | A kind of resource lock management method, relevant device and system | |
| CN110597922A (en) | Data processing method, device, terminal and storage medium | |
| CN106027356B (en) | A kind of conversion method and device of Tunnel Identifier | |
| CN103595808B (en) | A kind of file update information method for pushing and device | |
| CN109962961A (en) | A kind of reorientation method and system of content distribution network CDN service node | |
| CN102857547B (en) | The method and apparatus of distributed caching | |
| CN106209875A (en) | Method for processing business based on multi-services device and service server | |
| CN106899564A (en) | A kind of login method and device | |
| CN108614750A (en) | The restoration methods and device of delay machine data | |
| CN109525633B (en) | Block chain network, message sending method and message receiving method based on block chain network | |
| CN107870940A (en) | A kind of file memory method and device | |
| CN108206839A (en) | One kind is based on majority's date storage method, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170104 |