CN103327052A - Data storage method and system and data visiting method and system - Google Patents
Data storage method and system and data visiting method and system Download PDFInfo
- Publication number
- CN103327052A CN103327052A CN201210079243XA CN201210079243A CN103327052A CN 103327052 A CN103327052 A CN 103327052A CN 201210079243X A CN201210079243X A CN 201210079243XA CN 201210079243 A CN201210079243 A CN 201210079243A CN 103327052 A CN103327052 A CN 103327052A
- Authority
- CN
- China
- Prior art keywords
- key
- value
- storage server
- file
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data storage method and system and a data visiting method and system. The data storage method includes the following steps: obtaining file identifications of files to be stored and all objects under the files; taking the file identifications as keys, taking all the objects under the files as values to constitute key value pairs, and sending the key value pairs to a storage server corresponding to the keys; storing the key value pairs through the storage server. According to the data storage method, the file identifications and all the objects under the files are stored in a key value pair mode. Compared with a traditional transaction model storage method of a relation database, the data storage method is good in availability and partition expansibility, and high in data storage efficiency. Besides, the invention provides the data visiting method, a data storage system and a data visiting system.
Description
[technical field]
The present invention relates to networking technology area, relate to especially a kind of date storage method and system and data access method and system.
[background technology]
The cloud storage, refer to by functions such as cluster application, network technology or distributed file systems, a large amount of various dissimilar memory devices in the network are gathered collaborative work by application software, an as cloud computing system take the data store and management as core can be regarded the cloud storage by the common system that data storage and Operational Visit function externally are provided.
Cloud storage system provides the data access service for the user of magnanimity, therefore has the demand of mass data access, and in addition, the memory device in the cloud storage system is to disperse, so cloud storage system has again the distributed storage demand.Traditional cloud storage system adopts relevant database usually, and the extensibility of relevant database is relatively poor, can not carry out horizontal extension, and along with the increase of data volume, its readwrite performance obviously reduces.Therefore, be necessary to provide a kind of and can satisfy cloud storage data access amount greatly and distributed storage needs and the high data managing method of data access efficiency.
[summary of the invention]
Based on this, be necessary the date storage method that provides a kind of data storage efficiency high.
A kind of date storage method may further comprise the steps:
Obtain the folder identifier of file to be stored and all objects under this document folder;
All objects take described folder identifier as key, under described file form key-value pair as being worth, and key-value pair is sent to storage server corresponding to described key;
Described storage server is stored described value according to described key.
Therein among embodiment, the described step that key-value pair is sent to storage server corresponding to described key comprises:
Described key is converted to the integer number by preset algorithm;
Obtain storage server corresponding to integer number;
Described key-value pair is sent to the storage server of described correspondence.
Among embodiment, the step that described storage server is stored described value according to described key comprises therein:
Store described value by described storage server, obtain the memory address of described value, described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of described value is preserved in position corresponding to this side-play amount in the array of described value address.
Therein among embodiment, described folder identifier comprises the combination of the user folder ID of the user ID of user under the described file and described file.
Therein among embodiment, describedly store described value by described storage server and comprise:
Described value is packaged into packet according to predetermined format, and the form of this packet with data block is stored in the described storage server.
Based on this, also be necessary the data access method that provides a kind of access efficiency high.
A kind of data access method may further comprise the steps:
Obtain data access request, comprise the folder identifier of file to be visited in the described data access request;
Take described folder identifier as key, described key is sent to storage server corresponding to described key;
Obtain value corresponding to described key by described storage server, described value has been stored all objects under the described file;
Receive value corresponding to described key that described storage server returns, the value that described key is corresponding is reduced to the object under the described file.
Therein among embodiment, the described step that described key is sent to storage server corresponding to described key comprises:
Described key is converted to the integer number by preset algorithm;
Obtain storage server corresponding to integer number;
Described key is sent to the storage server of described correspondence.
Therein among embodiment, the described step of obtaining value corresponding to described key by described storage server comprises:
By described storage server described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of the value that the described key of position acquisition corresponding to this side-play amount is corresponding in the array of described value address is obtained described value according to described memory address.
Therein among embodiment, described folder identifier comprises the combination of the user folder ID of the user ID of user under the described file and described file.
Therein among embodiment, describedly obtain described value according to described memory address and comprise:
Obtain corresponding packet according to described memory address, described packet is decoded according to default form obtains described value.
Based on this, also be necessary the data-storage system that provides a kind of storage efficiency high.
A kind of data-storage system comprises at least one cloud file system and a plurality of storage server, wherein:
Described cloud file system is used for obtaining the folder identifier of file to be stored and all objects under this document folder, all objects take described folder identifier as key, under described file form key-value pair as being worth, and key-value pair is sent to storage server corresponding to described key;
Described storage server is used for receiving the key-value pair that described cloud file system sends, and stores described value according to described key.
Therein among embodiment, described cloud file system also is used for when key-value pair being sent to storage server corresponding to described key, described key is converted to the integer number by preset algorithm, obtain storage server corresponding to integer number, described key-value pair is sent to the storage server of described correspondence.
Therein among embodiment, when described storage server is used for receiving described key-value pair, store described value, obtain the memory address of described value, described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of described value is preserved in position corresponding to this side-play amount in the array of described value address.
Therein among embodiment, described folder identifier comprises the user ID in the cloud file system of user under the described file and the combination of the user folder ID of described file.
Based on this, also be necessary the data access system that provides a kind of access efficiency high.
A kind of data access system comprises at least one cloud file system and a plurality of storage server;
Described cloud file system is used for obtaining data access request, comprises the folder identifier of file to be visited in the described data access request;
Described cloud file system also is used for take described folder identifier as key, and described key is sent to storage server corresponding to described key;
Described storage server is used for receiving the key that described cloud file system sends, and obtains value corresponding to described key, and the value that described key is corresponding returns to
Cloud file system, described value have been stored all objects under the described file;
Described cloud file system also is used for the value that described key is corresponding and is reduced to object under the described file.
Therein among embodiment, described cloud file system also is used for when described key being sent to storage server corresponding to described key, described key is converted to the integer number by preset algorithm, obtains storage server corresponding to integer number, described key is sent to corresponding storage server.
Therein among embodiment, described storage server also is used for when obtaining value corresponding to described key, described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of the value that the described key of position acquisition corresponding to this side-play amount is corresponding in the array of described value address is obtained described value according to described memory address.
Therein among embodiment, described folder identifier comprises the combination of the user folder ID of the user ID in the cloud file system of user under the described file and described file.
Above-mentioned date storage method and system, with file as storage cell, take folder identifier as key, the value corresponding according to the key storage key, all objects under the storage folder in this value.The storage mode of above-mentioned key-value pair is than the transaction model storage mode of traditional relevant database, more extensive more high performance storage can be provided, and on availability and subregion autgmentability than more excellent, be suitable for mass memory demand and the distributed storage demand of cloud storage, improved data storage efficiency high.
Above-mentioned data access method and system during access file folder data, take folder identifier as key, are distributed to key first storage server corresponding to key, again by the storage server value of all objects under the file of having obtained storage corresponding to key.Aforesaid way not only is suitable for accessing the data of storing in the cloud distributed storage environment, and based on the data access mode of the key-value pair data access mode than traditional relevant database, quicker, the magnanimity requirements for access that is suitable for internet, applications, improved data access efficiency, for the user, access cloud storage file sensation is as good as with local IP access, so that user habit transition nature.
[description of drawings]
Fig. 1 is the schematic flow sheet of a date storage method among the embodiment;
Fig. 2 is the schematic flow sheet that key-value pair is sent to storage server corresponding to key among the embodiment;
Fig. 3 is the schematic flow sheet of a data access method among the embodiment;
Fig. 4 is the structural representation of a data-storage system among the embodiment;
Fig. 5 is the structural representation of a data access system among the embodiment;
Fig. 6 be one among the embodiment date storage method and the schematic diagram of data access method.
[embodiment]
As shown in Figure 1, a kind of date storage method may further comprise the steps:
Step S101 obtains the folder identifier of file to be stored and all objects under this document folder.
In one embodiment, folder identifier is the combination of the user folder ID of the user ID of user under the file and file.Preferably, user ID can be the account that the user logins the cloud file system, also can be that the cloud file system is the account of user assignment.
For a user, user's file and user folder ID are one to one.The user folder ID of this document folder can sequentially be set according to the establishment of user's file in one embodiment.For example, the file ID that the file of first establishment in user's the file can be set is that the file ID of the file of 1, the second establishment is 2, and file ID increases progressively along with the growth of number of folders.Each file has corresponding user folder ID, if the path of object is :/root/subdirectory/object, then the file ID at object place is the user folder ID of " subdirectory " this file.
The user that cloud is stored as magnanimity provides the data access service, and each user has user ID to be used for access data.The user ID of user under the file and the user folder ID of file are combined into folder identifier, but the different file of folder identifier unique identification mass users then.For different users, user's file is corresponding one by one with folder identifier.
In the present embodiment, folder identifier can be used for distinguishing the file between the different user, all users' file is a corresponding unique folder identifier all, file ID then can only be used for distinguishing same user's different file, some users' different file, its file ID are also different.
Above-mentioned folder identifier compound mode is applicable to data all objects take folder identifier as key, under file of user are stored as value is combined into key-value pair, and makes things convenient for the object of user under pressing from both sides take folder identifier as the key access file.
Object under the file includes but not limited to sub-folder and file.File includes but not limited to application program, mail, picture, contact person's business card etc. all kinds.
In one embodiment, step S101 can obtain the data entity of the file in the file to be stored, and/or, the quoting of file, and/or, the attribute information of sub-folder.
The data entity of file is the data of expression file attribute information (title, size, position, creation-time etc.) and content information, for example represents the string of binary characters of file attribute information and content information.The concrete form of quoting of file can be the chained address of file on Internet, or the link information of file in third party service provider's memory cell, may be read into the data of file according to this link information.The attribute information of sub-folder comprises title, size, position, creation-time of sub-folder etc.
Step S102, all objects take folder identifier as key, under file form key-value pair as being worth, and key-value pair is sent to storage server corresponding to key.
Key is called again key, and the corresponding value of key is called again value, and the key-value pair that key and value form can be designated as key-value.
Concrete, in one embodiment, can with the data entity of the file in the file to be stored or file quote and the attribute information of sub-folder as value, form key-value with folder identifier, the key-value that forms is packaged into packet according to default form, with Packet Generation to storage server corresponding to key.
Concrete, storage server is with the object in the form storage cloud file system of key-value.Among the key-value of storage server storage, the file in the corresponding cloud file system of the key user interface, all objects under the corresponding this document folder of value.Key-value in the storage of file in the user interface and cloud is corresponding one by one.
In one embodiment, can come object under storage folder and the file according to false code form as shown in table 1.
Table 1
In the table 1, key is file ID, items[] data that comprise of part are all objects under the file.For example, the file on the root " root ", its file ID is 1, it comprises two sub-folders " sub_folder_1 " and " sub_folder_2 ".The file ID of " sub_folder_1 " and " sub_folder_2 " is respectively 2 and 3.The lower sub-folder " my_photo " that comprises again of file " sub_folder_1 ".
In the present embodiment, take file as memory cell, during storage folder, only need storage to be located immediately at the relevant information of the object under the file, and do not need object under the sub-folder of storage folder, when having object to be modified or to have new object to be created, the file at object place can be stored as memory cell, and the sub-folder that does not need memory contents not to be modified, thereby can improve storage efficiency.
As shown in Figure 2, in one embodiment, the detailed process that key-value pair is sent to storage server corresponding to described key is:
Step S201 is converted to the integer number with key by preset algorithm.
Concrete, key can be converted to the integer number by specific hash digest algorithm.
Step S202 obtains storage server corresponding to integer number.
In one embodiment, can preserve the mapping table of integer number and storage server, in mapping table, search storage server corresponding to integer number according to the integer number.
In another embodiment, the integer number that the key of the value of storage server storage is corresponding is adjacent.For example, the integer number that key is corresponding is that 0～999 value is stored in the same storage server.Different storage servers is stored the value of key corresponding to the integer number of different segments.In the present embodiment, can find corresponding storage server according to segment corresponding to integer number.
In the present embodiment, according to key is converted to the integer number by preset algorithm, and obtain segment under the integer number, find corresponding storage server according to segment, can navigate to fast target storage server, improve the efficient of data storage in the cloud distributed storage environment.
In another embodiment, the value of different storage servers storage equal numbers, and integer number corresponding to the key of the value of same storage server storage is adjacent.Then storage server can be used the natural number number consecutively since 0, calculate the integer quotient that the integer number obtains divided by the storable value quantity of storage server, the storage server that integer quotient is corresponding is as target storage server.For example, 1000 value of every station server storage, establishing the integer number is 2330, and then obtaining the merchant with 2330 divided by 1000 is 2, and the storage server that obtains 2 correspondences is target storage server.
In the present embodiment, can obtain target storage server corresponding to integer number through simple calculating, improve the efficient of data storage in the cloud distributed storage environment.
Step S203 is sent to corresponding storage server with key-value pair.
Step S103 stores described value by storage server according to described key.
In one embodiment, can store by storage server the value of key-value pair, the memory address of the value of obtaining, key is converted to the integer number by preset algorithm, obtain the side-play amount of integer number in the array of the value address of this storage server, the memory address of position save value corresponding to this side-play amount in value address array.
Concrete, in one embodiment, the packet of value for forming according to default form packing.The cloud accumulation layer is stored in the form of packet with data block in the database.Preferably, the cloud accumulation layer is stored in value not in the non-relational database of fixed data structure (being nosql type database).The value that packs in the packet can be data entity, for example, the string of binary characters of expression value, the value that packs in the packet also can be quoting of data, for example, the chained address of data on Internet, or the link information of data in third party service provider's memory cell may be read into data according to this link information.
In one embodiment, save value address array in storage server in advance, the length of value address array equals the quantity of the storable value of storage server.After obtaining the memory address of value, can pass through specific hash digest algorithm, key is converted to the integer number.Further, in one embodiment, the length of integer digital-to-analogue except value address array can be obtained the side-play amount of integer number correspondence in the array of value address, namely be worth the subscript of address array.And the memory address of storing value according to the subscript of the value address array of obtaining.
For example, storage server can be stored 1000 value, and the length of array is 1000, and the integer number that obtains is 2330, then calculates 2330%1000=330, and namely side-play amount is 330.First array element in the side-play amount 0 corresponding array.Side-play amount is 330, but then the 331st of settings address array the array element is the memory address of value.
In the present embodiment, obtain corresponding storage server according to key, after value corresponding to the storage server storage key of correspondence, obtain the subscript of corresponding value address array according to key, and with position corresponding to subscript in the memory address value of being kept at address array of value.Aforesaid way can file be storage cell, fast all objects under the file is stored in cloud distributed storage environment, and can not stored the rising of memory data output and performance because of cloud, and storage efficiency is high.
Above-mentioned date storage method, with file as storage cell, take folder identifier as key, the value corresponding according to the key storage key, the object in this value under the storage folder.The storage mode of above-mentioned key-value pair is than the transaction model storage mode of traditional relevant database, more extensive more high performance storage can be provided, and on availability and subregion autgmentability than more excellent, be suitable for mass memory demand and the distributed storage demand of cloud storage, improved data storage efficiency.
As shown in Figure 3, in one embodiment, a kind of data access method may further comprise the steps:
Step S301 obtains data access request, comprises the folder identifier of file to be visited in the data access request.
In one embodiment, when the user logins, can trigger the request of the root file of calling party.When the user carries out the operation of opened file folder in user interface, can trigger the request of access this document folder.
In one embodiment, folder identifier is the combination of the user folder ID of the user ID of user under the file and file.For a user, user's file and user folder ID are one to one.For different users, user's file is corresponding one by one with folder identifier.
Above-mentioned folder identifier compound mode, make things convenient for the user in cloud storage take folder identifier as the key access file object under the folder.
Step S302 take folder identifier as key, sends to storage server corresponding to key with key.
In one embodiment, the detailed process that key is sent to the step of storage server corresponding to key is: key is converted to the integer number by preset algorithm, obtains storage server corresponding to integer number, key is sent to corresponding storage server.
Concrete, key can be converted to the integer number by specific hash digest algorithm.
In one embodiment, the integer number that the key of the value of storage server storage is corresponding is adjacent.Different storage servers is stored the value of key corresponding to the integer number of different segments.In the present embodiment, can find corresponding storage server according to segment corresponding to integer number.
In the present embodiment, obtain the affiliated segment of integer number, find corresponding storage server according to segment, can navigate to fast target storage server, improved the efficient of data access in the cloud distributed storage environment.
In another embodiment, the value of different storage servers storage equal numbers, and integer number corresponding to the key of the value of same storage server storage is adjacent.Then storage server can be used the natural number number consecutively since 0, calculate the integer quotient that the integer number obtains divided by the storable value quantity of storage server, the storage server that integer quotient is corresponding is as target storage server.
In the present embodiment, can obtain target storage server corresponding to integer number through simple calculating, improve the efficient of data access in the cloud distributed storage environment.
Step S303 obtains value corresponding to key by storage server, and this value has been stored all objects under the file.
In one embodiment, the object under the file comprises sub-folder and file.File comprises application program, mail, picture, contact person's business card etc. all kinds.In one embodiment, above-mentioned value has been stored the attribute information of quoting of the data entity of the file in the file or file and sub-folder.
In the present embodiment, access file when folder, only need to obtain the data that are located immediately at the lower object of this document folder, and not need to obtain the data of the object under the sub-folder that this document presss from both sides, thereby can improve access efficiency.
Concrete, in one embodiment, can key be converted to the integer number by preset algorithm by storage server, obtain the side-play amount of integer number in the array of the value address of this storage server, the memory address of the value that position acquisition key corresponding to this side-play amount is corresponding in value address array is obtained value corresponding to key according to memory address.
In the present embodiment, by key being converted to the integer number, according to the memory address of integer number side-play amount value of obtaining in the array of value address, obtain value corresponding to key according to the memory address of value again, the efficient of this mode reading out data is very high.
Step S304 receives value corresponding to key that storage server returns, and the value that key is corresponding is reduced to the object under the file to be visited.
In one embodiment, the value that the key that storage server returns is corresponding is generally packet, packet can be decoded according to default form, further decoded data are reduced into application program in the user interface, mail, text, associated person information etc.If stored quoting of file in the above-mentioned value, then step S304 can obtain file data according to quoting of the file of storing in the above-mentioned value.
The user accesses this data, comprises object is created, rename, the operation that is similar to native object such as checks, moves, deletes, opens.
Above-mentioned data access method during access file folder data, take folder identifier as key, is distributed to key first storage server corresponding to key, again by the storage server value of the object under the file of having obtained storage corresponding to key.Aforesaid way not only is suitable for accessing the data of storing in the cloud distributed storage environment, and based on the data access mode of the key-value pair data access mode than traditional relevant database, quicker, be suitable for the magnanimity requirements for access of internet, applications, improved data access efficiency.
As shown in Figure 4, a kind of data-storage system comprises at least one cloud file system 401 and a plurality of storage server 402, wherein:
In one embodiment, folder identifier is the combination of the user folder ID of the user ID of user under the file and file.For a user, user's file and user folder ID are one to one.For different users, user's file is corresponding one by one with folder identifier.
Above-mentioned folder identifier compound mode makes things convenient for the object of user under pressing from both sides take folder identifier as the key access file.
In one embodiment, cloud file system 401 can be obtained the attribute information of quoting of the data entity of the file in the file to be stored or file and sub-folder.
In one embodiment, cloud file system 401 can with the data entity of the file in the file to be stored or file quote and the attribute information of sub-folder as value, form key-value with folder identifier, the key-value that forms is packaged into packet according to default form, with Packet Generation to storage server 402 corresponding to key.
In the present embodiment, take file as memory cell, during storage folder, only need storage to be located immediately at the relevant information of the object under the file, and do not need object under the sub-folder of storage folder, when having object to be modified or to have new object to be created, the file at object place can be stored as memory cell, and the sub-folder that does not need memory contents not to be modified, thereby can improve storage efficiency.
Concrete, in one embodiment, when cloud file system 401 sends to storage server corresponding to key with key-value pair, key can be converted to the integer number by preset algorithm, obtain storage server corresponding to integer number, key-value pair is sent to corresponding storage server.
In the present embodiment, key is converted to the integer number by preset algorithm, obtains storage server corresponding to integer number as target storage server, can navigate to fast target storage server, improved the efficient of data storages.
In one embodiment, storage server 402 can be stored the value of key-value pair, the memory address of the value of obtaining, key is converted to the integer number by preset algorithm, obtain the side-play amount of integer number in the array of the value address of this storage server, the memory address of position save value corresponding to this side-play amount in value address array.
In the present embodiment, cloud file system 401 is obtained corresponding storage server 402 according to key, after value corresponding to corresponding storage server 402 storage keys, obtain the subscript of corresponding value address array according to key, and with position corresponding to subscript in the memory address value of being kept at address array of value.Aforesaid way can file be storage cell, fast all objects under the file are stored in distributed environment, and can be because of the rising of memory data output performance, storage efficiency is high.
Above-mentioned data-storage system, with file as storage cell, take folder identifier as key, the value corresponding according to the key storage key, the object in this value under the storage folder.The storage mode of above-mentioned key-value pair is than the transaction model storage mode of traditional relevant database, more extensive more high performance storage can be provided, and on availability and subregion autgmentability than more excellent, be suitable for mass memory demand and the distributed storage demand of cloud storage, improved data storage efficiency.
As shown in Figure 5, in one embodiment, a kind of data access system comprises at least one cloud file system 501 and a plurality of storage server 502, wherein:
In one embodiment, cloud file system 501 can provide application interface for user's login to the user, when the user logins, can trigger the request of the root file of calling party.When the user carries out the operation of opened file folder in user interface, can trigger the request of access this document folder.
In one embodiment, folder identifier is the combination of the user folder ID of the user ID of user under the file and file.For a user, user's file and user folder ID are one to one.For different users, user's file is corresponding one by one with folder identifier.
Above-mentioned folder identifier compound mode, make things convenient for the user in cloud storage take folder identifier as the key access file object under the folder.
In one embodiment, cloud file system 501 can be converted to the integer number by preset algorithm with key, obtains storage server corresponding to integer number, and key is sent to corresponding storage server.
Concrete, cloud file system 501 can be converted to the integer number by specific hash digest algorithm with key.
In one embodiment, the integer number that the key of the value of storage server 502 storages is corresponding is adjacent.The value of the key that the integer number of the different different segments of storage server 502 storages is corresponding.In the present embodiment, Distributor 502 can find corresponding storage server 502 according to segment corresponding to integer number.
In the present embodiment, obtain the affiliated segment of integer number, find corresponding storage server according to segment, can navigate to fast target storage server, improved the efficient of data access.
In another embodiment, the value of different storage servers 502 storage equal numbers, and integer number corresponding to the key of the value of same storage server storage is adjacent.Cloud file system 501 can be calculated the integer quotient that the integer number obtains divided by the storable value quantity of storage server with the natural number number consecutively of storage server 502 usefulness since 0, and the storage server 502 that integer quotient is corresponding is as target storage server.
In the present embodiment, can obtain target storage server corresponding to integer number through simple calculating, improve the efficient of data storage in the cloud distributed storage environment.
In one embodiment, the object under the file comprises sub-folder and file.In one embodiment, above-mentioned value has been stored the attribute information of quoting of the data entity of the file in the file or file and sub-folder.
In the present embodiment, access file when folder, only need to obtain the data that are located immediately at the lower object of this document folder, and not need to obtain the data of the object under the sub-folder that this document presss from both sides, thereby can improve access efficiency.
Concrete, in one embodiment, storage server 502 can be converted to the integer number by preset algorithm with key, obtain the side-play amount of integer number in the array of the value address of this storage server, the memory address of the value that position acquisition key corresponding to this side-play amount is corresponding in value address array is obtained value corresponding to key according to memory address.
In the present embodiment, by key being converted to the integer number, according to the memory address of integer number side-play amount value of obtaining in the array of value address, obtain value corresponding to key according to the memory address of value again, the efficient of this mode reading out data is very high.
Value corresponding to key that cloud file system 501 is used for that also storage server 502 is returned is reduced to the object under the file to be visited.
In one embodiment, the value that the key that storage server 502 returns is corresponding is generally packet, cloud file system 501 can be decoded packet according to default form, further decoded data are reduced into application program in the user interface, mail, text, associated person information etc.If stored quoting of file in the above-mentioned value, cloud file system 501 can be obtained file data according to quoting of the file of storing in the above-mentioned value.
Above-mentioned data access system during access file folder data, take folder identifier as key, is distributed to key first storage server corresponding to key, again by the storage server value of the object under the file of having obtained storage corresponding to key.Aforesaid way not only is suitable for accessing the data of storing in the cloud distributed storage environment, and based on the data access mode of the key-value pair data access mode than traditional relevant database, quicker, be suitable for the magnanimity requirements for access of internet, applications, improved data access efficiency.
The principle of date storage method and data access method is described with concrete example below in conjunction with Fig. 6.
In one embodiment, the detailed process of storage data is as follows:
(1) the cloud file system is obtained the folder identifier of file to be stored and all objects under this document folder, and all objects take folder identifier as key, under file form key-value pair as being worth, and key-value pair is sent to storage server corresponding to key.
Object under the file comprises various types of files such as sub-folder and application program, mail, text, associated person information.
(2) storage server is used for receiving the key-value pair that the cloud file system sends, and stores described value according to described key.
The form that storage server will be worth with data block is stored in the database, and is preferred, with data block store in the non-relational database of fixed data structure (being nosql type database) not.
In one embodiment, the detailed process of visit data is as follows:
(1) the cloud file system is used for obtaining data access request, comprises the folder identifier of file to be visited in the data access request, and wherein folder identifier is the combination of user ID and user folder ID.
(2) the cloud file system sends to storage server corresponding to key take folder identifier as key with key.
(3) storage server receives the key that the cloud file system sends, and obtains value corresponding to key according to key, and the value that key is corresponding returns to the cloud file system, and this value has been stored all objects under the file to be visited.
The data that are accessed frequently can be stored in the cache assembly.Storage server obtains value corresponding to key from database or cache assembly.
(4) the cloud file system value that key is corresponding is reduced into the object under the file.Object under the file comprises various types of files and the sub-folders such as application program, mail, text, associated person information.
The above embodiment has only expressed several execution mode of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to claim of the present invention.Should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection range of patent of the present invention should be as the criterion with claims.
Claims (18)
1. date storage method may further comprise the steps:
Obtain the folder identifier of file to be stored and all objects under this document folder;
All objects take described folder identifier as key, under described file form key-value pair as being worth, and key-value pair is sent to storage server corresponding to described key;
Described storage server is stored described value according to described key.
2. date storage method according to claim 1 is characterized in that, the described step that key-value pair is sent to storage server corresponding to described key comprises:
Described key is converted to the integer number by preset algorithm;
Obtain storage server corresponding to integer number;
Described key-value pair is sent to the storage server of described correspondence.
3. date storage method according to claim 1 and 2 is characterized in that, the step that described storage server is stored described value according to described key comprises:
Store described value by described storage server, obtain the memory address of described value, described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of described value is preserved in position corresponding to this side-play amount in the array of described value address.
4. date storage method according to claim 1 is characterized in that, described folder identifier comprises the combination of the user folder ID of the user ID of user under the described file and described file.
5. date storage method according to claim 3 is characterized in that, describedly stores described value by described storage server and comprises:
Described value is packaged into packet according to predetermined format, and the form of this packet with data block is stored in the described storage server.
6. data access method may further comprise the steps:
Obtain data access request, comprise the folder identifier of file to be visited in the described data access request;
Take described folder identifier as key, described key is sent to storage server corresponding to described key;
Obtain value corresponding to described key by described storage server, described value has been stored all objects under the described file;
Receive value corresponding to described key that described storage server returns, the value that described key is corresponding is reduced to the object under the described file.
7. data access method according to claim 6 is characterized in that, the described step that described key is sent to storage server corresponding to described key comprises:
Described key is converted to the integer number by preset algorithm;
Obtain storage server corresponding to integer number;
Described key is sent to the storage server of described correspondence.
8. according to claim 6 or 7 described data access methods, it is characterized in that, the described step of obtaining value corresponding to described key by described storage server comprises:
By described storage server described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of the value that the described key of position acquisition corresponding to this side-play amount is corresponding in the array of described value address is obtained described value according to described memory address.
9. data access method according to claim 6 is characterized in that, described folder identifier comprises the combination of the user folder ID of the user ID of user under the described file and described file.
10. data access method according to claim 8 is characterized in that, describedly obtains described value according to described memory address and comprises:
Obtain corresponding packet according to described memory address, described packet is decoded according to default form obtains described value.
11. a data-storage system is characterized in that, comprises at least one cloud file system and a plurality of storage server, wherein:
Described cloud file system is used for obtaining the folder identifier of file to be stored and all objects under this document folder, all objects take described folder identifier as key, under described file form key-value pair as being worth, and key-value pair is sent to storage server corresponding to described key;
Described storage server is used for receiving the key-value pair that described cloud file system sends, and stores described value according to described key.
12. data-storage system according to claim 11, it is characterized in that, described cloud file system also is used for when key-value pair being sent to storage server corresponding to described key, described key is converted to the integer number by preset algorithm, obtain storage server corresponding to integer number, described key-value pair is sent to the storage server of described correspondence.
13. according to claim 11 or 12 described data-storage systems, it is characterized in that, when described storage server is used for receiving described key-value pair, store described value, obtain the memory address of described value, described key is converted to the integer number by preset algorithm, obtains the side-play amount of described integer number in the array of the value address of this storage server, the memory address of described value is preserved in position corresponding to this side-play amount in the array of described value address.
14. data-storage system according to claim 11 is characterized in that, described folder identifier comprises the user ID in the cloud file system of user under the described file and the combination of the user folder ID of described file.
15. a data access system is characterized in that, comprises at least one cloud file system and a plurality of storage server;
Described cloud file system is used for obtaining data access request, comprises the folder identifier of file to be visited in the described data access request;
Described cloud file system also is used for take described folder identifier as key, and described key is sent to storage server corresponding to described key;
Described storage server is used for receiving the key that described cloud file system sends, and obtains value corresponding to described key, and the value that described key is corresponding returns to the cloud file system, and described value has been stored all objects under the described file;
Described cloud file system also is used for the value that described key is corresponding and is reduced to object under the described file.
16. data access system according to claim 15, it is characterized in that, described cloud file system also is used for when described key being sent to storage server corresponding to described key, described key is converted to the integer number by preset algorithm, obtain storage server corresponding to integer number, described key is sent to corresponding storage server.
17. according to claim 15 or 16 described data access systems, it is characterized in that, described storage server also is used for when obtaining value corresponding to described key, described key is converted to the integer number by preset algorithm, obtain the side-play amount of described integer number in the array of the value address of this storage server, the memory address of the value that the described key of position acquisition corresponding to this side-play amount is corresponding in the array of described value address is obtained described value according to described memory address.
18. data access system according to claim 15 is characterized in that, described folder identifier comprises the combination of the user folder ID of the user ID in the cloud file system of user under the described file and described file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210079243.XA CN103327052B (en) | 2012-03-22 | 2012-03-22 | Date storage method and system and data access method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210079243.XA CN103327052B (en) | 2012-03-22 | 2012-03-22 | Date storage method and system and data access method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103327052A true CN103327052A (en) | 2013-09-25 |
| CN103327052B CN103327052B (en) | 2018-04-03 |
Family
ID=49195582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210079243.XA Active CN103327052B (en) | 2012-03-22 | 2012-03-22 | Date storage method and system and data access method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103327052B (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104077374A (en) * | 2014-06-24 | 2014-10-01 | 华为技术有限公司 | Method and device for achieving internet protocol (IP) disk file storage |
| CN104184822A (en) * | 2014-08-29 | 2014-12-03 | 海视云（北京）科技有限公司 | Data synchronism method and system |
| CN104537076A (en) * | 2014-12-31 | 2015-04-22 | 北京奇艺世纪科技有限公司 | File reading and writing method and device |
| CN104537023A (en) * | 2014-12-19 | 2015-04-22 | 华为技术有限公司 | Storage method and device for reverse index records |
| CN104951540A (en) * | 2015-06-19 | 2015-09-30 | 阿里巴巴集团控股有限公司 | File processing methods and device |
| WO2015154625A1 (en) * | 2014-09-05 | 2015-10-15 | 中兴通讯股份有限公司 | Timing event processing method, storage method, execution method and corresponding device |
| WO2015180650A1 (en) * | 2014-05-29 | 2015-12-03 | 华为技术有限公司 | Method and device for generating key value in storage area network |
| CN105930104A (en) * | 2016-05-17 | 2016-09-07 | 百度在线网络技术（北京）有限公司 | Data storing method and device |
| CN106202848A (en) * | 2015-05-05 | 2016-12-07 | 北大方正集团有限公司 | The acquisition methods of medical image file, user terminal and master server |
| CN106649451A (en) * | 2016-09-22 | 2017-05-10 | 北京奇虎科技有限公司 | Data update method and device |
| CN107301177A (en) * | 2016-04-14 | 2017-10-27 | 杭州海康威视数字技术股份有限公司 | A kind of file memory method and device |
| CN107301183A (en) * | 2016-04-14 | 2017-10-27 | 杭州海康威视数字技术股份有限公司 | A kind of file memory method and device |
| CN108460054A (en) * | 2017-02-22 | 2018-08-28 | 北京京东尚科信息技术有限公司 | A kind of mthods, systems and devices improving cloud storage system performance |
| CN108846129A (en) * | 2018-07-02 | 2018-11-20 | 腾讯科技（深圳）有限公司 | Storing data access method, device and storage medium |
| CN109063051A (en) * | 2018-07-19 | 2018-12-21 | 佛山科学技术学院 | A kind of storage method of industry big data |
| CN109164977A (en) * | 2018-07-23 | 2019-01-08 | 中国建设银行股份有限公司 | A kind of data-storage system and method, storage medium |
| CN109474827A (en) * | 2018-12-03 | 2019-03-15 | 四川巧夺天工信息安全智能设备有限公司 | The method of monitor video fast transcoding |
| CN112035402A (en) * | 2019-06-04 | 2020-12-04 | 顺丰科技有限公司 | File storage method and device and terminal equipment |
| CN112437078A (en) * | 2020-11-20 | 2021-03-02 | 腾讯科技（深圳）有限公司 | File storage method, device, equipment and computer readable storage medium |
| CN112637327A (en) * | 2020-12-21 | 2021-04-09 | 北京奇艺世纪科技有限公司 | Data processing method, device and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090177721A1 (en) * | 2008-01-09 | 2009-07-09 | Yasuyuki Mimatsu | Management of quality of services in storage systems |
| CN101496005A (en) * | 2005-12-29 | 2009-07-29 | 亚马逊科技公司 | Distributed replica storage system with web services interface |
| CN101504670A (en) * | 2009-03-04 | 2009-08-12 | 成都市华为赛门铁克科技有限公司 | Data operation method, system, client terminal and data server |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102298641A (en) * | 2011-09-14 | 2011-12-28 | 清华大学 | Method for uniformly storing files and structured data based on key value bank |
-
2012
- 2012-03-22 CN CN201210079243.XA patent/CN103327052B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101496005A (en) * | 2005-12-29 | 2009-07-29 | 亚马逊科技公司 | Distributed replica storage system with web services interface |
| US20090177721A1 (en) * | 2008-01-09 | 2009-07-09 | Yasuyuki Mimatsu | Management of quality of services in storage systems |
| CN101504670A (en) * | 2009-03-04 | 2009-08-12 | 成都市华为赛门铁克科技有限公司 | Data operation method, system, client terminal and data server |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN102298641A (en) * | 2011-09-14 | 2011-12-28 | 清华大学 | Method for uniformly storing files and structured data based on key value bank |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180650A1 (en) * | 2014-05-29 | 2015-12-03 | 华为技术有限公司 | Method and device for generating key value in storage area network |
| US10437849B2 (en) | 2014-06-24 | 2019-10-08 | Huawei Technologies Co., Ltd. | Method and apparatus for implementing storage of file in IP disk |
| CN104077374A (en) * | 2014-06-24 | 2014-10-01 | 华为技术有限公司 | Method and device for achieving internet protocol (IP) disk file storage |
| WO2015196899A1 (en) * | 2014-06-24 | 2015-12-30 | 华为技术有限公司 | Method and device for realizing ip disk file storage |
| CN104184822A (en) * | 2014-08-29 | 2014-12-03 | 海视云（北京）科技有限公司 | Data synchronism method and system |
| WO2015154625A1 (en) * | 2014-09-05 | 2015-10-15 | 中兴通讯股份有限公司 | Timing event processing method, storage method, execution method and corresponding device |
| CN104537023B (en) * | 2014-12-19 | 2018-07-31 | 华为技术有限公司 | A kind of storage method and device of reverse indexing record |
| CN104537023A (en) * | 2014-12-19 | 2015-04-22 | 华为技术有限公司 | Storage method and device for reverse index records |
| CN104537076A (en) * | 2014-12-31 | 2015-04-22 | 北京奇艺世纪科技有限公司 | File reading and writing method and device |
| CN104537076B (en) * | 2014-12-31 | 2017-12-08 | 北京奇艺世纪科技有限公司 | A kind of file read/write method and device |
| CN106202848A (en) * | 2015-05-05 | 2016-12-07 | 北大方正集团有限公司 | The acquisition methods of medical image file, user terminal and master server |
| CN104951540A (en) * | 2015-06-19 | 2015-09-30 | 阿里巴巴集团控股有限公司 | File processing methods and device |
| CN107301177A (en) * | 2016-04-14 | 2017-10-27 | 杭州海康威视数字技术股份有限公司 | A kind of file memory method and device |
| CN107301183A (en) * | 2016-04-14 | 2017-10-27 | 杭州海康威视数字技术股份有限公司 | A kind of file memory method and device |
| CN107301183B (en) * | 2016-04-14 | 2020-02-18 | 杭州海康威视数字技术股份有限公司 | File storage method and device |
| CN105930104A (en) * | 2016-05-17 | 2016-09-07 | 百度在线网络技术（北京）有限公司 | Data storing method and device |
| CN106649451A (en) * | 2016-09-22 | 2017-05-10 | 北京奇虎科技有限公司 | Data update method and device |
| CN108460054A (en) * | 2017-02-22 | 2018-08-28 | 北京京东尚科信息技术有限公司 | A kind of mthods, systems and devices improving cloud storage system performance |
| CN108846129A (en) * | 2018-07-02 | 2018-11-20 | 腾讯科技（深圳）有限公司 | Storing data access method, device and storage medium |
| CN108846129B (en) * | 2018-07-02 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Storage data access method, device and storage medium |
| CN109063051A (en) * | 2018-07-19 | 2018-12-21 | 佛山科学技术学院 | A kind of storage method of industry big data |
| CN109164977B (en) * | 2018-07-23 | 2022-01-11 | 中国建设银行股份有限公司 | Data storage system and method, and storage medium |
| CN109164977A (en) * | 2018-07-23 | 2019-01-08 | 中国建设银行股份有限公司 | A kind of data-storage system and method, storage medium |
| CN109474827A (en) * | 2018-12-03 | 2019-03-15 | 四川巧夺天工信息安全智能设备有限公司 | The method of monitor video fast transcoding |
| CN109474827B (en) * | 2018-12-03 | 2022-11-29 | 四川巧夺天工信息安全智能设备有限公司 | Method for rapidly transcoding monitoring video |
| CN112035402A (en) * | 2019-06-04 | 2020-12-04 | 顺丰科技有限公司 | File storage method and device and terminal equipment |
| CN112437078A (en) * | 2020-11-20 | 2021-03-02 | 腾讯科技（深圳）有限公司 | File storage method, device, equipment and computer readable storage medium |
| CN112637327A (en) * | 2020-12-21 | 2021-04-09 | 北京奇艺世纪科技有限公司 | Data processing method, device and system |
| CN112637327B (en) * | 2020-12-21 | 2022-07-22 | 北京奇艺世纪科技有限公司 | Data processing method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103327052B (en) | 2018-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103327052A (en) | Data storage method and system and data visiting method and system | |
| CN103141060B (en) | Content router forwarding unit | |
| US9268716B2 (en) | Writing data from hadoop to off grid storage | |
| US20140181116A1 (en) | Method and device of cloud storage | |
| CN104133882A (en) | HDFS (Hadoop Distributed File System)-based old file processing method | |
| CN103577123A (en) | Small file optimization storage method based on HDFS | |
| CN108717457B (en) | Electronic commerce platform big data processing method and system | |
| CN104408111A (en) | Method and device for deleting duplicate data | |
| CN104679665A (en) | Method and system for achieving block storage of distributed file system | |
| CN103761102B (en) | A kind of uniform data service platform and its implementation | |
| CN103049574B (en) | Realize key assignments file system and the method for file dynamic copies | |
| CN105260429A (en) | ICN network information name searching method based on multiple Bloom filters | |
| CN104217023A (en) | Method for realizing map tile storage by package technology | |
| US20220374407A1 (en) | Multi-tenant partitioning in a time-series database | |
| CN105354250A (en) | Data storage method and device for cloud storage | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| CN103970875A (en) | Parallel repeated data deleting method | |
| CN105045762A (en) | Management method and apparatus for configuration file | |
| Von der Weth et al. | Multiterm keyword search in NoSQL systems | |
| CN102622352B (en) | Method, server and system for achieving shared file | |
| CN102378407B (en) | Object name resolution system and method in internet of things | |
| CN106919696A (en) | SEO websites construction method and the response method of SEO requests | |
| CN104391947B (en) | Magnanimity GIS data real-time processing method and system | |
| CN103136294A (en) | File operating method and device | |
| US9626378B2 (en) | Method for handling requests in a storage system and a storage node for a storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |