CN103885887B - User data storage method, read method and system - Google Patents
User data storage method, read method and system Download PDFInfo
- Publication number
- CN103885887B CN103885887B CN201210564064.5A CN201210564064A CN103885887B CN 103885887 B CN103885887 B CN 103885887B CN 201210564064 A CN201210564064 A CN 201210564064A CN 103885887 B CN103885887 B CN 103885887B
- Authority
- CN
- China
- Prior art keywords
- user data
- user
- basic document
- storage
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to internet mass technical field of data storage, disclose a kind of user data storage method, including step：The user data of user is stored in the file body for the basic document being pre-created, makes the user data Coutinuous store of same user, the user data of basic document storage predesignated subscriber number；The location index of storage location of the user data in basic document is recorded in the file header of the basic document.Also disclose a kind of user data storage system, user data read method and system.The present invention stores the user data of predesignated subscriber's number by using basic document, and user data is continuously stored in basic document, to improve data inquiry or reading efficiency；And first data nodal information of full dose user need not be safeguarded as the prior art, additional memory source expense is avoided, O＆M cost is reduced.
Description
Technical field
The present invention relates to internet mass technical field of data storage, more particularly to a kind of user data storage method is read
Take method and system.
Background technology
User's original content（User Generated Content, UGC）It is the new paragon that a kind of user uses internet,
Become to download based on download by original and upload is laid equal stress on.Community network, video sharing and blog etc. are all the main of UGC
Application form.With the continuous development of global interconnection network service, UGC business increasingly emerges, and causes the extensive pass of industry
Note.
Since data are that user generates, the user of magnanimity expedites the emergence of out the data of magnanimity, while can bring the reading of magnanimity again
The amount of writing.How these user data are effectively stored, and how to provide the read-write service of high concurrent is that technical field necessarily faces
Problem.
User data is stored in the form of data file in the prior art, All Files all string together, composition one
A chain, file number is more, and chain is longer.One file is made of several " nodes ", one data sheet of each node on behalf
Member, each data cell can only store the data of a user.The data of each user are made of several discontinuous nodes,
All nodes of one user are logically from beginning to end, form a chain.That is, having multiple sections in a file
Point, thereby increases and it is possible to store the data of multiple users.The data of one user are made of multiple discontinuous nodes, that is to say, that one
The data of a user are likely distributed in multiple files.Its file storage structure schematic diagram in memory index as shown in Figure 1, mark
The position of all users first node hereof.The processing logic of read request is：Memory index is first read, obtains and uses
The position of first node of family hereof.The data that the node obtains the node are found, are returned if meeting reading conditions
It returns, otherwise by first node, continues the data for looking for second node（Each node, which has, is directed toward next node position
Information）, and so on, until all nodes of user, which are all read, to be finished or meet reading conditions.
The storage method of above-mentioned user data has the disadvantage that：
If 1, each file size is 2GB, over time, data volume is more and more, and file number is also increasingly
More, the data of user are also increasingly longer, and the number of files of user data distribution can be more and more, and the data for reading a user may
The multiple data files of opening operation are wanted, the complexity of processing is O (n), can lead to internet product operation cost height, data in this way
Search efficiency is low, in poor service；
2, memory index preserves the data of first node of user, if the loss of data in memory index, will lose
The information of first node of user, also just can not find the position of descendant node, the data of that user will also lose；
3, memory index needs to preserve first nodal information of full dose user, and number of users is very more, this needs disappears
Consume very big memory.
Especially for microblog data, mass users continually read, update microblog data, and drawbacks described above is more obvious,
Increase severely over time and space with the increase of user volume.
If can have a kind of simple, efficiently, inexpensive method, it will greatly reduce O＆M complexity and cost,
A kind of new thinking can be provided for later similar service model.
Invention content
The purpose of the present invention is to provide a kind of user data storage method and systems, it is intended to solve mass users data and exist
Efficiency low problem when resource consumption is big when storage, O＆M cost is high and inquires.
In order to solve the above technical problems, the present invention provides a kind of user data storage method, including step：
The user data of user is stored in the file body for the basic document being pre-created, makes the number of users of same user
According to Coutinuous store, the user data of basic document storage predesignated subscriber number；
The location index of storage location of the user data in basic document is recorded in the file header of the basic document.
Wherein, the user data of user is stored in the file body for the basic document being pre-created, makes same user's
The step of user data Coutinuous store, specifically includes：
Calculate the basic document ID for storing the user data of the user by User ID, basic document ID=（User ID/
Predesignated subscriber's number）+ 1 ,/it is to divide exactly operation；
If the basic document has created, the user data is continuously stored in the corresponding region of basic document；If should
Basic document does not create, then creates the basic document, and the user data is continuously stored in basic document.
Wherein, the position of storage location of the user data in basic document is recorded in the file header of the basic document
The step of index, specifically includes：
The address offset of the storage location index in the header file of the basic document, address are calculated by User ID
Offset=User ID % predesignated subscriber's number, % is modulo operation；
The storage unit location index being stored in where the address offset of the header file of basic document.
Wherein, the user data orderly stores in chronological order, and the user data is stored simultaneously when storing user data
Temporal information when being stored, when the data volume of the user data of Coutinuous store is more than predetermined amount, by predetermined amount of use
User data divides a logical blocks, and by the temporal informations of the originating subscriber data of the logical blocks, ending user data
The mapping relations of time, initial address and end addresses are stored in as the location index in the file header.
Wherein, when the user data being stored in the file body of the basic document, if the size of user data is super
Predetermined threshold is crossed, then creates extension file, will be stored in the migration of subscriber data to the extension file, and in the storage
Position records address information of the user data in the extension file.
Wherein, the extension file includes the user data of one or more users.
The present invention also provides a kind of user data storage systems, including：
User data memory module, the file body for the user data of user to be stored in the basic document being pre-created
In, make the user data Coutinuous store of same user, the user data of basic document storage predesignated subscriber number；
Location index logging modle, for recording user data in the file header of the basic document in basic document
Storage location location index.
Wherein, the user data memory module specifically includes：
Basic document ID computing modules, for calculating the basic document for storing the user data of the user by User ID
ID, basic document ID=（User ID/predesignated subscriber's number）+1；
Memory module is judged, for the user data to be continuously stored in basic document when the basic document has created
Corresponding region；The basic document is created when not creating, and the user data is continuously stored in basic document.
Wherein, the location index logging modle specifically includes：
Address offset computing module, the storage institute rheme in the header file for calculating the basic document by User ID
Set the address offset of index, address offset=User ID % predesignated subscriber's number；
Location index memory module, the address offset institute of the header file for the location index to be stored in basic document
Storage unit.
Wherein, further include：Logical blocks division module, the data volume for the user data when Coutinuous store are more than predetermined
When amount, by predetermined amount of user data divide a logical blocks, and by the originating subscriber data of the logical blocks when
Between information, the mapping relations of end up time of user data, initial address and end addresses be stored in as the location index
In the file header, the user data orderly stores in chronological order, and the user data is stored simultaneously when storing user data
Temporal information when being stored.
Wherein, further include：File creation module is extended, for when the size of the user data of storage is more than predetermined threshold
Extension file is created, will be stored in the migration of subscriber data to the extension file, and described in storage location record
Address information of the user data in the extension file.
The present invention also provides a kind of user data read method, read step includes：
Basic document where the location index of bearing direction user data, and the basic document is loaded into memory,
The basic document includes file body for storing user data and for storing storage of the user data in basic document
The file header of the location index of position；
The location index is inquired, the storage location where the user data of user is positioned；
The user data is read from the storage location.
Wherein, the concrete mode of the basic document where the location index of bearing direction user data is：
Calculate the basic document ID for storing the user data by User ID, basic document ID=（User ID/predetermined use
Amount）+1.
Wherein, the location index is inquired, the concrete mode for positioning the storage location where the user data of user is：
The address offset of the storage location index in the header file of the basic document, address are calculated by User ID
Offset=User ID % predesignated subscriber's number reads the location index from the storage unit where described address offset.
Wherein, the location index is inquired, the concrete mode for positioning the storage location where the user data of user is：
The address offset of the storage location index in the header file of the basic document, address are calculated by User ID
Offset=User ID % predesignated subscriber's number, the storage unit where time set by user or period from described address offset
Time, initial address and the end addresses of the temporal information of the originating subscriber data of the logical blocks of record, the user data that ends up
Mapping table in read the initial address and end addresses of corresponding with the time or period logical blocks.
Wherein, reading the user data concrete mode from the storage location is：
If the storage location in the basic document of location index meaning is stored with user data, the user is directly read
Data；If what the storage location in the basic document of location index meaning stored is address information, pointed by address information
Extension file in read the user data, after reading user data, required data are screened from the user data of reading.
The present invention also provides a kind of user data to read system, including：
Basic document locating module, for the basic document where the location index of bearing direction user data, and by institute
It states basic document and is loaded into memory, the basic document includes file body for storing user data and for storing number of users
According to the file header of the location index of the storage location in basic document；
Storage location locating module positions the storage position where the user data of user for inquiring the location index
It sets；
User data read module, for reading the user data from the storage location.
Wherein, the basic document locating module is specifically used for calculating the basis for storing the user data by User ID
File ID, basic document ID=（User ID/predesignated subscriber's number）+1.
Wherein, the storage location locating module is specifically used in the header file for calculating the basic document by User ID
The storage location index address offset, address offset=User ID % predesignated subscriber's number, from where described address offset
Storage unit reads the location index.
Wherein, the storage location locating module is specifically used in the header file for calculating the basic document by User ID
The storage location index address offset, address offset=User ID % predesignated subscriber's number, according to the time set by user or
The temporal information of the originating subscriber data of the logical blocks of unit records where period from described address offset, ending
Patrol corresponding with the time or period is read in the mapping table of time of user data, initial address and end addresses
Collect the initial address and end addresses of block.
Wherein, if the user data read module is specifically used for the storage location in the basic document of location index meaning
It is stored with user data, then directly reads the user data；If the storage location in the basic document of location index meaning is deposited
Storage is address information, then the user data is read from the extension file pointed by address information, after reading user data,
Required data are screened from the user data of reading.
The present invention stores user data using basic document, and the data of each user are continuously stored in basic document,
To improve data inquiry or reading efficiency；And first number of full dose user need not be safeguarded as the prior art
According to nodal information, additional memory source expense is avoided, O＆M cost is reduced.
Description of the drawings
Fig. 1 is subscriber data file structural schematic diagram in the prior art；
Fig. 2 is a kind of user data storage method flow chart of the embodiment of the present invention；
Fig. 3 is the subscriber data file structural schematic diagram established by method in Fig. 2；
Fig. 4 is a kind of user data memory system architecture schematic diagram of the embodiment of the present invention；
Fig. 5 is a kind of user data read method flow chart of the embodiment of the present invention；
Fig. 6 is that a kind of user data of the embodiment of the present invention reads system structure diagram.
Specific implementation mode
With reference to the accompanying drawings and examples, the specific implementation mode of the present invention is described in further detail.Implement below
Example is not limited to the scope of the present invention for illustrating the present invention.
The method of the present invention is suitable for the storage of magnanimity user data on internet, such as the storage of microblog data, micro-
In rich, the personal page information of user can show what the inbox storage user of microblogging and user that this user delivers received
Microblogging.Memory storage can only newest several of limited storage user（About 200 or so）, and file storage needs full dose to store
All data of this user.When amount of user data is more, it is necessary to which special file storage structure stores this user
Data.As shown in Fig. 2, for the user data storage method flow chart of the embodiment of the present invention, including：
The user data of user is stored in the file body for the basic document being pre-created by step S201, makes same use
The user data Coutinuous store at family, to facilitate the reading of user data, and the user of basic document storage predesignated subscriber number
Data.100,000 users are pressed in the present embodiment（Predesignated subscriber's number can be according to actual total number of users and admissible basis text
The size of part determines）It it is one group, i.e. a basic document stores the data of 100,000 users.
When user uploads data, pass through the User ID of the user（One user of User ID unique mark, usually counts
Word, if not number can also convert characters to binary number）Calculate the basic document ID for storing its user data, basis text
Part ID=（User ID/predesignated subscriber's number）+ 1 ,/it is to divide exactly operation.Usual basic document is also named with basic document ID.
It is corresponding when certain user uploads data since a basic document stores the user data of 100,000 users
Basic document may have been created or not created also.If the basic document has created, user data is continuously stored in basic text
The corresponding region of part；If the basic document does not create, the basic document is created, and user data is continuously stored in basic text
In part.
Wherein, the corresponding region that the user data of some user is stored in basic document can be the row by User ID in advance
A certain size storage region that sequence divides, can also be the user data of each user of Coutinuous store at the beginning, as a certain user
When new upload data, the end for the corresponding region for storing its data is found, is inserted into the data newly uploaded, is then moved backward below
The user data of user.When storage in order to save memory space using tighten store by the way of by the data of different users successively
It is stored in basic document.
The process that user uploads a data at random is generally as follows：First this data is recorded in memory, at this time may be used
It is operated with modification with externally providing the reading to this data in real time；Then this operation note is added into write-in binlog texts
Part（The difference of binlog files and basic document is：Binlog file records are operations, and basic document is that user is real
Data）End；Final cycle since binlog files designated position read operation record, the operation of same subscriber is closed
And get up, it is merged with original data in memory, new user data can be generated in this way, and new user data is deposited
Storage is in basic document.
Step S202 records the position of storage location of the user data in basic document in the file header of basic document
Index.Location index can pass through initial address and end for identifying the address field that the user data is stored in basic document
Only address identifies, and can also be identified by initial address and current data length.Record position indexes in the present embodiment
Mode is as follows：
The ground of the location index of the user data for storing the user in the header file of basic document is calculated by User ID
Location deviates, and address offset=User ID % predesignated subscriber's number, % is modulo operation.Location index is stored in the head text of basic document
Storage unit where the address offset of part.
The present embodiment stores the user data of predetermined quantity, and the user data of each user using above-mentioned basic document
It is continuously stored in basic document, without being distributed across in different files, disposably can inquire or read user data,
Its complexity is O (1), improves the efficiency of inquiry or the reading of data；And it need not safeguard that full dose is used as the prior art
First data nodal information at family, avoids additional memory source expense, reduces O＆M cost；In addition, its basis text
Part itself contains the index of user data, it is no longer necessary to which memory indexes, and avoids server system power down memory index and loses
Caused by user data loss the phenomenon that.
It is disposable all readings to read this user data, then again in these use when amount of user data is smaller
It is screened in user data（Such as：By keyword lookup input by user）Go out the data of needs.When the data volume of user is larger, such as
Microblog data, i.e., when user to deliver microblogging more frequent, in the case that data volume is larger（More than 3000）, disposable whole
The microblog data for reading this user has clearly a need for expending larger time and space resources.
Further, for the ease of subsequently rapidly reading required user data, in the case that amount of user data is small with
Different storage modes is used in the case that amount of user data is big.In the present embodiment, user data orderly stores in chronological order,
The temporal information when user data is stored is stored when storing user data simultaneously.When the data of the user data of Coutinuous store
Amount is more than predetermined amount（It it is 3000 or more for microblog data）When, the user data of predetermined amounts divides a logical blocks,
And by the temporal information of the originating subscriber data of the logical blocks, end up user data time, initial address and end addresses
Mapping relations be stored in the file header as the location index of the logical blocks.In this way, can basis in searching data
Time or period set by user find corresponding logical blocks, then required number of users is filtered out from the logical blocks
According to, rather than entire user data is read, it goes to search wherein, to accelerate reading speed.
When taking user data due to subsequent reads, needs first to read basic document, i.e., basic document is loaded into memory.In order to
It avoids basic document excessive, influences loading velocity, further, when the user data size of some user is more than predetermined threshold
When, then extension file is created, and will be stored in the migration of subscriber data to extension file, and storage was used originally in basic document
Address field of the storage location record user data of user data in extending file.For the storage mode by logical blocks, base
Originally the ground of each logical blocks of the storage location record user data of user data in extending file was stored in plinth file
Location section.
The subscriber data file storage organization formed by above-mentioned user data storage method is as shown in figure 3, a basis text
Part corresponds to one or more extension files, can also be without extension file.Wherein extension file may include one or more use
The user data at family.
Wherein, predetermined threshold can be set according to actual conditions, such as：For text data, it is preferably set to 2GB ~ 4GB.2GB
~ 4GB can meet user and store a large amount of user data, therefore can guarantee that user data is all consecutively stored in basic document,
And with little need for or only need less extension file to reduce O＆M to improve the reading speed of user data
Cost.It is then its establishment extension file for the data more than predetermined threshold, it is preferable that in order to facilitate maintenance management, an expansion
Exhibition file only stores the user data of a user.
The present invention also provides a kind of subscriber data file structures to establish system, as shown in figure 4, including：User data is deposited
Store up module 401 and location index logging modle 402.
User data memory module 401 is used to the user data of user being stored in the file for the basic document being pre-created
In body, the user data Coutinuous store of same user, basic document is made to store the user data of predesignated subscriber's number.
Location index logging modle 402 is used to record user data in the file header of basic document in basic document
The location index of storage location.
Wherein, user data memory module 401 includes：
Basic document ID computing modules, for calculating the basic document for storing the user data of the user by User ID
ID, basic document ID=（User ID/predesignated subscriber's number）+1；
Memory module is judged, for the user data to be continuously stored in basic document when the basic document has created
Corresponding region；The basic document is created when not creating, and the user data is continuously stored in basic document.
Wherein, location index logging modle 402 includes：
Address offset computing module, the storage institute rheme in the header file for calculating the basic document by User ID
Set the address offset of index, address offset=User ID % predesignated subscriber's number；
Location index memory module, the address offset institute of the header file for the location index to be stored in basic document
Storage unit.
Further, which further includes：Logical blocks division module, the data for the user data when Coutinuous store
When amount is more than predetermined amount, a logical blocks are divided by predetermined amount of user data, and the starting of the logical blocks is used
The temporal information of user data, the time for the user data that ends up, initial address and end addresses mapping relations as the position
Index is stored in the file header, and the user data orderly stores in chronological order, is stored simultaneously when storing user data
The temporal information when user data is stored.
Further, which further includes extension file creation module, and the size for the user data when storage is more than
Extension file is created when predetermined threshold, will be stored in the migration of subscriber data to the extension file, and in the storage position
Set the address information for recording the user data in the extension file.
Based on the corresponding user data read method of above-mentioned user data storage method as shown in figure 5, including：
Step S501, the basic document where the location index of bearing direction user data, and the basic document is added
It is downloaded to memory.When user's requests data reading, the basic document ID of storage user data, basic document are calculated by User ID
ID=（User ID/predesignated subscriber's number）+ 1, to position the basic document for storing the user data of the user.
Step S502, inquiring position index, positions the storage location where the user data of user.It is calculated by User ID
The address offset of storage location index in the header file of basic document, address offset=User ID % predesignated subscriber's number.Obtain ground
The address of the storage location of storage actual user data can be obtained after the offset of location from the storage unit where the address offset.
Corresponding to storage mode of amount of user data when larger, step S502 calculates the head of basic document by User ID
The address offset of storage location index in file, address offset=User ID % predesignated subscriber's number, according to the time set by user
Or the temporal information of the originating subscriber data of the logical blocks of the unit records where the period from address offset, ending are used
Logical blocks corresponding with time or period are read in the mapping table of time of user data, initial address and end addresses
Initial address and end addresses.
It should be noted that the possible more than one of logical blocks corresponding with time set by user or period, such as：
Certain section very short of time data amount may be very big, may exceed scheduled data volume（Such as 3000 microblog datas）Or the period
Initial time and terminate the time inherently span more than one logical blocks, it is at this moment, the time or period is corresponding
Logical blocks are all read.User can correspond to less logical blocks with control time parameter.Due to only reading one or several
A a small amount of logical blocks only screen required number in follow-up step S503 in a small amount of logic area data in the block
According to improve the reading efficiency of user data.User can also extract the time parameter pair by only setting time parameter
The user data answered.
Step S503 reads user data from storage location.At the time of reading, if in the basic document of location index meaning
Storage location is stored with user data, then directly reads user data；If the storage position in the basic document of location index meaning
Set storage is address information, then reads user data from the extension file pointed by address information.After reading user data,
It is screened from the user data of reading（Such as：The keyword query specified according to user）Required data.
The user data storage organization of extension file, user data is added to read speed as a result of above-mentioned basic document
Degree greatly improves.
The present invention also provides a kind of user data to read system, as shown in fig. 6, including：Basic document locating module
601, storage location locating module 602 and user data read module 603.
Basic document locating module 601 is used for the basic document where the location index of bearing direction user data, and will
The basic document is loaded into memory, is specifically used for calculating the basic document ID for storing the user data, base by User ID
Plinth file ID=（User ID/predesignated subscriber's number）+ 1, to obtain basic document ID.
Storage location locating module 602 positions the storage where the user data of user for inquiring the location index
Position calculates the address offset of the storage location index in the header file of the basic document, ground especially by User ID
Location offset=User ID % predesignated subscriber's number reads the location index from the storage unit where described address offset.
In the case of amount of user data is larger, storage location locating module 602 is specifically used for calculating institute by User ID
State the address offset of the storage location index in the header file of basic document, address offset=User ID % predesignated subscriber's number,
The starting of the logical blocks of unit records where time set by user or period from described address offset is used
The temporal information of user data, the time for the user data that ends up, initial address and end addresses mapping table in read and institute
State the initial address and end addresses of time or period corresponding logical blocks.
User data read module 603 is used to read the user data from the storage location.The user data is read
Module 603 is when reading user data, if the storage location in the basic document of location index meaning is stored with user data,
Directly read the user data；If what the storage location in the basic document of location index meaning stored is address information,
The user data is read from the extension file pointed by address information, after reading user data, from the user data of reading
Data needed for middle screening.
User data storage method, read method and the system of the present invention can be widely applied to mass users to user data
The scene for being stored and being read, such as：Storage and reading of the microblog users to microblog data.
Embodiment of above is merely to illustrate the present invention, and not limitation of the present invention, in relation to the common of technical field
Technical staff can also make a variety of changes and modification without departing from the spirit and scope of the present invention, therefore all
Equivalent technical solution also belongs to scope of the invention, and scope of patent protection of the invention should be defined by the claims.
Claims (19)
1. a kind of user data storage method, which is characterized in that including step：
The user data of user is stored in the file body for the basic document being pre-created, the user data of same user is made to connect
Renew storage, the user data of basic document storage predesignated subscriber number；
The location index of storage location of the user data in basic document is recorded in the file header of the basic document；
When the data volume of the user data of Coutinuous store is more than predetermined amount, a use is divided by predetermined amount of user data
It is stored in the logical blocks of storage user data；By the temporal information of the originating subscriber data of the logical blocks, ending
The mapping relations of time of user data, initial address and end addresses are stored in the file header as the location index
In.
2. user data storage method as described in claim 1, which is characterized in that the user data of user to be stored in advance
In the file body of the basic document of establishment, the step of user data Coutinuous store for making same user, specifically includes：
The basic document ID for storing the user data of the user, basic document ID=(User ID/predetermined are calculated by User ID
Number of users)+1 ,/it is to divide exactly operation；
If the basic document has created, the user data is continuously stored in the corresponding region of basic document；If the basis
File does not create, then creates the basic document, and the user data is continuously stored in basic document.
3. user data storage method as described in claim 1, which is characterized in that remember in the file header of the basic document
The step of location index for employing storage location of the user data in basic document, specifically includes：
The address offset of the storage location index in the header file of the basic document, address offset are calculated by User ID
=User ID % predesignated subscriber's number, % are modulo operation；
The storage unit location index being stored in where the address offset of the header file of basic document.
4. user data storage method as described in claim 1, which is characterized in that the user data is orderly in chronological order
Storage stores the temporal information when user data is stored simultaneously when storing user data.
5. user data storage method as described in any one of claims 1 to 4, which is characterized in that by the user data
When storing in the file body of the basic document, if the size of user data is more than predetermined threshold, extension file is created, it will
It is stored in the migration of subscriber data to the extension file, and the user data is recorded in the expansion in the storage location
Open up the address information in file.
6. user data storage method as claimed in claim 5, which is characterized in that the extension file includes one or more
The user data of user.
7. a kind of user data storage system, which is characterized in that including：
User data memory module, for the user data of user to be stored in the file body for the basic document being pre-created,
Make the user data Coutinuous store of same user, the user data of basic document storage predesignated subscriber number；
Location index logging modle, for recording user data depositing in basic document in the file header of the basic document
The location index that storage space is set；
The system also includes：Logical blocks division module, the data volume for the user data when Coutinuous store are more than predetermined
When amount, divides one by predetermined amount of user data and stored for storing the logical blocks of user data；It is additionally operable to
By the temporal information of the originating subscriber data of the logical blocks, time, initial address and the end addresses of the user data that ends up
Mapping relations be stored in the file header as the location index.
8. user data storage system as claimed in claim 7, which is characterized in that the user data memory module is specifically wrapped
It includes：
Basic document ID computing modules, for calculating the basic document ID for storing the user data of the user by User ID,
Basic document ID=(User ID/predesignated subscriber's number)+1；
Judge memory module, the phase for the user data to be continuously stored in basic document when the basic document has created
Answer region；The basic document is created when not creating, and the user data is continuously stored in basic document.
9. user data storage system as claimed in claim 7, which is characterized in that the location index logging modle is specifically wrapped
It includes：
Address offset computing module, the position rope of the storage in header file for calculating the basic document by User ID
The address offset drawn, address offset=User ID % predesignated subscriber's number；
Location index memory module, where the address offset of the header file for the location index to be stored in basic document
Storage unit.
10. user data storage system as claimed in claim 7, which is characterized in that the user data has in chronological order
Sequence stores, and stores the temporal information when user data is stored when storing user data simultaneously.
11. the user data storage system as described in any one of claim 7~10, which is characterized in that further include：Extension text
Part creation module, for creating extension file when the size of the user data of storage is more than predetermined threshold, by the number of users
It is stored according to moving in the extension file, and the user data is recorded in the extension file in the storage location
Address information.
12. a kind of user data read method, which is characterized in that read step includes：
Basic document where the location index of bearing direction user data, and the basic document is loaded into memory, it is described
Basic document includes file body for storing user data and for storing storage location of the user data in basic document
Location index file header；
The location index is inquired, the storage location where the user data of user is positioned；
The user data is read from the storage location；
Wherein, the inquiry location index, the concrete mode for positioning the storage location where the user data of user are：
The starting of the logical blocks of unit records where time set by user or period from address offset is used
The temporal information of user data, the time for the user data that ends up, initial address and end addresses mapping table in read and institute
State the initial address and end addresses of time or period corresponding logical blocks.
13. user data read method as claimed in claim 12, which is characterized in that the position rope of bearing direction user data
The concrete mode of basic document where drawing is：
The basic document ID for storing the user data, basic document ID=(User ID/predesignated subscriber are calculated by User ID
Number)+1.
14. user data read method as claimed in claim 12, which is characterized in that inquire the location index, positioning is used
The concrete mode of storage location where the user data at family is：
The address offset of the storage location index in the header file of the basic document, address offset are calculated by User ID
=User ID % predesignated subscriber's number reads the location index from the storage unit where described address offset.
15. the user data read method as described in any one of claim 12~14, which is characterized in that from the storage position
Setting the reading user data concrete mode is：
If the storage location in the basic document of location index meaning is stored with user data, the number of users is directly read
According to；If what the storage location in the basic document of location index meaning stored is address information, pointed by address information
The user data is read in extension file, after reading user data, required data are screened from the user data of reading.
16. a kind of user data reads system, which is characterized in that including：
Basic document locating module, for the basic document where the location index of bearing direction user data, and by the base
Plinth file is loaded into memory, and the basic document includes file body for storing user data and exists for storing user data
The file header of the location index of storage location in basic document；
Storage location locating module positions the storage location where the user data of user for inquiring the location index；
User data read module, for reading the user data from the storage location；
The storage location locating module is specifically used for：According to depositing where time set by user or period from address offset
Time, initial address and the knot of the temporal information of the originating subscriber data of the logical blocks of storage unit record, the user data that ends up
The initial address and end addresses of logical blocks corresponding with the time or period are read in the mapping table of tail address.
17. user data as claimed in claim 16 reads system, which is characterized in that the basic document locating module is specific
For calculating the basic document ID for storing the user data, basic document ID=(User ID/predesignated subscriber by User ID
Number)+1.
18. user data as claimed in claim 16 reads system, which is characterized in that the storage location locating module is specific
The address offset of the storage location index in header file for calculating the basic document by User ID, address offset
=User ID % predesignated subscriber's number reads the location index from the storage unit where described address offset.
19. the user data as described in any one of claim 16~18 reads system, which is characterized in that the user data
If the storage location that read module is specifically used in the basic document of location index meaning is stored with user data, directly read
The user data；If what the storage location in the basic document of location index meaning stored is address information, believe from address
The user data is read in the pointed extension file of breath, after reading user data, institute is screened from the user data of reading
The data needed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210564064.5A CN103885887B (en) | 2012-12-21 | 2012-12-21 | User data storage method, read method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210564064.5A CN103885887B (en) | 2012-12-21 | 2012-12-21 | User data storage method, read method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103885887A CN103885887A (en) | 2014-06-25 |
| CN103885887B true CN103885887B (en) | 2018-07-27 |
Family
ID=50954795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210564064.5A Active CN103885887B (en) | 2012-12-21 | 2012-12-21 | User data storage method, read method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103885887B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346458B (en) * | 2014-10-31 | 2018-05-18 | 阿里巴巴（中国）有限公司 | Date storage method and storage device |
| CN104484332A (en) * | 2014-11-11 | 2015-04-01 | 珠海天琴信息科技有限公司 | Method and device for reading and writing data in embedded system |
| CN106547784B (en) * | 2015-09-22 | 2020-04-28 | 阿里巴巴集团控股有限公司 | Data splitting and storing method and device |
| CN105488225B (en) * | 2015-12-31 | 2019-01-25 | 新浪网技术(中国)有限公司 | Hbase data export method and device in PaaS system |
| CN107085900B (en) * | 2017-03-22 | 2019-05-07 | 百富计算机技术（深圳）有限公司 | Data processing method, device, system and POS terminal |
| CN109992535B (en) * | 2017-12-29 | 2024-01-30 | 华为技术有限公司 | Storage control method, device and system |
| CN110109886B (en) * | 2018-02-01 | 2022-11-18 | 中兴通讯股份有限公司 | File storage method of distributed file system and distributed file system |
| CN110109865A (en) * | 2019-05-07 | 2019-08-09 | 苏州浪潮智能科技有限公司 | A kind of date storage method, device, equipment and readable storage medium storing program for executing |
| WO2023143099A1 (en) * | 2022-01-29 | 2023-08-03 | 华为技术有限公司 | Data aggregation method and computation system |
| CN115630655B (en) * | 2022-12-22 | 2023-04-14 | 星汉智能科技股份有限公司 | Method and system for reading record file of SIM card and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101175247A (en) * | 2007-09-28 | 2008-05-07 | 中兴通讯股份有限公司 | Method for centralized storing and using user data |
| CN101620702A (en) * | 2008-06-30 | 2010-01-06 | 中国移动通信集团黑龙江有限公司 | Expenditure presentation processing method, device and system |
| CN102694824A (en) * | 2011-03-22 | 2012-09-26 | 中国移动通信集团公司 | User data storage system and data access method thereof |
| CN102810092A (en) * | 2011-05-31 | 2012-12-05 | 腾讯科技（深圳）有限公司 | Data read-write method and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101505472B (en) * | 2008-02-05 | 2011-07-20 | 华为技术有限公司 | User data server system and apparatus |
| CN102799628B (en) * | 2012-06-21 | 2015-10-07 | 新浪网技术(中国)有限公司 | The method and apparatus of data partition is carried out in key-value database |
-
2012
- 2012-12-21 CN CN201210564064.5A patent/CN103885887B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101175247A (en) * | 2007-09-28 | 2008-05-07 | 中兴通讯股份有限公司 | Method for centralized storing and using user data |
| CN101620702A (en) * | 2008-06-30 | 2010-01-06 | 中国移动通信集团黑龙江有限公司 | Expenditure presentation processing method, device and system |
| CN102694824A (en) * | 2011-03-22 | 2012-09-26 | 中国移动通信集团公司 | User data storage system and data access method thereof |
| CN102810092A (en) * | 2011-05-31 | 2012-12-05 | 腾讯科技（深圳）有限公司 | Data read-write method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103885887A (en) | 2014-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103885887B (en) | User data storage method, read method and system | |
| CN103019953B (en) | Construction system and construction method for metadata | |
| CN101620609B (en) | Multi-tenant data storage and access method and device | |
| CN107463447B (en) | B + tree management method based on remote direct nonvolatile memory access | |
| CN104794177B (en) | A kind of date storage method and device | |
| CN100468402C (en) | Sort data storage and split catalog inquiry method based on catalog tree | |
| CN103605758B (en) | The method and device that a kind of mobile terminal document is searched | |
| CN110413612A (en) | A kind of mixing internal memory performance optimization method and system based on hybrid index | |
| CN102332029A (en) | Hadoop-based mass classifiable small file association storage method | |
| CN103593436A (en) | File merging method and device | |
| CN106775446A (en) | Based on the distributed file system small documents access method that solid state hard disc accelerates | |
| CN103327052A (en) | Data storage method and system and data visiting method and system | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN102111438B (en) | Method and device for parameter adjustment and distributed computation platform system | |
| CN102541985A (en) | Organization method of client directory cache in distributed file system | |
| CN109240607B (en) | File reading method and device | |
| CN107832423A (en) | A kind of file read/write method for distributed file system | |
| CN103500089A (en) | Small file storage system suitable for Mapreduce calculation model | |
| CN103176750B (en) | Based on mobile Internet data storage system and the method thereof of staggered time subregion | |
| CN103049574B (en) | Realize key assignments file system and the method for file dynamic copies | |
| CN109766318A (en) | File reading and device | |
| CN112947860A (en) | Hierarchical storage and scheduling method of distributed data copies | |
| CN103399915A (en) | Optimal reading method for index file of search engine | |
| CN103793468A (en) | Data storage method and device and data reading method and device | |
| CN103310008A (en) | Cloud control server and file index method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |