CN108427728A - Management method, equipment and the computer-readable medium of metadata - Google Patents
Management method, equipment and the computer-readable medium of metadata Download PDFInfo
- Publication number
- CN108427728A CN108427728A CN201810150300.6A CN201810150300A CN108427728A CN 108427728 A CN108427728 A CN 108427728A CN 201810150300 A CN201810150300 A CN 201810150300A CN 108427728 A CN108427728 A CN 108427728A
- Authority
- CN
- China
- Prior art keywords
- metadata
- daily record
- disk
- mirror image
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0727—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a storage system, e.g. in a DASD or network based storage system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24573—Query processing with adaptation to user needs using data annotations, e.g. user-defined metadata
Abstract
The present invention provides a kind of management method of metadata, equipment and computer-readable medium.Its method includes：Receive operation requests of the user to metadata；It is modified according to the metadata of the operation requests of metadata, the tree to being stored in memory；According to the operation requests of metadata, corresponding daily record is generated, and be written in disk.Compared with the metadata that the relation table using third normal form of the prior art stores, the metadata of the management of the present invention is stored in memory with tree, it can ensure in this way when accessing, avoid using a large amount of join sentences, the access efficiency of metadata can be effectively improved.And in the present invention, also according to the operation requests of metadata, generate corresponding daily record, and disk is written, it can be when the corresponding process of metadata kills or clicks delay machine, the metadata restored in memory is helped to avoid the risk that metadata is lost completely to ensure persistence and the safety of metadata.
Description
【Technical field】
The present invention relates to a kind of computer application technology more particularly to management method of metadata, equipment and calculating
Machine readable medium.
【Background technology】
Database software will also store the information for describing user data, these are retouched while storing user data
Metadata can be referred to as by stating the information of user data.Existing metadata includes but not limited to database-name, table name, row
Information, Data distribution information, user right information and the various information for describing current database state.
Metadata is extremely important for database, and the almost all of operation to database such as the inquiry of user imports number
According to etc., it is required for accessing metadata, and according to the corresponding data of metadata access.In view of the importance of metadata, metadata
Management should at least meet following demand：It is efficient to access, to ensure the performance of database；Distinctive persistence, to protect
Card metadata can not lose, and the single machine for especially storing metadata still can have after delay machine recovery.Fig. 1 is the prior art
In table structure of the metadata in relational database.As shown in Figure 1, existing metadata is stored with relevant database (such as
Mysql form), is stored in single machine.Metadata is stored in a manner of relation table, and meets database third normal form.
But it since the relation table of metadata is using third normal form design, and when accessing certain metadata, often needs
A large amount of join sentences are used, cause the access efficiency of metadata very low.And it is low that metadata itself is stored in disk etc.
In fast medium, there is the performance bottleneck accessed in itself.Therefore, cause the access efficiency of existing metadata very low.
【Invention content】
The present invention provides a kind of management method of metadata, equipment and computer-readable mediums.For improving metadata
Access efficiency.
The present invention provides a kind of management method of metadata, the method includes：
Receive operation requests of the user to metadata；
It is modified according to the metadata of the operation requests of the metadata, the tree to being stored in memory；
And according to the operation requests of the metadata, corresponding daily record is generated, and be written in disk.
Still optionally further, in method as described above, further include：
After the single machine delay machine where the metadata is restored, ascending sequence is identified according to each daily record, according to
It is secondary that each daily record is read from the disk；Each daily record mark is passed according to the generated time of the daily record by small arrival
Increase；
And play back the operation of each daily record successively in the memory according to reading order, restore the institute in the memory
State metadata.
Still optionally further, in method as described above, further include：
Whether the item number for detecting the daily record in the disk reaches predetermined threshold value；
If reaching, further judge to whether there is metadata mirror image in the disk；
If in the presence of the metadata mirror image in the disk is updated according to the daily record of the predetermined threshold value item number.
Still optionally further, in method as described above, further include：
If metadata mirror image is not present in the disk, first number is generated according to the daily record of the predetermined threshold value item number
According to mirror image, and it is stored in the disk.
Still optionally further, in method as described above, further include：
Record the corresponding maximum daily record mark of the metadata mirror image；
Delete the corresponding each daily record of the metadata mirror image.
Still optionally further, in method as described above, further include：
After the single machine delay machine where the metadata is restored, the existing metadata mirror is read from the disk
Picture；
The metadata mirror image read is loaded in the memory；
From each daily record that the corresponding maximum daily record of the metadata mirror image identifies after corresponding daily record, according to
Each daily record identifies ascending sequence, is successively read each daily record；
And play back each daily record corresponding operation on the basis of the metadata mirror image successively according to reading order, into
And restore the metadata in the memory.
Still optionally further, in method as described above, further include：
Backup front end node on other single machines sends the corresponding daily record of operation requests of the metadata, for
The daily record pair is played back on the basis of metadata in corresponding memory of other backup front end nodes on other described single machines
The operation answered, to synchronize the metadata on other described single machines.
The present invention provides a kind of front end node equipment, and the equipment includes：
Receiving module, for receiving operation requests of the user to metadata；
Metadata processing module, for according to the operation requests of the metadata, to the tree that is stored in memory
Metadata is modified；
Log processing module generates corresponding daily record, and disk is written for the operation requests according to the metadata
In.
Still optionally further, in equipment as described above, further include：
Read module, for after the single machine delay machine where the metadata is restored, being identified by small according to each daily record
To big sequence, each daily record is read from the disk successively；When each daily record identifies the generation according to the daily record
Between by it is small reach be incremented by；
Recovery module, the operation for playing back each daily record successively in the memory according to reading order restore institute
State the metadata in memory.
Still optionally further, in equipment as described above, further include：
Whether detection module, the item number for detecting the daily record in the disk reach predetermined threshold value；
The detection module is further sentenced if the item number for the daily record being additionally operable in the disk reaches the predetermined threshold value
It whether there is metadata mirror image in the disconnected disk；
The log processing module, if there are the metadata mirror images for the disk, according to the predetermined threshold value item
Several daily records updates the metadata mirror image in the disk.
Still optionally further, in equipment as described above, the log processing module is additionally operable to：
If metadata mirror image is not present in the disk, first number is generated according to the daily record of the predetermined threshold value item number
According to mirror image, and it is stored in the disk.
Still optionally further, in equipment as described above, the equipment further includes：
Logging modle, for recording the corresponding maximum daily record mark of the metadata mirror image；
Removing module, for deleting the corresponding each daily record of the metadata mirror image.
Still optionally further, in equipment as described above, the equipment further includes load-on module；
The read module is additionally operable to after the single machine delay machine where the metadata is restored, read from the disk
The existing metadata mirror image；
The load-on module, for loading the metadata mirror image read in the memory；
The read module is additionally operable to after the corresponding maximum daily record of the metadata mirror image identifies corresponding daily record
Each daily record in, identify ascending sequence according to each daily record, be successively read each daily record；
The recovery module is additionally operable to play back on the basis of the metadata mirror image successively according to reading order each described
The corresponding operation of daily record, and then restore the metadata in the memory.
Still optionally further, in equipment as described above, further include：
Sending module, the operation requests that the metadata is sent for the backup front end node on other single machines are corresponding
The daily record, for being returned on the basis of metadata of other backup front end nodes on other described single machines in corresponding memory
The corresponding operation of the daily record is put, to synchronize the metadata on other described single machines.
The present invention also provides a kind of computer equipment, the equipment includes：
One or more processors；
Memory, for storing one or more programs；
When one or more of programs are executed by one or more of processors so that one or more of processing
Device realizes the management method of metadata as described above.
The present invention also provides a kind of computer-readable mediums, are stored thereon with computer program, which is held by processor
The management method of metadata as described above is realized when row.
Management method, equipment and the computer-readable medium of the metadata of the present invention, by receiving user to metadata
Operation requests；It is modified according to the metadata of the operation requests of metadata, the tree to being stored in memory；According to first number
According to operation requests, generate corresponding daily record, and be written in disk.It is stored with the relation table using third normal form of the prior art
Metadata compare, the metadata of management of the invention is stored in memory with tree, can be ensured in this way when accessing,
It avoids using a large amount of join sentences, the access efficiency of metadata can be effectively improved.And in the present invention, also according to first number
According to operation requests, generate corresponding daily record, and disk is written, can kill or click delay machine in the corresponding process of metadata
When, help the metadata restored in memory to avoid what metadata was lost completely to ensure persistence and the safety of metadata
Risk.
【Description of the drawings】
Fig. 1 is table structure of the metadata in the prior art in relational database.
Fig. 2 is the flow chart of the management method embodiment one of the metadata of the present invention.
Fig. 3 is a kind of structure of metadata provided in an embodiment of the present invention.
Fig. 4 is the flow chart of the management method embodiment two of the metadata of the present invention.
Fig. 5 is the flow chart of the management method embodiment three of the metadata of the present invention.
Fig. 6 is the flow chart of the management method example IV of the metadata of the present invention.
Fig. 7 is the scene graph of the High Availabitity of the metadata of the present invention.
Fig. 8 is the structure chart of the front end node apparatus embodiments one of the present invention.
Fig. 9 is the structure chart of the front end node apparatus embodiments two of the present invention.
Figure 10 is the structure chart of the computer equipment embodiment of the present invention.
Figure 11 is a kind of exemplary plot of computer equipment provided by the invention.
【Specific implementation mode】
To make the objectives, technical solutions, and advantages of the present invention clearer, right in the following with reference to the drawings and specific embodiments
The present invention is described in detail.
Fig. 2 is the flow chart of the management method embodiment one of the metadata of the present invention.As shown in Fig. 2, the member of the present embodiment
The management method of data, can specifically include following steps：
100, operation requests of the user to metadata are received；
The executive agent of the management method of the metadata of the present embodiment can be front end (Front End；FE) node, example
Such as, in the single machine of storage metadata, which can also be embodied in one and operate in the operating system of the single machine
A process, which can be with all metadata requests of service processing.
The operation requests of the metadata of the present embodiment may include the establishment to the information in metadata, the modification of information with
And the processing such as deletion of information.The metadata of the present embodiment is stored in the memory of the single machine, and number described in the metadata
It can store in memory according to the data in library, there may also be in the disk of the single machine, not limit in the present embodiment.
In the present embodiment, user can be realized by human-machine interface module asks the operation of metadata to single machine input
It asks, accordingly, FE nodes can receive operation requests of the user to metadata.
101, according to the operation requests of metadata, the metadata of the tree to being stored in memory is modified；
It, can be right according to the operation requests of metadata after FE nodes get user to the operation requests of metadata
The metadata of the tree stored in memory is modified.Difference and the prior art store metadata in the present embodiment
In memory, since disk is low velocity medium, there is the performance bottleneck accessed in itself, and in the present embodiment, metadata is existed in
In depositing, relative to the disk of single machine, memory has higher access performance, in this way, when receiving access of the user to metadata
Request or when operation requests, the metadata in memory can be quickly accessed in FE nodes, and can further change in turn or
Person does other processing.Therefore the present embodiment can effectively improve the visit to metadata by storing metadata in memory
Ask efficiency.
Further, the metadata of the storage of the present embodiment in memory is stored using tree, with the prior art
Use third normal form, the metadata that is stored in a manner of relation table is compared, avoid using a large amount of join sentences, access very clever
Living, access efficiency is very high.
For example, Fig. 3 is a kind of structure of metadata provided in an embodiment of the present invention.As shown in figure 3, the structure of the metadata
For a tree, the wherein root node Database of tree can be the title of the corresponding database of metadata.So
Afterwards according to the service attribute of database, under root node Database that can be in the metadata, each service attribute pair is established
The Table answered, such as Fig. 3 is for establishing Table1, Table2 and Table3.In practical application, according to the kind of service attribute
The quantity of class can establish the Table of arbitrary corresponding number.For each Table, corresponding data can be deposited with piecemeal
Storage, accordingly, in the metadata, can divide polylith partition.Such as Fig. 3 is divided into 3 pieces of partition i.e. with Table2
For partition1, partition2 and partition3, in practical application, which can also be divided into other quantity
Piecemeal.And accordingly, Table1 and Table3 can also be respectively classified into polylith partition.Further, for each piece
Partition can also be divided into multistage, for example, being divided into Tablet1, Tablet2 in embodiment illustrated in fig. 3 with partition2
For totally 3 sections of Tablet3, in practical application, one piece of partition is segmented into arbitrary integer section, i.e. partition1 and
Partition3 can also be divided into Tablet sections multiple according to actual demand.Then, multiple Tablet sections can be made more
A copy is respectively present on different single machines, if when ensureing a wherein single machine delay machine, remains able to obtain from other single machines
The information for getting data, to ensure the normal access of the data in the corresponding database of metadata.Such as embodiment illustrated in fig. 3
In, by taking Tablet2 saves 3 parts of copies Replica1, Replica2 and Replica3 as an example, Replica1, Replica2 and
Replica3 is respectively present on 3 single machines.Similarly, Tablet1 and Tablet3 can also preserve more parts on more single machines respectively
Copy.Embodiment illustrated in fig. 3 is by taking the tree of metadata includes above-mentioned multilayer as an example, in practical application, according to above-mentioned principle,
Metadata can be the tree for including the arbitrary number of plies, and no longer citing repeats one by one herein.
In the present embodiment, by storing the metadata of above-mentioned tree in the memory of single machine, FE nodes receive number
When according to request, metadata can be efficiently accessed from memory, to carry out data access according to metadata, can effectively be carried
The access efficiency and performance of high data.
102, according to the operation requests of metadata, corresponding daily record is generated, and be written in disk.
It is stored in memory by metadata in this present embodiment, for disk, inside saves as volatile storage
Medium, the data in memory can lose when single machine power down delay machine or corresponding process are killed, in order to ensure metadata energy
Enough to restore, in the present embodiment, any modification to metadata operates, and daily record (LOG) information can be serialized into, for remembering
Record content of the operation to metadata operation.In this way, for the operation requests of metadata each time, FE nodes can also be by this yuan
The corresponding LOG of operation requests of data is written in disk.If single machine power down delay machine or corresponding process are killed,
After internal storage data is lost, the metadata in memory can be restored according to the content of the operation recorded in the LOG of disk, to ensure
The persistence of metadata in memory, and then can ensure the access efficiency of metadata.
Step 101 and step 102 in the present embodiment can be not limited in sequence, in practical operation, it is necessary to assure
After step 101 and step 102 are performed both by, FE nodes just return to user and operate successful response message.
It should be noted that in the present embodiment, step 102 " according to the operation requests of metadata, corresponding daily record is generated,
And be written in disk " during, each LOG in disk is identified for clarity, can be each when generating LOG
A LOG configures a LOG mark (id), and LOG id ascending can be successively increased according to the generated time of LOG.In this way, magnetic
LOG in disk can identify all operations such as establishment, the modification of metadata in memory.All LOG phases i.e. in the disk
When in another form of metadata.
The management method of the metadata of the present embodiment, by receiving operation requests of the user to metadata；According to metadata
Operation requests, the metadata of the tree to being stored in memory modifies；According to the operation requests of metadata, generation pair
The daily record answered, and be written in disk.Compared with the metadata that the relation table using third normal form of the prior art stores, this implementation
The metadata of the management of example is stored in memory with tree, can ensure, when accessing, to avoid using a large amount of in this way
Join sentences can effectively improve the access efficiency of metadata.And in the present embodiment, the operation also according to metadata is asked
It asks, generates corresponding daily record, and disk is written, can be helped extensive when the corresponding process of metadata kills or clicks delay machine
Metadata in multiple memory avoids the risk that metadata is lost completely to ensure persistence and the safety of metadata.
Fig. 4 is the flow chart of the management method embodiment two of the metadata of the present invention.As shown in figure 4, the member of the present embodiment
The management method of data is further described the technical side of the present invention on the basis of technical solution of above-mentioned embodiment illustrated in fig. 2
Case.As shown in figure 4, the management method of the metadata of the present embodiment, can specifically include following steps：
200, when where metadata single machine delay machine restore after, ascending sequence is identified according to each daily record, successively from
Each daily record is read in disk；
201, it plays back the operation of each daily record successively in memory according to reading order, restores the metadata in memory.
The management method of the metadata of the present embodiment described in above-mentioned Fig. 2 on the basis of the technical solution of embodiment, is introduced
Metadata is after single machine delay machine or the corresponding process of metadata are killed, the recovery scheme of the metadata in memory.
Specifically, volatile storage medium is saved as due to interior, when single machine power down delay machine or corresponding process are killed,
Metadata in memory is all lost, can be according in above-mentioned embodiment illustrated in fig. 2 at this time to restore the metadata in memory
The step of the corresponding LOG of operation requests of each secondary metadata that is written in disk, to restore the metadata in memory, to ensure
The persistence of metadata in memory.
Specifically, according to the ascending incremental sequences of LOG id in disk, it is successively read each LOG.Often read one
The LOG can be deserialized as internal memory operation object by LOG, and re-execute the corresponding metadata behaviour of the LOG in memory
Make.By all executing one time the corresponding operations of all LOG in disk successively again, can regenerate in memory
A complete metadata, just ensure that the metadata information in memory will not be lost in this way.
The management method of the metadata of the present embodiment, after the single machine delay machine where metadata is restored, by according to each day
Will identifies ascending sequence, reads each daily record from disk successively；Each day is played back successively in memory according to reading order
The operation of will restores the metadata in memory, to which to ensure persistence and the safety of metadata, it is complete to avoid metadata
The risk of loss.
Fig. 5 is the flow chart of the management method embodiment three of the metadata of the present invention.As shown in figure 5, the member of the present embodiment
The management method of data is further described the technical side of the present invention on the basis of technical solution of above-mentioned embodiment illustrated in fig. 2
Case.As shown in figure 5, the management method of the metadata of the present embodiment, can specifically include following steps：
Whether the item number for 300, detecting the daily record in disk reaches predetermined threshold value；If reaching, step 301 is executed；Otherwise after
Continuous detection；
301, further judge to whether there is metadata mirror image in disk；If in the presence of step 302 is executed；Otherwise, if not depositing
Executing step 303；
302, the metadata mirror image in disk is updated according to the daily record of predetermined threshold value item number；Execute step 304；
303, metadata mirror image is generated according to the daily record of predetermined threshold value item number, and be stored in disk；Execute step 304；
304, the corresponding maximum daily record mark of record metadata mirror image；Execute step 305；
305, the corresponding each daily record of metadata mirror image is deleted.
Because over time, being recorded in disk in the way of the step 102 of above-mentioned embodiment illustrated in fig. 2
The item number of LOG can constantly add up, in this way, in single machine delay machine, according to the step 200 of above-mentioned embodiment illustrated in fig. 3 and 201
The time for restoring the metadata in memory according to the LOG in disk can increasingly be grown.In order to accelerate the time that delay machine restores metadata,
The mode that metadata mirror image may be used in the technical solution provided in the present embodiment realizes the storage of the LOG in disk, in order to
Fast quick-recovery when follow-up delay machine.
Specifically, " according to the operation requests of metadata, corresponding day is generated in the step 102 of above-mentioned embodiment illustrated in fig. 1
Will, and be written in disk " after, whether the item number for detecting the LOG in disk reaches predetermined threshold value.The predetermined threshold value of the present embodiment
It can be arranged according to actual demand, such as predetermined threshold value can be 1000 either 2000 or other integer item numbers.If
The item number of LOG in disk reaches predetermined threshold value, then it represents that can generate a metadata mirror according to the LOG of predetermined threshold value item number
Picture, instead of each LOG.If the item number of the LOG in disk reaches predetermined threshold value, can also further detect judge be in disk
It is no that there are metadata mirror images；It, can be on the basis of already present metadata mirror image, to predetermined threshold value item number if having existed
LOG according to the ascending sequence of each LOG id, execute the corresponding operations of each LOG successively, with more new metadata mirror image, directly
LOG to predetermined threshold value item number is all finished, and the metadata mirror image update in disk is completed.
And if in disk be not present metadata mirror image, directly according to the LOG of predetermined threshold value item number generate metadata mirror image, and
It is stored in disk；For generating process in addition to no already present metadata mirror image, remaining is similar with renewal process, no longer superfluous herein
It states.
In the manner described above, can by the corresponding operation notes of LOG of predetermined threshold value item number in metadata mirror image, in order to
The correspondence for identifying the metadata mirror image and LOG in disk, can also record the corresponding maximum of metadata mirror image in disk
LOG id can know from next after the corresponding maximum LOG id of the metadata mirror image in this way, when next update
The corresponding LOG of a LOG id starts more new metadata mirror image.Furthermore due to being stored with predetermined threshold value item in metadata mirror image
The corresponding operations of several LOG, it is repeated and misleading in order to avoid information, it is corresponding each that metadata mirror image can be deleted
LOG.The corresponding each daily record of the metadata mirror image can not certainly be deleted, subsequently according to each LOG id and metadata mirror image pair
The maximum LOG id answered determine which LOG corresponding informance is metadata mirror image correspondence include.
The management method of the metadata of the present embodiment can be in LOG quantity by using the technical solution of above-described embodiment
It when enough, avoids storing without several LOG in disk, but a plurality of LOG is stored by the way of metadata mirror image, with
Simplify the storage mode of the LOG in disk.
Fig. 6 is the flow chart of the management method example IV of the metadata of the present invention.As shown in fig. 6, the member of the present embodiment
The management method of data is further described the technical side of the present invention on the basis of technical solution of above-mentioned embodiment illustrated in fig. 5
Case.As shown in fig. 6, the management method of the metadata of the present embodiment, can specifically include following steps：
400, after the single machine delay machine where metadata is restored, existing metadata mirror image is read from disk；
401, the metadata mirror image read is loaded in memory；
402, it is identified in each daily record after corresponding daily record from the corresponding maximum daily record of metadata mirror image, according to each day
Will identifies ascending sequence, is successively read each daily record；
403, each daily record corresponding operation is played back on the basis of metadata mirror image successively according to reading order, and then inside
Deposit middle recovery metadata.
Technical solution is specifically described and is adopted on the basis of the technical solution of above-mentioned embodiment illustrated in fig. 5 in the present embodiment
After the storage for realizing the LOG in memory with the mode of memory mirror, after the recovery of single machine delay machine, how in fast quick-recovery memory
Metadata.
Specifically, which can create a Checkpoint thread, when the single machine delay machine where metadata is restored
Afterwards, Checkpoint threads can first read existing metadata mirror image (image) from disk；And the metadata that will be read
Mirror image loads in memory.If then there is corresponding LOG in disk after the corresponding maximum LOG id of metadata mirror image, this
When Checkpoint threads continue from each LOG after the corresponding LOG of the corresponding maximum LOG id of the metadata mirror image of disk
In, according to the sequence that each LOG id are ascending, it is successively read each LOG；Then according to reading order successively in metadata mirror image
On the basis of play back the corresponding operations of each LOG, and then restore complete metadata in memory.The technical solution of the present embodiment with
Above-mentioned embodiment illustrated in fig. 4 is compared, and a large amount of times for re-executing LOG operation contents can be effectively saved, to significantly carry
Metadata resume speed after high delay machine, is effectively improved the recovery efficiency of metadata.
First number may be used by using the technical solution of above-described embodiment in the management method of the metadata of the present embodiment
The storage that the LOG in disk is realized according to the mode of mirror image, in order to quick according to the metadata mirror image in disk when follow-up delay machine
Restore the metadata in memory, so as to effectively shorten the time of the recovery of the metadata after delay machine, improves the member after delay machine
Data resume speed improves the recovery efficiency of metadata.
Based on the above, it is recognised that the management method of the metadata of the present invention, can make the metadata in memory
At least there is following performance：
1) efficient access of metadata, the metadata in the embodiment of the present invention are stored entirely in memory, and using high
Data structure, that is, tree structure of effect is stored, and access efficiency is greatly improved；
2) persistence simultaneously can operate each time although the metadata storage in the embodiment of the present invention is in memory
Content is stored on disk, to restore after delay machine, to ensure that metadata will not lose.
Still optionally further, the metadata that the metadata management method of the embodiment of the present invention is managed can also have such as
Lower performance：
3) High Availabitity can be by the multiple copies of meta-data preservation shown in the afterbody of metadata structure as shown in Figure 3
On different single machines.In this way, after a wherein single machine delay machine or thoroughly damage, user still can be by other single
Machine node continues to access complete metadata.
Corresponding at this time, the operation that the backup FE nodes that current FE nodes can be on other single machines send metadata is asked
Corresponding LOG is sought, for being played back on the basis of metadata of other backups FE nodes on other single machines in corresponding memory
The corresponding operations of LOG, to synchronize the metadata on other single machines.
For example, can introduce bdbje components in the embodiment of the present invention, bdbje components are that developed by oracle one is embedding
Enter the distributed kv databases of formula.Using the characteristic of its more copy strongly consistent, realize that the height of the metadata of the embodiment of the present invention can
With.
Fig. 7 is the scene graph of the High Availabitity of the metadata of the present invention.As shown in fig. 7, FE nodes are written by each LOG
When disk, it is also necessary to by each LOG, all be written in bdbje components, bdbje itself can ensure that the persistence of data is written.
Meanwhile the LOG can be synchronized in the bdbje component copies on more FE nodes by bdbje.Wherein more FE nodes distinguish subordinate
On different single machines.The FE nodes that will be responsible for write-in LOG to bdbje components are known as Leader, and remaining FE node is known as
Follower.As shown in fig. 7, including a Leader with system, for two Follower, Follower can be monitored respectively
LOG variations on bdbje copies.After having new LOG to reach, Follower can play back this LOG at once, to ensure
Metadata information in Follower memories is synchronous in Leader.Metadada IN MEM indicate to store in memory in Fig. 7
Metadata, Checkpoint.10 and Checkpoint.13 indicate mirror image (image) file in disk respectively.LOG.11、
LOG.12, LOG.13 and LOG.14 respectively indicate the LOG stored in disk.
After Leader delay machines, system can elect new Leader from remaining Follower automatically, to ensure member
The lasting progress of data access flow.
In this way, even if single machine delay machine where one or more FE node and irrecoverable, can still be deposited by residue
Complete metadata information is accessed on FE nodes living.Meanwhile the single machine node of delay machine can also pass through in recovery process
Other surviving nodes, uninterrupted access metadata, to be effectively guaranteed the high availability of metadata.
4) support autgmentability, based on it is above-mentioned 3) in high availability, there may be a Leader and multiple in system
Follower, in this way, Metadata Service can access multiple Follower by extending transversely, so as to support service
Autgmentability effectively improves service processing ability.
Fig. 8 is the structure chart of the front end node apparatus embodiments one of the present invention.As shown in figure 8, the preceding end segment of the present embodiment
Point device can specifically include：
Receiving module 10 is for receiving operation requests of the user to metadata；
The operation requests for the metadata that metadata processing module 11 is used to be received according to receiving module 10, to being stored in memory
The metadata of tree modify；
The operation requests for the metadata that log processing module 12 is used to be received according to receiving module 10, generate corresponding day
Will, and be written in disk.
The front end node equipment of the present embodiment, by using above-mentioned module realize metadata management realization principle and
Technique effect is identical as the realization of above-mentioned related method embodiment, can refer to the record of above-mentioned related method embodiment in detail,
Details are not described herein.
Fig. 9 is the structure chart of the front end node apparatus embodiments two of the present invention.As shown in figure 9, the preceding end segment of the present embodiment
Point device can also include further following technical solution on the basis of the technical solution of above-mentioned embodiment illustrated in fig. 8.
As shown in figure 9, the front end node equipment of the present embodiment, can also include：
Read module 13 is used to after single machine delay machine recovery where metadata, identify according to each daily record ascending suitable
Sequence reads each daily record that log processing module 12 is written from disk successively；Each daily record mark according to daily record generated time by
Small reach is incremented by；
Recovery module 14 is used to play back the operation of each daily record successively in memory according to 13 reading order of read module, restores
Metadata in memory.
Still optionally further, as shown in figure 9, the front end node equipment of the present embodiment, can also include：
Whether the item number that detection module 15 is used to detect the daily record that log processing module 12 is written in disk reaches default threshold
Value；
If the item number that detection module 15 is additionally operable to the daily record in disk reaches predetermined threshold value, further judge in disk whether
There are metadata mirror images；
If log processing module 12 for detection module 15 detects disk, there are metadata mirror images, according to predetermined threshold value item
Metadata mirror image in several daily record update disks.
Still optionally further, in the front end node equipment of the present embodiment, log processing module 12 is additionally operable to：
If detection module 15 detects in disk there is no when metadata mirror image, generated according to the daily record of predetermined threshold value item number
Metadata mirror image, and be stored in disk.
Still optionally further, as shown in figure 9, the front end node equipment of the present embodiment, further includes：
Logging modle 16 is for the corresponding maximum daily record mark of metadata mirror image that record log processing module 12 is handled；
Removing module 17 is used to delete log processing module 12 and handles the obtained corresponding each daily record of metadata mirror image.
Still optionally further, as shown in figure 9, the front end node equipment of the present embodiment, further includes：Load-on module 18.
Read module 13 is additionally operable to that after the single machine delay machine where metadata is restored, log processing module is read from disk
The existing metadata mirror image of 12 processing；
Load-on module 18 is for the metadata mirror image that load read module 13 is read in memory；
Read module 13 is additionally operable to identify each day after corresponding daily record from the corresponding maximum daily record of metadata mirror image
In will, ascending sequence is identified according to each daily record, is successively read each daily record；
Recovery module 14 is additionally operable to play back on the basis of metadata mirror image successively according to the reading order of read module 13
The corresponding operation of each daily record, and then restore metadata in memory.
Still optionally further, as shown in figure 9, the front end node equipment of the present embodiment, further includes：
Sending module 19 sends first number that log processing module 12 generates for the backup front end node on other single machines
According to the corresponding daily record of operation requests, for other metadata of backup front end node in corresponding memory on other single machines
On the basis of the corresponding operation of playback daily record, to synchronize the metadata on other single machines.
The front end node equipment of the present embodiment, by using above-mentioned module realize metadata management realization principle and
Technique effect is identical as the realization of above-mentioned related method embodiment, can refer to the record of above-mentioned related method embodiment in detail,
Details are not described herein.
Figure 10 is the structure chart of the computer equipment embodiment of the present invention.As shown in Figure 10, the computer of the present embodiment is set
It is standby, including：One or more processors 30 and memory 40, memory 40 work as storage for storing one or more programs
The one or more programs stored in device 40 are executed by one or more processors 30 so that one or more processors 30 are realized
Such as the management method of the metadata of figure 1 above-embodiment illustrated in fig. 7.To include that multiple processors 30 are in embodiment illustrated in fig. 10
Example.
For example, Figure 11 is a kind of exemplary plot of computer equipment provided by the invention.Figure 11 is shown suitable for being used for realizing
The block diagram of the exemplary computer device 12a of embodiment of the present invention.The computer equipment 12a that Figure 11 is shown is only one and shows
Example, should not bring any restrictions to the function and use scope of the embodiment of the present invention.
As shown in figure 11, computer equipment 12a is showed in the form of universal computing device.The component of computer equipment 12a
It can include but is not limited to：One or more processor 16a, system storage 28a, connection different system component (including be
Unite memory 28a and processor 16a) bus 18a.
Bus 18a indicates one or more in a few class bus structures, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor or the local bus using the arbitrary bus structures in a variety of bus structures.It lifts
For example, these architectures include but not limited to industry standard architecture (ISA) bus, microchannel architecture (MAC)
Bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) bus.
Computer equipment 12a typically comprises a variety of computer system readable media.These media can be it is any can
The usable medium accessed by computer equipment 12a, including volatile and non-volatile media, moveable and immovable Jie
Matter.
System storage 28a may include the computer system readable media of form of volatile memory, such as deposit at random
Access to memory (RAM) 30a and/or cache memory 32a.Computer equipment 12a may further include it is other it is removable/
Immovable, volatile/non-volatile computer system storage medium.Only as an example, storage system 34a can be used for reading
Write immovable, non-volatile magnetic media (Figure 11 do not show, commonly referred to as " hard disk drive ").Although not showing in Figure 11
Go out, can provide for the disc driver to moving non-volatile magnetic disk (such as " floppy disk ") read-write, and to removable
The CD drive of anonvolatile optical disk (such as CD-ROM, DVD-ROM or other optical mediums) read-write.In these cases,
Each driver can be connected by one or more data media interfaces with bus 18a.System storage 28a may include
There is one group of (for example, at least one) program module, these program modules to be configured at least one program product, the program product
To execute the function of the above-mentioned each embodiments of Fig. 1-Fig. 9 of the present invention.
Program with one group of (at least one) program module 42a/utility 40a can be stored in such as system and deposit
In reservoir 28a, such program module 42a include --- but being not limited to --- operating system, one or more application program,
Other program modules and program data may include the reality of network environment in each or certain combination in these examples
It is existing.Program module 42a usually executes the function and/or method in above-mentioned each embodiments of Fig. 1-Fig. 9 described in the invention.
Computer equipment 12a can also be with one or more external equipment 14a (such as keyboard, sensing equipment, display
24a etc.) communication, the equipment interacted with computer equipment 12a communication can be also enabled a user to one or more, and/or
(such as network interface card is adjusted with any equipment that computer equipment 12a communicated with one or more of the other computing device is enable
Modulator-demodulator etc.) communication.This communication can be carried out by input/output (I/O) interface 22a.Also, computer equipment
12a can also by network adapter 20a and one or more network (such as LAN (LAN), wide area network (WAN) and/or
Public network, such as internet) communication.As shown, network adapter 20a by bus 18a and computer equipment 12a its
Its module communicates.It should be understood that although not shown in the drawings, other hardware and/or software can be used in conjunction with computer equipment 12a
Module, including but not limited to：Microcode, device driver, redundant processor, external disk drive array, RAID system, tape
Driver and data backup storage system etc..
Processor 16a is stored in program in system storage 28a by operation, to perform various functions application and
Data processing, such as realize the management method of metadata shown in above-described embodiment.
The present invention also provides a kind of computer-readable mediums, are stored thereon with computer program, which is held by processor
The management method of the metadata as shown in above-described embodiment is realized when row.
The computer-readable medium of the present embodiment may include in the system storage 28a in above-mentioned embodiment illustrated in fig. 11
RAM30a, and/or cache memory 32a, and/or storage system 34a.
With the development of science and technology, the route of transmission of computer program is no longer limited by tangible medium, it can also be directly from net
Network is downloaded, or is obtained using other modes.Therefore, the computer-readable medium in the present embodiment may include not only tangible
Medium can also include invisible medium.
The arbitrary combination of one or more computer-readable media may be used in the computer-readable medium of the present embodiment.
Computer-readable medium can be computer-readable signal media or computer readable storage medium.Computer-readable storage medium
Matter for example may be-but not limited to-system, device or the device of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or
The arbitrary above combination of person.The more specific example (non exhaustive list) of computer readable storage medium includes：There are one tools
Or the electrical connections of multiple conducting wires, portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM),
Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light
Memory device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer readable storage medium can
With to be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
Person is in connection.
Computer-readable signal media may include in a base band or as the data-signal that a carrier wave part is propagated,
Wherein carry computer-readable program code.Diversified forms may be used in the data-signal of this propagation, including --- but
It is not limited to --- electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be
Any computer-readable medium other than computer readable storage medium, which can send, propagate or
Transmission for by instruction execution system, device either device use or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited
In --- wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
It can be write with one or more programming languages or combinations thereof for executing the computer that operates of the present invention
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
Further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partly executes or executed on a remote computer or server completely on the remote computer on the user computer.
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including LAN (LAN) or
Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as carried using Internet service
It is connected by internet for quotient).
In several embodiments provided by the present invention, it should be understood that disclosed system, device and method can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of division of logic function, formula that in actual implementation, there may be another division manner.
The unit illustrated as separating component may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, you can be located at a place, or may be distributed over multiple
In network element.Some or all of unit therein can be selected according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it can also
It is that each unit physically exists alone, it can also be during two or more units be integrated in one unit.Above-mentioned integrated list
The form that hardware had both may be used in member is realized, can also be realized in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can be stored in one and computer-readable deposit
In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer
It is each that equipment (can be personal computer, server or the network equipment etc.) or processor (processor) execute the present invention
The part steps of embodiment the method.And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (Read-
Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disc or CD etc. it is various
The medium of program code can be stored.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
With within principle, any modification, equivalent substitution, improvement and etc. done should be included within the scope of protection of the invention god.
Claims (16)
1. a kind of management method of metadata, which is characterized in that the method includes：
Receive operation requests of the user to metadata；
It is modified according to the metadata of the operation requests of the metadata, the tree to being stored in memory；
And according to the operation requests of the metadata, corresponding daily record is generated, and be written in disk.
2. according to the method described in claim 1, it is characterized in that, the method further includes：
When where the metadata single machine delay machine restore after, identify ascending sequence according to each daily record, successively from
Each daily record is read in the disk；Each daily record mark is incremented by according to the generated time of the daily record by small reach；
And play back the operation of each daily record successively in the memory according to reading order, restore the member in the memory
Data.
3. according to the method described in claim 1, it is characterized in that, the method further includes：
Whether the item number for detecting the daily record in the disk reaches predetermined threshold value；
If reaching, further judge to whether there is metadata mirror image in the disk；
If in the presence of the metadata mirror image in the disk is updated according to the daily record of the predetermined threshold value item number.
4. according to the method described in claim 3, it is characterized in that, the method further includes：
If metadata mirror image is not present in the disk, the metadata mirror is generated according to the daily record of the predetermined threshold value item number
Picture, and be stored in the disk.
5. according to the method described in claim 3 or 4, which is characterized in that the method further includes：
Record the corresponding maximum daily record mark of the metadata mirror image；
Delete the corresponding each daily record of the metadata mirror image.
6. according to the method described in claim 5, it is characterized in that, the method further includes：
After the single machine delay machine where the metadata is restored, the existing metadata mirror image is read from the disk；
The metadata mirror image read is loaded in the memory；
From each daily record that the corresponding maximum daily record of the metadata mirror image identifies after corresponding daily record, according to each institute
It states daily record and identifies ascending sequence, be successively read each daily record；
And play back the corresponding operation of each daily record, Jin Er on the basis of the metadata mirror image successively according to reading order
Restore the metadata in the memory.
7. according to any method of claim 1-4 and 6, which is characterized in that the method further includes：
Backup front end node on other single machines sends the corresponding daily record of operation requests of the metadata, for described
It is corresponding that the daily record is played back on the basis of metadata in corresponding memory of other backup front end nodes on other single machines
Operation, to synchronize the metadata on other described single machines.
8. a kind of front end node equipment, which is characterized in that the equipment includes：
Receiving module, for receiving operation requests of the user to metadata；
Metadata processing module, for the operation requests according to the metadata, first number to the tree stored in memory
According to modifying；
Log processing module generates corresponding daily record, and be written in disk for the operation requests according to the metadata.
9. equipment according to claim 8, which is characterized in that the equipment further includes：
Read module, for after the single machine delay machine where the metadata is restored, being identified according to each daily record ascending
Sequence, each daily record is read from the disk successively；Each daily record mark according to the daily record generated time by
Small reach is incremented by；
Recovery module, the operation for playing back each daily record successively in the memory according to reading order restore described interior
The metadata in depositing.
10. equipment according to claim 9, which is characterized in that the equipment further includes：
Whether detection module, the item number for detecting the daily record in the disk reach predetermined threshold value；
The detection module further judges institute if the item number for the daily record being additionally operable in the disk reaches the predetermined threshold value
It states and whether there is metadata mirror image in disk；
The log processing module, if there are the metadata mirror images for the disk, according to the predetermined threshold value item number
Daily record updates the metadata mirror image in the disk.
11. equipment according to claim 10, which is characterized in that the log processing module is additionally operable to：
If metadata mirror image is not present in the disk, the metadata mirror is generated according to the daily record of the predetermined threshold value item number
Picture, and be stored in the disk.
12. according to the equipment described in claim 10 or 11, which is characterized in that the equipment further includes：
Logging modle, for recording the corresponding maximum daily record mark of the metadata mirror image；
Removing module, for deleting the corresponding each daily record of the metadata mirror image.
13. equipment according to claim 12, which is characterized in that the equipment further includes load-on module；
The read module is additionally operable to after the single machine delay machine where the metadata is restored, read and have from the disk
The metadata mirror image；
The load-on module, for loading the metadata mirror image read in the memory；
The read module, be additionally operable to from the corresponding maximum daily record of the metadata mirror image identify corresponding daily record after it is each
In the daily record, ascending sequence is identified according to each daily record, is successively read each daily record；
The recovery module is additionally operable to play back each daily record on the basis of the metadata mirror image successively according to reading order
Corresponding operation, and then restore the metadata in the memory.
14. according to any equipment of claim 8-11 and 13, which is characterized in that the equipment further includes：
Sending module, the operation requests that the metadata is sent for the backup front end node on other single machines are corresponding described
Daily record, for playing back institute on the basis of metadata of other backup front end nodes on other described single machines in corresponding memory
The corresponding operation of daily record is stated, to synchronize the metadata on other described single machines.
15. a kind of computer equipment, which is characterized in that the equipment includes：
One or more processors；
Memory, for storing one or more programs；
When one or more of programs are executed by one or more of processors so that one or more of processors are real
The now method as described in any in claim 1-7.
16. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that the program is executed by processor
Methods of the Shi Shixian as described in any in claim 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810150300.6A CN108427728A (en) | 2018-02-13 | 2018-02-13 | Management method, equipment and the computer-readable medium of metadata |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810150300.6A CN108427728A (en) | 2018-02-13 | 2018-02-13 | Management method, equipment and the computer-readable medium of metadata |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108427728A true CN108427728A (en) | 2018-08-21 |
Family
ID=63157049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810150300.6A Pending CN108427728A (en) | 2018-02-13 | 2018-02-13 | Management method, equipment and the computer-readable medium of metadata |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108427728A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109521963A (en) * | 2018-11-14 | 2019-03-26 | 郑州云海信息技术有限公司 | A kind of metadata rule method and device |
| CN109947733A (en) * | 2019-03-29 | 2019-06-28 | 众安信息技术服务有限公司 | Data storage device and method |
| CN110134550A (en) * | 2019-05-15 | 2019-08-16 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and computer readable storage medium |
| CN110286850A (en) * | 2019-05-15 | 2019-09-27 | 青岛镕铭半导体有限公司 | Wiring method, restoration methods and the solid state hard disk of solid state hard disk metadata |
| CN112000623A (en) * | 2020-08-07 | 2020-11-27 | 北京浪潮数据技术有限公司 | Metadata access method and device and computer readable storage medium |
| CN113032349A (en) * | 2019-12-25 | 2021-06-25 | 阿里巴巴集团控股有限公司 | Data storage method and device, electronic equipment and computer readable medium |
| CN113901276A (en) * | 2021-09-30 | 2022-01-07 | 苏州浪潮智能科技有限公司 | Data management method, B + tree loading method, device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567445A (en) * | 2011-10-25 | 2012-07-11 | 无锡城市云计算中心有限公司 | Method for guaranteeing consistency of metadata in distributed file system |
| CN102662795A (en) * | 2012-03-20 | 2012-09-12 | 浪潮电子信息产业股份有限公司 | Metadata fault-tolerant recovery method in distributed storage system |
| US20140040698A1 (en) * | 2012-08-06 | 2014-02-06 | Advanced Micro Devices, Inc. | Stacked memory device with metadata mangement |
| CN103795801A (en) * | 2014-02-12 | 2014-05-14 | 浪潮电子信息产业股份有限公司 | Metadata group design method based on real-time application group |
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN107357688A (en) * | 2017-07-28 | 2017-11-17 | 广东神马搜索科技有限公司 | Distributed system and its fault recovery method and device |
| CN107402870A (en) * | 2017-07-31 | 2017-11-28 | 郑州云海信息技术有限公司 | The processing method and processing device of daily record section in a kind of meta data server |
-
2018
- 2018-02-13 CN CN201810150300.6A patent/CN108427728A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102567445A (en) * | 2011-10-25 | 2012-07-11 | 无锡城市云计算中心有限公司 | Method for guaranteeing consistency of metadata in distributed file system |
| CN102662795A (en) * | 2012-03-20 | 2012-09-12 | 浪潮电子信息产业股份有限公司 | Metadata fault-tolerant recovery method in distributed storage system |
| US20140040698A1 (en) * | 2012-08-06 | 2014-02-06 | Advanced Micro Devices, Inc. | Stacked memory device with metadata mangement |
| CN103795801A (en) * | 2014-02-12 | 2014-05-14 | 浪潮电子信息产业股份有限公司 | Metadata group design method based on real-time application group |
| CN103902479A (en) * | 2014-03-27 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Quick reconstruction mechanism for metadata cache on basis of metadata log |
| CN107357688A (en) * | 2017-07-28 | 2017-11-17 | 广东神马搜索科技有限公司 | Distributed system and its fault recovery method and device |
| CN107402870A (en) * | 2017-07-31 | 2017-11-28 | 郑州云海信息技术有限公司 | The processing method and processing device of daily record section in a kind of meta data server |
Non-Patent Citations (3)
| Title |
|---|
| 贾志淳: "《移动云计算技术专题研究》", 30 April 2016 * |
| 陈晓晖: "《档案网站建设》", 28 February 2014 * |
| 黄维勤: "《数据密集型计算和模型》", 31 January 2015 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109521963A (en) * | 2018-11-14 | 2019-03-26 | 郑州云海信息技术有限公司 | A kind of metadata rule method and device |
| CN109521963B (en) * | 2018-11-14 | 2021-08-10 | 郑州云海信息技术有限公司 | Metadata dropping method and device |
| CN109947733A (en) * | 2019-03-29 | 2019-06-28 | 众安信息技术服务有限公司 | Data storage device and method |
| CN110134550A (en) * | 2019-05-15 | 2019-08-16 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and computer readable storage medium |
| CN110286850A (en) * | 2019-05-15 | 2019-09-27 | 青岛镕铭半导体有限公司 | Wiring method, restoration methods and the solid state hard disk of solid state hard disk metadata |
| CN110286850B (en) * | 2019-05-15 | 2023-05-09 | 镕铭微电子(济南)有限公司 | Writing method and recovery method of metadata of solid state disk and solid state disk |
| CN113032349A (en) * | 2019-12-25 | 2021-06-25 | 阿里巴巴集团控股有限公司 | Data storage method and device, electronic equipment and computer readable medium |
| CN112000623A (en) * | 2020-08-07 | 2020-11-27 | 北京浪潮数据技术有限公司 | Metadata access method and device and computer readable storage medium |
| CN113901276A (en) * | 2021-09-30 | 2022-01-07 | 苏州浪潮智能科技有限公司 | Data management method, B + tree loading method, device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427728A (en) | Management method, equipment and the computer-readable medium of metadata | |
| US7350052B2 (en) | Network system and method for setting volume group in the network system | |
| CN102158540A (en) | System and method for realizing distributed database | |
| CN109086388A (en) | Block chain date storage method, device, equipment and medium | |
| JP2016505930A (en) | System, method, and apparatus for collaborative CAX editing | |
| CN106062742A (en) | Systems and methods for improving snapshot performance | |
| WO2019231526A1 (en) | Persistent version control for data transfer between heterogeneous data stores | |
| CN109471851A (en) | Data processing method, device, server and storage medium | |
| US9031909B2 (en) | Provisioning and/or synchronizing using common metadata | |
| US20200364239A1 (en) | Asynchronous replication of in-scope table data | |
| CN104750755B (en) | A kind of data covering method and system after database active-standby switch | |
| CN108255855A (en) | Date storage method and device | |
| CN113806300B (en) | Data storage method, system, device, equipment and storage medium | |
| CN107818104A (en) | A kind of database portion arranging method, database deployment device, control device and system | |
| CN107729536A (en) | A kind of date storage method and device | |
| CN107580032A (en) | Data processing method, device and equipment | |
| CN107291876A (en) | A kind of DDM method | |
| CN109684298A (en) | Database migration method, apparatus, equipment and storage medium | |
| CN104899161A (en) | Cache method based on continuous data protection of cloud storage environment | |
| CN102890651B (en) | The method of testing of a kind of contextual data and device | |
| WO2007099636A1 (en) | File system migration method, program and apparatus | |
| US20190243807A1 (en) | Replication of data in a distributed file system using an arbiter | |
| CN104517067A (en) | Method, device and system for data access | |
| CN110019527B (en) | Slave library reading method, related device and equipment | |
| CN104951528B (en) | A kind of method and terminal of Data Migration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |