CN104967639A - Information reporting method, information reporting apparatus and data node server - Google Patents
Information reporting method, information reporting apparatus and data node server Download PDFInfo
- Publication number
- CN104967639A CN104967639A CN201410364737.1A CN201410364737A CN104967639A CN 104967639 A CN104967639 A CN 104967639A CN 201410364737 A CN201410364737 A CN 201410364737A CN 104967639 A CN104967639 A CN 104967639A
- Authority
- CN
- China
- Prior art keywords
- server
- metadata information
- back end
- data block
- meta data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides an information reporting method, an information reporting apparatus and a data node server, and is used for reducing the time-delay for meta-data information synchronization between a data node server and a meta-data server, and reducing the burden which is brought by reporting of large amount of meta-data information to the meta-data server. The method comprises determining the meta-data information, of a data block, to be reported to the meta-data server when there is requirement for reporting the meta-data information of the data block to the meta-data server, wherein if it is the first time for a data node server to report the meta-data information of the data block, the reported meta-data information of the data block includes the meta-data information of all the data blocks in the data node server, and if it is not the first time for the data node server to report the meta-data information of the data block, the reported meta-data information of the data block is the meta-data information of part of the data blocks in the data node server; and reporting the reported meta-data information of the data blocks to the meta-data server by the data node server.
Description
Technical field
The present invention relates to communication technical field, particularly relate to a kind of information uploading method, device and back end server.
Background technology
Distributed object file system, for managing mass memory resource, to provide at a high speed, the service of stable, linear expansion.In distributed object file system, each file is divided into multiple object (Object), each object is further subdivided into N number of data block (Block), and this N number of data block is stored on N number of mutually different back end server respectively.
The structure of distributed file system as shown in Figure 1.Wherein, back end server (being called for short DataNode) is for storing data block, and the metadata information of these data blocks.Meta data server (is called for short Metadata Server, MDS) for the metadata information of stored data base, namely described metadata information for the information of data of description block, specifically comprises: the information of side-play amount of the size of data block, the index of data block and data block and so on.
In existing distributed file system, when back end server is registered to meta data server, consistent for ensureing the metadata information of the data block between back end server and meta data server, back end server need will be positioned at the metadata information Complete Synchronization of all data blocks of this back end server to meta data server, and the full dose that namely back end server carries out the metadata information of data block to meta data server reports.
When back end server is registered to meta data server, need to metadata information corresponding to all data blocks on meta data server reported data node server, due to each back end server store a large amount of metadata information, therefore, when having more back end server to register to meta data server simultaneously, or back end server to meta data server registration, can carry out the synchronous of a large amount of metadata informations frequently.
When meta data server starts or active and standby meta data server carries out active-standby switch, the back end server associated with meta data server can be unified to initiate to register to meta data server, due to the immense pressure of offered load and meta data server process, metadata information synchronously will expend the longer time, and, before metadata information synchronously completes, meta data server outwardly cannot provide the read-write service on basis.
When the metadata information of the data block that back end server and meta data server store is inconsistent, meta data server needs to carry out a large amount of comparisons to the metadata information of data block, with the consistency of the metadata information of service data block, a large amount of metadata informations synchronously also can give the CPU of meta data server, internal memory brings very big burden.Wherein, about meta data server, a large amount of comparisons is carried out to the metadata information of data block, such as: back end server rolls off the production line, then pull out the disk of back end server, make like this on back end server, the metadata information of data block likely can be lost.But, on meta data server, the metadata information of data block or complete.Therefore, need the comparison of the metadata information carrying out data block, whether the metadata information of the data block namely on comparison all back end server is consistent with the metadata information of the data block that meta data server stores.If inconsistent, then can carry out recovery operation or the deletion action of data block.
In addition, when back end server is registered to meta data server, a large amount of metadata informations synchronously significantly increases network burden, and back end server may because offered load be too high, and the lose network of short duration with meta data server is connected.When back end server and meta data server network reconnect, to again register to meta data server, again corresponding to data blocks all on this back end server metadata information carries out full dose and reports, offered load will be strengthened further, cause back end server timeout to roll off the production line.Like this, vicious circle will be caused.
In sum, in prior art, between back end server and meta data server, synchronizing metadata information required time is longer, and a large amount of metadata information reports and can bring great burden to meta data server, and whole offered load also can be made excessive.
Summary of the invention
Embodiments provide a kind of method of information reporting, device and back end server, in order to reduce the time delay of synchronizing metadata information between back end server and meta data server, and reduce a large amount of metadata information and report the burden that meta data server brings.
The method of a kind of information reporting that the embodiment of the present invention provides, be applied in distributed object file system, this system comprises back end server and meta data server, and the method comprises:
Back end server, when needing the metadata information to meta data server reported data block, determines the metadata information needing the data block reported to meta data server; Wherein, if described back end server needs the metadata information to described meta data server reported data block first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, if described back end server is not need the metadata information to described meta data server reported data block first, then the metadata information of data block that described needs report comprises the metadata information not reporting the data block of meta data server in this back end server;
The metadata information of the data block that described needs report is reported meta data server by back end server.
Pass through the method, back end server optionally reports metadata information to meta data server, thus reduce a large amount of metadata information and report the burden that meta data server brings, and reduce the time delay of synchronizing metadata information between back end server and meta data server.
Preferably, if described back end server is not need the metadata information to described meta data server reported data block first, then described back end server determines the metadata information needing the data block reported to meta data server, comprising:
Described back end server, according to the index of the data block of the current record corresponding with described meta data server, determines that the metadata information of the data block that current needs report to described meta data server is for the metadata information according to the data block corresponding to the index of index order after the index of current record; Wherein, the index of described current record, for the described back end server last time reports in the metadata information of the data block of described meta data server according to the index of index order in last data block.
Preferably, described index is numeral; Described index order is numeral order sequence from small to large.
Preferably, the index of the data block of current record, is recorded in described back end server.
Preferably, the index of the data block of described current record, is recorded in described meta data server;
Described back end server, when needing the metadata information to described meta data server reported data block, also comprises:
Described back end server sends the request of asking for the index of the data block of the current record corresponding with described meta data server to described meta data server, and receives the index of the data block of the current record corresponding with described meta data server that described meta data server returns.
Preferably, described back end server determines the metadata information of current needs to described meta data server reported data block in the following way:
Described back end server, when monitoring disk plug information, determines the metadata information of current needs to described meta data server reported data block.
Pass through the method, node server is when monitoring disk plug information, to the metadata information of the data block corresponding to the index after meta data server increment reports the index of current record, thus under disk plugging condition, decrease the number of times that full dose reports, reduction full dose reports the burden to network.
Preferably, the metadata information of all data blocks in this back end server, according to predetermined period, is reported described meta data server by described back end server; And/or,
The metadata information of all data blocks in this back end server, when receiving the Trigger message that described meta data server sends, is reported described meta data server by described back end server.
Pass through the method, node server reports the metadata information of the data block of this node server to meta data server cycle full dose, or when meta data server monitors this meta data server and node server metadata information is inconsistent, forced node server full dose reports node server metadata information, thus avoids increment and report the meta data server that causes and the inconsistent problem of back end server data.
A kind of information reporting device that the embodiment of the present invention provides, be applied in distributed object file system, this system comprises back end server and meta data server, and this device comprises:
Determine metadata information unit, for when back end server needs the metadata information to meta data server reported data block, determine the metadata information needing the data block reported to meta data server; Wherein, if described back end server needs the metadata information to described meta data server reported data block first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, if described back end server is not need the metadata information to described meta data server reported data block first, then the metadata information of data block that described needs report comprises the metadata information not reporting the data block of meta data server in this back end server;
Report metadata information unit, the metadata information for the data block reported by described needs reports meta data server.
By this device, back end server optionally reports metadata information to meta data server, thus reduce a large amount of metadata information and report the burden that meta data server brings, and reduce the time delay of synchronizing metadata information between back end server and meta data server.
Preferably, if described back end server is not need the metadata information to described meta data server reported data block first, then described when determining that metadata information unit determines the metadata information needing the data block reported to meta data server, specifically for:
Determine the index of metadata information unit according to the data block of the current record corresponding with described meta data server, determine that the metadata information of the data block that current needs report to described meta data server is for the metadata information according to the data block corresponding to the index of index order after the index of current record; Wherein, the index of described current record, for the described back end server last time reports in the metadata information of the data block of described meta data server according to the index of index order in last data block.
Preferably, described index is numeral; Described index order is numeral order sequence from small to large.
Preferably, the index of the data block of described current record, is recorded in described back end server.
Preferably, the index of the data block of described current record, is recorded in described meta data server;
When described back end server needs the metadata information to described meta data server reported data block, described in report metadata information unit also for:
Send the request of asking for the index of the data block of the current record corresponding with described meta data server to described meta data server, and receive the index of the data block of the current record corresponding with described meta data server that described meta data server returns.
Preferably, describedly determine that metadata information unit determines the metadata information of current needs to described meta data server reported data block in the following way:
Described back end server, when monitoring the disk plug information of described back end server, determines the metadata information of current needs to described meta data server reported data block.
By this device, node server is when monitoring disk plug information, to the metadata information of the data block corresponding to the index after meta data server increment reports the index of current record, thus under disk plugging condition, decrease the number of times that full dose reports, reduction full dose reports the burden to network.
Preferably, described in report metadata information unit according to predetermined period, the metadata information of all data blocks in this back end server is reported described meta data server; And/or,
The metadata information of all data blocks in this back end server, when receiving the Trigger message that described meta data server sends, is reported described meta data server by the described metadata information unit that reports.
What the embodiment of the present invention provided is applied to back end server in distributed object file system, comprises above-mentioned arbitrary described information reporting device.
Accompanying drawing explanation
Fig. 1 is the structural representation of existing distributed file system;
The schematic flow sheet of a kind of information uploading method that Fig. 2 provides for the embodiment of the present invention;
Fig. 3 is that the embodiment of the present invention is for the corresponding relation schematic diagram between different situations metadata information reporting schemes;
Fig. 4 be in the embodiment of the present invention back end server to a kind of idiographic flow schematic diagram of the metadata information of meta data server reported data block;
Fig. 5 be in the embodiment of the present invention back end server to the another kind of idiographic flow schematic diagram of the metadata information of meta data server reported data block;
The structural representation of a kind of information reporting device that Fig. 6 provides for the embodiment of the present invention.
Embodiment
Embodiments provide a kind of method of information reporting, device and back end server, roll off the production line in order to solve of short duration time-out, the drawback that back end server reports metadata information to cause to meta data server full dose, and the inconsistent full dose of carrying out of the metadata information of back end server and meta data server reports the problem caused.
In the technical scheme that the embodiment of the present invention provides, when back end server is registered to meta data server first, report the metadata information of all data blocks on this back end server to meta data server, and record reports in the metadata information of this meta data server according to the index startIndex of index order in last data block.
When back end server is non-register to meta data server first time, back end server is by the index startIndex of the data block according to the current record corresponding with this meta data server, report the metadata information of partial data block in this back end server to this meta data server, such as, report call number to be greater than metadata information corresponding to the data block of startIndex.Meta data server, according to the metadata information of the metadata information of the data block reported before same back end server and the current data block reported, just can obtain the metadata information of all data blocks of this back end server current.
Below in conjunction with accompanying drawing, the technical scheme that the embodiment of the present invention provides is described.
See Fig. 2, a kind of information uploading method that the embodiment of the present invention provides, be applied in distributed object file system, this system comprises back end server and meta data server, and the method concrete steps comprise:
S201, back end server, when needing the metadata information to meta data server reported data block, determine the metadata information needing the data block reported to meta data server; Wherein, if described back end server needs the metadata information to described meta data server reported data block first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, if described back end server is not need the metadata information to described meta data server reported data block first, then the metadata information of data block that described needs report comprises the metadata information not reporting the data block of meta data server in this back end server;
The metadata information of the data block that described needs report is reported meta data server by S202, back end server.
In S201, when back end server is to the metadata information of meta data server reported data block, determine to be explained in detail explanation to both of these case below by the metadata information that the data block that needs to report to meta data server is corresponding:
Preferably, when back end server is registered to meta data server first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, meanwhile, record the index of metadata information corresponding to last data block, wherein, described index is numeral, and this index order, be numeral order sequence from small to large.
Preferably, described index also can be letter, and this index order, be the vertical order sequence of letter.
Preferably, the index of the metadata information that last data block of record is corresponding, both can be kept in back end server, also can be kept in meta data server.
Owing to registering first, do not have data block or only have a small amount of data block in back end server, therefore, the full dose of now carrying out metadata information corresponding to data block reports, and brings very large burden can't to network and meta data server.
Preferably, when back end server is non-register to meta data server first time, then back end server is according to the index of the data block of the current record corresponding with meta data server, determines that the metadata information of the data block that current needs report to meta data server is for the metadata information according to the data block corresponding to the index of index order after the index of current record; Wherein, the index of current record, for the back end server last time reports in the metadata information of the data block of meta data server according to the index of index order in last data block.
After back end server is registered first, meta data server can give back end server-assignment general unique identifier (Unive rsally Unique Identifier, UUID), back end server and meta data server all can preserve this UUID.Therefore, the registration if back end server is reached the standard grade again, meta data server and back end server all can detect whether this back end server is register first.
Preferably, the index of the data block of current record, both can be recorded in back end server, can be recorded in again in meta data server.
When the index record of the data block of current record is in meta data server, back end server sends the request of asking for the index of the data block of the current record corresponding with meta data server to meta data server, and receives the index of the data block of the current record corresponding with this meta data server that meta data server returns.Wherein, if the index of the data block of the current record that multiple back end server is corresponding is all recorded in meta data server, then meta data server will record the corresponding relation of the index of the data block of different back end servers and current record, namely for each back end server, recording this back end server the last time reports in the metadata information of the data block of this meta data server, according to the index of the data block of index order corresponding to the metadata information of last data block.
Therefore, the metadata information corresponding to partial data block according to index reports, compare and all reported by metadata information corresponding for the data block in back end server, metadata information amount obviously reduces, and brings very large burden can not to meta data server and whole network.
Preferably, back end server determines that current needs are to metadata information corresponding to meta data server reported data block in the following way:
Particularly, back end server when monitor disk plug information time, the metadata information that the data block after meta data server reports the index startIndex of acquisition is corresponding.Thus ensure that, when disk plugs, the metadata information that back end server and meta data server store also is consistent.
Particularly, metadata information corresponding for all data blocks in this back end server, according to predetermined period, is reported meta data server by back end server;
The time cycle (as one week or one month) of the metadata information that setting data node server reports all data blocks corresponding on meta data server, force the metadata information that all data blocks of back end server periodic report are corresponding.Meta data server is triggered successively the back end server that this meta data server is corresponding, and force the metadata information that back end server reports all data blocks of this back end server corresponding to corresponding meta data server, therefore, the situation of meta data server overload can be avoided.
Or, on back end server, setting reports the time cycle (as one week or one month) of the metadata information that all data blocks are corresponding in this back end server, the metadata information that back end server active period reports all data blocks corresponding.Back end server is in the time cycle of setting, the request of the metadata information reporting all data blocks corresponding is initiatively sent to meta data server, whether this meta data server will arrange this request to perform immediately according to its load capacity, if load capacity reaches highest level, so can request be reported to put into waiting list this, perform again after a while.Therefore, the situation of meta data server overload can be avoided.
Particularly, meta data server monitor the metadata information corresponding with the data block of back end server inconsistent after, trigger data node server.When back end server receives the Trigger message of meta data server transmission, the metadata information of all data blocks in this back end server is reported described meta data server, thus ensure that the metadata information that meta data server is corresponding with the data block of back end server is consistent.
Composition graphs 3, carry out summary to the situation of above-mentioned four kinds of information reportings and illustrate, when back end server is registered first, reported by the passive metadata information corresponding to data blocks all on this back end server, namely full dose reports;
When back end server is non-register first time, back end server is by the index startIndex of the data block according to the current record corresponding with meta data server, report call number to be greater than metadata information corresponding to the data block of startIndex to meta data server, namely increment reports;
When disk plugs, back end server will carry out increment to meta data server and report;
When meta data server monitors inconsistent with the metadata information of back end server, by active trigger data node server, the passive full dose of back end server is forced to report.
Below in conjunction with accompanying drawing, respectively the concrete grammar flow process that metadata information reports is illustrated.
See Fig. 4, the handling process that back end server initiates registration to meta data server comprises:
S401, back end server initiate registration request to meta data server;
Whether S402, meta data server Monitoring Data node server are register first, if register first, then performing S403, if not registering first, then performing S404;
S403, back end server are to metadata information corresponding to all data blocks of meta data server reported data node server;
The metadata information of the data block after S404, back end server statistics index startIndex;
In the present embodiment, the index arranged for the metadata information of each data block is numeral, index order is carried out according to numeral order from small to large, such as, last back end server reports index from metadata information corresponding to the data block of 0 to 9 to meta data server, then the index of current record is 9, and current what need to report this meta data server is metadata information corresponding to the data block corresponding to index from 10.
S405, back end server are to metadata information corresponding to the data block after the index startIndex that meta data server reported data node server is added up;
S406, meta data server arrange the new index startIndex corresponding with this back end server, and namely this index order reported is at the index startIndex of the metadata information of last data block;
The index startIndex of S407, this back end server update current record;
S408, every a time cycle, the metadata information that back end server reports all data block of back end server corresponding to meta data server.Wherein, the described time cycle can be arranged according to actual needs, such as one week or one month.
See Fig. 5, due to disk plug cause back end server and meta data server information inconsistent, carry out back end server and report the flow process of the metadata information of block to comprise:
S501, back end monitoring server are to disk plug event;
The disk plug event monitored is notified client by socket Socket by S502, back end server;
S503, back end server monitoring service termination receive this disk plug event;
S504, back end server are to metadata information corresponding to the data block after meta data server reports index startIndex;
The index startIndex that S505, back end server update record are new.
Correspondingly, see Fig. 6, a kind of information reporting device that the embodiment of the present invention provides, be applied in distributed object file system, this system comprises back end server and meta data server, and this device comprises:
Determine metadata information unit 601, for when back end server needs the metadata information to meta data server reported data block, determine the metadata information needing the data block reported to meta data server; Wherein, if described back end server needs the metadata information to described meta data server reported data block first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, if described back end server is not need the metadata information to described meta data server reported data block first, then the metadata information of data block that described needs report comprises the metadata information not reporting the data block of meta data server in this back end server;
Report metadata information unit 602, the metadata information for the data block reported by described needs reports meta data server.
Preferably, if described back end server is not need the metadata information to described meta data server reported data block first, then describedly determine that metadata information unit 601 determines to need the metadata information of the data block reported to meta data server, specifically for:
Determine the index of metadata information unit 601 according to the data block of the current record corresponding with described meta data server, determine that the metadata information of the data block that current needs report to described meta data server is for the metadata information according to the data block corresponding to the index of index order after the index of current record; Wherein, the index of described current record, for the described back end server last time reports in the metadata information of the data block of described meta data server according to the index of index order in last data block.
Preferably, described index is numeral; Described index order is numeral order sequence from small to large.
Preferably, the index of the data block of described current record, is recorded in described back end server.
Preferably, the index of the data block of described current record, is recorded in described meta data server;
The described metadata information unit 602 that reports when described back end server needs the metadata information to described meta data server reported data block, described in report metadata information unit also for:
The described metadata information unit 602 that reports sends the request of asking for the index of the data block of the current record corresponding with described meta data server to described meta data server, and receives the index of the data block of the current record corresponding with described meta data server that described meta data server returns.
Preferably, describedly determine that metadata information unit 601 determines the metadata information of current needs to described meta data server reported data block in the following way:
Described back end server, when monitoring the disk plug information of described back end server, determines the metadata information of current needs to described meta data server reported data block.
Preferably, described in report metadata information unit 602 according to predetermined period, the metadata information of all data blocks in this back end server is reported described meta data server; And/or,
The metadata information of all data blocks in this back end server, when receiving the Trigger message that described meta data server sends, is reported described meta data server by the described metadata information unit 602 that reports.
Preferably, determination metadata information unit described in the embodiment of the present invention 601 and report metadata information unit 602, can be realized by processor.
Preferably, the information reporting device described in the embodiment of the present invention, is specifically as follows back end server.
The embodiment of the present invention additionally provides one and is applied to back end server in distributed object file system, comprises any one the information reporting device described in the embodiment of the present invention.
In sum, a kind of method of information reporting, device and back end server is embodiments provided.Due to back end server register to meta data server first time, back end server does not have data block or only has the data block of seldom amount, therefore, the metadata information that the data block to meta data server reported data node server is corresponding, can't bring very large burden to network.When back end server is common roll off the production line reach the standard grade again registration time, report the metadata information that the data block after current data block index is corresponding, therefore, the metadata information amount of reporting obviously reduces, reduce information reporting to the burden of network, alleviate the processing pressure of meta data server.When back end monitoring server is to disk plug information, initiatively initiate the request of the metadata information reporting all data blocks of this back end server corresponding to meta data server.Thus ensure, even if there is disk plug event, still can ensure that the metadata information of meta data server and back end server stores is consistent.When meta data server monitors inconsistent with the metadata information of back end server, namely can initiatively trigger, force the metadata information that back end server reports all data blocks of this back end server corresponding, the time cycle to be updated can be waited again, force the metadata information that back end server reports all data blocks of this back end server corresponding.Thus reduce the time of delay of meta data server active-standby switch, it also avoid increment and report the meta data server that causes and the inconsistent problem of back end server data.
Those skilled in the art should understand, embodiments of the invention can be provided as method, system or computer program.Therefore, the present invention can adopt the form of complete hardware embodiment, completely software implementation or the embodiment in conjunction with software and hardware aspect.And the present invention can adopt in one or more form wherein including the upper computer program implemented of computer-usable storage medium (including but not limited to magnetic disc store and optical memory etc.) of computer usable program code.
The present invention describes with reference to according to the flow chart of the method for the embodiment of the present invention, equipment (system) and computer program and/or block diagram.Should understand can by the combination of the flow process in each flow process in computer program instructions realization flow figure and/or block diagram and/or square frame and flow chart and/or block diagram and/or square frame.These computer program instructions can being provided to the processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing device to produce a machine, making the instruction performed by the processor of computer or other programmable data processing device produce device for realizing the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
These computer program instructions also can be stored in can in the computer-readable memory that works in a specific way of vectoring computer or other programmable data processing device, the instruction making to be stored in this computer-readable memory produces the manufacture comprising command device, and this command device realizes the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
These computer program instructions also can be loaded in computer or other programmable data processing device, make on computer or other programmable devices, to perform sequence of operations step to produce computer implemented process, thus the instruction performed on computer or other programmable devices is provided for the step realizing the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.
Claims (15)
1. an information uploading method, is applied in distributed object file system, and this system comprises back end server and meta data server, it is characterized in that, the method comprises:
Back end server, when needing the metadata information to meta data server reported data block, determines the metadata information needing the data block reported to meta data server; Wherein, if described back end server needs the metadata information to described meta data server reported data block first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, if described back end server is not need the metadata information to described meta data server reported data block first, then the metadata information of data block that described needs report comprises the metadata information not reporting the data block of meta data server in this back end server;
The metadata information of the data block that described needs report by described back end server reports meta data server.
2. method according to claim 1, it is characterized in that, if described back end server is not need the metadata information to described meta data server reported data block first, then described back end server determines the metadata information needing the data block reported to meta data server, comprising:
Described back end server, according to the index of the data block of the current record corresponding with described meta data server, determines that the metadata information of the data block that current needs report to described meta data server is for the metadata information according to the data block corresponding to the index of index order after the index of current record; Wherein, the index of described current record, for the described back end server last time reports in the metadata information of the data block of described meta data server according to the index of index order in last data block.
3. method according to claim 2, is characterized in that, described index is numeral; Described index order is numeral order sequence from small to large.
4. method according to claim 3, is characterized in that, the index of the data block of described current record, is recorded in described back end server.
5. method according to claim 3, is characterized in that, the index of the data block of described current record, is recorded in described meta data server;
Described back end server is when needing the metadata information to described meta data server reported data block, and the method also comprises:
Described back end server sends the request of asking for the index of the data block of the current record corresponding with described meta data server to described meta data server, and receives the index of the data block of the current record corresponding with described meta data server that described meta data server returns.
6. method according to claim 1, is characterized in that, described back end server determines the metadata information of current needs to described meta data server reported data block in the following way:
Described back end server, when monitoring local disk plug information, determines the metadata information of current needs to described meta data server reported data block.
7. the method according to the arbitrary claim of claim 1-6, it is characterized in that, the method also comprises:
The metadata information of all data blocks in this back end server, according to predetermined period, is reported described meta data server by described back end server; And/or,
The metadata information of all data blocks in this back end server, when receiving the Trigger message that described meta data server sends, is reported described meta data server by described back end server.
8. an information reporting device, is applied in distributed object file system, and this system comprises back end server and meta data server, it is characterized in that, this device comprises:
Determine metadata information unit, for when back end server needs the metadata information to meta data server reported data block, determine the metadata information needing the data block reported to meta data server; Wherein, if described back end server needs the metadata information to described meta data server reported data block first, the metadata information of data block that then described needs report comprises the metadata information of all data blocks in this back end server, if described back end server is not need the metadata information to described meta data server reported data block first, then the metadata information of data block that described needs report comprises the metadata information not reporting the data block of meta data server in this back end server;
Report metadata information unit, the metadata information for the data block reported by described needs reports meta data server.
9. device according to claim 8, it is characterized in that, if described back end server is not need the metadata information to described meta data server reported data block first, then described when determining that metadata information unit determines the metadata information needing the data block reported to meta data server, specifically for:
According to the index of the data block of the current record corresponding with described meta data server, determine that the metadata information of the data block that current needs report to described meta data server is for the metadata information according to the data block corresponding to the index of index order after the index of current record; Wherein, the index of described current record, for the described back end server last time reports in the metadata information of the data block of described meta data server according to the index of index order in last data block.
10. device according to claim 9, is characterized in that, described index is numeral; Described index order is numeral order sequence from small to large.
11. devices according to claim 10, is characterized in that, the index of the data block of described current record, are recorded in described back end server.
12. devices according to claim 10, is characterized in that, the index of the data block of described current record, are recorded in described meta data server;
When described back end server needs the metadata information to described meta data server reported data block, described in report metadata information unit also for:
Send the request of asking for the index of the data block of the current record corresponding with described meta data server to described meta data server, and receive the index of the data block of the current record corresponding with described meta data server that described meta data server returns.
13. devices according to claim 8, is characterized in that, describedly determine that metadata information unit determines the metadata information of current needs to described meta data server reported data block in the following way:
When monitoring the disk plug information of described back end server, determine the metadata information of current needs to described meta data server reported data block.
14. devices according to Claim 8 described in-13 arbitrary claims, is characterized in that, described in report metadata information unit also for:
According to predetermined period, the metadata information of all data blocks in this back end server is reported described meta data server; And/or,
When receiving the Trigger message that described meta data server sends, the metadata information of all data blocks in this back end server is reported described meta data server.
15. 1 kinds are applied to back end server in distributed object file system, it is characterized in that, comprise the device described in the arbitrary claim of claim 8-14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410364737.1A CN104967639B (en) | 2014-07-28 | 2014-07-28 | Information reporting method, information reporting apparatus and data node server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410364737.1A CN104967639B (en) | 2014-07-28 | 2014-07-28 | Information reporting method, information reporting apparatus and data node server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104967639A true CN104967639A (en) | 2015-10-07 |
| CN104967639B CN104967639B (en) | 2017-03-22 |
Family
ID=54221580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410364737.1A Active CN104967639B (en) | 2014-07-28 | 2014-07-28 | Information reporting method, information reporting apparatus and data node server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104967639B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110196871A (en) * | 2019-03-07 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Data storage method and system |
| CN111400407A (en) * | 2020-04-10 | 2020-07-10 | 浙江大华技术股份有限公司 | Data synchronization method and device, storage medium and electronic device |
| CN113206891A (en) * | 2021-05-31 | 2021-08-03 | 浙江大华技术股份有限公司 | Scheduling method, storage system, scheduling device and storage medium for metadata reporting |
| CN113541786A (en) * | 2018-12-06 | 2021-10-22 | 华为技术有限公司 | Data acquisition method and related equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1330828A (en) * | 1998-12-15 | 2002-01-09 | 艾利森电话股份有限公司 | Method and arrangement in distributed system |
| CN101582920A (en) * | 2009-06-09 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for verifying and synchronizing data blocks in distributed file system |
-
2014
- 2014-07-28 CN CN201410364737.1A patent/CN104967639B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1330828A (en) * | 1998-12-15 | 2002-01-09 | 艾利森电话股份有限公司 | Method and arrangement in distributed system |
| CN101582920A (en) * | 2009-06-09 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for verifying and synchronizing data blocks in distributed file system |
Non-Patent Citations (1)
| Title |
|---|
| 李相娜等: "加快分布式文件系统元数据访问的研究与实现", 《微处理机》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113541786A (en) * | 2018-12-06 | 2021-10-22 | 华为技术有限公司 | Data acquisition method and related equipment |
| CN110196871A (en) * | 2019-03-07 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Data storage method and system |
| CN110196871B (en) * | 2019-03-07 | 2024-05-17 | 腾讯科技（深圳）有限公司 | Data warehousing method and system |
| CN111400407A (en) * | 2020-04-10 | 2020-07-10 | 浙江大华技术股份有限公司 | Data synchronization method and device, storage medium and electronic device |
| CN111400407B (en) * | 2020-04-10 | 2023-09-26 | 浙江大华技术股份有限公司 | Data synchronization method and device, storage medium and electronic device |
| CN113206891A (en) * | 2021-05-31 | 2021-08-03 | 浙江大华技术股份有限公司 | Scheduling method, storage system, scheduling device and storage medium for metadata reporting |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104967639B (en) | 2017-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107241281B (en) | Data processing method and device | |
| CN113568884B (en) | File management method and device, electronic equipment and storage medium | |
| CN110830283B (en) | Fault detection method, device, equipment and system | |
| EP3217735B1 (en) | M2m node deletion and registration method, m2m node and storage medium | |
| CN107315825B (en) | Index updating system, method and device | |
| EP3248361B1 (en) | Timers in stateless architecture | |
| CN104967639A (en) | Information reporting method, information reporting apparatus and data node server | |
| CN103237060B (en) | A kind of data object acquisition, Apparatus and system | |
| CN107404509B (en) | Distributed service configuration system and information management method | |
| WO2017162079A1 (en) | Data synchronization method and apparatus | |
| CN105338078A (en) | Data storage method and device used for storing system | |
| CN103024065A (en) | System configuration management method for cloud storage system | |
| CN108063832B (en) | Cloud storage system and storage method thereof | |
| US20160142778A1 (en) | Network camera, network camera control terminal, and video recording/delivering system | |
| CN111541762B (en) | Data processing method, management server, device and storage medium | |
| KR101768365B1 (en) | Notification Failure Management Method and System for Subscription-Notification in M2M/IoT System | |
| CN112069195A (en) | Database-based message transmission method and device, electronic equipment and storage medium | |
| EP3570169A1 (en) | Method and system for processing device failure | |
| CN111885103B (en) | Resource processing method, server, electronic device and storage medium | |
| CN110278109B (en) | Disaster recovery method and system | |
| CN104683288A (en) | Message continuous transmission method and device | |
| EP3710929B1 (en) | Optimized reconciliation in a controller switch network | |
| CN101730119B (en) | Reverse acquisition method and device of network element configuration data | |
| CN114374705A (en) | Service cluster and message pushing method | |
| CN111405313B (en) | Method and system for storing streaming media data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |