CN104967639B - Information reporting method, information reporting apparatus and data node server - Google Patents
Information reporting method, information reporting apparatus and data node server Download PDFInfo
- Publication number
- CN104967639B CN104967639B CN201410364737.1A CN201410364737A CN104967639B CN 104967639 B CN104967639 B CN 104967639B CN 201410364737 A CN201410364737 A CN 201410364737A CN 104967639 B CN104967639 B CN 104967639B
- Authority
- CN
- China
- Prior art keywords
- server
- metadata information
- back end
- data
- data block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides an information reporting method, an information reporting apparatus and a data node server, and is used for reducing the time-delay for meta-data information synchronization between a data node server and a meta-data server, and reducing the burden which is brought by reporting of large amount of meta-data information to the meta-data server. The method comprises determining the meta-data information, of a data block, to be reported to the meta-data server when there is requirement for reporting the meta-data information of the data block to the meta-data server, wherein if it is the first time for a data node server to report the meta-data information of the data block, the reported meta-data information of the data block includes the meta-data information of all the data blocks in the data node server, and if it is not the first time for the data node server to report the meta-data information of the data block, the reported meta-data information of the data block is the meta-data information of part of the data blocks in the data node server; and reporting the reported meta-data information of the data blocks to the meta-data server by the data node server.
Description
Technical field
The present invention relates to communication technical field, more particularly to a kind of information uploading method, device and back end server.
Background technology
Distributed object file system is used for managing mass memory resource, there is provided high speed, the stable, service of linear expansion.
In distributed object file system, each file is divided into into multiple objects (Object), each object is further subdivided into N number of number
According to block (Block), this N number of data block is stored respectively on N number of mutually different back end server.
The structure of distributed file system is as shown in Figure 1.Wherein, back end server (abbreviation Data Node) is used for
Data storage block, and the metadata information of these data blocks.Meta data server (abbreviation Metadata Server, MDS) is used
In the metadata information in data storage storehouse, the metadata information is used for describing the information of data block, specifically includes：Data block
The side-play amount of size, the index of data block and data block etc information.
In existing distributed file system, when back end server is registered to meta data server, it is to ensure number
Metadata information according to the data block between node server and meta data server is consistent, and back end server needs to be located at
The metadata information Complete Synchronization of all data blocks of the back end server is to meta data server, i.e. back end service
Device carries out the full dose of the metadata information of data block and reports to meta data server.
When back end server is registered to meta data server, need to meta data server reported data node to take
The corresponding metadata information of all data blocks on business device, due to store a large amount of metadata letters on each back end server
Breath, therefore, when there is more back end server to register to meta data server simultaneously, or back end server is frequent
To meta data server register, the synchronization of substantial amounts of metadata information can be carried out.
When meta data server starts or active and standby meta data server carries out active-standby switch, with meta data server
The back end server of association can be unified to initiate to register to meta data server, at offered load and meta data server
The immense pressure of reason, metadata information will synchronously expend longer time, also, before metadata information is synchronously completed,
Meta data server cannot outwardly provide the read-write service on basis.
When the metadata information of the data block stored on back end server with meta data server is inconsistent, first number
The metadata information to data block is needed to carry out substantial amounts of comparison according to server, to safeguard the consistent of the metadata information of data block
Property, substantial amounts of metadata information is synchronous also to bring greatly burden to the CPU of meta data server, internal memory.Wherein, with regard to first number
Substantial amounts of comparison is carried out to the metadata information of data block according to server, for example：Back end server is offline, then pulls out number
According to the disk of node server, so that on back end server, the metadata information of data block is possible to lose.
But, on meta data server, the metadata information of data block or complete.Accordingly, it would be desirable to carry out first number of data block
It is believed that the comparison of breath, that is, compare and deposit on the metadata information and meta data server of the data block on all back end servers
Whether the metadata information of the data block of storage is consistent.If it is inconsistent, the recovery operation of data block can be carried out or deletes behaviour
Make.
In addition, back end server to meta data server register when, substantial amounts of metadata information is synchronous greatly to be increased
Network burden is added, and back end server, probably due to offered load is too high, of short duration with meta data server has lost net
Network connects.When back end server is reconnected with meta data server network, will register to meta data server again,
Full dose is carried out to the corresponding metadata information of all of data block on the back end server again to report, it will further plus
Macroreticular is loaded, and causes back end server timeout offline.So, it will cause vicious cycle.
In sum, in prior art between back end server and meta data server needed for synchronous metadata information
Time is longer, and a large amount of metadata informations are reported and can bring great burden to meta data server, can also cause whole network
Load excessive.
The content of the invention
A kind of method of information reporting, device and back end server is embodiments provided, to reduce number
According to the time delay of synchronous metadata information between node server and meta data server, and a large amount of metadata informations of reduction are reported
To the burden that meta data server brings.
A kind of method of information reporting provided in an embodiment of the present invention, is applied in distributed object file system, and this is
System includes back end server and meta data server, and the method includes：
Back end server when need to meta data server reported data block metadata information when, it is determined that need to
The metadata information of the data block that meta data server is reported；Wherein, if the back end server is to be needed to institute first
The metadata information of meta data server reported data block is stated, then the metadata information of the data block for needing to report includes this
The metadata information of all data blocks in back end server, if the back end server is not to need first to described
The metadata information of meta data server reported data block, then the metadata information of data block for needing to report include the number
According to the metadata information of the data block for not reporting meta data server in node server；
The metadata information of the data block for needing and reporting is reported meta data server by back end server.
With it, back end server selectively reports metadata information to meta data server, so as to drop
Low a large amount of metadata informations report the burden that meta data server brings, and reduce back end server with metadata clothes
The time delay of synchronous metadata information between business device.
If it is preferred that the back end server is not to be needed to the meta data server reported data block first
Metadata information, then the back end server determination need the metadata letter of the data block reported to meta data server
Breath, including：
Rope of the back end server according to the data block of the current record corresponding with the meta data server
Draw, it is determined that the metadata information of data block for being currently needed for the meta data server reporting is current according to index order
The metadata information of the data block corresponding to index after the index of record；Wherein, the index of the current record, is described
According to index order in the metadata information of the data block that the back end server last time reports the meta data server
In the index of last data block.
It is preferred that the index is numeral；The index order, is numeral order sequence from small to large.
It is preferred that the index of the data block of current record, records in the back end server.
It is preferred that the index of the data block of the current record, records in the meta data server；
The back end server is gone back when metadata information of the needs to the meta data server reported data block
Including：
The back end server is asked for corresponding with the meta data server to meta data server transmission
Current record data block index request, and it is returning with the meta data server to receive the meta data server
The index of the data block of corresponding current record.
It is preferred that the back end server is determined as follows and is currently needed on the meta data server
The metadata information of report data block：
The back end server when monitor disk plug information when, it is determined that being currently needed for the Metadata Service
The metadata information of device reported data block.
With it, node server is reported to meta data server increment and is worked as when disk plug information is monitored
The metadata information of the data block corresponding to index after the index of front record, so as to, under disk plugging condition, reduce
The number of times that full dose is reported, reduces full dose and reports the burden to network.
It is preferred that the back end server is according to predetermined period, by all data blocks in the back end server
Metadata information report the meta data server；And/or,
The back end server when the triggering message that the meta data server sends is received, by the data section
In point server, the metadata information of all data blocks reports the meta data server.
With it, node server reports the data block of the node server to meta data server cycle full dose
Metadata information, or when meta data server monitors that the meta data server is inconsistent with node server metadata information
When, forced node server full dose reports node server metadata information, reports the metadata for causing so as to avoid increment
The inconsistent problem of server and data node server data.
A kind of information reporting device provided in an embodiment of the present invention, is applied in distributed object file system, the system
Including back end server and meta data server, the device includes：
Determine metadata information unit, for needing to meta data server reported data block when back end server
During metadata information, it is determined that needing to meta data server the metadata information of data block for reporting；Wherein, if the data section
Point server is to need the metadata information to the meta data server reported data block first, then the number for needing to report
Include the metadata information of all data blocks in the back end server according to the metadata information of block, if back end clothes
Business device is not to need the metadata information to the meta data server reported data block first, then the data for needing to report
The metadata information of block does not report the metadata information of the data block of meta data server in including the back end server；
Metadata information unit is reported, for the metadata information for needing the data block for reporting is reported metadata
Server.
By the device, back end server selectively reports metadata information to meta data server, so as to drop
Low a large amount of metadata informations report the burden that meta data server brings, and reduce back end server with metadata clothes
The time delay of synchronous metadata information between business device.
If it is preferred that the back end server is not to be needed to the meta data server reported data block first
Metadata information, then it is described to determine that metadata information unit determines the metadata of the data block for needing to report to meta data server
During information, specifically for：
Determine rope of the metadata information unit according to the data block of the current record corresponding with the meta data server
Draw, it is determined that the metadata information of data block for being currently needed for the meta data server reporting is current according to index order
The metadata information of the data block corresponding to index after the index of record；Wherein, the index of the current record, is described
According to index order in the metadata information of the data block that the back end server last time reports the meta data server
In the index of last data block.
It is preferred that the index is numeral；The index order, is numeral order sequence from small to large.
It is preferred that the index of the data block of the current record, records in the back end server.
It is preferred that the index of the data block of the current record, records in the meta data server；
When the back end server needs the metadata information to the meta data server reported data block, institute
State and report metadata information unit to be additionally operable to：
The data block for asking for the current record corresponding with the meta data server is sent to the meta data server
Index request, and receive the current record corresponding with the meta data server that the meta data server is returned
The index of data block.
It is preferred that the determination metadata information unit is determined as follows and is currently needed for the Metadata Service
The metadata information of device reported data block：
The back end server when monitor the back end server disk plug information when, it is determined that currently
Need the metadata information to the meta data server reported data block.
By the device, node server is reported to meta data server increment and is worked as when disk plug information is monitored
The metadata information of the data block corresponding to index after the index of front record, so as to, under disk plugging condition, reduce
The number of times that full dose is reported, reduces full dose and reports the burden to network.
It is preferred that described report metadata information unit according to predetermined period, by all numbers in the back end server
The meta data server is reported according to the metadata information of block；And/or,
It is described to report metadata information unit when the triggering message that the meta data server sends is received, this is counted
The meta data server is reported according to the metadata information of all data blocks in node server.
It is provided in an embodiment of the present invention to be applied to back end server in distributed object file system, including above-mentioned
Information reporting device described in one.
Description of the drawings
Fig. 1 is the structural representation of existing distributed file system；
Fig. 2 is a kind of schematic flow sheet of information uploading method provided in an embodiment of the present invention；
Fig. 3 is the embodiment of the present invention for the corresponding relation schematic diagram between different situations metadata information reporting schemes；
Fig. 4 be the embodiment of the present invention in metadata information from back end server to meta data server reported data block
A kind of idiographic flow schematic diagram；
Fig. 5 be the embodiment of the present invention in metadata information from back end server to meta data server reported data block
Another kind of idiographic flow schematic diagram；
Fig. 6 is a kind of structural representation of information reporting device provided in an embodiment of the present invention.
Specific embodiment
A kind of method of information reporting, device and back end server is embodiments provided, it is short to solve
Temporary is overtime offline, back end server to meta data server full dose report metadata information to cause the drawbacks of, and data
Node server reports the problem for causing with the inconsistent full dose that carries out of metadata information of meta data server.
In technical scheme provided in an embodiment of the present invention, when back end server is registered to meta data server first
When, the metadata information of all data blocks on the back end server is reported to meta data server, and record reports this
In the metadata information of meta data server according to index order last data block index startIndex.
When back end server is non-to be registered to meta data server first, back end server will according to this yuan
The index startIndex of the data block of the corresponding current record of data server, reports the data to the meta data server
The metadata information of partial data block in node server, for example, report call number corresponding more than the data block of startIndex
Metadata information.Meta data server is according to the metadata information of the data block reported before same back end server and works as
Before the metadata information of data block that reports, it is possible to obtain the metadata of all data blocks of the current back end server
Information.
Below in conjunction with the accompanying drawings technical scheme provided in an embodiment of the present invention is illustrated.
Referring to Fig. 2, a kind of information uploading method provided in an embodiment of the present invention is applied to distributed object file system
In, the system includes back end server and meta data server, and the method concrete steps include：
S201, back end server when need to meta data server reported data block metadata information when, it is determined that
The metadata information of data block for reporting is needed to meta data server；Wherein, if the back end server is to need first
Will be to the metadata information of the meta data server reported data block, then the metadata information of the data block for needing to report
Including the metadata information of all data blocks in the back end server, if the back end server is not to need first
To the metadata information of the meta data server reported data block, then the metadata information bag of the data block for needing to report
The metadata information of the data block of meta data server is not reported in including the back end server；
The metadata information of the data block for needing and reporting is reported Metadata Service by S202, back end server
Device.
In S201, metadata information from back end server to meta data server reported data block when, it is determined that need
The corresponding metadata information of data block to report to meta data server, is explained in detail to both of these case below
It is bright：
It is preferred that when back end server is registered to meta data server first, then the data that the needs are reported
The metadata information of block includes the metadata information of all data blocks in the back end server, at the same time, records most
The index of the corresponding metadata information of latter data block, wherein, the index is numeral, and the index order, be numeral from
It is little to sort to big order.
It is preferred that the index can also be letter, and the index order, it is alphabetical vertical order sequence.
It is preferred that the index of the corresponding metadata information of last data block of record, can both be stored in data section
In point server, it is also possible to be stored in meta data server.
Due to registering first, in back end server, there is no data block or only a small amount of data block, therefore, now enter
The full dose of the corresponding metadata information of row data block is reported, and can't bring very big burden to network and meta data server.
It is preferred that when back end server is non-register to meta data server first when, then back end server root
According to the index of the data block of the current record corresponding with meta data server, it is determined that being currently needed for meta data server reporting
The metadata information of data block be data block corresponding to the index according to index order after the index of current record
Metadata information；Wherein, the index of current record, is data that the back end server last time reports meta data server
In the metadata information of block according to index order last data block index.
After back end server is registered first, meta data server can be general to back end server-assignment one
Unique identifier (Unive rsally Unique Identifier, UUID), back end server and meta data server
The UUID will be preserved.Therefore, the registration if back end server is reached the standard grade again, meta data server and data node serve
Device can all detect whether the back end server is to register first.
It is preferred that the index of the data block of current record, can both record in back end server, can record again
In meta data server.
When the index record of the data block of current record is in meta data server, back end server is taken to metadata
Business device sends the request of the index of the data block for asking for the current record corresponding with meta data server, and receives metadata clothes
The index of the data block of the current record corresponding with the meta data server that business device is returned.Wherein, if multiple back end
The index of the data block of the corresponding current record of server is all recorded in meta data server, then meta data server will be recorded
The corresponding relation of the index of the data block of different back end servers and current record, i.e., for each back end service
Device, records in the metadata information of the data block that back end server the last time reports the meta data server, presses
According to the index of the data block corresponding to metadata information of the index order in a last data block.
Therefore, the corresponding metadata information of partial data block is reported according to index, compares and back end is serviced
The corresponding metadata information of data block in device is all reported, and metadata information amount is significantly reduced, and will not be taken to metadata
Business device and whole network bring very big burden.
It is preferred that back end server is determined as follows being currently needed for meta data server reported data block
Corresponding metadata information：
Specifically, back end server reports acquisition when disk plug information is monitored to meta data server
The corresponding metadata information of data block after index startIndex.So as to ensure in the case where disk is plugged, data section
The metadata information that point server is stored with meta data server is also consistent.
Specifically, back end server, will be all data blocks in the back end server corresponding according to predetermined period
Metadata information report meta data server；
On meta data server setting data node server report the corresponding metadata information of all data blocks when
Between the cycle (such as one week or one month), all data blocks of pressure data node server periodic report corresponding metadata letter
Breath.Meta data server is triggered the corresponding back end server of the meta data server, and pressure data node clothes successively
Business device reports the corresponding metadata information of all data blocks of the back end server to corresponding meta data server, because
This, you can avoid the situation of meta data server overload.
Or, the setting on the back end server reports the corresponding first number of all data blocks in the back end server
It is believed that the time cycle (such as one week or one month) of breath, back end server active period reports all data blocks correspondingly
Metadata information.Back end server was actively sent to meta data server and reports all numbers in the time cycle of setting
According to the request of the corresponding metadata information of block, according to its load capacity, the meta data server will arrange whether the request holds immediately
OK, if load capacity has reached top level, then request can be reported to be put into waiting list this, be performed again after a while.Therefore,
The situation of meta data server overload can be avoided.
Specifically, meta data server monitors that metadata information corresponding with the data block of back end server differs
After cause, trigger data node server.When back end server receives the triggering message of meta data server transmission, will
In the back end server, the metadata information of all data blocks reports the meta data server, so as to ensure that first number
It is consistent according to server metadata information corresponding with the data block of back end server.
With reference to Fig. 3, summary explanation is carried out to the situation of above-mentioned four kinds of information reportings, when back end server is registered first
When, passively the corresponding metadata information of all of data block on the back end server is reported, i.e., full dose is reported；
When back end server is non-to be registered first, back end server will be according to corresponding with meta data server
Current record data block index startIndex, to meta data server report call number more than startIndex number
According to the corresponding metadata information of block, i.e. increment is reported；
When disk is plugged, back end server will carry out increment to meta data server and report；
When meta data server monitors inconsistent with the metadata information of back end server, number is actively triggered
According to node server, the passive full dose of pressure data node server is reported.
Below in conjunction with the accompanying drawings, the concrete grammar flow process for reporting to metadata information respectively is illustrated.
Referring to Fig. 4, back end server includes to the handling process that meta data server initiates registration：
S401, back end server initiate registration request to meta data server；
Whether S402, meta data server Monitoring Data node server are to register first, if registering first, are then held
Row S403, if not registering first, then performs S404；
S403, back end server are to the corresponding unit of all data blocks of meta data server reported data node server
Data message；
The metadata information of the data block after S404, back end server statistics index startIndex；
In the present embodiment, it is that the index of the metadata information setting of each data block is numeral, according to numeral from small to large
Order be indexed sequence, for example, last back end server to meta data server report index from 0 to 9 number
According to the corresponding metadata information of block, then the index of current record is 9, and be currently needed for reporting the meta data server is from 10
The corresponding metadata information of data block corresponding to the index of beginning.
The index that S405, back end server are counted to meta data server reported data node server
The corresponding metadata information of data block after startIndex；
S406, meta data server arrange new index startIndex corresponding with the back end server, i.e., originally
Index startIndex of the secondary index order for reporting in the metadata information of last data block；
S407, the index startIndex of the back end server update current record；
S408, every a time cycle, back end server reports back end clothes to meta data server
The corresponding metadata information of all data blocks of business device.Wherein, the time cycle can be configured according to actual needs, for example
One week or one month.
Referring to Fig. 5, as disk plug causes back end server inconsistent with meta data server information, enter line number
The flow process of the metadata information of block is reported to include according to node server：
S501, back end monitoring server are to disk hot plug event；
The disk hot plug event for monitoring is notified client by socket Socket by S502, back end server；
S503, back end server are monitored service end and receive the disk hot plug event；
Data block after S504, back end server report from index startIndex to meta data server is corresponding
Metadata information；
S505, back end server update record new index startIndex.
Correspondingly, referring to Fig. 6, a kind of information reporting device provided in an embodiment of the present invention is applied to distributed objects text
In part system, the system includes back end server and meta data server, and the device includes：
Determine metadata information unit 601, for needing to meta data server reported data when back end server
During the metadata information of block, it is determined that needing to meta data server the metadata information of data block for reporting；Wherein, if the number
To need the metadata information to the meta data server reported data block first according to node server, then it is described to need to report
The metadata information of data block include the metadata information of all data blocks in the back end server, if the data section
Point server is not to need the metadata information to the meta data server reported data block first, then described to need what is reported
The metadata information of data block does not report the metadata of the data block of meta data server to believe in including the back end server
Breath；
Metadata information unit 602 is reported, for the metadata information for needing the data block for reporting is reported unit
Data server.
If it is preferred that the back end server is not to be needed to the meta data server reported data block first
Metadata information, then it is described to determine that metadata information unit 601 determines the unit of the data block for needing to report to meta data server
Data message, specifically for：
Determine metadata information unit 601 according to the data block of the current record corresponding with the meta data server
Index, it is determined that the metadata information of data block for being currently needed for the meta data server reporting is to work as according to index order
The metadata information of the data block corresponding to index after the index of front record；Wherein, the index of the current record, is institute
Arrange according to index in stating the metadata information of the data block that the back end server last time reports the meta data server
Index of the sequence in last data block.
It is preferred that the index is numeral；The index order, is numeral order sequence from small to large.
It is preferred that the index of the data block of the current record, records in the back end server.
It is preferred that the index of the data block of the current record, records in the meta data server；
It is described to report metadata information unit 602 to need on the meta data server when the back end server
It is during the metadata information of report data block, described to report metadata information unit to be additionally operable to：
It is described to report metadata information unit 602 to ask for and the meta data server to meta data server transmission
The request of the index of the data block of corresponding current record, and it is returning with the metadata to receive the meta data server
The index of the data block of the corresponding current record of server.
It is preferred that the determination metadata information unit 601 is determined as follows and is currently needed for the metadata
The metadata information of server reported data block：
The back end server when monitor the back end server disk plug information when, it is determined that currently
Need the metadata information to the meta data server reported data block.
It is preferred that described report metadata information unit 602 according to predetermined period, will own in the back end server
The metadata information of data block reports the meta data server；And/or,
It is described to report metadata information unit 602 when the triggering message that the meta data server sends is received, incite somebody to action
In the back end server, the metadata information of all data blocks reports the meta data server.
It is preferred that determination metadata information unit 601 described in the embodiment of the present invention and reporting metadata information unit
602, can be realized by processor.
It is preferred that the information reporting device described in the embodiment of the present invention, is specifically as follows back end server.
The embodiment of the present invention additionally provides one kind and is applied to back end server in distributed object file system, including
Any one information reporting device described in the embodiment of the present invention.
In sum, a kind of method of information reporting, device and back end server are embodiments provided.By
When back end server is registered to meta data server first, back end server does not have data block or only minimal amount
Data block, therefore, to the corresponding metadata information of data block of meta data server reported data node server, can't
Very big burden is brought to network.When back end server it is common it is offline reach the standard grade again registration when, report current data block rope
The corresponding metadata information of data block after drawing, therefore, the metadata information amount of reporting is significantly reduced, and reduces information reporting to net
The burden of network, mitigates the processing pressure of meta data server.When back end monitoring server is to disk plug information, Xiang Yuan
Data server actively initiates to report the request of the corresponding metadata information of all data blocks of back end server.So as to protect
Card, even if there is disk plug event, remains able to ensure the metadata of meta data server and back end server storage
Information is consistent.When meta data server monitors inconsistent with the metadata information of back end server, you can with
Actively trigger, pressure data node server reports the corresponding metadata information of all data blocks of back end server, and
The time cycle to be updated can be waited, pressure data node server reports all data blocks of back end server corresponding
Metadata information.So as to reduce the time delay of meta data server active-standby switch, unit's number caused by increment is reported is it also avoid
According to the inconsistent problem of server and data node server data.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, system or computer program
Product.Therefore, the present invention can adopt complete hardware embodiment, complete software embodiment or with reference to the reality in terms of software and hardware
Apply the form of example.And, the present invention can be using the computer for wherein including computer usable program code at one or more
The shape of the computer program implemented in usable storage medium (including but not limited to disk memory and optical memory etc.)
Formula.
The present invention be with reference to method according to embodiments of the present invention, equipment (system), and computer program flow process
Figure and/or block diagram are describing.It should be understood that can be by computer program instructions flowchart and/or each stream in block diagram
The combination of journey and/or square frame and flow chart and/or the flow process in block diagram and/or square frame.These computer programs can be provided
The processor of general purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced for reality by the instruction of computer or the computing device of other programmable data processing devices
The device of the function of specifying in present one flow process of flow chart or one square frame of multiple flow processs and/or block diagram or multiple square frames.
These computer program instructions may be alternatively stored in and can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory is produced to be included referring to
Make the manufacture of device, the command device realize in one flow process of flow chart or one square frame of multiple flow processs and/or block diagram or
The function of specifying in multiple square frames.
These computer program instructions can be also loaded in computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented process, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow process of flow chart or multiple flow processs and/or block diagram one
The step of function of specifying in individual square frame or multiple square frames.
Obviously, those skilled in the art can carry out the essence of various changes and modification without deviating from the present invention to the present invention
God and scope.So, if these modifications of the present invention and modification belong to the scope of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to comprising these changes and modification.
Claims (13)
1. a kind of information uploading method, is applied in distributed object file system, the system include back end server and
Meta data server, it is characterised in that the method includes：
Back end server when need to meta data server reported data block metadata information when, it is determined that needing to first number
The metadata information of the data block reported according to server；Wherein, if the back end server is to be needed to the unit first
The metadata information of data server reported data block, then the metadata information of data block for needing to report include the data
The metadata information of all data blocks in node server, if the back end server is not to be needed to first number first
According to the metadata information of server reported data block, then the metadata information of the data block for needing to report includes the data section
The metadata information of the data block of meta data server is not reported in point server；
The metadata information of the data block for needing and reporting is reported meta data server by the back end server；
If the back end server is not to need the metadata information to the meta data server reported data block first,
Then the back end server determines the metadata information of the data block for needing to report to meta data server, including：
Index of the back end server according to the data block of the current record corresponding with the meta data server, really
Surely the metadata information of data block for being currently needed for the meta data server reporting is in current record according to index order
Index after index corresponding to data block metadata information；Wherein, the index of the current record, is the data
The node server last time is reported in the metadata information of the data block of the meta data server according to index order most
The index of data block afterwards.
2. method according to claim 1, it is characterised in that the index is numeral；The index order, be numeral from
It is little to sort to big order.
3. method according to claim 2, it is characterised in that the index of the data block of the current record, records in institute
State in back end server.
4. method according to claim 2, it is characterised in that the index of the data block of the current record, records in institute
State in meta data server；
The back end server when need to the meta data server reported data block metadata information when, the method
Also include：
The back end server is sent to the meta data server asks for corresponding with the meta data server working as
The request of the index of the data block of front record, and receive the relative with the meta data server of the meta data server return
The index of the data block of the current record answered.
5. method according to claim 1, it is characterised in that the back end server is determined as follows works as
The front metadata information needed to the meta data server reported data block：
The back end server when monitor local disk plug information when, it is determined that being currently needed for the Metadata Service
The metadata information of device reported data block.
6. the method according to claim 1-5 any claim, it is characterised in that the method also includes：
The back end server according to predetermined period, by the metadata information of all data blocks in the back end server
Report the meta data server；And/or,
The back end is taken by the back end server when the triggering message that the meta data server sends is received
In business device, the metadata information of all data blocks reports the meta data server.
7. a kind of information reporting device, is applied in distributed object file system, the system include back end server and
Meta data server, it is characterised in that the device includes：
Determine metadata information unit, for the first number to meta data server reported data block being needed when back end server
It is believed that during breath, it is determined that the metadata information of data block for reporting is needed to meta data server；Wherein, if back end clothes
Business device is to need the metadata information to the meta data server reported data block first, then the data block for needing to report
Metadata information include the metadata information of all data blocks in the back end server, if the back end server
To need the metadata information to the meta data server reported data block first, then the data block that the needs are reported
Metadata information does not report the metadata information of the data block of meta data server in including the back end server；
Metadata information unit is reported, for the metadata information for needing the data block for reporting is reported Metadata Service
Device；
If the back end server is not to need the metadata information to the meta data server reported data block first,
When then the determination metadata information unit determines the metadata information of the data block for needing to report to meta data server, specifically
For：
According to the index of the data block of the current record corresponding with the meta data server, it is determined that being currently needed for the unit
The metadata information of the data block that data server is reported is the index institute according to index order after the index of current record
The metadata information of corresponding data block；Wherein, the index of the current record, is on the back end server last time
In the metadata information of the data block for offering the meta data server according to index order last data block index.
8. device according to claim 7, it is characterised in that the index is numeral；The index order, be numeral from
It is little to sort to big order.
9. device according to claim 8, it is characterised in that the index of the data block of the current record, records in institute
State in back end server.
10. device according to claim 8, it is characterised in that the index of the data block of the current record, records in institute
State in meta data server；
When the back end server needs the metadata information to the meta data server reported data block, it is described on
Report metadata information unit is additionally operable to：
The rope of the data block for asking for the current record corresponding with the meta data server is sent to the meta data server
The request drawn, and receive the data of the current record corresponding with the meta data server that the meta data server is returned
The index of block.
11. devices according to claim 7, it is characterised in that the determination metadata information unit is in the following way
It is determined that being currently needed for the metadata information to the meta data server reported data block：
When the disk for monitoring the back end server plugs information, it is determined that being currently needed for the meta data server
The metadata information of reported data block.
12. devices according to claim 7-11 any claim, it is characterised in that described to report metadata information list
Unit is additionally operable to：
According to predetermined period, the metadata information of all data blocks in the back end server is reported into the metadata clothes
Business device；And/or,
When the triggering message that the meta data server sends is received, by all data blocks in the back end server
Metadata information reports the meta data server.
13. one kind are applied to back end server in distributed object file system, it is characterised in that including claim 7-
Device described in 12 any claims.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410364737.1A CN104967639B (en) | 2014-07-28 | 2014-07-28 | Information reporting method, information reporting apparatus and data node server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410364737.1A CN104967639B (en) | 2014-07-28 | 2014-07-28 | Information reporting method, information reporting apparatus and data node server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104967639A CN104967639A (en) | 2015-10-07 |
| CN104967639B true CN104967639B (en) | 2017-03-22 |
Family
ID=54221580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410364737.1A Active CN104967639B (en) | 2014-07-28 | 2014-07-28 | Information reporting method, information reporting apparatus and data node server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104967639B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113541786A (en) * | 2018-12-06 | 2021-10-22 | 华为技术有限公司 | Data acquisition method and related equipment |
| CN110196871B (en) * | 2019-03-07 | 2024-05-17 | 腾讯科技（深圳）有限公司 | Data warehousing method and system |
| CN111400407B (en) * | 2020-04-10 | 2023-09-26 | 浙江大华技术股份有限公司 | Data synchronization method and device, storage medium and electronic device |
| CN113206891B (en) * | 2021-05-31 | 2022-10-14 | 浙江大华技术股份有限公司 | Scheduling method, storage system, scheduling device and storage medium for metadata reporting |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1330828A (en) * | 1998-12-15 | 2002-01-09 | 艾利森电话股份有限公司 | Method and arrangement in distributed system |
| CN101582920A (en) * | 2009-06-09 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for verifying and synchronizing data blocks in distributed file system |
-
2014
- 2014-07-28 CN CN201410364737.1A patent/CN104967639B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1330828A (en) * | 1998-12-15 | 2002-01-09 | 艾利森电话股份有限公司 | Method and arrangement in distributed system |
| CN101582920A (en) * | 2009-06-09 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for verifying and synchronizing data blocks in distributed file system |
Non-Patent Citations (1)
| Title |
|---|
| 加快分布式文件系统元数据访问的研究与实现;李相娜等;《微处理机》;20080415(第02期);120-122 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104967639A (en) | 2015-10-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107241281B (en) | Data processing method and device | |
| CN104967639B (en) | Information reporting method, information reporting apparatus and data node server | |
| CN108512876B (en) | Data pushing method and device | |
| CN102769549A (en) | Network security monitoring method and device | |
| CN108744504B (en) | Game data processing method and device, game server and game system | |
| CN110311831A (en) | System resource monitoring method and relevant device based on container cloud | |
| WO2020134840A1 (en) | Data distribution method and related product | |
| CN105119763A (en) | RIA-based Web network management client big data rapid export method and system | |
| TW201737107A (en) | Data synchronization method and apparatus | |
| CN110471749A (en) | Task processing method, device, computer readable storage medium and computer equipment | |
| CN108063832B (en) | Cloud storage system and storage method thereof | |
| CN114675983A (en) | Business data processing method and device, storage medium and electronic equipment | |
| CN107040576A (en) | Information-pushing method and device, communication system | |
| CN101925093A (en) | Method and equipment for acquiring terminal information | |
| CN108595121B (en) | Data storage method and device | |
| CN103747484B (en) | A kind of RIM flow systems information processing method and service node device | |
| CN107231284B (en) | Message sending method and terminal equipment | |
| CN105472739A (en) | Traffic channel allocation method and base station | |
| CN109005246B (en) | Data synchronization method, device and system | |
| CN103259863B (en) | Based on the system and method that the control zookeeper of cluster serves | |
| CN104317906A (en) | Processing method for large-scale storage of reported data of GPS (global positioning system) monitoring radios | |
| CN110471922B (en) | Content extension method and device for accounting node and computer readable storage medium | |
| EP3346671A1 (en) | Service processing method and equipment | |
| CN104580113B (en) | A kind of method and device of processing external request | |
| CN105992194A (en) | Network data content acquiring method and network data content acquiring device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |