CN104408063A - Distributed file management method and device - Google Patents
Distributed file management method and device Download PDFInfo
- Publication number
- CN104408063A CN104408063A CN201410594431.5A CN201410594431A CN104408063A CN 104408063 A CN104408063 A CN 104408063A CN 201410594431 A CN201410594431 A CN 201410594431A CN 104408063 A CN104408063 A CN 104408063A
- Authority
- CN
- China
- Prior art keywords
- node
- data file
- file
- unique
- upload request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/122—File system administration, e.g. details of archiving or snapshots using management policies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
An embodiment of the invention provides a distributed file management method and device. The distributed file management method comprises the following steps of receiving an uploading request of a data file, wherein the uploading request carries a corresponding relationship between node sequence serial numbers and file processing operations; obtaining a self-node serial number according to the uploading request; inquiring the file processing operation which is corresponding to the self-node serial number according to the corresponding relationship between the node serial numbers and the file processing operations; determining the only next-hop node of the self-node according to a preset route relationship if the file processing operation is a transmission processing operation, adding self-node identification information into the uploading request, transmitting the data file and the uploading request with the added self-node identification information to the only next-hop node to enable the only next-hop node to obtain a self-node serial number according to the received uploading request and perform the corresponding file processing operation inquired according to the self-node sequence serial number. The distributed file management method can balance node loads and reduce node expenditures.
Description
Technical field
The present invention relates to computer realm, particularly relate to a kind of distributed file management method and device.
Background technology
Along with the development of bank finance industry, the data files such as the voucher bill that its business produces get more and more, and this just needs a system capacious to manage these data files, thus conveniently stores it and call.At present, the main method of bank management data file is that cloud stores, and during upload file, client sends files passe request to the management node of cloud storage system, upload file is stored into database by management node, and records fileinfo and the memory address thereof of upload file; During download file, client sends file download request to the management node of cloud storage system, management node inquires about the memory address of this download file according to the fileinfo of the download file comprised in download request, then sends client to by this file of web download.The management node that cloud stores needs all upload request and the download request that process client transmission, and load overhead is large.
Summary of the invention
The embodiment of the present invention provides a kind of distributed file management method and device, can balanced node load, reduces node overheads.
First aspect present invention provides a kind of distributed file management method, can comprise:
The upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation;
Self node sequence numbering is obtained according to described upload request;
According to the corresponding relation of described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores also forward process;
If described document processing operation is forward process, then according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and the described upload request that with the addition of own node identification information, obtain self node sequence to make described unique next-hop node according to the upload request received and number and perform the document processing operation of the correspondence inquired according to own node serial number.
Second aspect present invention provides a kind of distributed file management method, can comprise:
The download request of received data file, described download request comprises the file unique identification of described data file;
According to the path computing rule preset, calculated the target query path of described data file by described file unique identification;
Judge position corresponding to target query path described in the memory device whether described data file is present in own node management;
If not, then according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge the position that described in the memory device whether described data file is present in own node management, target query path is corresponding.
Third aspect present invention provides a kind of distributed file management device, can comprise:
Receiving element, for the upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation;
Numbering acquiring unit, for obtaining self node sequence numbering according to described upload request;
Query unit, for the corresponding relation according to described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores and forward process;
Forward processing unit, for be forward process or described document processing operation at described document processing operation for store and forward process time, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and described upload request, to make described unique next-hop node obtain self node sequence numbering according to described upload request, and the document processing operation corresponding according to the inquiry of own node serial number.
Fourth aspect present invention provides a kind of distributed file management device, can comprise:
Receiving element, for the download request of received data file, described download request comprises the file unique identification of described data file;
Computing unit, for according to the path computing rule preset, is calculated the target query path of described data file by described file unique identification;
Request forward unit, during for judging position corresponding to target query path described in the memory device that described data file is not present in own node management at file judging unit, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge whether described data file is present in position corresponding to target query path described in memory device that own node manages.
Implement the embodiment of the present invention, there is following beneficial effect:
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the schematic flow sheet of a kind of distributed file management method in the embodiment of the present invention;
Fig. 2 is the schematic flow sheet of a kind of distributed file management method in another embodiment of the present invention;
Fig. 3 is the schematic flow sheet of a kind of distributed file management method in another embodiment of the present invention;
Fig. 4 is the schematic flow sheet of a kind of distributed file management method in another embodiment of the present invention;
Fig. 5 is the schematic flow sheet of a kind of distributed file management method in further embodiment of this invention;
Fig. 6 is the structural representation of a kind of distributed file management device in the embodiment of the present invention;
Fig. 7 is the structural representation of a kind of distributed file management device in another embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
The data file mentioned in the embodiment of the present invention, can include but are not limited to all kinds of voucher bill image files produced in bank finance industry.The memory device of the node administration mentioned in the embodiment of the present invention can include but are not limited to NAS (Network Attached Storage, network interconnection system stores) equipment.The application scenarios of the distributed file management method of the embodiment of the present invention can but be not limited only to all kinds of voucher bill image files of bank finance industry storage and in calling.
Fig. 1 is the schematic flow sheet of a kind of distributed file management method in the embodiment of the present invention, and the distributed file management method as shown in the figure in the present embodiment can comprise:
S101, the upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation.
Concrete, comprise the fileinfo of described data file in the upload request that own node receives, described fileinfo comprises the corresponding relation of described node sequence numbering and document processing operation.Above-mentioned upload request directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.Such as, the corresponding relation of described node sequence numbering and document processing operation can be: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.Above-mentioned source node be first send and and the node of upload request described in non-forwarded, can comprise and upload client.
S102, obtains self node sequence numbering according to described upload request.
Concrete, obtain self node sequence numbering according to described upload request, namely obtaining own node is which hop node that described upload request sends from source node.
S103, according to the corresponding relation of described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores also forward process.
Concrete, the node sequence numbering of carrying according to described upload request and the corresponding relation of document processing operation, the document processing operation that the own node serial number obtained in query steps S102 is corresponding.Such as, the node sequence numbering of carrying in described upload request and the corresponding relation of document processing operation are: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.Own node number order is one, and namely own node is the first jumping that described upload request sends from source node, and the document processing operation that so own node is corresponding is exactly forward process.
S104, if described document processing operation is forward process, then according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and the described upload request that with the addition of own node identification information, obtain self node sequence to make described unique next-hop node according to the upload request received and number and perform the document processing operation of the correspondence inquired according to own node serial number.
Concrete, above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.Before receiving described upload request, in own files node, set up routing relation in advance, described routing relation can determine unique next-hop node of described own node.Own node comprises at least two ports, one for sending upload request and data file, one for receiving upload request and data file.Above-mentioned routing relation defines in own node and only with unique next-hop node is connected with the port of data file for receiving upload request with the port of data file for sending upload request.This routing relation can be maintained by setting up the long connection of Socket.
Such as, in the routing relation set up in advance in own node, the port numbers that own node sends upload request and data file is 30, routing relation specifies that the port numbers of reception upload request and the data file be connected with this port numbers is 31, and the node so having No. 31 ports is exactly unique next-hop node of own node.If inquiring document processing operation corresponding to own node serial number in S103 step is forward process, the port numbers 30 of own node is added in described upload request, to having the node for data forwarding file of No. 31 ports and with the addition of the upload request of own node port numbers, obtain self node sequence to make the node having No. 31 ports according to the upload request received and number and perform the document processing operation of the correspondence inquired according to own node serial number.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Fig. 2 is the schematic flow sheet of a kind of distributed file management method in another embodiment of the present invention, and the distributed file management method as shown in the figure in the present embodiment can comprise:
S201, the upload request of received data file, described upload request carries node sequence numbering and the corresponding relation of document processing operation and the file unique identification of described data file.
Concrete, comprise the fileinfo of described data file in the upload request that own node receives, described fileinfo comprises described node sequence numbering and the corresponding relation of document processing operation and the file unique identification of described file.Above-mentioned file unique identification can include but are not limited to No. ID of described data file.No. ID of data file comprises the building maths modec of data file, uniquely represents a data file, such as No. ID " 20140916.50 ", uniquely represents the 50th data file set up on September 16th, 2014.Above-mentioned upload request directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.
S202, judges whether described upload request comprises own node identification information, if not, then performs step S203; If so, then step S206 is performed.
In the embodiment of the present invention, only inquire about document processing operation corresponding to its node sequence numbering for the upload request not comprising own node identification information and perform, can avoid when the routing relation set up in advance is ring-type, all nodes be forward process or store and forward process time the endless loop that formed.Above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.
S203, obtains self node sequence numbering according to described upload request.
Such as, obtain self node sequence according to described upload request and be numbered one, represent that own node is the first hop node that described upload request sends from source node.
S204, according to the corresponding relation of described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores also forward process.
Concrete, the node sequence numbering of carrying according to described upload request and the corresponding relation of document processing operation, the document processing operation that the own node serial number obtained in query steps S203 is corresponding.Such as, the node sequence numbering of carrying in described upload request and the corresponding relation of document processing operation are: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.Own node serial number is one, and namely own node is the first jumping that described upload request sends from source node, and the document processing operation that so own node is corresponding is exactly forward process.
S205, if described document processing operation is stores processor, then according to the path computing rule preset, is calculated the target store path of described data file, stores described data file according to described target store path by described file unique identification.
Concrete, before receiving described upload request, preestablish path computing rule at own node.Described path computing rule can be obtained the target store path of the memory device that described data file manages at own node by the file unique identification of data file.Above-mentioned file unique identification can include but are not limited to No. ID of described data file.Optionally, above-mentioned target store path can be the time of setting up according to data file, forms according to year, month, day classification.Such as, No. ID of data file is " 20140916.50 ", and by path computing rule, the target store path obtaining this data file is exactly " 2014 09 16 ".
S206, by unique upper hop node of the routing relation determination own node preset, returns the prompting of upload request refusal to described unique upper hop node.
Concrete, before the upload request of received data file, in own node, preestablish routing relation.Described routing relation can determine unique upper hop node of own node.Own node comprises at least two ports, one for sending upload request and data file, one for receiving upload request and data file.Above-mentioned routing relation defines in own node and only with unique upper hop node is connected with the port of data file for sending upload request with the port of data file for receiving upload request.This routing relation can be maintained by setting up the long connection of Socket.
Such as, in the routing relation set up in advance in own node, the port numbers that own node receives upload request and data file is 29, routing relation specifies that the port numbers of transmission upload request and the data file be connected with this port numbers is 28, and the node so having No. 28 ports is exactly unique upper hop node of own node.If step S202 judges that described upload request comprises own node identification information, return the prompting of upload request refusal to the node having No. 28 ports.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Fig. 3 is the schematic flow sheet of a kind of distributed file management method in another embodiment of the present invention, and the security documents disposal route as shown in the figure in the present embodiment can comprise:
S301, the upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation.
Concrete, comprise the fileinfo of described data file in the upload request that own node receives, described fileinfo comprises the corresponding relation of described node sequence numbering and document processing operation.Above-mentioned upload request directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.Such as, the corresponding relation of described node sequence numbering and document processing operation can be: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.
S302, calculates the total number of node identification information in described upload request, is added determine own node serial number by the total number of node identification information.
Concrete, judge whether have node identification information in described upload request, if not, then node identification information number is zero, and own node serial number is the first jumping; If so, add up node identification information total number, own node serial number is that the total number of node identification information adds and determines.
S303, according to the corresponding relation of described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores also forward process.
Concrete, the node sequence numbering of carrying according to described upload request and the corresponding relation of document processing operation, the document processing operation that the own node serial number obtained in query steps S302 is corresponding.Such as, the node sequence numbering of carrying in described upload request and the corresponding relation of document processing operation are: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.Own node serial number is two, and namely own node is the second jumping that described upload request sends from source node, and the document processing operation that so own node is corresponding stores and forward process exactly.
S304, if described document processing operation is for storing and forward process, is then stored in the memory device of own node management by described data file; According to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and the described upload request that with the addition of own node identification information, obtain self node sequence to make described unique next-hop node according to the upload request received and number and perform the document processing operation corresponding according to the inquiry of own node serial number.
Concrete, before the described upload request of reception, in own files node, set up routing relation.Described routing relation can determine unique next-hop node of own node.Above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.Such as, if the data file that file ID number is " 20140916.50 " at document processing operation corresponding to own node for store and to forward, then according to unique next-hop node of the routing relation determination own node preset, data file being stored in store path in the memory device of own node management is the position of " 2014 09 16 ", and the port numbers of own node is added in upload request, to unique next-hop node forwarding data file and the described upload request that with the addition of own node port numbers of own node, obtain self node sequence to make described unique next-hop node according to the upload request received number and perform the document processing operation corresponding according to the inquiry of own node serial number.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Fig. 4 is the schematic flow sheet of a kind of distributed file management method in another embodiment of the present invention, and the security documents disposal route as shown in the figure in the present embodiment can comprise:
S401, the download request of received data file, described download request comprises the file unique identification of described data file.
Concrete, comprise the fileinfo of described data file in the download request that own node receives, described fileinfo comprises the file unique identification of described file.Above-mentioned file unique identification can include but are not limited to No. ID of described data file.No. ID of data file comprises the building maths modec of data file, uniquely represents a data file, such as No. ID " 20140916.50 ", uniquely represents the 50th data file set up on September 16th, 2014.Above-mentioned download request directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.Above-mentioned source node be first send and and the node of download request described in non-forwarded, can download client be comprised.
S402, according to the path computing rule preset, is calculated the target query path of described data file by described file unique identification.
Concrete, before the download request of received data file, preestablish path computing rule at own node.Described path computing rule can be obtained the target query path of the memory device that own node manages by the file unique identification of data file.Optionally, above-mentioned target query path can be the time of setting up according to data file, forms according to year, month, day classification.Such as, the file unique identification of data file is No. ID " 20140916.50 ", and by path computing rule, the target query path obtaining this data file is exactly " 2014 09 16 ".
S403, judges position corresponding to target query path described in the memory device whether described data file is present in own node management, if not, then performs step S404.
Concrete, target query path in the memory device managed at own node according to the described data file obtained in step S402, the position that target query path in the memory device of inquiry own node management is corresponding, and judge whether described data file exists, if there is no then perform step S404.
S404, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge the position that described in the memory device whether described data file is present in own node management, target query path is corresponding.
Concrete, above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.Before receiving described download request, in own files node, set up routing relation in advance, described routing relation can determine unique next-hop node of described own node.Own node comprises at least two ports, one for sending download request, one for receiving download request.Above-mentioned routing relation defines in own node and is only connected for the port receiving download request with unique next-hop node for the port sending download request.This routing relation can be maintained by setting up the long connection of Socket.
Such as, in the routing relation set up in advance in own node, the port numbers that own node sends download request is 30, and routing relation specifies that the port numbers of the reception download request be connected with this port numbers is 31, and the node so having No. 31 ports is exactly unique next-hop node of own node.If judge position corresponding to target query path described in the memory device that described data file is not present in own node management in step S403, the port numbers 30 of own node is added in described download request, and forward to the node having No. 31 ports the download request that with the addition of port numbers 30, calculate the target query path of described data file according to the path computing rule preset by described file unique identification with the node making to have No. 31 ports and judge whether described data file is present in the position that described in memory device that own node manages, target query path is corresponding.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Fig. 5 is the schematic flow sheet of a kind of distributed file management method in further embodiment of this invention, and the security documents disposal route as shown in the figure in the present embodiment can comprise:
S501, the download request of received data file, described download request comprises the file unique identification of described data file.
Concrete, the download request that own node receives directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.Above-mentioned file unique identification can include but are not limited to No. ID of described data file.
S502, judges whether comprise described own node identification information in described download request, if so, then performs step S506; If not, then step S503 is performed.
Concrete, before the download request of received data file, preestablish routing relation, described routing relation can determine unique next-hop node of own node.In the embodiment of the present invention, only for not comprising the download request of own node identification information according to the path computing rule preset, the target query path of described data file is calculated by described file unique identification, can avoid when the routing relation set up in advance is ring-type, and the endless loop formed when data file does not all exist described in the position that the target query path of self is corresponding in the memory device of all node administrations.
S503, according to the path computing rule preset, is calculated the target query path of described data file by described file unique identification.
Concrete, before the download request of received data file, preestablish path computing rule at own node.Described path computing rule can be obtained the target query path of the memory device that own node manages by the file unique identification of data file.Optionally, above-mentioned target query path can be the time of setting up according to data file, forms according to year, month, day classification.Such as, the file unique identification of data file is No. ID " 20140916.50 ", and by path computing rule, the target query path obtaining this data file is exactly " 2014 09 16 ".
S504, judges position corresponding to target query path described in the memory device whether described data file is present in own node management, if so, then performs step S507, if not, then perform step S505.
Such as, No. ID that obtains in step S503 is target query path in the memory device that manages at own node of the data file of " 20140916.50 " is " 2014 09 16 ", in the memory device of inquiry own node management, target query path is the position of " 2014 09 16 ", and judge whether No. ID " 20140916.50 " exist, if exist, perform step S507, if do not exist, perform step S505.
S505, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge the position that described in the memory device whether described data file is present in own node management, target query path is corresponding.
Concrete, before the described download request of reception, in own files node, set up routing relation.Above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.If position corresponding to target query path described in the memory device that described data file is not present in own node management, then the unique next-hop node to own node forwards described download request.Such as, according to unique next-hop node of the routing relation determination own node preestablished, own node IP address is added in described download request, and the download request that with the addition of own node IP address is forwarded to described unique next-hop node, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge the position that described in the memory device whether described data file is present in own node management, target query path is corresponding.
S506, is determined the unique upper hop node of self by the routing relation preset, the source node to described unique upper hop node and the described download request of transmission returns the prompting of download request refusal.
Concrete, before the described download request of reception, in own files node, set up routing relation in advance.Described routing relation can determine unique upper hop node of own node.Own node comprises at least two ports, one for sending download request, one for receiving download request.Above-mentioned routing relation defines in own node and only with unique upper hop node is connected for the port sending download request for the port receiving download request.This routing relation can be maintained by setting up the long connection of Socket.
Such as, in the routing relation set up in advance in own node, the port numbers that own node receives download request is 29, and routing relation specifies that the port numbers of the transmission download request be connected with this port numbers is 28, and the node so having No. 28 ports is exactly unique upper hop node of own node.If judge to comprise self port numbers 29 in described download request, then the source node to the node and the described download request of transmission that have No. 28 ports returns the prompting of download request refusal.
S507, reads described data file, described data file is sent to the source node sending described download request.
If position corresponding to target query path described in the memory device that described data file is present in own node management, then read described data file, and described data file sent to the source node sending described download request.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Fig. 6 is the structural representation of a kind of distributed file management device in the embodiment of the present invention, and this device can comprise: receiving element 602, numbering acquiring unit 603, query unit 604 and forward processing unit 605.
Receiving element 602, for the upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation.
Concrete, comprise the fileinfo of described data file in the upload request that receiving element 602 receives, described fileinfo comprises the corresponding relation of described node sequence numbering and document processing operation.Above-mentioned upload request directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.Above-mentioned source node be first send and and the node of upload request described in non-forwarded, can comprise and upload client.Such as, the corresponding relation of described node sequence numbering and document processing operation can be: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.
Numbering acquiring unit 603, for obtaining self node sequence numbering according to described upload request.
Obtain self node sequence numbering according to described upload request, namely own node is which hop node that described upload request sends from source node.Optionally, calculate the total number of node identification information in described upload request, added by the total number of node identification information and determine described own node serial number.Concrete, judge whether have node identification information in described upload request, if not, then node identification information number is zero, and own node serial number is the first jumping; If so, add up node identification information total number, own node serial number is that the total number of node identification information adds and determines.
Query unit 604, for the corresponding relation according to described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores and forward process.
Concrete, the node sequence numbering of carrying according to the upload request of step receiving element 602 reception and the corresponding relation of document processing operation, the document processing operation that the own node serial number that number of queries acquiring unit 603 obtains is corresponding.Such as, the node sequence numbering of carrying in described upload request and the corresponding relation of document processing operation are: first jumps forward process; Second jumps storage and forward process; 3rd jumps stores processor.Own node serial number is one, and namely own node is the first jumping that described upload request sends from source node, and the document processing operation that so own node is corresponding is exactly forward process.
Forward processing unit 605, for be forward process or described document processing operation at described document processing operation for store and forward process time, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and described upload request, to make described unique next-hop node obtain self node sequence numbering according to described upload request, and the document processing operation corresponding according to the inquiry of own node serial number.
Concrete, above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.Before receiving element 602 receives described upload request, set up routing relation in advance in own files node, described routing relation can determine unique next-hop node of described own node.
Optionally, this device also can comprise storage processing unit 606, for be stores processor or described document processing operation at described document processing operation for store and forward process time, described data file is stored in the memory device of own node management.
Concrete, according to the path computing rule preset, calculated the target store path of described data file by the file unique identification of the data file comprised in described upload request, and store described data file according to described target store path.Optionally, above-mentioned target store path can be the time of setting up according to data file, forms according to year, month, day classification.
Further alternative, this device also can comprise request judging unit 601, before obtaining self node sequence numbering at described numbering acquiring unit 603 according to described upload request, judge in described upload request, whether to comprise described own node identification information, if, then by unique upper hop node of the routing relation determination own node preset, return the prompting of upload request refusal to described unique upper hop node; If not, then obtain self node sequence by described numbering acquiring unit according to described upload request to number.
In the embodiment of the present invention, query unit is only inquired about document processing operation corresponding to its node sequence numbering for the upload request not comprising own node identification information and is performed, can avoid when the routing relation set up in advance is ring-type, all nodes be forward process or store and forward process time the endless loop that formed.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
Fig. 7 is the structural representation of a kind of distributed file management device in another embodiment of the present invention, and this device can comprise: receiving element 702, computing unit 703, file judging unit 704 and request forward unit 705.
Receiving element 702, for the download request of received data file, described download request comprises the file unique identification of described data file.
Concrete, comprise the fileinfo of described data file in the download request that receiving element 702 receives, described fileinfo comprises the file unique identification of described file.Above-mentioned download request directly can be sent by source node, also can be to be forwarded by unique upper hop node of own node.Above-mentioned source node be first send and and the node of download request described in non-forwarded, can download client be comprised.
Computing unit 703, for according to the path computing rule preset, is calculated the target query path of described data file by described file unique identification.
Concrete, before the download request of receiving element 702 received data file, in own node, preestablish path computing rule.Described path computing rule can be obtained the target query path of the memory device that own node manages by the file unique identification of data file.Optionally, above-mentioned target query path can be the time of setting up according to data file, forms according to year, month, day classification.
File judging unit 704, for judging position corresponding to target query path described in the memory device whether described data file is present in own node management.
Concrete, target query path in the memory device that the described data file obtained according to computing unit 703 manages at own node, the position that target query path in the memory device of inquiry own node management is corresponding, and judge whether described data file exists.
Request forward unit 705, during for judging position corresponding to target query path described in the memory device that described data file is not present in own node management at file judging unit, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge whether described data file is present in position corresponding to target query path described in memory device that own node manages.
Concrete, before the download request of receiving element 702 received data file, preestablish routing relation in own node, described routing relation can determine unique next-hop node of own node.Above-mentioned own node mark includes but are not limited to the IP address of own node or the port numbers of own node.
Optionally, this device also can comprise request judging unit 701, for regular according to the path computing preset at described computing unit, calculated the target query path of described data file by described file unique identification before, judge in described download request, whether to comprise described own node identification information; If so, then determined the unique upper hop node of self by the routing relation preset, the source node to described unique upper hop node and the described download request of transmission returns the prompting of download request refusal; If not, then regular according to the path computing preset by described computing unit, the target query path of described data file is calculated by described file unique identification.
Concrete, before the download request of receiving element 702 received data file, preestablish routing relation in own node, described routing relation can determine unique next-hop node of own node.In the embodiment of the present invention, computing unit 703 is only for the download request not comprising own node identification information, according to the path computing rule preset, the target query path of described data file is calculated by described file unique identification, can avoid when the routing relation set up in advance is ring-type, and the endless loop formed when data file does not all exist described in the position that the target query path of self is corresponding in the memory device of all node administrations.Above-mentioned own node identification information can include but are not limited to the IP address of own node or the port numbers of own node.
Further alternative, this device also can comprise download unit 706, during for judging position corresponding to target query path described in the memory device that described data file is present in own node management at described file judging unit, read described data file, and described data file is sent to the source node sending described download request.
Concrete, when file judging unit 704 judges position corresponding to target query path described in the memory device that described data file is present in own node management, download unit 706 reads described data file, and described data file is sent to the source node sending described download request.
By implementing the embodiment of the present invention, when the document processing operation that own node serial number is corresponding is forward process, the upload request of data file is transmitted to unique down hop Archive sit process of own node, or described in the memory device that data file is not present in own node management during position corresponding to target query path, the download request of data file is transmitted to unique next-hop node process of own node, thus balanced node load, reduce node overheads.
One of ordinary skill in the art will appreciate that all or part of flow process realized in above-described embodiment method, that the hardware that can carry out instruction relevant by computer program has come, described program can be stored in a computer read/write memory medium, this program, when performing, can comprise the flow process of the embodiment as above-mentioned each side method.Wherein, described storage medium can be magnetic disc, CD, read-only store-memory body (Read-Only Memory, ROM) or random store-memory body (Random Access Memory, RAM) etc.
Above disclosedly be only present pre-ferred embodiments, certainly can not limit the interest field of the present invention with this, therefore according to the equivalent variations that the claims in the present invention are done, still belong to the scope that the present invention is contained.
Claims (17)
1. a distributed file management method, is characterized in that, comprising:
The upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation;
Self node sequence numbering is obtained according to described upload request;
According to the corresponding relation of described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores also forward process;
If described document processing operation is forward process, then according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and the described upload request that with the addition of own node identification information, obtain self node sequence to make described unique next-hop node according to the upload request received and number and perform the document processing operation of the correspondence inquired according to own node serial number.
2. method according to claim 1, is characterized in that, also comprise:
If described document processing operation is stores processor, then described data file is stored in the memory device of own node management.
3. method according to claim 1, is characterized in that, also comprise:
If described document processing operation is for storing and forward process, then described data file is stored in the memory device of own node management;
According to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and the described upload request that with the addition of own node identification information, obtain self node sequence to make described unique next-hop node according to the upload request received and number and perform the document processing operation of the correspondence inquired according to own node serial number.
4. method according to claim 1, is characterized in that, describedly obtains self node sequence numbering according to described upload request and comprises:
Calculate the total number of node identification information in described upload request;
Described own node serial number is added by the total number of node identification information to be determined.
5. method according to Claims 2 or 3, is characterized in that, described upload request also comprises the file unique identification of described data file;
The described memory device described data file being stored in own node management comprises:
According to the path computing rule preset, calculated the target store path of described data file by described file unique identification;
Described data file is stored according to described target store path.
6. method according to claim 1, is characterized in that, described obtain self node sequence numbering according to described upload request before, described method also comprises:
Judge whether described upload request comprises described own node identification information; If so, then by unique upper hop node of the routing relation determination own node preset, the prompting of upload request refusal is returned to described unique upper hop node; If not, then the described step obtaining self node sequence numbering according to described upload request is triggered.
7. a distributed file management method, is characterized in that, comprising:
The download request of received data file, described download request comprises the file unique identification of described data file;
According to the path computing rule preset, calculated the target query path of described data file by described file unique identification;
Judge position corresponding to target query path described in the memory device whether described data file is present in own node management;
If not, then according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge the position that described in the memory device whether described data file is present in own node management, target query path is corresponding.
8. method according to claim 7, is characterized in that, the path computing rule that described basis is preset, and calculated the target query path of described data file by described file unique identification before, described method also comprises:
Judge in described download request, whether to comprise described own node identification information; If so, then by unique upper hop node of the routing relation determination own node preset, the source node to described unique upper hop node and the described download request of transmission returns the prompting of download request refusal; If not, then trigger the path computing rule that described basis is preset, calculated the step in the target query path of described data file by described file unique identification.
9. method according to claim 7, is characterized in that, also comprise:
If judge position corresponding to target query path described in the memory device that described data file is present in own node management, then read described data file, described data file is sent to the source node sending described download request.
10. a distributed file management device, is characterized in that, comprising:
Receiving element, for the upload request of received data file, described upload request carries the corresponding relation of node sequence numbering and document processing operation;
Numbering acquiring unit, for obtaining self node sequence numbering according to described upload request;
Query unit, for the corresponding relation according to described node sequence numbering and document processing operation, the document processing operation that inquiry own node serial number is corresponding, described document processing operation comprises stores processor, forward process or stores and forward process;
Forward processing unit, for be forward process or described document processing operation at described document processing operation for store and forward process time, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described upload request, to described unique next-hop node forwarding data file and described upload request, to make described unique next-hop node obtain self node sequence numbering according to described upload request, and the document processing operation corresponding according to the inquiry of own node serial number.
11. devices according to claim 10, it is characterized in that, also comprise storage processing unit, for be stores processor or described document processing operation at described document processing operation for store and forward process time, described data file is stored in the memory device of own node management.
12. devices according to claim 10, it is characterized in that, described numbering acquiring unit is used for:
Calculate the total number of node identification information in described upload request, and added by the total number of node identification information and determine described own node serial number.
13., according to device described in claim 11, is characterized in that, described upload request also comprises the file unique identification of described data file;
Described storage processing unit, for according to the path computing rule preset, is calculated the target store path of described data file, and stores described data file according to described target store path by described file unique identification.
14. devices according to claim 10, is characterized in that, also comprise:
Request judging unit, before obtaining self node sequence numbering at described numbering acquiring unit according to described upload request, judge in described upload request, whether to comprise described own node identification information, if, then by unique upper hop node of the routing relation determination own node preset, return the prompting of upload request refusal to described unique upper hop node; If not, then obtain self node sequence by described numbering acquiring unit according to described upload request to number.
15. 1 kinds of distributed file management devices, is characterized in that, comprising:
Receiving element, for the download request of received data file, described download request comprises the file unique identification of described data file;
Computing unit, for according to the path computing rule preset, is calculated the target query path of described data file by described file unique identification;
Request forward unit, during for judging position corresponding to target query path described in the memory device that described data file is not present in own node management at file judging unit, according to unique next-hop node of the routing relation determination own node preset, own node identification information is added in described download request, and described in forwarding to described unique next-hop node, with the addition of the download request of own node identification information, calculate the target query path of described data file according to the path computing rule preset by described file unique identification to make described unique next-hop node and judge whether described data file is present in position corresponding to target query path described in memory device that own node manages.
16., according to device described in claim 15, is characterized in that, also comprise:
Request judging unit, for regular according to the path computing preset at described computing unit, before being calculated the target query path of described data file, judges whether comprise described own node identification information in described download request by described file unique identification; If so, then by unique upper hop node of the routing relation determination own node preset, the source node to described unique upper hop node and the described download request of transmission returns the prompting of download request refusal; If not, then regular according to the path computing preset by described computing unit, the target query path of described data file is calculated by described file unique identification.
17., according to device described in claim 15, is characterized in that, also comprise download unit:
During for judging position corresponding to target query path described in the memory device that described data file is present in own node management at described file judging unit, read described data file, and described data file is sent to the source node sending described download request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410594431.5A CN104408063B (en) | 2014-10-29 | 2014-10-29 | A kind of distributed file management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410594431.5A CN104408063B (en) | 2014-10-29 | 2014-10-29 | A kind of distributed file management method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104408063A true CN104408063A (en) | 2015-03-11 |
| CN104408063B CN104408063B (en) | 2019-04-02 |
Family
ID=52645695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410594431.5A Active CN104408063B (en) | 2014-10-29 | 2014-10-29 | A kind of distributed file management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104408063B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111327524A (en) * | 2018-12-14 | 2020-06-23 | 中国电信股份有限公司 | Flow forwarding method and system, SDN controller and computer readable storage medium |
| CN115170095A (en) * | 2022-09-07 | 2022-10-11 | 浪潮通信信息系统有限公司 | Order processing method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080310302A1 (en) * | 2007-06-18 | 2008-12-18 | Sony Computer Entertainment Inc. | Load balancing distribution of data to multiple recipients on a peer-to-peer network |
| US20100082774A1 (en) * | 2005-09-09 | 2010-04-01 | Pitts William M | Distributed File System Consistency Mechanism Extension for Enabling Internet Video Broadcasting |
| CN102571968A (en) * | 2012-01-17 | 2012-07-11 | 华中科技大学 | Directional data forwarding method in cloud backup system |
| CN103283189A (en) * | 2010-12-27 | 2013-09-04 | 日本电气株式会社 | Communication system and communication method |
| CN103535011A (en) * | 2011-12-14 | 2014-01-22 | 华为技术有限公司 | Routing method, device, and system in content delivery network (CDN) |
-
2014
- 2014-10-29 CN CN201410594431.5A patent/CN104408063B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100082774A1 (en) * | 2005-09-09 | 2010-04-01 | Pitts William M | Distributed File System Consistency Mechanism Extension for Enabling Internet Video Broadcasting |
| US20080310302A1 (en) * | 2007-06-18 | 2008-12-18 | Sony Computer Entertainment Inc. | Load balancing distribution of data to multiple recipients on a peer-to-peer network |
| CN103283189A (en) * | 2010-12-27 | 2013-09-04 | 日本电气株式会社 | Communication system and communication method |
| CN103535011A (en) * | 2011-12-14 | 2014-01-22 | 华为技术有限公司 | Routing method, device, and system in content delivery network (CDN) |
| CN102571968A (en) * | 2012-01-17 | 2012-07-11 | 华中科技大学 | Directional data forwarding method in cloud backup system |
Non-Patent Citations (2)
| Title |
|---|
| HAOJUN LIAO,JIZHONG HAN,JINYUN FANG: "Multi-dimensional Index on Hadoop Distributed File System", 《2010 FIFTH IEEE INTERNATIONAL CONFERENCE ON NETWORKING, ARCHITECTURE, AND STORAGE》 * |
| 张宇: "基于对象存储的分布式小文件系统的研究与设计", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111327524A (en) * | 2018-12-14 | 2020-06-23 | 中国电信股份有限公司 | Flow forwarding method and system, SDN controller and computer readable storage medium |
| CN111327524B (en) * | 2018-12-14 | 2022-06-07 | 中国电信股份有限公司 | Flow forwarding method and system, SDN controller and computer readable storage medium |
| CN115170095A (en) * | 2022-09-07 | 2022-10-11 | 浪潮通信信息系统有限公司 | Order processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104408063B (en) | 2019-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10959089B2 (en) | Data management microservice in a microservice domain | |
| US12015666B2 (en) | Systems and methods for distributing partial data to subnetworks | |
| US11181382B2 (en) | Generating maps of private spaces using mobile computing device sensors | |
| US7995504B2 (en) | Locality-based routing table generation | |
| CN104247371A (en) | Name-based neighbor discovery and multi-hop service discovery in information-centric networks | |
| CN106790131B (en) | Parameter modification method and device and distributed platform | |
| CN104809601A (en) | Express delivery mutual aid system based on electronic map | |
| CN103139081A (en) | Update method and nodes for distributed hash table routing lists | |
| CN112217655B (en) | Network equipment configuration method and device in SD-WAN system and computer equipment | |
| CN109977635B (en) | Method, device, equipment and medium for processing traceable data based on block chain | |
| CN103067521A (en) | Distributed-type nodes and distributed-type system in a crawler cluster | |
| CN103200196A (en) | Accessing method, system and device between user equipment and accessing target | |
| CN104298687A (en) | Hash partitioning management method and device | |
| CN105450585A (en) | Information transmission method and device | |
| US20170318521A1 (en) | Method for realizing resource attribute notification, and common service entity | |
| CN114024886A (en) | Cross-resource-pool network intercommunication method, electronic equipment and readable storage medium | |
| CN104408063A (en) | Distributed file management method and device | |
| JP2006221423A (en) | Data distribution method and device and storage medium | |
| EP3345345B1 (en) | Systems and methods for remote network topology discovery | |
| RU2715285C1 (en) | Method of increasing stability of a secure connection between elements of a corporate management system using information and communication resources, which are controlled by two or more communication operators | |
| CN110611591B (en) | Network topology establishing method and device | |
| JP2010271953A (en) | Information exchange/share system, method and program thereof | |
| CN105282025A (en) | Method of determining end-to-end routing and apparatus thereof | |
| CN108011991B (en) | Data stream forwarding method, main control board, interface board, engine board and distributed firewall | |
| CN102035663B (en) | Multi-domain routing maintenance method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhang Xiaobiao Inventor after: Zheng Bangdong Inventor after: Xu Li Inventor before: Zhang Xiaobiao Inventor before: Zheng Bangdong Inventor before: Xu Li |