CN104408063B - A kind of distributed file management method and device - Google Patents
A kind of distributed file management method and device Download PDFInfo
- Publication number
- CN104408063B CN104408063B CN201410594431.5A CN201410594431A CN104408063B CN 104408063 B CN104408063 B CN 104408063B CN 201410594431 A CN201410594431 A CN 201410594431A CN 104408063 B CN104408063 B CN 104408063B
- Authority
- CN
- China
- Prior art keywords
- node
- file
- data file
- unique
- upload request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/122—File system administration, e.g. details of archiving or snapshots using management policies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
Abstract
The embodiment of the present invention provides a kind of distributed file management method and device, which comprises receives the upload request of data file, the upload request carries the corresponding relationship of node sequence number and document processing operation；Itself node sequence number is obtained according to the upload request；According to the corresponding relationship of node sequence number and document processing operation, the corresponding document processing operation of inquiry own node serial number；If the document processing operation is forward process, unique next-hop node of own node is then determined according to preset routing relation, own node identification information is added in upload request, it to the unique next-hop node forwarding data file and is added to the upload request of own node identification information, numbers so that unique next-hop node obtains itself node sequence according to the upload request received and execute the corresponding document processing operation inquired according to own node serial number.The present invention can balanced node load, reduce node overheads.
Description
Technical field
The present invention relates to computer field more particularly to a kind of distributed file management method and devices.
Background technique
With the development of bank finance industry, the data files such as voucher bill that business generates are more and more, this is just needed
One capacity big system manages these data files, so that convenient stored and called to it.Currently, bank management number
Main method according to file is cloud storage, and when upper transmitting file, client sends file to the management node of cloud storage system and uploads
Request, management node store upper transmitting file into database, and record the file information and its storage address of upper transmitting file；Downloading
When file, client sends file download request to the management node of cloud storage system, and management node is wrapped according in downloading request
The file information of the downloading file contained inquires the storage address of the downloading file, then downloads this document by network and sends to
Client.One management node of cloud storage needs to handle all upload requests of client transmission and downloading request, load are opened
Pin is big.
Summary of the invention
The embodiment of the present invention provides a kind of distributed file management method and device, can balanced node load, reduce node
Expense.
First aspect present invention provides a kind of distributed file management method, it may include:
The upload request of data file is received, the upload request carries pair of node sequence number and document processing operation
It should be related to；
Itself node sequence number is obtained according to the upload request；
According to the corresponding relationship of node sequence number and document processing operation, inquiry own node serial number is corresponding
Document processing operation, the document processing operation include storage processing, forward process or storage and forward process；
If the document processing operation is forward process, determine own node only according to preset routing relation
Own node identification information is added in the upload request by one next-hop node, is forwarded to unique next-hop node
Data file and the upload request for being added to own node identification information, so that unique next-hop node is according to reception
To upload request obtain itself node sequence and number and execute the corresponding file inquired according to own node serial number
Processing operation.
Second aspect of the present invention provides a kind of distributed file management method, it may include:
The downloading request of data file is received, the downloading request includes the file unique identification of the data file；
According to preset path computing rule, the target query road of the data file is calculated by the file unique identification
Diameter；
Judge the data file with the presence or absence of the target query path pair described in the storage equipment of own node management
The position answered；
If it is not, unique next-hop node of own node is then determined according to preset routing relation, by own node
Identification information is added in the downloading request, and is added to own node mark to described in unique next-hop node forwarding
The downloading of information is requested, so that unique next-hop node is according to preset path computing rule by the file unique identification
It calculates the target query path of the data file and judges that the data file whether there is in the storage of own node management
The corresponding position of target query path described in equipment.
Third aspect present invention provides a kind of distributed file management device, it may include:
Receiving unit, the upload request of file, the upload request carry node sequence number and text for receiving data
The corresponding relationship of part processing operation；
Number acquiring unit, for obtaining itself node sequence number according to the upload request；
Query unit inquires itself section for the corresponding relationship according to node sequence number and document processing operation
Dot sequency numbers corresponding document processing operation, and the document processing operation includes storage processing, forward process or storage and turns
Hair processing；
Forward processing unit, for being forward process in the document processing operation or the document processing operation is storage
And when forward process, unique next-hop node of own node is determined according to preset routing relation, by own node mark
Know information to be added in the upload request, forwards data file and the upload request to unique next-hop node, with
So that unique next-hop node is obtained itself node sequence number according to the upload request, and is compiled according to own node sequence
Number corresponding document processing operation of inquiry.
Fourth aspect present invention provides a kind of distributed file management device, it may include:
Receiving unit, the downloading request of file for receiving data, the downloading request includes the text of the data file
Part unique identification；
Computing unit, for calculating the data text by the file unique identification according to preset path computing rule
The target query path of part；
Retransmission unit is requested, for judging that the data file is not present in own node management in file judging unit
When storing the corresponding position in target query path described in equipment, determine own node only according to preset routing relation
One next-hop node by the addition of own node identification information in downloading request, and turns to unique next-hop node
The downloading request of own node identification information is added to described in hair, so that unique next-hop node is counted according to preset path
Rule is calculated to calculate the target query path of the data file by the file unique identification and whether judge the data file
It is present in the corresponding position in target query path described in the storage equipment of own node management.
The implementation of the embodiments of the present invention has the following beneficial effects:
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow diagram of one of embodiment of the present invention distributed file management method；
Fig. 2 is a kind of flow diagram of distributed file management method in another embodiment of the present invention；
Fig. 3 is a kind of flow diagram of distributed file management method in another embodiment of the present invention；
Fig. 4 is a kind of flow diagram of distributed file management method in another embodiment of the present invention；
Fig. 5 is a kind of flow diagram of distributed file management method in further embodiment of this invention；
Fig. 6 is the structural schematic diagram of one of embodiment of the present invention distributed file management device；
Fig. 7 is a kind of structural schematic diagram of distributed file management device in another embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Data file as mentioned in the embodiments of the present invention may include but be not limited only to generate in bank finance industry all kinds of
Voucher bill image file.The storage equipment of node administration as mentioned in the embodiments of the present invention may include but be not limited only to NAS
(Network Attached Storage, the storage of network connection formula) equipment.The distributed file management side of the embodiment of the present invention
The application scenarios of method can with but be not limited only in the storage and calling of all kinds of voucher bill image files of bank finance industry.
Fig. 1 is the flow diagram of one of embodiment of the present invention distributed file management method, this reality as shown in the figure
The distributed file management method applied in example may include:
S101, receives the upload request of data file, and the upload request carries node sequence number and file process behaviour
The corresponding relationship of work.
Specifically, including the file information of the data file, the file in the upload request that own node receives
Information includes the corresponding relationship of the node sequence number and document processing operation.Above-mentioned upload request can be straight by source node
It receives and sends, is also possible to by unique upper hop node forwarding of own node.For example, the node sequence number and file
The corresponding relationship of processing operation may is that the first jump forward process；Second jumps storage and forward process；Third jumps storage processing.
Above-mentioned source node be first issue and and the non-forwarded upload request node, may include upload client.
S102 obtains itself node sequence number according to the upload request.
Specifically, obtaining itself node sequence number according to the upload request, i.e. acquisition own node is the upload
Which hop node issued from source node requested.
S103, according to the corresponding relationship of node sequence number and document processing operation, inquiry own node sequence is compiled
Number corresponding document processing operation, the document processing operation include storage processing, forward process or storage and forward process.
Specifically, the corresponding relationship of the node sequence number and document processing operation carried according to the upload request, is looked into
Ask the corresponding document processing operation of own node serial number obtained in step S102.For example, being carried in the upload request
Node sequence number and document processing operation corresponding relationship are as follows: first jump forward process；Second jumps storage and forward process；
Third jumps storage processing.Own node number order is one, i.e., own node is that the upload request is issued from source node
One jumps, then the corresponding document processing operation of own node is exactly forward process.
S104 determines that itself is saved according to preset routing relation if the document processing operation is forward process
Unique next-hop node of point, own node identification information is added in the upload request, to unique next-hop section
Point forwarding data file and the upload request for being added to own node identification information, so that unique next-hop node root
Itself node sequence is obtained according to the upload request received to number and execute the correspondence inquired according to own node serial number
Document processing operation.
Specifically, above-mentioned own node identification information may include but be not limited only to the IP address or itself section of own node
The port numbers of point.Before receiving the upload request, routing relation, the routing relation are pre-established in own files node
It can determine unique next-hop node of the own node.Own node contains at least two port, and one for sending
Biography asks summed data file, and one for receiving upload request and data file.Above-mentioned routing relation defines in own node
For send the port of upload request and data file only in unique next-hop node for receiving upload request and data text
The port of part is connected.The routing relation can be maintained by establishing Socket long connection.
For example, own node sends upload request and data file in the routing relation pre-established in own node
Port numbers be 30, the port numbers of reception upload request and data file that routing relation regulation is connected with the port numbers are 31,
The node for so possessing No. 31 ports is exactly unique next-hop node of own node.If inquiring itself section in S103 step
It is forward process that dot sequency, which numbers corresponding document processing operation, and the port numbers 30 of own node are added to the upload request
In, the node for data forwarding file of No. 31 ports Xiang Yongyou and it is added to the upload request of own node port numbers, so as to possess
The node of No. 31 ports obtains itself node sequence according to the upload request received and numbers and execute according to own node sequence
Number the corresponding document processing operation inquired.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Fig. 2 is a kind of flow diagram of distributed file management method in another embodiment of the present invention, as shown in the figure originally
Distributed file management method in embodiment may include:
S201, receives the upload request of data file, and the upload request carries node sequence number and file process behaviour
The file unique identification of the corresponding relationship of work and the data file.
Specifically, including the file information of the data file, the file in the upload request that own node receives
Information includes the node sequence number and the corresponding relationship of document processing operation and the file unique identification of the file.On
State the ID number that file unique identification may include but be not limited only to the data file.The ID number of data file includes data file
Building maths modec, uniquely represent a data file, such as ID number " 20140916.50 ", uniquely represent on September 16th, 2014 and build
The 50th vertical data file.Above-mentioned upload request can be to be directly transmitted by source node, is also possible to by own node
Unique upper hop node forwarding.
S202 judges whether the upload request includes own node identification information, if it is not, thening follow the steps S203；If
It is to then follow the steps S206.
In the embodiment of the present invention, its node sequence is inquired only for the upload request for not including own node identification information and is compiled
Number corresponding document processing operation simultaneously executes, can be to avoid when the routing relation pre-established is ring-type, and all nodes are
The endless loop formed when forward process or storage and forward process.Above-mentioned own node identification information may include but not only limit
In the IP address of own node or the port numbers of own node.
S203 obtains itself node sequence number according to the upload request.
For example, obtaining itself node sequence number according to the upload request is one, indicate that own node is the upload
Request the first hop node issued from source node.
S204, according to the corresponding relationship of node sequence number and document processing operation, inquiry own node sequence is compiled
Number corresponding document processing operation, the document processing operation include storage processing, forward process or storage and forward process.
Specifically, the corresponding relationship of the node sequence number and document processing operation carried according to the upload request, is looked into
Ask the corresponding document processing operation of own node serial number obtained in step S203.For example, being carried in the upload request
Node sequence number and document processing operation corresponding relationship are as follows: first jump forward process；Second jumps storage and forward process；
Third jumps storage processing.Own node serial number is one, i.e., own node is that the upload request is issued from source node
One jumps, then the corresponding document processing operation of own node is exactly forward process.
S205, if the document processing operation is storage processing, according to preset path computing rule, by the file
Unique identification calculates the target store path of the data file, stores the data file according to the target store path.
Specifically, preestablishing path computing rule in own node before receiving the upload request.The path computing
Rule can be obtained the data file in the mesh of the storage equipment of own node management by the file unique identification of data file
Mark store path.Above-mentioned file unique identification may include but be not limited only to the ID number of the data file.Optionally, above-mentioned mesh
Mark store path can be the time established according to data file, is classified and constitutes according to year, month, day.For example, the ID of data file
Number be " 20140916.50 ", passage path computation rule obtain the data file target store path be exactly " 2014 09
16”。
S206 is determined unique upper hop node of own node by preset routing relation, to unique upper hop section
Point returns to upload request refusal prompt.
Specifically, preestablishing routing relation in own node before the upload request of reception data file.The routing
Relationship can determine unique upper hop node of own node.Own node contains at least two port, and one for sending
Biography asks summed data file, and one for receiving upload request and data file.Above-mentioned routing relation defines in own node
For receive the port of upload request and data file only in unique upper hop node for sending upload request and data text
The port of part is connected.The routing relation can be maintained by establishing Socket long connection.
For example, own node receives upload request and data file in the routing relation pre-established in own node
Port numbers be 29, the port numbers of transmission upload request and data file that routing relation regulation is connected with the port numbers are 28,
The node for so possessing No. 28 ports is exactly unique upper hop node of own node.If step S202 judges that the upload is asked
It asks comprising own node identification information, the node of No. 28 ports Xiang Yongyou returns to upload request refusal prompt.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Fig. 3 is a kind of flow diagram of distributed file management method in another embodiment of the present invention, as shown in the figure originally
Security documents processing method in embodiment may include:
S301, receives the upload request of data file, and the upload request carries node sequence number and file process behaviour
The corresponding relationship of work.
Specifically, including the file information of the data file, the file in the upload request that own node receives
Information includes the corresponding relationship of the node sequence number and document processing operation.Above-mentioned upload request can be straight by source node
It receives and sends, is also possible to by unique upper hop node forwarding of own node.For example, the node sequence number and file
The corresponding relationship of processing operation may is that the first jump forward process；Second jumps storage and forward process；Third jumps storage processing.
S302, calculates the node identification information total number in the upload request, adds one by node identification information total number
Determine own node serial number.
Specifically, judging whether there is node identification information in the upload request, if it is not, then node identification information number is
Zero, own node serial number is the first jump；If so, statistics node identification information total number, own node serial number is section
Point identification information total number adds one to determine.
S303, according to the corresponding relationship of node sequence number and document processing operation, inquiry own node sequence is compiled
Number corresponding document processing operation, the document processing operation include storage processing, forward process or storage and forward process.
Specifically, the corresponding relationship of the node sequence number and document processing operation carried according to the upload request, is looked into
Ask the corresponding document processing operation of own node serial number obtained in step S302.For example, being carried in the upload request
Node sequence number and document processing operation corresponding relationship are as follows: first jump forward process；Second jumps storage and forward process；
Third jumps storage processing.Own node serial number is two, i.e., own node is that the upload request is issued from source node
Two jump, then the corresponding document processing operation of own node is exactly storage and forward process.
The data file is stored in itself section if the document processing operation is storage and forward process by S304
In the storage equipment of point management；Unique next-hop node that own node is determined according to preset routing relation, by itself
Node identification information is added in the upload request, to the unique next-hop node forwarding data file and described is added to
The upload request of own node identification information, so that unique next-hop node obtains itself according to the upload request received
Node sequence, which is numbered and executed, inquires corresponding document processing operation according to own node serial number.
Specifically, establishing routing relation in own files node before receiving the upload request.The routing is closed
System can determine unique next-hop node of own node.Above-mentioned own node identification information may include but be not limited only to itself
The IP address of node or the port numbers of own node.For example, if file ID number is the data file of " 20140916.50 " at itself
The corresponding document processing operation of node is to store and forward, then determines the unique of own node according to preset routing relation
Data file is stored in the position that store path in the storage equipment of own node management is " 2014 09 16 " by next-hop node
It sets, and by the addition of the port numbers of own node in upload request, to unique next-hop node of own node forwarding data text
Part and the upload request for being added to own node port numbers, so that unique next-hop node is according to the upload received
Request own node serial number is simultaneously executed according to the corresponding document processing operation of own node serial number inquiry.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Fig. 4 is a kind of flow diagram of distributed file management method in another embodiment of the present invention, as shown in the figure originally
Security documents processing method in embodiment may include:
S401, receives the downloading request of data file, and the downloading request is uniquely marked comprising the file of the data file
Know.
Specifically, including the file information of the data file, the file in the downloading request that own node receives
Information includes the file unique identification of the file.Above-mentioned file unique identification may include but be not limited only to the data file
ID number.The ID number of data file includes the building maths modec of data file, uniquely represents a data file, such as ID number
" 20140916.50 " uniquely represent on September 16th, 2014 the 50th data files established.Above-mentioned downloading request can be by
What source node directly transmitted, it is also possible to by unique upper hop node forwarding of own node.Above-mentioned source node is first
Issue and and it is non-forwarded it is described downloading request node, may include downloading client.
S402 is calculated the target of the data file by the file unique identification according to preset path computing rule
Query path.
Specifically, preestablishing path computing rule in own node before the downloading for receiving data file is requested.It is described
Path computing rule can be obtained the target query of the storage equipment of own node management by the file unique identification of data file
Path.Optionally, above-mentioned target query path can be the time established according to data file, be classified structure according to year, month, day
At.For example, the file unique identification of data file is ID number " 20140916.50 ", passage path computation rule obtains the data
The target query path of file is exactly " 2014 09 16 ".
S403 judges the data file with the presence or absence of the target query road described in the storage equipment of own node management
The corresponding position of diameter, if it is not, thening follow the steps S404.
Specifically, mesh of the data file according to obtained in step S402 in the storage equipment of own node management
Query path is marked, inquires the corresponding position in target query path in the storage equipment of own node management, and judge the number
It whether there is according to file, if there is no thening follow the steps S404.
S404 determines unique next-hop node of own node according to preset routing relation, by own node mark
Know information to be added in the downloading request, and is added to own node mark letter to described in unique next-hop node forwarding
The downloading of breath is requested, so that unique next-hop node is according to preset path computing rule by the file unique identification meter
It calculates the target query path of the data file and judges that the data file whether there is and set in the storage of own node management
The corresponding position in target query path described in standby.
Specifically, above-mentioned own node identification information may include but be not limited only to the IP address or itself section of own node
The port numbers of point.Before receiving the downloading request, routing relation, the routing relation are pre-established in own files node
It can determine unique next-hop node of the own node.Own node contains at least two port, under one for sending
Request is carried, one for receiving downloading request.Above-mentioned routing relation defines in own node for sending the end of downloading request
Mouth is only connected with unique next-hop node for receiving the port of downloading request.The routing relation can be by establishing Socket
Long connection is maintained.
For example, in the routing relation pre-established in own node, the port numbers that own node sends downloading request are
30, the port numbers for the reception downloading request that routing relation regulation is connected with the port numbers are 31, then possessing the section of No. 31 ports
Point is exactly unique next-hop node of own node.If judging that the data file is not present in own node in step S403
The port numbers 30 of own node are added to the downloading by the corresponding position in target query path described in the storage equipment of management
In request, and the downloading request for being added to port numbers 30 is forwarded to the node for possessing No. 31 ports, so as to possess No. 31 ports
Node calculates the target query path of the data file simultaneously by the file unique identification according to preset path computing rule
Judge the data file with the presence or absence of the corresponding position in target query path described in the storage equipment of own node management.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Fig. 5 is a kind of flow diagram of distributed file management method in further embodiment of this invention, as shown in the figure originally
Security documents processing method in embodiment may include:
S501, receives the downloading request of data file, and the downloading request is uniquely marked comprising the file of the data file
Know.
Specifically, the downloading request that own node receives can be and be directly transmitted by source node, be also possible to by from
Unique upper hop node forwarding of body node.Above-mentioned file unique identification may include but be not limited only to the data file
ID number.
Whether S502 judges in the downloading request comprising the own node identification information, if so, thening follow the steps
S506；If it is not, thening follow the steps S503.
Specifically, preestablish routing relation before receiving the downloading request of data file, the routing relation can be with
Determine unique next-hop node of own node.In the embodiment of the present invention, only for not comprising under own node identification information
Request is carried according to preset path computing rule, the target query road of the data file is calculated by the file unique identification
Diameter, can be to avoid when the routing relation pre-established be ring-type, and itself the mesh in the storage equipment of all node administrations
The endless loop formed in the absence of data file described in the corresponding position of mark query path is equal.
S503 is calculated the target of the data file by the file unique identification according to preset path computing rule
Query path.
Specifically, preestablishing path computing rule in own node before receiving the downloading request of data file.Institute
The target for stating the storage equipment that path computing rule can obtain own node management by the file unique identification of data file is looked into
Ask path.Optionally, above-mentioned target query path can be the time established according to data file, be classified structure according to year, month, day
At.For example, the file unique identification of data file is ID number " 20140916.50 ", passage path computation rule obtains the data
The target query path of file is exactly " 2014 09 16 ".
S504 judges the data file with the presence or absence of the target query road described in the storage equipment of own node management
The corresponding position of diameter, if so, S507 is thened follow the steps, if it is not, thening follow the steps S505.
For example, ID number obtained in step S503 is data file the depositing in own node management of " 20140916.50 "
Storing up the target query path in equipment is " 2014 09 16 ", inquires target query path in the storage equipment of own node management
For the position of " 2014 09 16 ", and judge that ID number " 20140916.50 " whether there is, and if it exists, step S507 is executed, if not
In the presence of execution step S505.
S505 determines unique next-hop node of own node according to preset routing relation, by own node mark
Know information to be added in the downloading request, and is added to own node mark letter to described in unique next-hop node forwarding
The downloading of breath is requested, so that unique next-hop node is according to preset path computing rule by the file unique identification meter
It calculates the target query path of the data file and judges that the data file whether there is and set in the storage of own node management
The corresponding position in target query path described in standby.
Specifically, establishing routing relation in own files node before receiving the downloading request.Itself above-mentioned section
Point identification information may include but be not limited only to the IP address of own node or the port numbers of own node.If the data file
It is not present in the corresponding position in target query path described in the storage equipment of own node management, then to the unique of own node
Next-hop node forwards the downloading request.For example, determining the unique next of own node according to the routing relation preestablished
Own node IP address is added in the downloading request by hop node, and is added to unique next-hop node forwarding
The downloading of own node IP address is requested, so that unique next-hop node is according to preset path computing rule by the text
Part unique identification calculates the target query path of the data file and judges that the data file whether there is in own node
The corresponding position in target query path described in the storage equipment of management.
S506 is determined the unique upper hop node of itself by preset routing relation, to unique upper hop node and
The source node for sending the downloading request returns to downloading request refusal prompt.
Specifically, pre-establishing routing relation in own files node before receiving the downloading request.The road
Unique upper hop node of own node can be determined by relationship.Own node contains at least two port, and one for sending
Downloading request, one for receiving downloading request.Above-mentioned routing relation defines in own node for receiving downloading request
Port is only connected with unique upper hop node for sending the port of downloading request.The routing relation can pass through foundation
Socket long connection is maintained.
For example, in the routing relation pre-established in own node, the port numbers that own node receives downloading request are
29, the port numbers for the transmission downloading request that routing relation regulation is connected with the port numbers are 28, then possessing the section of No. 28 ports
Point is exactly unique upper hop node of own node.If it is determined that include itself port numbers 29 in downloading request, then to gathering around
There is the node of No. 28 ports and sends the source node return downloading request refusal prompt of the downloading request.
S507 reads the data file, and the data file is sent to the source node for sending the downloading request.
If the data file is present in the corresponding position in target query path described in the storage equipment of own node management
It sets, then reads the data file, and the data file is sent to the source node for sending the downloading request.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Fig. 6 is the structural schematic diagram of one of embodiment of the present invention distributed file management device, the device can include:
Receiving unit 602, number acquiring unit 603, query unit 604 and forward processing unit 605.
Receiving unit 602, the upload request of file for receiving data, the upload request carry node sequence number and
The corresponding relationship of document processing operation.
Specifically, including the file information of the data file, the text in the upload request that receiving unit 602 receives
Part information includes the corresponding relationship of the node sequence number and document processing operation.Above-mentioned upload request can be by source node
It directly transmits, is also possible to by unique upper hop node forwarding of own node.Above-mentioned source node be first issue and
And the node of the non-forwarded upload request, it may include uploading client.For example, the node sequence number and file process
The corresponding relationship of operation may is that the first jump forward process；Second jumps storage and forward process；Third jumps storage processing.
Number acquiring unit 603, for obtaining itself node sequence number according to the upload request.
Itself node sequence number is obtained according to the upload request, i.e., own node is the upload request from source node
Which hop node issued.Optionally, the node identification information total number in the upload request is calculated, by node identification information
Total number adds one to determine the own node serial number.Specifically, judging whether there is node identification letter in the upload request
Breath, if it is not, then node identification information number is zero, own node serial number is the first jump；If so, statistics node identification information
Total number, own node serial number are that node identification information total number adds one to determine.
Query unit 604 inquires itself for the corresponding relationship according to node sequence number and document processing operation
Node sequence numbers corresponding document processing operation, and the document processing operation includes storage processing, forward process or stores simultaneously
Forward process.
Specifically, the node sequence number and file process that are carried according to the received upload request of step receiving unit 602
The corresponding relationship of operation, the corresponding document processing operation of own node serial number that number of queries acquiring unit 603 obtains.Example
Such as, the corresponding relationship of the node sequence number and document processing operation that are carried in the upload request are as follows: first jumps forward process；
Second jumps storage and forward process；Third jumps storage processing.Own node serial number is one, i.e., own node is the upload
Issue from source node first is requested to jump, then the corresponding document processing operation of own node is exactly forward process.
Forward processing unit 605, for being forward process in the document processing operation or the document processing operation is
When storing simultaneously forward process, unique next-hop node of own node is determined according to preset routing relation, itself is saved
Point identification information is added in the upload request, is asked to unique next-hop node forwarding data file and the upload
It asks, is numbered so that unique next-hop node obtains itself node sequence according to the upload request, and according to own node
Serial number inquires corresponding document processing operation.
Specifically, above-mentioned own node identification information may include but be not limited only to the IP address or itself section of own node
The port numbers of point.Before receiving unit 602 receives the upload request, routing relation is pre-established in own files node,
The routing relation can determine unique next-hop node of the own node.
Optionally, which may also include storage processing unit 606, for being storage processing in the document processing operation
Or the document processing operation is that when storing simultaneously forward process, the storage that the data file is stored in own node management is set
In standby.
Specifically, according to preset path computing rule, by the data file for including in the upload request file only
One mark calculates the target store path of the data file, and stores the data file according to the target store path.
Optionally, above-mentioned target store path can be the time established according to data file, is classified and constitutes according to year, month, day.
Further alternative, which may also include request judging unit 601, in the number acquiring unit 603
Before obtaining itself node sequence number according to the upload request, whether judge in the upload request comprising itself section
Point identification information, if so, unique upper hop node of own node is determined by preset routing relation, to described unique upper one
Hop node returns to upload request refusal prompt；If it is not, then obtaining itself according to the upload request by the number acquiring unit
Node sequence number.
In the embodiment of the present invention, query unit inquires its section only for the upload request for not including own node identification information
Dot sequency is numbered corresponding document processing operation and is executed, and can own to avoid when the routing relation pre-established is ring-type
The endless loop that is formed when being forward process or storage and forward process of node.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Fig. 7 is that a kind of structural schematic diagram of distributed file management device, the device can wrap in another embodiment of the present invention
It includes: receiving unit 702, computing unit 703, file judging unit 704 and request retransmission unit 705.
Receiving unit 702, for receiving data the downloading request of file, the downloading request include the data file
File unique identification.
Specifically, including the file information of the data file, the text in the downloading request that receiving unit 702 receives
Part information includes the file unique identification of the file.Above-mentioned downloading request can be to be directly transmitted by source node, can also be with
It is to be forwarded by unique upper hop node of own node.Above-mentioned source node be first issue and and the non-forwarded downloading ask
The node asked may include downloading client.
Computing unit 703, for calculating the data by the file unique identification according to preset path computing rule
The target query path of file.
Specifically, preestablishing path in own node before receiving unit 702 receives the downloading request of data file
Computation rule.The path computing rule can be set by the storage that the file unique identification of data file obtains own node management
Standby target query path.Optionally, above-mentioned target query path can be according to data file establish time, according to year,
The moon, day classification are constituted.
File judging unit 704, for judging that the data file whether there is in the storage equipment of own node management
Described in the corresponding position in target query path.
Specifically, the data file obtained according to computing unit 703 is in the storage equipment of own node management
The corresponding position in target query path in the storage equipment of own node management is inquired in target query path, and described in judgement
Data file whether there is.
Retransmission unit 705 is requested, for judging that the data file is not present in own node pipe in file judging unit
When the corresponding position in target query path described in the storage equipment of reason, own node is determined according to preset routing relation
Unique next-hop node, by the addition of own node identification information in the downloading is requested, and to unique next-hop section
The downloading request of own node identification information is added to described in point forwarding, so that unique next-hop node is according to preset road
Diameter computation rule calculates the target query path of the data file by the file unique identification and judges the data file
With the presence or absence of the corresponding position in target query path described in the storage equipment in own node management.
Specifically, preestablishing routing in own node before receiving unit 702 receives the downloading request of data file
Relationship, the routing relation can determine unique next-hop node of own node.Above-mentioned own node mark includes but not only
It is limited to the IP address of own node or the port numbers of own node.
Optionally, which may also include request judging unit 701, be used in the computing unit according to preset path
Computation rule before the target query path for calculating the data file by the file unique identification, judges that the downloading is asked
It whether include the own node identification information in asking；If so, determining the unique upper hop of itself by preset routing relation
Node returns to downloading request refusal prompt to unique upper hop node and the source node for sending the downloading request；If it is not,
The mesh of the data file is then calculated by the file unique identification according to preset path computing rule by the computing unit
Mark query path.
Specifically, preestablishing routing in own node before receiving unit 702 receives the downloading request of data file
Relationship, the routing relation can determine unique next-hop node of own node.In the embodiment of the present invention, computing unit 703
It is requested only for the downloading for not including own node identification information, it is unique by the file according to preset path computing rule
Mark calculates the target query path of the data file, can to avoid when the routing relation pre-established is ring-type, and
Shape in the absence of data file described in the corresponding position in target query path of itself is equal in the storage equipment of all node administrations
At endless loop.Above-mentioned own node identification information may include but be not limited only to the IP address or own node of own node
Port numbers.
Further alternative, which may also include download unit 706, described in judging in the file judging unit
When data file is present in the corresponding position in target query path described in the storage equipment of own node management, the number is read
According to file to, and the data file is sent to the source node for sending the downloading request.
Specifically, file judging unit 704 judges that the data file is present in the storage equipment of own node management
When the corresponding position in the target query path, download unit 706 reads the data file, and the data file is transmitted
To the source node for sending the downloading request.
It is forward process in the corresponding document processing operation of own node serial number by implementing the embodiment of the present invention
When, the upload request of data file is transmitted to unique next-hop file node processing of own node, or in data file
When being not present in the corresponding position in target query path described in the storage equipment of own node management, by the downloading of data file
Request is transmitted to unique next-hop node processing of own node, so that balanced node load, reduces node overheads.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in a computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the storage medium can be magnetic
Dish, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access
Memory, RAM) etc..
The above disclosure is only the preferred embodiments of the present invention, cannot limit the right model of the present invention with this certainly
It encloses, therefore equivalent changes made in accordance with the claims of the present invention, is still within the scope of the present invention.
Claims (17)
1. a kind of distributed file management method characterized by comprising
The upload request of data file is received, the upload request carries node sequence number and the corresponding of document processing operation is closed
System；
Itself node sequence number is obtained according to the upload request；
According to the corresponding relationship of node sequence number and document processing operation, the corresponding text of inquiry own node serial number
Part processing operation, the document processing operation include storage processing, forward process or storage and forward process；
If the document processing operation be forward process, according to preset routing relation determine own node it is unique under
Own node identification information is added in the upload request by one hop node, forwards data to unique next-hop node
File and the upload request for being added to the own node identification information, so that unique next-hop node is according to receiving
Upload request obtains itself node sequence and numbers and execute the corresponding file process inquired according to own node serial number
Operation.
2. method according to claim 1, which is characterized in that further include:
If the document processing operation is storage processing, the data file is stored in the storage equipment of own node management
In.
3. method according to claim 1, which is characterized in that further include:
If the document processing operation is storage and forward process, the data file is stored in depositing for own node management
It stores up in equipment；
Unique next-hop node that own node is determined according to preset routing relation, own node identification information is added
To in the upload request, to unique next-hop node forwarding data file and described it is added to own node identification information
Upload request number and hold so that unique next-hop node obtains itself node sequence according to the upload request received
The corresponding document processing operation that row is inquired according to own node serial number.
4. method according to claim 1, which is characterized in that described to obtain itself node sequence volume according to the upload request
Number include:
Calculate the node identification information total number in the upload request；
The own node serial number adds one to determine by node identification information total number.
5. according to Claims 2 or 3 the method, which is characterized in that the upload request further includes the text of the data file
Part unique identification；
Include: in the storage equipment that the data file is stored in own node management
According to preset path computing rule, the target store path of the data file is calculated by the file unique identification；
The data file is stored according to the target store path.
6. method according to claim 1, which is characterized in that described to obtain itself node sequence volume according to the upload request
Before number, the method also includes:
Judge whether the upload request includes the own node identification information；If so, being determined by preset routing relation
Unique upper hop node of own node returns to upload request refusal prompt to unique upper hop node；If it is not, then triggering
Described the step of itself node sequence number is obtained according to the upload request.
7. a kind of distributed file management method characterized by comprising
The downloading request of data file is received, the downloading request includes the file unique identification of the data file；
According to preset path computing rule, the target query path of the data file is calculated by the file unique identification；
Judge that the data file is corresponding with the presence or absence of the target query path described in the storage equipment of own node management
Position；
If it is not, then determining unique next-hop node of own node according to preset routing relation, own node is identified
Information is added in the downloading request, and is added to the own node identification information to unique next-hop node forwarding
Downloading request so that unique next-hop node is calculated according to preset path computing rule by the file unique identification
The target query path of the data file simultaneously judges that the data file whether there is in the storage equipment of own node management
Described in the corresponding position in target query path.
8. method according to claim 7, which is characterized in that it is described according to preset path computing rule, by the file
Before unique identification calculates the target query path of the data file, the method also includes:
Whether judge in the downloading request comprising the own node identification information；If so, true by preset routing relation
The unique upper hop node for determining own node is returned to unique upper hop node and the source node for sending the downloading request
Downloading request refusal prompt；If it is not, then triggering described according to preset path computing rule, calculated by the file unique identification
The step of target query path of the data file.
9. method according to claim 7, which is characterized in that further include:
If judging, the data file is present in the corresponding position in target query path described in the storage equipment of own node management
It sets, then reads the data file, the data file is sent to the source node for sending the downloading request.
10. a kind of distributed file management device characterized by comprising
Receiving unit, the upload request of file, the upload request carry at node sequence number and file for receiving data
Manage the corresponding relationship of operation；
Number acquiring unit, for obtaining itself node sequence number according to the upload request；
Query unit, for the corresponding relationship according to node sequence number and document processing operation, inquiry own node is suitable
Sequence numbers corresponding document processing operation, and the document processing operation includes at storage processing, forward process or storage and forwarding
Reason；
Forward processing unit, for the document processing operation be forward process or the document processing operation be store and turn
When hair processing, unique next-hop node of own node is determined according to preset routing relation, and own node is identified and is believed
Breath is added in the upload request, to unique next-hop node forwarding data file and the upload request, so that institute
It states unique next-hop node and itself node sequence number is obtained according to the upload request, and looked into according to own node serial number
Ask corresponding document processing operation.
11. device according to claim 10, which is characterized in that further include storage processing unit, at the file
When reason operation is storage processing or the document processing operation is storage and forward process, the data file is stored in itself
In the storage equipment of node administration.
12. device according to claim 10, which is characterized in that the number acquiring unit is used for:
The node identification information total number in the upload request is calculated, and is added described in a determination as node identification information total number
Own node serial number.
13. device according to claim 11, which is characterized in that the upload request further includes the file of the data file
Unique identification；
The storage processing unit, for calculating the number by the file unique identification according to preset path computing rule
The data file is stored according to the target store path of file, and according to the target store path.
14. device according to claim 10, which is characterized in that further include:
Judging unit is requested, numbers it for obtaining itself node sequence according to the upload request in the number acquiring unit
Before, judge whether comprising the own node identification information in the upload request, if so, being determined by preset routing relation
Unique upper hop node of own node returns to upload request refusal prompt to unique upper hop node；If it is not, then by institute
It states number acquiring unit and itself node sequence number is obtained according to the upload request.
15. a kind of distributed file management device characterized by comprising
Receiving unit, the downloading request of file for receiving data, file of the downloading request comprising the data file is only
One mark；
Computing unit, for calculating the data file by the file unique identification according to preset path computing rule
Target query path；
Retransmission unit is requested, for judging that the data file is not present in the storage of own node management in file judging unit
When the corresponding position of target query path described in equipment, according to preset routing relation determine own node it is unique under
One hop node by the addition of own node identification information in downloading request, and adds to unique next-hop node forwarding
The downloading of the own node identification information has been added to request, so that unique next-hop node is advised according to preset path computing
The target query path of the data file is then calculated by the file unique identification and judges that the data file whether there is
The corresponding position in target query path described in the storage equipment of own node management.
16. device according to claim 15, which is characterized in that further include:
Judging unit is requested, for regular according to preset path computing in the computing unit, by the file unique identification
Before the target query path for calculating the data file, judge whether identify comprising the own node in the downloading request
Information；If so, unique upper hop node of own node is determined by preset routing relation, to unique upper hop node
Downloading request refusal prompt is returned with the source node for sending the downloading request；If it is not, then by the computing unit according to default
Path computing rule, the target query path of the data file is calculated by the file unique identification.
17. device according to claim 15, which is characterized in that further include download unit:
For judging that the data file is present in described in the storage equipment of own node management in the file judging unit
When the corresponding position in target query path, the data file is read, and the data file is sent to and sends the downloading
The source node of request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410594431.5A CN104408063B (en) | 2014-10-29 | 2014-10-29 | A kind of distributed file management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410594431.5A CN104408063B (en) | 2014-10-29 | 2014-10-29 | A kind of distributed file management method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104408063A CN104408063A (en) | 2015-03-11 |
| CN104408063B true CN104408063B (en) | 2019-04-02 |
Family
ID=52645695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410594431.5A Active CN104408063B (en) | 2014-10-29 | 2014-10-29 | A kind of distributed file management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104408063B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111327524B (en) * | 2018-12-14 | 2022-06-07 | 中国电信股份有限公司 | Flow forwarding method and system, SDN controller and computer readable storage medium |
| CN115170095B (en) * | 2022-09-07 | 2022-11-29 | 浪潮通信信息系统有限公司 | Order processing method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102571968A (en) * | 2012-01-17 | 2012-07-11 | 华中科技大学 | Directional data forwarding method in cloud backup system |
| CN103283189A (en) * | 2010-12-27 | 2013-09-04 | 日本电气株式会社 | Communication system and communication method |
| CN103535011A (en) * | 2011-12-14 | 2014-01-22 | 华为技术有限公司 | Routing method, device, and system in content delivery network (CDN) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8504597B2 (en) * | 2005-09-09 | 2013-08-06 | William M. Pitts | Distributed file system consistency mechanism extension for enabling internet video broadcasting |
| US8238237B2 (en) * | 2007-06-18 | 2012-08-07 | Sony Computer Entertainment Inc. | Load balancing distribution of data to multiple recipients on a peer-to-peer network |
-
2014
- 2014-10-29 CN CN201410594431.5A patent/CN104408063B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103283189A (en) * | 2010-12-27 | 2013-09-04 | 日本电气株式会社 | Communication system and communication method |
| CN103535011A (en) * | 2011-12-14 | 2014-01-22 | 华为技术有限公司 | Routing method, device, and system in content delivery network (CDN) |
| CN102571968A (en) * | 2012-01-17 | 2012-07-11 | 华中科技大学 | Directional data forwarding method in cloud backup system |
Non-Patent Citations (2)
| Title |
|---|
| Multi-dimensional Index on Hadoop Distributed File System;Haojun Liao,Jizhong Han,Jinyun Fang;《2010 Fifth IEEE International Conference on Networking, Architecture, and Storage》;20100715;全文 |
| 基于对象存储的分布式小文件系统的研究与设计;张宇;《中国优秀硕士学位论文全文数据库信息科技辑》;20140115;全文 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104408063A (en) | 2015-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2020113283A5 (en) | ||
| CN111147388B (en) | Estimating bandwidth in a network | |
| CN100413290C (en) | Method for setting up notification function for route selection according to border gateway protocol | |
| US8250171B2 (en) | Content delivery apparatus, content delivery method, and content delivery program | |
| CN101854299A (en) | Dynamic load balancing method of release/subscription system | |
| CN109491797A (en) | Date storage method, system and computer readable storage medium | |
| CN104660507B (en) | The control method and device of forwarding data flow routing | |
| CN106453122B (en) | A kind of choosing method and device of Stream Data Transmission node | |
| CN104539681A (en) | Distributed GIS accelerating system and GIS service processing method | |
| CN103259809A (en) | Load balancer, load balancing method and stratified data center system | |
| CN109787868A (en) | A kind of method, system and server for choosing routed path | |
| CN109117275A (en) | Account checking method, device, computer equipment and storage medium based on data fragmentation | |
| CN102006234A (en) | Method and device for optimizing P2P (Peer-to-Peer) network flow | |
| KR102441087B1 (en) | Method and Apparatus for Proof of Delivery in a Trustless Network | |
| CN105656964B (en) | The implementation method and device of data-pushing | |
| CN105049262B (en) | Adjust the method and device of port flow load | |
| CN104408063B (en) | A kind of distributed file management method and device | |
| CN109246190A (en) | network addressing method, data editing method, device and storage medium | |
| CN103312607A (en) | Transmission path selection method and device | |
| CN104967677A (en) | File transmission method and apparatus based on NDN cache optimization | |
| CN110414979A (en) | Endorsement endorsement method, device, equipment and storage medium on line | |
| CN108989438A (en) | Implementation method, the device and system of data distribution network | |
| CN105847380A (en) | UDP acceleration method and system in content distribution network | |
| CN110266766A (en) | A kind of construction method, system and the terminal device of attack resistance distributed network node | |
| CN103731376A (en) | Method and system for selecting chain aggregation root ports on stacked equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhang Xiaobiao Inventor after: Zheng Bangdong Inventor after: Xu Li Inventor before: Zhang Xiaobiao Inventor before: Zheng Bangdong Inventor before: Xu Li |
|
| CB03 | Change of inventor or designer information |