CN100550848C - The method and system of transferring large number of data - Google Patents
The method and system of transferring large number of data Download PDFInfo
- Publication number
- CN100550848C CN100550848C CNB2006101564276A CN200610156427A CN100550848C CN 100550848 C CN100550848 C CN 100550848C CN B2006101564276 A CNB2006101564276 A CN B2006101564276A CN 200610156427 A CN200610156427 A CN 200610156427A CN 100550848 C CN100550848 C CN 100550848C
- Authority
- CN
- China
- Prior art keywords
- state
- node
- transmission
- data
- child node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Invention provides a kind of method of transferring large number of data, be applied in the banking business data processing, comprise: transmission course is divided into different states, action and the state carried out at this state at different conditions transform, and under the situation that the transmission interruption takes place, from the recovering state transmission course of interrupting.The present invention also provides a kind of system of transferring large number of data.Utilize the present invention, the state of interdependent node register node in data transmission procedure, can realize monitoring to transmission course, and if cause transmission course to be interrupted owing to make mistakes, can be in follow-up process when interrupting the transmission state of registration recover transmission.
Description
Technical field
The present invention relates to the transfer of data in the network, particularly a kind of method and system of transferring large number of data.
Background technology
The regular deal with data of inter-agency needs of the different stage of bank.The data type that needs to handle generally has two classes: real-time transaction data and non-real-time service data.Real-time transaction data refers generally to send to the transaction data of handling in the kernel business of bank system by real-time transaction system, and these type of data are very strict to time requirement, generally all requires can get access in some milliseconds; The non-real-time service data are meant similar bank return, account table or other to the undemanding data of time requirement, and these type of data are looser to time requirement, generally get access to get final product in several hours even several days.Sometimes transmission is mass data, and these class data generally are non real-times, needs the time of transmission long, needs several hours to several days.
In the prior art, the transmission course of mass data is fairly simple, transmit leg with DSR after, just begin to transmit to the recipient.In transmission course, may cause transmission course to be interrupted for various reasons, for example not response of recipient, transmit leg or recipient produce mistake and cause situation such as transmission ending.Under these situations, though may transmit a lot of contents, or even the data end of transmission, the affirmation that only needs to obtain the recipient can be finished the transmission course of whole data, but existing transmission method with transmission course as a simple transfer process, can not determine which state was data passed to when transmission was interrupted, also just can not from the transmission of interrupting, recover.
Summary of the invention
The method and system that the purpose of this invention is to provide a kind of transferring large number of data is to overcome can not determine which state that has been transferred in the prior art, the shortcoming that can not recover from the transmission of interrupting.
For solving the problems of the technologies described above, the method and system that the invention provides a kind of transferring large number of data is achieved in that
A kind of method of transferring large number of data is divided into different states with transmission course, and action and the state carried out at this state at different conditions transform, and when taking place to transmit under the situation of interrupting, from the recovering state transmission course of interrupting;
Wherein, when child node when father node sends data, described transmission course is divided into different states, carry out to transform at different conditions and comprise at the action of this state and state:
It is the task standby condition that child node is set, to the data of needs transmission pack, compression, cryptographic operation;
The ready back of child node task configuration state is ready for transmission, send application to father node and send file request, and configuration state is ftp transmission beginning after receiving the request-reply that returns;
Father node is received after the application of sending sends file request oneself state is made as the ftp transmission wait, waits for that child node begins the ftp transmission;
Child node is transmitted data to father node, and state is changed to the ftp end of transmission behind end of transmission;
Child node is confirmed to finish data to father node and is sent, and oneself state is made as task termination, and it is ready that the father node state is changed to transmission;
Father node receives that affirmation that child node is sent finishes that the task that will self be made as arrives state after data send, and after the data that arrive are decrypted, decompress, unpack state is made as task termination.
When father node when child node receives data, also comprised before father node sends data in child node:
Put the father node state for to child node request transformation task, send reception file task requests, and receiving that replying the back configuration state is the request transmission wait to child node;
Child node receives that configuration state is the solicited status inspection after the reception file task requests of sending, and echo reply message is to father node;
Child node checks whether father node is request transmission wait state, if it is the task standby condition that child node is put oneself state.
When father node sends data to child node, described transmission course is divided into different states, carry out to transform at different conditions and comprise at the action of this state and state:
Father node is packed, compresses, is encrypted the data of needs transmission, finish after these operations state is changed to transmit ready;
Father node notice child node is got file, state is changed to the ftp transmission wait after receiving replying of returning;
Child node is changed to the ready state inspection with state after receiving the notice of sending of getting file, confirms whether the father node state is the ftp transmission wait; If putting oneself state is ftp transmission beginning;
Father node transmits data to child node, and the child node state is changed to the ftp end of transmission behind the end of transmission, and the father node state is changed to task termination, and child node enters the ready state of transmission;
The child node task that is changed to arrives state, and to the data decryption that arrives, decompress, unpack, executing these operation back configuration states is task termination.
When child node uncle node receives data, before sending data to child node, father node also comprises:
The child node state is set for to child node request transformation task, sends file, and after receiving the replying of father node, state is made as the request transmission wait to the father node application;
Father node is received after the application of sending sends file request oneself state is made as the solicited status inspection check whether the child node state is the request transmission wait, if then put certainly as the task standby condition.
When intermediate node receives when father node sends from child node, described transmission course is divided into different states, action and the state conversion carried out at this state at different conditions comprise:
Child node sends data to intermediate node and sends the data procedures execution according to described child node to father node, and when intermediate node arrives the ready state of transmission, intermediate node sends data to father node, sends data procedures according to described child node to father node and carries out.
When intermediate node receives when child node sends from child node, described transmission course is divided into different states, action and the state conversion carried out at this state at different conditions comprise:
Child node sends data to intermediate node and sends the data procedures execution according to described child node to father node, and when intermediate node arrives the ready state of transmission, intermediate node sends data to child node, sends data to the child node process according to described father node and carries out.
When intermediate node uncle node receives when child node sends, described transmission course is divided into different states, action and the state conversion carried out at this state at different conditions comprise:
Father node sends data to intermediate node and sends the data procedures execution according to described father node to child node, and when intermediate node arrives the ready state of transmission, intermediate node sends data to child node, sends data procedures according to described father node to child node and carries out.
When the data of first node request are striding across on the 3rd node of Section Point, when Section Point need receive the forwarding of file request, described transmission course is divided into different states, carry out to transform at different conditions and comprise at the action of this state and state:
The first node configuration state is to child node request transformation task, to Section Point application transfer files, and is receiving that the back configuration state of replying of Section Point is the request transmission wait;
Section Point receives that the rearmounted oneself state of application transfer files request is the solicited status inspection, checks whether first node is the request transmission wait, if oneself state is changed to child node request transformation task;
Section Point is to the 3rd node application transfer files, and is the request transmission wait at the back configuration state of replying of receiving the 3rd node;
The 3rd node receives that the rearmounted oneself state of application transfer files request is the solicited status inspection, checks whether Section Point is the request transmission wait, if will self be changed to the task standby condition;
The 3rd node is packed, compresses, is encrypted the data that will transmit.
The mode that described encryption adopts the file key that the data file head of transmission is encrypted.
Described employing file key is encrypted the data file head of transmission and is realized by following mode:
The password of registering by file transfer obtains the file key, when system begins in task, adopt this document key to encrypt, and the file key is transferred to destination node as the attribute of this document by the command channel through the header information of the packet after packing, the compression;
After task arrived destination node, system's utilization was deciphered packet and reduction from the file key that the source node transmission comes.
Comprise the node that participates in transfer of data, wherein,
Described node comprises state table, and this node utilizes this state table that transmission course is divided into different states, and action and the state carried out at this state at different conditions transform;
Described node comprises monitoring module, is used to monitor the residing state of transmission course, and when transmission course takes place to interrupt the state that monitors is sent to the recovery module;
Described node comprises the recovery module, is used for according to the interrupt status received transmission course from this recovering state.
Described state table comprises following state:
The task standby condition, this state to the data that will transmit pack, compression, cryptographic operation;
Task arrives state, and this state is decrypted, decompress(ion), separates package operation the transformation task that arrives;
Transmit ready state, this state according to node in transmission course the position and the difference on opportunity of appearance be divided into:
This state appears at child node and this node is the transmit leg of data, the expression data have been finished preliminary treatment, wait for to the father node transmission, then send the request that begins to transmit to father node according to this state, and behind the request echo reply, system changes into ftp transmission initial state with state;
This state appears at child node and this node is the recipient of data, represents that then the data of this task are all obtained end from father node, and system arrives this state task that is changed in state;
This state appears at father node and this node is the transmit leg of data, the expression data have been finished preliminary treatment, wait for to the child node transmission, then send to its child node according to this state to begin the request of obtaining, and behind the request echo reply, system is changed to ftp transmission wait state with state;
This state appears at father node and this node is the recipient of data, represents that then the data of this task are all obtained end from child node, and system arrives this state task that is changed in state;
Ready state is checked, this state appears at child node, when father node transmits data to child node, or child node is when father node obtains data, child node is to the father node inspection, confirm whether its state has become the ftp transmission wait, if then child node starts ftp request beginning transmission or receives data;
Ftp transmits initial state, and this state appears at child node, and system starts the ftp request according to this state, sends or received data file to its father node, and after the ftp request finished, system was changed to ftp end of transmission state with this state;
Ftp end of transmission state, this state appears at child node, is divided into according to the transmission of ftp or the difference of receive direction:
The ftp direction is for sending, and system sends the request that transmits ED to the other side's node, and it is ready that the other side's node state is changed to transmission;
The ftp direction is for receiving, and system sends the request that transmits ED to the other side's node, and the other side's node state is changed to task termination;
Task termination, this state are end-state, and the expression task finishes smoothly;
Described state table also comprises following state:
Done state checks that this state appears at father node, and system checks to child node whether this node task finishes according to this state;
To child node request transformation task, this state occurs in child node and obtains data or father node when child node is obtained data to father node, system sends the application of acquisition request data to the other side's node according to this state, after the other side's node is received application, configuration state is the solicited status inspection, simultaneously echo reply is to requesting node, and requesting node is changed to the request transmission wait with local state after replying receiving;
Solicited status checks that system checks that to the other side's node whether the other side's node state is the request transmission wait, after affirmation, is changed to the task standby condition with local state according to this state;
Ftp transmission wait, this state are passive wait states, after the state of the other side's node begins to become the ftp end of transmission from the ftp transmission, send request by the other side's node local state is changed to the ready or task termination of transmission;
The request transmission wait, this state is passive wait state, when the state of the other side's node become from the task standby condition transmission ready after, send request by the other side's node local state be changed to the ready state inspection;
Mission failure, this state are end-state, the expression mission failure.
By above technical scheme provided by the invention as seen, the present invention adds data transmission procedure to and finishes to be divided into different task statuses from task, and concrete processing and the consequential state of setting different task state correspondence change, like this, the state of interdependent node register node in data transmission procedure, can realize monitoring to transmission course, and if cause transmission course to be interrupted owing to make mistakes, can be in follow-up process when interrupting the transmission state of registration recover transmission.
Description of drawings
Fig. 1 is the inter-agency tree-like graph of a relation of banking system different stage;
Fig. 2 is basic data transmission relations figure, is divided into Fig. 2-1 and Fig. 2-2;
Fig. 3 is divided into Fig. 3-1, Fig. 3-2 and Fig. 3-3 for the data transmission relations figure of combination;
Fig. 4 is the flow chart of child node to father node transmission data;
Fig. 5 is child node receives data from father node a flow chart;
Fig. 6 is the flow chart of father node to child node transmission data;
Fig. 7 is father node receives data from child node a flow chart;
Fig. 8 receives the flow chart of transmitting to father node from child node;
Fig. 9 receives the flow chart of transmitting to child node from child node;
Figure 10 receives the flow chart of transmitting to child node from father node;
Figure 11 is the flow chart of the forwarding of reception file request.
Embodiment
Core of the present invention provides a kind of method and system of transferring large number of data, transmission course is divided into different states,, finishes whole transmission course in the action of each state execution at this state, and under the situation that the transmission interruption takes place, from the recovering state transmission course of interrupting.
In order to make those skilled in the art person understand the present invention program better, the present invention is described in further detail below in conjunction with drawings and embodiments.
The mechanism that one skilled in the art will appreciate that bank can be divided into different ranks, for example head office, one-level branch, secondary branch and site etc., and banking system need be in the inter-agency transfer of data of carrying out of different stage.Relation between these banking institution can be represented that Fig. 1 shows the inter-agency tree-like relation of banking system different stage by tree structure.In the node of this tree-like relation, any one node can carry out transfer of data with father node or child node, also can carry out transfer of data, and can the rest may be inferred obtain any two data between nodes transmission and can realize by the combination of these situations with other child node by father node and this father node.
In these transmission relation, as father node be generally server (Server), as child node be generally client (Client), data are in the difference of server and client side's transmission direction, node is different to the processing of data accordingly.Therefore, basic data transmission relations can be divided into following two kinds: child node transmits data to father node, and father node transmits data to child node.Fig. 2 shows basic data transmission relations, and the node table of shade shows the node at data source place.For child node transmits data to father node, send the difference that reception concerns according to node among Fig. 2-1, can be divided into again: a1 sends (put) to father node; B1 receives (get) from child node.Similarly, for father node transmits data to child node, send the difference that reception concerns according to node among Fig. 2-2, can be divided into again: a2 sends to child node; B2 receives from father node.Bottom transmission mechanism of the present invention can be the proftpd technology, this technology comparative maturity and opening, and except basic data-transformation facility is provided, also provide data breakpoint transmission, flow control and self-defined user's mechanism, and, the transmission of this technical support mass data file, and can be not the data file not be done the transmission that realizes mass data under the situation of any change.
It is as follows that child node transmits data to the flow chart of data processing of father node:
(1) file place node (being child node) detects the data file that need send to father node.
(2) to father node registration put task, father node is done the inter-related task registration record.
(3) child node is submitted upload request as client to father node server, and file is sent to father node.
(4) both sides finish file transfer, write down task treatment state (the child node logger task is finished, and the father node log file arrives) separately respectively.
(4 ') if both sides do not finish file transfer and accidental interruption, and then node recording status separately by both party triggers breakpoint transmission or transmission again, until the file transfer success next time when finding this transformation task by child node.
It is as follows that father node transmits data to the flow chart of data processing of child node:
(1) file place node (being father node) detects the data file that need send to child node.
(2) to child node registration get task, child node is done the inter-related task registration record.
(3) child node is submitted download request as client to father node server, receives file.
(4) both sides finish file transfer, write down task treatment state (the father node logger task is finished, and the child node log file arrives) separately respectively.
(4 ') if both sides do not finish file transfer and accidental interruption, and then node recording status separately by both party triggers breakpoint transmission or transmission again by child node, until file transfer successfully next time when finding this transformation task.
Based on above-mentioned master data transfer process, also has the transfer of data flow process of combination, as shown in Figure 3.As intermediate node, receive from child node among Fig. 3-1, send to father node; Receive from child node among Fig. 3-2, send to child node; Receive from father node from child node among Fig. 3-3, send to child node.
The present invention adds data transmission procedure to end (success or failure) from task and is divided into different task statuses according to foregoing, and sets the concrete processing and the change of consequential state of different task state correspondence.Shown in the table specific as follows:
| Task status | Status Name | The concrete processing |
| PREPARE | The task standby condition | To the data that will transmit pack, compress, operation such as encryption |
| ARRIVE | Task arrives state | The transformation task that arrives is decrypted, decompress(ion), behaviour such as unpacks |
| Do | |||
| XFER_OK | Transmit ready | This state according to node in transmission course position (child node or father node) and the difference on opportunity (task initiation/termination) that occurs different tupes is arranged respectively: 1, this state appears at child node and this node is a data receiver, the expression data have been finished preliminary treatment, wait is transmitted to father node, then send the request that begins to transmit to its father node according to this state, and behind the request echo reply, just state is changed into the XFER_ST state; 2, this state appears at child node and this node is the recipient of data, represents that then the data of this task are all obtained end from father node, and system will make this state into the ARRIVE state; 3, this state appears at father node, and this node is the transmit leg of data, the expression data have been finished preliminary treatment, wait is transmitted to father node, then send the request that begins to obtain to its child node according to this state, and behind the request echo reply, just state is changed into the XFER_ED state; 4, this state appears at father node and this node is the recipient of data, with 2 | |
| XFER_CHK | Ready state is checked | This state only appears at child node, when father node during to child node transmission data (or child node is obtained data to father node), child node need be to the father node inspection, confirm whether its state has become XFER_ED, if confirm then child node just starts ftp request beginning put or get data, can guarantee the state consistency on both sides like this | |
| XFER_ST | Ftp transmits beginning | Child node only appears in this state, and system will start according to this state | |
| A ftp request, to its father node put or get data file, after the ftp request finished, system can make this state into the XFER_END state automatically | ||
| XFER_END | The ftp end of transmission | Child node only appears in this state, direction (put or get) difference according to ftp, its operation is also inequality: if 1 ftp direction is put, system will send a request that transmits ED to the other side's node, and the other side's node state is revised as XFER_OK; If 2 ftp directions are get, system will send a request that transmits ED to the other side's node, and the other side's node state is revised as FINISH |
| FIN_CHK | Done state is checked | This state only appears at father node, and system checks to child node whether this node task finishes according to this state |
| GET_REQ | To child node request transformation task | This state occurs in child node and obtains data or father node obtains under the data conditions to child node to father node, system will send the application of acquisition request data according to this state to the other side's node, the other side's node is after receiving application, this node state is revised as REQ_CHK, simultaneously echo reply is given requesting node, and requesting node is REQ_WAIT with local status modifier after obtaining replying |
| REQ_CHK | Solicited status is checked | System will send one to the other side's node according to this state and check whether the other side's node state is the request of REQ_WAIT, after affirmation, is PREPARE with local status modifier |
| XFER_ED | The ftp transmission wait | This state is a passive wait state, has only after the state of the other side's node becomes XFER_END from XFER_ST, can send request by the other side's node local state is revised as XFER_OK or FINISH from XFER_ED |
| REQ_WAIT | The request transmission wait | This state is a passive wait state, has only after the state of the other side's node becomes XFER_OK from PREPARE, and can send request by the other side's node is XFER_CHK with local status modifier |
| FINISH | Task termination | This state is an end-state, and the expression task finishes smoothly |
| IMPOSSIBLE | Mission failure | This state is an end-state, and the expression task is failed because of certain mistake that cannot repair |
The state of table 1. task and concrete the processing
Like this, be divided into different states with sending with the process of receiving node place transfer of data, in this process of transfer of data, the state of interdependent node register node, if cause transmission course to be interrupted owing to make mistakes, can be in follow-up process when interrupting the transmission state of registration recover transmission.
Same internodal state is converted into local state and transforms, and handles the triggering state by the data transmission procedure of local node and transforms, and mainly comprise: PREPARE is to XFER_OK, and ARRIVE is to FINISH.
The state current between different nodes according to task, conditions such as the destination node of task and transmission direction, the state that triggers the other side's node transforms.Concrete, transmit the control command request by sending to Object node, and according to request results task switching state.The transmission of transmission control command request and the reception of replying must be finished by the command channel, and content as shown in the table is supported to comprise in the command channel:
Table 2. transmission control command table
Each state is not fixing and certain order combines, but according to command execution of the dynamic selection of condition, such as: the XFER_OK state, can corresponding a plurality of orders, as 0XGETBEG, 0XPUTBEG, along with position (child node, the father node) difference or task direction (put or the get) difference of node, the order of selection also can be different.Below enumerating different embodiment describes.
Fig. 4 shows the process of child node to father node transmission data.The initiator of this task is child node A, and data source is initially on the child node A, and father node B is the recipient of data, also is the destination node of task.Whole transmission course is driven by child node, and father node only needs passive response (father node only the processing after the active process end of transmission and finish).
This process is specially:
PREPARE: child node to the data of needs transmission pack, compress, operation such as encryption;
XFER_OK: child node is sent application to father node and is sent file, promptly carries out 0XPUTBEG, and after receiving the request-reply that returns, and self state is changed to XFER_ST;
XFER_ED: father node is passive wait state at this moment;
XFER_ST:ftp transmits beginning, and child node is transmitted data to father node, and behind the ftp end of transmission, the change oneself state is XFER_END;
XFER_END: the child node ftp end of transmission, confirm that to father node having sent file finishes, promptly carry out 0XPUTEND, the father node state is changed to XFER_OK, the child node oneself state changes to FINISH;
XFER_OK: father node receives ED from child node, and oneself state is changed to ARRIVE;
ARRIVE: father node is to operations such as the data that arrive are decrypted, decompress, unpack, and oneself state is changed to FINISH;
FINISH: father node finishes transformation task smoothly.
As seen, this process one has 13 states.
Fig. 5 shows child node receives data from father node process.It is child node A that this task is initiated, and data source is on father node B, and A is the recipient of data, and this is a kind of comparatively complicated transformation task, and this mission mode kind is made of 11 kinds of task statuses altogether.The existing part that is driven by A of the whole process of task also has the part that is driven by B, and common coordination of both sides finished.In the whole process of task, guarantee the coordination on two side's states of task by twice inspection, prevent to cause the failure of task because the both sides state is inconsistent.
This process is specially:
GET_REQ: child node sends file to the father node application, promptly carries out 0XGETREQ, and father node receives that request back change state is REQ_CHK, and echo reply message gives child node, and child node changes to REQ_WAIT with state after obtaining replying;
REQ_CHK: father node sends to child node whether an inspection state is the request 0XREQCHK of REQ_WAIT, after affirmation, is PREPARE with local status modifier;
PREPARE: father node to the data of needs transmission pack, compress, operation such as encryption, and after finishing, state is changed to XFER_OK;
XFER_OK: father node notice child node is got file, promptly carries out 0XGETBEG, and behind echo reply, changes into the XFER_ED state with regard to state;
XFER_CHK: child node confirms that to the father node inspection its state has become XFER_ED, promptly carries out 0XGETCHK, and child node starts ftp request beginning get data, can guarantee the state consistency on both sides like this;
XFER_ST:ftp transmits beginning, and father node transmits data to child node, and behind the ftp end of transmission, child node change oneself state is XFER_END;
XFER_END: child node is finished from father node and is got file, confirms to father node, promptly carries out 0XPUTEND, and is FINISH with the father node status modifier;
XFER_OK: child node receives ED from father node, and self node is changed to ARRIVE;
ARRIVE: child node is to operations such as the data that arrive are decrypted, decompress, unpack, and oneself state is changed to FINISH;
FINISH: child node finishes transformation task smoothly.
Fig. 6 shows the process of father node to child node transmission data.This task initiator and data source all are father node B, and A is the recipient of data, are made of 8 kinds of task statuses altogether in this mission mode.The existing part that is driven by A of whole process also has the part that is driven by B, and common coordination of both sides finished.This process is the part of process shown in Figure 5, therefore omits the description to this process.
Fig. 7 shows father node receives data from child node process.It is father node B that this task is initiated, but data source is on child node A, and B is the recipient of data.This mission mode kind is made of 10 kinds of task statuses altogether.The existing part that is driven by A of the whole process of task also has the part that is driven by B, and common coordination of both sides finished.In the whole process of task, guarantee the coordination on two side's states of task by the several times inspection, prevent to cause the failure of task because the both sides state is inconsistent.
This process is specially:
GET_REQ: father node sends to child node and receives the file task requests, promptly carries out 0XGETREQ, and child node receives that request back change state is REQ_CHK, and echo reply message gives father node, and father node changes to REQ_WAIT with oneself state after obtaining replying;
REQ_CHK: child node sends one to father node and checks whether the other side's node state is the request of REQ_WAIT, and after affirmation, child node is PREPARE with local status modifier;
Process afterwards is identical with Fig. 4, repeats no more.
Transfer of data does not exist only between two nodes, also often is present in the transmission between cross-node.In fact the cross-node document transmission process can resolve into is the combination of several Basic Transmission Unit, but the forwarding between single and single transmission unit also has different processing modes according to concrete transmission direction or transmission combination different, and the cross-node transmission mainly exists following 3 kinds of transmission modes and a kind of cross-node request pattern:
Transmit to father node from the child node reception;
Transmit to child node from the child node reception;
Transmit to child node from the father node reception;
Receive the forwarding of file request.
Below introduce respectively.
Fig. 8 shows from child node and receives the process of transmitting to father node.As shown in the figure, intermediate node B is responsible for the route of task and judges, if the task object node is not this node, will send the request that begins to transmit to next node, if this node is a destination node, then directly finishes this task, and state is updated to FINISH.This process can be decomposed into two Fig. 4 processes, and child node A is equivalent to child node to Centroid B transmission data and sends data to father node, also is equivalent to child node to father node C transmission data after Centroid B receives and sends data to father node.
Fig. 9 shows from child node and receives the process of transmitting to child node.As shown in the figure, this process can be decomposed into Fig. 4 and Fig. 6.Task is initiated from child node A, sends data to Centroid B, is equivalent to child node and sends data to father node, and Centroid B receives the back and sends data to child node C, is equivalent to father node and sends data to child node.
Figure 10 shows from father node and receives the process of transmitting to child node.Similarly, this process can be decomposed into two Fig. 6.
Figure 11 shows the forwarding that receives file request.This situation is specially: the data file of node A request is placed on the node C, and therefore the transmission request just need be forwarded to node C by cross-node, is handled by node C then, is transferred to node A again.
This process is specially:
GET_REQ: node A promptly carries out 0XGETREQ to Node B application transfer files, and Node B receives that request back change state is REQ_CHK, and echo reply message gives node A, and node A changes to REQ_WAIT with state after obtaining replying;
REQ_CHK: Node B sends to node A whether an inspection state is the request 0XREQCHK of REQ_WAIT, after affirmation, is GET_REQ with local status modifier;
GET_REQ: Node B is promptly carried out 0XGETREQ to node C application transfer files, and node C receives that request back change state is REQ_CHK, and echo reply message gives Node B, and Node B changes to REQ_WAIT with state after obtaining replying;
REQ_CHK: node C sends to Node B whether an inspection state is the request 0XREQCHK of REQ_WAIT, after affirmation, is PREPARE with local status modifier;
PREPARE: node C to the data of needs transmission pack, compress, operation such as encryption.
By above each embodiment as seen, each state is not fixing and certain order combines, but select a command execution according to condition, for example: this state of XFER_OK can corresponding a plurality of order 0XGETBEG, 0XPUTBEG, along with position (child node, the father node) difference or task direction (put or the get) difference of node, the selection of system also can be different.When in the transmission unit of " child node sends to father node ", system will send the 0XPUTBEG order to father node when handling the XFER_OK state, and the notice father node its state such as changes into (XFER_ED) to be transmitted state; And " child node receives from father node " transmission unit, then just in time opposite, system will send the order of 0XGETBEG to child node, the notice child node by proftpd from the father node file that fetches data.
In addition, the entity of data transmission system transmission is a data file, and the data in the banking are extremely important data, must guarantee its safety in transmission course, so need encrypt data entity.
Three grades of traditional key code systems require must carry out the operation of key conversion under the situation of cross-node.As general message, change to encrypt and do not have what problem, if but encrypt to as if the situation of the file of big data quantity under must cause a large amount of system resource waste and transmission time to increase, unrealistic from realizing using three grades of key code systems of traditional mode.
Document transmission system of the present invention based on this, adopts the file secret key encryption from three grades of key code systems.Concrete, only to encrypt with file key-pair file head, this document key is that the password of registering by file transfer obtains through computing, and is kept at this locality.When system begins in task; the header information of the packet after will packing, compress with the file key is encrypted; then the file key is transferred to the destination node of task by the command channel as an attribute of this task; and; this document key is subjected to three grades of key code system protections in the communication process of transmission command passage; after task arrived destination node, system just used from the next file key of source node transmission packet is deciphered, and reduction.Thereby guaranteed the safe transmission of data file in enormous quantities with minimum resource, time loss.
Below introduce the system of transferring large number of data provided by the invention.
The system of transferring large number of data comprises the node that participates in transfer of data, wherein,
Transmission node comprises state table, and transmission node utilizes this state table that transmission course is divided into different states, and action and the state carried out at this state at different conditions transform; Described state table is as shown in table 1;
Transmission node comprises monitoring module, is used to monitor the residing state of transmission course, and when transmission course takes place to interrupt the state that monitors is sent to the recovery module;
Transmission node comprises the recovery module, is used for according to the interrupt status received transmission course from this recovering state.
By above embodiment as seen, the present invention will send with the process of receiving node place transfer of data and be divided into different states, in this process of transfer of data, the state of interdependent node register node, can realize monitoring to data transmission procedure, if cause transmission course to be interrupted owing to make mistakes, can be in follow-up process when interrupting the transmission state of registration recover transmission.
Though described the present invention by embodiment, those of ordinary skills know, the present invention has many distortion and variation and do not break away from spirit of the present invention, wish that appended claim comprises these distortion and variation and do not break away from spirit of the present invention.
Claims (13)
1, a kind of method of transferring large number of data is characterized in that, transmission course is divided into different states, and action and the state carried out at this state at different conditions transform, and when taking place to transmit under the situation of interrupting, from the recovering state transmission course of interrupting;
Wherein, when child node when father node sends data, described transmission course is divided into different states, carry out to transform at different conditions and comprise at the action of this state and state:
It is the task standby condition that child node is set, to the data of needs transmission pack, compression, cryptographic operation;
The ready back of child node task configuration state is ready for transmission, send application to father node and send file request, and configuration state is ftp transmission beginning after receiving the request-reply that returns;
Father node is received after the application of sending sends file request oneself state is made as the ftp transmission wait, waits for that child node begins the ftp transmission;
Child node is transmitted data to father node, and state is changed to the ftp end of transmission behind end of transmission;
Child node is confirmed to finish data to father node and is sent, and oneself state is made as task termination, and it is ready that the father node state is changed to transmission;
Father node receives that affirmation that child node is sent finishes that the task that will self be made as arrives state after data send, and after the data that arrive are decrypted, decompress, unpack state is made as task termination.
2, the method for claim 1 is characterized in that, when father node when child node receives data, also comprised before father node sends data in child node:
Put the father node state for to child node request transformation task, send reception file task requests, and receiving that replying the back configuration state is the request transmission wait to child node;
Child node receives that configuration state is the solicited status inspection after the reception file task requests of sending, and echo reply message is to father node;
Child node checks whether father node is request transmission wait state, if it is the task standby condition that child node is put oneself state.
3, the method for claim 1 is characterized in that, when father node sends data to child node, described transmission course is divided into different states, carries out to transform at the action of this state and state at different conditions to comprise:
Father node is packed, compresses, is encrypted the data of needs transmission, finish after these operations state is changed to transmit ready;
Father node notice child node is got file, state is changed to the ftp transmission wait after receiving replying of returning;
Child node is changed to the ready state inspection with state after receiving the notice of sending of getting file, confirms whether the father node state is the ftp transmission wait; If putting oneself state is ftp transmission beginning;
Father node transmits data to child node, and the child node state is changed to the ftp end of transmission behind the end of transmission, and the father node state is changed to task termination, and child node enters the ready state of transmission;
The child node task that is changed to arrives state, and to the data decryption that arrives, decompress, unpack, executing these operation back configuration states is task termination.
4, method as claimed in claim 3 is characterized in that, when child node uncle node receives data, also comprises before father node sends data to child node:
The child node state is set for to child node request transformation task, sends file, and after receiving the replying of father node, state is made as the request transmission wait to the father node application;
Father node is received after the application of sending sends file request oneself state is made as the solicited status inspection check whether the child node state is the request transmission wait, if then put certainly as the task standby condition.
5, the method for claim 1 is characterized in that, when intermediate node receives when father node sends from child node, described transmission course is divided into different states, and action and the state conversion carried out at this state at different conditions comprise:
Child node sends data to intermediate node and sends the data procedures execution according to described child node to father node, and when intermediate node arrives the ready state of transmission, intermediate node sends data to father node, sends data procedures according to described child node to father node and carries out.
6, method as claimed in claim 3 is characterized in that, when intermediate node receives when child node sends from child node, described transmission course is divided into different states, and action and the state conversion carried out at this state at different conditions comprise:
Child node sends data to intermediate node and sends the data procedures execution according to described child node to father node, and when intermediate node arrives the ready state of transmission, intermediate node sends data to child node, sends data to the child node process according to described father node and carries out.
7, method as claimed in claim 3 is characterized in that, when intermediate node uncle node receives when child node sends, described transmission course is divided into different states, and action and the state conversion carried out at this state at different conditions comprise:
Father node sends data to intermediate node and sends the data procedures execution according to described father node to child node, and when intermediate node arrives the ready state of transmission, intermediate node sends data to child node, sends data procedures according to described father node to child node and carries out.
8, the method for claim 1, it is characterized in that, when the data of first node request are striding across on the 3rd node of Section Point, when Section Point need receive the forwarding of file request, described transmission course is divided into different states, carries out to transform at different conditions and comprise at the action of this state and state:
The first node configuration state is to child node request transformation task, to Section Point application transfer files, and is receiving that the back configuration state of replying of Section Point is the request transmission wait;
Section Point receives that the rearmounted oneself state of application transfer files request is the solicited status inspection, checks whether first node is the request transmission wait, if oneself state is changed to child node request transformation task;
Section Point is to the 3rd node application transfer files, and is the request transmission wait at the back configuration state of replying of receiving the 3rd node;
The 3rd node receives that the rearmounted oneself state of application transfer files request is the solicited status inspection, checks whether Section Point is the request transmission wait, if will self be changed to the task standby condition;
The 3rd node is packed, compresses, is encrypted the data that will transmit.
9, as each described method in the claim 1 to 8, it is characterized in that the mode that described encryption adopts the file key that the data file head of transmission is encrypted.
10, method as claimed in claim 9 is characterized in that, described employing file key is encrypted the data file head of transmission and realized by following mode:
The password of registering by file transfer obtains the file key, when system begins in task, adopt this document key to encrypt, and the file key is transferred to destination node as the attribute of this document by the command channel through the header information of the packet after packing, the compression;
After task arrived destination node, system's utilization was deciphered packet and reduction from the file key that the source node transmission comes.
11, a kind of system of transferring large number of data is characterized in that, comprises the node that participates in transfer of data, wherein,
Described node comprises state table, and this node utilizes this state table that transmission course is divided into different states, and action and the state carried out at this state at different conditions transform;
Described node comprises monitoring module, is used to monitor the residing state of transmission course, and when transmission course takes place to interrupt the state that monitors is sent to the recovery module;
Described node comprises the recovery module, is used for according to the interrupt status received transmission course from this recovering state.
12, system as claimed in claim 11 is characterized in that, described state table comprises following state:
The task standby condition, this state to the data that will transmit pack, compression, cryptographic operation;
Task arrives state, and this state is decrypted, decompress(ion), separates package operation the transformation task that arrives;
Transmit ready state, this state according to node in transmission course the position and the difference on opportunity of appearance be divided into:
This state appears at child node and this node is the transmit leg of data, the expression data have been finished preliminary treatment, wait for to the father node transmission, then send the request that begins to transmit to father node according to this state, and behind the request echo reply, system changes into ftp transmission initial state with state;
This state appears at child node and this node is the recipient of data, represents that then the data of this task are all obtained end from father node, and system arrives this state task that is changed in state;
This state appears at father node and this node is the transmit leg of data, the expression data have been finished preliminary treatment, wait for to the child node transmission, then send to its child node according to this state to begin the request of obtaining, and behind the request echo reply, system is changed to ftp transmission wait state with state;
This state appears at father node and this node is the recipient of data, represents that then the data of this task are all obtained end from child node, and system arrives this state task that is changed in state;
Ready state is checked, this state appears at child node, when father node transmits data to child node, or child node is when father node obtains data, child node is to the father node inspection, confirm whether its state has become the ftp transmission wait, if then child node starts ftp request beginning transmission or receives data;
Ftp transmits initial state, and this state appears at child node, and system starts the ftp request according to this state, sends or received data file to its father node, and after the ftp request finished, system was changed to ftp end of transmission state with this state;
Ftp end of transmission state, this state appears at child node, is divided into according to the transmission of ftp or the difference of receive direction:
The ftp direction is for sending, and system sends the request that transmits ED to the other side's node, and it is ready that the other side's node state is changed to transmission;
The ftp direction is for receiving, and system sends the request that transmits ED to the other side's node, and the other side's node state is changed to task termination;
Task termination, this state are end-state, and the expression task finishes smoothly;
13, system as claimed in claim 11 is characterized in that, described state table also comprises following state:
Done state checks that this state appears at father node, and system checks to child node whether this node task finishes according to this state;
To child node request transformation task, this state occurs in child node and obtains data or father node when child node is obtained data to father node, system sends the application of acquisition request data to the other side's node according to this state, after the other side's node is received application, configuration state is the solicited status inspection, simultaneously echo reply is to requesting node, and requesting node is changed to the request transmission wait with local state after replying receiving;
Solicited status checks that system checks that to the other side's node whether the other side's node state is the request transmission wait, after affirmation, is changed to the task standby condition with local state according to this state;
Ftp transmission wait, this state are passive wait states, after the state of the other side's node begins to become the ftp end of transmission from the ftp transmission, send request by the other side's node local state is changed to the ready or task termination of transmission;
The request transmission wait, this state is passive wait state, when the state of the other side's node become from the task standby condition transmission ready after, send request by the other side's node local state be changed to the ready state inspection;
Mission failure, this state are end-state, the expression mission failure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006101564276A CN100550848C (en) | 2006-12-31 | 2006-12-31 | The method and system of transferring large number of data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006101564276A CN100550848C (en) | 2006-12-31 | 2006-12-31 | The method and system of transferring large number of data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101009641A CN101009641A (en) | 2007-08-01 |
| CN100550848C true CN100550848C (en) | 2009-10-14 |
Family
ID=38697784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006101564276A Active CN100550848C (en) | 2006-12-31 | 2006-12-31 | The method and system of transferring large number of data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100550848C (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101252506B (en) * | 2007-12-29 | 2012-07-04 | 中国建设银行股份有限公司 | Data transmission system |
| CN102036102A (en) * | 2009-09-25 | 2011-04-27 | 腾讯科技（深圳）有限公司 | System and method for transcoding multimedia and system and method for playing multimedia |
| CN102169439B (en) * | 2010-02-26 | 2014-05-07 | 英业达股份有限公司 | Data transmission system |
| CN104184753B (en) | 2013-05-20 | 2018-04-27 | 腾讯科技（深圳）有限公司 | A kind of document transmission method and device |
| CN109388618B (en) * | 2018-10-15 | 2021-02-12 | 密卡思(深圳)电讯有限公司 | Method and device for compressing, decompressing, encrypting and decrypting file of embedded system |
| CN110602359B (en) * | 2019-09-02 | 2022-01-18 | Oppo广东移动通信有限公司 | Image processing method, image processor, photographing device and electronic equipment |
| CN111245929B (en) * | 2020-01-09 | 2022-11-01 | 卡斯柯信号有限公司 | Method for realizing breakpoint continuous transmission of COCC non-real-time channel data |
| CN113660393A (en) * | 2021-07-08 | 2021-11-16 | 上海途悠信息科技有限公司 | Service window double-recording system |
-
2006
- 2006-12-31 CN CNB2006101564276A patent/CN100550848C/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN101009641A (en) | 2007-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100550848C (en) | The method and system of transferring large number of data | |
| CN109493020B (en) | Block chain based secure transaction method and device | |
| CN106503098B (en) | Block chain cloud service framework system built in Paas service layer | |
| US6128647A (en) | Self configuring peer to peer inter process messaging system | |
| CN110881063B (en) | Storage method, device, equipment and medium of private data | |
| CN109284627B (en) | Credit credit method and device based on block chain intelligent contract | |
| WO2022062976A1 (en) | Cross-blockchain system for executing transactions, and cross-blockchain transaction method and device | |
| CN110009494B (en) | Method and device for monitoring transaction content in block chain | |
| CN109428899A (en) | File security transfer management method and system | |
| CN102710759A (en) | Web server, business logging method and system | |
| CN103562882A (en) | System and method for providing a messaging application program interface | |
| WO2016056473A1 (en) | Secret calculation system and relay device, and method, program, and storage medium thereof | |
| US10728045B2 (en) | Authentication device, authentication system, authentication method, and program | |
| CN102404326A (en) | Method, system and device for validating safety of messages | |
| CN101175089A (en) | Data transmission method between server and client terminal based on HTTP protocol and .NET structure | |
| CN112532387B (en) | Key service operation system and method thereof | |
| CN111448812A (en) | Information transmission method, storage medium, information transmission system, and unmanned aerial vehicle | |
| CN103092932A (en) | Distributed document transcoding system | |
| CN114091059A (en) | Data security processing method, device, terminal, medium and system | |
| CN111125734B (en) | Data processing method and system | |
| US7908333B2 (en) | Self configuring peer to peer inter process messaging system | |
| Daodu et al. | A data encryption standard (DES) based web services security architecture | |
| KR101997996B1 (en) | Security communication method using key management server in software defined network controller and apparatus for perfoming the same | |
| CN101827108B (en) | Method for describing and acquiring right object of digital work in digital right management | |
| CN116566698B (en) | Secret-related data exchange method and system based on multistage cross-network isolation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |