CN104410675A - Data transmission method, data system and related devices - Google Patents

Data transmission method, data system and related devices Download PDF

Info

Publication number
CN104410675A
CN104410675A CN201410645843.7A CN201410645843A CN104410675A CN 104410675 A CN104410675 A CN 104410675A CN 201410645843 A CN201410645843 A CN 201410645843A CN 104410675 A CN104410675 A CN 104410675A
Authority
CN
China
Prior art keywords
data
transmitted
cluster
server
proxy server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201410645843.7A
Other languages
Chinese (zh)
Inventor
赵剑锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Qihoo Technology Co Ltd
Qizhi Software Beijing Co Ltd
Original Assignee
Beijing Qihoo Technology Co Ltd
Qizhi Software Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Qihoo Technology Co Ltd, Qizhi Software Beijing Co Ltd filed Critical Beijing Qihoo Technology Co Ltd
Priority to CN201410645843.7A priority Critical patent/CN104410675A/en
Publication of CN104410675A publication Critical patent/CN104410675A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services

Abstract

The invention provides a data transmission method, a data system and related devices. The data system comprises a source server, a plurality of target servers respectively positioned at different clusters and proxy servers respectively provided for each cluster, wherein the source server is suitable for transmitting data to be transmitted to the proxy server of each cluster; the proxy server of each cluster is suitable for receiving the data to be transmitted and forwarding the data to be transmitted to all target servers in the cluster for any one cluster in each cluster; each target server in the cluster is suitable for receiving the data to be transmitted, which is issued by the proxy server of each cluster. According to the embodiment of the invention, the resource utilization rate of the source server can be improved.

Description

Data transmission method, data system and relevant apparatus
Technical field
The present invention relates to field of data transmission, particularly relate to a kind of data transmission method, data system and relevant apparatus.
Background technology
Along with the fast development of network technology, at the source server of large-scale website, there are various types of data such as much newly-generated web page files, code file and configuration file all the time.For ensureing that Internet user can access in time or use, source server needs with speed faster, newly-generated Data dissemination to be positioned on the destination server of different location to tens of, even hundreds.
In prior art, the transfer of data of generation to the mode of each destination server is had two kinds by source server: serial process mode and parallel processing manner:
Serial process mode, namely often a collection of data to be transmitted is transferred on all destination servers by source server serially, when there is many batches of data to be transmitted, repeats above process.Although this transmission means good stability, transmission speed is slow, causes data to be transmitted to be transferred on destination server in time efficiently.
Parallel processing manner, namely often a collection of data to be transmitted is transferred on all destination servers by source server concurrently.Although this transmission means transmission speed is faster than the first transmission means, at least there are following two shortcomings:
(1) source server needs simultaneously to set up network with all destination servers and is connected, therefore network connection and process/thread quantity wayward;
(2) multiple destination servers of each data transfer operation are in different regions usually, and potential range source server distance, the therefore poor stability of transmitting procedure.
Summary of the invention
In view of the above problems, embodiments of the invention propose a kind of data transmission method, data system and relevant apparatus, to overcome the problems referred to above or to solve the problem at least in part.
According to one aspect of the present invention, provide a kind of source server, comprise: transport module, be suitable for transmitting the proxy server that the most each cluster of described data to be transmitted is arranged respectively, wherein, the proxy server of each cluster is used for each destination server described data to be transmitted being forwarded to its place cluster.
Alternatively, described source server also comprises: distribution module, is suitable for according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, wherein, N be not less than 1 positive integer; Transport module, is suitable for utilizing the described N number of queue distributed to transmit described data to be transmitted to the proxy server of described each cluster.
Alternatively, described source server also comprises: monitoring modular, is suitable in transmitting procedure, and whether the parameter of monitoring current data to be transmitted changes; And, when the parameter monitoring current data to be transmitted changes, trigger described distribution module; Described distribution module, is also suitable for the triggering receiving described monitoring modular, and redistributes M for transmitting the queue of described data to be transmitted according to the parameter after change; Described transport module, is also suitable for utilizing described M the queue redistributed to proceed data transfer operation.
Alternatively, described distribution module is also suitable for: the configuration information determining this source server according to the parameter of described data to be transmitted, and wherein, described configuration information comprises number of processes information, number of network connections information; And, distribute N number of queue for transmitting described data to be transmitted according to described configuration information.
Alternatively, described transport module comprises further: determining unit, is suitable for the priority-level determining described data to be transmitted according to the parameter of described data to be transmitted; Sequencing unit, is suitable for utilizing the priority-level of described data to be transmitted to sort to the described data to be transmitted of buffer memory in described N number of queue; And transmission unit, be suitable for carrying out described data transfer operation according to ranking results.
Alternatively, described sequencing unit is also suitable for: before data to be transmitted higher for priority being placed in the lower data to be transmitted of priority, with the data to be transmitted that priority described in prioritised transmission is higher.
Alternatively, described source server also comprises: receiver module, the transmission result that the proxy server being suitable for receiving described each cluster returns.
Alternatively, described source server also comprises: processing module, is suitable for carrying out transmission state analysis according to described transmission result, obtains analysis result, wherein, described transmission state comprise following one of at least: transmission speed, transmission success rate, transmission packet loss; Described distribution module, is also suitable for utilizing described analysis result distribution queue number.
Alternatively, described transport module is also suitable for: after described receiver module receives described transmission result, if the transmission result that arbitrary proxy server returns is transmission operation failure, again transmits described data to be transmitted to this proxy server.
According to another aspect of the present invention, additionally provide a kind of proxy server, be arranged at a cluster, wherein, also there is multiple destination server in described cluster, comprising: receiver module, be suitable for receiving the data to be transmitted from source server; Sending module, is suitable for each destination server sent in described data to be transmitted to this cluster.
Alternatively, described sending module is also suitable for: return transmission result to described source server, for described source server according to described transmission result adjustment subsequent transmission operations.
According to another aspect of the invention, additionally provide a kind of data system, comprise above-mentioned source server and at least one cluster, wherein, each cluster comprises multiple destination server, also comprises above-mentioned proxy server, wherein, described source server, is suitable for transmitting the proxy server of data to be transmitted to described each cluster; For any one cluster in described each cluster, the proxy server in this cluster, is suitable for receiving described data to be transmitted, and forwards each destination server in described data to be transmitted to this cluster; And each destination server in this cluster, is suitable for receiving the described data to be transmitted that this cluster proxy server issues.
Alternatively, in described each cluster, the proxy server of arbitrary cluster is also suitable for being arranged by following steps: in this cluster, select arbitrary destination server as proxy server; Or, for this cluster arranges separately a proxy server.
According to another aspect of the present invention, additionally provide a kind of data transmission method being applied to data system, described data system comprises source server, the multiple destination servers laid respectively at least one cluster, and described method comprises: for each cluster arranges proxy server; Data to be transmitted is transferred to corresponding proxy server; And utilize the proxy server of described correspondence described data to be transmitted to be forwarded to destination server in this cluster.
According to another aspect of the invention, additionally provide a kind of data transmission method being applied to source server, comprise: the proxy server that the most each cluster of transmission data to be transmitted is arranged respectively, wherein, described proxy server is used for each destination server described data to be transmitted being forwarded to its place cluster.
Alternatively, described method also comprises: before the proxy server that transmission data to be transmitted most each cluster is arranged respectively, according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, wherein, N be not less than 1 positive integer; And utilize the described N number of queue distributed to transmit described data to be transmitted to the proxy server of described each cluster.
Alternatively, described method also comprises: in transmitting procedure, and whether the parameter of monitoring current data to be transmitted changes; When the parameter monitoring current data to be transmitted changes, redistribute M for transmitting the queue of described data to be transmitted according to the parameter after change; And utilize described M the queue redistributed to proceed data transfer operation.
Alternatively, according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, comprise: the configuration information determining this source server according to the parameter of described data to be transmitted, wherein, described configuration information comprises number of processes information, number of network connections information; And distribute N number of queue for transmitting described data to be transmitted according to described configuration information.
Alternatively, described N number of queue that described utilization distributes transmits described data to be transmitted to the proxy server of described each cluster, comprising: the priority-level determining described data to be transmitted according to the parameter of described data to be transmitted; The priority-level of described data to be transmitted is utilized to sort to the described data to be transmitted of buffer memory in described N number of queue; And carry out described data transfer operation according to ranking results.
Alternatively, the priority-level of described data to be transmitted is utilized to sort to the described data to be transmitted of buffer memory in described N number of queue, comprise: before data to be transmitted higher for priority being placed in the lower data to be transmitted of priority, with the data to be transmitted that priority described in prioritised transmission is higher.
Alternatively, described method also comprises: after the proxy server that the most each cluster of transmission data to be transmitted is arranged respectively, the transmission result that the proxy server receiving each cluster returns.
Alternatively, described method also comprises: after the transmission result that the proxy server receiving each cluster returns, and carries out transmission state analysis, obtain analysis result according to described transmission result, wherein, described transmission state comprise following one of at least: transmission speed, transmission success rate, transmission packet loss; And utilize described analysis result distribution queue number.
Alternatively, described method also comprises: after the transmission result that the proxy server receiving each cluster returns, if the transmission result that arbitrary proxy server returns is transmission operation failure, again transmits described data to be transmitted to this proxy server.
According to another aspect of the present invention, additionally provide a kind of data transmission method being applied to proxy server, comprising: receive the data to be transmitted from source server; Forward each destination server of described data to be transmitted to this proxy server place cluster.
Alternatively, described method also comprises:
After receiving the data to be transmitted from described source server, return transmission result to described source server, perform subsequent operation for described source server according to described transmission result.
According to the data system of the embodiment of the present invention, source server transmission data to be transmitted is to the proxy server of each cluster.For any one cluster in each cluster, proxy server in cluster receives data to be transmitted, and forward data to be transmitted to each destination server in this cluster, and each destination server in cluster receives the data to be transmitted that this cluster proxy server issues.Directly transmit the situation of the multiple destination servers in data to be transmitted to same cluster at source server under, if directly pass through serial mode, transmission speed cannot be ensured, and then the real-time of data to be transmitted cannot be ensured, if pass through parallel mode, then source server needs all to set up network with the multiple destination servers in this cluster and be connected, the network of source server is caused to connect and the bad control of process/thread quantity, and this cluster may be in different regions from source server, distant between the two, cause the network connective stability set up between source server and multiple destination server poor.And the data system of the embodiment of the present invention can set up proxy server for same cluster, source server needs the data to be transmitted of the multiple destination servers being sent to same cluster originally, only need the proxy server being sent to this cluster, then data to be transmitted is forwarded to the multiple destination servers in this cluster by proxy server.The system of a proxy server is only there is for each cluster, the transmission of the across a network of large quantity is changed into single across a network transmission (by source server to proxy server) and add the interior transmission of the network of large quantity (proxy server is to multiple destination server), such scheme is while guarantee data transmission bauds, reduce across a network the number of transmissions, improve data transmission efficiency rationally and effectively.Transmission between the proxy server of cluster and the multiple destination servers in this cluster is connected in netting, and therefore, the communication speed of transmitting procedure is fast, and good stability.In addition, source server needs the network equal with the multiple destination server numbers in same cluster set up to connect originally, be kept to and only need foundation to be connected with the network set up between the proxy server of this cluster, such way it also avoid the problem of too much number of network connections and process/thread number, and then the wasting of resources of source server can be avoided to connect and process/thread junction to a large amount of networks, therefore, the embodiment of the present invention can substantially increase the resource utilization of source server.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to technological means of the present invention can be better understood, and can be implemented according to the content of specification, and can become apparent, below especially exemplified by the specific embodiment of the present invention to allow above and other objects of the present invention, feature and advantage.
According to hereafter by reference to the accompanying drawings to the detailed description of the specific embodiment of the invention, those skilled in the art will understand above-mentioned and other objects, advantage and feature of the present invention more.
Accompanying drawing explanation
By reading the detailed description of following description, various other advantage and benefit will become cheer and bright for those of ordinary skill in the art.Accompanying drawing only for illustrating the object of execution mode, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts by identical reference symbol.In the accompanying drawings:
Fig. 1 shows the structural representation of data system according to an embodiment of the invention;
Fig. 2 shows source server according to an embodiment of the invention and transmits the process chart of data to be transmitted to proxy server;
Fig. 3 A shows the schematic diagram of the data to be transmitted of buffer memory in queue according to an embodiment of the invention;
Fig. 3 B shows according to an embodiment of the invention to the schematic diagram of the result that the data to be transmitted of buffer memory in queue sorts;
Fig. 4 shows the structural representation of source server according to an embodiment of the invention;
Fig. 5 shows the structural representation of source server in accordance with another embodiment of the present invention;
Fig. 6 shows the structural representation of the source server according to another embodiment of the present invention;
Fig. 7 shows the structural representation of the source server according to another embodiment of the present invention;
Fig. 8 shows the structural representation of proxy server according to an embodiment of the invention;
Fig. 9 shows the process chart of data transmission method according to an embodiment of the invention;
Figure 10 shows the process chart of the data transmission method being applied to source server according to an embodiment of the invention;
Figure 11 shows the process chart of the data transmission method being applied to source server in accordance with another embodiment of the present invention;
Figure 12 shows the process chart of the data transmission method being applied to source server in accordance with another embodiment of the present invention;
Figure 13 shows the process chart of the data transmission method being applied to source server in accordance with another embodiment of the present invention;
Figure 14 shows the process chart of the data transmission method being applied to source server in accordance with another embodiment of the present invention;
Figure 15 shows the process chart of the data transmission method being applied to proxy server according to an embodiment of the invention; And
Figure 16 shows the data transmission method being applied to proxy server in accordance with another embodiment of the present invention.
Embodiment
Below with reference to accompanying drawings exemplary embodiment of the present disclosure is described in more detail.Although show exemplary embodiment of the present disclosure in accompanying drawing, however should be appreciated that can realize the disclosure in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the disclosure can be understood, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
For solving the problems of the technologies described above, embodiments provide a kind of data system.Fig. 1 shows the structural representation of data system according to an embodiment of the invention.See Fig. 1, the data system of the embodiment of the present invention at least comprises source server 110, lays respectively at multiple destination servers 130 of different cluster, and the proxy server 120 arranged respectively for each cluster.
Now introduce the annexation between each device of the data system of the embodiment of the present invention or the function of composition and each several part:
Source server 110: be suitable for transmitting the proxy server 120 of data to be transmitted to each cluster;
For any one cluster in each cluster,
Proxy server 120 in this cluster: be suitable for receiving data to be transmitted, and forward all destination servers 130 in data to be transmitted to this cluster;
Each destination server 130 in this cluster: receive the data to be transmitted that this cluster proxy server 120 issues.
It should be noted that, as illustrated three clusters in Fig. 1, in practical operation, the number of cluster can be arbitrary integer, and the destination server 130 in cluster also can be arbitrary integer, and the embodiment of the present invention is not limited this.In actual applications, the cluster in the embodiment of the present invention can be the server being positioned at same machine room, can be the server being positioned at same local area network (LAN), can also be to provide the server being positioned at same region of certain particular application services, etc.In addition, the numbering of each cluster shown in Fig. 1 only for distinguishing each cluster, the framework of each cluster and the effect performed in the data system of the embodiment of the present invention similar, therefore, Fig. 1 only shows the framework of cluster 1, and other clusters repeat no more.In like manner, for the proxy server 120 in any cluster and destination server 130, seemingly, role is identical for the class of operation performed in the data system of the embodiment of the present invention, and therefore, the embodiment of the present invention is only introduced with the example in Fig. 1.
According to the data system of the embodiment of the present invention, source server 110 transmits the proxy server 120 of data to be transmitted to each cluster.For any one cluster in each cluster, proxy server 120 in cluster receives data to be transmitted, and forward data to be transmitted to each destination server 130 in this cluster, and each destination server 130 in cluster receives the data to be transmitted that this cluster proxy server 120 issues.If source server 110 directly transmits the multiple destination servers 130 in data to be transmitted to same cluster, then source server 110 needs all to set up network with the multiple destination servers 130 in this cluster and be connected, the network of source server 110 is caused to connect and the bad control of process/thread quantity, and this cluster may be in different regions from source server 110, distant between the two, cause the network connective stability set up between source server 110 and multiple destination server 130 poor.And the data system of the embodiment of the present invention is same cluster sets up proxy server 120, then source server 110 needs the data to be transmitted of the multiple destination servers 130 being sent to same cluster originally, only need the proxy server 120 being sent to this cluster, then data to be transmitted is forwarded to the multiple destination servers 130 in this cluster by proxy server 120.For the system that only there is a proxy server in each cluster, the transmission of the across a network of large quantity is changed into single across a network transmission (by source server to proxy server) and add the interior transmission of the network of large quantity (proxy server is to multiple destination server), such scheme reduces across a network the number of transmissions, improves data transmission efficiency rationally and effectively.Transmission between the proxy server 120 of cluster and the multiple destination servers 130 in this cluster is connected in netting, and therefore, the communication speed of transmitting procedure is fast, and good stability.In addition, source server 110 needs the network equal with multiple destination servers 130 number in same cluster set up to connect originally, be kept to and only need foundation to be connected with the network set up between the proxy server 120 of this cluster, such way it also avoid the problem of too much number of network connections and process/thread number, and then the resource of source server 110 can be avoided to be wasted to the connection of a large amount of networks and process/thread junction, therefore, the above embodiment of the present invention can improve the resource utilization of source server 110 greatly.
From above, the embodiment of the present invention is by effectively improving data transmission efficiency for multiple destination servers 130 of same cluster arrange proxy server 120.No matter destination server 130 is in consolidated network with source server 110 or is in heterogeneous networks, by the multiple destination servers 130 for same cluster, proxy server 120 is set, the network that the embodiment of the present invention all can reduce source server 110 in a large number connects and/process number and/or Thread Count, and then effectively can improve the resource utilization of source server 110.Especially source server 110 and destination server 130 are in the situation of heterogeneous networks, due to distant between source server 110 and destination server 130, therefore transmission speed may be caused slower, and transmission stability is poor, and the data system of the foundation embodiment of the present invention, the across a network transmitted by multiple across a networks between source server 110 and destination server 130 between the proxy server 120 being reduced to source server 110 and destination server 130 place cluster transmits, and by transmitting in the multiple nets between proxy server 120 and destination server 130, data to be transmitted is forwarded to destination server 130, greatly improve the stability of network transfer speeds and transmission.It can thus be appreciated that, there is when the data system of the embodiment of the present invention is applied to across a network transmission more significant advantage.
For the setting of the proxy server of cluster, consider the enforceability in practice, can be arranged by two kinds of modes in the embodiment of the present invention: a kind of mode directly chooses the proxy server of the arbitrary destination server in cluster as this cluster; Another kind of mode is that cluster arranges separately a proxy server (as shown in label in Fig. 1 120) for this reason.From destination servers different in the different corresponding same cluster of data to be transmitted possibility of source server, proxy server needs the different data to be transmitted according to receiving, and sends it to corresponding different destination server.Therefore, in data transmission procedure, the data load amount of proxy server is larger, Data Update speed is very fast, if the destination server directly chosen in cluster is as proxy server, harmful effect may be caused to user to the access of this destination server and/or to the use procedure of the more new data of this destination server in some cases.But under other situations, if the data load amount of proxy server is reasonable, the destination server directly chosen in cluster then can reduce overhead as proxy server, improves resource utilization.It should be appreciated by those skilled in the art, above-mentioned two kinds of modes arranging proxy server can be carried out choice for use or be combined according to concrete application feature, such execution mode also should be within the scope of the present invention.The above embodiment of the present invention have selected as each cluster arranges separately a proxy server, to guarantee the data to be transmitted from source server to be transferred to destination server in time by proxy server.
It should be noted that, above-described embodiment (is the mode that each cluster arranges separately a proxy server) only in the second for example is described.Those skilled in the art should understand, the present invention (namely chooses the mode of the arbitrary destination server in cluster as the proxy server of this cluster) and carries out the situation implemented in the first manner, or combine (for a part of cluster arranges separately a proxy server in two ways, another part cluster then chooses the proxy server of the arbitrary destination server in cluster as this cluster) carry out the situation implemented, therefore not to repeat here for detail.
In the data system of the embodiment of the present invention, for improving data transmission efficiency further, at source server 110, data to be transmitted being transferred in the process of the proxy server of each cluster, using the handling process shown in Fig. 2 to transmit.Please refer to Fig. 2, this flow process at least comprises the steps:
Step S202: according to the N number of queue for transmitting data to be transmitted of the parametric distribution of current data to be transmitted, wherein, N be not less than 1 positive integer;
Step S204: N number of queue that utilization distributes transmits data to be transmitted to the proxy server 120 of each cluster;
Step S206: in transmitting procedure, when the parameter of data to be transmitted changes, redistributes M for transmitting the queue of data to be transmitted according to the parameter after change, wherein, M be not less than 1 positive integer;
Step S208: utilize M the queue redistributed to proceed data transfer operation.
Source server 110 transmits data to be transmitted to proxy server 120 according to the flow chart shown in Fig. 2, namely, in data transmission procedure, source server 110 can be redistributed the queue that transmission data to be transmitted utilizes in time according to the parameter of data to be transmitted, and the resource of source server 110 is more reasonably used by redistributing queue, while ensureing data transmission efficiency, improve the resource utilization of source server 110.
In addition, in practical operation, the data to be transmitted that source server 110 transfers to the proxy server 120 of each cluster is all transmitted in generation and wait all the time.And then the parameter of data to be transmitted may repeatedly change in transmitting procedure.Therefore, do not limit in the embodiment of the present invention in transmitting procedure according to the number of times that the queue of the change of the parameter of data to be transmitted to transmission data to be transmitted is redistributed.Namely, to perform after the step S208 shown in Fig. 2 (M queue redistributing of use proceed data transfer operation after), when the parameter of data to be transmitted changes again, the embodiment of the present invention repeats the operation and data transfer operation that queue redistributes, until data transfer operation terminates.
From above, in data transmission procedure, the embodiment of the present invention can be changed in real time according to the queue of the change of the parameter of data to be transmitted to transmission data to be transmitted, therefore, the embodiment of the present invention by the resource repeatedly redistributing more reasonably use source server 110 to queue, can improve the resource utilization of source server 110.As shown in the step S202 in Fig. 2 and step S206, the embodiment of the present invention according to the parametric distribution of data to be transmitted for transmitting the queue of data to be transmitted.Particularly, the embodiment of the present invention according to the configuration information of the parameter determination source server 110 of data to be transmitted, and can divide the queue being used in transmission data to be transmitted according to configuration information.Wherein, the configuration information of source server 110 can comprise the configuration category information of any source server 110 such as number of processes information, number of network connections information.
Such as, be 100 according to the quantity of the destination server 130 place cluster of first data to be transmitted one of the parameter of data to be transmitted (i.e.), the number of network connections information one of the configuration information of source server 110 (i.e.) determining source server 110 is 100, and be that 100 distribution 10 is for transmitting the queue of first data to be transmitted according to the number of network connections information of source server 110, then in 10 queues, each queue performs 10 transmission operations, namely first data to be transmitted can be transferred to 100 clusters.
When getting second batch data to be transmitted, if the quantity determining the destination server 130 place cluster of second batch data to be transmitted is 9, then the number of network connections information of source server 110 is 9.10 queues transmitting first data to be transmitted before using if continue are transmitted second batch data to be transmitted, even if each queue only performs once transmit operation, still there is a queue free time, namely source server 110 resource is wasted on vacant transmit queue, therefore, be that 9 queues can redistributing any number being not more than 9 are transmitted second batch data to be transmitted according to the number of network connections information of source server 110, as being the multiple of 3 according to 9, distributing 3 queues and second batch data to be transmitted is transmitted.
Introduce according to the process of the parametric distribution of data to be transmitted for the queue transmitting data to be transmitted the embodiment of the present invention above, afterwards, the embodiment of the present invention utilizes the queue distributed to carry out data transfer operation.Particularly, the embodiment of the present invention is first according to the priority-level of the parameter determination data to be transmitted of data to be transmitted, secondly, the priority-level of data to be transmitted is utilized to sort to the data to be transmitted of buffer memory in the queue for transmitting data to be transmitted, finally, transmit the proxy server 120 of data to be transmitted to each cluster according to ranking results, and issued each destination server 130 in data to be transmitted to this cluster by each proxy server 120.
Such as, Fig. 3 A shows according to an embodiment of the invention to the schematic diagram that the data to be transmitted of buffer memory in queue sorts.Queue 1 to queue 5 as shown in Figure 3A.Assuming that the data to be transmitted of current buffer memory in queue 1 to queue 5 is the first data, the first data need to transfer to 15 proxy servers 120, and namely queue 1 to queue 5 sends and then the first data all can be sent to corresponding proxy server 120 for 3 times.For simple and clear signal, namely show in queue 1 to queue 5 in Fig. 3 A, each queue buffer memory 3 parts of data to be transmitted are waited for and being transmitted.Determine the importance of the first data and/or emergency and/or requirement of real-time according to the parameter of the first data, and determine that the priority-level of the first data is low according to determination result (as low in importance, emergency is low and requirement of real-time is low).Now the second newly-generated data and the 3rd data are put in queue medium to be sent.Wherein, the second data need to transfer to 4 proxy servers the 120, three data needs to transfer to 9 proxy servers 120.And obtaining determination result according to the parameter of the second data is: importance is high, emergency is high and requirement of real-time is high, then the priority-level of the second data is high; And obtain determination result according to the parameter of the 3rd data and be: in importance, emergency is low and requirement of real-time is high, then during the priority-level of the 3rd data is.Sort to the first data, the second data and the 3rd data in queue 1 to queue 5, ranking results as shown in Figure 3 B.See Fig. 3 B, in the arbitrary queue of queue 1 to queue 5, priority-level is before the second high data are placed in the 3rd data that priority-level is, and priority-level be in the 3rd data be placed in before priority-level is first data at the end.Namely before data to be transmitted higher for priority is placed in the lower data to be transmitted of priority by the embodiment of the present invention, with the data to be transmitted that prioritised transmission priority is higher.In the embodiment of the present invention, data to be transmitted in queue is sorted according to priority-level, can ensure requirement of real-time higher, urgent data to be transmitted is by queue prioritised transmission, therefore improve the guarantee to real-time property to be transmitted, ensure that urgent data to be transmitted is transferred to the proxy server 120 of each cluster in time, and transfer to the destination server 130 in each cluster by proxy server 120.
It should be noted that, above to the determination mode of the priority-level of data to be transmitted and data to be transmitted is dispensed to the mode that queue transmits and is only used for example, not limitation of the present invention.Those skilled in the art is on reading basis of the present invention, the defining method according to the parameter of data to be transferred or other various priority levels of other relevant informations can be implemented, and the queue assignment method of other various data to be transmitted is to realize the present invention, it all should comprise within the scope of the present invention.
After utilization introduction is above sorted to the data to be transmitted of buffer memory in queue, the embodiment of the present invention transmits the proxy server 120 of data to be transmitted to each cluster according to ranking results.Further, after data are transmitted, the transmission result that the proxy server 120 that can receive each cluster again returns.For carrying out perfect to data transfer operation further, the transmission result that the embodiment of the present invention can return according to the proxy server 120 of each cluster carries out transmission state analysis, and to adjust subsequent transmission operations according to analysis result and/or optimize (such as redistributing queue according to analysis result).In addition, after the transmission result that the proxy server 120 receiving each cluster returns, if the transmission result that arbitrary proxy server 120 returns is transmission operation failure, again transmit data to be transmitted to this proxy server 120, effectively to increase data transmission success.
Based on the data system that each embodiment above provides, based on same inventive concept, embodiments provide a kind of source server.Fig. 4 shows the structural representation of source server according to an embodiment of the invention.See Fig. 4, the source server of the embodiment of the present invention at least comprises transport module 420.Wherein, transport module 420, is suitable for transmitting the proxy server of data to be transmitted to each cluster, and wherein, the proxy server of each cluster is used for each destination server data to be transmitted being forwarded to its place cluster.
Fig. 5 shows the structural representation of source server in accordance with another embodiment of the present invention.Please also refer to Fig. 4 and Fig. 5, compare the source server shown in Fig. 4, the source server (shown in Fig. 5) of the embodiment of the present invention also comprises:
Distribution module 410: be suitable for according to the N number of queue for transmitting data to be transmitted of the parametric distribution of data to be transmitted, wherein, N be not less than 1 positive integer;
Transport module 420: be coupled with distribution module 410, is suitable for the proxy server transmission data to be transmitted utilizing N number of queue of distribution to each cluster;
In another embodiment, above-mentioned source server can further include:
Monitoring modular 430: be coupled with distribution module 410, is suitable in transmitting procedure, and whether the parameter of monitoring current data to be transmitted changes; And
When the parameter monitoring current data to be transmitted changes, trigger distribution module.
In this situation, distribution module 410 can receive the triggering of monitoring modular, and redistributes M for transmitting the queue of data to be transmitted according to the parameter after change; Transport module 420 then can utilize M the queue redistributed to proceed data transfer operation.
In another embodiment, distribution module 410 is also suitable for:
Determine the configuration information of this source server according to the parameter of data to be transmitted, and distribute N number of queue for transmitting data to be transmitted according to configuration information, wherein, configuration information comprises number of processes information, number of network connections information.
Fig. 6 shows the structural representation of the source server according to another embodiment of the present invention.Please also refer to Fig. 4, Fig. 5 and Fig. 6, compare the source server shown in Fig. 4 and Fig. 5, the transport module 420 (shown in Fig. 6) of the embodiment of the present invention comprises further:
Determining unit 421: the priority-level being suitable for the parameter determination data to be transmitted according to data to be transmitted;
Sequencing unit 422: be coupled with determining unit 421, is suitable for utilizing the priority-level of data to be transmitted to sort to the data to be transmitted of buffer memory in N number of queue; And
Transmission unit 423: be coupled with sequencing unit 422, is suitable for carrying out data transfer operation according to ranking results.
In one embodiment, sequencing unit 422 is also suitable for:
Before data to be transmitted higher for priority being placed in the lower data to be transmitted of priority, with the data to be transmitted that prioritised transmission priority is higher.
Fig. 7 shows the structural representation of the source server according to another embodiment of the present invention.Please also refer to Fig. 4, Fig. 5, Fig. 6 and Fig. 7, the source server compared in Fig. 4, Fig. 5 and Fig. 6, the source server (shown in Fig. 7) of the embodiment of the present invention also comprises:
Receiver module 440: be coupled with transport module 420, the transmission result that the proxy server being suitable for receiving each cluster returns.
In one embodiment, the source server compared in Fig. 4, Fig. 5 and Fig. 6, the source server (shown in Fig. 7) of the embodiment of the present invention also comprises further:
Processing module 450: be coupled with receiver module 440, is suitable for carrying out transmission state analysis according to transmission result, obtains analysis result, wherein, transmission state comprise following one of at least: transmission speed, transmission success rate, transmission packet loss;
Distribution module 410: be also suitable for utilizing above-mentioned analysis result distribution queue number.
In another embodiment, transport module 420 is also suitable for:
After receiver module receives transmission result, if the transmission result that arbitrary proxy server returns is transmission operation failure, again transmit data to be transmitted to this proxy server.
Based on the data system that each embodiment above provides, based on same inventive concept, the embodiment of the present invention additionally provides a kind of proxy server, is arranged at a cluster, wherein, also there is multiple destination server in described cluster.Fig. 8 shows the structural representation of proxy server according to an embodiment of the invention.See Fig. 8, the proxy server of the embodiment of the present invention at least comprises receiver module 810 and sending module 820.
Now introduce the annexation between each device of the proxy server of the embodiment of the present invention or the function of composition and each several part:
Receiver module 810: be suitable for receiving the data to be transmitted from source server;
Sending module 820: be coupled with receiver module 810, is suitable for sending each destination server in data to be transmitted to this cluster.
In one embodiment, sending module 820 is also suitable for:
Return transmission result to source server, for source server according to transmission result adjustment subsequent transmission operations.
The data system provided based on each embodiment above and device (comprising source server, destination server and proxy server), based on same inventive concept, the embodiment of the present invention additionally provides a kind of data transmission method being applied to data system.Fig. 9 shows the process chart of data transmission method according to an embodiment of the invention.Please refer to Fig. 9, this flow process at least comprises the steps:
Step S902: for each cluster arranges proxy server;
Step S904: data to be transmitted is transferred to corresponding proxy server;
Step S906: data to be transmitted is forwarded to the destination server in this cluster by the proxy server of utilization correspondence.
Based on data system, device (comprising source server and proxy server) and method that each embodiment above provides, based on same inventive concept, the embodiment of the present invention additionally provides a kind of data transmission method being applied to source server.Figure 10 shows the process chart of the data transmission method being applied to source server according to an embodiment of the invention.See Figure 10, this flow process at least comprises the steps:
Step S1002: the proxy server that the most each cluster of transmission data to be transmitted is arranged respectively, wherein, proxy server is used for each destination server data to be transmitted being forwarded to its place cluster.
Figure 11 shows the process chart of the data transmission method being applied to source server in accordance with another embodiment of the present invention.Please also refer to Figure 10 and Figure 11, the data transmission method of the Figure 10 that compares, the data transmission method (shown in Figure 11) of the embodiment of the present invention comprises further:
Step S1004: before the proxy server that transmission data to be transmitted most each cluster is arranged respectively, according to the N number of queue for transmitting data to be transmitted of the parametric distribution of data to be transmitted, wherein, N be not less than 1 positive integer;
Step S1006: utilize N number of queue of distribution to the proxy server transmission data to be transmitted of each cluster.
Known with reference to Figure 10 and Figure 11 in the lump, the step S1006 in Figure 11 is an embodiment of step S1002 in Figure 10.Utilize step S1004 and step S1006 to illustrate in this example in Figure 11 and can distribute N individual queue (i.e. step S1004), and the transmission (i.e. step S1006) of the data to be transmitted utilizing queue to realize mentioning in step S1002.
Figure 12 shows the process chart of the data transmission method being applied to source server in accordance with another embodiment of the present invention.Please also refer to Figure 11 and Figure 12, the data transmission method of the Figure 11 that compares, the data transmission method (as shown in figure 12) of the embodiment of the present invention, except comprising the step S1004 shown in Figure 11 and step S1006, also comprises:
Step S1008: in transmitting procedure, whether the parameter of monitoring current data to be transmitted changes;
Step S1010: when the parameter monitoring current data to be transmitted changes, redistributes M for transmitting the queue of data to be transmitted according to the parameter after change;
Step S1012: utilize M the queue redistributed to proceed data transfer operation.
In another embodiment, according to the N number of queue for transmitting data to be transmitted of the parametric distribution of data to be transmitted, comprise further: the configuration information determining this source server according to the parameter of data to be transmitted, wherein, configuration information comprises number of processes information, number of network connections information; And distribute N number of queue for transmitting data to be transmitted according to configuration information.
In another embodiment, utilize the N number of queue distributed to the proxy server transmission data to be transmitted of each cluster, comprise further: according to the priority-level of the parameter determination data to be transmitted of data to be transmitted; The priority-level of data to be transmitted is utilized to sort to the data to be transmitted of buffer memory in N number of queue; And carry out data transfer operation according to ranking results.
In another embodiment, the priority-level of data to be transmitted is utilized to sort to the data to be transmitted of buffer memory in N number of queue, comprise further: before data to be transmitted higher for priority being placed in the lower data to be transmitted of priority, with the data to be transmitted that prioritised transmission priority is higher.
In another embodiment, the data transmission method of the embodiment of the present invention also comprises:
Step S1014: after the proxy server that the most each cluster of transmission data to be transmitted is arranged respectively, the transmission result that the proxy server receiving each cluster returns.
In one embodiment, the data transmission method of the embodiment of the present invention, after step S1014, also comprises step S1016, step S1018, as shown in figure 13.In another embodiment, the data transmission method of the embodiment of the present invention, after step S1014, also comprises step S1020, as shown in figure 14.
Please also refer to Figure 11 and Figure 13, compare the data transmission method shown in Figure 11, in the data transmission method shown in Figure 13, except comprising the step S1004 shown in Figure 11 and step S1006, also comprises step S1014 mentioned above, and,
Step S1016: after the transmission result that the proxy server receiving each cluster returns, carries out transmission state analysis according to transmission result, obtains analysis result, wherein, transmission state comprise following one of at least: transmission speed, transmission success rate, transmission packet loss;
Step S1018: utilize analysis result distribution queue number.
Please also refer to Figure 11 and Figure 14, compare the data transmission method shown in Figure 11, in the data transmission method shown in Figure 14, except comprising the step S1004 shown in Figure 11 and step S1006, also comprises step S1014 mentioned above, and,
Step S1020: after the transmission result that the proxy server receiving each cluster returns, if the transmission result that arbitrary proxy server returns is transmission operation failure, transmits data to be transmitted again to this proxy server.
It should be noted that, above to the compound mode of each step in the data transmission method being applied to source server, built-up sequence only for example, not limitation of the present invention.Those skilled in the art is on reading basis of the present invention, can be combined with each other each step in said method according to performance, and adjustment order and/or the operation such as deletion and/or merging are carried out to above-mentioned steps, to realize similar technique effect, its execution mode all the present invention includes within scope.
Based on data system, device (comprising source server and proxy server) and method that each embodiment above provides, based on same inventive concept, the embodiment of the present invention additionally provides a kind of data transmission method being applied to proxy server.Figure 15 shows the process chart of the data transmission method being applied to proxy server according to an embodiment of the invention.See Figure 15, this flow process at least comprises the steps:
Step S1502: receive the data to be transmitted from source server;
Step S1504: forward each destination server of data to be transmitted to this proxy server place cluster.
Figure 16 shows the data transmission method being applied to proxy server in accordance with another embodiment of the present invention.Please also refer to Figure 15 and Figure 16, except comprising above-mentioned steps S1502 and S1504, the data transmission method shown in Figure 16 also comprises:
Step S1506: after receiving the data to be transmitted from source server, returns transmission result to source server, for source server according to transmission result adjustment subsequent operation.
According to the combination of any one embodiment above-mentioned or multiple embodiment, the embodiment of the present invention can reach following beneficial effect:
According to the data system of the embodiment of the present invention, source server transmission data to be transmitted is to the proxy server of each cluster.For any one cluster in each cluster, proxy server in cluster receives data to be transmitted, and forward data to be transmitted to each destination server in this cluster, and each destination server in cluster receives the data to be transmitted that this cluster proxy server issues.Directly transmit the situation of the multiple destination servers in data to be transmitted to same cluster at source server under, if directly pass through serial mode, transmission speed cannot be ensured, and then the real-time of data to be transmitted cannot be ensured, if pass through parallel mode, then source server needs all to set up network with the multiple destination servers in this cluster and be connected, the network of source server is caused to connect and the bad control of process/thread quantity, and this cluster may be in different regions from source server, distant between the two, cause the network connective stability set up between source server and multiple destination server poor.And the data system of the embodiment of the present invention can set up proxy server for same cluster, source server needs the data to be transmitted of the multiple destination servers being sent to same cluster originally, only need the proxy server being sent to this cluster, then data to be transmitted is forwarded to the multiple destination servers in this cluster by proxy server.The system of a proxy server is only there is for each cluster, the transmission of the across a network of large quantity is changed into single across a network transmission (by source server to proxy server) and add the interior transmission of the network of large quantity (proxy server is to multiple destination server), such scheme is while guarantee data transmission bauds, reduce across a network the number of transmissions, improve data transmission efficiency rationally and effectively.Transmission between the proxy server of cluster and the multiple destination servers in this cluster is connected in netting, and therefore, the communication speed of transmitting procedure is fast, and good stability.In addition, source server needs the network equal with the multiple destination server numbers in same cluster set up to connect originally, be kept to and only need foundation to be connected with the network set up between the proxy server of this cluster, such way it also avoid the problem of too much number of network connections and process/thread number, and then the wasting of resources of source server can be avoided to connect and process/thread junction to a large amount of networks, therefore, the above embodiment of the present invention can improve the resource utilization of source server greatly.
In specification provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this specification (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this specification (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in detail in the claims, the one of any of embodiment required for protection can use with arbitrary compound mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that the some or all functions that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the some or all parts in the device of the embodiment of the present invention or equipment.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and does not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computer of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.
So far, those skilled in the art will recognize that, although multiple exemplary embodiment of the present invention is illustrate and described herein detailed, but, without departing from the spirit and scope of the present invention, still can directly determine or derive other modification many or amendment of meeting the principle of the invention according to content disclosed by the invention.Therefore, scope of the present invention should be understood and regard as and cover all these other modification or amendments.
The invention also discloses A1. source server, comprising:
Transport module, is suitable for transmitting the proxy server that the most each cluster of described data to be transmitted is arranged respectively,
Wherein, the proxy server of each cluster is used for each destination server described data to be transmitted being forwarded to its place cluster.
A2. the source server according to A1, wherein, also comprises:
Distribution module, is suitable for according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, wherein, N be not less than 1 positive integer;
Transport module, is suitable for utilizing the described N number of queue distributed to transmit described data to be transmitted to the proxy server of described each cluster.
A3. the source server according to A2, wherein, also comprises:
Monitoring modular, is suitable in transmitting procedure, and whether the parameter of monitoring current data to be transmitted changes; And when the parameter monitoring current data to be transmitted changes, trigger described distribution module;
Described distribution module, is also suitable for the triggering receiving described monitoring modular, and redistributes M for transmitting the queue of described data to be transmitted according to the parameter after change;
Described transport module, is also suitable for utilizing described M the queue redistributed to proceed data transfer operation.
A4. the source server according to A2 or A3, wherein, described distribution module is also suitable for:
Determine the configuration information of this source server according to the parameter of described data to be transmitted, wherein, described configuration information comprises number of processes information, number of network connections information; And
N number of queue for transmitting described data to be transmitted is distributed according to described configuration information.
A5. the source server according to any one of A2 to A4, wherein, described transport module comprises further:
Determining unit, is suitable for the priority-level determining described data to be transmitted according to the parameter of described data to be transmitted;
Sequencing unit, is suitable for utilizing the priority-level of described data to be transmitted to sort to the described data to be transmitted of buffer memory in described N number of queue; And
Transmission unit, is suitable for carrying out described data transfer operation according to ranking results.
A6. the source server according to A5, wherein, described sequencing unit is also suitable for:
Before data to be transmitted higher for priority being placed in the lower data to be transmitted of priority, with the data to be transmitted that priority described in prioritised transmission is higher.
A7. the source server according to any one of A1 to A6, wherein, also comprises:
Receiver module, the transmission result that the proxy server being suitable for receiving described each cluster returns.
A8. the source server according to A7, wherein, also comprises:
Processing module, is suitable for carrying out transmission state analysis according to described transmission result, obtains analysis result, wherein, described transmission state comprise following one of at least: transmission speed, transmission success rate, transmission packet loss;
Described distribution module, is also suitable for utilizing described analysis result distribution queue number.
A9. the source server according to A7, wherein, described transport module is also suitable for:
After described receiver module receives described transmission result, if the transmission result that arbitrary proxy server returns is transmission operation failure, again transmit described data to be transmitted to this proxy server.
The invention also discloses B10. proxy server, be arranged at a cluster, wherein, also there is multiple destination server in described cluster, comprising:
Receiver module, is suitable for receiving the data to be transmitted from source server;
Sending module, is suitable for each destination server sent in described data to be transmitted to this cluster.
B11. the proxy server according to B10, wherein, described sending module is also suitable for:
Return transmission result to described source server, for described source server according to described transmission result adjustment subsequent transmission operations.
The invention also discloses C12. data system, comprise the source server described in any one of A1 to A9 and at least one cluster, wherein, each cluster comprises multiple destination server, also comprises the proxy server described in B10 or B11, wherein,
Described source server, is suitable for transmitting the proxy server of data to be transmitted to described each cluster;
For any one cluster in described each cluster,
Proxy server in this cluster, is suitable for receiving described data to be transmitted, and forwards each destination server in described data to be transmitted to this cluster; And
Each destination server in this cluster, is suitable for receiving the described data to be transmitted that this cluster proxy server issues.
C13. the system according to C12, wherein, in described each cluster, the proxy server of arbitrary cluster is also suitable for being arranged by following steps:
In this cluster, select arbitrary destination server as proxy server; Or
For this cluster arranges separately a proxy server.
The invention also discloses the data transmission method that D14. mono-kind is applied to data system, described data system comprise source server, the multiple destination servers laid respectively at least one cluster, described method comprises:
For each cluster arranges proxy server;
Data to be transmitted is transferred to corresponding proxy server; And
The proxy server of described correspondence is utilized described data to be transmitted to be forwarded to destination server in this cluster.
The invention also discloses the data transmission method that E15. mono-kind is applied to source server, comprising:
The proxy server that the most each cluster of transmission data to be transmitted is arranged respectively,
Wherein, described proxy server is used for each destination server described data to be transmitted being forwarded to its place cluster.
E16. the method according to E15, wherein, described method also comprises:
Before the proxy server that transmission data to be transmitted most each cluster is arranged respectively, according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, wherein, N be not less than 1 positive integer; And
Described N number of queue that utilization distributes transmits described data to be transmitted to the proxy server of described each cluster.
E17. the method according to E16, wherein, described method also comprises:
In transmitting procedure, whether the parameter of monitoring current data to be transmitted changes;
When the parameter monitoring current data to be transmitted changes, redistribute M for transmitting the queue of described data to be transmitted according to the parameter after change; And
Described M the queue redistributed is utilized to proceed data transfer operation.
E18. the method according to E16 or E17, wherein, according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, comprising:
Determine the configuration information of this source server according to the parameter of described data to be transmitted, wherein, described configuration information comprises number of processes information, number of network connections information; And
N number of queue for transmitting described data to be transmitted is distributed according to described configuration information.
E19. the method according to any one of E16 to E18, wherein, described N number of queue that described utilization distributes transmits described data to be transmitted to the proxy server of described each cluster, comprising:
The priority-level of described data to be transmitted is determined according to the parameter of described data to be transmitted;
The priority-level of described data to be transmitted is utilized to sort to the described data to be transmitted of buffer memory in described N number of queue; And
Described data transfer operation is carried out according to ranking results.
E20. the method according to E19, wherein, utilizes the priority-level of described data to be transmitted to sort to the described data to be transmitted of buffer memory in described N number of queue, comprising:
Before data to be transmitted higher for priority being placed in the lower data to be transmitted of priority, with the data to be transmitted that priority described in prioritised transmission is higher.
E21. the method according to any one of E15 to E20, wherein, described method also comprises:
After the proxy server that the most each cluster of transmission data to be transmitted is arranged respectively, the transmission result that the proxy server receiving each cluster returns.
E22. the method according to E21, wherein, described method also comprises:
After the transmission result that the proxy server receiving each cluster returns, carry out transmission state analysis according to described transmission result, obtain analysis result, wherein, described transmission state comprise following one of at least: transmission speed, transmission success rate, transmission packet loss; And
Utilize described analysis result distribution queue number.
E23. the method according to E21, wherein, described method also comprises:
After the transmission result that the proxy server receiving each cluster returns, if the transmission result that arbitrary proxy server returns is transmission operation failure, again transmit described data to be transmitted to this proxy server.
The invention also discloses the data transmission method that F24. mono-kind is applied to proxy server, comprising:
Receive the data to be transmitted from source server;
Forward each destination server of described data to be transmitted to this proxy server place cluster.
F25. the method according to F24, wherein, described method also comprises:
After receiving the data to be transmitted from described source server, return transmission result to described source server, for described source server according to described transmission result adjustment subsequent operation.

Claims (10)

1. a source server, comprising:
Transport module, is suitable for transmitting the proxy server that the most each cluster of described data to be transmitted is arranged respectively,
Wherein, the proxy server of each cluster is used for each destination server described data to be transmitted being forwarded to its place cluster.
2. source server according to claim 1, wherein, also comprises:
Distribution module, is suitable for according to the N number of queue for transmitting described data to be transmitted of the parametric distribution of described data to be transmitted, wherein, N be not less than 1 positive integer;
Transport module, is suitable for utilizing the described N number of queue distributed to transmit described data to be transmitted to the proxy server of described each cluster.
3. source server according to claim 2, wherein, also comprises:
Monitoring modular, is suitable in transmitting procedure, and whether the parameter of monitoring current data to be transmitted changes; And when the parameter monitoring current data to be transmitted changes, trigger described distribution module;
Described distribution module, is also suitable for the triggering receiving described monitoring modular, and redistributes M for transmitting the queue of described data to be transmitted according to the parameter after change;
Described transport module, is also suitable for utilizing described M the queue redistributed to proceed data transfer operation.
4. the source server according to Claims 2 or 3, wherein, described distribution module is also suitable for:
Determine the configuration information of this source server according to the parameter of described data to be transmitted, wherein, described configuration information comprises number of processes information, number of network connections information; And
N number of queue for transmitting described data to be transmitted is distributed according to described configuration information.
5. the source server according to any one of claim 2 to 4, wherein, described transport module comprises further:
Determining unit, is suitable for the priority-level determining described data to be transmitted according to the parameter of described data to be transmitted;
Sequencing unit, is suitable for utilizing the priority-level of described data to be transmitted to sort to the described data to be transmitted of buffer memory in described N number of queue; And
Transmission unit, is suitable for carrying out described data transfer operation according to ranking results.
6. a proxy server, is arranged at a cluster, wherein, also there is multiple destination server, comprising in described cluster:
Receiver module, is suitable for receiving the data to be transmitted from source server;
Sending module, is suitable for each destination server sent in described data to be transmitted to this cluster.
7. a data system, comprise the source server described in any one of claim 1 to 5 and at least one cluster, wherein, each cluster comprises multiple destination server, also comprises proxy server according to claim 6, wherein,
Described source server, is suitable for transmitting the proxy server of data to be transmitted to described each cluster;
For any one cluster in described each cluster,
Proxy server in this cluster, is suitable for receiving described data to be transmitted, and forwards each destination server in described data to be transmitted to this cluster; And
Each destination server in this cluster, is suitable for receiving the described data to be transmitted that this cluster proxy server issues.
8. be applied to a data transmission method for data system, described data system comprise source server, the multiple destination servers laid respectively at least one cluster, described method comprises:
For each cluster arranges proxy server;
Data to be transmitted is transferred to corresponding proxy server; And
The proxy server of described correspondence is utilized described data to be transmitted to be forwarded to destination server in this cluster.
9. be applied to a data transmission method for source server, comprise:
The proxy server that the most each cluster of transmission data to be transmitted is arranged respectively,
Wherein, described proxy server is used for each destination server described data to be transmitted being forwarded to its place cluster.
10. be applied to a data transmission method for proxy server, comprise:
Receive the data to be transmitted from source server;
Forward each destination server of described data to be transmitted to this proxy server place cluster.
CN201410645843.7A 2014-11-12 2014-11-12 Data transmission method, data system and related devices Pending CN104410675A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410645843.7A CN104410675A (en) 2014-11-12 2014-11-12 Data transmission method, data system and related devices

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410645843.7A CN104410675A (en) 2014-11-12 2014-11-12 Data transmission method, data system and related devices

Publications (1)

Publication Number Publication Date
CN104410675A true CN104410675A (en) 2015-03-11

Family

ID=52648273

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410645843.7A Pending CN104410675A (en) 2014-11-12 2014-11-12 Data transmission method, data system and related devices

Country Status (1)

Country Link
CN (1) CN104410675A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105119978A (en) * 2015-07-21 2015-12-02 浪潮(北京)电子信息产业有限公司 Metadata cluster parallel distribution processing method and metadata cluster parallel distribution processing system
CN105281976A (en) * 2015-09-17 2016-01-27 国网北京市电力公司 Method and apparatus for monitoring transmission of proxy service data
CN107465644A (en) * 2016-06-03 2017-12-12 中兴通讯股份有限公司 Data transmission method, data transmission client and data transmission actuator
CN107911257A (en) * 2017-12-28 2018-04-13 北京声智科技有限公司 The control device and method of data transfer
CN110830587A (en) * 2019-11-14 2020-02-21 腾讯科技(深圳)有限公司 Data transmission method, device, equipment and storage medium
CN111259227A (en) * 2020-01-16 2020-06-09 北京旷视科技有限公司 Method and apparatus for sharing target retrieval service between multiple retrieval clusters

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101127623A (en) * 2007-09-27 2008-02-20 腾讯科技(深圳)有限公司 Data processing method, device and system
US20100153363A1 (en) * 2008-12-12 2010-06-17 Hitachi, Ltd. Stream data processing method and system
CN101883039A (en) * 2010-05-13 2010-11-10 北京航空航天大学 Data transmission network of large-scale clustering system and construction method thereof
CN102123168A (en) * 2011-01-14 2011-07-13 广州市动景计算机科技有限公司 Web page pre-reading and integration method and system based on relay server
WO2012054347A1 (en) * 2010-10-19 2012-04-26 Alibaba Group Holding Limited Communication method and server of transmission control protocol
CN103634346A (en) * 2012-08-23 2014-03-12 腾讯科技(深圳)有限公司 Data distribution method and system for network data center
US8700750B2 (en) * 2009-01-16 2014-04-15 Microsoft Corporation Web deployment functions and interfaces

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101127623A (en) * 2007-09-27 2008-02-20 腾讯科技(深圳)有限公司 Data processing method, device and system
US20100153363A1 (en) * 2008-12-12 2010-06-17 Hitachi, Ltd. Stream data processing method and system
US8700750B2 (en) * 2009-01-16 2014-04-15 Microsoft Corporation Web deployment functions and interfaces
CN101883039A (en) * 2010-05-13 2010-11-10 北京航空航天大学 Data transmission network of large-scale clustering system and construction method thereof
WO2012054347A1 (en) * 2010-10-19 2012-04-26 Alibaba Group Holding Limited Communication method and server of transmission control protocol
CN102123168A (en) * 2011-01-14 2011-07-13 广州市动景计算机科技有限公司 Web page pre-reading and integration method and system based on relay server
CN103634346A (en) * 2012-08-23 2014-03-12 腾讯科技(深圳)有限公司 Data distribution method and system for network data center

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105119978A (en) * 2015-07-21 2015-12-02 浪潮(北京)电子信息产业有限公司 Metadata cluster parallel distribution processing method and metadata cluster parallel distribution processing system
CN105119978B (en) * 2015-07-21 2019-05-24 浪潮(北京)电子信息产业有限公司 A kind of parallel distribution processing method of metadata cluster and system
CN105281976A (en) * 2015-09-17 2016-01-27 国网北京市电力公司 Method and apparatus for monitoring transmission of proxy service data
CN107465644A (en) * 2016-06-03 2017-12-12 中兴通讯股份有限公司 Data transmission method, data transmission client and data transmission actuator
CN107465644B (en) * 2016-06-03 2021-02-23 中兴通讯股份有限公司 Data transmission method, data transmission client and data transmission executor
CN107911257A (en) * 2017-12-28 2018-04-13 北京声智科技有限公司 The control device and method of data transfer
CN107911257B (en) * 2017-12-28 2021-04-23 北京声智科技有限公司 Data transmission control device and method
CN110830587A (en) * 2019-11-14 2020-02-21 腾讯科技(深圳)有限公司 Data transmission method, device, equipment and storage medium
CN110830587B (en) * 2019-11-14 2022-01-18 腾讯科技(深圳)有限公司 Data transmission method, device, equipment and storage medium
CN111259227A (en) * 2020-01-16 2020-06-09 北京旷视科技有限公司 Method and apparatus for sharing target retrieval service between multiple retrieval clusters
CN111259227B (en) * 2020-01-16 2023-11-10 北京旷视科技有限公司 Method and apparatus for sharing a targeted search service among multiple search clusters

Similar Documents

Publication Publication Date Title
CN104410675A (en) Data transmission method, data system and related devices
CN110896355B (en) Network slice selection method and device
US8166185B2 (en) System and method for enterprise software distribution
CN105812276B (en) Message forwarding adaptive system and method adopting multi-core processor
CN105511954A (en) Method and device for message processing
CN105653365A (en) Task processing method and device
CN102833294A (en) File processing method and system based on cloud storage, and server cluster system
CN111654386A (en) Method and system for establishing service function chain
CN104539713A (en) Service request processing method and device
US20130145034A1 (en) Agile hostpool allocator
CN107301178A (en) Data query processing method, apparatus and system
CN103781107A (en) Simulation, simulation processing method and device for wireless communication network
CN106649640A (en) Instant search method and device
Ke et al. Aggregation on the fly: Reducing traffic for big data in the cloud
CN111222810B (en) Method and device for distributing grid openings in real time
CN104394095A (en) Data transmission method, data transmission system and source server
CN112261125B (en) Centralized unit cloud deployment method, device and system
CN107426109B (en) Traffic scheduling method, VNF module and traffic scheduling server
CN113395183B (en) Virtual node scheduling method and system for network simulation platform VLAN interconnection
CN100499564C (en) Packet processing engine
US20170142227A1 (en) Data processing apparatus and data processing method
CN102929721B (en) Balanced scheduling system and method based on station quota
US11706097B2 (en) Task processing method applied to network topology, electronic device and storage medium
CN101909283B (en) M2M (Machine-to-Machine) platform and method for sharing load downloaded with in high capacity
CN103905515A (en) Distributed Internet information downloading system and method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20150311