CN107624231B - Method, device, server and computer program product for transmitting and receiving data - Google Patents
Method, device, server and computer program product for transmitting and receiving data Download PDFInfo
- Publication number
- CN107624231B CN107624231B CN201680002898.3A CN201680002898A CN107624231B CN 107624231 B CN107624231 B CN 107624231B CN 201680002898 A CN201680002898 A CN 201680002898A CN 107624231 B CN107624231 B CN 107624231B
- Authority
- CN
- China
- Prior art keywords
- cache
- data
- level cache
- level
- bandwidth utilization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
Abstract
The invention provides a method, a device, a server and a computer program product for sending and receiving data, wherein the data sending method comprises the following steps: detecting the link quality between a first-level cache and a second-level cache or detecting the bandwidth utilization rate of the second-level cache to obtain a detection result; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN; and when the detection result shows that the link quality between the first-level cache and the second-level cache is lower than a first threshold value or the bandwidth utilization rate of the second-level cache is lower than a second threshold value, based on ARQ, UDP is adopted to send the data in the first-level cache, so that data transmission can be effectively accelerated, and the bandwidth utilization efficiency is improved.
Description
Technical Field
The present invention relates to the field of data transmission technologies, and in particular, to a method, an apparatus, a server, and a computer program product for transmitting and receiving data.
Background
The cloud robot is an intelligent robot which is formed by placing a cognitive system in the cloud, placing a body, a drive and a sensor on a robot body and connecting the body, the drive and the sensor through mobile communication; the cloud robot is the development direction of the intelligent humanoid robot.
A Content Delivery Network (CDN) is a policy-based overall system, and includes four aspects, namely distributed storage, load balancing, Network request redirection, and Content management, where Content management and global Network traffic management are the core of the CDN, and the CDN ensures that Content provides services for a user's request in an extremely efficient manner by determining user proximity and server load.
The application of CDNs is becoming more and more common today. The CDN sets a cache on an edge server through a large number of Point of presence (edge server) deployments, so as to cache data accessed by a user, so as to improve a data access speed of the user.
The network architecture of the CDN generally divides Pop into two levels, namely: the cache service distributed close to the user is a first-level cache, and the cache service concentrated back to the source is a second-level cache, so that the pressure of the source station server of the client is reduced, and meanwhile, the service quality can be improved. In general, the first-level cache is deployed in each operator network, and the second-level cache is deployed in a BGP (Border Gateway Protocol) machine room that is interconnected and interworked. The primary cache and the secondary cache perform data transmission through a standard HTTP (Hyper Text Transfer Protocol).
The prior art is not enough: at present, the IDC (Internet Data Center) market mostly charges according to the bandwidth, and even if the actual bandwidth utilization rate is low, a user may pay according to the purchased bandwidth. The utilization rate of data transmission bandwidth between a primary cache and a secondary cache in the existing CDN is low by adopting an HTTP protocol, and bandwidth resources are not reasonably utilized.
Disclosure of Invention
The embodiment of the invention provides a technical scheme for mainly solving the problem of low utilization rate of data transmission bandwidth between a primary cache and a secondary cache in the existing CDN.
A first aspect of an embodiment of the present invention provides a data transmission method, including the following steps:
detecting the link quality between a first-level cache and a second-level cache or detecting the bandwidth utilization rate of the second-level cache to obtain a detection result; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN;
and when the detection result shows that the link quality between the primary cache and the secondary cache is lower than a first threshold value or the bandwidth utilization rate of the secondary cache is lower than a second threshold value, transmitting the data in the primary cache by adopting UDP based on ARQ.
A second aspect of the embodiments of the present invention provides a data transmission apparatus, including:
the detection module is used for detecting the link quality between the first-level cache and the second-level cache or detecting the bandwidth utilization rate of the second-level cache to obtain a detection result; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN;
a first sending module, configured to send data in the primary cache by using UDP based on ARQ when the detection result indicates that the link quality between the primary cache and the secondary cache is lower than a first threshold, or that the bandwidth utilization of the secondary cache is lower than a second threshold; the first-level cache is used for caching data accessed by a user.
A third aspect of the embodiments of the present invention provides a data receiving method, including the following steps:
when the link quality between a primary cache and a secondary cache is lower than a first threshold value or the bandwidth utilization rate of the secondary cache is lower than a second threshold value, receiving data from the primary cache sent by adopting UDP based on ARQ control, and converting the data from the UDP format to the HTTP format; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN;
and caching the data in the HTTP format to a second-level cache.
A fourth aspect of an embodiment of the present invention provides a data receiving apparatus, including:
the receiving and converting module is used for receiving data from the primary cache sent by adopting UDP based on ARQ control when the link quality between the primary cache and the secondary cache is lower than a first threshold or the bandwidth utilization rate of the secondary cache is lower than a second threshold, and converting the data from the UDP format to the HTTP format; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second level cache is a cache in the CDN for centralizing back to the source.
And the cache module is used for caching the data in the HTTP format to the second-level cache.
A fifth aspect of the embodiments of the present invention provides a server, where the server includes:
the system comprises a memory, one or more processors, a communication interface, a bus and one or more modules, wherein the processor, the memory and the communication interface are connected through the bus and complete mutual communication; the one or more modules are stored in the memory and configured to be executed by the one or more processors, the one or more modules including instructions for performing the steps of the data transmission method.
A sixth aspect of embodiments of the present invention provides a computer program product, which includes a computer-readable storage medium and a computer program mechanism embedded therein, where the computer program mechanism includes instructions for executing steps of the data transmission method.
A seventh aspect of the embodiments of the present invention provides a server, where the server includes:
the system comprises a memory, one or more processors, a communication interface, a bus and one or more modules, wherein the processor, the memory and the communication interface are connected through the bus and complete mutual communication; the one or more modules are stored in the memory and configured to be executed by the one or more processors, the one or more modules including instructions for performing the steps of the data reception method.
An eighth aspect of the embodiments of the present invention provides a computer program product, which includes a computer-readable storage medium and a computer program mechanism embedded therein, where the computer program mechanism includes instructions for executing each step in the data receiving method.
The beneficial effects are as follows:
the most significant characteristic is that each request sent by a client needs a server to feed back a response, and needs a plurality of times of handshake communication, resulting in the problems of slow data Transmission and low bandwidth utilization rate between the primary cache and the secondary cache in the CDN network.
Because the technical proposal provided by the embodiment of the invention establishes the ARQ transmission tunnel between the first-level cache and the second-level cache, a detection module is added at the side of the first-level cache, the existing transmission control module of the first-level cache is improved, the link quality between the first-level cache and the second-level cache is detected or the bandwidth load of the second-level cache is detected, when the detection result is that the link quality between the first level cache and the second level cache is lower than a first threshold value, or, when the bandwidth utilization rate of the second-level cache is lower than the second threshold, the data in the first-level cache is sent by using the UDP based on the ARQ, because the UDP is not connection-oriented and does not establish connection with the other party before transmitting the data, no acknowledgement signal is sent for the received data, and therefore, based on the ARQ controlled UDP transport, the data transmission speed and efficiency of the first-level cache and the second-level cache in the CDN network can be effectively improved.
Drawings
Specific embodiments of the present invention will now be described with reference to the accompanying drawings, in which:
fig. 1 is a flowchart illustrating a data transmission method according to a first embodiment of the present invention;
FIG. 2 is a diagram of an ARQ module according to an embodiment;
fig. 3 is a schematic structural diagram of a data transmission apparatus according to a first embodiment of the present invention;
fig. 4 is a schematic flowchart of a data receiving method according to an embodiment of the present invention;
FIG. 5 is a block diagram of a data receiving device according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a server (I) according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a server (ii) according to an embodiment of the present invention;
fig. 8 is a schematic diagram of an agent structure of a device on a CDN first-level cache side according to an embodiment of the present invention;
fig. 9 is a schematic processing flow diagram of an agent on the CDN first-level cache side in the embodiment of the present invention;
fig. 10 is a schematic flow chart of sending data by a device on the CDN first-level cache side after the scheme provided by the embodiment of the present invention is adopted.
Detailed Description
In order to make the technical solutions and advantages of the present invention more apparent, the following further detailed description of exemplary embodiments of the present invention is provided with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and are not exhaustive of all embodiments. And the embodiments and features of the embodiments in the present description may be combined with each other without conflict.
The inventor notices in the process of invention that: the HTTP protocol is an application established on a TCP, and needs to be subjected to multiple handshake communication before the TCP protocol transmits data, each request sent by a client needs to be responded by a server feedback, and multiple handshake communication is needed, which may cause the problems of slow data transmission and low bandwidth utilization rate between the primary cache and the secondary cache in the existing CDN network.
In addition, most of the existing IDCs (Internet Data centers) charge according to bandwidth, and even if the actual bandwidth utilization rate is low, the user may pay according to the purchased bandwidth. In this case, the bandwidth utilization rate can be increased to make reasonable use of the bandwidth resources.
To facilitate the practice of the invention, the following examples are given.
Example one
Based on this, embodiments of the present invention propose a method, an apparatus, a server, and a computer program product for data transmission and reception, which are described below.
Fig. 1 is a schematic flow chart of a data transmission method according to a first embodiment of the present invention, and as shown in fig. 1, the data transmission method may include the following steps:
and 102, when the detection result is that the link quality between the first-level cache and the second-level cache is lower than a first threshold value, or the bandwidth utilization rate of the second-level cache is lower than a second threshold value, transmitting the data in the first-level cache by using UDP based on ARQ.
In a specific implementation, the data sending method may be implemented on a device on a CDN first-level cache side. The improvement may be made on the basis of the existing CDN device, for example, a detection module and a processing function module for controlling UDP based on ARQ are added to an agent of the device on the existing CDN primary cache side, the link quality between the primary cache and the secondary cache is detected or the bandwidth utilization of the secondary cache is detected, and data in the primary cache is sent based on a detection result, specifically: when the detection result is that the link quality between the first-level cache and the second-level cache is lower than a first threshold value, or the bandwidth utilization rate of the second-level cache is lower than a second threshold value, based on ARQ, adopting UDP to send data in the first-level cache; otherwise, when the detection result indicates that the link quality between the first-level cache and the second-level cache is not lower than a first threshold value and the bandwidth utilization rate of the second-level cache is not lower than a second threshold value, the data in the first-level cache is sent by adopting HTTP.
Correspondingly to the device on the CDN primary cache side, an ARQ processing function module is added in the proxy of the device on the CDN secondary cache side, and the ARQ processing function module may receive data in the UDP format based on ARQ control from the primary cache, convert the data into the HTTP format, and cache the converted data into the secondary cache.
The link quality may be measured by a data packet loss rate, and in specific implementation, parameters such as an error rate and a time delay may be considered comprehensively, which is not limited specifically here.
In a specific implementation, if the bandwidth utilization of the second-level cache needs to be considered, the bandwidth utilization of the second-level cache can be detected.
The bandwidth utilization rate of the secondary cache may be detected by SNMP (Simple Network Management Protocol).
The SNMP mode is adopted in the implementation, because the SNMP mode is common and mainstream, and is easy to use/understand by the technicians in the field, the SNMP is taken as an example; however, theoretically, other manners are also possible, as long as the bandwidth utilization rate of the second-level cache can be detected, and the SNMP is only used for teaching a person skilled in the art how to implement the present invention specifically, but does not mean that the SNMP can be used only, and the implementation process can be combined with practical needs to determine the corresponding manner.
Fig. 2 is a schematic diagram of an ARQ module in this embodiment, and as shown in fig. 2, an ARQ transmission tunnel is established between a device on the CDN primary cache side and a device on the CDN secondary cache side through an ARQ client 1 and an ARQ client 2, so as to implement data transmission between the CDN primary cache and the CDN secondary cache based on ARQ control UDP.
ARQ is an automatic repeat request protocol, which is one of error correction protocols of a data link layer and a transport layer in an OSI (Open System Interconnection) model, and realizes reliable information transmission on the basis of unreliable services by using two mechanisms of acknowledgement and timeout. If the sender does not receive the acknowledgement frame from the receiver within a period of time after the sending, the sender retransmits. ARQ includes stop-and-wait ARQ protocols, error detection, positive acknowledgements, timed-out retransmissions and negative acknowledgements and retransmissions.
ARQ protocols are widely used in wireless networks, and are applied to the link layer of wireless communications. In wired networks, there are also implementations based on ARQ. For example, the open-source KCP is a quick and reliable realization method of the ARQ protocol, the KCP can replace the transmission effect that the average time delay is reduced by 30% -40% and the maximum delay is reduced by three times at the cost of wasting 10% -20% of bandwidth compared with TCP, and the data transmission speed can be improved by nearly ten times. The KCP only provides an algorithm for transmission control, and does not provide data transceiving of a bottom layer protocol. The user can compile the implementation code according to the business requirement of the user. The KCP is in https:// github. com/skywind 3000/KCP. The whole protocol only has two source files of ikcp.h and ikcp.c, and can be conveniently integrated into the protocol stack of the user. The two files are integrated into a self-developed project, and a set of complete ARQ reliable protocol can be realized by adding a plurality of lines of codes.
The KCP and UDP systems are used in the implementation because they are relatively common and mainstream and are easy to be used/understood by those skilled in the art, so the KCP and UDP are taken as examples; however, in theory, other manners are also possible, as long as data transmission based on ARQ and simplified connection-oriented or data transmission handshake protocol can be realized, KCP and UDP are only used for teaching one skilled in the art how to implement the present invention specifically, but do not mean that only KCP and UDP can be used, and the implementation process can be combined with practical needs to determine the corresponding manner.
For a detailed description of ARQ, reference may be made to standard protocol RFC3366, which is not described in detail here.
The first-level cache caches data accessed by users, and periodically or aperiodically sends the cached data to the second-level cache which is centralized and returned to the source, so that subsequent users can conveniently access the data.
The first threshold and the second threshold in the embodiment of the present invention may be manually set according to technical experience, or may be obtained through historical data statistics, which is not specifically limited herein.
The data transmission method provided by the embodiment has the advantage of reasonably utilizing the ARQ data transmission accuracy, the link quality between the first-level cache and the second-level cache can be firstly detected, and if the link quality is lower than a first threshold, UDP is adopted for data transmission based on ARQ; if the link quality is not lower than the first threshold, the bandwidth utilization rate of the secondary cache is continuously detected, if the bandwidth utilization rate of the secondary cache is lower than the second threshold, data transmission is carried out by adopting UDP based on ARQ, otherwise, data transmission is carried out by adopting HTTP.
In the specific implementation, a person skilled in the art may also, in combination with specific needs, first detect the bandwidth utilization rate of the secondary cache, and if the bandwidth utilization rate of the secondary cache is lower than a second threshold, perform data transmission by using UDP based on ARQ; otherwise, continuously detecting the link quality between the first-level cache and the second-level cache, if the link quality is lower than a first threshold value, adopting UDP to transmit data based on ARQ, and otherwise, adopting HTTP to transmit data.
The link quality between the first-level cache and the second-level cache and the bandwidth utilization detection sequence of the second-level cache are not limited.
In implementation, the implementing the data sending method may further include:
receiving a connection request from a second level cache prior to detecting link quality between the first level cache and the second level cache or detecting bandwidth utilization of the second level cache.
Has the advantages that: detection can be performed after a connection request from the second-level cache is received, and detection can be reasonably performed as required.
In implementation, sending data in the first-level buffer by using UDP based on ARQ control may specifically include:
and after converting the data in the primary cache from the HTTP format to the UDP format, transmitting the data in the primary cache by adopting the UDP based on the ARQ.
The data transmission method may further include:
and when the detection result shows that the link quality between the first-level cache and the second-level cache is not lower than a first threshold value and the bandwidth utilization rate of the second-level cache is not lower than a second threshold value, adopting HTTP to send data to the second-level cache.
Has the advantages that: when the detection result is that the link quality between the first-level cache and the second-level cache is not lower than a first threshold value and the bandwidth utilization rate of the second-level cache is not lower than a second threshold value, HTTP is adopted to send data to the second-level cache, and a data transmission mode is selected reasonably by combining the detection result.
The data transmission method provided by the embodiment of the invention obtains a detection result by detecting the link quality between the first-level cache and the second-level cache or detecting the bandwidth utilization rate of the second-level cache; when the detection result is that the link quality between the primary cache and the secondary cache is lower than a first threshold, or the bandwidth utilization rate of the secondary cache is lower than a second threshold, the data in the primary cache is sent by using the UDP based on the ARQ, and when the link quality of the primary cache side and the secondary cache side of the CDN is poor or the bandwidth utilization rate of the secondary cache side is low, the data in the primary cache is sent by using the UDP based on the ARQ, so that the data transmission is effectively accelerated, and the bandwidth utilization efficiency is improved.
Based on the same inventive concept of the data transmission method, the embodiment of the present invention further provides a data transmission device, and the implementation of the data transmission device may refer to the related description of the data transmission method, and repeated parts are not described again.
Fig. 3 is a schematic structural diagram of a data transmitting apparatus according to a first embodiment of the present invention, and as shown in fig. 3, the data transmitting apparatus may include:
a detection module 301, configured to detect link quality between a first-level cache and a second-level cache or detect a bandwidth utilization rate of the second-level cache, to obtain a detection result; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN;
a first sending module 302, configured to send data in the primary cache by using UDP based on ARQ when the detection result indicates that the link quality between the primary cache and the secondary cache is lower than a first threshold, or the bandwidth utilization of the secondary cache is lower than a second threshold; the first-level cache is used for caching data accessed by a user.
In a specific implementation, the data sending apparatus may correspond to a CDN primary cache side, and may be improved on the basis of an existing CDN device, for example, a detection module and a processing function module that controls UDP based on ARQ are added to the existing CDN primary cache side, the detection module detects link quality between the primary cache and the secondary cache or detects bandwidth utilization of the secondary cache, and the first sending module sends data in the primary cache based on a detection result, specifically: when the detection result is that the link quality between the first-level cache and the second-level cache is lower than a first threshold, or the bandwidth utilization rate of the second-level cache is lower than a second threshold, the first sending module sends the data in the first-level cache by adopting UDP based on ARQ; otherwise, when the detection result indicates that the link quality between the first-level cache and the second-level cache is not lower than a first threshold and the bandwidth utilization rate of the second-level cache is not lower than a second threshold, the first sending module sends the data in the first-level cache by using HTTP.
The data sending device provided in the embodiment of the present invention includes a detection module capable of detecting the link quality between the primary cache and the secondary cache or detecting the bandwidth utilization rate of the secondary cache, and when the detection result fed back by the detection module is that the link quality between the primary cache and the secondary cache is lower than a first threshold or the bandwidth utilization rate of the secondary cache is lower than a second threshold, the first sending module sends the data in the primary cache by using UDP based on ARQ, and when the link quality of the primary cache and the secondary cache is not good or the bandwidth utilization rate of the secondary cache side is low, the data in the primary cache is sent by using UDP based on ARQ, so that the data transmission is effectively accelerated, and the bandwidth utilization efficiency is improved.
In an implementation, the data transmitting apparatus may further include: a first receiving module 303, configured to receive a connection request from the second level cache before detecting a link quality between the first level cache and the second level cache or detecting a bandwidth utilization of the second level cache.
Has the advantages that: the detection module can perform detection after the first receiving module receives the connection request from the second-level cache, and perform reasonable detection as required.
In an implementation, the first sending module may be further configured to send data to the secondary cache using HTTP when the detection result indicates that the link quality between the primary cache and the secondary cache is not lower than a first threshold and the bandwidth utilization of the secondary cache is not lower than a second threshold.
Has the advantages that: when the detection result is that the link quality between the first-level cache and the second-level cache is not lower than a first threshold value and the bandwidth utilization rate of the second-level cache is not lower than a second threshold value, HTTP is adopted to send data to the second-level cache, and a data transmission mode is selected reasonably by combining the detection result.
In an implementation, the first sending module may be specifically configured to convert the data in the primary cache from a hypertext transfer protocol HTTP format to a UDP format, and then send the data in the primary cache by using a UDP based on ARQ.
In a specific implementation, because the output of the existing primary cache is basically in the HTTP format, the first sending module needs to convert the HTTP formatted data from the primary cache into the UDP format before sending the data, and then sends the data by using the UDP based on the ARQ.
Based on the data sending method and device, which belong to a general inventive concept, the embodiment of the invention also provides a data receiving method and device, and related repeated parts are not described again.
Fig. 4 is a schematic flow chart of a data receiving method according to an embodiment of the present invention, and as shown in fig. 4, the data receiving method may include the following steps:
The data receiving method provided by the embodiment of the invention can be implemented on equipment on a CDN (content distribution network) secondary cache side, when the link quality between the primary cache and the secondary cache is lower than a first threshold value, or the bandwidth utilization rate of the secondary cache is lower than a second threshold value, the data from the primary cache sent by adopting UDP (user datagram protocol) based on ARQ control is received, the data is converted from the UDP format to the HTTP format, the data in the HTTP format is cached to the secondary cache, and when the link quality of the CDN primary cache side and the secondary cache side is poor or the bandwidth utilization rate of the secondary cache side is lower, the data in the primary cache sent by adopting UDP based on ARQ is received and cached, so that the data transmission is effectively accelerated, and the bandwidth utilization efficiency is improved.
In an implementation, the data receiving method may further include:
sending a connection request to the primary cache before receiving data from the primary cache sent using UDP based on ARQ control.
Has the advantages that: before receiving the data from the first-level cache, the connection request is sent, so that the CDN first-level cache side can conveniently perform reasonable detection as required.
In implementation, when the link quality between the primary cache and the secondary cache is not lower than a first threshold and the bandwidth utilization of the secondary cache is not lower than a second threshold, the HTTP formatted data from the primary cache may be cached to the secondary cache.
Has the advantages that: when the link quality between the first-level cache and the second-level cache is not lower than a first threshold and the bandwidth utilization rate of the second-level cache is not lower than a second threshold, the data in the HTTP format from the first-level cache is cached to the second-level cache, and the data transmission mode is reasonably selected by combining the actual data transmission condition.
Based on the same inventive concept of the above data receiving method, the embodiment of the present invention further provides a data receiving apparatus, and the implementation of the data receiving apparatus may refer to the related description of the data receiving method, and repeated parts are not described again.
Fig. 5 is a schematic structural diagram of a data receiving apparatus according to an embodiment of the present invention, and as shown in fig. 5, the data receiving apparatus may include:
a receiving conversion module 501, configured to receive data from a primary cache sent by using UDP based on ARQ control when a link quality between the primary cache and a secondary cache is lower than a first threshold or a bandwidth utilization of the secondary cache is lower than a second threshold, and convert the data from a UDP format to an HTTP format; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second level cache is a cache in the CDN for centralizing back to the source.
The caching module 502 is configured to cache the HTTP formatted data in the second level cache.
In a specific implementation, the data receiving apparatus corresponds to the data sending apparatus, an ARQ processing function module may be added at the CDN secondary cache side, and the ARQ processing function module may receive data in the UDP format based on ARQ control from the primary cache, convert the data into the HTTP format, and cache the converted data in the secondary cache.
The data receiving device provided by the embodiment of the invention comprises a receiving and converting module which receives the data sent by the first-level cache by using UDP based on ARQ control and converts the data from the UDP format to the HTTP format when the link quality between the first-level cache and the second-level cache is lower than a first threshold or the bandwidth utilization rate of the second-level cache is lower than a second threshold, so that the data sent by the first-level cache side of the CDN by using UDP based on ARQ can be received, and the data in the first-level cache sent by the first-level cache side of the CDN by using UDP based on ARQ can be received and cached when the link quality of the first-level cache and the second-level cache is poor or the bandwidth utilization rate of the second-level cache is lower in cooperation with the first-level cache side of the CDN, the data.
In an implementation, the data receiving apparatus may further include:
a second sending module 503, configured to send a connection request to the primary cache before receiving data from the primary cache sent in UDP based on ARQ control.
Has the advantages that: before receiving data from the CDN first-level cache side, the second sending module sends a connection request, so that the CDN first-level cache side can conveniently perform reasonable detection as required.
In an implementation, the caching module may be further configured to cache, to the second level cache, the HTTP-formatted data from the first level cache when a link quality between the first level cache and the second level cache is not lower than a first threshold and a bandwidth utilization of the second level cache is not lower than a second threshold.
In specific implementation, when the link quality between the first-level cache and the second-level cache is not lower than a first threshold and the bandwidth utilization rate of the second-level cache is not lower than a second threshold, the HTTP format data from the first-level cache side may be directly cached in the second-level cache along with a data receiving manner in the prior art.
Has the advantages that: when the link quality between the first-level cache and the second-level cache is not lower than a first threshold and the bandwidth utilization rate of the second-level cache is not lower than a second threshold, the data in the HTTP format from the first-level cache is cached, and the data transmission mode is reasonably selected by combining the actual data transmission condition.
Based on the same inventive concept of the data transmission method, the embodiment of the invention provides a server (I).
Fig. 6 is a schematic structural diagram of a server (i) according to an embodiment of the present invention. As shown in fig. 6, the server (one) may include:
a memory 620, one or more processors 610, a communication interface 630, a bus and one or more modules, said processor, said memory and said communication interface being connected via said bus and performing communication with each other; the one or more modules are stored in the memory and configured to be executed by the one or more processors, the one or more modules including instructions for performing the steps of the data transmission method described above.
Based on the same inventive concept of the above data receiving method, an embodiment of the present invention further provides a computer program product, where the computer program product includes a computer-readable storage medium and a computer program mechanism embedded therein, and the computer program mechanism includes instructions for executing each step in the above data sending method.
Based on the same inventive concept of the data receiving method, the embodiment of the invention provides a server (II).
Fig. 7 is a schematic structural diagram of a server (ii) according to an embodiment of the present invention, and as shown in fig. 7, the server (ii) may include:
a memory 720, one or more processors 720, a communication interface 730, a bus and one or more modules, wherein the processor, the memory and the communication interface are connected through the bus and complete communication with each other; the one or more modules are stored in the memory and configured to be executed by the one or more processors, the one or more modules including instructions for performing the steps of the data reception method.
Based on the same inventive concept of the above data receiving method, an embodiment of the present invention further provides a computer program product, which includes a computer-readable storage medium and a computer program mechanism embedded therein, where the computer program mechanism includes instructions for executing each step in the data receiving method.
To facilitate the practice of the invention, the following examples are given.
In the second embodiment of the present invention, a CDN actual application scenario is taken as an example, and the first embodiment of the present invention is described in detail.
Example two
An agent is deployed on equipment on the CDN first-level cache side, the agent not only has the function of an HTTP proxy server, but also supports an ARQ protocol, has a detection function, can detect the link quality between the first-level cache and the second-level cache, and obtains the bandwidth use condition of the second-level cache from the second-level cache through an SNMP protocol. Specifically, the proxy includes an HTTP proxy function module, an ARQ-enabled function module, a probing module, and a configuration module, and in a specific implementation, the probing module and the ARQ-enabled function module may be added to an existing device proxy on the primary cache side. The functional module supporting ARQ can realize a fast and reliable UDP transmission tunnel according to KCP protocol; the detection module is responsible for detecting the link quality between the first-level cache and the second-level cache of the CDN, and the detection mode may adopt an ICMP (Internet Control Message Protocol) Protocol and may detect the bandwidth utilization rate of the second-level cache through, for example, an SNMP Protocol; the configuration module is responsible for configuring variable information, and is not specifically limited herein.
Fig. 8 is a schematic diagram of a proxy structure of a device on the CDN level-one cache side according to an embodiment of the present invention, where a relationship among the HTTP proxy function module, the ARQ supporting function module 1, the probe module, and the configuration module is shown in fig. 8.
Fig. 9 is a schematic processing flow diagram of an agent at a CDN first-level cache side in the embodiment of the present invention, as shown in fig. 9, specifically as follows:
and step 907, ending.
An application supporting an ARQ protocol is deployed on a device on the CDN secondary cache side to cooperate with a device on the primary cache side to perform data reception processing.
Fig. 10 is a schematic flow chart of sending data by a device on the CDN first-level cache side after the solution provided by the embodiment of the present invention is adopted, and as shown in fig. 10, the following specifically:
step 1001, the probing module of the device agent on the first-level cache side probes periodically or aperiodically (for example, after receiving a connection request of the device on the second-level cache side), where the specific probing content includes: link quality between devices at the first and second-level cache sides of the CDN or bandwidth utilization rate of the devices at the second-level cache side is detected;
And 1004, if the HTTP transmission is determined to be directly adopted, directly sending the data requested by the equipment at the side of the second-level cache and located in the first-level cache to the second-level cache in an HTTP format, and returning the response information from the second-level cache to the first-level cache.
The scheme provided by the embodiment of the invention determines whether ARQ transmission or HTTP direct transmission is specifically adopted according to the detected link quality between the devices at the first-level cache side and the second-level cache side of the CDN or the bandwidth utilization rate of the devices at the second-level cache side based on the improvement of the devices at the first-level cache side and the second-level cache side of the CDN, can flexibly select a data transmission mode reasonably according to the link quality of the first-level cache and the second-level cache of the CDN or the bandwidth utilization rate of the second-level cache, effectively accelerates data transmission and provides bandwidth utilization efficiency.
In the above embodiments, all the modules may be implemented by using existing functional components. For example, the processing module may use an existing data processing component, and at least, a positioning server used in the existing positioning technology is equipped to implement the functional component; the receiving module is a component which any equipment with a signal transmission function has; meanwhile, A, n parameter calculation, intensity adjustment and the like performed by the processing module are all the existing technical means, and can be realized by corresponding design and development by a person skilled in the art.
For convenience of description, each part of the above-described apparatus is separately described as being functionally divided into various modules or units. Of course, the functionality of the various modules or units may be implemented in the same one or more pieces of software or hardware in practicing the invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
Claims (14)
1. A data transmission method, comprising the steps of:
detecting the link quality between a first-level cache and a second-level cache or detecting the bandwidth utilization rate of the second-level cache to obtain a detection result; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN; the first-level cache and the second-level cache are caches arranged on an edge server;
when the detection result is that the link quality between the first-level cache and the second-level cache is lower than a first threshold value, or the bandwidth utilization rate of the second-level cache is lower than a second threshold value, transmitting data in the first-level cache by adopting UDP based on ARQ;
further comprising:
receiving a connection request from the secondary cache prior to detecting a link quality between the primary cache and the secondary cache or detecting a bandwidth utilization of the secondary cache.
2. The method of claim 1, wherein sending the data in the primary buffer using UDP based on ARQ control comprises:
and after converting the data in the primary cache from the HTTP format to the UDP format, transmitting the data in the primary cache by adopting the UDP based on the ARQ.
3. The method of claim 1, further comprising:
and when the detection result shows that the link quality between the first-level cache and the second-level cache is not lower than a first threshold value and the bandwidth utilization rate of the second-level cache is not lower than a second threshold value, adopting HTTP to send data to the second-level cache.
4. A data transmission apparatus, comprising:
the detection module is used for detecting the link quality between the first-level cache and the second-level cache or detecting the bandwidth utilization rate of the second-level cache to obtain a detection result; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN; the first-level cache and the second-level cache are caches arranged on an edge server;
a first sending module, configured to send data in the primary cache by using UDP based on ARQ when the detection result indicates that the link quality between the primary cache and the secondary cache is lower than a first threshold, or that the bandwidth utilization of the secondary cache is lower than a second threshold; the first-level cache is used for caching data accessed by a user;
further comprising:
a first receiving module, configured to receive a connection request from the second-level cache before detecting a link quality between the first-level cache and the second-level cache or detecting a bandwidth utilization of the second-level cache.
5. The apparatus as claimed in claim 4, wherein the first sending module is specifically configured to send the data in the primary buffer in UDP based on ARQ after converting the data from HTTP format to UDP format.
6. The apparatus of claim 4, wherein the first sending module is further configured to send data to the secondary cache using HTTP when the probing result is that a link quality between the primary cache and the secondary cache is not below a first threshold and a bandwidth utilization of the secondary cache is not below a second threshold.
7. A data receiving method, comprising the steps of:
when the link quality between a primary cache and a secondary cache is lower than a first threshold value or the bandwidth utilization rate of the secondary cache is lower than a second threshold value, receiving data from the primary cache sent by adopting UDP based on ARQ control, and converting the data from the UDP format to the HTTP format; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN; the first-level cache and the second-level cache are caches arranged on an edge server;
caching the data in the HTTP format to a second-level cache;
further comprising:
sending a connection request to the primary cache before receiving data from the primary cache sent using UDP based on ARQ control.
8. The method of claim 7, wherein when a link quality between the level one cache and the level two cache is not below a first threshold and a bandwidth utilization of the level two cache is not below a second threshold, caching HTTP formatted data from the level one cache to the level two cache.
9. A data receiving device, comprising:
the receiving and converting module is used for receiving data from the primary cache sent by adopting UDP based on ARQ control when the link quality between the primary cache and the secondary cache is lower than a first threshold or the bandwidth utilization rate of the secondary cache is lower than a second threshold, and converting the data from the UDP format to the HTTP format; the first-level cache is a cache distributed close to the user in the content delivery network CDN and used for caching data accessed by the user; the second-level cache is a cache used for centralized returning to a source in the CDN; the first-level cache and the second-level cache are caches arranged on an edge server;
the cache module is used for caching the data in the HTTP format to a second-level cache;
further comprising:
a second sending module, configured to send a connection request to the primary cache before receiving data from the primary cache sent by using UDP based on ARQ control.
10. The apparatus of claim 9, wherein the caching module is further to cache HTTP formatted data from the level one cache to a level two cache when a link quality between the level one cache and the level two cache is not below a first threshold and a bandwidth utilization of the level two cache is not below a second threshold.
11. A server, characterized in that the server comprises:
the system comprises a memory, one or more processors, a communication interface, a bus and one or more modules, wherein the processor, the memory and the communication interface are connected through the bus and complete mutual communication; the one or more modules stored in the memory and configured to be executed by the one or more processors, the one or more modules comprising instructions for performing the steps of the method of any of claims 1-3.
12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 3.
13. A server, characterized in that the server comprises:
the system comprises a memory, one or more processors, a communication interface, a bus and one or more modules, wherein the processor, the memory and the communication interface are connected through the bus and complete mutual communication; the one or more modules stored in the memory and configured to be executed by the one or more processors, the one or more modules comprising instructions for performing the steps of the method of any of claims 7-8.
14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 7-8.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2016/108430 WO2018098820A1 (en) | 2016-12-02 | 2016-12-02 | Method and device for sending and receiving data, server, and computer program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107624231A CN107624231A (en) | 2018-01-23 |
| CN107624231B true CN107624231B (en) | 2021-06-04 |
Family
ID=61087325
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201680002898.3A Active CN107624231B (en) | 2016-12-02 | 2016-12-02 | Method, device, server and computer program product for transmitting and receiving data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107624231B (en) |
| WO (1) | WO2018098820A1 (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111093089B (en) * | 2018-10-23 | 2022-04-01 | 中国电信股份有限公司 | Method for managing video, edge cache scheduling center and communication system |
| CN109194772A (en) * | 2018-11-13 | 2019-01-11 | 杭州领智云画科技有限公司 | CDN fringe node returns source method and system |
| CN111106950B (en) * | 2019-11-25 | 2022-08-16 | 深圳市网心科技有限公司 | Reverse proxy service quality monitoring method, electronic device, system and medium |
| CN111103834A (en) * | 2019-12-25 | 2020-05-05 | 北京航天云路有限公司 | Method for reducing remote control delay of industrial control equipment based on KCP protocol |
| CN113301071B (en) * | 2020-04-09 | 2022-08-12 | 阿里巴巴集团控股有限公司 | Network source returning method, device and equipment |
| CN111479137B (en) * | 2020-04-16 | 2022-02-18 | 广州酷狗计算机科技有限公司 | Method and device for providing line address, server and storage medium |
| CN111628911B (en) * | 2020-05-27 | 2021-10-15 | 北京金山云网络技术有限公司 | CDN back source abnormity testing method and device and terminal equipment |
| CN112015674B (en) * | 2020-08-27 | 2023-05-30 | 平安科技（深圳）有限公司 | Multi-level-based cache resource access method and device and computer equipment |
| CN112099949B (en) * | 2020-09-11 | 2023-09-05 | 北京奇艺世纪科技有限公司 | Task distribution control method and device, electronic equipment and storage medium |
| CN113890896A (en) * | 2021-09-24 | 2022-01-04 | 中移（杭州）信息技术有限公司 | Network access method, communication device, and computer-readable storage medium |
| CN114338686A (en) * | 2021-12-21 | 2022-04-12 | 江苏云工场信息技术有限公司 | Source returning method and device of CDN node server |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103078831A (en) * | 2011-10-26 | 2013-05-01 | 新太科技股份有限公司 | Self-adaption method on basis of TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) transmission supported by streaming media |
| CN104967685A (en) * | 2015-06-25 | 2015-10-07 | 广东德诚网络科技有限公司 | Streaming media multi-level cache network acceleration method based on Flash P2P |
| GB2533434A (en) * | 2014-12-16 | 2016-06-22 | Cisco Tech Inc | Networking based redirect for CDN scale-down |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143204A (en) * | 2010-11-26 | 2011-08-03 | 华为技术有限公司 | Method, device and system for realizing reorientation of hypertext transfer protocol in content delivery network |
| CN102137006B (en) * | 2010-12-31 | 2013-12-18 | 华为技术有限公司 | Data transmission method and equipment in CDN (Content Delivery Network) |
| US8874750B2 (en) * | 2011-03-29 | 2014-10-28 | Mobitv, Inc. | Location based access control for content delivery network resources |
-
2016
- 2016-12-02 WO PCT/CN2016/108430 patent/WO2018098820A1/en active Application Filing
- 2016-12-02 CN CN201680002898.3A patent/CN107624231B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103078831A (en) * | 2011-10-26 | 2013-05-01 | 新太科技股份有限公司 | Self-adaption method on basis of TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) transmission supported by streaming media |
| GB2533434A (en) * | 2014-12-16 | 2016-06-22 | Cisco Tech Inc | Networking based redirect for CDN scale-down |
| CN104967685A (en) * | 2015-06-25 | 2015-10-07 | 广东德诚网络科技有限公司 | Streaming media multi-level cache network acceleration method based on Flash P2P |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107624231A (en) | 2018-01-23 |
| WO2018098820A1 (en) | 2018-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107624231B (en) | Method, device, server and computer program product for transmitting and receiving data | |
| BE1023180B9 (en) | HARQ-ACK SUPPORT FOR UNINTENDED DOWNLINK PACKETS | |
| CN109412946B (en) | Method, device, server and readable storage medium for determining back source path | |
| US9253015B2 (en) | Transparent proxy architecture for multi-path data connections | |
| Xu et al. | CMT-NC: improving the concurrent multipath transfer performance using network coding in wireless networks | |
| CN103401946B (en) | HTTP uploads accelerated method and system | |
| CN111769968A (en) | Method and network equipment for processing message in hybrid access network | |
| CN108200158B (en) | Request Transmission system, method, apparatus and storage medium | |
| GB2519491A (en) | Method and system for increasing data flow transmission | |
| CN104683259A (en) | TCP congestion control method and device | |
| CN109067796A (en) | A kind of data transmission method and device | |
| CN104967613A (en) | Data transmission system and method under mobile network environments | |
| CN105392157B (en) | A kind of method of adjustment of congestion window, relevant apparatus and system | |
| CN1774894A (en) | Method for implementing a connection-oriented data transfer protocol in wireless transmission networks | |
| CN105763297A (en) | Cloud computing system-based remote data optimized transmission method and device | |
| WO2014012724A1 (en) | Content optimization based on real time network dynamics | |
| CN105792359B (en) | Resource allocation methods, data mapping method and device | |
| WO2017107148A1 (en) | Method of transmitting data and network equipment | |
| CN113301000A (en) | Data transmission method, device, medium and equipment | |
| EP3038312A1 (en) | Data transmission method, user equipment and proxy equipment | |
| KR20190011898A (en) | Terminal, base station, and transmitting method thereof in a wireless communication system | |
| CN106161561A (en) | A kind of realize the method for Differentiated Services, device and load-balanced server | |
| CN105897452A (en) | Data retransmission method and device | |
| Lindquister et al. | Proxy pair optimizations for increased service reliability in DIL networks | |
| CN109217978A (en) | The methods, devices and systems of data transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210325 Address after: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Applicant after: Dalu Robot Co.,Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: Shenzhen Qianhaida Yunyun Intelligent Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 201111 Building 8, No. 207, Zhongqing Road, Minhang District, Shanghai Patentee after: Dayu robot Co.,Ltd. Address before: 201111 2nd floor, building 2, no.1508, Kunyang Road, Minhang District, Shanghai Patentee before: Dalu Robot Co.,Ltd. |
|
| CP03 | Change of name, title or address |