CN112256495A - Data transmission method and device, computer equipment and storage medium - Google Patents
Data transmission method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112256495A CN112256495A CN202011125164.9A CN202011125164A CN112256495A CN 112256495 A CN112256495 A CN 112256495A CN 202011125164 A CN202011125164 A CN 202011125164A CN 112256495 A CN112256495 A CN 112256495A
- Authority
- CN
- China
- Prior art keywords
- access resource
- network
- resource
- user
- source station
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000003068 static effect Effects 0.000 claims abstract description 45
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 7
- 230000001133 acceleration Effects 0.000 abstract description 18
- 230000000694 effects Effects 0.000 abstract description 12
- 238000010276 construction Methods 0.000 abstract description 2
- 230000007613 environmental effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2002—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where interconnections or communication control functionality are redundant
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
- G06F11/2023—Failover techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
- H04L43/0847—Transmission error
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0852—Delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Environmental & Geological Engineering (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a data transmission method, a data transmission device, computer equipment and a storage medium.A first network node is determined according to an access resource request sent to a source station by a user, an access resource type is determined according to the access resource request, and when the access resource is a static resource, the corresponding access resource in the cache of the first network node is sent to the user; and when the access resource is a dynamic resource, calculating a plurality of paths between the first network node and the source station, and selecting a corresponding path according to the network condition to send the corresponding access resource in the source station to the user, thereby ensuring the optimal acceleration effect and the stable acceleration effect. The method and the system can be applied to smart government affairs/smart city management/smart community/smart security/smart logistics/smart medical treatment/smart education/smart environmental protection/smart traffic scenes, and accordingly construction of smart cities is promoted.
Description
Technical Field
The present invention relates to the field of content distribution networks, and in particular, to a data transmission method, apparatus, computer device, and storage medium.
Background
A CDN (Content Delivery Network) is an intelligent virtual Network constructed on the basis of an existing Network, and by means of edge servers deployed in various places, users can obtain required Content nearby through functional modules of load balancing, Content Delivery, scheduling, and the like of a central platform, so that Network congestion is reduced, and the access response speed and hit rate of the users are improved.
Most of the CDNs perform static file delivery through cache, and with the rise of e-commerce, finance, social contact and cloud games, the CDNs are also used for dynamic acceleration, and have the characteristics of obtaining the latest content without cache in real time, but the acceleration method in the existing market also has the problems of unobvious acceleration effect, unstable acceleration effect and the like.
Disclosure of Invention
In view of this, the present invention provides a data transmission method, an apparatus, a computer device and a storage medium, which are used to solve the problems of unobvious network acceleration effect and unstable acceleration effect in the data transmission process in the prior art.
First, in order to achieve the above object, the present invention provides a data transmission method applied in a content distribution network, where the method includes:
determining a first network node according to a resource access request sent by a user to a source station;
determining an access resource type according to the access resource request, wherein the access resource type comprises static resources and dynamic resources;
when the access resource is a static resource, sending the corresponding access resource in the first network node cache to the user;
and when the access resource is a dynamic resource, calculating a plurality of paths between the first network node and the source station, and selecting a corresponding path according to the network condition to send the corresponding access resource in the source station to the user.
Preferably, the content distribution network includes a plurality of edge nodes and at least one parent layer node, and the determining the first network node according to the resource access request sent by the user to the source station includes:
receiving a domain name corresponding to the source station output by a user;
and resolving the domain name to point the access resource request to an edge node corresponding to the first network node.
Preferably, when the access resource is a static resource, sending the access resource corresponding to the network node cache closest to the user includes:
when the access resource is a static resource, sending the access resource corresponding to the edge node cache corresponding to the first network node to the user;
when the access resource corresponding to the cache of the edge node is not cached, the access resource corresponding to the cache of the parent layer node of the edge node is sent to the user;
and when the parent layer node does not cache the corresponding access resource, acquiring the access resource from the source station through the content distribution network.
Preferably, when the access resource is a dynamic resource, calculating a plurality of paths between the network node closest to the user and the source station, and selecting a corresponding path according to a network condition to send the corresponding access resource in the source station to the user includes:
when the access resource is a dynamic resource, calculating a plurality of paths between the edge node and the source station according to the current network delay, packet loss rate or jitter value in the content distribution network;
selecting the network delay, the packet loss rate or the corresponding path with the jitter value meeting the first condition, and sending the corresponding access resource in the source station to the user;
and when the network condition of the path corresponding to the first condition changes, selecting other paths to send the corresponding access resource in the source station to the user.
Preferably, when the access resource is a dynamic resource, calculating a plurality of paths between the edge node and the source station according to a current network delay, a packet loss rate, or a jitter value in the content distribution network includes:
network detection is mutually executed between the edge node and the parent layer node to obtain network delay of data transmission between the nodes;
and determining a first path, a second path and a third path between the edge node and the source station according to the total network delay size sequence of different paths between the edge node and the source station.
Preferably, the selecting the corresponding path where the network delay, the packet loss rate, or the jitter value satisfies the first condition, and sending the corresponding access resource in the source station to the user includes:
and selecting a path with the minimum total network delay between the edge node and the source station as a first path so as to send the corresponding access resource in the source station to the user.
Preferably, when the network condition of the path corresponding to the first condition changes, selecting another path to send the access resource corresponding to the source station to the user includes:
when the first path has network failure, sending the corresponding access resource in the source station to the user through the second path;
and when the second path has network failure, sending the corresponding access resource in the source station to the user through a third path.
In order to achieve the above object, the present invention further provides a data transmission device, including:
the network access module is used for determining a first network node according to a resource access request sent to a source station by a user;
the resource determining module is used for determining the type of the access resource according to the access resource request, wherein the type of the access resource comprises a static resource and a dynamic resource;
a static processing module, configured to send the access resource corresponding to the cache of the first network node to the user when the access resource is a static resource;
and the dynamic processing module is used for calculating a plurality of paths between the first network node and the source station when the access resource is a dynamic resource, and selecting a corresponding path according to the network condition to send the corresponding access resource in the source station to the user.
To achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
To achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above method.
Compared with the prior art, the data transmission method, the data transmission device, the computer equipment and the storage medium in the embodiment of the invention ensure the optimal acceleration effect through multi-mode link quality detection, and have a multi-layer degradation or fault-tolerant mechanism to quickly switch when the delay of a front link is judged to be larger or a fault occurs so as to ensure the stable acceleration effect.
Drawings
FIG. 1 is a schematic diagram of an application environment of an embodiment of the present invention;
FIG. 2 is a schematic diagram of a content distribution network according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a data transmission method according to a first embodiment of the invention;
FIG. 4 is a schematic flow chart illustrating the process of determining the network node closest to the user in FIG. 3;
FIG. 5 is a schematic flow chart illustrating the accessing of the resource in FIG. 3 as a static resource;
FIG. 6 is a schematic diagram of the architecture of a single network node in FIG. 2;
FIG. 7 is a schematic flow chart illustrating the accessing of the resource in FIG. 3 when the resource is a dynamic resource;
fig. 8 is a schematic flowchart of the process of fig. 6 and 7 for sending the access resource corresponding to the source station to the user according to different priority paths;
FIG. 9 is a diagram illustrating a scenario of computing nodes with different priority paths according to an embodiment of the present invention;
FIG. 10 is a block diagram of a second embodiment of a data transmission apparatus according to the present invention;
fig. 11 is a schematic hardware configuration diagram of a third embodiment of the computer apparatus according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
Referring to fig. 1, a schematic diagram of an implementation environment of the embodiment of the invention is shown. The implementation environment includes: a user terminal 10 and a server 12.
The user terminal 10 is an electronic device with network access function, and the device may be a smart phone, a tablet computer, a personal computer, or the like.
The user terminal 10 is installed with a program 11 that can access the server terminal 12, and after the program 11 logs in to access the server terminal 12 through an account and a password, the user can perform specific operations and inputs on the server terminal 12.
The server 12 is a server, a server cluster formed by a plurality of servers, or a cloud computing center. The server 12 stores a program 13 corresponding to a source station, the program 13 corresponding to the source station includes a front end module and a back end module, the front end module and the back end module can be called by an interface, and a user can perform specific operation and input on the program 13 corresponding to the source station after the program 11 logs in or accesses the program 13 corresponding to the source station of the server 12 through an account and a password.
In the present embodiment, the server 12 stores a plurality of databases and data tables. Different databases may obtain the data in the databases through the triggering instruction of the user 10.
In this embodiment, the database and data tables include, but are not limited to, a user attribute table and a product attribute table.
In other embodiments, the database may also be stored in different servers in the server 12 cluster, or in different network nodes connected via a network, or locally in the user segment 10.
Referring to fig. 2, the user terminal 10 and the service terminal 12 are connected through a content distribution network, which may include n network nodes, where the n network nodes include a plurality of edge nodes and at least one parent layer node.
Specifically, in this embodiment, the CDN network includes 8 edge nodes e1., e8, and two parent-level nodes e (n) and e (n-1), where the parent-level node e (n) is a parent-level node of the nodes e1, e2, e3, e4, and e8, the parent-level node e (n-1) is a parent-level node of the nodes e5, e6, and e7, and the parent-level nodes e (n) and e (n-1) are interconnected through the network.
The content distribution network is accessed to m source station contents, each source station is distributed at different node positions of the content distribution network, and the nearby edge nodes are accessed according to physical positions.
The method and the system can be applied to smart government affairs/smart city management/smart community/smart security/smart logistics/smart medical treatment/smart education/smart environmental protection/smart traffic scenes, and accordingly construction of smart cities is promoted.
Referring to fig. 3, in the data transmission method of this embodiment, the optimal acceleration effect is ensured by multi-mode link quality detection, and when it is determined that the forward link has a larger delay or a fault, a multi-layer degradation or fault-tolerant mechanism is provided to enable fast switching to ensure stable acceleration effect.
Specifically, the data transmission method of the present embodiment is applied to a content distribution network, and includes the following steps:
step S100: determining a first network node according to a resource access request sent by a user to a source station;
the content distribution network is an intelligent virtual network established on the basis of the existing network, a user can obtain required content nearby by means of edge nodes deployed in various places through functional modules such as load balancing, content distribution and scheduling, network congestion is reduced, user access response speed and hit rate are improved, and the edge node with the network physical geographic position closest to the user is used as a first network node by analyzing the network physical geographic position of each edge node in the CDN.
For example, by analyzing the IP address of the user, when the physical geographic location of the user is Shenzhen is monitored, the CDN network includes an edge node such as south China, north China, east China, and the like, and then the south China node, which is the nearest edge node to Shenzhen, is used as the first network node.
Specifically, referring to fig. 4, step S100 includes:
step S110: receiving a domain name corresponding to the source station output by a user;
step S120: and resolving the domain name to point the access resource request to an edge node corresponding to the first network node.
Specifically, a user obtains an IP address of a nearby CDN edge node through DNS (Domain Name System) resolution and sends an access request to the node.
In a content distribution network, a complete domain name resolution process includes: a user sends a request- > a local DNS analysis server; local resolution server- > top level DNS resolution server; the top layer resolving domain name returns a server address- > a local DNS resolving server; a local DNS resolution server- > a server of a domain name registrar- > local; local DNS- > IP (CDN domain name resolution server) returned by domain name registrars; the CDN directs the request to a CDN node server closest to the user;
referring to fig. 2, for example, if a user needs to obtain an access resource in network a, the user first inputs a domain name corresponding to website a. Com, the domain name of network a is obtained through DNS resolution and corresponds to the server ip address of the source station S1 storing the corresponding network a, where the source station S1 is stored in the server and accessed into the CDN network, the source station S1 corresponds to the node e8 in the CDN network, and the user is closest to the node e4 in the CDN network.
The user obtains the IP of the node e4 through DNS resolution and initiates an HTTP (HyperText Transfer Protocol) network request or an HTTP (HyperText Transfer Protocol over secure session Layer) network request to the node e 4.
Step S200: determining an access resource type according to the access resource request, wherein the access resource type comprises static resources and dynamic resources;
the edge node intelligently determines whether the accessed resource is static or dynamic content, e.g., node e4 determines whether the resource is static or dynamic content by reading the domain name configuration and self learning.
Static network is web page without data interaction, does not support data transmission, page suffix is generally html, and the static network must download and upload the file of the server once again every time a page is added, deleted and modified. Once the web page content is published to the web server, whether accessed by a user or not, the content of each static web page is stored on the web server, that is, the static web pages are files stored on the server, and each web page is a separate file
The dynamic network supports a network of data interaction, programs are nested in pages of the dynamic network, the dynamic network has the same frames, content and form separation is carried out on information pages which are updated quickly, and the information content is stored in a database of the network in a recorded form so as to be convenient for calling at various positions of the network. Unlike static networks, a page of a dynamic network may not correspond to a html document on the server, and the web page frame houses contents in records in many databases. That is, the suffix of the web page URL is not a common form of static web pages such as html, shtml, xml, but a suffix in the form of asp, jsp, php, perl, cgi, etc.
Specifically, whether the request content is static content or dynamic content is judged through a resource judgment module in the node, the resource judgment module is specifically an httpserver access module, a domain Name configuration and a self-learning history library are read according to a request SNI (Server Name Indication), whether the dynamic content or the static content, such as the static content, moves through a normal rear-end cache path, and if the dynamic content is submitted to a routing module, is judged according to the request resource.
Step S300: when the access resource is a static resource, sending the corresponding access resource in the first network node cache to the user;
specifically, referring to fig. 5, step S300 includes:
step S310: when the access resource is a static resource, sending the access resource corresponding to the first network node in the edge node cache to the user;
step S320: when the access resource corresponding to the cache of the edge node is not cached, the access resource corresponding to the cache of the parent layer node of the edge node is sent to the user;
step S330: and when the parent layer node does not cache the corresponding access resource, acquiring the access resource from the source station through the content distribution network.
Specifically, when the access resource is a static resource, the corresponding access resource in the network node cache closest to the user is sent to the user; if the static content is judged, the local cache is accessed first, and if the static content is not hit, the parent layer and the source station are accessed to finally respond to the user.
For example, when the content of the source station S1 requested to be obtained by the user is shot content, such as an html file, first, it is queried in the local cache of the node e4 whether there is history access to the html file cached locally at the node e4, if it is queried that there is no html file locally at the node e4, it is queried whether there is cached html file in the parent node e (n) of the node e4, and if it is queried that there is html file locally at the node e4, the html file is directly sent to the user.
And if the html file is cached in the query parent node e (n), directly sending the html file to the user. If the html file is not cached in the query parent node e (n), the html file corresponding to the source station S1 is finally sent to the user through the relay of the adjacent node e 8.
Step S400: and when the access resource is a dynamic resource, calculating a plurality of paths between the first network node and the source station, and selecting a corresponding path according to the network condition to send the corresponding access resource in the source station to the user.
Specifically, if the dynamic content is judged, the optimal source return path agent request is selected to the source station, and finally the optimal source return path agent request is returned to the user.
Referring to fig. 6, a single node architecture operates as follows:
the detectNode is a detection node deployed on the LVS and comprises an active mode and a passive mode, the active mode is to detect (including ping, tcping, httping and the like) other nodes (including edges, parent layers and source stations), the passive mode is to record the response history of normal user requests, and collect rtt and other data and report the data to the detectmgr.
The detectMgr is a detection center and is responsible for collecting data reported by the detectNode, summarizing and calculating factors such as time delay, packet loss rate, jitter and the like, and calculating optimal, suboptimal and secondary optimal paths (IP lists) through a single-source shortest path algorithm and issuing the optimal, suboptimal and secondary optimal paths (IP lists) to the redis.
The access is an httpserver access module, the domain name configuration and the self-learning history library are read according to the SNI request, whether the dynamic content or the static content exists is judged according to the request resource, if the static content runs a normal rear-end cache path, if the dynamic content is submitted to the route module
The Route module is an intelligent routing module, firstly judges that a Route parameter is obtained from HTTP Header, does not indicate that the node is an originating node, reads the optimal, suboptimal and next-to-best paths from the node to a source station to a redis, adds the optimal path to the HTTP Header, removes the first IP (namely the node) to save transmission content if the node is a relay node, and when the next node is connected and has a fault or large delay, quickly degrades (from optimal to suboptimal, suboptimal to secondary and optimized again to return to a source of a public network) routing to ensure faster delivery instead of waiting for overtime.
the transport module is a transmission module, it utilizes openness dynamic upstream technology to deliver http request to next node, the transmission module needs to make connection (pool) management, overtime processing, etc., and then to the last node, according to the heat, it will keep long connection with the source station to reduce the overhead of establishing connection, when the connection between the last node and the source station fails, it will not re-select the route but use the address of the next source station to directly return to the source in public network.
Referring to fig. 7, step S400 includes:
step S410, when the access resource is a dynamic resource, calculating a plurality of paths between the edge node and the source station according to the current network delay, packet loss rate or jitter value in the content distribution network;
specifically, in this embodiment, a plurality of network delay values, packet loss rate values, or jitter values of data transmitted by different edge nodes and parent nodes are calculated according to the network delay, packet loss rate, or jitter value of each node acquired in the current content distribution network.
Step S420, selecting a corresponding path where the network delay, the packet loss rate, or the jitter value satisfies a first condition, and sending the corresponding access resource in the source station to the user;
specifically, in this embodiment, a plurality of network delay values are set according to the obtained plurality of network delay values, packet loss values or jitter values, and the first condition is satisfied when the packet loss value or jitter value is minimum.
In this embodiment, only a single condition of the network delay, the packet loss rate, or the jitter value is considered, but in other embodiments, the network delay, the packet loss rate, or the jitter value may be considered at the same time, a plurality of total values including the weight are finally obtained by giving different weights to the network delay, the packet loss rate, or the jitter value, and the first condition is set when the value among the plurality of total values including the weight is the minimum.
Step S430, when the network status of the path corresponding to the first condition changes, selecting a path corresponding to another priority according to the priority order, and sending the access resource corresponding to the source station to the user.
Specifically, referring to fig. 8 and 9, step S410 includes:
step S411: network detection is mutually executed between the edge node and the parent layer node to obtain network delay of data transmission between the nodes;
step S412: and determining a first path, a second path and a third path between the edge node and the source station according to the total network delay size sequence of different paths between the edge node and the source station.
Correspondingly, step S420 includes:
step S421: and selecting a path with the minimum total network delay between the edge node and the source station as a first path so as to send the corresponding access resource in the source station to the user.
Correspondingly, step S430 includes:
step S431: when the first path has network failure, sending the corresponding access resource in the source station to the user through a second path;
step S432: and when the second path has network failure, sending the corresponding access resource in the source station to the user through a third path.
Specifically, the detectMgr collects data reported by the detectNode, performs summary operation to calculate factors such as time delay, packet loss rate, jitter, and the like, and calculates an optimal path, a suboptimal path, and a secondary optimal path (IP list) through a single-source shortest path algorithm to be sent to a remote dictionary server, where the optimal path, the suboptimal path, and the secondary optimal path correspond to a first path, a second path, and a third path.
For example, network delay time of data transmission between nodes is obtained by mutually detecting between nodes in the CDN network, and three paths with different priorities are obtained according to the magnitude order of network delay, as shown in fig. 8:
the first path is e8-e7-e (n-1) -e5-e 4; the delay is: 6 seconds;
a second path e8-e (n) -e 4; the delay is: 8 seconds;
a third path e8-e1-e2-e3-e 4; the delay is: 12 seconds;
the content corresponding to the access resource in the corresponding source station is sent to the user by selecting a first path, and when the network state of the node in the first path changes, a second path is selected to send the content corresponding to the access resource in the corresponding source station to the user;
and when the network state of the node in the second path changes, selecting a third path to send the content corresponding to the access resource in the corresponding source station to the user.
In this embodiment, three CDN network transmission paths with priority levels are obtained through calculation, in other embodiments, all possible transmission paths may be generated according to different network delays, or packet loss rates, or jitter values, and when a network state changes in a current network transmission process, for example, a network fault occurs, a corresponding path is sequentially selected to transmit dynamic resources according to node network delays, or packet loss rates, or jitter values obtained through calculation in different paths.
In the data transmission method of this embodiment, the optimal acceleration effect is ensured by multi-mode link quality detection, and when it is determined that the delay of the front link is large or there is a fault, a multi-layer degradation or fault-tolerant mechanism is provided to enable fast switching to ensure stable acceleration effect.
Dynamic or static contents are intelligently distinguished through modes such as a built-in common static template, manual configuration and self-learning, the dynamic contents are enabled to be accelerated to a dynamic link as soon as possible, a software framework can be multiplexed no matter the dynamic acceleration or the static acceleration, the traditional three-layer tree framework is kept during the static acceleration, father-layer nodes are degraded and flattened during the dynamic acceleration to form a whole-network graph framework, the CDN uplink bandwidth or a server is used to the maximum, and the resource utilization rate is high.
Example two
Referring to fig. 10, a program module diagram of the data transmission device of the present invention is shown. In this embodiment, the data transmission device 20 may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to implement the present invention and implement the above-mentioned method for generating an image of enterprise asset data. The program modules referred to in the embodiments of the present invention refer to a series of computer program instruction segments capable of performing specific functions, and are more suitable than the program itself for describing the execution process of the data transmission device 20 in the storage medium. The following description will specifically describe the functions of the program modules of the present embodiment:
a network access module 201, configured to determine a first network node according to a resource access request sent by a user to a source station;
a resource determining module 202, configured to determine an access resource type according to the access resource request, where the access resource type includes a static resource and a dynamic resource;
a static processing module 203, configured to send the access resource corresponding to the cache of the first network node to the user when the access resource is a static resource;
a dynamic processing module 204, configured to calculate multiple paths between the first network node and the source station when the access resource is a dynamic resource, and select a corresponding path according to a network status to send the corresponding access resource in the source station to the user.
EXAMPLE III
Fig. 11 is a schematic diagram of a hardware architecture of a computer device according to a third embodiment of the present invention. In the present embodiment, the computer device 2 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a preset or stored instruction. The computer device 2 may be a rack server, a blade server, a tower server or a rack server (including an independent server or a server cluster composed of a plurality of servers), and the like. As shown in fig. 11, the computer device 2 includes, but is not limited to, at least a memory 21, a processor 22, a network interface 23, and a data transmission device 20, which are communicatively connected to each other through a system bus. Wherein:
in this embodiment, the memory 21 includes at least one type of computer-readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the storage 21 may be an internal storage unit of the computer device 2, such as a hard disk or a memory of the computer device 2. In other embodiments, the memory 21 may also be an external storage device of the computer device 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like provided on the computer device 2. Of course, the memory 21 may also comprise both internal and external memory units of the computer device 2. In this embodiment, the memory 21 is generally used for storing an operating system installed in the computer device 2 and various types of application software, such as the program codes of the data transmission apparatus 20 described in the above embodiments. Further, the memory 21 may also be used to temporarily store various types of data that have been output or are to be output.
The network interface 23 may comprise a wireless network interface or a wired network interface, and the network interface 23 is generally used for establishing communication connection between the computer device 2 and other electronic apparatuses. For example, the network interface 23 is used to connect the computer device 2 to an external terminal through a network, establish a data transmission channel and a communication connection between the computer device 2 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), Wi-Fi, and the like.
It is noted that fig. 11 only shows the computer device 2 with components 20-23, but it is to be understood that not all shown components are required to be implemented, and that more or less components may be implemented instead.
In this embodiment, the data transmission device 20 stored in the memory 21 can be further divided into one or more program modules, and the one or more program modules are stored in the memory 21 and executed by one or more processors (in this embodiment, the processor 22) to complete the present invention.
Example four
The present embodiment also provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application mall, etc., on which a computer program is stored, which when executed by a processor implements corresponding functions. The computer readable storage medium of the embodiment is used for storing the data transmission device 20, and when being executed by a processor, the computer readable storage medium implements the method for generating the enterprise asset data representation according to the above embodiment.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A data transmission method applied to a content distribution network, the method comprising:
determining a first network node according to a resource access request sent by a user to a source station;
determining an access resource type according to the access resource request, wherein the access resource type comprises static resources and dynamic resources;
when the access resource is a static resource, sending the corresponding access resource in the first network node cache to the user;
and when the access resource is a dynamic resource, calculating a plurality of paths between the first network node and the source station, and selecting a corresponding path according to the network condition to send the corresponding access resource in the source station to the user.
2. The data transmission method of claim 1, wherein the content distribution network comprises a plurality of edge nodes and at least one parent layer node, and wherein determining the first network node based on the access resource request sent by the user to the source station comprises:
receiving a domain name corresponding to the source station output by a user;
and resolving the domain name to point the access resource request to an edge node corresponding to the first network node.
3. The data transmission method according to claim 2, wherein the sending the corresponding access resource in the network node cache closest to the user when the access resource is a static resource comprises:
when the access resource is a static resource, sending the access resource corresponding to the edge node cache corresponding to the first network node to the user;
when the access resource corresponding to the cache of the edge node is not cached, the access resource corresponding to the cache of the parent layer node of the edge node is sent to the user;
and when the parent layer node does not cache the corresponding access resource, acquiring the access resource from the source station through the content distribution network.
4. The data transmission method according to claim 1 or 3, wherein when the access resource is a dynamic resource, calculating a plurality of paths between the network node closest to the user and the source station, and selecting a corresponding path according to a network condition to send the corresponding access resource in the source station to the user comprises:
when the access resource is a dynamic resource, calculating a plurality of paths between the edge node and the source station according to the current network delay, packet loss rate or jitter value in the content distribution network;
selecting the network delay, the packet loss rate or the corresponding path with the jitter value meeting the first condition, and sending the corresponding access resource in the source station to the user;
and when the network condition of the path corresponding to the first condition changes, selecting other paths to send the corresponding access resource in the source station to the user.
5. The data transmission method according to claim 4, wherein when the access resource is a dynamic resource, calculating the plurality of paths between the edge node and the source station according to a current network delay, a packet loss rate or a jitter value in the content distribution network comprises:
network detection is mutually executed between the edge node and the parent layer node to obtain network delay of data transmission between the nodes;
and determining a first path, a second path and a third path between the edge node and the source station according to the total network delay size sequence of different paths between the edge node and the source station.
6. The data transmission method according to claim 5, wherein the selecting the corresponding path where the network delay, the packet loss rate, or the jitter value satisfies the first condition to send the corresponding access resource in the source station to the user comprises:
and selecting a path with the minimum total network delay between the edge node and the source station as a first path so as to send the corresponding access resource in the source station to the user.
7. The data transmission method according to claim 6, wherein the selecting other paths to send the corresponding access resources in the source station to the user when the network condition of the path corresponding to the first condition changes comprises:
when the first path has network failure, sending the corresponding access resource in the source station to the user through the second path;
and when the second path has network failure, sending the corresponding access resource in the source station to the user through a third path.
8. A data transmission apparatus, characterized in that the data transmission apparatus comprises:
the network access module is used for determining a first network node according to a resource access request sent to a source station by a user;
the resource determining module is used for determining the type of the access resource according to the access resource request, wherein the type of the access resource comprises a static resource and a dynamic resource;
a static processing module, configured to send the access resource corresponding to the cache of the first network node to the user when the access resource is a static resource;
and the dynamic processing module is used for calculating a plurality of paths between the first network node and the source station when the access resource is a dynamic resource, and selecting a corresponding path according to the network condition to send the corresponding access resource in the source station to the user.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the data transmission method according to any of claims 1 to 7 are implemented by the processor when executing the computer program.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when being executed by a processor realizes the steps of the data transmission method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011125164.9A CN112256495A (en) | 2020-10-20 | 2020-10-20 | Data transmission method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011125164.9A CN112256495A (en) | 2020-10-20 | 2020-10-20 | Data transmission method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112256495A true CN112256495A (en) | 2021-01-22 |
Family
ID=74245099
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011125164.9A Pending CN112256495A (en) | 2020-10-20 | 2020-10-20 | Data transmission method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112256495A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113726749A (en) * | 2021-08-13 | 2021-11-30 | 刘应森 | Data management system based on big data and intelligent security |
| CN113783787A (en) * | 2021-08-05 | 2021-12-10 | 山东省计算中心（国家超级计算济南中心） | Non-real-time data transmission method and device based on cloud edge cooperation |
| CN113810461A (en) * | 2021-08-04 | 2021-12-17 | 网宿科技股份有限公司 | Bandwidth control method, device, equipment and readable storage medium |
| CN114466018A (en) * | 2022-03-22 | 2022-05-10 | 北京有竹居网络技术有限公司 | Scheduling method and device for content distribution network, storage medium and electronic equipment |
| CN114500170A (en) * | 2021-12-28 | 2022-05-13 | 杭州安恒信息安全技术有限公司 | VPN line screening method, device, equipment and storage medium |
| CN115174690A (en) * | 2022-09-08 | 2022-10-11 | 中国人民解放军国防科技大学 | System and method for accelerating high-flow service under weak network or broken network condition |
| CN115208955A (en) * | 2021-03-29 | 2022-10-18 | 贵州白山云科技股份有限公司 | Resource request processing method and device, computer equipment and medium |
| CN115379016A (en) * | 2022-08-22 | 2022-11-22 | 深信服科技股份有限公司 | Resource access method, access service platform, device, equipment and storage medium |
| CN116170447A (en) * | 2023-04-19 | 2023-05-26 | 杭州非奇科技股份有限公司 | Data transmission method, system and storage medium based on edge node |
| CN116366524A (en) * | 2023-05-31 | 2023-06-30 | 天翼云科技有限公司 | Path calculation method and device based on content distribution network |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051709A (en) * | 2012-12-20 | 2013-04-17 | 新浪网技术(中国)有限公司 | Data transmission path determination method, network node and content delivery network system |
| CN104753965A (en) * | 2013-12-25 | 2015-07-01 | 中国电信股份有限公司 | Web acceleration method and system |
| CN105897581A (en) * | 2015-11-20 | 2016-08-24 | 乐视云计算有限公司 | Media file back source path determination method and device in CDN network |
| CN107105029A (en) * | 2017-04-18 | 2017-08-29 | 北京友普信息技术有限公司 | A kind of CDN dynamic contents accelerated method and system based on Docker technologies |
| CN110830572A (en) * | 2019-11-06 | 2020-02-21 | 中移（杭州）信息技术有限公司 | CDN access optimization method and system |
| CN111181849A (en) * | 2018-11-09 | 2020-05-19 | 北京嘀嘀无限科技发展有限公司 | Return source path determining method, determining device, computer equipment and storage medium |
| CN111355612A (en) * | 2020-02-28 | 2020-06-30 | 中国工商银行股份有限公司 | CDN node emergency processing method and system for source station fault |
| US20200304555A1 (en) * | 2019-03-18 | 2020-09-24 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for determining access path of content delivery network |
-
2020
- 2020-10-20 CN CN202011125164.9A patent/CN112256495A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051709A (en) * | 2012-12-20 | 2013-04-17 | 新浪网技术(中国)有限公司 | Data transmission path determination method, network node and content delivery network system |
| CN104753965A (en) * | 2013-12-25 | 2015-07-01 | 中国电信股份有限公司 | Web acceleration method and system |
| CN105897581A (en) * | 2015-11-20 | 2016-08-24 | 乐视云计算有限公司 | Media file back source path determination method and device in CDN network |
| CN107105029A (en) * | 2017-04-18 | 2017-08-29 | 北京友普信息技术有限公司 | A kind of CDN dynamic contents accelerated method and system based on Docker technologies |
| CN111181849A (en) * | 2018-11-09 | 2020-05-19 | 北京嘀嘀无限科技发展有限公司 | Return source path determining method, determining device, computer equipment and storage medium |
| US20200304555A1 (en) * | 2019-03-18 | 2020-09-24 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for determining access path of content delivery network |
| CN110830572A (en) * | 2019-11-06 | 2020-02-21 | 中移（杭州）信息技术有限公司 | CDN access optimization method and system |
| CN111355612A (en) * | 2020-02-28 | 2020-06-30 | 中国工商银行股份有限公司 | CDN node emergency processing method and system for source station fault |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115208955B (en) * | 2021-03-29 | 2024-04-19 | 贵州白山云科技股份有限公司 | Method, device, computer equipment and medium for processing resource request |
| CN115208955A (en) * | 2021-03-29 | 2022-10-18 | 贵州白山云科技股份有限公司 | Resource request processing method and device, computer equipment and medium |
| CN113810461A (en) * | 2021-08-04 | 2021-12-17 | 网宿科技股份有限公司 | Bandwidth control method, device, equipment and readable storage medium |
| CN113783787A (en) * | 2021-08-05 | 2021-12-10 | 山东省计算中心（国家超级计算济南中心） | Non-real-time data transmission method and device based on cloud edge cooperation |
| CN113783787B (en) * | 2021-08-05 | 2023-06-13 | 山东省计算中心（国家超级计算济南中心） | Cloud edge cooperation-based non-real-time data transmission method and device |
| CN113726749B (en) * | 2021-08-13 | 2023-04-28 | 山西鑫博睿科技有限公司 | Data management system based on big data and intelligent security |
| CN113726749A (en) * | 2021-08-13 | 2021-11-30 | 刘应森 | Data management system based on big data and intelligent security |
| CN114500170A (en) * | 2021-12-28 | 2022-05-13 | 杭州安恒信息安全技术有限公司 | VPN line screening method, device, equipment and storage medium |
| CN114500170B (en) * | 2021-12-28 | 2024-04-26 | 杭州安恒信息安全技术有限公司 | VPN line screening method, device, equipment and storage medium |
| CN114466018A (en) * | 2022-03-22 | 2022-05-10 | 北京有竹居网络技术有限公司 | Scheduling method and device for content distribution network, storage medium and electronic equipment |
| CN115379016A (en) * | 2022-08-22 | 2022-11-22 | 深信服科技股份有限公司 | Resource access method, access service platform, device, equipment and storage medium |
| CN115174690A (en) * | 2022-09-08 | 2022-10-11 | 中国人民解放军国防科技大学 | System and method for accelerating high-flow service under weak network or broken network condition |
| CN116170447A (en) * | 2023-04-19 | 2023-05-26 | 杭州非奇科技股份有限公司 | Data transmission method, system and storage medium based on edge node |
| CN116170447B (en) * | 2023-04-19 | 2023-07-14 | 杭州非奇科技股份有限公司 | Data transmission method, system and storage medium based on edge node |
| CN116366524A (en) * | 2023-05-31 | 2023-06-30 | 天翼云科技有限公司 | Path calculation method and device based on content distribution network |
| CN116366524B (en) * | 2023-05-31 | 2023-08-04 | 天翼云科技有限公司 | Path calculation method and device based on content distribution network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112256495A (en) | Data transmission method and device, computer equipment and storage medium | |
| CN107517228B (en) | Dynamic acceleration method and device in content distribution network | |
| CN107872402B (en) | Global flow scheduling method and device and electronic equipment | |
| US10404790B2 (en) | HTTP scheduling system and method of content delivery network | |
| CN106031130B (en) | Content distribution network framework with edge proxies | |
| CN108173976B (en) | Domain name resolution method and device | |
| US8065417B1 (en) | Service provider registration by a content broker | |
| CN111800458B (en) | Dynamic load balancing method and system for Kubernetes container cloud platform | |
| CN102638561B (en) | Use the Inter-cache communication of http resource | |
| JP2012257338A (en) | Dns wildcard beaconing to determine client location and resolver load for global traffic load balancing | |
| US8341285B2 (en) | Method and system for transferring files | |
| CN107347015B (en) | Method, device and system for identifying content distribution network | |
| US11734367B2 (en) | Direct data center request for proxy web scraping | |
| CN108540505B (en) | Content updating method and device | |
| CN110677492A (en) | Access request processing method and device, electronic equipment and storage medium | |
| CN113301079B (en) | Data acquisition method, system, computing device and storage medium | |
| CN109873855A (en) | A kind of resource acquiring method and system based on block chain network | |
| US20090150564A1 (en) | Per-user bandwidth availability | |
| CN107682271B (en) | Method and device for generating routing line | |
| CN110392073B (en) | Scheduling method and device based on dynamic acceleration | |
| CN110545453B (en) | Content distribution method, device and system of content distribution network | |
| EP4227829A1 (en) | Web scraping through use of proxies, and applications thereof | |
| CN115883657A (en) | Cloud disk service accelerated scheduling method and system | |
| CN112087382A (en) | Service routing method and device | |
| CN103188323A (en) | System and method for providing Web services based on user mainframe cache equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |