CN108566370B - Method and device for returning data to source - Google Patents
Method and device for returning data to source Download PDFInfo
- Publication number
- CN108566370B CN108566370B CN201810142517.2A CN201810142517A CN108566370B CN 108566370 B CN108566370 B CN 108566370B CN 201810142517 A CN201810142517 A CN 201810142517A CN 108566370 B CN108566370 B CN 108566370B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- node
- determining
- core
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a method and a device for returning data to a source, wherein the method is applied to an edge node, the edge node is used for returning data to the source from a core node, and the core node stores global video data synchronized from a global storage node; the method comprises the following steps: receiving a first data acquisition request at an edge node side, and if the edge node does not have a target file corresponding to the first data acquisition request, determining attribute information of the target file; logically dividing the target file into a plurality of data blocks based on the attribute information; determining one or more target data chunks from the plurality of data chunks; determining a core node corresponding to the one or more target data blocks; and requesting the core node corresponding to the one or more target data blocks for the target video data. The invention can improve the success rate and efficiency of returning the source and save the bandwidth and the storage resource.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for returning data to a source.
Background
In recent years, with the rapid development of internet technology, the access demand of users of each large website is increasing. In order to improve the access response speed, enable a user to obtain requested Content more quickly, and solve the state of network congestion, a CDN (Content delivery network) plays an important role.
The back sourcing is an important technology in the CDN, and the back sourcing means that when an IDC (Internet Data Center) node in the CDN, which receives a user Data request, does not have Data requested by the user, the IDC node requests the Data requested by the user from a back-sourced node.
In the existing source returning mode, when one source returning node goes to another returned source node for data source returning, if the returned source node does not have data needing to be returned, the source returning node needs to make a data request to other returned source nodes, the source returning success rate is low, and the bandwidth and the storage resources are greatly wasted;
secondly, in the existing source returning mode, the server firstly obtains an available source returning IP address list through a certain method (the existing methods such as static configuration, dynamic configuration, real-time obtaining and the like); then, when the file needs to be returned to the source, an IP address is selected from the list for downloading through a certain algorithm (polling, consistent hashing and other existing methods, hereinafter referred to as a positioning algorithm), and the whole file only selects one IP address for downloading, so that a subsequent request for the same file is caused, and when data which is not downloaded is requested, the waiting time is long, and even the situation of blocking occurs.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a method of data back-sourcing and a corresponding apparatus of data back-sourcing that overcome or at least partially solve the above problems.
In order to solve the above problem, an embodiment of the present invention discloses a method for returning data to a source, where the method is applied to an edge node, the edge node is used for returning data to a source from a core node, and the core node stores global video data synchronized from a global storage node;
the method comprises the following steps:
receiving a first data acquisition request at an edge node side, and if the edge node does not have a target file corresponding to the first data acquisition request, determining attribute information of the target file;
logically dividing the target file into a plurality of data blocks based on the attribute information, wherein each data block at least comprises two data segments;
determining one or more target data chunks from the plurality of data chunks;
determining a core node corresponding to the one or more target data blocks;
and requesting the core nodes corresponding to the one or more target data blocks for the target video data.
Preferably, the requesting target video data from the core node corresponding to the one or more target data partitions includes:
when the number of the target data blocks is larger than a value 1, determining the downloading sequence of the target data blocks;
and according to the downloading sequence, when the proportion of the target video data downloaded from the corresponding core node of the previous target data block reaches a preset threshold value, starting the downloading of the next target data block.
Preferably, the determining a core node corresponding to the one or more target data chunks includes:
determining the downloading speed of a client;
determining the back-source speed of the edge node;
and if the downloading speed is greater than the back-source speed, respectively allocating at least two core nodes to the target data blocks.
Preferably, the determining one or more target data partitions from the plurality of data partitions includes
Receiving a second data acquisition request aiming at the target file at an edge node side, wherein the second data acquisition request comprises a start byte and/or an end byte;
determining one or more target data chunks from the plurality of data chunks according to the start byte and/or the end byte.
Preferably, the attribute information includes a file size, and the dividing the target file into a plurality of data blocks based on the attribute information includes:
logically dividing the target file into a plurality of data fragments according to the file size of the target file;
and averaging all the data fragments, and taking the obtained data fragment set as data blocks.
The embodiment of the invention also discloses a device for returning data to the source, which is applied to the edge node, wherein the edge node is used for returning data to the source from the core node, and the core node stores the global video data synchronized from the global storage node;
the device comprises:
an attribute information determining module, configured to receive a first data obtaining request at an edge node side, and if the edge node does not have a target file corresponding to the first data obtaining request, determine attribute information of the target file;
a data block determination module, configured to logically divide the target file into a plurality of data blocks based on the attribute information, where each data block includes at least two data segments;
a target data block determination module for determining one or more target data blocks from the plurality of data blocks;
a core node determining module, configured to determine a core node corresponding to the one or more target data chunks;
and the first data request module is used for requesting the core nodes corresponding to the one or more target data blocks for the target video data.
Preferably, the first data request module includes:
the download order determining submodule is used for determining the download order of the target data blocks when the number of the target data blocks is greater than a numerical value 1;
and the first parallel downloading submodule is used for starting the downloading of the next target data block when the proportion of the target video data downloaded from the corresponding core node by the current target data block reaches a preset threshold value according to the downloading sequence.
Preferably, the core node determining module includes:
the download speed determining submodule is used for determining the download speed of the client;
the source returning speed determining submodule is used for determining the source returning speed of the edge node;
and the second parallel downloading submodule is used for respectively distributing at least two core nodes for the target data block if the downloading speed is greater than the back-to-source speed.
Preferably, the target data blocking determination module includes:
a second request receiving submodule, configured to receive, at an edge node side, a second data obtaining request for the target file, where the second data obtaining request includes a start byte and/or an end byte;
and the target data block determining submodule is used for determining one or more target data blocks from the plurality of data blocks according to the starting byte and/or the ending byte.
Preferably, the data blocking determination module includes:
the data segment dividing submodule is used for logically dividing the target file into a plurality of data segments according to the file size of the target file;
and the equipartition submodule is used for equipartition of all the data segments and taking the obtained data segment set as the data blocks.
Compared with the background art, the embodiment of the invention has the following advantages:
in the embodiment of the invention, after the edge node receives the first data acquisition request, the target file can be logically divided into a plurality of data blocks according to the attribute information of the target file corresponding to the first data acquisition request, the target data blocks are determined, the core nodes are distributed to the target data blocks, the corresponding target video data are requested from the core nodes corresponding to the target data blocks, the globally stored video data are stored in the core nodes, the core nodes can return the corresponding target video data after receiving the data requests of the target data blocks, the edge node does not need to request data from other nodes, the source return success rate is improved, and the bandwidth and the storage resources are saved.
In addition, the embodiment of the invention only needs to request the content of the target data block from the core node, and does not need to request the content of the whole target file, thereby improving the efficiency of data acquisition and further saving bandwidth and storage resources.
Drawings
FIG. 1 is a schematic structural diagram of an embodiment of a CDN system of the present invention;
FIG. 2 is a flow chart of the steps of one embodiment of a method of data feed back of the present invention;
FIG. 3 is a flow chart of steps in another embodiment of a method of data feed back of the present invention;
FIG. 4 is a block diagram of data in another embodiment of a method of data feed back of the present invention;
FIG. 5 is a block diagram of an embodiment of an apparatus for returning data to a source according to the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, the present invention is described in detail with reference to the accompanying drawings and the detailed description thereof.
Referring to fig. 1, a schematic structural diagram of an embodiment of a CDN system of the present invention is shown, where the CDN system improves an existing CDN architecture, and specifically, the CDN system may include an edge node 101, a core node 102, and a global storage node 103; wherein, the first and the second end of the pipe are connected with each other,
the edge node 101 is used for returning data from the core node 102 to the source;
the core node 102 is configured to serve as a returned source node and return video data to the edge node 101;
the global storage node 103 is configured to issue global video data to the core node 102.
Specifically, in the embodiment of the present invention, all CDN nodes are divided into three layers, i.e., a global storage node 103, a core node 102, and an edge node 101, where the edge node 101 is located at the lowest layer and is a node that directly provides a service for a user; the core node 102 is located in the middle layer and is a node serving the edge node 101; the global storage node 103 is located at the uppermost layer, and is a node for storing global data.
In one embodiment, the number of edge nodes 101 is greater than the number of core nodes 102, and the number of core nodes 102 is greater than the number of global storage nodes 103.
In one embodiment, the storage capacity of a single edge node is less than the storage capacity of a single core node, which is less than the storage capacity of a single global storage node.
As an example, the ratio of the number of edge nodes 101, core nodes 102, and global storage nodes 103 may be 25:5: 1; the ratio of the storage capacity of each node can be 100:10: 1.
The above ratio is an empirical value, and the embodiment of the present invention does not limit the above ratio, and may be set as needed by a person skilled in the art according to actual situations.
According to the CDN system of the embodiment of the present invention, the global storage node 103 issues the globally stored global video data to all the core nodes 102 according to the preset time interval, and then each core node 102 stores the global video data, so that the core nodes 102 in the embodiment of the present invention do not perform the consistent hash calculation, and all the file requests are scheduled on each server in a polling manner. Therefore, no matter what content is stored in the edge node 101, it is feasible to connect any core node 102 for returning the source, so that the success rate of returning the source is improved, and the bandwidth and the storage resource are saved.
It should be noted that the CDN system in the embodiment of the present invention is a content delivery network, and the CDN system described in the embodiment of the present invention may be a video CDN network, where the edge node 101, the core node 102, and the global storage node 103 are used to provide video resources, and the data return source is directed to a video return source. Of course, those skilled in the art may understand that the CDN system according to the embodiment of the present invention may also be a non-video CDN network, where the edge node 101, the core node 102, and the global storage node 103 are configured to provide a non-video resource, where the data feed back is directed to a non-video feed back, and the non-video may be a multimedia resource other than a video, such as audio, or the like, or the non-video may also be a non-multimedia resource, such as a file object, or the like, that is, any data feed back that has a feed back requirement may use the implementation manner provided by the embodiment of the present invention. The embodiments provided in the embodiments of the present invention are all described by taking the video feed back as an example.
Based on the CDN system in fig. 1, referring to fig. 2, a flowchart of steps of an embodiment of a method for returning data to a source of the present invention is shown, where the method specifically includes the following steps:
in a specific implementation, when a user needs to obtain video data, a first data obtaining request may be triggered in a client, and sent to an edge node, where the first data obtaining request may include information such as a type of a target file and a name of the target file, as an example.
After receiving the first data acquisition request, the edge node may determine whether a corresponding target file is stored locally according to information such as a target file type and a name of the target file in the first data acquisition request, and if the corresponding target file does not exist in the edge node, the edge node may request the upper core node for the target file.
In the process of requesting the target file from the core node, the edge node may first determine the attribute information of the target file, and as an example, the attribute information of the target file may include a file size of the target file. Specifically, in implementation, the edge node may determine the target file according to the name of the target file, and determine the file size of the target file.
In practice, the target file may exist in various data forms, such as an audio file type, a video file type, etc., and thus, after the target file is determined, the file size of the target file may also be determined according to the target file type.
when a source return is required, the edge node may logically divide a target file to be source returned into a plurality of data block blocks according to attribute information of the target file, where each data block includes at least two data segments piece.
In a preferred embodiment of the present invention, step 202 may comprise the following sub-steps:
a substep S11 of logically dividing the target file into a plurality of data segments according to the file size of the target file;
in a specific implementation, the edge node may use a file cutting method to logically divide the target file into a plurality of data fragments piece according to the file size of the target file, for example, assuming that the file size of the target file is 10MB, 1MB of data may be used as one piece. Wherein each piece may include a start byte and an end byte.
And a substep S12, averaging all the data segments to obtain a data segment set as a data block.
Then, the edge node may divide all pieces equally, and organize a preset number of piece sets into one data block, for example, assuming that the target file is divided into 10 pieces, each two pieces may be used as one set, that is, one block.
In practice, according to the file size, about 10-20 blocks can be obtained from a target file, and the number of the blocks and the number of pieces contained in the blocks are not limited in the embodiment of the invention.
in a specific implementation, if there is no behavior such as dragging when the user watches the video, each data chunk in the target file may be used as a target data chunk.
In practice, however, since there may be behaviors such as dragging when the user watches the video, when going back to the source, the entire target file does not need to be downloaded, but the block and piece are selectively downloaded according to the request of the user. Then in a preferred embodiment of the present invention, step 203 may comprise the following sub-steps:
receiving a second data acquisition request aiming at the target file at an edge node side, wherein the second data acquisition request comprises a start byte and/or an end byte; determining one or more target data chunks from the plurality of data chunks according to the start byte and/or the end byte.
Specifically, if the user has behaviors such as dragging, the client may determine a start byte and/or an end byte according to the behaviors such as dragging of the user, generate a second data acquisition request, and send the second data acquisition request to the edge node, and after the edge node receives the second data acquisition request, determine the data block where the start byte and the end byte of the user request are located, and use the data block as the target data block.
In practice, the target data block may comprise at least one block.
after determining the target data blocks, a core node may be further allocated to each target data block, that is, an IP address of the core node for downloading the target data block is determined.
In one embodiment, after the target file is logically divided into a plurality of blocks, each block may be allocated a corresponding core node in advance.
In a particular implementation, a core node list may be set for all core nodes. For each block, a hash algorithm such as a preset positioning algorithm can be adopted, at least one available core node is selected from the core node list and allocated to the block. After the target data block is determined, the IP address of the core node corresponding to the target data block may be obtained.
In another embodiment, instead of allocating a corresponding core node to each block in advance, after a target data block is determined, a core node may be allocated to a block included in the target data block. Then preferably step 204 may comprise the sub-steps of:
determining the downloading speed of a client; determining the back source speed of the edge node; and if the downloading speed is greater than the back-source speed, respectively allocating at least two core nodes to the target data blocks.
Specifically, the download speed of the client refers to the download speed of the user, for example, the download speed may be calculated as follows:
1024 × X (access bandwidth value) ÷ 9
Taking 2M bandwidth as an example, the download speed is about 1024 × 2 ÷ 9 ÷ 228 KB/s.
The edge node can measure the downloading speed of the video data by the user in real time, and can also obtain the downloading speed of the user by reading the measured data of other measuring application programs.
In one embodiment, the return source velocity is calculated as follows:
and the source return speed is the download data volume/download time of the edge node from the core node corresponding to the target data block.
After the downloading speed and the source returning speed are obtained, if the downloading speed is greater than or equal to the source returning speed, in order to avoid waiting for a user, parallel downloading may be performed, that is, at least two core nodes are allocated to each target data block, and the target video data is downloaded in parallel through the at least two core nodes, where the parallel downloading may be performed after a previous piece downloads a certain proportion, or may be performed synchronously, which is not limited herein, with reference to the following sub-step S32.
It should be noted that, in order to save traffic, the number of pieces downloaded in parallel may be limited, for example, to download three pieces at most.
On the other hand, if the downloading speed is less than the back-to-source speed, parallel downloading is not carried out, namely, a core node is distributed to each target data block, and the service is carried out in a conventional mode.
In the embodiment of the invention, at least two core nodes can be allocated for the data block, and the data block can select the IP addresses of the at least two core nodes for data downloading, so that the data downloading efficiency is improved.
After the core node corresponding to the target data block is determined, the edge node may establish http connection between the target data block and the IP address of the corresponding core node, and download the target video data to the IP address corresponding to the core node.
In a preferred embodiment of the present invention, when the number of target data blocks is greater than the value 1, that is, when the content requested by the user crosses the block, step 205 may include the following sub-steps:
a substep S31, determining a downloading order of the target data chunks;
in a specific implementation, piece contained in one block may be adjacent piece, and a downloading order between at least two blocks may be determined according to a precedence order of bytes.
And a substep S32, according to the downloading sequence, when the proportion of the target video data downloaded from the corresponding target core node by the previous target data block reaches a preset threshold, starting the downloading of the next target data block.
Specifically, after the downloading sequence of at least two target data blocks is determined, parallel downloading may be performed according to the downloading sequence.
In practice, since there may be actions such as dragging during the process of watching a video by a user, in this case, sometimes it is not necessary to download a next block, so as to avoid wasting resources, the parallel downloading may be implemented as follows: and when the proportion of the target video data downloaded from the corresponding core node of the previous target data block reaches a preset threshold value, starting the downloading of the next target data block. For example, when the current block content is downloaded more than 90%, the next block is downloaded.
It should be noted that, when the edge node performs the parallel multi-path downloading to the core node, the network traffic of the core node may be increased, so that when the parallel multi-path downloading of the embodiment of the present invention is implemented, the edge node may determine a next time period first, if the time period is the traffic peak time period, still use the conventional method for downloading, and if the time period is the non-traffic peak time period, use the downloading method of the embodiment of the present invention, where the traffic peak time period may be set manually, for example, set 19:00-23:00 per night as the traffic peak time period.
In the embodiment of the invention, after the edge node receives the first data acquisition request, the target file can be logically divided into a plurality of data blocks according to the attribute information of the target file corresponding to the first data acquisition request, the target data block corresponding to the first data acquisition request is determined, and the corresponding target video data is requested from the core node corresponding to the target data block.
In addition, the embodiment of the invention only needs to request the content of the target data block from the core node, and does not need to request the content of the whole target file, thereby improving the efficiency of data acquisition and further saving bandwidth and storage resources.
Referring to fig. 3, a flowchart illustrating steps of another embodiment of a data source returning method of the present invention is shown, which may specifically include the following steps:
in a specific implementation, in the process of acquiring target video data corresponding to the target data block, if the edge node receives a third data acquisition request, the third data acquisition request may be processed synchronously.
As an example, the third data acquisition request may be a request initiated by another user.
If the third data obtaining request is also a request for a target file, the edge node may determine, with reference to the implementation manner in step 203, a target data chunk corresponding to the third data obtaining request, and if the target data chunk corresponding to the third data obtaining request is a data chunk for which video data has not been downloaded, request, with reference to the implementation manner in step 205, the target video data from the core node of the corresponding target data chunk.
For example, as shown in fig. 4, assume that a second data acquisition request from a user a requests to download 3M video data, i.e. P in the figure0To P6The server finds that there is no data to download the data from the source, and divides the target file to be sourced back into 6 blocks, which are respectively: (P)0，P1)、(P1，P2)、(P2，P3)、(P3，P4)、(P4，P5)、(P5，P6) When t time edge node downloads PtAt the time, there is another new user B to request a part of the data in the 3M data, i.e. PsTo P6In between, at which time the edge node may be (P)3，P4)、(P4，P5)、(P5，P6) And taking the three blocks as second target data blocks, and requesting the target video data from the corresponding core nodes. For user B, only waiting for P is required3To PsUnder this dataAnd (4) loading time. If the embodiment of the invention is not adopted, the user B needs to wait for PtTo PsThe download time of the data is long, and the delay of returning data to the B user is increased due to the long waiting time, so that the B user is very likely to be jammed. Therefore, after the scheme of the embodiment of the invention is adopted, for a plurality of requests of the same target file, block or piece is taken as a unit, and data can be requested to the corresponding core node, so that parallel processing is realized, the edge node can provide video data for subsequent users as soon as possible, the waiting time of the users can be reduced, the video playing time delay and the video pause frequency are reduced, and the service quality of the users is improved.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 5, a block diagram of an embodiment of a data source returning device according to the present invention is shown, where the device is applied in an edge node, the edge node is used for performing data source returning from a core node, and the core node stores global video data synchronized from a global storage node;
the device may specifically include the following modules:
an attribute information determining module 501, configured to receive a first data obtaining request at an edge node side, and if there is no target file corresponding to the first data obtaining request in the edge node, determine attribute information of the target file;
a data block determining module 502, configured to logically divide the target file into a plurality of data blocks based on the attribute information, where each data block includes at least two data segments;
a target data block determination module 503, configured to determine one or more target data blocks from the plurality of data blocks;
a core node determining module 504, configured to determine a core node corresponding to the one or more target data partitions;
a first data request module 505, configured to request target video data from a core node corresponding to the one or more target data chunks.
In a preferred embodiment of the present invention, the first data request module 505 includes:
the downloading sequence determining sub-module is used for determining the downloading sequence of the target data blocks when the number of the target data blocks is greater than a numerical value 1;
and the first parallel downloading submodule is used for starting the downloading of the next target data block when the proportion of the target video data downloaded from the corresponding core node of the current target data block reaches a preset threshold value according to the downloading sequence.
In a preferred embodiment of the present invention, the core node determining module 504 includes:
the download speed determining submodule is used for determining the download speed of the client;
the source returning speed determining submodule is used for determining the source returning speed of the edge node;
and the second parallel downloading submodule is used for respectively distributing at least two core nodes for the target data block if the downloading speed is greater than the back-to-source speed.
In a preferred embodiment of the present invention, the target data block determining module 503 includes:
a second request receiving submodule, configured to receive, at an edge node side, a second data obtaining request for the target file, where the second data obtaining request includes a start byte and/or an end byte;
and the target data block determining submodule is used for determining one or more target data blocks from the plurality of data blocks according to the starting byte and/or the ending byte.
In a preferred embodiment of the present invention, the data block determining module 502 includes:
the data segment dividing submodule is used for logically dividing the target file into a plurality of data segments according to the file size of the target file;
and the equipartition submodule is used for equipartition of all the data fragments and taking the obtained data fragment set as the data block.
For the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points.
The embodiment of the invention also discloses a system, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and when the program is executed by the processor, the method in the figures 1-3 can be realized
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of fig. 1-3.
For the system and computer-readable storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or terminal equipment comprising the element.
The method and the apparatus for returning data to source provided by the present invention are described in detail above, and a specific example is applied in the present document to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understand the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (6)
1. The method for data source returning is characterized in that the method is applied to an edge node, the edge node is used for performing data source returning from a core node, and global video data synchronized from a global storage node is stored in the core node; the core node is positioned at the upper layer of the edge node, and the global storage node is positioned at the upper layer of the core node;
the method comprises the following steps:
receiving a first data acquisition request at an edge node side, and if the edge node does not have a target file corresponding to the first data acquisition request, determining attribute information of the target file;
logically dividing the target file into a plurality of data blocks based on the attribute information, wherein each data block at least comprises two data segments;
determining one or more target data chunks from the plurality of data chunks;
determining a core node corresponding to the one or more target data blocks;
requesting target video data from a core node corresponding to the one or more target data blocks;
wherein the determining one or more target data chunks from the plurality of data chunks comprises:
receiving a second data acquisition request aiming at the target file at the side of an edge node, wherein the second data acquisition request comprises a start byte and/or an end byte, and the start byte and/or the end byte are determined based on the dragging behavior of a user on a video;
determining one or more target data chunks from the plurality of data chunks according to the start byte and/or the end byte;
wherein the determining the core node corresponding to the one or more target data partitions includes:
determining the downloading speed of a client;
determining the source returning speed of the edge node when downloading data from the core node;
and if the downloading speed is greater than the back-source speed, respectively allocating at least two core nodes to the target data blocks.
2. The method of claim 1, wherein requesting target video data from a core node corresponding to the one or more target data chunks comprises:
when the number of the target data blocks is larger than a value 1, determining the downloading sequence of the target data blocks;
and according to the downloading sequence, when the proportion of the target video data downloaded from the corresponding core node of the previous target data block reaches a preset threshold value, starting the downloading of the next target data block.
3. The method of claim 1, wherein the attribute information comprises a file size, and wherein dividing the target file into a plurality of data chunks based on the attribute information comprises:
logically dividing the target file into a plurality of data fragments according to the file size of the target file;
and averaging all the data fragments, and taking the obtained data fragment set as data blocks.
4. The device for data source return is applied to an edge node, wherein the edge node is used for data source return from a core node, and global video data synchronized from a global storage node is stored in the core node; the core node is positioned at the upper layer of the edge node, and the global storage node is positioned at the upper layer of the core node;
the device comprises:
the attribute information determining module is used for receiving a first data acquisition request at an edge node side, and if the edge node does not have a target file corresponding to the first data acquisition request, determining the attribute information of the target file;
a data block determination module, configured to logically divide the target file into a plurality of data blocks based on the attribute information, where each data block includes at least two data segments;
a target data block determination module for determining one or more target data blocks from the plurality of data blocks;
a core node determining module, configured to determine a core node corresponding to the one or more target data partitions;
the first data request module is used for requesting the core nodes corresponding to the one or more target data blocks for target video data;
wherein the target data blocking determination module comprises:
a second request receiving submodule, configured to receive, at an edge node side, a second data obtaining request for the target file, where the second data obtaining request includes a start byte and/or an end byte, and the start byte and/or the end byte are determined based on a drag behavior of a user on a video;
a target data block determination submodule, configured to determine one or more target data blocks from the multiple data blocks according to the start byte and/or the end byte;
wherein the core node determining module comprises:
the download speed determining submodule is used for determining the download speed of the client;
the source returning speed determining submodule is used for determining the source returning speed when the edge node downloads data from the core node;
and the second parallel downloading submodule is used for respectively distributing at least two core nodes for the target data block if the downloading speed is greater than the back-to-source speed.
5. The apparatus of claim 4, wherein the first data request module comprises:
the download order determining submodule is used for determining the download order of the target data blocks when the number of the target data blocks is greater than a numerical value 1;
and the first parallel downloading submodule is used for starting the downloading of the next target data block when the proportion of the target video data downloaded from the corresponding core node of the current target data block reaches a preset threshold value according to the downloading sequence.
6. The apparatus of claim 4, wherein the data chunking determination module comprises:
the data segment dividing submodule is used for logically dividing the target file into a plurality of data segments according to the file size of the target file;
and the equipartition submodule is used for equipartition of all the data segments and taking the obtained data segment set as the data blocks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810142517.2A CN108566370B (en) | 2018-02-11 | 2018-02-11 | Method and device for returning data to source |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810142517.2A CN108566370B (en) | 2018-02-11 | 2018-02-11 | Method and device for returning data to source |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108566370A CN108566370A (en) | 2018-09-21 |
| CN108566370B true CN108566370B (en) | 2022-07-12 |
Family
ID=63532244
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810142517.2A Active CN108566370B (en) | 2018-02-11 | 2018-02-11 | Method and device for returning data to source |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108566370B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110022482B (en) * | 2019-03-05 | 2021-07-27 | 咪咕视讯科技有限公司 | Video playing starting method, video service system and storage medium |
| CN112995251B (en) * | 2019-12-13 | 2023-02-03 | 北京金山云网络技术有限公司 | Source returning method and device, electronic equipment and storage medium |
| CN113301087B (en) * | 2020-07-21 | 2024-04-02 | 阿里巴巴集团控股有限公司 | Resource scheduling method, device, computing equipment and medium |
| CN112130931B (en) * | 2020-09-27 | 2023-01-06 | 联想(北京)有限公司 | Application deployment method, node, system and storage medium |
| CN112769957B (en) * | 2021-02-05 | 2023-09-22 | 上海哔哩哔哩科技有限公司 | Point-to-point network scheduling method and system |
| EP4075691B1 (en) * | 2021-02-20 | 2024-05-01 | Wangsu Science & Technology Co., Ltd. | Resource requesting method and terminal |
| CN114363358B (en) * | 2021-12-30 | 2024-03-12 | 南京中孚信息技术有限公司 | Document distribution method, device, electronic equipment and storage medium |
| CN115022177B (en) * | 2022-06-08 | 2023-10-24 | 阿里巴巴（中国）有限公司 | CDN system, source returning method, CDN node and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634361A (en) * | 2012-08-28 | 2014-03-12 | 腾讯科技（深圳）有限公司 | Method and apparatus for downloading file |
| CN104580067A (en) * | 2013-10-11 | 2015-04-29 | 北京大学 | Multiserver-based stream media transmitting device and stream media transmitting method |
| CN104811323A (en) * | 2014-01-23 | 2015-07-29 | 腾讯科技（深圳）有限公司 | Data requesting method, data requesting device, node server and CDN (content delivery network) system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104253838A (en) * | 2013-06-27 | 2014-12-31 | 北京世纪互联宽带数据中心有限公司 | CDN (content delivery network) node and CDN service system |
| CN106550023A (en) * | 2016-10-31 | 2017-03-29 | 北京百度网讯科技有限公司 | A kind of method and apparatus for providing the user specified file piecemeal |
-
2018
- 2018-02-11 CN CN201810142517.2A patent/CN108566370B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103634361A (en) * | 2012-08-28 | 2014-03-12 | 腾讯科技（深圳）有限公司 | Method and apparatus for downloading file |
| CN104580067A (en) * | 2013-10-11 | 2015-04-29 | 北京大学 | Multiserver-based stream media transmitting device and stream media transmitting method |
| CN104811323A (en) * | 2014-01-23 | 2015-07-29 | 腾讯科技（深圳）有限公司 | Data requesting method, data requesting device, node server and CDN (content delivery network) system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108566370A (en) | 2018-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108566370B (en) | Method and device for returning data to source | |
| JP6984097B2 (en) | Content delivery network architecture with edge proxies | |
| CN107872489B (en) | File slice uploading method and device and cloud storage system | |
| CN108173774B (en) | Client upgrading method and system | |
| US20170164020A1 (en) | Content delivery method for content delivery network platform and scheduling proxy server | |
| CN109600437B (en) | Downloading method of streaming media resource and cache server | |
| WO2017096830A1 (en) | Content delivery method and scheduling proxy server for cdn platform | |
| EP2976887A1 (en) | Bandwidth management for over-the-top adaptive streaming | |
| CN107333148B (en) | Method for processing video frequency and device, computer storage medium and equipment | |
| CN111355971B (en) | Live streaming transmission method and device, CDN server and computer readable medium | |
| JP6972714B2 (en) | Data acquisition programs, equipment, and methods | |
| CN110324406B (en) | Method for acquiring business data and cloud service system | |
| CN109862376B (en) | Live content delivery system, method and device, list server and storage medium | |
| WO2023098029A1 (en) | Resource scheduling method and system | |
| EP3103264B1 (en) | Methods and apparatus for naming video content chunks | |
| CN106657182B (en) | Cloud file processing method and device | |
| CN111031405A (en) | Transmission method, device, equipment and medium for slice file in multimedia playback | |
| CN115208955B (en) | Method, device, computer equipment and medium for processing resource request | |
| CN112565829B (en) | Network scheduling method, device and system and readable storage medium | |
| CN107908730B (en) | Method and device for downloading data | |
| CN113676514A (en) | File source returning method and device | |
| CN106850273B (en) | Server and streaming media service providing method thereof | |
| CN110300308B (en) | Streaming media playback method and device | |
| CN112118461B (en) | Audio and video file storage and distribution method and device | |
| CN112860432A (en) | Process management method, device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |