CN112199340A - Multithreading file downloading method and device, electronic equipment and storage medium - Google Patents
Multithreading file downloading method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112199340A CN112199340A CN202011149048.0A CN202011149048A CN112199340A CN 112199340 A CN112199340 A CN 112199340A CN 202011149048 A CN202011149048 A CN 202011149048A CN 112199340 A CN112199340 A CN 112199340A
- Authority
- CN
- China
- Prior art keywords
- cdn
- downloading
- target file
- information
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 239000012634 fragment Substances 0.000 claims abstract description 92
- 238000013467 fragmentation Methods 0.000 claims abstract description 38
- 238000006062 fragmentation reaction Methods 0.000 claims abstract description 38
- 238000013441 quality evaluation Methods 0.000 claims description 28
- 230000002159 abnormal effect Effects 0.000 claims description 27
- 230000015654 memory Effects 0.000 claims description 14
- 238000012544 monitoring process Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 7
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 2
- 238000001303 quality assessment method Methods 0.000 claims 1
- 230000005856 abnormality Effects 0.000 abstract description 4
- 230000009467 reduction Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the application discloses a multithreading file downloading method, a multithreading file downloading device, electronic equipment and a storage medium, wherein the method comprises the following steps: carrying out logic fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file; running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information; and constructing the target file by using all the acquired fragment data. Therefore, when multi-thread downloading is carried out, multi-thread parallel downloading is realized by simultaneously establishing connection with a plurality of different CDN nodes, the downloading stability can be ensured, and the downloading speed is increased, so that the problem of downloading interruption or downloading speed reduction caused by CDN node abnormality is avoided.
Description
Technical Field
The present application relates to computer network technologies, and in particular, to a multithreading file downloading method and apparatus, an electronic device, and a storage medium.
Background
A Content Delivery Network (CDN) is a traffic distribution management Network platform formed by node server clusters distributed in different areas, and configured to provide a user with distributed storage and cache of Content, where a request of the user is directed to a service node closest to the user according to Network dynamic traffic and a load condition. The method aims to enable the user to obtain the required content nearby, solve the problem of congestion of the Internet network and improve the response speed of the user for accessing the website.
The CDN server is a distributed server, and the client can download a file by sending an http request to the CDN server. The file is downloaded from the CDN server, and the prior art scheme mainly has single-thread download and multi-thread (segmented) download. However, the quality of the CDN servers is uneven, the network environment where the user terminal is located is complex, and the connection quality and the download speed between the user terminal and the CDN server are difficult to guarantee. The risk of a low download speed with poor connection quality cannot be avoided by adopting a simple single-thread or multi-thread download scheme.
Disclosure of Invention
In order to solve the foregoing technical problems, embodiments of the present application are intended to provide a multithreading file downloading method, apparatus, electronic device, and storage medium.
The technical scheme of the application is realized as follows:
in a first aspect, a multithreading file downloading method is provided, which is applied to a terminal device, and includes:
carrying out logic fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file;
running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information;
and constructing the target file by using all the acquired fragment data.
In a second aspect, there is provided a multi-threaded download apparatus, the apparatus comprising:
the slicing unit is used for carrying out logic slicing on the target file to be downloaded to obtain M pieces of slicing information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file;
the communication unit is used for running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information;
and the construction unit is used for constructing the target file by utilizing all the acquired fragment data.
In a third aspect, a terminal device is provided, which includes: a processor and a memory configured to store a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of the aforementioned method when running the computer program.
In a fourth aspect, the present application further provides a computer storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of the foregoing method.
The embodiment of the application also provides a multithreading file downloading method, a multithreading file downloading device, electronic equipment and a storage medium, wherein the method comprises the following steps: carrying out logic fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file; running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information; and constructing the target file by using all the acquired fragment data. Therefore, when the target file is subjected to multithreading downloading, the target file is simultaneously connected with a plurality of different CDN nodes, multithreading parallel downloading is realized, the downloading stability can be ensured, and the downloading speed is increased, so that the problem of downloading interruption or downloading speed reduction caused by CDN node abnormality is solved.
Drawings
FIG. 1 is a first flowchart of a multithread file download method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of an application layer implementing an http request in an embodiment of the present application;
FIG. 3 is a diagram illustrating an overall network architecture for multi-threaded downloading according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a network framework for executing an http request in an embodiment of the present application;
FIG. 5 is a second flowchart of a multi-thread file downloading method according to an embodiment of the present application;
fig. 6 is a connection framework between a terminal device and a CDN network in an embodiment of the present application;
FIG. 7 is a third flowchart of a multithread file downloading method according to an embodiment of the present application;
FIG. 8 is a block diagram of a multi-thread file downloading apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a terminal device in the embodiment of the present application.
Detailed Description
So that the manner in which the features and elements of the present embodiments can be understood in detail, a more particular description of the embodiments, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings.
Fig. 1 is a first flowchart of a multithread file downloading method in an embodiment of the present application, and as shown in fig. 1, the method may specifically include:
step 101: carrying out logic fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file;
it should be noted that the target file is logically fragmented, which may be uniform fragmentation or non-uniform fragmentation, and the fragmentation may also be understood as segmenting the target file, so as to implement multi-thread concurrent downloading. Each piece information includes the start position and the end position of each piece in the target file. For example, the size of the target file is 500 bytes, the first fragment information is 0-99 bytes, the second fragment information is 100 th-199 bytes, the third fragment information is 200 th-299 bytes, the fourth fragment information is 300 th-399 bytes, and the fifth fragment information is 400 th-499 bytes; according to the fragment information, the CDN node can obtain corresponding fragment data from the target file and send the fragment data to the terminal.
Step 102: running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information;
in some embodiments, M download threads are run, and TCP connections are established with M CDN nodes through the application layer. That is to say, in order to establish a connection with multiple CDN nodes to implement multi-line downloading, an application layer needs to implement Dns, TCP connection, TLS handshake and http request independently.
Specifically, the establishing of the connection with the M content delivery network CDN nodes includes: sending a DNS request to a domain name system DNS server through an application layer, and receiving an Internet Protocol (IP) list of CDN nodes returned by the DNS server; selecting M IP addresses from the IP list; initiating a Transmission Control Protocol (TCP) connection request to the M CDN nodes through an application layer by utilizing the M IP addresses; and performing TLS handshake with the M CDN nodes through an application layer to establish TCP connection.
In practical application, after the application layer initiates a TCP connection request to the M CDN nodes, the application layer performs identity authentication through three TLS handshakes, and after the authentication passes, the application layer determines to establish a TCP connection. Specifically, an application layer initiates a security transport layer protocol (TLS) handshake to the first CDN node and receives a digital certificate of the first CDN node; verifying that the digital certificate is legal, and establishing TLS handshake with the first CDN node; the first CDN node is any one of the M CDN nodes.
Fig. 2 is a schematic flow diagram of an implementation procedure of an http request by an application layer in an embodiment of the present application, and as shown in fig. 2, a process of the http request is as follows:
1. domain name resolution; the application layer sends a DNS request to a DNS server, the DNS request carries domain name information, the DNS server analyzes the domain name, and an IP list corresponding to the domain name is determined;
2. TCP connection; the application layer can acquire a plurality of IP addresses from the IP list and initiate a connection request to the appointed IP address;
3. TLS handshake; initiating TLS handshake to the specified IP address to ensure that an SNI (server name indicator) is a target CDN domain name, thereby obtaining a correct digital certificate of the CDN node and processing handshake domain name verification;
4. the http request is realized, the CDN service is ensured to be capable of resolving the correct domain name request, and the corresponding downloading service is provided; specifically, a host is added to the http request header to indicate the requested service, and the http request header also includes the URL of the target file and the fragment information.
In some embodiments, the obtaining, based on the M pieces of fragment information, pieces of fragment data sent by the M CDN nodes respectively includes: adding the M pieces of fragmentation information to an http request header to generate M http requests; respectively sending different http requests to the M CDN nodes through an application layer, and receiving http responses returned by the M CDN nodes; and the http response comprises the downloaded fragment data.
In some embodiments, the method further comprises: pre-creating N downloading threads in a thread pool; wherein N is an integer greater than 1; and based on the slicing result, selecting the M downloading threads for executing the multithread concurrent downloading task from the N downloading threads.
It should be noted that N download threads are established in advance, and then M download threads are selected according to the slicing result to perform multi-thread downloading. Or directly establishing M downloading threads according to the fragment result.
Step 103: and constructing the target file by using all the acquired fragment data.
It should be noted that, since the target file is downloaded in a fragmented manner, the data obtained from the CDN node is part of the data of the target file. Therefore, after the fragment data is obtained, the complete target file needs to be restored according to the position of the fragment data in the target file.
In some embodiments, the step may specifically include: sequentially caching the fragment data into a cache queue according to the acquired sequence; and reading the fragment data from the cache queue, and writing the fragment data into the corresponding position of the target file according to the position indication information.
FIG. 3 is a schematic diagram of an overall network architecture of multi-thread downloading in the embodiment of the present application, as shown in FIG. 3, a binary task includes 3 downloading threads, each of which includes a Connection operation and a Retry mechanism (Retry Policy); and the downloading thread sequentially stores the acquired fragment data into the cache queue according to the acquisition sequence, each fragment data comprises position indication information, and the writing thread writes the fragment data in the cache queue into the target file according to the position indication information to obtain a complete target file.
Fig. 4 is a schematic diagram of a network framework for executing an http request in an embodiment of the present application, where, for example, the connection pool in fig. 4 includes 3 connections, and each connection includes an IP connected to the CDN node.
Here, the execution subject of steps 101 to 103 may be a processor of the terminal device.
The terminal described in the present application may include devices such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a camera, and the like.
By adopting the technical scheme, when the target file is subjected to multithreading downloading, the multithreading parallel downloading is realized by simultaneously establishing connection with a plurality of different CDN nodes, the downloading stability can be ensured, and the downloading speed is increased, so that the problem of downloading interruption or downloading speed reduction caused by CDN node abnormality is avoided.
On the basis of the foregoing embodiment of the present application, a multithreading file downloading method of the present application is further illustrated, as shown in fig. 5, the method specifically includes:
step 501: carrying out logic fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein M is an integer greater than 1, and the fragment information includes the start position and the end position of each fragment in the target file
It should be noted that the target file is logically fragmented, which may be uniform fragmentation or non-uniform fragmentation, and the fragmentation may also be understood as segmenting the target file, so as to implement multi-thread concurrent downloading. Each piece information includes the start position and the end position of each piece in the target file. For example, the size of the target file is 500 bytes, the first fragment information is 0-99 bytes, the second fragment information is 100 th-199 bytes, the third fragment information is 200 th-299 bytes, the fourth fragment information is 300 th-399 bytes, and the fifth fragment information is 400 th-499 bytes; according to the fragment information, the CDN node can obtain corresponding fragment data from the target file and send the fragment data to the terminal.
Step 502: running M downloading threads and establishing connection with M CDN nodes; respectively acquiring fragment data sent by the M CDN nodes based on the M pieces of fragment information;
in some embodiments, the establishing connections with M content delivery network CDN nodes includes: sending a DNS request to a domain name system DNS server through an application layer, and receiving an Internet Protocol (IP) list of CDN nodes returned by the DNS server; selecting M IP addresses from the IP list; initiating a Transmission Control Protocol (TCP) connection request to the M CDN nodes through an application layer by utilizing the M IP addresses; and performing TLS handshake with the M CDN nodes through an application layer to establish TCP connection.
Step 503: monitoring the connection quality between the CDN nodes and the M CDN nodes to obtain a quality evaluation result;
it should be noted that, affected by the connection stability and the quality of different CDN nodes, the connection quality of the terminal and different CDN nodes may vary, and in the embodiment of the present application, by monitoring the connection quality with different CDN nodes, an abnormal CDN node with an abnormal connection or a slow download speed may be found in time, so that the connection with the abnormal CDN node is disconnected, and the abnormal CDN node is connected to a normal CDN node to continue to execute an incomplete download task.
In some embodiments, the monitoring the connection quality with the M CDN nodes to obtain a quality evaluation result includes: monitoring the connection quality between the CDN nodes and the CDN nodes to obtain M quality evaluation information; the quality evaluation information comprises at least one of download speed, connection time consumption and connection interruption times; and performing quality evaluation based on a preset quality evaluation standard and the M pieces of quality evaluation information to obtain M pieces of quality scores.
Here, the quality evaluation criterion specifies how to perform quality evaluation based on different quality evaluation information. For example, the score of each quality evaluation information is determined, and all the scores are summed up to obtain a total quality score.
Step 504: determining an abnormal CDN node with abnormal connection based on the quality evaluation result;
in some embodiments, when the quality evaluation result is represented by a quality score, the abnormal node may be determined by setting a score threshold. Specifically, when the quality score is smaller than a first threshold value, the abnormal CDN node is determined; and when the quality score is greater than a second threshold value, determining the CDN node as a high-quality CDN node.
Step 505: switching the connection with the abnormal CDN node to other normal CDN nodes to continue to execute the unfinished downloading task of the abnormal CDN node;
here, a disconnection instruction is sent to the abnormal CDN node to disconnect, and then TCP connection is established with other normal CDN nodes through a connection operation.
In some embodiments, the other normal CDN nodes include CDN nodes corresponding to other IP addresses in the IP list except for the M IP addresses.
In some embodiments, the other normal nodes may also be high-quality nodes in the above-determined M CDN nodes. Specifically, after the downloading task corresponding to the high-quality CDN node is completed, the connection with the abnormal CDN node is switched to the high-quality CDN node.
Fig. 6 is a connection framework between a terminal device and a CDN network in the embodiment of the present application, and as shown in fig. 6, the terminal device may establish a connection with M CDN nodes (i.e., Node1, Node2 to Node M), initiate an http request on each connection, request a binary stream of a corresponding file fragment, read the binary stream from the CDN Node corresponding to each thread, and write the binary stream into a target file, thereby implementing multi-thread parallel downloading of the target file.
Step 506: and constructing the target file by using all the acquired fragment data.
It should be noted that, since the target file is downloaded in a fragmented manner, the data obtained from the CDN node is part of the data of the target file. Therefore, after the fragment data is obtained, the complete target file needs to be restored according to the position of the fragment data in the target file.
In some embodiments, the step may specifically include: sequentially caching the fragment data into a cache queue according to the acquired sequence; and reading the fragment data from the cache queue, and writing the fragment data into the corresponding position of the target file according to the position indication information.
Therefore, the problems of low downloading speed and downloading interruption caused by uneven node downloading speed and connection quality can be completely avoided by using multi-source downloading, the downloading speed and stability are improved, and the downloading experience of a user is improved.
On the basis of the foregoing embodiment of the present application, a multithread file downloading method of the present application is further illustrated, as shown in fig. 7, the method specifically includes:
step 701: requesting a server to acquire the header information of the target file;
step 702: acquiring the size of the target file from the header information of the target file;
in practical applications, the header information of the target file further includes a Uniform Resource Locator (URL) of the target file, which is used to indicate the location of the target file.
Step 703: performing logic fragmentation on the target file based on a preset fragmentation length and the size of the target file to obtain M pieces of fragmentation information;
step 704: running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information;
step 705: and constructing the target file by using all the acquired fragment data.
By adopting the technical scheme, when the target file is subjected to multithreading downloading, the multithreading parallel downloading is realized by simultaneously establishing connection with a plurality of different CDN nodes, the downloading stability can be ensured, and the downloading speed is increased, so that the problem of downloading interruption or downloading speed reduction caused by CDN node abnormality is avoided.
In order to implement the method of the embodiment of the present application, based on the same inventive concept, an embodiment of the present application further provides a multithread downloading apparatus, as shown in fig. 8, the apparatus includes:
the fragmentation unit 801 is configured to perform logical fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file;
a communication unit 802, configured to run M download threads, establish connections with M content delivery network CDN nodes, and obtain, based on the M pieces of fragment information, fragment data sent by the M CDN nodes, respectively;
a constructing unit 803, configured to construct the target file by using all the obtained fragment data.
In some embodiments, the apparatus further includes a monitoring unit, configured to monitor connection quality with the M CDN nodes, to obtain a quality evaluation result; determining an abnormal CDN node with abnormal connection based on the quality evaluation result; and switching the connection with the abnormal CDN node to other normal CDN nodes so as to continuously execute the unfinished downloading task of the abnormal CDN node.
In some embodiments, the monitoring unit is specifically configured to monitor connection quality with the M CDN nodes to obtain M quality evaluation information; the quality evaluation information comprises at least one of download speed, connection time consumption and connection interruption times; and performing quality evaluation based on a preset quality evaluation standard and the M pieces of quality evaluation information to obtain M pieces of quality scores.
In some embodiments, the monitoring unit is specifically configured to determine that the CDN node is an abnormal CDN node when the quality score is smaller than a first threshold; and when the quality score is greater than a second threshold value, determining the CDN node as a high-quality CDN node.
In some embodiments, the monitoring unit is specifically configured to switch the connection with the abnormal CDN node to the high-quality CDN node after a download task corresponding to the high-quality CDN node is completed.
In some embodiments, the communication unit 802 is specifically configured to send a DNS request to a domain name system DNS server through an application layer, and receive an internet protocol IP list of a CDN node returned by the DNS server; selecting M IP addresses from the IP list; initiating a Transmission Control Protocol (TCP) connection request to the M CDN nodes through an application layer by utilizing the M IP addresses; and performing TLS handshake with the M CDN nodes through an application layer to establish TCP connection.
In some embodiments, the other normal CDN nodes include CDN nodes corresponding to other IP addresses in the IP list except for the M IP addresses.
In some embodiments, the communication unit 802 is specifically configured to add the M pieces of fragmentation information to an http request header to generate M http requests; respectively sending different http requests to the M CDN nodes through an application layer, and receiving http responses returned by the M CDN nodes; and the http response comprises the downloaded fragment data.
In some embodiments, the communication unit 802 is further configured to request a server to obtain header information of the target file; acquiring the size of the target file from the header information of the target file;
correspondingly, the fragmentation unit 801 is specifically configured to perform logical fragmentation on the target file based on a preset fragmentation length and a target file size to obtain M pieces of fragmentation information.
In some embodiments, the apparatus further comprises: the processing unit is used for pre-creating N downloading threads in the thread pool; wherein N is an integer greater than 1; and based on the slicing result, selecting the M downloading threads for executing the multithread concurrent downloading task from the N downloading threads.
In some embodiments, the communication unit 802 is further configured to buffer the fragmented data in a buffer queue in sequence according to the obtained sequence;
and the constructing unit 803 is configured to read the fragment data from the buffer queue, and write the fragment data into a corresponding location of the target file according to the location indication information.
Based on the hardware implementation of each unit in the multi-thread downloading device, an embodiment of the present application further provides a terminal device, as shown in fig. 9, where the terminal device includes: a processor 901 and a memory 902 configured to store a computer program capable of running on the processor;
wherein the processor 901 is configured to execute the method steps in the previous embodiments when running the computer program.
Of course, in actual practice, the various components in the terminal device are coupled together by a bus system 903, as shown in FIG. 9. It is understood that the bus system 903 is used to enable communications among the components. The bus system 903 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as the bus system 903 in FIG. 9.
In practical applications, the processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, and a microprocessor. It is understood that the electronic devices for implementing the above processor functions may be other devices, and the embodiments of the present application are not limited in particular.
The Memory may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (HDD), or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor.
In an exemplary embodiment, the present application further provides a computer readable storage medium, such as a memory including a computer program, which is executable by a processor of a terminal device to perform the steps of the foregoing method.
It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The expressions "having", "may have", "include" and "contain", or "may include" and "may contain" in this application may be used to indicate the presence of corresponding features (e.g. elements such as values, functions, operations or components) but does not exclude the presence of additional features.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another, and are not necessarily used to describe a particular order or sequence. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention.
The technical solutions described in the embodiments of the present application can be arbitrarily combined without conflict.
In the several embodiments provided in the present application, it should be understood that the disclosed method, apparatus, and device may be implemented in other ways. The above-described embodiments are merely illustrative, and for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.
Claims (14)
1. A multithreading file downloading method is applied to terminal equipment and is characterized by comprising the following steps:
carrying out logic fragmentation on a target file to be downloaded to obtain M pieces of fragmentation information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file;
running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information;
and constructing the target file by using all the acquired fragment data.
2. The method of claim 1, wherein after establishing connections with the M content delivery network CDN nodes, the method further comprises:
monitoring the connection quality between the CDN nodes and the M CDN nodes to obtain a quality evaluation result;
determining an abnormal CDN node with abnormal connection based on the quality evaluation result;
and switching the connection with the abnormal CDN node to other normal CDN nodes so as to continuously execute the unfinished downloading task of the abnormal CDN node.
3. The method of claim 2, wherein the monitoring connection quality with the M CDN nodes to obtain a quality assessment result comprises:
monitoring the connection quality between the CDN nodes and the CDN nodes to obtain M quality evaluation information; the quality evaluation information comprises at least one of download speed, connection time consumption and connection interruption times;
and performing quality evaluation based on a preset quality evaluation standard and the M pieces of quality evaluation information to obtain M pieces of quality scores.
4. The method of claim 3, wherein the determining, based on the quality evaluation result, an abnormal CDN node having an abnormal connection comprises:
when the quality score is smaller than a first threshold value, determining the CDN node as an abnormal CDN node;
and when the quality score is greater than a second threshold value, determining the CDN node as a high-quality CDN node.
5. The method of claim 4, wherein handing off the connection with the abnormal CDN node to other normal CDN nodes comprises:
and after the downloading task corresponding to the high-quality CDN node is completed, the connection with the abnormal CDN node is switched to the high-quality CDN node.
6. The method of claim 2, wherein establishing connections with M Content Delivery Network (CDN) nodes comprises:
sending a DNS request to a domain name system DNS server through an application layer, and receiving an Internet Protocol (IP) list of CDN nodes returned by the DNS server;
selecting M IP addresses from the IP list;
initiating a Transmission Control Protocol (TCP) connection request to the M CDN nodes through an application layer by utilizing the M IP addresses;
and performing TLS handshake with the M CDN nodes through an application layer to establish TCP connection.
7. The method of claim 6, wherein the other normal CDN nodes comprise CDN nodes corresponding to other IP addresses in the IP list except the M IP addresses.
8. The method according to claim 1, wherein the obtaining, based on the M pieces of fragment information, pieces of fragment data sent by the M CDN nodes, respectively, includes:
adding the M pieces of fragmentation information to an http request header to generate M http requests;
respectively sending different http requests to the M CDN nodes through an application layer, and receiving http responses returned by the M CDN nodes; and the http response comprises the downloaded fragment data.
9. The method according to claim 1, wherein before performing logical fragmentation on the target file to be downloaded to obtain M pieces of fragmentation information, the method further comprises:
requesting a server to acquire the header information of the target file;
acquiring the size of the target file from the header information of the target file;
the logic fragmentation is performed on the target file to be downloaded to obtain M pieces of fragmentation information, and the method comprises the following steps:
and performing logic fragmentation on the target file based on the preset fragmentation length and the size of the target file to obtain M pieces of fragmentation information.
10. The method of claim 1, further comprising:
pre-creating N downloading threads in a thread pool; wherein N is an integer greater than 1;
and based on the slicing result, selecting the M downloading threads for executing the multithread concurrent downloading task from the N downloading threads.
11. The method according to claim 1, wherein the constructing the object file using all the obtained shard data comprises:
sequentially caching the fragment data into a cache queue according to the acquired sequence;
and reading the fragment data from the cache queue, and writing the fragment data into the corresponding position of the target file according to the position indication information.
12. An apparatus for multi-threaded downloading, the apparatus comprising:
the slicing unit is used for carrying out logic slicing on the target file to be downloaded to obtain M pieces of slicing information; wherein, M is an integer larger than 1, and the fragment information comprises the starting position and the ending position of each fragment in the target file;
the communication unit is used for running M downloading threads, establishing connection with M Content Delivery Network (CDN) nodes, and respectively acquiring fragment data sent by the M CDN nodes based on the M fragment information;
and the construction unit is used for constructing the target file by utilizing all the acquired fragment data.
13. A terminal device, characterized in that the terminal device comprises: a processor and a memory configured to store a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of the method of any one of claims 1 to 11 when running the computer program.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011149048.0A CN112199340A (en) | 2020-10-23 | 2020-10-23 | Multithreading file downloading method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011149048.0A CN112199340A (en) | 2020-10-23 | 2020-10-23 | Multithreading file downloading method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112199340A true CN112199340A (en) | 2021-01-08 |
Family
ID=74011217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011149048.0A Pending CN112199340A (en) | 2020-10-23 | 2020-10-23 | Multithreading file downloading method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112199340A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112769957A (en) * | 2021-02-05 | 2021-05-07 | 上海哔哩哔哩科技有限公司 | Point-to-point network scheduling method and system |
| CN113778673A (en) * | 2021-08-27 | 2021-12-10 | 深圳市广和通无线股份有限公司 | Data processing method, storage medium and device |
| CN114513503A (en) * | 2022-02-16 | 2022-05-17 | 中银金融科技有限公司 | File downloading method and device, electronic equipment and computer storage medium |
| CN114860260A (en) * | 2022-04-07 | 2022-08-05 | 北京三快在线科技有限公司 | Over-the-air downloading method and device, storage medium and electronic equipment |
| CN115242780A (en) * | 2022-07-26 | 2022-10-25 | 北京知道创宇信息技术股份有限公司 | File downloading method and device, downloading manager and readable storage medium |
| CN117459542A (en) * | 2023-10-30 | 2024-01-26 | 华南理工大学 | Internet data downloading method, device, system and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017084313A1 (en) * | 2015-11-17 | 2017-05-26 | 乐视控股（北京）有限公司 | File downloading method and apparatus, and electronic device |
| CN107040615A (en) * | 2017-06-22 | 2017-08-11 | 深圳Tcl数字技术有限公司 | Method for down loading, terminal and the computer-readable recording medium of media slicing |
| CN108574883A (en) * | 2018-02-11 | 2018-09-25 | 广东南方新媒体股份有限公司 | The method for realizing multiple CDN Parallel downloads video segments |
| CN109862065A (en) * | 2018-12-06 | 2019-06-07 | 北京字节跳动网络技术有限公司 | Document down loading method, device and electronic equipment |
| CN111343274A (en) * | 2020-02-28 | 2020-06-26 | 国铁吉讯科技有限公司 | Data synchronization interaction method |
-
2020
- 2020-10-23 CN CN202011149048.0A patent/CN112199340A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017084313A1 (en) * | 2015-11-17 | 2017-05-26 | 乐视控股（北京）有限公司 | File downloading method and apparatus, and electronic device |
| CN107040615A (en) * | 2017-06-22 | 2017-08-11 | 深圳Tcl数字技术有限公司 | Method for down loading, terminal and the computer-readable recording medium of media slicing |
| CN108574883A (en) * | 2018-02-11 | 2018-09-25 | 广东南方新媒体股份有限公司 | The method for realizing multiple CDN Parallel downloads video segments |
| CN109862065A (en) * | 2018-12-06 | 2019-06-07 | 北京字节跳动网络技术有限公司 | Document down loading method, device and electronic equipment |
| CN111343274A (en) * | 2020-02-28 | 2020-06-26 | 国铁吉讯科技有限公司 | Data synchronization interaction method |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112769957A (en) * | 2021-02-05 | 2021-05-07 | 上海哔哩哔哩科技有限公司 | Point-to-point network scheduling method and system |
| WO2022166584A1 (en) * | 2021-02-05 | 2022-08-11 | 上海哔哩哔哩科技有限公司 | Point-to-point network scheduling method and system |
| CN112769957B (en) * | 2021-02-05 | 2023-09-22 | 上海哔哩哔哩科技有限公司 | Point-to-point network scheduling method and system |
| CN113778673A (en) * | 2021-08-27 | 2021-12-10 | 深圳市广和通无线股份有限公司 | Data processing method, storage medium and device |
| CN113778673B (en) * | 2021-08-27 | 2024-02-09 | 深圳市广和通无线股份有限公司 | Data processing method, storage medium and apparatus |
| CN114513503A (en) * | 2022-02-16 | 2022-05-17 | 中银金融科技有限公司 | File downloading method and device, electronic equipment and computer storage medium |
| CN114513503B (en) * | 2022-02-16 | 2024-04-02 | 中银金融科技有限公司 | File downloading method and device, electronic equipment and computer storage medium |
| CN114860260A (en) * | 2022-04-07 | 2022-08-05 | 北京三快在线科技有限公司 | Over-the-air downloading method and device, storage medium and electronic equipment |
| CN114860260B (en) * | 2022-04-07 | 2024-08-20 | 北京三快在线科技有限公司 | Over-the-air downloading method and device, storage medium and electronic equipment |
| CN115242780A (en) * | 2022-07-26 | 2022-10-25 | 北京知道创宇信息技术股份有限公司 | File downloading method and device, downloading manager and readable storage medium |
| CN117459542A (en) * | 2023-10-30 | 2024-01-26 | 华南理工大学 | Internet data downloading method, device, system and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112199340A (en) | Multithreading file downloading method and device, electronic equipment and storage medium | |
| US10893031B2 (en) | Dynamically serving digital certificates based on secure session properties | |
| US9871850B1 (en) | Enhanced browsing using CDN routing capabilities | |
| US10560543B2 (en) | Rule based cache processing in application delivery controller for load balancing | |
| US9600787B2 (en) | Deferring authentication and resource loading while starting an enterprise system | |
| US10616314B1 (en) | Dynamic source routing for data transfer | |
| US10165039B1 (en) | Pre-connecting to a cacheable resource based on page profiles | |
| US8903946B1 (en) | Reduction in redirect navigation latency via speculative preconnection | |
| JP6491352B2 (en) | Method and apparatus for regulating batch billing of services | |
| US20240073274A1 (en) | Accelerating connections to a host server | |
| KR102154788B1 (en) | A DNS resolution method in a terminal of a wireless communication system, and an apparatus therefor | |
| US8762563B2 (en) | Method and apparatus for improving the adaptive bit rate behavior of a streaming media player | |
| CN113536168B (en) | Component processing method and device | |
| EP3046038A1 (en) | A computer-implemented method, a network device and computer program products for tracking OPT-OUT user preferences in a global communication network | |
| CN110191203B (en) | Method for realizing dynamic access of server and electronic equipment | |
| US9674220B2 (en) | System and method thereof for optimizing delivery of content over a network | |
| EP2701068B1 (en) | Network access system | |
| CN115022278A (en) | CDN processing method and system | |
| CN110324366B (en) | Data processing method, device and system | |
| EP3487152A1 (en) | Auctioning the serving and/or caching of a data object | |
| CN111695018B (en) | Data processing method and device, distributed network system and computer equipment | |
| CN114466032B (en) | CDN system merging and source returning method, device and storage medium | |
| Bocovich et al. | Addressing denial of service attacks on free and open communication on the internet | |
| CN115334163A (en) | Service data processing method and device and server | |
| CN110138872B (en) | Computer readable medium for network proxy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |