CN113596125B - Resource downloading method, device, data system and readable storage medium - Google Patents
Resource downloading method, device, data system and readable storage medium Download PDFInfo
- Publication number
- CN113596125B CN113596125B CN202110817580.3A CN202110817580A CN113596125B CN 113596125 B CN113596125 B CN 113596125B CN 202110817580 A CN202110817580 A CN 202110817580A CN 113596125 B CN113596125 B CN 113596125B
- Authority
- CN
- China
- Prior art keywords
- host
- downloading
- resource
- task
- independent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000004891 communication Methods 0.000 claims abstract description 29
- 230000015654 memory Effects 0.000 claims description 18
- 238000011144 upstream manufacturing Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 5
- 238000009434 installation Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0894—Packet rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a resource downloading method, a device, a data system and a readable storage medium, and belongs to the technical field of information. The resource downloading method comprises the following steps: obtaining and transmitting running numbers according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading target resources in the cloud host at any time; and controlling the independent host to execute the downloading task of the target resource according to the concurrent running number. The invention can improve the efficiency of downloading resources by a plurality of independent hosts in the cloud host.
Description
Technical Field
The present invention relates to the field of information technologies, and in particular, to a method and apparatus for downloading resources, a data system, and a readable storage medium.
Background
A cloud host is a part that virtualizes multiple similar independent hosts on one host, each of which may have a separate operating system. Therefore, a cloud host can virtualize more than tens of hundred mobile phones, and each virtual mobile phone can be provided with resources such as mobile phone application programs of the same type of hand tour and the like and run the mobile phone application programs.
However, when a certain mobile phone application program installed by each independent host in the cloud host needs to be updated, the cloud host needs to download resources such as a mobile phone application program installation package and the like for each independent host in turn, which takes a long time.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a resource downloading method, a device, a data system and a readable storage medium, and aims to solve the technical problem of long resource consumption of a cloud host in the prior art.
In order to achieve the above object, the present invention provides a resource downloading method, which includes the following steps:
obtaining and transmitting running numbers according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading target resources in the cloud host at any time;
and controlling the independent host to execute the downloading task of the target resource according to the concurrent running number.
In an embodiment, the network communication bandwidth information includes a maximum uplink bandwidth of the resource server, a maximum downlink bandwidth of the independent host, and a router bandwidth of the router; the resource server stores target resources;
The step of obtaining and transmitting the running number according to the network communication bandwidth information and the data size information of the target resource comprises the following steps:
obtaining the minimum instantaneous bandwidth under the current network environment according to the minimum value in the maximum uplink bandwidth, the maximum downlink bandwidth and the router bandwidth;
and obtaining and transmitting the running number according to the minimum instantaneous bandwidth and the data size information of the target resource.
In an embodiment, when the resource server includes at least two resources, before the step of obtaining the minimum instantaneous bandwidth in the current network environment according to the minimum value in the maximum uplink bandwidth, the maximum downlink bandwidth and the router bandwidth, the method further includes:
acquiring a first number of resource servers, an uplink bandwidth of each resource server, a router bandwidth, a second number of independent hosts in a cloud host, a third number of network ports of the cloud host and a port bandwidth of each network port;
obtaining maximum uplink bandwidth according to the first quantity and the uplink bandwidth;
and obtaining the maximum downlink bandwidth according to the second quantity, the third quantity and the port bandwidth.
In an embodiment, the step of controlling the independent host to execute the download task of the target resource according to the concurrent running number includes:
Selecting at least one host from all independent hosts which do not execute the downloading task as an executing host, wherein the number of the executing hosts is smaller than or equal to the number of concurrent running lines;
adding the execution host into a downloading task execution host queue so that the execution host executes the downloading task of the target resource; the downloading task execution host queue comprises a plurality of downloading channels, the number of the downloading channels is smaller than or equal to the number of concurrent running lines, and each independent host uses one downloading channel to download target resources;
after any independent host in the downloading task execution host queue finishes the downloading task, removing any independent host from the downloading task execution host queue, and returning to execute the step of selecting at least one from all independent hosts which do not execute the downloading task as an execution host, and circulating to all independent hosts in the cloud host to finish the downloading task.
In one embodiment, before the step of selecting at least one host from all independent hosts that do not perform the download task as the executing host, the method further includes:
monitoring the actual uplink bandwidth of each resource server, and obtaining the actual uplink bandwidth historical data of the resource server;
Obtaining an uplink bandwidth predicted value of the resource server at the target moment according to the actual uplink bandwidth historical data;
according to the uplink bandwidth predicted values of all the resource servers, updating the maximum uplink bandwidth at the target moment to obtain the updated concurrent running number;
after any independent host in the downloading task execution host queue finishes the downloading task, returning to execute the step of selecting at least one from all independent hosts which do not execute the downloading task as an executing host, and circulating to all independent hosts in the cloud host to finish the downloading task, wherein the step comprises the following steps:
after any independent host in the downloading task execution host queue finishes the downloading task, returning to execute the step of obtaining the uplink bandwidth predicted value of the resource server at the target moment according to the actual uplink bandwidth historical data, and circulating to all the independent hosts in the cloud host to finish the downloading task.
In one embodiment, the step of adding the execution host to the download task execution host queue to cause the execution host to execute the download task of the target resource includes:
adding the execution host to an idle download channel in a download task execution host queue, and controlling a download task thread corresponding to the download channel to send a task execution instruction to the execution host in a wake-up state; the downloading task threads are in one-to-one correspondence with the downloading channels;
Controlling the downloading task thread to switch from the awakening state to the dormant state;
after any independent host in the downloading task execution host queue finishes the downloading task, returning to execute the step of selecting at least one from all independent hosts which do not execute the downloading task as an executing host, and circulating to all independent hosts in the cloud host to finish the downloading task, wherein the step comprises the following steps:
receiving identification information sent by any independent host in a downloading task execution host queue after completing the downloading task;
in response to receiving the identification information, removing any independent host from the downloading task execution host queue to obtain an idle downloading channel;
and controlling the download task thread corresponding to the idle download channel to switch from the dormant state to the wake state, returning to the step of selecting at least one execution host from all independent hosts which do not execute the download task, and circulating to the step of completing the download task by all the independent hosts in the cloud host.
In a second aspect, the present invention also provides a resource downloading device, where the resource downloading device includes:
the data calculation module is used for obtaining and transmitting the running number according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading target resources in the cloud host at any time;
And the task control module is used for controlling the independent host to execute the downloading task of the target resource according to the concurrent running number.
In a third aspect, the present invention also provides a data system comprising:
at least one resource server, each resource server stores target resources;
a router;
the cloud host is internally provided with a plurality of independent hosts; and
the resource downloading device comprises a memory, a processor and a resource downloading program stored in the memory and capable of running on the processor, wherein the resource downloading program is configured to realize the steps of the method so as to download the target resource into all independent hosts when the cloud host sends the target resource downloading request to the resource server.
In one embodiment, the resource server is a network file system server.
In a fourth aspect, the invention also provides a computer readable storage medium comprising executable program code, wherein the program code when executed by a processor performs the steps of the method as described above.
The invention provides a resource downloading method, which calculates the number of independent hosts in a cloud host allowing to simultaneously execute downloading tasks in the current communication environment through network communication environment information, namely the concurrent running number, so as to control the independent hosts in the cloud host to execute the downloading tasks according to the concurrent running number pipeline until all the independent hosts in the cloud host complete the downloading tasks, thereby improving the efficiency of downloading resources by a plurality of independent hosts in the cloud host and shortening the downloading time.
Drawings
FIG. 1 is a schematic diagram of an embodiment of a data system according to the present invention;
FIG. 2 is a schematic diagram of a resource downloading device according to the present invention;
FIG. 3 is a flowchart of a first embodiment of a resource downloading method according to the present invention;
FIG. 4 is a detailed flowchart of step S102 of the second embodiment of the resource downloading method of the present invention;
FIG. 5 is a flowchart of a third embodiment of a resource downloading method according to the present invention;
FIG. 6 is a flowchart of a fourth embodiment of a resource downloading method according to the present invention;
fig. 7 is a schematic diagram of functional modules of an embodiment of the resource downloading device of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
A cloud host is a part that virtualizes multiple similar independent hosts on one host, each of which may have a separate operating system. For example, a cloud host can virtualize more than tens of hundred mobile phones. Each virtual mobile phone has an independent operating system, so that each virtual mobile phone can be provided with resources such as mobile phone application programs such as the same hand tour and the like, and the mobile phone application programs can be operated. In the prior art, a cloud host can be adopted to test a certain mobile phone application program so as to improve the test efficiency, or a hand game studio provides hand game service through the cloud host.
However, when a certain mobile phone application program installed by each independent host in the cloud host needs to be updated, particularly for a hand-tour studio or a mobile phone APP test department, the starting speed of the mobile phone application program in the cloud host is required, if an update installation package or an installation package of the mobile phone application program only has hundreds of M, the working efficiency of the hand-tour studio or the mobile phone APP test department is not greatly affected, but for mobile phone application programs such as large games of several G, each virtual independent host in one cloud host sequentially downloads and updates the mobile phone application program, the cloud host needs to sequentially download resources such as the mobile phone application installation package for each independent host, and the time consumption is long.
Therefore, the embodiment of the application provides a resource downloading method, which calculates the number of independent hosts in a cloud host allowing to simultaneously execute downloading tasks in the current communication environment through a network communication environment, namely the concurrent running number, so as to control the independent hosts in the cloud host to execute the downloading tasks according to the concurrent running number pipeline until all the independent hosts in the cloud host complete the downloading tasks.
The inventive concept of the present application is further elucidated below in connection with a few specific embodiments.
Referring to fig. 1, the present invention shows a data system including at least one resource server 10, a router 20, a cloud host 40, and a resource download device.
The resource server 10 stores target resources that are pre-downloaded by the cloud host 40. For resources such as mobile phone application programs or large data files, the resource server 10 may be downloaded and stored in advance from a website such as a mobile phone application program web. In an embodiment, the resource server 10 may be an NFS (Network File System ) server. The NFS server can enable computers with different operating systems, such as independent hosts like a virtual mobile phone in the cloud host 40 to share files or data, and by taking the streaming media server blade as a network file system client to plug in a network file system server, the storage capacity of the streaming media server is greatly expanded.
The cloud host 40 is a virtualization technology device similar to a VPS (Virtual Private Server ) host, and the VPS adopts virtual software to virtualize a plurality of parts similar to independent hosts on one host, so that single machine multi-user can be realized, each part can be used as an independent operating system, and the management method is the same as that of the host. Each independent host has a respective operating system. It will be appreciated that the operating system of the stand-alone host may be a cell phone operating system or a computer operating system, i.e., the stand-alone host within the cloud host 40 may be considered a cell phone or a personal computer.
The resource download device may be installed on the cloud host 40, and may also be installed on the coordination server 30 connected to the router. The following specifically describes an example of the resource download device as the coordination server 30.
Referring to fig. 1, fig. 1 also shows a schematic structural diagram of a recommended electronic device of a resource downloading method according to an embodiment of the present invention.
The resource downloading device includes: at least one processor 301, a memory 302 and a resource download program stored on the memory and executable on the processor, the resource download program being configured to implement the steps of the resource download method as before.
Processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 301 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central ProcessingUnit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 301 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen.
Memory 302 may include one or more computer-readable storage media, which may be non-transitory. Memory 302 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 302 is used to store at least one instruction for execution by processor 301 to implement the resource download method provided by the method embodiments of the present application.
The resource download device also includes a communication interface 303. The processor 301, the memory 302 and the communication interface 303 may be connected by a bus or signal lines. The respective peripheral devices may be connected to the communication interface 303 through a bus, signal line, or circuit board.
The communication interface 303 may be used to connect at least one peripheral device associated with an I/O (Input/Output) to the processor 301 and the memory 302. In some embodiments, processor 301, memory 302, and communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 301, the memory 302, and the communication interface 303 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.
Those skilled in the art will appreciate that the architecture shown in fig. 1 is not limiting of the resource download device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The application provides a first embodiment of a resource downloading method. Referring to fig. 3, fig. 3 is a flowchart illustrating a first embodiment of a resource downloading method according to the present application.
In this embodiment, the method includes the following steps:
step S101, obtaining and transmitting running numbers according to network communication bandwidth information and data size information of target resources; the concurrent running number is the maximum number of independent hosts for simultaneously downloading the target resources in the cloud host at any time.
Multiple independent hosts are virtualized in the cloud host 40, each of which has an independent operating system, such as a mobile phone operating system, where each independent host can be considered a mobile phone. The following further describes the case where the independent host is regarded as a mobile phone. Those skilled in the art will readily appreciate that each individual host has a computer operating system installed therein, and the like, based on the teachings of the present application.
All or at least some of the individual hosts within cloud host 40 need to download the same target resource, which may be a data file, such as an installation package for a mobile game application or a larger compressed package. When the cloud host 40 sends a target resource download request to the resource server 10, the coordination server 30 connected with the router 20 listens for or receives the target resource download request, and then obtains and transmits the running number according to the network communication bandwidth information and the data size information of the target resource. The number of concurrent operations is any time, and the maximum number of independent hosts capable of simultaneously downloading the target resources in the cloud host 40. At this time, the independent host of the cloud host 40 performs the download task according to the concurrent running number pipeline, and can maximally utilize the network environment under the current data system, so as to improve the resource download efficiency.
In some embodiments, the network communication bandwidth information includes a maximum upstream bandwidth of the resource server 10, a maximum downstream bandwidth of the independent hosts, and a router bandwidth of the router 20.
At this time, step S101 includes:
(1) And obtaining the minimum instantaneous bandwidth under the current network environment according to the minimum value in the maximum uplink bandwidth, the maximum downlink bandwidth and the router bandwidth.
(2) And obtaining and transmitting the running number according to the minimum instantaneous bandwidth and the data size information of the target resource.
Specifically, in the data system, the instantaneous traffic cannot exceed any of the maximum uplink bandwidth of any resource server 10, the maximum downlink bandwidth of the independent hosts, and the minimum value of the router bandwidth of the router 20, and each independent host can only download one target resource at the same time, so that the number of independent hosts capable of concurrently executing the download task in the current network environment can be obtained by dividing the minimum value of the three by the data size information of the target resource.
In one embodiment, the number of concurrent runs may be determined by:
；
wherein,for concurrent run number, +.>For a first number of resource servers 10, < >>Upstream bandwidth for each resource server 10, < > for each resource server 10 >Router bandwidth for router 20, < >>A second number of independent hosts within cloud host 40, < > is provided>A third number of network ports for cloud host 40, < >>Port bandwidth for each network port.
Thus, prior to step S101, the method may further comprise:
step S10, acquiring a first number of resource servers 10Upstream bandwidth of each resource server 10 +.>Router bandwidth of router 20 +.>Second number of independent hosts within cloud host 40 +.>Third number of network ports of cloud host 40 +.>And port bandwidth +/for each network port>。
Coordination server 30 may manually configure the data based on the data entered by the administrator.
And step S20, obtaining the maximum uplink bandwidth according to the first quantity and the outlet flow.
The maximum uplink bandwidth is。
And step S30, obtaining the maximum downlink bandwidth according to the second quantity, the third quantity and the port bandwidth.
The maximum bandwidth is。
Step S102, the independent host is controlled to execute the downloading task of the target resource according to the concurrent running number.
After calculating and transmitting the running number, the coordination server 30 can control the corresponding number of independent hosts of the cloud host 40 to execute the downloading task of the target resource at any moment, and in a certain batch of independent hosts executing the downloading task, namely, the downloading task executing host queues, after any independent host is executed, the coordination server 30 selects an independent host from the rest independent hosts to immediately supplement the emptied downloading channel to execute the downloading resource. Since the resource servers 10 include a plurality of resource servers, all the resource servers 10 can be uniformly distributed to each independent host before step S102, so as to distribute the corresponding resource servers 10 to each independent host, thereby avoiding occupying the outlet flow of a certain resource server 10 in a centralized manner. When each independent host performs the download task, a connection is established with the allocated resource server 10 to perform the download task.
It should be noted that at any time, the number of independent hosts performing the download task may beA table, wherein,. Preferably, the->。
In this embodiment, based on the above steps, the independent hosts in the cloud host 40 select a corresponding number of independent hosts at any time to simultaneously execute the download task of the target resource based on the concurrent running number, thereby improving the download efficiency of the independent hosts in the cloud host 40 for downloading the same target resource.
On the basis of the first embodiment of the resource downloading method, a second embodiment of the resource downloading method is also provided.
Referring to fig. 4, in this embodiment, step S102 specifically includes the following steps:
and step A10, selecting at least one host from the independent hosts which do not execute the downloading task as an executing host, wherein the number of the executing hosts is smaller than or equal to the number of concurrent running lines.
Step A20, adding the execution host to a downloading task execution host queue so that the execution host executes the downloading task of the target resource; the downloading task execution host queue comprises a plurality of downloading channels, the number of the downloading channels is smaller than or equal to the number of concurrent running lines, and each independent host uses one downloading channel to download the target resource.
And step A30, after any independent host in the downloading task execution host queue finishes the downloading task, removing any independent host from the downloading task execution host queue, returning to the execution step A10, and circulating to all independent hosts in the cloud host to finish the downloading task.
Specifically, the concurrent run number defines the number of independent hosts that can simultaneously perform the download tasks. I.e. the number of independent hosts in the download task execution host queue.
When the number of concurrent running is 5, that is, the download task executing host queue has 5 download channels at most, and at most 5 independent hosts can be selected as executing hosts to be added into the download task executing host queue, so that each independent host occupies one download channel to execute the download task. After any independent host in the downloading task execution host queue completes the downloading task, the independent host exits from the downloading task execution host queue, and a corresponding number of idle downloading channels are reserved. At this time, a corresponding number of execution hosts are selected from the independent hosts which do not execute the downloading task to be supplemented, and the downloading target resources of the vacated downloading channel are occupied.
After each independent host downloads the target resource, the target resource can be started to be used or operated. If the target resource is a hand-tour program, the independent host computer can start to install and run the hand-tour program when finishing downloading the hand-tour program. The coordination server 30 selects an execution host from the independent hosts that do not install or update the hand-tour program to control the execution host to perform the download task.
In one embodiment, 88 "handsets" are separated from the cloud host 40, and a number of 5 are calculated and shipped. Then the first time step a10 is performed, the 88 independent hosts are all independent hosts that do not perform the download task. At this time, the coordination server 30 may randomly select 5 "handsets" as execution hosts to execute the download task of the download hand-stream installation package. One "mobile phone" finishes downloading the hand-tour installation package before the other 4 "mobile phones", and at this time, the coordination server 30 selects 1 "mobile phone" from the remaining 83 "mobile phones" as an execution host, and the execution host immediately executes the downloading task. The above steps are looped until all independent hosts within cloud host 40 have completed the download task.
On the basis of the first embodiment and the second embodiment of the resource downloading method, a third embodiment of the resource downloading method is also provided. Referring to fig. 5, fig. 5 is a flowchart illustrating a third embodiment of a resource downloading method according to the present invention.
In this embodiment, the resource downloading method includes:
step S201, obtaining and transmitting running numbers according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading the target resources in the cloud host at any time.
Step S202, monitoring the actual upstream bandwidth of each resource server 10, and obtaining the actual upstream bandwidth history data of the resource server 10.
And step 203, obtaining an uplink bandwidth predicted value of the resource server at the target moment according to the actual uplink bandwidth historical data.
And the running number is the calculated theoretical value, and in actual running, the actual quality of the network is also required to be combined for adjustment and running number due to network fluctuation and the like.
Therefore, in this step, the actual upstream bandwidth of the resource server 10 may be monitored in real time by the monitoring thread to form the actual upstream bandwidth history data of the resource server 10, so that the outlet flow of the resource server 10 at the target time is predicted based on the actual upstream bandwidth history data. In one embodiment, the supervisory thread may also time, such as collecting the actual upstream bandwidth of the resource server 10 every minute
In one embodiment, the outlet flow may be predicted by a weighted moving average method. As at time t, the outlet traffic of the resource server 10 is:
；
the weight of the exit flow of the mth nfs machine at t-1; />Weight of the mth nfs machine outlet flow at t-2; similarly, the case of->Is the weight of the mth nfs machine outlet flow of the t-n stage.
For the last 1 minute mth nfs machine actually upstream bandwidth value,/for>，/>And->Respectively are provided withRepresenting the actual upstream bandwidth for the first 2 minutes, the first 3 minutes up to the first n minutes. t-1 is 1 minute before the indicated distance target time t. n is the number of predicted time periods, e.g., n=1440.
And is also provided with+/>+...+/>=1. Optionally, a->For an equal ratio queue, initialize value +.>Is->In order->Is->Up to->Is->。
From the above equation, the exit traffic of any resource server 10 at time t can be predicted.
Step S204, according to the uplink bandwidth predicted values of all the resource servers 10, the maximum uplink bandwidth at the target moment is updated, and the updated concurrent running number is obtained.
At the target time t, the updated concurrent running number can be calculated by the following formula:
。
step S205, selecting at least one host from the independent hosts which do not execute the downloading task as an executing host, wherein the number of the executing hosts is smaller than or equal to the concurrent running number.
Step S206, adding the execution host to a download task execution host queue so that the execution host executes the download task of the target resource; the downloading task execution host queue comprises a plurality of downloading channels, the number of the downloading channels is smaller than or equal to the number of concurrent running lines, and each independent host uses one downloading channel to download the target resource.
Step S207, after any independent host in the downloading task execution host queue finishes the downloading task, returning to the execution step S203, and circulating to all independent hosts in the cloud host to finish the downloading task.
In one embodiment, 88 "handsets" are separated from the cloud host 40, and a number of 5 are calculated and shipped. At time 1, the coordination server 30 may randomly select 5 "handsets" to perform the downloading task of downloading the hand-stream installation package. However, the coordination server 30 predicts the actual upstream bandwidth of the resource server 10 and transmits 4 pieces of data at time 2. And downloading the hand-tour installation package is finished before the other 4 'mobile phones' in one 'mobile phone'. At this time, the coordination server 30 selects 0 "handsets" from the remaining 83 "handsets" as the download task, and at this time, a total of 4 independent hosts execute the download task. The above steps are looped until all independent hosts within cloud host 40 have completed the download task.
In this step, the coordination server 30 calculates the number of concurrent runs based on the updateAfter that, the download task of the target resource can be executed by the corresponding number of independent hosts of the cloud host 40 at any moment, and after any independent host of a certain batch of independent hosts executing the download task is executed, the coordination server 30 calculates the number of concurrent operations according to the updated number of concurrent operations according to the actual uplink bandwidth data at the moment again >Select +.>The independent host computer immediately supplements the vacated download channel to execute the download resource.
In this embodiment, based on the above steps, the calculated and transmitted running number can be more attached to the actual uplink bandwidth of the resource server 10, so that the downloading efficiency of the cloud host 40 is further improved as much as possible under the condition that the resource downloading task in the cloud host 40 is in line with the actual uplink bandwidth of the resource server 10.
On the basis of the first to third embodiments of the resource downloading method of the present invention, a fourth embodiment of the resource downloading method of the present invention is also provided. Referring to fig. 6, fig. 6 is a flowchart illustrating a fourth embodiment of a resource downloading method according to the present invention.
Step S301, obtaining and transmitting running numbers according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading the target resources in the cloud host at any time.
Step S302, selecting at least one host from all independent hosts which do not execute the downloading task as an executing host, wherein the number of the executing hosts is smaller than or equal to the number of concurrent running lines.
Step S303, adding the execution host to an idle download channel in a download task execution host queue, and controlling a download task thread corresponding to the download channel to send a task execution instruction to the execution host in an awake state; and the downloading task threads are in one-to-one correspondence with the downloading channels.
Step S304, the download task thread is controlled to switch from the wake-up state to the sleep state.
Step S305, receiving the identification information sent by any independent host in the download task execution host queue after completing the download task.
And step S306, in response to receiving the identification information, removing any independent host from the downloading task execution host queue to obtain an idle downloading channel.
Step 307, the download task thread corresponding to the idle download channel is controlled to switch from the sleep state to the wake state, and returns to step 302 to circulate to all independent hosts in the cloud host to complete the download task.
In this embodiment, the coordination server 30 establishes a download task thread according to the download task execution host queue, where the download task thread has an awake state and a sleep state. In the wake state, each download task thread is configured to send task execution instructions to an independent host. And after the task execution instruction is sent, the downloading task thread is switched from the awakening state to the dormant state. Each download task thread wakes up again after receiving the identification information sent by any execution host after completing the download task, and continues to send task execution instructions to the corresponding execution host.
In one embodiment, 88 "handsets" are separated from the cloud host 40, and a number of 5 are calculated and shipped. And at this time, the coordination server 30 can randomly select 5 "mobile phones" to execute the downloading task of downloading the hand-stream installation package. The coordination server 30 also establishes 5 download task threads. The 5 download task threads send task execution instructions to any one of the 5 'mobile phones' respectively, and then enter a dormant state. And 5 independent hosts start downloading the target resources after receiving the task execution instruction. After any one of the independent hosts completes the downloading task, the coordination server 30 selects 1 "mobile phone" from the remaining 83 "mobile phones" as an executing host, wakes up a downloading task thread, the downloading task thread sends a task executing instruction to the executing host, and the executing host immediately executes the downloading task. The above steps are looped until all independent hosts within cloud host 40 have completed the download task.
In this embodiment, the execution host is controlled to execute the download task by the download task thread established to switch between the sleep and awake states, so as to ensure that the number of execution hosts executing the download task coincides with the number of concurrent hosts at any time.
In addition, the invention also provides a resource downloading device, referring to fig. 6, the resource downloading device comprises:
the data calculation module 100 is configured to obtain and send an operation number according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading target resources in the cloud host at any time;
the task control module 200 is configured to control the independent host to execute the download task of the target resource according to the concurrent running number.
In an embodiment, the network communication bandwidth information includes a maximum uplink bandwidth of the resource server, a maximum downlink bandwidth of the independent host, and a router bandwidth of the router;
the data calculation module 100 includes:
the minimum instantaneous bandwidth calculation unit is used for obtaining the minimum instantaneous bandwidth in the current network environment according to the minimum value in the maximum uplink bandwidth, the maximum downlink bandwidth and the router bandwidth;
and the concurrent running line number calculation unit is used for obtaining the concurrent running line number according to the minimum instantaneous bandwidth and the data size information of the target resource.
In an embodiment, when the resource server includes at least two, the resource downloading device further includes:
the data acquisition module is used for acquiring the first number of the resource servers, the uplink bandwidth of each resource server, the router bandwidth, the second number of the independent hosts in the cloud host, the third number of the network ports of the cloud host and the port bandwidth of each network port; obtaining a maximum uplink bandwidth according to the first quantity and the uplink bandwidth; and obtaining the maximum downlink bandwidth according to the second quantity, the third quantity and the port bandwidth.
In one embodiment, the task control module 200 includes:
the host selecting unit is used for selecting at least one host from all independent hosts which do not execute the downloading task as an executing host, wherein the number of the executing hosts is smaller than or equal to the number of concurrent running lines;
the queue adding unit is used for adding the execution host into a download task execution host queue so as to enable the execution host to execute the download task of the target resource, wherein the download task execution host queue comprises a plurality of download channels, the number of the download channels is smaller than or equal to the concurrent running number, and each independent host uses one download channel to download the target resource;
and the circulation execution unit is used for removing any independent host from the downloading task execution host queue after any independent host in the downloading task execution host queue completes the downloading task, returning to execute the step of selecting at least one execution host from all independent hosts which do not execute the downloading task, and circulating to all the independent hosts in the cloud host to complete the downloading task.
In an embodiment, the resource downloading device further includes:
the flow monitoring unit is used for monitoring the actual uplink bandwidth of each resource server 10 and obtaining the actual uplink bandwidth historical data of the resource server 10;
The flow prediction unit is used for obtaining an uplink bandwidth predicted value of the resource server at the target moment according to the actual uplink bandwidth historical data;
the data updating unit is used for updating the maximum uplink bandwidth at the target moment according to the uplink bandwidth predicted values of all the resource servers to obtain updated concurrent running numbers;
the task control module 200 is further configured to return to execute the step of obtaining the predicted value of the uplink bandwidth of the resource server at the target moment according to the actual uplink bandwidth history data after any independent host in the queue of the download task execution host completes the download task, and circulate to all the independent hosts in the cloud host to complete the download task.
In an embodiment, the task control module 200 is further configured to add the execution host to an idle download channel in the download task execution host queue, and control a download task thread corresponding to the download channel to send a task execution instruction to the execution host in an awake state; after the task execution instruction is sent, the task thread is downloaded and switched from the awakening state to the dormant state; the downloading task threads are in one-to-one correspondence with the downloading channels; receiving identification information sent by any independent host in a downloading task execution host queue after completing the downloading task; in response to receiving the identification information, removing any independent host from the downloading task execution host queue to obtain an idle downloading channel; and controlling the download task thread corresponding to the idle download channel to switch from a dormant state to an awake state, returning to select at least one host from all independent hosts which do not execute the download task, and circulating to all independent hosts in the cloud host to complete the download task as a step of executing the host.
Furthermore, embodiments of the present application also propose a computer-readable storage medium comprising executable program code which, when executed by a processor, implements the steps of the resource downloading method as above. Therefore, a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium according to the present application, please refer to the description of the method embodiments of the present application. As an example, the program instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of computer programs, which may be stored on a computer-readable storage medium, and which, when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random access Memory (Random AccessMemory, RAM), or the like.
It should be further noted that the above-described apparatus embodiments are merely illustrative, and that the units described as separate units may or may not be physically separate, and that units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the embodiment of the device provided by the invention, the connection relation between the modules represents that the modules have communication connection, and can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the present invention may be implemented by means of software plus necessary general purpose hardware, or of course by means of special purpose hardware including application specific integrated circuits, special purpose CPUs, special purpose memories, special purpose components, etc. Generally, functions performed by computer programs can be easily implemented by corresponding hardware, and specific hardware structures for implementing the same functions can be varied, such as analog circuits, digital circuits, or dedicated circuits. However, a software program implementation is a preferred embodiment for many more of the cases of the present invention. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a readable storage medium, such as a floppy disk, a usb disk, a removable hard disk, a Read-only memory (ROM), a random-access memory (RAM, randomAccessMemory), a magnetic disk or an optical disk of a computer, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.
Claims (8)
1. A method of resource download, the method comprising the steps of:
obtaining and transmitting running numbers according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading the target resources in the cloud host at any time;
controlling an independent host to execute the downloading task of the target resource according to the concurrent running number;
the independent host computer is controlled to execute the downloading task of the target resource according to the concurrent running number, and the method comprises the following steps:
selecting at least one host from all independent hosts which do not execute the downloading task as an executing host, wherein the number of the executing hosts is smaller than or equal to the concurrent running number;
adding the execution host to an idle download channel in a download task execution host queue, and controlling a download task thread corresponding to the download channel to send a task execution instruction to the execution host in an awake state; the downloading task threads are in one-to-one correspondence with the downloading channels; the downloading task execution host queue comprises a plurality of downloading channels, the number of the downloading channels is smaller than or equal to the concurrent running number, and each independent host uses one downloading channel to download the target resource;
Controlling the downloading task thread to switch from the awakening state to the dormant state;
receiving identification information sent by any independent host in a downloading task execution host queue after completing the downloading task;
in response to receiving the identification information, removing any independent host from the downloading task execution host queue to obtain an idle downloading channel;
and controlling the download task thread corresponding to the idle download channel to switch from a dormant state to an awake state, returning to select at least one from all independent hosts which do not execute the download task, and circulating to all the independent hosts in the cloud host as a step of executing the host to complete the download task.
2. The resource downloading method as claimed in claim 1, wherein the network communication bandwidth information includes a maximum upstream bandwidth of a resource server, a maximum downstream bandwidth of the independent host, and a router bandwidth of a router; wherein the resource server stores the target resource;
the step of obtaining and transmitting the running number according to the network communication bandwidth information and the data size information of the target resource comprises the following steps:
Obtaining the minimum instantaneous bandwidth in the current network environment according to the minimum value in the maximum uplink bandwidth, the maximum downlink bandwidth and the router bandwidth;
and obtaining and transmitting the running number according to the minimum instantaneous bandwidth and the data size information of the target resource.
3. The method for downloading resources according to claim 2, wherein when the resource server includes at least two servers, before the step of obtaining the minimum instantaneous bandwidth in the current network environment according to the minimum value among the maximum upstream bandwidth, the maximum downstream bandwidth and the router bandwidth, the method further includes:
acquiring a first number of the resource servers, an uplink bandwidth of each resource server, the router bandwidth, a second number of the independent hosts in the cloud host, a third number of network ports of the cloud host and a port bandwidth of each network port;
obtaining a maximum uplink bandwidth according to the first quantity and the uplink bandwidth;
and obtaining the maximum downlink bandwidth according to the second quantity, the third quantity and the port bandwidth.
4. The resource downloading method as claimed in claim 1, wherein at least one of the independent hosts that do not perform the downloading task is selected as the host, and the method further comprises, before the step of executing the host:
Monitoring the actual uplink bandwidth of each resource server, and obtaining the actual uplink bandwidth historical data of the resource server;
obtaining an uplink bandwidth predicted value of the resource server at a target moment according to the actual uplink bandwidth historical data;
updating the maximum uplink bandwidth at the target moment according to the uplink bandwidth predicted values of all the resource servers to obtain updated concurrent running numbers;
and controlling the download task thread corresponding to the idle download channel to switch from a dormant state to an awake state, returning to a step of selecting at least one host from all independent hosts which do not execute the download task as an executing host, and circulating to all the independent hosts in the cloud host to complete the download task, wherein the method comprises the following steps:
and controlling the download task thread corresponding to the idle download channel to switch from a dormant state to an awake state, and returning to execute the step of obtaining the uplink bandwidth predicted value of the resource server at the target moment according to the actual uplink bandwidth historical data, and circulating to all the independent hosts in the cloud host to complete the download task.
5. A resource downloading apparatus, characterized in that the resource downloading apparatus comprises:
The data calculation module is used for obtaining and transmitting the running number according to the network communication bandwidth information and the data size information of the target resource; the concurrent running number is the maximum number of independent hosts for simultaneously downloading target resources in the cloud host at any time;
the task control module is used for controlling the independent host to execute the downloading task of the target resource according to the concurrent running number;
the task control module is specifically configured to select at least one host from all independent hosts that do not execute the download task, where the number of execution hosts is less than or equal to the number of concurrent operations; adding the execution host to an idle download channel in a download task execution host queue, and controlling a download task thread corresponding to the download channel to send a task execution instruction to the execution host in an awake state; the downloading task threads are in one-to-one correspondence with the downloading channels; the downloading task execution host queue comprises a plurality of downloading channels, the number of the downloading channels is smaller than or equal to the concurrent running number, and each independent host uses one downloading channel to download the target resource; controlling the downloading task thread to switch from the awakening state to the dormant state; receiving identification information sent by any independent host in a downloading task execution host queue after completing the downloading task; in response to receiving the identification information, removing any independent host from the downloading task execution host queue to obtain an idle downloading channel; and controlling the download task thread corresponding to the idle download channel to switch from a dormant state to an awake state, returning to select at least one from all independent hosts which do not execute the download task, and circulating to all the independent hosts in the cloud host as a step of executing the host to complete the download task.
6. A data system, comprising:
at least one resource server, each resource server stores target resources;
a router;
the cloud host is internally provided with a plurality of independent hosts; and
a resource downloading device comprising a memory, a processor and a resource downloading program stored on the memory and executable on the processor, the resource downloading program being configured to implement the steps of the method of any of claims 1 to 4.
7. The data system of claim 6, wherein the resource server is a network file system server.
8. A computer readable storage medium comprising executable program code, wherein the program code when executed by a processor performs the steps of the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110817580.3A CN113596125B (en) | 2021-07-21 | 2021-07-21 | Resource downloading method, device, data system and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110817580.3A CN113596125B (en) | 2021-07-21 | 2021-07-21 | Resource downloading method, device, data system and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113596125A CN113596125A (en) | 2021-11-02 |
| CN113596125B true CN113596125B (en) | 2023-11-14 |
Family
ID=78248345
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110817580.3A Active CN113596125B (en) | 2021-07-21 | 2021-07-21 | Resource downloading method, device, data system and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113596125B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101963946A (en) * | 2010-10-11 | 2011-02-02 | 中兴通讯股份有限公司 | Method and device for supporting a plurality of mobile terminals to download software version via one computer |
| CN105323106A (en) * | 2014-08-04 | 2016-02-10 | 中兴通讯股份有限公司 | Software upgrade management method, equipment and system |
| CN106209452A (en) * | 2016-07-11 | 2016-12-07 | 福建联迪商用设备有限公司 | It is applied to wireless downloading installation method and the system of smart machine |
| US10404700B1 (en) * | 2015-08-27 | 2019-09-03 | Cadence Design Systems, Inc. | Concurrent design process |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7379990B2 (en) * | 2002-08-12 | 2008-05-27 | Tsao Sheng Ted Tai | Distributed virtual SAN |
-
2021
- 2021-07-21 CN CN202110817580.3A patent/CN113596125B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101963946A (en) * | 2010-10-11 | 2011-02-02 | 中兴通讯股份有限公司 | Method and device for supporting a plurality of mobile terminals to download software version via one computer |
| CN105323106A (en) * | 2014-08-04 | 2016-02-10 | 中兴通讯股份有限公司 | Software upgrade management method, equipment and system |
| US10404700B1 (en) * | 2015-08-27 | 2019-09-03 | Cadence Design Systems, Inc. | Concurrent design process |
| CN106209452A (en) * | 2016-07-11 | 2016-12-07 | 福建联迪商用设备有限公司 | It is applied to wireless downloading installation method and the system of smart machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113596125A (en) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8910169B2 (en) | Methods and systems to perform a computer task in a reduced power consumption state | |
| US20060143617A1 (en) | Method, apparatus and system for dynamic allocation of virtual platform resources | |
| CN109726005B (en) | Method, server system and computer readable medium for managing resources | |
| CN112148489A (en) | Game resource scheduling method, device, equipment and storage medium | |
| CN108064377B (en) | Management method and device for multi-system shared memory | |
| CN112380020A (en) | Computing power resource allocation method, device, equipment and storage medium | |
| CN104252390A (en) | Resource scheduling method, device and system | |
| CN103810015A (en) | Virtual machine establishing method and equipment | |
| US20150301914A1 (en) | Dynamically Limiting Bios Post For Effective Power Management | |
| US10614542B2 (en) | High granularity level GPU resource allocation method and system | |
| US20170097854A1 (en) | Task placement for related tasks in a cluster based multi-core system | |
| CN105335211A (en) | System and method for dispatching FPGA (Field Programmable Gate Array) accelerator based on Xen virtualized cluster | |
| CN115033352A (en) | Task scheduling method, device and equipment for multi-core processor and storage medium | |
| CN109729113B (en) | Method, server system and computer program product for managing dedicated processing resources | |
| US9436505B2 (en) | Power management for host with devices assigned to virtual machines | |
| CN113596125B (en) | Resource downloading method, device, data system and readable storage medium | |
| US9785463B2 (en) | Using per task time slice information to improve dynamic performance state selection | |
| US10157066B2 (en) | Method for optimizing performance of computationally intensive applications | |
| CN102438230B (en) | Terminal and data service processing method | |
| US20230289194A1 (en) | Sled level boot management control of compute nodes for context switching using boot controllers | |
| CN114116220A (en) | GPU (graphics processing Unit) sharing control method, GPU sharing control device and storage medium | |
| CN106484074B (en) | Nonvolatile memory device, nonvolatile memory method, computer system, and method for implementing standby or hibernation | |
| CN117632457A (en) | Method and related device for scheduling accelerator | |
| US20170052827A1 (en) | Using Multiple Central Processing Unit Cores for Packet Forwarding in Virtualized Networks | |
| CN114661465A (en) | Resource management method, device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |