CN112445577B - Container adding method, device, terminal equipment and storage medium - Google Patents
Container adding method, device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN112445577B CN112445577B CN202011384769.XA CN202011384769A CN112445577B CN 112445577 B CN112445577 B CN 112445577B CN 202011384769 A CN202011384769 A CN 202011384769A CN 112445577 B CN112445577 B CN 112445577B
- Authority
- CN
- China
- Prior art keywords
- container
- storage device
- capacity
- container scheduling
- containers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000004891 communication Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a container adding method, a device, a terminal device and a storage medium.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and apparatus for adding a container, a terminal device, and a storage medium.
Background
In the field of data processing technology, for a scenario where a plurality of different data needs to be collected, the data needs to be stored in a container form into different storage devices, for example: in the automatic driving technique, data like point cloud data, acceleration operation data, left turn operation data, right turn operation data, and the like need to be stored in different storage devices in the form of containers, respectively.
In general, a plurality of containers are stored in one storage device, but the capacity requirements of different containers are different, so that there may be some containers with small capacity requirements on each storage device, and when one container with high capacity requirements needs to be added, there may be no available capacity.
Disclosure of Invention
The invention mainly aims to provide a container adding method, a device, a terminal device and a storage medium, and aims to solve the technical problem that in the prior art, as a plurality of containers with small capacity requirements are arranged on each storage device, when a container with high capacity requirements to be added is arranged, the situation that no capacity is available is caused.
In order to achieve the above object, the present invention provides a container adding method comprising the steps of:
when a container adding instruction is received, determining the target capacity of a container to be added according to the container adding instruction, and acquiring the occupation information of the container on each storage device;
if the target capacity of the container to be added exceeds the residual capacity of each storage device, generating a plurality of container scheduling strategies according to the occupation information;
determining loss coefficients corresponding to each container scheduling strategy;
And selecting a target container scheduling strategy from the container scheduling strategies according to the loss coefficient, and scheduling the containers according to the target container scheduling strategy.
Optionally, the step of generating a plurality of container scheduling policies according to the occupancy information includes:
taking the storage equipment with the total capacity larger than or equal to the target capacity as the storage equipment to be selected;
taking the container with the occupied information smaller than the target capacity in each storage device to be selected as a container to be selected;
traversing the container to be selected, taking the traversed container to be selected as a container to be moved, and taking a storage device to which the container to be moved belongs as a current storage device;
calculating the residual capacity of the current storage device after the container to be moved is removed;
and when the residual capacity of the current storage device is larger than or equal to the target capacity, generating a container scheduling policy for moving the container to be moved to other storage devices except the current storage device and storing the container to be added to the current storage device.
Optionally, the step of determining the loss coefficient corresponding to each container scheduling policy includes:
acquiring the capacity of a container to be moved corresponding to each container scheduling policy, acquiring resource related information corresponding to each container scheduling policy, and acquiring equipment occupancy rate information corresponding to each container scheduling policy;
And calculating loss coefficients respectively corresponding to the container scheduling strategies according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to the container scheduling strategies.
Optionally, the step of obtaining the resource-related information corresponding to each container scheduling policy includes:
acquiring the data association degree between the containers, wherein the data association degree is used for reflecting the probability of simultaneously reading data between two containers;
traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy;
determining containers in each storage device corresponding to the current container scheduling policy;
calculating the resource correlation degree corresponding to each storage device according to the determined containers in each storage device and the data correlation degree between the containers, wherein the resource correlation degree is used for reflecting the probability that each container in the storage device needs to read data at the same time;
and carrying out average value calculation on the data correlation degrees corresponding to the storage devices respectively, and taking an average value calculation result as resource correlation information corresponding to the current container scheduling strategy.
Optionally, the step of obtaining the device occupancy rate information corresponding to each container scheduling policy includes:
Traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy;
determining containers in each storage device corresponding to the current container scheduling policy;
calculating the capacity occupancy rate corresponding to each storage device according to the determined containers in each storage device and the occupancy information of each container;
and carrying out product calculation on the capacity occupancy rates corresponding to the storage devices respectively, and taking the product calculation result as the device occupancy rate information corresponding to the current container scheduling strategy.
Optionally, the calculation is performed according to the capacity, the resource related information and the equipment occupancy rate information of the container to be moved corresponding to each container scheduling policy
The step of loss coefficient corresponding to each container scheduling strategy specifically comprises the following steps:
calculating loss coefficients respectively corresponding to each container scheduling strategy according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to each container scheduling strategy through a preset loss coefficient formula;
the preset loss coefficient formula is as follows:
W＝aX+bY+cZ+d
wherein a, b, c, d is a constant, W is a loss coefficient, X is a capacity of a container to be moved, Y is resource-related information, and Z is equipment occupancy information.
Optionally, before the step of calculating the loss coefficients corresponding to the container scheduling policies respectively according to the capacity, the resource related information and the equipment occupancy rate information of the container to be moved corresponding to the container scheduling policies through a preset loss coefficient formula, the container adding method further includes:
acquiring sample capacity, sample resource related information, sample equipment occupancy information and a sample loss coefficient;
and performing fitting operation on the initial loss coefficient formula through the sample capacity, the sample resource related information, the sample equipment occupancy rate information and the sample loss coefficient to obtain a preset loss coefficient formula.
In addition, in order to achieve the above object, the present invention also provides a container adding device including:
the system comprises an occupation information module, a storage device and a storage device, wherein the occupation information module is used for determining the target capacity of a container to be added according to a container adding instruction when receiving the container adding instruction, and acquiring the occupation information of the container on each storage device;
the policy generation module is used for generating a plurality of container scheduling policies according to the occupation information when the target capacity of the container to be added exceeds the residual capacity of each storage device;
the coefficient determining module is used for determining loss coefficients corresponding to each container scheduling strategy;
And the container scheduling module is used for selecting a target container scheduling strategy from the container scheduling strategies according to the loss coefficient and performing container scheduling according to the target container scheduling strategy.
In addition, to achieve the above object, the present invention also provides a terminal device including: a memory, a processor, and a container add-on program stored on the memory and executable on the processor, the container add-on program configured to implement the steps of the container add-on method as described above.
In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a container addition program which, when executed by a processor, implements the steps of the container addition method as described above.
According to the invention, a plurality of container scheduling strategies are generated according to the occupation information, the target container scheduling strategy is selected from the container scheduling strategies according to the loss coefficient, and the container scheduling is carried out according to the target container scheduling strategy, so that part of containers can be scheduled to other storage devices according to the target container scheduling strategy, and the containers to be added can be added smoothly.
Drawings
FIG. 1 is a schematic flow chart of a first embodiment of a container adding method of the present invention;
FIG. 2 is a flowchart of step S20 in a second embodiment of the container adding method according to the present invention;
FIG. 3 is a flowchart of step S30 in a third embodiment of the container adding method according to the present invention;
FIG. 4 is a block diagram illustrating an embodiment of a container adding apparatus according to the present invention;
fig. 5 is a schematic structural diagram of a terminal device of a hardware running environment according to an embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a first embodiment of a container adding method according to the present invention.
In a first embodiment, the container adding method includes the steps of:
s10: when a container adding instruction is received, determining the target capacity of a container to be added according to the container adding instruction, and acquiring the occupation information of the container on each storage device.
It should be understood that the method of the present embodiment is executed by a terminal device, which may be a server, a personal computer, or the like, and the present embodiment is not limited thereto.
It should be noted that, for the container add instruction, it may be issued by an upper computer, for example: when the upper computer needs to use a new type of data, the upper computer can estimate how large a container is needed by the data in a certain time range, then a container adding instruction is sent, and the container adding instruction carries the target capacity of the container to be added, so that the terminal equipment can determine the target capacity of the container to be added according to the container adding instruction when receiving the container adding instruction.
It will be appreciated that, since the containers are stored on the storage devices themselves, the containers are already divided into corresponding capacities, that is, the occupation information of the containers on the storage devices is known, and the occupation information of the containers on the storage devices can be obtained.
S20: and if the target capacity of the container to be added exceeds the residual capacity of each storage device, generating a plurality of container scheduling strategies according to the occupation information.
It should be noted that, when the target capacity of the container to be added does not exceed the remaining capacity of at least part of the storage devices, the capacity to be added may be directly stored in the part of the storage devices, but sometimes, there may be a case that the target capacity of the container to be added exceeds the remaining capacity of each storage device, at this time, a plurality of container scheduling policies may be generated according to the occupation information, and of course, each container scheduling policy may ensure that the container to be added completes the addition.
S30: and determining loss coefficients corresponding to each container scheduling strategy.
It will be appreciated that for each container scheduling policy, it is generally necessary to schedule the containers, and therefore, there is a cost, and the loss factor is a parameter reflecting the cost.
S40: and selecting a target container scheduling strategy from the container scheduling strategies according to the loss coefficient, and scheduling the containers according to the target container scheduling strategy.
In a specific implementation, because the loss coefficient corresponding to each container scheduling policy is determined, at this time, a target container scheduling policy can be selected from the container scheduling policies according to the loss coefficient, and the container scheduling is performed according to the target container scheduling policy.
According to the embodiment, a plurality of container scheduling strategies are generated according to the occupation information, a target container scheduling strategy is selected from the container scheduling strategies according to the loss coefficient, and container scheduling is carried out according to the target container scheduling strategy, so that part of containers can be scheduled to other storage devices according to the target container scheduling strategy, and the containers to be added can be added smoothly.
As shown in fig. 2, a second embodiment of the container adding method according to the present invention is provided based on the first embodiment, in this embodiment, in step S20, a plurality of container scheduling policies are generated according to the occupancy information, and specifically includes:
s21: and taking the storage equipment with the total capacity larger than or equal to the target capacity as the storage equipment to be selected.
It should be noted that, for the storage device, there may be a case that the total capacity is not uniform, and it is assumed that the storage devices have 3 storage devices, which are respectively denoted as storage device a, storage device B and storage device C, where the total capacity of storage device a is 8GB, the total capacity of storage device B is 16GB, and the total capacity of storage device C is 16GB, if the target capacity is 10GB, only storage device B and storage device C can add the container to be added, and the original container in storage device a cannot accommodate the container to be added at any rate, so that storage device B and storage device C may be used as storage devices to be selected.
S22: and taking the container with the occupied information smaller than the target capacity in each storage device to be selected as a container to be selected.
It can be understood that, generally, in order to ensure that the container to be added can be added smoothly, a part of containers in the storage device to be selected need to be scheduled to other storage devices, and if the occupation information of the scheduled containers is greater than or equal to the target capacity, the cost of container scheduling will be excessive at this time, so in this embodiment, the containers with the occupation information smaller than the target capacity in each storage device to be selected are used as the containers to be selected.
S23: traversing the container to be selected, taking the traversed container to be selected as a container to be moved, and taking the storage device to which the container to be moved belongs as the current storage device.
In a specific implementation, since the containers to be selected can be scheduled, the containers to be selected can be traversed, the traversed containers to be selected are used as containers to be moved, and the storage device to which the containers to be moved belong is used as the current storage device.
S24: and calculating the residual capacity of the current storage device after the container to be moved is removed.
It should be understood that, for the container to be moved, after being dispatched to other storage devices, the remaining capacity of the current storage device after the container to be moved is removed needs to be ensured, and the target capacity needs to be greater than or equal to the target capacity, so that the container to be added can be ensured to be stored in the current storage device, and therefore, the remaining capacity of the current storage device after the container to be moved is removed can be calculated first.
Assuming that the total capacity of the current storage device is 16GB, the current storage device has a container a and a container B, the occupation information of the container a is 4GB, the occupation information of the container B is 5GB, at this time, when the container to be moved is the container a, the remaining capacity of the current storage device after removing the container a is 11GB, and when the container to be moved is the container B, the storage capacity of the current storage device after removing the container B is 12GB.
S25: and when the residual capacity of the current storage device is larger than or equal to the target capacity, generating a container scheduling policy for moving the container to be moved to other storage devices except the current storage device and storing the container to be added to the current storage device.
Assuming that three storage devices are provided, namely a storage device A, a storage device B and a storage device C, wherein the total capacity of the storage device A is 8GB, a container X is stored in the storage device A, and the occupation information of the container X is 6GB; the total capacity of the storage equipment B is 8GB, wherein a container Y and a container Z are stored, the occupation information of the container Y is 2GB, and the occupation information of the container Z is 2GB; the total capacity of the storage equipment C is 8GB, the container W is stored in the storage equipment C, the occupation information of the container W is 3GB, the target capacity of the container to be added is 6GB, and the storage equipment A, the storage equipment B and the storage equipment C are selected because the total capacity of the three storage equipment is larger than the target capacity;
for the containers stored in the storage device a, the storage device B, and the storage device C, the container Y, the container Z, and the container W are each smaller than the target capacity, and thus, the container Y, the container Z, and the container W can each be a container to be selected;
When the container Y is a container to be moved, the remaining capacity of the storage device B after the container to be moved is 6GB, at this time, a container scheduling policy for moving the container Y to the storage device a and storing the container to be added to the storage device B may be generated, and a container scheduling policy for moving the container Y to the storage device C and storing the container to be added to the storage device B may be generated.
When the container Z is a container to be moved, the remaining capacity of the storage device B after the container to be moved is removed is 6GB, and at this time, a container scheduling policy for moving the container Z to the storage device a and storing the container to be added to the storage device B may be generated, and a container scheduling policy for moving the container Z to the storage device C and storing the container to be added to the storage device B may be generated.
When the container W is a container to be moved, the remaining capacity of the storage device C after the container to be moved is 8GB, at this time, a container scheduling policy may be generated to move the container W to the storage device B and store the container to be added to the storage device C.
According to the embodiment, a to-be-selected container is determined from containers in all storage devices through preset conditions, then the to-be-selected container is traversed, the traversed to-be-selected container is used as a to-be-moved container, the storage device to which the to-be-moved container belongs is used as a current storage device, the residual capacity of the current storage device after the to-be-moved container is removed is calculated, when the residual capacity of the current storage device is greater than or equal to the target capacity, the to-be-moved container is generated to move to other storage devices except the current storage device, and the to-be-added container is stored in a container scheduling strategy of the current storage device.
As shown in fig. 3, a third embodiment of the container adding method according to the present invention is provided based on the first embodiment, in this embodiment, step S30 specifically includes:
s31: the method comprises the steps of obtaining the capacity of a container to be moved corresponding to each container scheduling policy, obtaining resource related information corresponding to each container scheduling policy, and obtaining equipment occupancy rate information corresponding to each container scheduling policy.
It will be understood that, for each container scheduling policy, moving the container to be moved to a storage device other than the current storage device is involved, so that the capacity of the container to be moved corresponding to each container scheduling policy is the capacity of the container to be moved in each container scheduling policy, and since the movement of the container causes that data is difficult to be normally used during the movement, and the movement of the container also occupies data processing resources, for the container to be moved, the larger the capacity of the container to be moved, the larger the cost will be, and accordingly, the larger the loss coefficient will be.
In a specific implementation, since each container scheduling policy corresponds to a different container storage form, that is, different container scheduling policies may make the containers stored in each storage device have a distinction, and when data processing is performed on the data in the containers, the situation that the data in the containers a need to be read simultaneously may be related to, and if the data in the containers a are image data acquired through a camera in an unmanned process, the data in the containers B are point cloud data acquired through radar in an unmanned process, and the data in the containers C are position data acquired through GPS positioning in an unmanned process, because the data in the containers a, B and C are usually used for training or analyzing an automatic driving model in the unmanned process, the data in the containers a, B and C need to be read simultaneously in many cases, but if the data in the containers a, B and C are in the same storage device, at the same time, the data in the containers a, B and C can only be read sequentially in a similar manner, but because the data in the containers a data amount in the containers a is usually larger than the containers B and C are acquired, that is, the relative information loss in the same serial process can be reflected in the same device, that the relative information is larger, and the relative information can be stored in the same device.
Accordingly, in order to facilitate quick acquisition of the resource-related information, in this embodiment, the data association degree between each container may be acquired first, where the data association degree is used to reflect the probability that data needs to be read simultaneously between two containers, then each container scheduling policy is traversed, the traversed container scheduling policy is used as a current container scheduling policy, then the container in each storage device corresponding to the current container scheduling policy is determined, then the resource association degree corresponding to each storage device is calculated according to the determined container in each storage device and the data association degree between each container, where the resource association degree is used to reflect the probability that each container in the storage device needs to read data simultaneously, and finally the average value calculation is performed on the data association degree corresponding to each storage device, and the average value calculation result is used as the resource-related information corresponding to the current container scheduling policy.
In a specific implementation, the degree of data association between containers is the probability that the data in the containers are read simultaneously, for example: the data in the container a is read 20 times, the data in the container B is read 30 times, and the number of times of simultaneous reading of the container a and the container B is 10 times, at this time, the degree of data association between the container a and the container B may be calculated to be 10×2/(20+30) =40%.
It is understood that when calculating the resource correlation corresponding to each storage device according to the determined container in each storage device and the data correlation between each container, it is assumed that, for one storage device, there is a container a, a container B, and a container C in the storage device, the data correlation between the container a and the container B is 40%, the data correlation between the container B and the container C is 50%, and the data correlation between the container a and the container C is 30%, at this time, the three data correlations may be averaged, and therefore, the resource correlation of the storage device may be determined to be 40%, and of course, it is assumed that only one container is present in one storage device, at this time, the resource correlation of the storage device may be determined to be 0.
Assuming that the storage devices have 3 storage devices, and the resource relatedness of the storage devices is 40%,60% and 50%, at this time, it may be determined that the resource relatedness information corresponding to the current container scheduling policy is 50%.
It should be noted that, since each container scheduling policy corresponds to different container storage forms, that is, different container scheduling policies may make containers stored in each storage device different, and the higher the capacity utilization rate in the storage device, the more beneficial to the subsequent container addition, in order to facilitate the measurement of the capacity utilization rate of each storage device, the measurement can be performed through the device occupancy rate information, at this time, the lower the corresponding device occupancy rate information, and accordingly, the smaller the loss coefficient.
In order to facilitate rapid determination of the device occupancy rate information corresponding to each container scheduling policy, in this embodiment, each container scheduling policy may be traversed first, the traversed container scheduling policy is used as a current container scheduling policy, then the container in each storage device corresponding to the current container scheduling policy is determined, then the capacity occupancy rate corresponding to each storage device is calculated according to the determined container in each storage device and the occupancy information of each container, finally the product calculation is performed on the capacity occupancy rates corresponding to each storage device, and the product calculation result is used as the device occupancy rate information corresponding to the current container scheduling policy.
Assuming that three storage devices are provided, namely a storage device A, a storage device B and a storage device C, wherein the total capacity of the storage device A is 8GB, a container X is stored in the storage device A, and the occupation information of the container X is 6GB; the total capacity of the storage equipment B is 8GB, wherein a container Y and a container Z are stored, the occupation information of the container Y is 2GB, and the occupation information of the container Z is 2GB; the total capacity of the storage equipment C is 8GB, the container W is stored in the storage equipment C, the occupation information of the container W is 3GB, the target capacity of the container to be added is 6GB, and the storage equipment A, the storage equipment B and the storage equipment C are selected because the total capacity of the three storage equipment is larger than the target capacity;
When the current container scheduling policy is a container scheduling policy of moving the container Y to the storage device a and storing the container to be added to the storage device B, the capacity occupancy rate of the storage device a is 100%, the capacity occupancy rate of the storage device B is 100%, and the capacity occupancy rate of the storage device C is 37.5%, at this time, the device occupancy rate information corresponding to the current container scheduling policy is 37.5%.
When the current container scheduling policy is a container scheduling policy of moving the container Y to the storage device C and storing the container to be added to the storage device B, the capacity occupancy rate of the storage device a is 75%, the capacity occupancy rate of the storage device B is 100%, and the capacity occupancy rate of the storage device C is 62.5%, at this time, the device occupancy rate information corresponding to the current container scheduling policy is 46.9%.
S32: and calculating loss coefficients respectively corresponding to the container scheduling strategies according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to the container scheduling strategies.
It should be noted that, according to the above description, the capacity of the to-be-moved container, the resource related information, and the equipment occupancy rate information are all in direct proportion to the loss coefficients, so when determining the loss coefficients corresponding to the container scheduling policies, the loss coefficients corresponding to the container scheduling policies can be calculated according to the capacity of the to-be-moved container, the resource related information, and the equipment occupancy rate information corresponding to the container scheduling policies through a preset loss coefficient formula;
The preset loss coefficient formula is as follows:
W＝aX+bY+cZ+d
wherein a, b, c, d is a constant, W is a loss coefficient, X is a capacity of a container to be moved, Y is resource-related information, and Z is equipment occupancy information.
In a specific implementation, as can be seen from the above description, the unit of capacity X of the container to be moved is generally GB, for example: in order to avoid the problem that the units are not uniform, in this embodiment, the capacity of the container to be moved may be divided by GB units when the loss coefficient is calculated, that is, assuming that the capacity of the container to be moved is 2GB, when the preset loss coefficient is introduced, x=2, assuming that the capacity of the container to be moved is 0.5GB, and x=0.5 when the preset loss coefficient is introduced, the problem that the units are not uniform may be avoided.
Assuming that a=0.5, b=1, c=1.2, and d=0, at this time, if the capacity of the container to be moved corresponding to the first container scheduling policy is 2GB, that is, x=2, the resource-related information is 40%, that is, y=0.4, and the device occupancy information is 50%, that is, z=0.5, at this time, w=0.5x2+1×0.4+1.2x0.5=2 can be calculated.
If the capacity of the container to be moved corresponding to the second container scheduling policy is 3GB, i.e., x=3, the resource-related information is 30%, i.e., y=0.3, the device occupancy information is 40%, i.e., z=0.4, at this time, w=0.5×3+1×0.3+1.2×0.4=2.28 can be obtained by calculation. It can be understood that, since the constants a, b, c, d belong to unknown parameters, if the parameters are simply set, the problem that the weights of part of the parameters are too heavy easily occurs, and in order to avoid the problem, in this embodiment, a plurality of sample capacities, sample resource related information, sample equipment occupancy information, and sample loss coefficients may be preset, and the initial loss coefficient formula is fitted through the sample capacities, the sample resource related information, the sample equipment occupancy information, and the sample loss coefficients, so as to determine the constants in the initial loss coefficient formula to obtain the preset loss coefficient formula.
Of course, the expression of the initial loss coefficient formula is the same as the preset loss coefficient formula, and the preset loss coefficient formula is the initial loss coefficient formula determined by the constant a, b, c, d.
According to the method and the device for calculating the loss coefficients, the loss coefficients corresponding to the container scheduling strategies are calculated according to the capacity, the resource related information and the equipment occupancy rate information of the container to be moved, which correspond to the container scheduling strategies.
In addition, an embodiment of the present invention further provides a container adding device, referring to fig. 4, where the container adding device includes:
the occupation information module 10 is used for determining the target capacity of a container to be added according to the container adding instruction when the container adding instruction is received, and acquiring the occupation information of the container on each storage device;
a policy generation module 20, configured to generate a plurality of container scheduling policies according to the occupancy information when the target capacity of the container to be added exceeds the remaining capacity of each storage device;
a coefficient determining module 30, configured to determine a loss coefficient corresponding to each container scheduling policy;
and the container scheduling module 40 is configured to select a target container scheduling policy from the container scheduling policies according to the loss coefficient, and perform container scheduling according to the target container scheduling policy.
According to the scheme, the plurality of container scheduling strategies are generated according to the occupation information, the target container scheduling strategy is selected from the container scheduling strategies according to the loss coefficient, and the container scheduling is carried out according to the target container scheduling strategy, so that part of containers can be scheduled to other storage devices according to the target container scheduling strategy, and the containers to be added can be added smoothly.
It should be noted that each module in the above apparatus may be used to implement each step in the above method, and achieve a corresponding technical effect, which is not described herein again.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a removable carrier of a hardware running environment according to an embodiment of the present invention.
As shown in fig. 5, the movable carrier may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 5 is not limiting of the movable carrier and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 5, an operating system, a network communication module, a user interface module, and a container addition program may be included in the memory 1005, which is one type of computer storage medium.
In the removable carrier shown in fig. 5, the network interface 1004 is mainly used for data communication with an external network; the user interface 1003 is mainly used for receiving an input instruction of a user; the removable carrier calls a container addition program stored in the memory 1005 by the processor 1001 and performs the following operations:
when a container adding instruction is received, determining the target capacity of a container to be added according to the container adding instruction, and acquiring the occupation information of the container on each storage device;
if the target capacity of the container to be added exceeds the residual capacity of each storage device, generating a plurality of container scheduling strategies according to the occupation information;
determining loss coefficients corresponding to each container scheduling strategy;
and selecting a target container scheduling strategy from the container scheduling strategies according to the loss coefficient, and scheduling the containers according to the target container scheduling strategy.
Further, the processor 1001 may call a container addition program stored in the memory 1005, and further perform the following operations:
Taking the storage equipment with the total capacity larger than or equal to the target capacity as the storage equipment to be selected;
taking the container with the occupied information smaller than the target capacity in each storage device to be selected as a container to be selected;
traversing the container to be selected, taking the traversed container to be selected as a container to be moved, and taking a storage device to which the container to be moved belongs as a current storage device;
calculating the residual capacity of the current storage device after the container to be moved is removed;
and when the residual capacity of the current storage device is larger than or equal to the target capacity, generating a container scheduling policy for moving the container to be moved to other storage devices except the current storage device and storing the container to be added to the current storage device.
Further, the processor 1001 may call a container addition program stored in the memory 1005, and further perform the following operations:
acquiring the capacity of a container to be moved corresponding to each container scheduling policy, acquiring resource related information corresponding to each container scheduling policy, and acquiring equipment occupancy rate information corresponding to each container scheduling policy;
and calculating loss coefficients respectively corresponding to the container scheduling strategies according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to the container scheduling strategies.
Further, the processor 1001 may call a container addition program stored in the memory 1005, and further perform the following operations:
acquiring the data association degree between the containers, wherein the data association degree is used for reflecting the probability of simultaneously reading data between two containers;
traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy;
determining containers in each storage device corresponding to the current container scheduling policy;
calculating the resource correlation degree corresponding to each storage device according to the determined containers in each storage device and the data correlation degree between the containers, wherein the resource correlation degree is used for reflecting the probability that each container in the storage device needs to read data at the same time;
and carrying out average value calculation on the data correlation degrees corresponding to the storage devices respectively, and taking an average value calculation result as resource correlation information corresponding to the current container scheduling strategy.
Further, the processor 1001 may call a container addition program stored in the memory 1005, and further perform the following operations:
traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy;
Determining containers in each storage device corresponding to the current container scheduling policy;
calculating the capacity occupancy rate corresponding to each storage device according to the determined containers in each storage device and the occupancy information of each container;
and carrying out product calculation on the capacity occupancy rates corresponding to the storage devices respectively, and taking the product calculation result as the device occupancy rate information corresponding to the current container scheduling strategy.
Further, the processor 1001 may call a container addition program stored in the memory 1005, and further perform the following operations:
the step of loss coefficient corresponding to each container scheduling strategy specifically comprises the following steps:
calculating loss coefficients respectively corresponding to each container scheduling strategy according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to each container scheduling strategy through a preset loss coefficient formula;
the preset loss coefficient formula is as follows:
W＝aX+bY+cZ+d
wherein a, b, c, d is a constant, W is a loss coefficient, X is a capacity of a container to be moved, Y is resource-related information, and Z is equipment occupancy information.
Further, the processor 1001 may call a container addition program stored in the memory 1005, and further perform the following operations:
Acquiring sample capacity, sample resource related information, sample equipment occupancy information and a sample loss coefficient;
and performing fitting operation on the initial loss coefficient formula through the sample capacity, the sample resource related information, the sample equipment occupancy rate information and the sample loss coefficient to obtain a preset loss coefficient formula.
According to the scheme, the plurality of container scheduling strategies are generated according to the occupation information, the target container scheduling strategy is selected from the container scheduling strategies according to the loss coefficient, and the container scheduling is carried out according to the target container scheduling strategy, so that part of containers can be scheduled to other storage devices according to the target container scheduling strategy, and the containers to be added can be added smoothly.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (6)
1. A method of adding a container, the method comprising the steps of:
when a container adding instruction is received, determining the target capacity of a container to be added according to the container adding instruction, and acquiring the occupation information of the container on each storage device;
if the target capacity of the container to be added exceeds the residual capacity of each storage device, generating a plurality of container scheduling strategies according to the occupation information;
determining loss coefficients corresponding to each container scheduling strategy;
selecting a target container scheduling strategy from the container scheduling strategies according to the loss coefficient, and scheduling containers according to the target container scheduling strategy;
the step of determining the loss coefficient corresponding to each container scheduling policy comprises the following steps:
acquiring the capacity of a container to be moved corresponding to each container scheduling policy, acquiring resource related information corresponding to each container scheduling policy, and acquiring equipment occupancy rate information corresponding to each container scheduling policy;
calculating loss coefficients respectively corresponding to each container scheduling policy according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to each container scheduling policy;
the step of obtaining the resource related information corresponding to each container scheduling policy includes:
Acquiring the data association degree between the containers, wherein the data association degree is used for reflecting the probability of simultaneously reading data between two containers;
traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy;
determining containers in each storage device corresponding to the current container scheduling policy;
calculating the resource correlation degree corresponding to each storage device according to the determined containers in each storage device and the data correlation degree between the containers, wherein the resource correlation degree is used for reflecting the probability that each container in the storage device needs to read data at the same time;
carrying out average value calculation on the data correlation degrees corresponding to the storage devices respectively, and taking an average value calculation result as resource correlation information corresponding to the current container scheduling strategy;
the step of obtaining the device occupancy rate information corresponding to each container scheduling policy comprises the following steps:
traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy;
determining containers in each storage device corresponding to the current container scheduling policy;
calculating the capacity occupancy rate corresponding to each storage device according to the determined containers in each storage device and the occupancy information of each container;
Performing product calculation on capacity occupancy rates corresponding to the storage devices respectively, and taking the product calculation result as device occupancy rate information corresponding to the current container scheduling strategy;
the step of calculating loss coefficients corresponding to each container scheduling policy respectively according to the capacity, the resource related information and the equipment occupancy rate information of the container to be moved corresponding to each container scheduling policy specifically includes:
calculating loss coefficients respectively corresponding to each container scheduling strategy according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to each container scheduling strategy through a preset loss coefficient formula;
the preset loss coefficient formula is as follows:
W=aX+bY+cZ+d
wherein a, b, c, d is a constant, W is a loss coefficient, X is a capacity of a container to be moved, Y is resource-related information, and Z is equipment occupancy information.
2. The container adding method according to claim 1, wherein the step of generating a plurality of container scheduling policies from the occupancy information comprises:
taking the storage equipment with the total capacity larger than or equal to the target capacity as the storage equipment to be selected;
taking the container with the occupied information smaller than the target capacity in each storage device to be selected as a container to be selected;
Traversing the container to be selected, taking the traversed container to be selected as a container to be moved, and taking a storage device to which the container to be moved belongs as a current storage device;
calculating the residual capacity of the current storage device after the container to be moved is removed;
and when the residual capacity of the current storage device is larger than or equal to the target capacity, generating a container scheduling policy for moving the container to be moved to other storage devices except the current storage device and storing the container to be added to the current storage device.
3. The method for adding containers according to claim 1, wherein before the step of calculating the loss coefficients corresponding to the container scheduling policies respectively according to the capacity, the resource-related information, and the equipment occupancy information of the container to be moved corresponding to the container scheduling policies by using a preset loss coefficient formula, the method for adding containers further comprises:
acquiring sample capacity, sample resource related information, sample equipment occupancy information and a sample loss coefficient;
and performing fitting operation on the initial loss coefficient formula through the sample capacity, the sample resource related information, the sample equipment occupancy rate information and the sample loss coefficient to obtain a preset loss coefficient formula.
4. A container adding apparatus, characterized in that the container adding apparatus comprises:
the system comprises an occupation information module, a storage device and a storage device, wherein the occupation information module is used for determining the target capacity of a container to be added according to a container adding instruction when receiving the container adding instruction, and acquiring the occupation information of the container on each storage device;
the policy generation module is used for generating a plurality of container scheduling policies according to the occupation information when the target capacity of the container to be added exceeds the residual capacity of each storage device;
the coefficient determining module is used for determining loss coefficients corresponding to each container scheduling strategy;
the container scheduling module is used for selecting a target container scheduling strategy from the container scheduling strategies according to the loss coefficient and performing container scheduling according to the target container scheduling strategy;
the coefficient determining module is further used for obtaining the capacity of the container to be moved corresponding to each container scheduling policy, obtaining resource related information corresponding to each container scheduling policy and obtaining equipment occupancy rate information corresponding to each container scheduling policy; calculating loss coefficients respectively corresponding to each container scheduling policy according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to each container scheduling policy;
The coefficient determining module is further used for obtaining the data association degree between the containers, and the data association degree is used for reflecting the probability that the two containers need to read data at the same time; traversing each container scheduling strategy, and taking the traversed container scheduling strategy as the current container scheduling strategy; determining containers in each storage device corresponding to the current container scheduling policy; calculating the resource correlation degree corresponding to each storage device according to the determined containers in each storage device and the data correlation degree between the containers, wherein the resource correlation degree is used for reflecting the probability that each container in the storage device needs to read data at the same time; carrying out average value calculation on the data correlation degrees corresponding to the storage devices respectively, and taking an average value calculation result as resource correlation information corresponding to the current container scheduling strategy;
the coefficient determining module is further used for traversing each container scheduling strategy and taking the traversed container scheduling strategy as a current container scheduling strategy; determining containers in each storage device corresponding to the current container scheduling policy; calculating the capacity occupancy rate corresponding to each storage device according to the determined containers in each storage device and the occupancy information of each container; performing product calculation on capacity occupancy rates corresponding to the storage devices respectively, and taking the product calculation result as device occupancy rate information corresponding to the current container scheduling strategy;
The coefficient determining module is further configured to calculate loss coefficients corresponding to the container scheduling policies respectively according to the capacity of the container to be moved, the resource related information and the equipment occupancy rate information corresponding to the container scheduling policies through a preset loss coefficient formula;
wherein, the preset loss coefficient formula is: w=ax+by+cz+d, wherein a, b, c, d is a constant, W is a loss coefficient, X is a capacity of a container to be moved, Y is resource-related information, and Z is equipment occupancy information.
5. A terminal device, characterized in that the terminal device comprises: a memory, a processor and a container add-on program stored on the memory and executable on the processor, the container add-on program configured to implement the steps of the container add-on method of any one of claims 1 to 3.
6. A computer-readable storage medium, on which a container addition program is stored, which, when executed by a processor, implements the steps of the container addition method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011384769.XA CN112445577B (en) | 2020-11-30 | 2020-11-30 | Container adding method, device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011384769.XA CN112445577B (en) | 2020-11-30 | 2020-11-30 | Container adding method, device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112445577A CN112445577A (en) | 2021-03-05 |
| CN112445577B true CN112445577B (en) | 2023-11-24 |
Family
ID=74740333
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011384769.XA Active CN112445577B (en) | 2020-11-30 | 2020-11-30 | Container adding method, device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112445577B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109213597A (en) * | 2018-08-01 | 2019-01-15 | 平安科技（深圳）有限公司 | Resource allocation methods, device, computer equipment and computer readable storage medium |
| CN109343801A (en) * | 2018-10-23 | 2019-02-15 | 深圳前海微众银行股份有限公司 | Date storage method, equipment and computer readable storage medium |
| CN109753356A (en) * | 2018-12-25 | 2019-05-14 | 北京友信科技有限公司 | A kind of container resource regulating method, device and computer readable storage medium |
| CN110018893A (en) * | 2019-03-12 | 2019-07-16 | 平安普惠企业管理有限公司 | A kind of method for scheduling task and relevant device based on data processing |
| CN110515704A (en) * | 2019-08-30 | 2019-11-29 | 广东浪潮大数据研究有限公司 | Resource regulating method and device based on Kubernetes system |
| WO2020103723A1 (en) * | 2018-11-21 | 2020-05-28 | 腾讯科技（深圳）有限公司 | Method, apparatus and device for scheduling virtual object in virtual environment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8776065B2 (en) * | 2006-11-16 | 2014-07-08 | Oracle America, Inc. | Real time monitoring and tracing of scheduler decisions |
-
2020
- 2020-11-30 CN CN202011384769.XA patent/CN112445577B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109213597A (en) * | 2018-08-01 | 2019-01-15 | 平安科技（深圳）有限公司 | Resource allocation methods, device, computer equipment and computer readable storage medium |
| CN109343801A (en) * | 2018-10-23 | 2019-02-15 | 深圳前海微众银行股份有限公司 | Date storage method, equipment and computer readable storage medium |
| WO2020103723A1 (en) * | 2018-11-21 | 2020-05-28 | 腾讯科技（深圳）有限公司 | Method, apparatus and device for scheduling virtual object in virtual environment |
| CN109753356A (en) * | 2018-12-25 | 2019-05-14 | 北京友信科技有限公司 | A kind of container resource regulating method, device and computer readable storage medium |
| CN110018893A (en) * | 2019-03-12 | 2019-07-16 | 平安普惠企业管理有限公司 | A kind of method for scheduling task and relevant device based on data processing |
| CN110515704A (en) * | 2019-08-30 | 2019-11-29 | 广东浪潮大数据研究有限公司 | Resource regulating method and device based on Kubernetes system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112445577A (en) | 2021-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110837410B (en) | Task scheduling method and device, electronic equipment and computer readable storage medium | |
| CN110389816B (en) | Method, apparatus and computer readable medium for resource scheduling | |
| CN112615731B (en) | Method and device for distributing multi-operator combined network slice resources | |
| JP2013542530A (en) | Application lifetime management | |
| CN103067524A (en) | Ant colony optimization computing resource distribution method based on cloud computing environment | |
| CN105378668B (en) | The interruption of operating system management in multicomputer system guides | |
| CN110502321A (en) | A kind of resource regulating method and system | |
| CN110389822A (en) | The node scheduling method, apparatus and server of execution task | |
| CN105824705B (en) | A kind of method for allocating tasks and electronic equipment | |
| CN103548324A (en) | Method for virtual machine assignment and device for virtual machine assignment | |
| CN105677744A (en) | Method and apparatus for increasing service quality in file system | |
| US11513866B1 (en) | Method and system for managing resource utilization based on reinforcement learning | |
| CN110188975B (en) | Resource acquisition method and device | |
| CN112445577B (en) | Container adding method, device, terminal equipment and storage medium | |
| US9501321B1 (en) | Weighted service requests throttling | |
| CN112445617A (en) | Load strategy selection method and system based on mobile edge calculation | |
| CN115174582B (en) | Data scheduling method and related device | |
| CN112036697A (en) | Task allocation method and device, readable storage medium and electronic equipment | |
| CN115437781B (en) | GPU resource management method and system | |
| CN110069319A (en) | A kind of multiple target dispatching method of virtual machine and system towards cloudlet resource management | |
| CN113296951B (en) | Resource allocation scheme determining method and equipment | |
| CN115712486A (en) | Method and device for controlling live migration of virtual machine, medium and computer equipment | |
| CN108762938A (en) | Task processing method, apparatus and system in a kind of cloud computing platform | |
| JPWO2018168695A1 (en) | Distributed machine learning device, distributed machine learning method, and distributed machine learning program | |
| CN114610465A (en) | Task processing method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |