CN117149099B - Calculation and storage split server system and control method - Google Patents
Calculation and storage split server system and control method Download PDFInfo
- Publication number
- CN117149099B CN117149099B CN202311429320.4A CN202311429320A CN117149099B CN 117149099 B CN117149099 B CN 117149099B CN 202311429320 A CN202311429320 A CN 202311429320A CN 117149099 B CN117149099 B CN 117149099B
- Authority
- CN
- China
- Prior art keywords
- calculation
- module
- storage module
- storage
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 107
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004891 communication Methods 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000009825 accumulation Methods 0.000 claims description 3
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000000611 regression analysis Methods 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000013403 standard screening design Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to the technical field of servers, in particular to a calculation and storage split server system and a control method, wherein the calculation and storage split server system comprises a calculation module and a storage module which are arranged as independent working modules, and the calculation module and the storage module are in communication connection; load parameters of the calculation module and the storage module are obtained, and the load parameters of all the calculation modules and the storage module are recorded to form a record reference group; taking the combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group as a benchmark; and (3) establishing a weighted calculation model, and optionally combining the rest calculation modules and the storage modules. The scheme provided by the invention enables the user to simply expand the computing capacity or the storage capacity of the server according to the requirements. In contrast, the architecture of the present invention allows users to match appropriate independent computing modules and storage modules as needed to meet changing business needs.
Description
Technical Field
The invention relates to the technical field of servers, in particular to a computing and storing split type server system and a control method.
Background
In modern computing environments, servers are core devices that support a variety of computing and storage tasks. However, conventional server architectures have inherent differences between computing and storage capabilities. In general, servers are relatively more computationally powerful but less powerful, or more powerful but less computationally powerful. This imbalance limits the overall performance and flexibility of the server system.
To address this problem, one solution is to introduce a computing storage split server architecture. This architecture separates the computing and storage functions into separate modules, allowing users to flexibly extend computing or storage capabilities according to actual needs. The user can select to add a computing module or a storage module according to the application requirement so as to realize the dynamic configuration and optimization of the resources.
The most recent prior implementation to the present invention is the concept of "Composable Infrastructure" (combinable infrastructure). The combinable infrastructure is a server architecture and management model that aims to provide flexible resource configuration and management. It divides computing, storage and network resources into independent components based on abstraction and virtualization of physical resources to meet the needs of different workloads.
The combinable infrastructure combines different hardware components together in a software-defined manner to form a customized server solution. The user can combine the computing module and the storage module according to the requirement to meet the requirement of a specific application scene. This architecture allows for flexible configuration and expansion of resources, providing better flexibility and scalability.
However, the combinable infrastructure is more concerned with the combination and management of resources, and does not explicitly address the differences between computing and storage capabilities, resulting in the problem of computing and storage capability imbalance.
Disclosure of Invention
The invention aims to provide a calculation and storage split type server system and a control method, which are used for solving the problem of unbalanced calculation and storage capacity in the prior art.
The embodiment of the invention is realized by the following technical scheme:
in a first aspect, the present invention provides a method for controlling a computing storage split server system, including;
setting a calculation module and a storage module as independent working modules, wherein the calculation module and the storage module are in communication connection;
load parameters of the calculation module and the storage module are obtained, and the load parameters of all the calculation modules and the storage module are recorded to form a record reference group;
taking the combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group as a benchmark;
building a weighted calculation model, carrying out random combination on the rest calculation modules and the storage module, carrying out weighted comparison through the weighted calculation model, and storing the weighted comparison result obtained by each group;
and performing task allocation through the weighted comparison result.
In one embodiment of the present invention, the establishing the weighted calculation model includes;
wherein:for the processing capacity of the reference calculation module, +.>Representing the processing capacity of i computing modules, < >>Representing the usage of the ith calculation module,/->Calculating the utilization rate of the module for the reference, +.>Representing reference storage module parameters->Representing the i-th storage module parameter,/>For the utilization of the ith storage module, < >>For the utilization of the reference storage module, +.>Network traffic for a reference storage module, +.>For the network traffic of the ith storage module, < >>For the transport capacity of the reference storage module, +.>The transmission quantity of the ith storage module; alpha represents the comparison weight of the calculation module, beta represents the comparison weight of the storage module, gamma represents the comparison weight of the network, theta represents the comparison weight of the load, and the initial values of alpha, beta, gamma and theta can be 1.
In one embodiment of the present invention, the task allocation by the weighted comparison result includes;
setting probability spaces of each group of the calculation module and the storage module;
setting the space of the whole server as 1;
acquiring a request task, and calculating a random number between {0,1 };
the group to be forwarded by the request task is judged according to the point of the random number falling in the probability space.
In one embodiment of the present invention, further comprising;
setting a website access amount threshold, acquiring the website access amount at the moment, judging whether the network is congested at the moment according to the website access amount threshold, and judging whether the server is in high load at the moment according to the network congestion;
if the network is congested at this time, the server is in high load;
converting the access information through a computing module to generate a data packet, wherein the data packet comprises request and distribution data;
judging the group to be forwarded by the request task according to the point of the random number falling in the probability space;
changing the physical address of the network card in the data packet into an updated address to continue to be sent;
after the transmission is completed, the calculation module continues to accept the data, generates a response packet and returns the response packet.
In one embodiment of the present invention, further comprising;
acquiring load values transmitted by each group of calculation modules and the storage module, and judging whether the current load value changes or not;
if the load values of a certain group of calculation modules and the storage module become smaller, a sorting binary tree structure is carried out;
inserting the calculation module and the storage module into a binary tree to finish data traversal;
and finishing sequencing according to the magnitude of the load value, and distributing corresponding tasks to a certain group of calculation modules and storage modules according to the sequencing.
In one embodiment of the present invention, further comprising;
inquiring the load conditions of the calculation module and the storage module, and adjusting the weights of gamma, alpha, beta and theta;
if the relation value between the load data and the weight value in a certain group of calculation modules and the storage module is larger, the load of the group of calculation modules and the storage module is high, and the weight value needs to be reduced;
in the method, in the process of the invention,for initial weight, ++>The number of the current movable connection is the number; />Is an adaptive weight.
In an embodiment of the present invention, the method further includes predicting a network access amount, the predicting a network access amount including;
data preprocessing, namely establishing an original data sequence, and generating a first data sequence through accumulation;
averaging two adjacent items of the first data sequence to obtain a second data sequence;
modelingA, b are unknown coefficients;
estimating a, b using regression analysis, the formula:
obtaining a predicted value:
the predicted values of the original sequence are:
constructing a matrix B and a data vector Y:
order theThen, according to the construction matrix B and the data vector Y, < ->；
A, b is obtained and substituted into the predicted value to obtain a predicted value function;
predicting the network access amount through a predicted value function;
preparing to forward the group in advance according to the predicted access quantity;
in the method, in the process of the invention,X (0) for the original data sequence, Z (1) For the second data sequence, X (1) Is the first data sequence.
In a second aspect, the present invention provides a computing storage split server system comprising;
the setting device is configured to set the computing module and the storage module as independent working modules, and the computing module and the storage module are in communication connection;
the acquisition device is configured to acquire load parameters of the calculation module and the storage module, record the load parameters of all the calculation modules and the storage module, and form a record reference group;
recording means configured to base on a combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group;
the modeling device is configured to establish a weighted calculation model, the rest calculation modules and the storage module are combined arbitrarily, weighted comparison is carried out through the weighted calculation model, and the weighted comparison results obtained by each group are stored;
the distribution device is configured to distribute tasks through the weighted comparison result;
and the central processing device is connected with the setting device, the acquisition device, the recording device, the modeling device and the distribution device and is used for executing the control method of the calculation and storage split server system.
In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements a method for controlling a computing storage split server system as described above when executing the computer program.
In a fourth aspect, the present invention further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements a method for controlling a split-type server system.
The technical scheme of the embodiment of the invention has at least the following advantages and beneficial effects:
the scheme provided by the invention enables the user to simply expand the computing capacity or the storage capacity of the server according to the requirements. In contrast, the architecture of the present invention allows users to match appropriate independent computing modules and storage modules as needed to meet changing business needs.
By the scheme for management and distribution, the distribution proportion of the calculation and storage resources can be flexibly adjusted according to the requirements. This approach provides greater flexibility and resilience, but may be limited by factors such as network bandwidth and delay.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
The terms first, second and the like in the description and in the claims of the present application and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. The naming or numbering of the steps in the present application does not mean that the steps in the method flow must be executed according to the time/logic sequence indicated by the naming or numbering, and the execution sequence of the steps in the flow that are named or numbered may be changed according to the technical purpose to be achieved, so long as the same or similar technical effects can be achieved.
The division of the modules presented in this application is a logical division, and there may be other manners of division in practical application, for example, multiple modules may be combined or integrated in another system, or some features may be omitted, or not performed.
The modules or sub-modules described separately may or may not be physically separate, may or may not be implemented in software, and may be implemented in part in software, where the processor invokes the software to implement the functions of the part of the modules or sub-modules, and where other parts of the templates or sub-modules are implemented in hardware, for example in hardware circuits. In addition, some or all of the modules may be selected according to actual needs to achieve the purposes of the present application.
In a first aspect, as shown in fig. 1, the present invention provides a method for controlling a computing storage split server system, including;
s101: setting a calculation module and a storage module as independent working modules, wherein the calculation module and the storage module are in communication connection;
the computing module mainly comprises a processor, a memory, a network interface and the like, and is also provided with a system management function, including remote management, monitoring and configuration. Through the system management functions, an administrator can remotely monitor, diagnose and configure the computing module to ensure stable operation and optimal performance of the server system.
The storage module typically contains a plurality of hard disk drives for persisting data. Hard disk drives employ disk storage technology that is capable of providing large amounts of storage space. HDDs are suitable for high-capacity storage and large-scale data storage, have relatively low cost and good durability.
Solid State Drive (SSD): the memory module may also include solid state drives that employ flash memory storage technology with faster read and write speeds and higher data access performance. SSDs are suitable for application scenarios requiring fast data access and response, such as large-scale databases, virtualized environments, and the like.
RAID controller: the storage module may integrate a RAID (redundant array of disk) controller for managing data redundancy and fault tolerance among the plurality of hard disk drives. RAID techniques may provide redundancy backup of data, increase read-write performance and fault tolerance to improve the reliability and performance of the storage system.
Storage management software: the storage module is provided with storage management software for managing storage resources, realizing functions of data backup, snapshot, data recovery and the like. The storage management software provides a user-friendly interface that enables an administrator to configure, monitor and manage the storage modules.
Each module has its own computing or storage resources, which can be independently operated and managed.
With respect to communication connections, they may be through Ethernet, fibre channel, infiniBand, or PCI Express connections.
S102: load parameters of the calculation module and the storage module are obtained, and the load parameters of all the calculation modules and the storage module are recorded to form a record reference group;
s103: taking the combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group as a benchmark;
the first calculation module and the first storage module may be random, or may be a module with relatively uniform performance from all calculation modules and storage modules.
S104: building a weighted calculation model, carrying out random combination on the rest calculation modules and the storage module, carrying out weighted comparison through the weighted calculation model, and storing the weighted comparison result obtained by each group;
and performing task allocation through the weighted comparison result.
Specifically, the building of the weighted calculation model comprises the following steps of;
in one embodiment of the present invention, the establishing the weighted calculation model includes;
wherein:for the processing capacity of the reference calculation module, +.>Representing the processing capacity of i computing modules, < >>Representing the usage of the ith calculation module,/->Calculating the utilization rate of the module for the reference, +.>Representing reference storage module parameters->Representing the i-th storage module parameter,/>For the utilization of the ith storage module, < >>For the utilization of the reference storage module, +.>Network traffic for a reference storage module, +.>For the network traffic of the ith storage module, < >>For the transport capacity of the reference storage module, +.>Transfer amount for the ith storage moduleThe method comprises the steps of carrying out a first treatment on the surface of the Alpha represents the comparison weight of the calculation module, beta represents the comparison weight of the storage module, gamma represents the comparison weight of the network, theta represents the comparison weight of the load, and the initial values of alpha, beta, gamma and theta can be 1.
Depending on the actual situation in which the cluster is operating, one of its weights may be increased or decreased to emphasize or decrease certain aspects of load performance.
According to the combined calculation module and storage module list, the combination of the calculation module and the storage module with the least loading is found, and if two heavier loads are placed on the same server, the response efficiency of the server client is very likely to be reduced; secondly, server load calculations are continuous, which can affect server performance to some extent.
Therefore, a minimum load strategy of the combination of the calculation module and the storage module can be adopted, a random probability model is selected, the probability space of the combination of each calculation module and the storage module is reasonably set through calculation and comparison of weights, and the space of the whole server system is set to be 1.
Assuming the dispatcher receives a new requested task, a random number between [0,1] is calculated, and the forwarded combination of the target calculation module and the storage module is judged according to the point that the random number falls in the probability space. If the relative weight of the combination of the calculation module and the storage module is smaller, the probability space is relatively larger, more allocation of the request tasks can be obtained, and the allocation probability is higher. On the other hand, each forwarding belongs to an independent event, and finally the forwarding cannot be interfered, and similarly, the forwarding operation is carried out again, so that the mutual influence cannot be generated. Therefore, the forwarding effect can be improved and the stability of the server can be ensured.
In one embodiment of the present invention, further comprising;
s201: setting a website access amount threshold, acquiring the website access amount at the moment, judging whether the network is congested at the moment according to the website access amount threshold, and judging whether the server is in high load at the moment according to the network congestion;
s202: if the network is congested at this time, the server is in high load;
s203: converting the access information through a computing module to generate a data packet, wherein the data packet comprises request and distribution data;
s204: judging the group to be forwarded by the request task according to the point of the random number falling in the probability space;
s205: changing the physical address of the network card in the data packet into an updated address to continue to be sent;
s206: after the transmission is completed, the calculation module continues to accept the data, generates a response packet and returns the response packet.
By the method, the load balancing processor is needed to be used, the return task can be completed without going through other paths, and the running times of the load balancing processor are reduced.
Secondly, also comprises;
acquiring load values transmitted by each group of calculation modules and the storage module, and judging whether the current load value changes or not; if the load values of a certain group of calculation modules and the storage module become smaller, a sorting binary tree structure is carried out; inserting the calculation module and the storage module into a binary tree to finish data traversal; and finishing sequencing according to the magnitude of the load value, and distributing corresponding tasks to a certain group of calculation modules and storage modules according to the sequencing.
Thus, related tasks can be distributed to a certain group of the computing module and the storage module more reasonably.
In an embodiment of the present invention, the adjusting of the weight further includes;
inquiring the load conditions of the calculation module and the storage module, and adjusting the weights of gamma, alpha, beta and theta;
if the relation value between the load data and the weight value in a certain group of calculation modules and the storage module is larger, the load of the group of calculation modules and the storage module is high, and the weight value needs to be reduced;
in the method, in the process of the invention,for initial weight, ++>The number of the current movable connection is the number; />Is an adaptive weight.
The cluster system selects a proper computing module and a storage module to provide service by comparing the joint load with the self-adaptive weight. The load values and weights of the computing module and the storage module group are required to be distributed proportionally according to a preset value so as to ensure the load balance of the computing module and the storage module group.
In one embodiment of the present invention, the method further comprises predicting a network access amount, wherein the predicting the network access amount comprises;
s301: data preprocessing, namely establishing an original data sequence, and generating a first data sequence through accumulation;
s302: averaging two adjacent items of the first data sequence to obtain a second data sequence;
taking the above example as an example:
s303: modelingA, b are unknown coefficients;
s304: estimating a, b using regression analysis, the formula:
obtaining a predicted value:
the predicted values of the original sequence are:
s305: constructing a matrix B and a data vector Y:
s306: order theThen, according to the construction matrix B and the data vector Y, < ->；
S307: a, b is obtained and substituted into the predicted value to obtain a predicted value function;
s308: predicting the network access amount through a predicted value function;
s309: preparing to forward the group in advance according to the predicted access quantity;
wherein X is (0) For the original data sequence, Z (1) For the second data sequence, X (1) Is the first data sequence.
In a second aspect, the present invention provides a computing storage split server system comprising;
the setting device is configured to set the computing module and the storage module as independent working modules, and the computing module and the storage module are in communication connection;
the acquisition device is configured to acquire load parameters of the calculation module and the storage module, record the load parameters of all the calculation modules and the storage module, and form a record reference group;
recording means configured to base on a combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group;
the modeling device is configured to establish a weighted calculation model, the rest calculation modules and the storage module are combined arbitrarily, weighted comparison is carried out through the weighted calculation model, and the weighted comparison results obtained by each group are stored;
the distribution device is configured to distribute tasks through the weighted comparison result;
and the central processing device is connected with the setting device, the acquisition device, the recording device, the modeling device and the distribution device and is used for executing the control method of the calculation and storage split server system.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. The computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A method for controlling a computing storage split server system, comprising;
setting a calculation module and a storage module as independent working modules, wherein the calculation module and the storage module are in communication connection;
load parameters of the calculation module and the storage module are obtained, and the load parameters of all the calculation modules and the storage module are recorded to form a record reference group;
taking the combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group as a benchmark;
building a weighted calculation model, carrying out random combination on the rest calculation modules and the storage module, carrying out weighted comparison through the weighted calculation model, and storing the weighted comparison result obtained by each group;
task allocation is carried out through the weighted comparison result;
the building of the weighted calculation model comprises the following steps of;
wherein:for the processing capacity of the reference calculation module, +.>Representing the processing capacity of i computing modules, < >>Representing the usage of the ith calculation module,/->Calculating the utilization rate of the module for the reference, +.>Representing reference storage module parameters->Representing the i-th storage module parameter,/>For the utilization of the ith storage module, < >>As a reference to the utilization rate of the storage module,network traffic for a reference storage module, +.>For the network traffic of the ith storage module, < >>For the transport capacity of the reference storage module, +.>The transmission quantity of the ith storage module; alpha represents the comparison weight of the calculation module, beta represents the comparison weight of the storage module, gamma represents the comparison weight of the network, theta represents the comparison weight of the load, and the initial values of alpha, beta, gamma and theta can be 1.
2. The method according to claim 1, wherein the task allocation by the weighted comparison result comprises;
setting probability spaces of each group of the calculation module and the storage module;
setting the space of the whole server as 1;
acquiring a request task, and calculating a random number between {0,1 };
the group to be forwarded by the request task is judged according to the point of the random number falling in the probability space.
3. The method for controlling a computing storage split server system according to claim 2, further comprising;
setting a website access amount threshold, acquiring the website access amount at the moment, judging whether the network is congested at the moment according to the website access amount threshold, and judging whether the server is in high load at the moment according to the network congestion;
if the network is congested at this time, the server is in high load;
converting the access information through a computing module to generate a data packet, wherein the data packet comprises request and distribution data;
judging the group to be forwarded by the request task according to the point of the random number falling in the probability space;
changing the physical address of the network card in the data packet into an updated address to continue to be sent;
after the transmission is completed, the calculation module continues to accept the data, generates a response packet and returns the response packet.
4. A method of controlling a computing storage split server system according to claim 3, further comprising;
acquiring load values transmitted by each group of calculation modules and the storage module, and judging whether the current load value changes or not;
if the load values of a certain group of calculation modules and the storage module become smaller, a sorting binary tree structure is carried out;
inserting the calculation module and the storage module into a binary tree to finish data traversal;
and finishing sequencing according to the magnitude of the load value, and distributing corresponding tasks to a certain group of calculation modules and storage modules according to the sequencing.
5. The method for controlling a computing storage split server system according to claim 4, further comprising;
inquiring the load conditions of the calculation module and the storage module, and adjusting the weights of gamma, alpha, beta and theta;
if the relation value between the load data and the weight value in a certain group of calculation modules and the storage module is larger, the load of the group of calculation modules and the storage module is high, and the weight value needs to be reduced;
in the method, in the process of the invention,for initial weight, ++>The number of the current movable connection is the number; />Is an adaptive weight.
6. The method of claim 4, further comprising predicting a network access amount, the predicting a network access amount comprising;
data preprocessing, namely establishing an original data sequence, and generating a first data sequence through accumulation;
averaging two adjacent items of the first data sequence to obtain a second data sequence;
modelingA, b are unknown coefficients;
estimating a, b using regression analysis, the formula:
obtaining a predicted value:
the predicted values of the original sequence are:
constructing a matrix B and a data vector Y:
order theThen, according to the construction matrix B and the data vector Y, < ->；
A, b is obtained and substituted into the predicted value to obtain a predicted value function;
predicting the network access amount through a predicted value function;
preparing to forward the group in advance according to the predicted access quantity;
wherein X is (0) For the original data sequence, Z (1) For the second data sequence, X (1) Is the first data sequence.
7. A computing storage split server system, comprising;
the setting device is configured to set the computing module and the storage module as independent working modules, and the computing module and the storage module are in communication connection;
the acquisition device is configured to acquire load parameters of the calculation module and the storage module, record the load parameters of all the calculation modules and the storage module, and form a record reference group;
recording means configured to base on a combination of the load parameter of the first calculation module and the load parameter of the first storage module in the record reference group;
the modeling device is configured to establish a weighted calculation model, the rest calculation modules and the storage module are combined arbitrarily, weighted comparison is carried out through the weighted calculation model, and the weighted comparison results obtained by each group are stored;
the distribution device is configured to distribute tasks through the weighted comparison result;
central processing means, connected to the setting means, the acquisition means, the recording means, the modeling means and the distribution means, for executing a calculation-storage split server system control method according to any one of claims 1 to 6.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a method of controlling a computing storage split server system as claimed in any one of claims 1 to 6 when the computer program is executed by the processor.
9. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, the computer program when executed by a processor implementing a method of controlling a computing storage split server system according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311429320.4A CN117149099B (en) | 2023-10-31 | 2023-10-31 | Calculation and storage split server system and control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311429320.4A CN117149099B (en) | 2023-10-31 | 2023-10-31 | Calculation and storage split server system and control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117149099A CN117149099A (en) | 2023-12-01 |
| CN117149099B true CN117149099B (en) | 2024-03-12 |
Family
ID=88903157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311429320.4A Active CN117149099B (en) | 2023-10-31 | 2023-10-31 | Calculation and storage split server system and control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117149099B (en) |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102089741A (en) * | 2007-09-17 | 2011-06-08 | 国际商业机器公司 | Executing computer-intensive database user-defined programs on an attached high-performance parallel computer |
| CN103327094A (en) * | 2013-06-19 | 2013-09-25 | 成都市欧冠信息技术有限责任公司 | Data distributed type memory method and data distributed type memory system |
| JP2014109886A (en) * | 2012-11-30 | 2014-06-12 | Seiko Precision Inc | Load distribution device, load distribution method and program |
| EP2963539A1 (en) * | 2014-07-02 | 2016-01-06 | VIA Alliance Semiconductor Co., Ltd. | Split-path fused multiply-accumulate operation using calculation control indicator cache |
| CN107357523A (en) * | 2017-06-27 | 2017-11-17 | 联想(北京)有限公司 | A kind of data processing method and electronic equipment |
| CN109088742A (en) * | 2017-06-14 | 2018-12-25 | 中国移动通信有限公司研究院 | A kind of traffic forecast method and network element device, computer readable storage medium |
| CN110442453A (en) * | 2019-08-01 | 2019-11-12 | 佛山普瑞威尔科技有限公司 | A kind of automatic telescopic strategy O&M method, system and readable storage medium storing program for executing |
| CN110781006A (en) * | 2019-10-28 | 2020-02-11 | 重庆紫光华山智安科技有限公司 | Load balancing method, device, node and computer readable storage medium |
| CN110933139A (en) * | 2019-11-05 | 2020-03-27 | 浙江工业大学 | System and method for solving high concurrency of Web server |
| CN112817836A (en) * | 2021-02-22 | 2021-05-18 | 深信服科技股份有限公司 | Method, device, equipment and storage medium for determining server load |
| WO2022183802A1 (en) * | 2021-03-05 | 2022-09-09 | 深圳前海微众银行股份有限公司 | Load balancing method, apparatus, and device, storage medium, and computer program product |
| CN115309538A (en) * | 2021-05-08 | 2022-11-08 | 戴尔产品有限公司 | Multi-index based workload balancing between storage resources |
| CN115659165A (en) * | 2022-08-15 | 2023-01-31 | 广东电网有限责任公司 | Method, device, equipment and storage medium for constructing park load data sample |
| WO2023050901A1 (en) * | 2021-09-30 | 2023-04-06 | 深圳前海微众银行股份有限公司 | Load balancing method and apparatus, device, computer storage medium and program |
| CN116016533A (en) * | 2022-12-22 | 2023-04-25 | 中国建设银行股份有限公司 | Automatic weighting load balancing method and system, electronic equipment and storage medium |
| CN116737373A (en) * | 2023-06-09 | 2023-09-12 | 兴业银行股份有限公司 | Load balancing method, device, computer equipment and storage medium |
| CN116886619A (en) * | 2023-08-08 | 2023-10-13 | 北京市大数据中心 | Load balancing method and device based on linear regression algorithm |
| CN116893900A (en) * | 2023-07-19 | 2023-10-17 | 合芯科技有限公司 | Cluster computing pressure load balancing method, system, equipment and IC design platform |
-
2023
- 2023-10-31 CN CN202311429320.4A patent/CN117149099B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102089741A (en) * | 2007-09-17 | 2011-06-08 | 国际商业机器公司 | Executing computer-intensive database user-defined programs on an attached high-performance parallel computer |
| JP2014109886A (en) * | 2012-11-30 | 2014-06-12 | Seiko Precision Inc | Load distribution device, load distribution method and program |
| CN103327094A (en) * | 2013-06-19 | 2013-09-25 | 成都市欧冠信息技术有限责任公司 | Data distributed type memory method and data distributed type memory system |
| EP2963539A1 (en) * | 2014-07-02 | 2016-01-06 | VIA Alliance Semiconductor Co., Ltd. | Split-path fused multiply-accumulate operation using calculation control indicator cache |
| CN109088742A (en) * | 2017-06-14 | 2018-12-25 | 中国移动通信有限公司研究院 | A kind of traffic forecast method and network element device, computer readable storage medium |
| CN107357523A (en) * | 2017-06-27 | 2017-11-17 | 联想(北京)有限公司 | A kind of data processing method and electronic equipment |
| CN110442453A (en) * | 2019-08-01 | 2019-11-12 | 佛山普瑞威尔科技有限公司 | A kind of automatic telescopic strategy O&M method, system and readable storage medium storing program for executing |
| CN110781006A (en) * | 2019-10-28 | 2020-02-11 | 重庆紫光华山智安科技有限公司 | Load balancing method, device, node and computer readable storage medium |
| CN110933139A (en) * | 2019-11-05 | 2020-03-27 | 浙江工业大学 | System and method for solving high concurrency of Web server |
| CN112817836A (en) * | 2021-02-22 | 2021-05-18 | 深信服科技股份有限公司 | Method, device, equipment and storage medium for determining server load |
| WO2022183802A1 (en) * | 2021-03-05 | 2022-09-09 | 深圳前海微众银行股份有限公司 | Load balancing method, apparatus, and device, storage medium, and computer program product |
| CN115309538A (en) * | 2021-05-08 | 2022-11-08 | 戴尔产品有限公司 | Multi-index based workload balancing between storage resources |
| WO2023050901A1 (en) * | 2021-09-30 | 2023-04-06 | 深圳前海微众银行股份有限公司 | Load balancing method and apparatus, device, computer storage medium and program |
| CN115659165A (en) * | 2022-08-15 | 2023-01-31 | 广东电网有限责任公司 | Method, device, equipment and storage medium for constructing park load data sample |
| CN116016533A (en) * | 2022-12-22 | 2023-04-25 | 中国建设银行股份有限公司 | Automatic weighting load balancing method and system, electronic equipment and storage medium |
| CN116737373A (en) * | 2023-06-09 | 2023-09-12 | 兴业银行股份有限公司 | Load balancing method, device, computer equipment and storage medium |
| CN116893900A (en) * | 2023-07-19 | 2023-10-17 | 合芯科技有限公司 | Cluster computing pressure load balancing method, system, equipment and IC design platform |
| CN116886619A (en) * | 2023-08-08 | 2023-10-13 | 北京市大数据中心 | Load balancing method and device based on linear regression algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117149099A (en) | 2023-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109218355B (en) | Load balancing engine, client, distributed computing system and load balancing method | |
| CN107430528B (en) | Opportunistic resource migration to optimize resource placement | |
| US8595364B2 (en) | System and method for automatic storage load balancing in virtual server environments | |
| CN102971724B (en) | The method and apparatus relevant with the management based on modular virtual resource in data center environment | |
| US9807159B2 (en) | Allocation of virtual machines in datacenters | |
| US20110307899A1 (en) | Computing cluster performance simulation using a genetic algorithm solution | |
| JP5998206B2 (en) | Scalable centralized dynamic resource distribution in cluster data grids | |
| US20160092108A1 (en) | Quality of Service Implementation in a Networked Storage System with Hierarchical Schedulers | |
| CN102947796A (en) | Methods and apparatus for movement of virtual resources within data center environment | |
| CN107450855B (en) | Model-variable data distribution method and system for distributed storage | |
| CN102082692A (en) | Method and equipment for migrating virtual machines based on network data flow direction, and cluster system | |
| CN108900626B (en) | Data storage method, device and system in cloud environment | |
| CN103098014A (en) | Storage system | |
| CN110221920B (en) | Deployment method, device, storage medium and system | |
| US10616134B1 (en) | Prioritizing resource hosts for resource placement | |
| US10810054B1 (en) | Capacity balancing for data storage system | |
| CN104679594A (en) | Middleware distributed calculating method | |
| CN115718644A (en) | Computing task cross-region migration method and system for cloud data center | |
| CN104683480A (en) | Distribution type calculation method based on applications | |
| Wang et al. | Provision of storage QoS in distributed file systems for clouds | |
| CN112118314A (en) | Load balancing method and device | |
| CN117149099B (en) | Calculation and storage split server system and control method | |
| CN113688115B (en) | Archive big data distributed storage system based on Hadoop | |
| Khodayarseresht et al. | A multi-objective cloud energy optimizer algorithm for federated environments | |
| CN116932156A (en) | Task processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |