RU2818490C1 - Method and system for distributing system resources for processing user requests - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: invention relates to computer engineering. Result is achieved by steps of: receiving a request from a user device to perform a current task; extracting task parameters from the request and determining the amount of system resources (SR) required to perform the current task; determining availability of SRs allocated to perform tasks of this type; determining the presence of a user task in the list of tasks in the queue to said dedicated SRs; determining the predicted execution time of the current task based on the predicted execution time of the task from the queue; comparing the predicted execution time of the current task with the allowable values of the execution time of the current task, wherein if the predicted execution time of the current task does not go beyond the values of the allowable execution time of the current task, then the current task is put into the list of tasks in the queue; if the predicted execution time of the current task exceeds the limits of the allowable time for executing the current task, then an SR is selected to launch a copy of the software for executing the current task and a command is sent to said dedicated SRs for executing the current task through the copy of the software.

EFFECT: high speed of executing user tasks and bandwidth of system resources.

9 cl, 2 dwg

Description

TECHNICAL FIELD

[0001] The invention relates, in general, to computer technology, and in particular to a method and system for distributing computing resources for processing user requests, including in a cloud environment.

BACKGROUND OF THE ART

[0002] Various solutions are known in the prior art aimed at allocating computing resources to provide services.

[0003] For example, methods and systems for providing a function as a service at predictable times, for example, on a set date or at a certain frequency, are known in the application US 20220357994 A1, publ. 11/10/2022. Disclosed herein is a system comprising at least one computing node providing a platform including a plurality of software-functional services to a plurality of users, wherein the at least one computing node is configured to: run a custom platform-level process associated with a service function; calling a service function to execute a specialized executor function from a configured process at the platform level; performing at least a first initialization step including allocating computing resources to the custom process at the platform level; and when an event associated with a service function is triggered: calling at least one service function performed by a specialized executor function to perform the service function using allocated computing resources; receiving by a specialized executor function a generalized return value from at least one service function; and applying a generic return value in response to the event.

[0004] The disadvantage of the known solution is the lack of tools to balance the load on computing resources, in particular to control the number of copies of containers with running applications.

DISCLOSURE OF INVENTION

[0005] The technical problem or task posed in this technical solution is to create a simple and reliable method and system for allocating computing resources for processing user requests.

[0006] The technical result is to increase the speed of performing user tasks and the throughput of computing resources.

[0007] This technical result is achieved by implementing a method for allocating computing resources for processing user requests, performed by means of a computing device, containing the stages of:

- receive a request from the user’s device to perform a task (current task), containing task parameters;

- extract task parameters from the request and, based on them, determine the amount of computing resources (BP) required to complete the current task;

- determine the availability of BP allocated to perform tasks of this type;

- determining the presence of at least one user task in the list of tasks in the queue to the mentioned allocated BPs;

- determine the predicted execution time of the current task, taking into account the predicted execution time of the task from the queue;

- compare the predicted execution time of the current task with the acceptable values of the current task execution time,

moreover, if the predicted execution time of the current task does not go beyond the limits of the permissible execution time of the current task, then the current task is placed in the list of tasks in the queue; if the predicted execution time of the current task goes beyond the limits of the permissible execution time of the current task, then a BP is allocated to launch a copy of the software to perform the current task and a command is sent to said allocated BPs to perform the current task through the copy of the software.

[0008] In one particular example of the implementation of the method, the placement of the user's current task in the list of tasks in the queue is carried out taking into account the priority of the current task and/or taking into account the values of the allowable execution time of the current task and/or the values of the allowable execution time of the task in the queue.

[0009] In another particular example of the implementation of the method, the stage of allocating a BP to run a copy of the software in it to perform the current task contains a stage at which it is determined whether there are free BPs to run a copy of the software to perform the current task.

[0010] In another particular example of the method, the stage of allocating a BP to run a copy of the software in it to perform the current task contains stages in which:

- determine the lack of free BP to launch a copy of the software to perform the current task;

- search for a container or virtual machine without a queue or with a minimum number of tasks in the queue;

- send a command to the said container or virtual machine without a queue or with a minimum number of tasks in the queue to stop its operation and release BP.

[0011] In another particular example of the method, the steps are additionally performed in which:

- they monitor the number of tasks of the first type in the list of tasks in the queue for each container or virtual machine, and if the number of tasks of the first type in the list exceeds a specified threshold value, then they search for a container or virtual machine without a queue or with a minimum number of tasks of the second type in the queue;

- send a command to the said container or virtual machine without a queue or with a minimum number of tasks in the queue to stop his/her work and release the BPs that were allocated to run software using this container or virtual machine, with the aim of running a task of the first type on the freed ones B.P.

[0012] In another particular example of the method, the steps are additionally performed in which:

- monitor the time spent by dedicated BPs with a running copy of the software in standby mode, and if, after the time interval specified by the developer, no additional user task has been received for the mentioned BPs, then a command is sent to the dedicated BPs to shut down the container or virtual machine with the running copy of the software, after which BP are released.

[0013] In another particular example of the method, the stage of placing the current task in the list of tasks in the queue additionally contains stages in which:

- determine the predicted execution time of the current task using a copy of the software, taking into account the time required to deploy a container or virtual machine with an application using BP;

- compare the predicted execution time of the current task, taking into account the time of the predicted execution of the task from the queue, with the predicted execution time of the current task using a copy of the software;

wherein the placement of the current task into the list of tasks in the queue is carried out after determining that the predicted execution time of the current task, taking into account the time of the predicted execution of the task from the queue, is less than the predicted execution time of the current task using a copy of the software.

[0014] In another particular example of the implementation of the method, the software through which the user's task is performed is launched through a container or virtual machine instance with the software running in it.

[0015] In another preferred embodiment of the claimed solution, a system for distributing computing resources for processing user requests is provided, comprising at least one computing device and at least one memory device containing computer-readable instructions that, when executed by at least one computing device, perform the above method.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The features and advantages of the present technical solution will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

- in Fig. 1 shows the general system for distributing computing resources;

- in Fig. Figure 2 shows an example of a general view of a computing device.

IMPLEMENTATION OF THE INVENTION

[0017] The concepts and terms necessary to understand this technical solution will be described below.

[0018] In this technical solution, a system means, including, a computer system (in particular, an information system, computing systems, computing clusters), a computer (electronic computer), CNC (computer numerical control), PLC (programmable logic controller ), computerized control systems, specialized devices that implement calculations using FPGA and ASIC technologies, and any other devices capable of performing a given, clearly defined sequence of operations (actions, instructions).

[0019] By command processing device is meant an electronic unit, a computing device, or an integrated circuit (microprocessor) that executes machine instructions (programs).

[0020] An instruction processing device reads and executes machine instructions (programs) from one or more storage devices. Storage devices can include, but are not limited to, hard drives (HDD), flash memory, ROM (read-only memory), solid-state drives (SSD), and optical drives.

[0021] Program - a sequence of instructions intended for execution by a computer control device or command processing device.

[0022] Database (DB) - a collection of data organized in accordance with a conceptual structure that describes the characteristics of that data and the relationships between them, and such a collection of data that supports one or more areas of application (ISO/IEC 2382:2015, 2121423 " database").

[0023] A signal is a material embodiment of a message for use in the transmission, processing and storage of information.

[0024] Logical element - an element that implements certain logical dependencies between input and output signals. Logic elements are usually used to construct logical circuits of computers and discrete automatic monitoring and control circuits. All types of logical elements, regardless of their physical nature, are characterized by discrete values of input and output signals.

[0025] Automated system (AS) is an organizational and technical system that ensures the development of solutions based on the automation of information processes.

[0026] According to the diagram shown in FIG. 1, the system 100 for distributing computing resources for processing user requests contains: user devices 10, 11, authorization server 20, computing task management device 30, executive device 40. To connect the mentioned devices, well-known wireless and/or wired data transmission channels can be used .

[0027] The user device (10, 11) may be a laptop or desktop computer, a computer terminal, an automatic terminal device, a software module of another information system or device, a telephone, a smartphone, a tablet, or other computing device equipped with wired and/or wireless means communications.

[0028] The authorization server 20 is a hardware and software complex that performs the function of automatically identifying users and checking for access to services provided by the system 100.

[0029] The computing task management device 30 may be at least one server and is equipped with a task scheduler 31 implemented on a server equipped with appropriate software and processing a queue of transport tasks. The task scheduler 31 polls the devices of the system 100 for the presence of suitable events that will trigger the initialization of the desired operations.

[0030] The execution device 40 can be implemented on a server and contains a resource allocation process control module 41 implemented on the basis of a computing device, a model storage 42, BP 43,44, 45, a computing task control module 46, a sequence control module 47, and module 49 for managing the request queue. To implement the mentioned modules, well-known logic elements on transistors, ADCs, DACs, clock generators, etc. can be used.

[0031] Real or virtual CPUs, GPUs, FPGAs and other known computing units, RAM, disk space memory, network resources, etc. can be used as BP.

[0032] The user, through the device 10, can submit a request to perform a task using the BP. A task request may contain:

- User ID, for example, username;

- information for user authorization, for example, login, password, token, etc.;

- task ID,

- task parameters, which may include: the name of the task being performed, the name of the container with the task being performed and the model, model ID, the name of the virtual machine with the executable computing program code, the address where the image is placed in the registry of containers or virtual machines, etc.;

- code or name of the computing region or cluster (if this parameter is absent, the task is executed in the default computing region);

- requested computing resources (if these parameters are missing, a default set of computing resources is provided)

- operating mode - synchronous or asynchronous

- parameters containing rules for provisioning and disabling computing resources

- a set of parameters that initialize a computational task (arguments of a calculated function or a dataset with data for performing calculations, for example, boundary conditions when solving a system of differential equations) and obtaining a model prediction (dataset, prompt or seed request), also the set of parameters can be empty,

- parameters describing the priority of executing a specific request/task,

- parameters describing the way to obtain calculation results, including transfer of the calculation result to the user in the form of a response to the user’s request, or saving data on permanent data storage (for example, HDD or SSD hard drives), or transfer to external program modules and systems.

[0033] A directed request to perform a task arrives at the authorization server 20. When a user request is received, the authorization server 20, using known methods, checks the request format, authorizes the user for the right to launch and execute a task and calculate the model prediction, checks and confirms the user’s rights to allocate and provide system resources and the rights to launch a task and containers to complete the task in accordance with rights granted to the user in the system, and the right to obtain the results of calculations and model predictions. If the user's authorization is confirmed, the notification server informs the user about successful authorization, or about refusal of authorization, or about the existence of restrictions that prevent the request from being completed, or about an authorization error. Information may include providing the user with a message code, a test message describing the result, or a message code describing the result. If necessary, user data can be stored in a user database integrated with the authorization servers 20.

[0034] If the authorization of the request source is not confirmed, the authorization server may, depending on the information security settings and the number of repeated requests that fail authorization:

- refuse authorization and send a notice of refusal of authorization to the source of the request without specifying the reasons;

- refuse authorization and send to the source of the request a notification of refusal of authorization indicating the reason and/or issuance of an error code;

- refuse authorization without sending a notification of refusal of authorization to the source of the request;

- limit the subsequent number of authorization attempts with or without notification to the source of the request;

- block further receipt of requests from the request source;

- block further execution of requests from the request source.

[0035] Another way to receive a request to deploy a computing environment and perform a computing task and compute a model prediction is to receive an API call arriving at an API gateway generated by a user application, an external automated system or software module, or generated and submitted by a user using tools for work in CLI program line mode (CurI utilities, Postman application and other tools). The API call must contain the same set of parameters as the user request.

[0036] Accordingly, if the authorization of the device 10 is successful, then a request to execute a task is sent to the computing task management device 30, which is further processed depending on the load of the system 100 and the queue of tasks to be executed.

[0037] In the first embodiment of the system, in the absence of a task queue, the device 30 extracts the ID and parameters of the task from the request to execute the task and based on the task parameters (for example, “name of the task being performed”, “name of the container with the task being performed and model”, "Model ID", "computing region code", "requested computing resources") determines the number of BPs required to complete the task. For example, to determine said number of BPs, device 30 may be equipped with a memory that stores a list of tasks, a list of their parameters, a list of software (e.g., machine learning models) that will be used to perform the task according to the parameters, and a list of BPs that should be use to run each machine learning model.

[0038] For example, a task request may contain task ID and parameters that require generating text data based on specified keywords (first task) or generating an image based on specified keywords (second task). For example, a request to perform a task may contain ID and task parameters, according to which it is necessary to generate text data based on specified keywords (first type of task) or generate an image based on specified keywords (second type of task). The parameters of the first task type of the "requested computing resources" task may indicate the need to use 4 CPUs and 32 GB of memory to run the model for text generation, and the parameters of the second task type of the "requested computing resources" task may indicate the need to use 16 CPUs, 4 GPUs and 320 GB of memory to run the model to generate the image. Machine learning models can be implemented using well-known methods, for example, based on deep learning neural networks.

[0039] Once the number of BPs required to perform a task has been determined, device 30 contacts BP scheduler 31 by sending an appropriate request to determine the availability of dedicated BPs or containers running software to perform the user's task, in particular dedicated BPs running a model for text generation or a model for image generation. If, in response to a request from the scheduler 31, information is received about the presence of the mentioned dedicated BPs, for example, BP 43, then the device 30 generates a request to the execution device 40 to process the user request to perform a task using the dedicated BPs 43, after which the results of the completed task, for example , the generated text or image is sent to the user's device 10, or redirected to another target storage location (in the form of saving to hard drives and generating an API call to external systems). In an alternative implementation of the substituted solution, instead of the mentioned container, an instance of a virtual machine with software running in it can be used to perform the user’s task.

[0040] In the second version of the system operation, in the absence of a task queue, the scheduler 31 can determine that there is no information in memory about the presence of the mentioned allocated BPs or containers with running software to perform the user's task, after which the scheduler 31 sends a request to the actuator 40 to determine the number of free BPs, information about which the actuator 40 sends in response to the request. The scheduler 31 compares information about the number of BPs required to complete a task with information about the number of free BPs, and if there are enough free BPs to complete the task or the sum of the involved BPs and the required BPs does not exceed a set threshold, then the scheduler 31 sends information about availability of free BPs to perform the task, after which the request to perform the task is sent by the device 30 to the actuator 40.

[0041] Accordingly, the mentioned request arrives at the computational task management module 46, which, based on the request information, allocates BPs (43, 44 or 45) and, using known methods, deploys a container to run software in it using the allocated BPs, in particular with a machine learning model to execute a task according to the task ID.

[0042] After the BPs have been allocated, the module 46 informs the flow control module 47 and the resource allocation process control module 41 about the allocated BPs and their readiness for operation. Also, module 46 informs scheduler 31 about allocated BPs with running software for storing this information in memory and for updating information about free BPs. Next, module 46 sends a command to the dedicated BPs to perform the user's task through software running in the container. This ensures an increase in the speed of completing user tasks and the throughput of computing resources.

[0043] The results of the completed task, for example, generated text or image, allocated by BP are transmitted through module 46 to device 40 for transfer to user device 10, or to another target storage location (in the form of saving to hard drives and generating an API call to external systems).

[0044] After the user task is completed, i.e. module 46 received the results of the completed task from the allocated BPs, module 46 records the time of the completed task and starts a counter with which module 46 can be equipped to track the time spent by the allocated BPs in idle mode. If, after a time interval specified by the developer, module 46 has not received an additional user task for the mentioned BPs, then module 46 sends a command to the allocated BPs to shut down the container with the running software, after which the BPs are released. Information about freed resources is sent by module 46 to scheduler 31 to update information about free BPs.

[0045] In the third embodiment of the system, if there is a task queue, the actuator 40 can be equipped with a request queue management module 49. Accordingly, after determining the presence of dedicated BPs running software in this embodiment of the presented solution, device 30 generates a request to execution device 40 to perform a user task using dedicated BPs, containing the ID of the BP or container running software, which is sent through module 46 to module 49 .

[0046] Next, module 49 accesses the list of tasks in the queue for the container intended to perform the task launched by BP and determines the presence of at least one user task in the list of tasks in the queue, after which module 49 enters information about the new one into the list. task in accordance with the request received from the device 30. Additionally, when queuing, parameters that describe the priority of task execution can be taken into account.

[0047] Accordingly, after the user's task from the list is completed using the allocated BPs, information about the completed task from the BP is sent to module 49, which excludes information about the completed task from the list of tasks for execution, after which module 49 sends information about the next task from the mentioned list to BP to perform the task using software running in the container.

[0048] Additionally, module 49 may be configured, after determining the presence of at least one or more user tasks in the queued task list, to send a request to scheduler 31 to check for free BPs to run a copy of the software to perform the user task. Alternatively, module 49, after determining the presence of at least one user task in the queued task list, may send a request to scheduler 31 to determine when to complete the user task. The execution time of a user's task can be determined by the scheduler 31 by a well-known method, for example, by analyzing previously allocated BPs when they previously performed similar user tasks.

[0049] Based on the task execution time received from the scheduler 31 and the start time of the task execution from the queue, which can also be recorded by known methods after the start of execution of the said task by means of BP, module 49 determines the predicted execution time of the task from the queue, and then, based on the predicted execution of the task from the queue and the execution time of the user task received from the scheduler 31, module 49 determines the predicted execution time of the user task received from the execution device 40, i.e. current task. The predicted execution time of the current task is compared by module 49 with the acceptable execution time of the current task, information about which may be contained in a previously received request to perform the task in accordance with user input.

[0050] If the predicted execution time of the current task is within the acceptable execution time of the current task, then module 49 may place the current task in the list of tasks in the queue.

[0051] If the predicted execution time of the current task is outside the acceptable execution time of the current task, then module 49 sends a request to scheduler 31 to check whether there are free BPs to run a copy of the software to complete the user's task. If there are enough free BPs to launch a copy of the software to complete the user’s task, then module 49 sends a corresponding notification to the computational task management module 46, which, based on the request information, allocates BPs (43, 44 or 45) and, using known methods, deploys a container to run a copy in it Software using dedicated BPs.

[0052] Next, module 49 sends a command to the allocated BPs to perform the user's task through a copy of the software running in the container, and the results of the completed task, the allocated BPs are transmitted through module 46 to device 40 for transmission to user device 10, or to another target location storage (in the form of saving to hard drives and generating an API call to external systems). Thus, the speed of completing user tasks and the throughput of computing resources are further increased.

[0053] After the user task is completed, i.e. module 46 received the results of the completed task from the allocated BPs, module 46 records the time of the completed task and starts a counter with which module 46 can be equipped to track the time spent by the allocated BPs running a copy of the software in standby mode. If, after a time interval specified by the developer, module 46 has not received an additional user task for the mentioned BPs, then module 46 sends a command to the allocated BPs to shut down the container with the running copy of the software, after which the BPs are released. Information about freed resources is sent by module 46 to scheduler 31 to update information about free BPs.

[0054] In an alternative embodiment, module 49 may be configured to determine the predicted completion time of a current task using a copy of the software designed to perform said task. To do this, module 49 requests information from scheduler 31 about the time it takes to deploy the application container using BP. Said container deployment time can be determined by the scheduler 31 by a well-known method, for example by polling allocated BPs when they deploy the container.

[0055] Based on the container deployment time and the task execution time, module 49 determines the predicted execution time of the current task using the copy of the software. Next, the device compares said predicted execution time of the current task with the previously obtained predicted execution time of the user's task, determined taking into account the predicted execution of the task from the queue.

[0056] If the predicted execution time of the user's task, determined taking into account the predicted execution time of the task from the queue, is less than the predicted execution time of the current task using the software copy, then the module 49 puts the current task into the task queue according to the priority value. If the predicted execution time of a user's task, determined taking into account the predicted execution time of a task from the queue, is greater than the predicted execution time of the current task using a copy of the software, then module 49 initiates the deployment of a container to run a copy of the software in it using dedicated BPs and process the task as described earlier way.

[0057] It can also be determined by module 49 that the predicted execution time of the current task, in particular the current first task, is beyond the acceptable execution time of the current task, and there are not enough free BPs to run a copy of the software to complete the user's task. Next, module 49 can analyze lists of queues for containers with running copies of the software to perform a second task to find a container without a queue or with a minimum number of tasks in the queue, after which module 49 can send a command to said container without a queue or with a minimum number of tasks in the queue to stopping its operation and freeing BPs that were allocated to run software using this container in order to run the current task or another task using these BPs (higher priority or running with a long latency).

[0058] Additionally, when queuing a task, module 49 may take into account the task execution time values so that all tasks in the queue are completed according to their execution time values.

[0059] In an alternative implementation, module 49 may monitor the number of tasks, in particular the first tasks, in the task list in the queue for each container, and if the number of tasks in the list exceeds a predetermined threshold, then module 49 can analyze the queue lists to containers with running copies of the software to perform a second task to search for a container without a queue or with a minimum number of tasks in the queue, after which module 49 can send a command to said container without a queue or with a minimum number of tasks in the queue to stop its operation and release the BPs that were allocated to run software using this container in order to run the current task or another task on these BPs (higher priority or running with a long latency)..

[0060] Accordingly, as soon as the BPs are released, module 49 initiates the deployment of a container to run a copy of the software in it using the allocated BPs to perform the current task in the manner previously described. Tasks in the queue list for a container whose operation has been stopped can be distributed by module 49 among other containers. Thus, the speed of completing user tasks and the throughput of computing resources are further increased.

[0061] In general (see Fig. 2), the computing device (200) contains one or more processors (201), united by a common information exchange bus, including computing ones for accelerating calculations - GPU modules (Graphics Processing Unit). , TPU (Tensor Processing Unit), IPU (Integrated Processing Unit), as well as modules based on FPGA and ASIC microprocessors, memory devices such as RAM (202) and ROM (203) , input/output interfaces (204), input/output devices (205), and network communication devices (206).

[0062] The processor (201) (or multiple processors, multi-core processor, etc.) may be implemented based on a CISC, RISC, ARM, etc. architecture, and be selected from a variety of devices commonly used today, such as manufacturers such as: Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, Huawei Kunpeng™ and Nvidia™, IBM™, Baikal™, IVA™, etc. The processor or one of the used processors in the device (200) must also take into account a computational accelerator implemented, for example, in the form of a graphics processor, for example an NVIDIA GPU with a programming model compatible with CUDA, Graphcore, TPU (Tensor Processing Unit), IPU (Integrated Processing Unit), as well as modules based on FPGA and ASIC microprocessors, the type of which is also suitable for full or partial implementation of the method, and can also be used for training and applying machine learning models in various information systems.

[0063] RAM (202) is a random access memory and is designed to store machine-readable instructions executed by the processor (201) to perform the necessary logical data processing operations. The RAM (202) typically contains executable operating system instructions and associated software components (applications, program modules, etc.). In this case, the available memory capacity of a graphics card, graphics processor, or other computing accelerators can act as RAM (202). [0064] The ROM (203) is one or more permanent storage devices, such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[0065] To organize the operation of device components (200) and organize the operation of external connected devices, various types of I/O interfaces (204) are used. The choice of appropriate interfaces depends on the specific design of the computing device, which can be, but is not limited to: PCI, AGP, PS/2, IrDa, FireWire, COM, SATA, IDE, USB (2.0, 3.0, 3.1, micro, mini, type C ), HDMI, DVI, VGA, RJ45, Ethernet RJ45, single-mode and multimode optical interfaces SC and ST, WiFi, NVLink, InfiniBand, STM, ATM, etc.

[0066] To ensure user interaction with the computing device (200), various means (205) of I/O information are used, for example, a keyboard, a display (monitor), a touch display, a touch pad, a joystick, a mouse, a light pen, a stylus, touch panel, trackball, speakers, microphone, augmented reality tools, optical sensors, tablet, light indicators, projector, camera, biometric identification tools (retina scanner, fingerprint scanner, voice recognition module), etc.

[0067] The network communication means (206) provides data transmission via an internal or external computer network, for example, an Intranet, the Internet, a LAN, etc. One or more means (206) may include, but are not limited to: Ethernet card, single-mode and multimode optical interface modules, NVLink, InfiniBand, STM, ATM, etc.

[0068] Additionally, satellite navigation tools can also be used as part of the device (200), for example, GPS, GLONASS, BeiDou, Galileo. Also, precise time signals received from the above-mentioned satellite navigation devices can be used to synchronize the operation of geographically dispersed devices.

[0069] The specific selection of device elements (200) for implementing various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[0070] Modifications and improvements to the above-described embodiments of the present technical solution will be apparent to those skilled in the art. The foregoing description is provided by way of example only and is not intended to be limiting. Thus, the scope of the present technical solution is limited only by the scope of the attached claims.

Claims (26)

1. A method for distributing computing resources to process user requests, performed by means of a computing device, containing the steps of: - receive a request from the user’s device to perform a task (current task), containing task parameters; - extract task parameters from the request and, based on them, determine the amount of computing resources (BP) required to complete the current task; - determine the availability of BP allocated to perform tasks of this type; - determining the presence of at least one user task in the list of tasks in the queue to the mentioned allocated BPs; - determine the predicted execution time of the current task, taking into account the predicted execution time of the task from the queue; - compare the predicted execution time of the current task with the acceptable values of the current task execution time, moreover, if the predicted execution time of the current task does not go beyond the limits of the permissible execution time of the current task, then the current task is placed in the list of tasks in the queue; if the predicted execution time of the current task goes beyond the limits of the permissible execution time of the current task, then a BP is allocated to launch a copy of the software to perform the current task and a command is sent to said allocated BPs to perform the current task through the copy of the software. 2. The method according to claim 1, characterized in that the placement of the user’s current task in the list of tasks in the queue is carried out taking into account the priority of the current task and/or taking into account the values of the allowable execution time of the current task and/or the values of the allowable execution time of the task in the queue. 3. The method according to claim 1, characterized in that the stage of allocating a BP to run a copy of the software in it to perform the current task contains a stage at which the availability of free BP to run a copy of the software to perform the current task is determined. 4. The method according to claim 1, characterized in that the stage of allocating a BP to run a copy of the software in it to perform the current task contains stages in which: - determine the lack of free BP to launch a copy of the software to perform the current task; - search for a container or virtual machine without a queue or with a minimum number of tasks in the queue; - send a command to the said container or virtual machine without a queue or with a minimum number of tasks in the queue to stop its operation and release BP. 5. The method according to claim 1, characterized in that additional steps are performed in which: - they monitor the number of tasks of the first type in the list of tasks in the queue for each container or virtual machine, and if the number of tasks of the first type in the list exceeds a specified threshold value, then they search for a container or virtual machine without a queue or with a minimum number of tasks of the second type in the queue; - send a command to the said container or virtual machine without a queue or with a minimum number of tasks in the queue to stop his/her work and release the BPs that were allocated to run software using this container or virtual machine, with the aim of running a task of the first type on the freed ones B.P. 6. The method according to claim 1, characterized in that additional steps are performed in which: - monitor the time spent by dedicated BPs with a running copy of the software in standby mode, and if, after the time interval specified by the developer, no additional user task has been received for the mentioned BPs, then a command is sent to the dedicated BPs to shut down the container or virtual machine with the running copy of the software, after which BP are released. 7. The method according to claim 1, characterized in that the stage of placing the current task in the list of tasks in the queue additionally contains stages in which: - determine the predicted execution time of the current task using a copy of the software, taking into account the time required to deploy a container or virtual machine with an application using BP; - compare the predicted execution time of the current task, taking into account the time of the predicted execution of the task from the queue, with the predicted execution time of the current task using a copy of the software; wherein the placement of the current task into the list of tasks in the queue is carried out after determining that the predicted execution time of the current task, taking into account the time of the predicted execution of the task from the queue, is less than the predicted execution time of the current task using a copy of the software. 8. The method according to claim 1, characterized in that the software through which the user’s task is performed is launched through a container or a virtual machine instance with the software running in it. 9. A system for distributing computing resources for processing user requests, comprising at least one computing device and at least one memory device containing machine-readable instructions, which, when executed by at least one computing device, perform the method according to any one of claims. 1-8.

Images