RU2775354C1 - Method and apparatus for launching arbitrary (untrusted) code on a cluster in an isolated environment - Google Patents

Abstract

FIELD: computing technology.

SUBSTANCE: method for launching user software code on a computing node, containing the stages of: receiving a request to launch user software code on a computing node in an isolated data processing environment; delivering the user software code to the computing node; launching an isolated environment on the computing node; loading the user software code on the isolated environment; determining data for the user software code to be processed by the user software code; uploading the data for the user software code to the isolated environment; processing the data for the user software code in the isolated environment; retrieving and storing the data processing results on at least one data storage apparatus.

EFFECT: provided possibility of automated launch of user software code on a computing node in an isolated environment.

6 cl, 2 dwg

Description

FIELD OF TECHNOLOGY

[0001] The presented technical solution relates, in general, to the field of digital data processing, and in particular to a method and device for running arbitrary user (untrusted) code on a cluster or server in an isolated environment.

BACKGROUND OF THE INVENTION

[0002] In the prior art, solutions are known that allow you to run user code in an isolated environment.

[0003] In particular, a solution is known that provides the ability to isolate Linux processes in their own small system environments, disclosed in the article "Separation Anxiety: A Tutorial for Isolating Your System with Linux Namespaces" (http://hadooptutorial.info/separation-anxiety -a-tutorial-for-isolating-vour-system-with-linux-namespaces/).

[0004] A cloud distributed computing environment (CBDCE) control method is also known, as disclosed in US 2020/0068010 A1, publ. 02/27/2020. In this solution, multiple services run concurrently on CBDCE compute nodes, with each service containing multiple service instances that run concurrently on multiple separate CBDCE compute nodes. During operation, the system uses a distributed database to keep track of the state of the CBDCE to ensure that the CBDCE is always stable and scalable. Upon receiving a request related to the CBDCE configuration, the service accesses the CBDCE status information from the distributed database in order to respond to the request.

[0005] Such solutions do not allow you to safely run user code on a cluster or server (on an arbitrary node), because have the following disadvantages:

- access to data by an untrusted code is not excluded;

- there is no possibility to safely run arbitrary code on a cluster or server (i.e. in an industrial (industrial) environment) and protection from malicious code, destructive actions, errors, etc.;

- there is no safe way to deliver data to an isolated environment for the possibility of secure processing of such data by untrusted code on a cluster or server;

- do not provide delivery and launch of untrusted code in prom. environment;

- do not allow to determine the necessary resources (CPU, Memory) to run untrusted code.

ESSENCE OF THE TECHNICAL SOLUTION

[0006] The technical problem or technical challenge posed in this technical solution is to create a new efficient, simple and secure solution for the automated execution of arbitrary (untrusted) code on a cluster or server in an isolated environment.

[0007] The technical result achieved when solving the above technical problem or technical problem is to enable automated launch of arbitrary (untrusted) code, in particular user code, on a computing node in an isolated environment.

[0008] The specified technical result is achieved due to the implementation of a method for launching a user (untrusted) code on a computing node, performed by at least one computing device, comprising the steps of:

- receive a request to run the user program code;

- deliver the user program code to the computing node

- running on the computing node an isolated environment designed for the safe execution of computer programs;

- upload user code to the isolated environment;

- loading into the sandbox data for the user code, which will be processed by the user code;

- processing data for the user code through the user code in an isolated environment to obtain data processing results;

- extract the results of data processing from the isolated environment and store them in at least one data storage device.

[0009] In one particular embodiment of the method, the request to run the user code contains user code and/or data for the user code, wherein the user code and/or data for the user code to load into the sandbox is retrieved from said request.

[0010] In another particular embodiment of the method, the request to run the user code contains a link to the user code and/or a link to data for the user code to be processed by the user code, wherein the user code and/or data for the user code to be loaded into the sandbox the medium is retrieved from the data storage device via said links contained in said request.

[0011] In another particular embodiment of the method for loading data for user code into an isolated environment and extracting the results of data processing, a named pipe or unix socket is formed to exchange data with running user code in an isolated environment or provide direct upload / download of the mentioned data and code in folder inside the sandbox.

[0012] In another particular embodiment of the method, the request to run the user program code further comprises computing resource parameters, the method further comprising determining a computing node to run the custom code based on said computing resource parameters depending on the computing characteristics of the computing node and their current computing load.

[0013] In another preferred embodiment of the claimed solution, a device for launching user (untrusted) code on a computing node is presented, containing at least one computing device and at least one memory containing machine-readable instructions that, when executed by at least one computing device perform the above method.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The features and advantages of the present technical solution will become apparent from the following detailed description of the technical solution and the accompanying drawings, in which:

[0015] In FIG. 1 is a diagram of a data processing system.

[0016] In FIG. 2 is an example of a general view of a computing device.

IMPLEMENTATION OF THE TECHNICAL SOLUTION

[0017] The concepts and terms necessary for understanding this technical solution will be described below.

[0018] In this technical solution, the system means, including a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given , a well-defined sequence of operations (actions, instructions).

[0019] A command processing device is an electronic unit, a computing device, or an integrated circuit (microprocessor) that executes machine instructions (programs).

[0020] The instruction processing device reads and executes machine instructions (programs) from one or more data storage devices. The role of a storage device can be, but not limited to, hard drives (HDD), flash memory, ROM (read only memory), solid state drives (SSD), optical drives.

[0021] A program is a sequence of instructions intended to be executed by a computer control device or command processing device.

[0022] A virtual machine is an abstract computer whose operation is implemented (emulated) on a real machine using software and/or hardware.

[0023] A trusted code is a code that, before running on prom. environment, has gone through certain stages that guarantee the safety of running such code in prom. environment. As a rule, these stages include: code review (by other developers); performing static code analysis with various tools (for example, Sonar) to identify weaknesses or malicious code; testing, including: functional, load testing and acceptance testing, fixing artifacts, for example, in the form of a distribution kit, which is installed on the industrial platform according to the established DevOps process. Wednesday. The trusted code problem is the output time in the prom. environment, because to test the code, you must first develop tests, etc.

[0024] Arbitrary user (untrusted) code is code that has not gone through the steps inherent in trusted code and therefore cannot be considered safe, since it may contain malicious code, destructive actions, errors, etc. Hereinafter, the terms "user" and "untrusted" as used in relation to the term "code" are used interchangeably.

[0025] The need to run untrusted code in prom. environment arises, for example, when data engineers (Data Engineer) or experts in the analysis of data arrays (Data Scientist) develop data transformation code or develop models written, for example, in Python code, and want to quickly and automatically implement such code in industry . environment, including those with minimal checks performed by the developers themselves, but without going through the stages for trusted code. The problem with untrusted code is its security. nobody checks such code.

[0026] The presented data processing system solves the problem of automated delivery and safe launch of arbitrary (untrusted) code on a cluster (industrial environment) due to:

- ensuring the delivery and launch of untrusted code in the prom. environment;

- providing such a code with access only to data intended for processing by said code (the trusted code of said system determines what data is required for processing, reads the data from the source and puts it in an isolated environment of the untrusted code);

- exclusion of the possibility of access to any other data by blocking access to the network from an isolated environment (access to any storages, databases, file systems, etc. is closed);

- allocation of a certain amount of resources (CPU, MEM, etc.) for the operation of untrusted code, which eliminates malicious actions associated with attempts to overload the system

- blocking any other malicious activity even if the account under which such code was run is compromised.

[0023] According to the diagram shown in FIG. 1, the data processing system includes: a user (untrusted) code launch control device 10 and a user (untrusted) code automated launcher 100. Said devices may be implemented on the basis of at least one computing device (including virtual machines/devices) configured in firmware in such a way as to perform the functions assigned to them below. For example, the user code launch control device 10 may be a laptop or desktop computer, telephone, smartphone, tablet, or other computing device configured to generate and transmit a request to run the user code to the automated user code launcher 100 . The request to run the user code may be transmitted using wired or wireless data channels widely known in the art.

[0027] The user code launcher 100 may include: a task scheduling module 101; a computing resource management module 102; at least one computing node control module 103; at least one sandbox launcher 104; at least one sandbox communication module 105; at least one module 106 run code in an isolated environment; and at least one data storage module 107 . All of the listed modules can be implemented on the basis of the firmware of the mentioned device 100.

[0028] For example, task scheduling module 101 may be implemented on Apache Oozie, a server-side workflow scheduling system for managing Hadoop jobs.

[0029] Compute resource management module 102 may be implemented on the basis of YARN (Yet Another Resource Negotiator), a module responsible for cluster resource management and job scheduling.

[0030] Compute node management module 103 can be implemented on the basis of a YARN NodeManager that manages all nodes in a YARN cluster (see, for example, the article https://www.ibm.com/developerworks/ru/library/bd-hadoopyarn/).

Sandbox launcher 104 may be implemented based on a Linux kernel capable of implementing a namespace function. Namespace (from the English namespaces) is a feature of the Linux kernel that allows you to isolate and virtualize the global system resources of many processes. Examples of resources that can be virtualized are process IDs, hostnames, user IDs, network access, interprocess communication, and filesystems. For a more detailed description of the namespace function, see the article "user_namespaces - overview of Linux user namespaces" posted on the Internet at: https://man7.org/linux/man-pages/man7/user_namespaces.7.html.

[0032] The sandbox communication module 105 is designed to interact with the data storage module 107, and to supply and receive data and user code to/from the sandbox. Said module 105 may be implemented based on the unix named pipes/sockets technology disclosed in the article "A Socket-based IPC Tutorial" published on the Internet at: http://www.qnx.com/developers/docs/qnx 4.25 docs /tcpip50/prog_guide/sock_ipc_tut.html.

[0033] The sandbox code launcher 106 may be implemented on at least one computing device equipped with a memory device and configured to run user code stored in the memory device.

[0034] The data storage unit 107 may be a hard disk drive (HDD), a solid state data drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media (CD-R/RW, DVD-R /RW, BlueRay Disc, MD), etc. In the data storage unit 107, at least one user code and data to be processed with the user code, hereinafter referred to as user code data, can be stored. In an alternative implementation of the presented technical solution, said device 100 may contain only one data storage module 107, which will be common to all computing nodes.

[0035] In the first step of the data processing system, the user code launch control device 10 generates a request to run the user program code, which is arbitrary (untrusted) code. The formation of said request may be initiated by the user of the device 10 by means of data input devices widely known in the art. For example, the user through a specialized graphical interface can specify: a link to the user code, which is stored in the module 107 data storage; a link to data for the user code that will be processed by the user code; and parameters characterizing computing resources for running user code.

[0036] Accordingly, after the user enters the above information, the user code launch control device 10 generates a request to run the user code, which includes a link to the user code, a link to data for the user code, and computing resource parameters, after which the generated request is sent to device 100 for automated launching of user code.

[0037] In an alternative implementation of the claimed solution, said request to run the user program code, instead of referring to the user code, may contain directly the user code, and instead of referring to data for the user code, directly the data for the user code, which should be processed by the user code. The user code and data for the user code can be downloaded via the dedicated graphical interface mentioned earlier and added to a request to run the user code by said device 10. Thus, delivery of the user code and data to the computing node can be performed.

[0038] When said device 100 receives a request to run a user program code, said request goes to the task scheduling module 101, which determines a computing node to run the user code. For example, to determine said computing node, said module 101 may generate and send a corresponding request to the computing resource management module 102, which, in response to the request, will send data about the computing nodes, their computing characteristics, and the current computing load. The list of computing nodes and their computing characteristics can be stored in advance in the mentioned module 102, and data on the current computing load of the computing nodes can be requested by the module 102 from at least one computing node management module 103 by methods widely known from the prior art.

[0039] When the module 101 receives data about the computing nodes, their computing characteristics and the current computing load, the said module 101, for example, selects the least loaded computing node, the computing characteristics of which correspond to the parameters characterizing the computing resources for running user code. Next, the module 101 generates a request to run the user code, which includes data about the computing node to run the user code, a link to the user code, and a link to the data for the user code, after which the generated request is transmitted to the module 102 of the computing resource management, which searches for the computing node specified in the compute node data to run the user code, and sends the generated request to this compute node.

[0040] Accordingly, the request to run the user code is received by the computing node control module 103, which, upon receiving the above request, starts the sandbox launch module 104 by sending the appropriate command, and then sends the received request to the said module 104. Upon receipt of the start request user code module 104 executes sandbox launch in sandbox code launcher 106 . For example, launching the sandbox can be accomplished by having module 104 place folders and files of an application (e.g., an operating system) intended to run user code, whereupon module 104 extracts from the received request a user code reference and a data reference for the user code, which are sent to module 105 communication with an isolated environment. Additionally, module 104 may direct module 105 to a folder containing sandbox files so that user code data and/or user code data can be placed in that folder.

[0041] The sandbox communication module 105 references the user code and retrieves the user code from the data storage module 107, after which the extracted data is loaded by module 105 into the sandbox running in module 106. To download the user code, for example, , the link to the folder received from the module 104 earlier is used. Once the user code has been loaded into module 106, module 105 sends a notification that the user code has been loaded, after which module 104 sends a command to module 106 to execute the user code in a sandbox (i.e., module 104 initiates the execution of user code in module 106) . Thus, the delivery of the user program code and data to the computing node is ensured.

[0042] Further, the mentioned module 105 creates a named pipe (named pipeline (English named pipe) or an alternative unix socket (for the case of two-way exchange) for exchanging data with running user code in an isolated environment, after which the mentioned module 105 retrieves the data by reference for user code from the storage module 107 and sends the data via a named pipe to the sandbox of the module 106 for processing the retrieved data to be processed by the user code.

[0043] In an alternative implementation of the claimed solution, after loading the user code into the sandbox, the data sandbox communication module 105 may load the data intended for processing by the user code into the sandbox in a specially allocated memory area of the module 106, for example, in a folder, designed to contain the said data. Accordingly, once the user code and data for the user code have been loaded into the module 106, the module 104 instructs the module 106 to execute the user code in a sandbox to process the stored data for the user code.

[0044] The data processing results for the user code can be stored in the code launcher 106 in an isolated environment, for example, in a dedicated memory area of the module 106 (in particular, located in a folder intended for storing data processing results), after which the processing results data can be retrieved by sandbox communication module 105 from said module 106, and stored in data storage module 107, or sent directly to user code launch control device 10 in response to a request to run user program code. The results of data processing can be viewed by the user, or another person having access to the data stored in the data storage unit 107, using well-known information display means.

[0045] Accordingly, if the request to run the user code instead of the references mentioned contains directly the user code and/or data for the user code, then this data is extracted from the request by the sandbox launcher 104 and placed in the sandbox code launcher 106 after the sandbox is started. environment, for example, by placing files of user code and/or data for user code in a dedicated memory area of the module 106, for example, in a folder designed to accommodate said data. Further, after the module 106 has processed the data intended for the user code, the said module 106 sends a command to the communication module 105 with an isolated environment to retrieve the data processing results from the module 106 and store them in the data storage module 107, or sent directly to the launch control device 10 user code in response to a request to run user code.

[0046] In general terms (see Fig. 2), the computing device (200) contains one or more processors (201), memory means, such as RAM (202) and ROM (203), and input / output (204).

[0047] The processor (201) (or multiple processors, multi-core processor, etc.) can be selected from a range of devices currently widely used, for example, manufacturers such as: Intel™, AMD™, Apple™, Samsung Exynos ™, MediaTEK™, Qualcomm Snapdragon™, etc. Under the processor or one of the processors used in the system (200), it is also necessary to take into account the graphics processor, for example, NVIDIA GPU with a CUDA-compatible software model, or Graphcore, the type of which is also suitable for full or partial execution of the method, and can also be used to training and application of machine learning models in various information systems.

[0048] RAM (202) is a random access memory and is designed to store machine-readable instructions executable by the processor (201) to perform the necessary data logical processing operations. The RAM (202) typically contains the executable instructions of the operating system and associated software components (applications, program modules, etc.). In this case, the RAM (202) may be the available memory of the graphics card or graphics processor.

[0049] A ROM (203) is one or more persistent storage devices such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[0050] Various types of I/O interfaces (204) are used to organize the operation of device components (200) and organize the operation of external connected devices. The choice of the appropriate interfaces depends on the specific design of the computing device, which can be, but not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0051] To ensure user interaction with the device (200), various means (205) of I/O information are used, for example, a keyboard, a display (monitor), a touch display, a touchpad, a joystick, a mouse, a light pen, a stylus, a touch panel, trackball, speakers, microphone, augmented reality tools, optical sensors, tablet, light indicators, projector, camera, biometric identification tools (retina scanner, fingerprint scanner, voice recognition module), etc.

[0052] The network communication means (206) provides data transmission via an internal or external computer network, for example, an Intranet, the Internet, a LAN, and the like. As one or more means (206) can be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and / or BLE module, Wi-Fi module and others

[0053] The specific choice of elements of the device (200) for the implementation of various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[0054] Modifications and improvements to the above embodiments of the present technical solution will be clear to experts in the art. The foregoing description is provided by way of example only and is not intended to be limiting in any way. Thus, the scope of this technical solution is limited only by the scope of the attached claims.

Claims (15)

1. A method for launching a user program code on a computing node, executed by at least one computing device, comprising the steps of: - receive a request to run a user program code on a computing node in an isolated environment for processing data intended for processing by said code; - delivering the user program code to the computing node; - running on the computing node an isolated environment designed for the safe execution of computer programs; - download the user program code in an isolated environment; - determining, based on the information of said request, data for the user program code to be processed by the user program code; - loading into the sandbox data for the user program code to be processed by the user program code; - determine that the mentioned data for the user program code is loaded into the sandbox; - processing data for the user program code through the user program code in an isolated environment to obtain data processing results; extracting data processing results obtained as a result of processing data for the user program code by the user program code from the isolated environment and storing them in at least one data storage device. 2. The method according to claim 1, characterized in that the request to run the user program code contains user program code and/or data for user program code, wherein the user program code and/or data for user program code to be loaded into the sandbox are retrieved from the mentioned request. 3. The method according to claim 1, characterized in that the request to run the user program code contains a link to the user program code and/or a link to data for the user program code to be processed by the user program code, wherein the user program code and/or data for custom program code to be loaded into the sandbox are retrieved from the storage device at the referenced links contained in the said request. 4. The method according to claim 1, characterized in that in order to load data for the user program code into the isolated environment and retrieve the results of data processing, a named pipe or unix socket is formed to exchange data with the running user program code in the isolated environment or provide direct upload / download mentioned data and code to a folder inside the sandbox. 5. The method according to claim 1, characterized in that the request to run the user program code further comprises computing resource parameters, and the method further comprises determining a computing node to run the user program code based on said computing resource parameters depending on computing characteristics of the computing node and their current computing load. 6. A device for launching user program code on a computing node, comprising at least one computing device and at least one memory containing machine-readable instructions that, when executed by at least one computing device, perform the method according to any one of paragraphs. 1-5.

Images