RU150421U1 - SYSTEM OF PLACING AND REMOVING FILES OF DIVIDED SERVERS - Google Patents

Abstract

1. A system for hosting and extracting files of dedicated servers, comprising: a host computer providing coordinated access to computer resources; a data store containing: - a collection of files of dedicated servers configured to support the file system service; - a collection of files of dedicated servers located in individual areas of server memory; - cache memory of a disk storage device configured to store currently used files of dedicated servers; - database data that stores data of internal system names of files of dedicated servers; - a user control and support unit configured to collect statistics of accesses to files of dedicated servers, in which: a user control and support unit is configured to copy the file from individual memory areas to the cache memory a disk storage device based on statistics of file accesses and data of internal system file names; the control and user support unit is designed to provide Highlighted servers access to data stored in the disk cache memory, the internal data name system dedicated servers contain checksum and file attributes faylov.2 length. The system of claim 1, wherein the user control and support unit is configured to share a dedicated server file from a cache of a disk storage device if the internal file data matches the files used in the database. The system of claim 1, wherein

Description

The technical field to which the utility model relates.

The utility model relates to virtual private servers (from English - Virtual Private Server, VPS). More specifically, the utility model relates to caching (i.e., the procedure for placing and retrieving from the cache) shared virtual private server files.

State of the art

The significant commercial importance of server technology is due to the popularity and success of the Internet. Typically, an individual server application (i.e., a computer program) runs on a dedicated physical (i.e., really existing) host computer serving client requests. However, providing a separate physical host computer for each server application is costly and inefficient.

For example, Internet service providers often provide commercial hosting services, and each consumer is usually provided with a dedicated physical host computer on which the server application runs. However, often the consumer who purchases hosting services does not need the processing power of the whole host computer, and the consumer is not inclined to pay for its use. As a rule, an individual consumer needs only part of the resources of the host computer: part of the processing power, part of the memory space of his storage devices, etc.

In this regard, it is desirable to host multiple server applications on one physical computer. For this, for each server application, a container application is required - an isolated virtual private server or a dedicated server that simulates the execution of the server application on a dedicated physical host computer. Such an organization of calculations cannot be implemented by conventional server technology, because instead of executing separate discrete processes, a virtual private server must contain many processes that seem to not interact with each other. Each container process performs various elements of the functionality required by the consumer, and each process uses the files belonging to it, stored in the isolated private zone of the virtual private server (dedicated server or “container”).

Thus, another problem in organizing multiple virtual private servers on a single physical host computer is to provide each virtual private server with a separate file system. A file system is an organized collection of data (in a data warehouse) stored on one or more physical storage devices, for example, hard disk drives or RAID drives (from the English RAID - Redundant Array of Inexpensive Disks, an excess array of inexpensive disks).

The data warehouse included in the system may contain: a collection of dedicated server files supported by the file system service. In this case, a file of dedicated servers can be located in individual areas of server memory. Also, the data warehouse may include a cache memory of a disk storage device, configured to store currently used files of dedicated servers. The data warehouse contains a database that stores data of internal system file names of dedicated servers. In addition, the system contains a control and user support unit configured to collect statistics on accesses to files of dedicated servers.

As a rule, data is stored in the form of files, for example, text documents, data tables, executable programs, etc. Files are stored in a variety of ″ memory elements ″ of the storage device, often referred to as the terms ″ disk block ″ or ″ unit of disk memory ″ (cluster).

Providing a separate physical device for storing the file system of each virtual private server is expensive and inefficient. Therefore, each virtual private server (dedicated server) is allocated a private (i.e., individual) memory area for storing files used by this virtual private server. However, if on a physical host computer there are many virtual private servers identical to each other, then in their work these virtual private servers possess and use identical sets of files.

Although sharing files with multiple virtual private servers is easy, equipping containers with an efficient method for sharing different matching files is problematic. Thus, an efficient and effective method of using and accessing shared files of virtual private servers (containers) is required.

Utility Model Disclosure

This utility model relates to caching of shared files of virtual private servers (containers), which largely eliminates one or more of the drawbacks of the prior art.

According to an embodiment, the system includes a main computer that provides coordinated access to the resources of the computer that runs the kernel of the operating system, which contains the file system service. The kernel of the operating system supports many virtual private servers. At least one application is available to users of the virtual private server service. Each virtual private server has its own virtual address space (or its own set of addresses), which includes: addresses of random access memory (RAM); IP addresses (from English Internet Protocol - Internet Protocol); hard drive addresses; names using the SMB protocol (from the English Server Message Block - a set of server messages, a network protocol for remote access to files, printers, and other network resources); names over TCP (from English Transmission Control Protocol - Transmission Control Protocol); channel names, etc. Each virtual private server has its own objects and files. From the point of view of the kernel of the operating system, each of the mentioned objects and files of each of the virtual private servers has a unique identifier.

Each of the virtual private servers cannot affect the files or objects of another virtual private server (container), and also cannot access information about processes running on another virtual private server. The composition of server resources includes: virtual memory space allocated to each user; swap memory space allocated in the kernel of the operating system to support virtual private servers; swap memory space used by the kernel of the operating system to support user processes either in the form of sharing or in an exclusive form (i.e., to support user processes of either one or multiple virtual private servers); resident memory allocated in the kernel of the operating system; physical memory used by user processes; shared resources of the central processing unit (CPU); secret descriptors (or other identifiers related to the delimitation of access rights of users and virtual private servers); objects and files used by the kernel of the operating system; I / O interfaces and their level of use by specific virtual private servers; file and (or) disk space of memory; restrictions imposed on individual user resources.

Each virtual private server contains: a set of processes, each of which contains at least one thread serving the corresponding users; a collection of objects and files associated with a specified set of threads. According to an embodiment, if the virtual private server uses files for read and write operations, then for each of the files, the main control operating system calculates hash values (checksums) using hash functions. Such hash values (checksums) are used as attributes (hash attributes) of the file. The specified values of the hash attributes of the files (checksums) are generated when the files are created (i.e., when the file close operation is performed). Hash attributes are part of the extended file attributes that are stored with the files.

When the virtual private server process accesses the file, it checks its hash attribute (checksum) against the database records. And if an identical file is detected with matching values of the checksum and file length in the index (from the English inode, i.e. index node, is a data structure that stores service information about files in the file system, including storing file attributes), this file is cached on the disk storage device. The hash attribute (checksum) is checked by a special utility of the operating system kernel. If the checksums of the files match, then the file is not copied. Instead, a redirect link is used to open the file stored in the container’s private directory. A file in the process space of a virtual private server can be reassigned (i.e. redirected) to the cache memory of a disk storage device using a special utility of the operating system kernel. Then, instead of accessing the private space of the virtual private server, the processes of the virtual private server access the file in the cache memory of the disk storage device.

Server software can be an addition to any of the operating systems: Microsoft Windows NT Server - Terminal Server Edition; Microsoft Windows 2000 Server - Terminal Server; Microsoft Windows Server 2003 - Terminal Services, or any server based on the Microsoft Windows platform. The operating system contains a set of threads designed to fulfill user requests. Virtual private servers are presented to the user essentially as dedicated servers and in general provide the functionality of a dedicated server or remote computer, including all administrative operations.

Additional features and advantages of the present utility model will be explained in the following description of the implementation of the utility model. Partially, they will become clear from the description, or they will be revealed by the practice of using the utility model. The advantages of the utility model will be realized and obtained in the structure disclosed in detail in the attached description and formula of the utility model, as well as in the accompanying drawings.

It should be borne in mind that both the preceding general description and the following detailed description are given as an example and for purposes of explanation and are intended to better explain the essence of the claimed utility model.

Brief Description of the Drawings

The drawings show:

in FIG. 1 is a cache diagram of a disk storage device according to an embodiment;

in FIG. 2 is a graphical representation of a procedure for generating a hash attribute (checksum), which is an attribute of a file in a container, according to an embodiment;

in FIG. 3 is a graphical representation of a file cache generation procedure according to an embodiment;

in FIG. 4 is a graphical representation of a method for caching computer files according to an embodiment;

in FIG. 5 is an example of a host computer system diagram that can be used in the present utility model.

Utility Model Implementation

The following is a detailed description of embodiments of the present utility model, which are illustrated by the accompanying drawings.

This utility model is directed to a system, method and computer software product for caching shared files of virtual private servers (containers). A virtual private server (container) is a limited set (set) of processes, system resources, users, user groups, objects, data structures. Each virtual private server has a unique identifier.

According to an embodiment, the virtual private server provides its users with services that are substantially functionally equivalent to those provided by the dedicated remote access server. In fact, from the point of view of the administrator, a virtual private server in this utility model is the same as a dedicated computer installed in a computer center.

In fact, from the user's point of view, a virtual private server functions in the same way as a server with remote access, and provides the same services, for example, through a dedicated IP address. Keep in mind that some processes running as part of the kernel of the operating system can provide services to many virtual private servers.

In a computer system with one operating system, each virtual private server virtualizes the specified operating system and uses the services of the operating system. Each virtual private server contains many processes, each of which contains at least one thread serving the corresponding users, as well as many objects and files associated with the specified stream (multiple threads). Virtual private server processes use a number of the same files. U.S. Patent Application 11 / 972,556, filed simultaneously with this application and incorporated by reference into this application, proposes a system for using links to shared files of virtual private servers located in the physical storage device of the host computer.

When the process of the virtual private server needs to access the file, the process finds a link to the file in the private zone of the virtual private server and accesses the file in the physical storage device. However, this procedure is not always operational after restoring a virtual private server from a backup. In such cases, usually a certain number of links to virtual private server files do not work or indicate an incorrect position, and therefore it is impossible to start the virtual private server.

According to an embodiment, when the virtual private server processes read and write files, the main control operating system calculates hash values (checksums) for each of the files. The operating system kernel utility intercepts file read and write operations and, when an unshared file (that is, a file without shared access) is detected, performs the required actions. Keep in mind that when creating files, a file cache may be generated. Hash values (checksums) are used as file attributes. The value of the hash attribute (checksum) of the container file is generated when the file is created (i.e., when the file close operation is performed). Hash attributes are stored in the database and arranged according to file size attribute values. Please note that disordered records may also be stored in the database. For example, hash attributes can be grouped.

The first group of hash attribute bits is used as the name of the folder (group) and corresponds to the ordered or unordered hash attributes stored in this folder. When a new hash value arrives, it is sent to the appropriate folder, this is a kind of ordering algorithm. For example, a file with a hash value of ″ ABCD ″ can be stored as a ″ ABCD ″ file, or as a ″ D ″ file in the ″ C ″ folder in the ″ B ″ folder located in the ″ A ″ root folder (which however, it may be a lower level folder for some other folder). The letters ″ A-D ″ can be numbers in hexadecimal code. The hash value can also be used as the address of the storage device in which the corresponding data is stored, for example, as the path to the shared file (i.e., a shared file). The proposed method does not necessarily require additional storage of any kind.

When the process of the virtual private server (container) accesses the file, the hash attribute (checksum) is compared with the contents of the database. And if an identical file is detected with matching values in the inode (checksum and file length), then the file is placed in the cache memory of the disk storage device in RAM, which can be located on the local server or on any server in the cluster or in the cloud computing service cluster. Note that for virtual private servers, it is preferable not to use client-server requests. It is preferable to use a single local computing node and / or a cluster of computing nodes. These operations can be performed using the control unit and user support, for example, made as part of a computer memory containing instructions of the control unit and user support that control the operation of the processor. The control and user support unit may be executed in user space. A file in the container’s private or private directory is assigned a redirect link to a copy of it in the cache memory of the disk storage device. The virtual private server can access the file from the cache of the disk storage device instead of the file from the private zone of the virtual private server.

In other words, the kernel or a separate kernel service of the operating system determines access to the disk on which the file inode is stored. If identical inodes are detected, then the inode value (checksum and file length) are transferred to the control and user support unit. The control and user support unit is a user-space application that checks all files in the container and collects statistics of file accesses. The control and user support unit can be used on the local server or on any server in the cluster or on the cloud computing service cluster. For example, if the control and user support unit detects a certain number of files in the private directory of the container that the container processes access regularly, then the user control and support unit processes this information and performs the corresponding actions.

The kernel of the operating system checks whether the files have identical inodes in the database and, if found, assigns redirect links to the location of the same files in the cache memory of the disk storage device. The control and user support unit also frees the cache memory of the disk storage device from obsolete files that have not been accessed for a long time.

If the file is open for writing, then the hash attribute (checksum) of the file is discarded, and when the write operation to the file is completed, a new attribute value is generated. These actions are performed by the updated I / O subsystem of the kernel of the operating system. According to an embodiment, a hash attribute (checksum) is generated when a file close operation is performed only for files located in a directory with the attribute ″ checksum = auto ″ (from English, ″ checksum = automatically ″). For example, the ″ / bin ″ directory has the ″ checksum = auto ″ attribute.

If the file ″ / bin / xxx ″ is written in the specified directory, then the hash attribute (checksum) ″ XXX ″ is calculated for this file and written to the file attribute ″ checksum = XXX ″. It should be noted that, for example, the ″ / home ″ directory does not have a checksum attribute, and the user files in this directory are not hashed (hash attributes are not calculated), since the probability of matching user files is extremely small compared to the probability of matching container files.

According to an embodiment, well-known hashing algorithms can be used, for example, SHA20, Gold Fish, ECB, SHA1, SHA2, SHA256, SHA512, etc. If the file is stored at the level of data blocks (clusters), then the value of the hash attribute may change when changing the blocks of data that make up the file. In this case, the value of the hash attribute is recalculated taking into account the corresponding data block. These operations should be performed quickly, and therefore it is recommended to use a simple (short) hash function. An alternative is to calculate and store hash attribute values for each data block.

According to an embodiment, the number of files (inodes) is not limited, however, the amount of disk memory available for files is limited in advance. Files that have not been accessed for a long time are deleted from the cache and discarded. Checking file accesses and deleting them can be performed by another user control and support unit, which scans the virtual storage device independently of users and deletes files. The time interval after which the file is considered obsolete is set by the system administrator. An alternative is to archive files in a compressed form.

It should be noted that the main advantage of the embodiment of the utility model is not so much in saving the amount of available memory of the disk storage device as in reducing the number of calls to devices of this type. That is, the advantages are the great performance and speed of the system. For example, if 100 identical virtual servers (containers) randomly access 10 pages, then the total number of hits is reduced by (100-1) × 10 compared to a conventional system. Keep in mind that 990 random accesses to a conventional traditional hard drive takes about 10 seconds.

In other words, instead of reading the same file five times from five different areas of disk memory, the file is read once from the cache memory of the disk storage device. The kernel of the operating system detects the presence of a common file for given inodes (i.e. sets of file metadata, such as file extension, checksum, file length) and quickly finds the file. The control unit and user space user support finds files common to containers based on file access statistics. The control and user support unit operates in the user space, however, it is not under the control of the users of the virtual private server. The proposed control and user support units can be controlled by the administrator's computing node (virtual private server with the ″ 0 ″ index), controlled by the system administrator. If the file is only renamed inside the container without changing the hash attribute a, then nothing changes fundamentally in the system, and the file can be read using the old value of the hash attribute. If changes in the contents of the file and metadata affect the value of the hash attribute, a new instance of the file can be created in the container’s memory space, after which the file can be shared in accordance with the above procedure.

According to an embodiment, the system periodically automatically scans containers in search of identical files using another control and user support unit. Files are copied to the cache memory and stored there for a set time, after which the files are deleted from the cache or transferred to the archive. This increases the overall system performance and RAM utilization, provided that the system operates with a large amount of redundant data (i.e., with language settings, geographic location-specific data, graphic primitives, etc.).

Keep in mind that if the file changes, then the changed file instance remains in the container’s private directory. When placing files in the high-speed cache of a disk storage device, a list of processes of a virtual private server (container) is used. The file is always stored in the container’s private directory. However, for quick file open operations, the file version provided by the ″ PFcache ″ mechanism can be used. According to an embodiment, the virtual private server (container) file remains in the container’s private directory even after it is copied to the cache memory of the disk storage device. If the container is migrated, or if it is necessary to restore the container from the backup copy, when the specified file from the cache memory of the disk storage device is unavailable, the file stored in the private directory is used.

When the container process tries to access the file from the container’s private directory, it receives a redirect to the file located in the cache memory of the disk storage device. However, if the redirection does not work (for example, if the container is migrated to another host server), then the process accesses the file in the private directory along the path stored in the file system tree (index tree). If access to the file specified in the redirection is not provided after several attempts, then an alternative is to discard the value of the checksum attribute. Then, upon completion of the changes to the file, a new value of the checksum attribute is generated.

According to an embodiment, template files, if any, are always used. Pattern cache checksums are calculated in advance. The checksum is calculated every time you modify or create a new file. Each file has an index, which includes an attribute indicating whether the checksum was calculated and whether it is stored with the file. If there is a checksum, then the container process is redirected to the file with the same index in the shared cache of the disk storage device. According to an embodiment, the checksums are stored automatically.

In FIG. 1 shows a cache diagram of a disk storage device according to an embodiment of the utility model. A file is created, the value of the hash attribute (checksum) is calculated, this value will be placed in one of the options for the software cache "centos-6.tgz" file template (the number 6 can indicate the version number of the file, "tgz" is the file extension meaning that it is a container of compressed data, not a virtual container). The file checksum attribute is assigned ("x-attr: csum = HEX16"). When creating the container, the specified file is copied along with the pre-calculated value "x-attr" (checksum value) from the template cache to the file (virtual memory, disk image) "ploop" (from English "p" + "loop", in OpenVZ virtualization technology - a file containing the container file system) 120. The “ploop” file 120 is a combination of a virtual block device and a corresponding driver that receives data from a file, and not from a physical storage device. File "ploop" 120 is the result of the transfer of ring storage technology (from English - loop device, a virtual storage device containing a file system) from the Linux operating system, which supports: modular architecture; various file formats (″ flat ″, QCOW2, etc.); network storage and the corresponding NFS network file system protocol; creating snapshots of the contents of the storage device and quick initialization through stack images and a tracker for combined recording for accelerated migration on the fly (live migration). The ″ ploop ″ 120 file is a virtual block ring disk device that, in addition to the capabilities of a ″ ploop ″ device, has an expanded set of capabilities, in particular, it provides dynamic volume change, instant creation of content copies, backup, etc. The basic idea is to place the container file system in a file. When creating the container, the file is copied to the 130 ″ / bin / bash / ″ directory. The ″ ploop ″ 120 virtual storage device checks files in the private directory of the ″ / vz / private / 100 / fs ″ container. Then, if the file has the hash attribute (checksum) ″ x-attr: csum = HEX16 ″, then the file is cached in the local template 140 ″ / opt / vzcached-file ″ together with the file checksum attribute ″ x-attr : csum = HEX16 ″. Then the file is saved (150) locally at the address ″ / opt / cached-file ″ with its own attribute ″ x-attr: csum = HEX16 ″ for use by the container in cases when the file stored in the cache is inaccessible (i.e. lost or truncated).

The file from the cache of 140 templates ″ / opt / vzcached-file ″ is moved to the cache 160 of the disk storage ″ / vz / pfcache / HEX1-2 / HEX3-16 ″. An alternative is that the file from the local storage device 130 of the ″ / bin / bash / ″ container does not fit into the cache 140, but is put into the cache storage 160 of the disk storage device if necessary (on request) or delayed. Delayed caching means that the control unit and user support collects statistics of accesses to container files, and the kernel of the operating system after some time places the files in the cache memory based on statistics processed by the control and user support unit.

Keep in mind that when you install the application template, the original Red Hat ″ RPM ″ package manager (package manager or package management system) installs the kernel files and assistants and calculates their checksums in deferred mode. The cache 160 then processes the file checksums, modifies the files, and sets the xattrs attributes of the files. In addition, the cache 160 detects copies (clones) of files having the same index value. Clones can be detected by hashing the file and then comparing the actual contents of the file. Since the probability of the initial bits of different files coinciding with the same values of the hash attributes is very small, only the predetermined parts of the files — file sizes — can be compared to decide on the identity of the files. Please note that the hash attribute values in this case will be identical.

2 illustrates a procedure for generating a hash attribute (checksum) of a container file according to an embodiment of the utility model. According to an embodiment, when creating the container, the file, along with the previously calculated value of the xattr checksum attribute, is extracted from the template cache and copied to the local storage 130 "/ bin / bash /" of the container.

When you install the appropriate application template, the original Red Hat package manager or another package manager or installer installs the files and, optionally, the templates. The kernel takes part in this procedure and calculates checksums of "x-attr: csum = HEX16" files in cache 210 "/ opt / plesk" in deferred mode. Cache 160 (see figure 1) generates checksums for the modified files and sets the values of their attributes "x-attr". Then, the attribute values "x-attr: csum = HEX16" of the files in the private directory "/ bin / Is" of the container are updated.

3 illustrates a cache filling procedure according to an embodiment of the utility model. Application templates are set by the Red Hat package manager (the kernel of the operating system contributes to this procedure), the values of the "x-attr" attributes are calculated on the fly. In addition, the cache 160 recounts the checksum values of those files that were changed from the state at the time of the previous calculation of the checksum values.

If the index value of a file (checksum and file length) matches the index value of another file being accessed, the kernel of the operating system may request file caching. For example, the attribute values of files located in the private directories of containers 310 and 320 are the same (checksums and file lengths coincide). The kernel 330 of the operating system detects the fact of using the same file (the checksum and length are the same) and notifies the control unit and the user support cache. The control and user support unit 340 in deferred mode decides whether and when to cache the file common to containers. ″ Ploop ″ 350 virtual storage device is used to limit the cache size by using a file image. ″ Ploop ″ virtual storage device is a type of virtual disk storage device or disk image with certain characteristics and properties described above, provided with appropriate driver programs. Please note that the ″ ploop ″ 350 virtual storage device can be used as a separate section of a disk storage device.

According to an embodiment, the global cache ″ / vz / pfcache ″ is used to store redirect links for both templates and cache-located files. Files are cached in deferred mode: initially, the ″ pfcache ″ cache of the disk storage device is empty, it is filled based on statistics on access to the container files collected by the control unit and user support 340 cache. The cache memory of the disk storage device is filled dynamically, it contains only the files common to the containers that are being accessed. Unused files are deleted from the cache after a specified time.

According to an embodiment, if the file is modified, the file checksum attribute is cleared by the kernel of the operating system. Files stored in the cache, which container processes have not accessed for a long time, are ″ cut off ″ by the control and user support unit. This can be done when the cache reaches the set limit.

When the cache reaches its limit value, ″ old ″ files are moved to a subdirectory (a directory with a lower hierarchy) ″ to-be-removed ″ (from English - ″ to remove ″). In turn, all files stored in this subdirectory should be deleted by the beginning of this operation. If the kernel of the operating system reports that some of the moved files are still in use, then these files are moved back by the user control and support unit. A preventive delete operation in the background is performed by a special computing thread.

Outdated files are identified based on file access time stamps. If a file is accessed from the cache using a redirect link based on a checksum attribute, the kernel of the operating system updates the timestamp for accessing the file (for example, if the current time differs by 12 hours from the value of the timestamp for accessing the file).

Keep in mind that templates can be placed in the cache in the form of "simfs" (a representation in the form of memory of a disk storage device or virtual memory used in the OpenVZ virtualization technology project), but with attributes containing file checksums. Templates can be deleted simply by deleting the "template.tgz" file placed in the cache (for example, when updating).

A virtual private server (container) is created by unpacking a cached container template containing files and attributes. A container can be deleted by deleting its private directory. Keep in mind that files stored in the cache are deleted later, depending on the time of the last access to the file.

According to an embodiment, virtual private servers (containers) can be restored, and can also migrate. A copy of the container can easily be created. The advantage is that no additional steps are required, since the same files located in the cache memory (if any) are automatically used. Otherwise, the required files are created on demand (i.e., in deferred mode) if another container uses the same files.

Note that copying the cache or templates is not required. The cache memory of the disk storage device is recreated automatically (in deferred mode and on demand) depending on file accesses. According to an embodiment, the virtual private server (container) may be updated compared to the current state. The container is mounted, the "ploop" image is mounted, then the files are copied. The "ploop" image looks similar to other images of disk drives, for example, ISO images (from the English International Organization for Standardization), etc. In other words, for a file or inode, the file system looks like it is real one. An alternative is to store files (templates) in the storage device along with the calculated checksums.

According to an embodiment, the application template is installed as follows:

user, debugging data, settings, etc. Packages are installed ″ as is ″, after which the data is sorted;

- the kernel of the operating system facilitates the installation and generates ″ on the fly ″ checksums of files created by the Red Hat package manager;

- to generate hash attributes (checksums), the enhanced SHA1 algorithm is used (from the English Secure Hash Algorithm 1 - type 1 cryptographic hashing algorithm).

Keep in mind that the computation speed of the SHA1 algorithm is quite high. Using modern software, the speed is about 300 MB / s. This value is comparable to the data rate of a high-speed disk storage device. The SHA1 algorithm generates long values (20 bytes or 40 hexadecimal words), in connection with which they must be encoded in the 27-character format of the checksum attribute using the alphabet [0, 1, ..., 9, a, b, ..., z , A, B, ..., Ζ]. According to an embodiment, the attribute values calculated by the ″ SHA1 + file length ″ rule are compared to identify matching inodes.

According to an embodiment, the cache management and user support unit is executed using a separate binocounter (user binocounter) or another resource controller. User binocounters or binocounters are a set of restrictions and can be used to prevent system overload from operating control units and user support. Resource binckunters allow you to take into account and control the consumption by a group of processes of the core resources of the operating system. A complete set of user binunchers allows you to control:

- memory space of the kernel of the operating system. All objects of the kernel of the operating system, placed at the request of the user, should be taken into account and limited to prevent a Denial of Service failure (for example, English DoS - Denial of Service), for example, page tables, data structures of the type "task struct" (data structure in the kernel of the operating system, containing the necessary information to control a specific process), "vmas", etc .;

- pages of virtual memory.

The user binometer allows you to limit the amount of memory used by the container, and introduces a two-level restriction of the "low memory" type, which monitors the container's consumption of storage resources. Pages shared by containers are loaded correctly as parts (custom procedure). Keep in mind that usually an error such as "out of memory" occurs when the operating system cannot increase the amount of virtual memory, since all available memory space of storage devices is full. Various operating systems, such as Linux, will try to overcome this lack of memory by stopping low-priority processes, this mechanism is known as "OOM Killer" (from the English Out of Memory Killer - eliminating an error like "out of memory"). In some cases, memory leaks (that is, an uncontrolled decrease in the amount of free computer memory associated with errors in the release of unnecessary sections of memory), the OOM Killer mechanism is vulnerable. A memory leak occurs when a computer program incorrectly performs placement operations in memory space. In object-oriented programming, a memory leak can occur if an object stored in memory is inaccessible to the program being executed.

A complete set of user binocounters also allows you to control:

- network buffer memory. This memory includes TCP / IP protocol transmit and receive buffers, dgram snd, unix, netlink, and others buffers;

- minor resources, taken into account by value:

- tasks and files; mechanisms and objects "flock", "ptys", "siginfo", "pinned dcache";

- sockets; mechanisms and objects "mem", "iptentry" (for containers with virtualized access to the local area network).

All or some of the listed mechanisms and objects can be used to limit the IOPS (from English Input / Output Operations Per Second - the number of input / output operations per second) or throughput or in relation to the image ″ ploop ″ (to limit the amount of memory of the disk storage device loaded into the cache of a disk storage device).

″ Virtuozzo ™ ″ resource controllers, in addition to managing the use of memory space and system resources, allow you to:

- control CPU utilization; detailed information about this resource is given in the section ″ Control of CPU time allocation ″;

- control disk quotas (see the user guide for ″ Parallels Virtuozzo Containers ″ software from ″ Parallels ″, ″ Resource Management ″);

- create and keep records of network traffic (see the user guide for ″ Parallels Virtuozzo Containers ″ software from ″ Parallels ″, section ″ Resource Management ″).

In this case, the amount of memory is limited. If there is no free memory space for new records, then the old data is deleted. Keep in mind that in order to minimize access time to the file in the cache, the file inode containing the redirection information must be limited in size so as to fit in one data block. Using a hash algorithm of type SHA1 allows you to create indexes of the required length.

If the control and user support unit is part of the resource manager, then it can access all container files and can calculate hash attribute values for these files. After that, the control unit and user support can initiate the movement and reassignment of files to which should be shared.

In FIG. 4 is a diagram of an algorithm implementing the method according to an embodiment of the present utility model. At step 410, a container is created. At 420, the container files are stored in the container’s private directory. At step 430, the kernel of the operating system identifies the files that the container processes are accessing. At step 440, hash attributes (checksums) are generated for files used by the container processes. In step 450, the file attributes are assigned checksum values and file lengths.

At step 460, the kernel of the operating system, by comparing the attribute values (checksums and lengths) of the files with the attribute values common to the containers stored in the database, identifies identical files used by other containers. At step 470, the condition is checked: if a fact of file identity is detected, then at step 480, the file from the container’s private directory is copied to the cache memory of the disk storage device; otherwise, it returns to step 460, and the kernel of the operating system checks another file. Finally, at step 490, a file from the container’s private directory is assigned a redirect link to its copy in the cache memory of the disk storage device. Thus, the next time the container process accesses the file, the process is redirected to a copy of the file in the cache memory of the disk storage device.

In FIG. 5 shows an example of a block diagram of a host computer that can be used in the present utility model. The example system shown in FIG. 5 comprises a general-purpose computing device in the form of a computer or server 20 or a similar calculator comprising a computing device 21, a system storage device 22 and a system bus 23 that integrates various system components, including system storage device 22 and computing device 21. The system bus 23 can be any type of bus, including a storage bus or a storage controller, a peripheral bus, a local bus, and can be built according to any arch tecture tires. The composition of the system memory includes read only memory (ROM) 24 and RAM 25.

The ROM 24 stores the basic input / output system 26, which contains basic procedures that facilitate the transfer of information between the components of the computer 20, in particular during boot.

The computer 20 may also comprise: a hard disk drive 27 for reading and writing information to the hard disks (in Fig. 5, the hard disks themselves are not conventionally shown); a drive 28 of magnetic disks for reading and writing information to removable magnetic disks 29; optical drive 30 for reading and writing information to removable optical disks 31, such as CD-ROM (from English Compact disk read-only memory - CD-ROM - read-only memory), DVD-ROM (from English Digital video disc read -only memory - digital video disc - read-only memory) or another type of optical computer-readable medium.

The hard disk drive 27, the magnetic disk drive 28 and the optical disk drive 30 are connected to the system bus 23 using the hard disk drive interface 32, the magnetic disk drive interface 33 and the optical disk drive interface 34, respectively. Drives and associated machine-readable media provide the computer 20 with non-volatile storage of machine-readable instructions, data structures, software modules, and other data.

Despite the fact that the described embodiment uses a hard disk, a magnetic disk 29, and a removable optical disk 31, it will be understood by those skilled in the art that other types of computer-readable media, in particular magnetic tapes, can be used to store data accessible to a computer, flash memory cards, digital video disks, cartridges for Bernoulli drives, drives on RAM and ROM, etc.

A number of program modules may be stored on a hard disk, magnetic disk 29, optical disk 31, ROM 24 or RAM 25, including the operating system 35. Computer 20 includes: a file system 36 connected to or included in the operating system 35; one or more application programs 37; other software modules 38; data from 39 programs. The user can enter commands and information into the computer 20 through input devices, such as a keyboard 40 and pointing device 42. Other input devices (not shown in FIG. 5) may include: a microphone; joystick; game control panel; satellite antenna; scanner; other similar devices.

These and other input devices are usually connected to the processor device 21 via the serial port interface 46, which is connected to the system bus. However, these input devices can also be connected via other interfaces, such as a parallel port, a game port, and a USB port (from the English Universal serial bus - universal serial bus). Also, a monitor 47 or another type of device for displaying video information is connected to the system bus 23 through an interface, such as a video adapter 48. In addition to the monitor 47, personal computers typically include other peripheral output devices (not shown in FIG. 5), such as speakers and printers.

Computer 20 may operate in a networked environment using logical connections to one or more remote computers 49. The remote computer (s) 49 may be: another computer; server; router network personal computer; a network device that directly interacts with other devices in a networked environment; another node on the same network. In addition, the remote computer 49 usually contains many or all of the functional nodes described above with respect to the computer 20, although only the storage device 50 is shown. The logical connections include a local area network (LAN) 51 and a global network 52. The described network environment is common in offices, enterprise computing networks, intranets and the Internet.

When used in a networked LAN environment, computer 20 is connected to the LAN 51 via a network interface (network adapter) 53. When used in a networked environment of a global computer network, computer 20 typically includes a modem 54 or other means of establishing a connection through a global computer network 52, such as the Internet .

The modem 54, which may be internal or external, connects to the system bus 23 via the serial port interface 46. In a networked environment, program modules depicted relative to computer 20 or parts thereof may be stored in a remote storage device. Note that the network connections shown serve as an example, and other means of establishing a communication connection between computers can be used.

Thus, in view of the above description of preferred embodiments, those skilled in the art will appreciate that the described method and system have certain advantages. In particular, it will be understood by those skilled in the art that the proposed system and method provides efficient caching and file sharing by containers.

It should also be understood that various modifications, adaptations, and alternative embodiments can be made within the scope of this utility model. A utility model is defined by the following utility model formula.

Claims (6)

1. A system for hosting and extracting dedicated server files, comprising: The main computer, providing coordinated access to computer resources; A data warehouse containing: - a collection of files of dedicated servers, configured to support the file system service; - a collection of files of dedicated servers located in individual areas of server memory; - cache memory of a disk storage device, configured to store the currently used files of dedicated servers; - a database storing data of internal system file names of dedicated servers; - a control and user support unit configured to collect statistics of accesses to files of dedicated servers, wherein: the control and user support unit is configured to copy a file from individual memory areas to the cache memory of a disk storage device based on statistics of file accesses and data of internal system file names; the control and user support unit is made providing dedicated servers access to data stored in the cache memory of the disk storage device, while the internal system names of the dedicated servers contain attributes of the checksum files and file lengths. 2. The system of claim 1, wherein the user control and support unit is configured to share the dedicated server file from the cache memory of the disk storage device if the internal file data matches the files used in the database. 3. The system of claim 1, wherein the file system service is configured to assign a redirect link from the individual memory areas of the dedicated servers to the dedicated file servers copied to the cache memory of the disk storage device. 4. The system of claim 1, wherein the file system service is configured to calculate the values of the checksum attributes of the container files when creating these files. 5. The system of claim 1, wherein the control and user support unit is configured to copy files of dedicated servers to the cache memory of a disk storage device on demand. 6. The system of claim 1, wherein the user control and support unit is configured to copy files of dedicated servers to the cache memory of a disk storage device in deferred mode, after files of dedicated servers that are often accessed and which are not have cache redirects.

Images