CN112988062A

CN112988062A - Metadata reading limiting method and device, electronic equipment and medium

Info

Publication number: CN112988062A
Application number: CN202110123455.2A
Authority: CN
Inventors: 姚任南; 方睿
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-01-28
Filing date: 2021-01-28
Publication date: 2021-06-18
Anticipated expiration: 2041-01-28
Also published as: CN112988062B

Abstract

The embodiment of the application discloses a metadata reading limiting method, a metadata reading limiting device, electronic equipment and a medium, which can be applied to the field of cloud technology and can also be used for reading mass data in the field of artificial intelligence during model training. The method comprises the following steps: receiving an input data operation command, the data operation command, when executed, being associated with a metadata read operation; acquiring path information of a target path corresponding to the data operation command; and determining whether to limit the metadata reading operation associated with the data operation command or not according to the path information. By adopting the embodiment of the application, the overload of the metadata server and the crash of the client are prevented. The metadata can be stored through the distributed cloud storage system, and the metadata can be read from the distributed cloud storage system when being read. In addition, the method and the device can be applied to a cloud computing platform, and various operation commands are executed through the container deployed by the cloud computing platform.

Description

Metadata reading limiting method and device, electronic equipment and medium

Technical Field

The present application relates to the field of cloud technologies, and in particular, to a metadata reading restriction method and apparatus, an electronic device, and a medium.

Background

With the development of storage technology, distributed storage systems are widely used, which can store a large amount of data and metadata, etc., and can read data or metadata based on various data operation commands. When some data operation commands are executed, a large amount of metadata may be read in a short time, which easily results in overloading a metadata server and crashing a client. The inventor finds in the research process that the metadata server can be prevented from being overloaded by increasing the number of server nodes and improving the performance of metadata cache at present, but the method increases the cost and cannot prevent the client from collapsing when the client requests to read mass files; or, the file merging can be realized by compressing and packaging the massive small files to reduce the storage quantity of the metadata and further reduce the access of the client to the metadata, but the method is long in time consumption for packaging the massive files, and decompression operation is needed for reading the files, so that the calculation amount is increased. Therefore, how to better prevent the metadata server overload and the client crash becomes a problem to be solved urgently.

Disclosure of Invention

The embodiment of the application provides a metadata reading limiting method, a metadata reading limiting device, electronic equipment and a medium, which are beneficial to preventing the metadata server from being overloaded and a client from being crashed.

In one aspect, an embodiment of the present application provides a metadata reading restriction method, where the method includes:

receiving an input data operation command, the data operation command, when executed, being associated with a metadata read operation;

acquiring path information of a target path corresponding to the data operation command;

and determining whether to limit the metadata reading operation associated with the data operation command or not according to the path information.

In another aspect, an embodiment of the present application provides a metadata reading restriction apparatus, including:

the acquisition module is used for receiving an input data operation command, and the data operation command is associated with metadata reading operation when being executed;

the acquisition module is further configured to acquire path information of a target path corresponding to the data operation command;

and the processing module is used for determining whether to limit the metadata reading operation associated with the data operation command according to the path information.

In another aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor and a storage device, where the processor is connected to the storage device, where the storage device is configured to store computer program instructions, and the processor is configured to perform the following steps:

In yet another aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer program instructions are stored, and when executed by a processor, the computer program instructions are configured to perform the following steps:

According to the embodiment of the application, when the data operation command associated with the metadata reading operation is received, the path information of the target path corresponding to the data operation command can be obtained, and whether the limitation processing is carried out on the metadata reading operation or not is determined according to the path information, so that the limitation on the reading operation of partial metadata can be realized, the metadata server overload and the client crash can be prevented, extra cost and calculation amount are not required to be increased, and the user experience is enhanced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1a is a schematic structural diagram of a computing platform according to an embodiment of the present invention;

FIG. 1b is a schematic structural diagram of a distributed storage system according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a metadata read restriction method according to an embodiment of the present invention;

FIG. 3a is a flow chart of another metadata reading restriction method according to an embodiment of the present invention;

FIG. 3b is a flowchart illustrating a metadata read restriction method according to another embodiment of the present invention;

FIG. 4a is a flowchart illustrating a metadata read restriction method according to another embodiment of the present invention;

FIG. 4b is a flowchart illustrating a metadata read restriction method according to another embodiment of the present invention;

FIG. 5a is a flowchart illustrating a metadata read restriction method according to another embodiment of the present invention;

FIG. 5b is a flowchart illustrating a metadata read restriction method according to another embodiment of the present invention;

FIG. 6 is a flowchart illustrating a metadata read restriction method according to another embodiment of the present invention;

fig. 7 is a schematic structural diagram of a metadata reading restriction apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

The application provides a metadata reading limitation scheme which can determine whether limitation processing is carried out on metadata reading operation associated with a data operation command when the data operation command is received. Limiting at least a portion of metadata read operations may thereby be implemented to help prevent metadata server overload and client crashes.

It can be understood that the technical solution of the present application can be applied to the field of cloud technology, for example, cloud computing and/or cloud storage technology can be involved, and can be specifically applied to an electronic device, which can be a terminal or a server. In addition, the metadata referred to herein may be stored in one or more servers, such as by a distributed storage system.

Optionally, in the present application, the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, and a big data and artificial intelligence platform. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein. That is, the present application may be applied to a server cluster or a distributed system configured by a plurality of physical servers. For example, the server cluster or distributed system formed by the plurality of physical servers may be deployed in a computing platform (e.g., a cloud computing platform), such as a machine learning computing platform, an artificial intelligence computing platform, and so on, which are not listed here.

The Cloud technology (Cloud technology) is a hosting technology for unifying series resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied in the cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.

Cloud computing (cloud computing) is a computing model that distributes computing tasks over a pool of resources formed by a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand. As a basic capability provider of cloud computing, a cloud computing resource pool (called as a cloud Platform in general, an Infrastructure as a Service) Platform is established, and multiple types of virtual resources are deployed in the resource pool for selective use by external clients, the cloud computing resource pool mainly includes a computing device (including an operating system, for a virtualized machine), a storage device, and a network device, and is divided according to logical functions, a PaaS (Platform as a Service) layer may be deployed on an IaaS (Infrastructure as a Service) layer, a SaaS (Software as a Service) layer may be deployed on the PaaS layer, or the SaaS may be directly deployed on the IaaS layer, the PaaS may be a Platform running on Software, such as a web database, a container, and the like, as business Software of various websites, a web portal, and the like, SaaS and PaaS are upper layers relative to IaaS. For example, the electronic devices described above may be computing devices,

in a possible implementation manner, please refer to fig. 1a, where fig. 1a is a schematic diagram of a cloud computing platform provided in an embodiment of the present application. As shown in fig. 1a, the cloud computing platform includes a distributed storage system, which may include a plurality of data servers and a plurality of metadata servers, and one or more electronic devices. Wherein the plurality of data servers are operable to store data in a distributed manner and the plurality of metadata servers are operable to store metadata associated with the data. The one or more electronic devices may be deployed with a user container and a distributed storage system client, and after the container is started, the container may execute various operation commands and may read metadata from the distributed storage system.

Optionally, the distributed storage system according to the embodiment of the present application may be a distributed cloud storage system, where cloud storage (cloud storage) is a new concept that is extended and developed on a cloud computing concept, and a distributed cloud storage system (hereinafter, referred to as a storage system) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network through application software or an application interface to cooperate through functions such as a cluster application, a grid technology, and a distributed storage file system, and provides a data storage function and a service access function to the outside. At present, a storage method of a storage system is as follows: logical volumes are created, and when created, each logical volume is allocated physical storage space, which may be the disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as data Identification (ID), the file system writes each object into a physical storage space of the logical volume, and the file system records storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

For example, the distributed storage system may be a Ceph distributed storage system, the Metadata Server may be a Ceph Metadata Server (MDS), and the distributed storage system client may be a Ceph fs client. The Ceph distributed storage system is a widely used distributed storage system and is specially designed for object storage and cloud infrastructure; MDS is a module for storing file metadata in a Ceph cluster; the CephFS is a POSIX-compatible file system, a Ceph storage cluster is used for storing data, and Ceph files (data) can be directly accessed through the CephFS like a local hard disk; the CephFS client may be configured to interact with the MDS to obtain metadata information of the Ceph file, for example, when the CephFS client accesses a file in the Ceph, the CephFS client requests the MDS to obtain the metadata information of the file.

Taking the application to a machine learning computing platform as an example, the machine learning computing platform includes a plurality of data servers and a plurality of metadata servers, and an electronic device including or connected to a distributed storage system client. Reading of a large amount of data may be involved in model training through the machine learning computing platform, for example, a hundred million level of files are required for model training in a computer vision direction. Optionally, a multi-machine multi-card distributed training mode may be adopted during training, the multi-machine multi-card distributed training may be performed by using a plurality of Graphics Processing Units (GPUs) of a plurality of electronic devices, and the trained files may be stored in a distributed storage system such as a cloud Ceph. During model training, a container (or called a user container) is created in each electronic device, the container is a lightweight and flexible virtualization processing mode, after the container is started, a data operation command can be input into the container, and a client of the distributed storage system, such as a CephFS client, can read metadata from a metadata server, such as an MDS, according to the data operation command. If the data operation command needs to perform metadata reading operation on a large number of files during execution, it may cause situations of metadata server overload and client crash, resulting in training failure. For example, a large amount of file metadata information is stored in the MDS, all file metadata information may be read in a short time when the CephFS client executes some commands, which may cause a sudden increase in the MDS load and affect the cluster stability performance, and a large amount of file information may be read in a short time when the CephFS client executes some commands, which may cause the CephFS client to crash and prevent the user from using Ceph normally. The metadata server overload and the client crash can be prevented by limiting the metadata reading operation associated with the data operation command, for example, the operation of reading a large amount of metadata information in a short time can be limited, MDS overload is prevented, the stability of a Ceph cluster is ensured, the CephFS client crash can be prevented, and the occurrence of faults is prevented. Moreover, massive metadata does not need to be packaged, and the universality is good.

In a possible implementation manner, the present application may limit the data operation command through a security mechanism, such as seccomp of a Linux kernel, and limit the execution of the data operation command by limiting the access of the process in the container to the system call. Since the associated metadata read operation is performed by the data operation command basically by calling a function (e.g., getdents) to execute the system call command through a read directory entry interface (e.g., readdir interface) of the function library (e.g., glibc). Therefore, the command of the system call can be forbidden through a safety mechanism, such as the command of forbidding getdents system call, so that the use of the data operation command is limited, and the overload of a metadata server and the crash of a client can be prevented. Since the scheme is fast and effective, but the limitation is globally effective in the whole container, the limitation cannot be only performed on partial paths such as ceph directories, and inconvenience may be brought to the user. Thus, optionally, in some special scenarios, this may be used, for example, the stored data has confidentiality (e.g., carries a confidential label, or has a higher confidential level than a preset level, or is specified by a user, etc.), and the user needs to be restricted from reading; as another example, the data volume of the stored data is large, and only needs to be stored, and does not need to be acquired by the user, so as to enhance the user experience.

In a possible implementation manner, the application may receive an input data operation command, obtain path information of a target path corresponding to the data operation command, and further determine whether to perform limitation processing on metadata reading operation associated with the data operation command according to the path information. This may help prevent metadata server overload and client crashes.

In a possible implementation manner, the scheme may further combine the above-mentioned two methods of limiting the data operation command through the security mechanism and determining whether to limit the data operation command according to the path information, and determine which scheme to use based on some judgment mechanisms. For example, a scheme of determining whether to perform limitation processing on the data operation command according to the path information is adopted, and when it is detected that limitation processing of a metadata reading operation associated with the data operation command input in the user container exceeds a threshold (e.g., 90%), or a switching instruction of a user is received and it is determined that the user has a switching right, the system can be switched to perform limitation processing on the data operation command through a security mechanism, and relevant system calls are disabled, such as disabling a getdents system call command. For another example, when a scheme of performing limitation processing on a data operation command through a security mechanism is adopted, if it is detected that the number of times of the data operation command input in the user container reaches a certain number threshold, it indicates that the user has a great demand for performing data operation, a prompt message may be output to prompt the user to switch/modify the metadata reading limitation scheme, for example, disabling related system calls such as getdents system call commands is cancelled, and when it is detected that the executing user has a switching right, switching is performed to determine whether to perform limitation processing on the data operation command according to path information, and so on, which are not listed one by one.

Alternatively, the data related to the embodiments of the present application may be stored in a blockchain, for example, a plurality of servers in the distributed storage system may be combined into a blockchain, and the servers are nodes on the blockchain. Referring To fig. 1b, fig. 1b is a schematic structural diagram of a block chain-based distributed storage system according To an embodiment of the present invention, which is formed by a plurality of nodes (computing devices in any form in an access network, such as servers and terminals), and connected To clients, where the nodes form a Peer-To-Peer (P2P, Peer To Peer) network, and a P2P Protocol is an application layer Protocol operating on a Transmission Control Protocol (TCP). In a distributed system, any machine, such as a server or a terminal, can join to become a node, and the node comprises a hardware layer, a middle layer, an operating system layer and an application layer. Referring to the functions of each node in the blockchain system shown in fig. 1b, the functions involved include: routing, a basic function that a node has for supporting communication between nodes; the application is used for being deployed in a block chain, realizing specific services according to actual service requirements, recording data related to the realization functions to form recording data, carrying a digital signature in the recording data to represent a source of task data, and sending the recording data to other nodes in the block chain system, so that the other nodes can add the recording data to a temporary block when the source and integrity of the recording data are verified successfully; and the Block chain comprises a series of blocks (blocks) which are mutually connected according to the generated chronological order, new blocks cannot be removed once being added into the Block chain, and recorded data submitted by nodes in the Block chain system are recorded in the blocks.

The metadata provided in the present application may also be referred to as intermediate data, relay data, and the like in some scenarios, and the present application is not limited thereto. Metadata may include information describing attributes of data, and may be used to support functions such as indicating storage locations, historical data, resource lookups, file records, and the like. Optionally, the data operation command may also be referred to as a file metadata operation command, a metadata operation command, or other names, which is not limited in this application.

It is to be understood that the foregoing scenarios are only examples, and do not constitute a limitation on application scenarios of the technical solutions provided in the embodiments of the present application, and the technical solutions of the present application may also be applied to other scenarios. For example, as can be known by those skilled in the art, with the evolution of system architecture and the emergence of new service scenarios, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

Based on the above description, the embodiment of the present application provides a metadata reading restriction method, and details will be described below with respect to a scheme for determining whether to perform restriction processing on a metadata reading operation associated with a data operation command according to path information.

Please refer to fig. 2, which is a flowchart illustrating a metadata read restriction method according to an embodiment of the present application. As shown in FIG. 2, the metadata read restriction method may include

steps

201 and 203.

201. An input data operation command is received, the data operation command, when executed, being associated with a metadata read operation.

The data operation command may be a command associated with a metadata read operation, that is, a metadata read is involved in the execution process. For example, the data operation command may be a command to obtain a file list, a command to calculate a file size, a command to search for a target file, a command to search for specified content, and the like, which are not listed here. Specifically, the command to obtain the file list may be used to list all file names in the target path, and distinguish the file types by color (for example, green represents an executable file, red represents a compressed file, and the like), for example, the command to obtain the file list may be an ls command of Linux; the command for calculating the file size may be used to list size information of all files and folders in the target path, for example, the command for calculating the file size may be a du command of Linux; the command for searching the target file can be used for searching the target file under the target path, and the target file can set the retrieval condition of the target file when a data operation command is input, for example, the command for searching the target file can be a find command of Linux; the command for searching for the specified content may be used to read all information of all files in the target path and search for the specified content in the files, where the specified content may be a character string meeting a condition in the file, and may be set when a data operation command is input, and for example, the command for searching for the specified content may be a grep command of Linux. Alternatively, the data manipulation command may be obtained after the container is started. That is, after the container of the user is started, if the data operation command is executed, the metadata of a large number of Ceph files may be read.

In this application, when the data operation command is executed, the operation corresponding to the data operation command is realized by reading the metadata, that is, the data operation command related to this application is associated with the metadata reading operation when being executed. For example, when a command to acquire a file list is executed, acquisition of the file list needs to be realized by reading of metadata.

202. And acquiring the path information of the target path corresponding to the data operation command.

The path information related to the present application may include any one or more of the following: information indicating whether the target path is an internal path of a distributed storage system (e.g., Ceph), information indicating whether the target path indicates a folder, information indicating whether the number of files within the target path exceeds a threshold, information indicating whether the target path is a path when the data operation command is input, information indicating whether the target path is a path indicated/included by the data operation command, information indicating whether the data operation command includes a recursion parameter, and the like. The target path is an internal path of the distributed storage system, which may mean that the target path belongs to a path in a folder for storing in the distributed storage system, such as a Ceph internal path. The information indicating whether the target path indicates a folder may refer to whether an object indicated by the target path is a folder, for example, one target path is: file _ list1, since File _ list1 is a folder, the path information indicates that the target path is a folder, if a target path: the/File _ list1/file.txt, since file.txt is a txt File, the path information indicates that the target path is not a folder. Whether the number of files in the target path exceeds the threshold value or not may refer to the number of files in the target path, and if one or more folders are included in the target path, the number of files in the one or more folders is also counted as the number of files in the target path.

Optionally, the target path corresponding to the data operation command may be a path indicated by the data operation command, or may be a path when the data operation command is input, that is, a current path or a default path, and the like.

For example, the command for inputting the list of acquired files is: ls/File _ list1, the target path corresponding to the command to obtain the File list is/File _ list1, and the command to obtain the File list indicates all files under the path of the get/File _ list 1. As another example, the command for inputting the file list is: ls, the target path corresponding to the data operation command is the path when the command for acquiring the File list is input, the command indicates that all files under the path when the data operation command is input are acquired, and if the path when the data operation command is input is/File _ list2, the command indicates that all files under the target path of/File _ list2 are acquired.

203. And determining whether to limit the metadata reading operation associated with the data operation command or not according to the path information.

Wherein, whether to limit the metadata reading operation associated with the data operation command can be determined according to whether the path information meets a limiting condition, and the limiting condition can be any one or more of the following items: the first condition, the data operation command, includes a recursion parameter, a second condition. The first condition may mean that the target path is an internal path of the distributed storage system, the target path indicates a folder, and/or the number of files within the target path exceeds a threshold. The second condition may indicate that the target path is an internal path of the distributed storage system and/or that the number of files within the target path exceeds a threshold. Therefore, the limitation processing of the data reading operation associated with the command can be realized when the path information meets the limitation condition.

Optionally, the limiting process may be to prohibit part or all of the operations in the original data operation command, such as prohibiting execution of the color distinguishing operation, or prohibit all operations of metadata reading associated with the data operation command, that is, no longer trigger the relevant metadata reading. Further optionally, a prompt message may also be returned, which may be used to indicate that the data operation command was not executed. For example, it may be output that the data operation command was not executed; and/or, return the number of files; and/or prompting the user that the number of files is too large, and overload or crash risks may exist; and/or return an error, etc., and the application is not limited.

Please refer to fig. 3a in conjunction with fig. 2, which is a flowchart illustrating another metadata reading restriction method according to an embodiment of the present application. In this embodiment, the data operation command may be a command to obtain a file list, where the command to obtain the file list may be used to list all file names in the target path and distinguish the file types by colors, such as green for executable files, red for compressed files, and so on, and may be an ls command of Linux, for example. As shown in FIG. 3a, the metadata read restriction method may include

steps

3101 and 3104.

3101. And receiving an input command for acquiring the file list.

3102. Whether the path information satisfies a first condition. If so, step 3103 may be performed, otherwise, step 3104 may be performed.

After receiving a command to acquire a file list, whether to restrict a metadata reading operation corresponding to the command may be determined according to whether path information corresponding to the command satisfies a first condition. The first condition includes: the target path is an internal path of the distributed storage system, such as Ceph, the target path indicates a folder, and the number of files within the target path exceeds a threshold, such as 100.

3103. And limiting the metadata reading operation associated with the command of acquiring the file list.

3104. And executing metadata reading operation according to the command of acquiring the file list.

When the path information satisfies the first condition, step 3103 may be performed to perform a restriction process with respect to a metadata read operation associated with the command to acquire the file list. Since the original command for obtaining the file list may list all file names and colors corresponding to files in the target path when executed, and directly disabling the data operation command may affect normal use of the user, the limiting process may be performed on the metadata reading operation associated with the command for obtaining the file list, such as disabling the file color display operation associated with the command for obtaining the file list (for example, setting a designated parameter color in the command to new), reading the metadata from the metadata server, and executing the data operation corresponding to the command for obtaining the file list. The file color display operation can be used for distinguishing different file categories, so that the display colors of the files in different categories are different. Optionally, when the path information does not satisfy the first condition, for example, the path is not a Ceph internal path, or is not a folder, or does not exceed the threshold, step 3104 may be executed, where a corresponding metadata reading operation is executed according to the command to obtain the file list, the file list under the target path is obtained, and the file type is distinguished by color, that is, the command to obtain the file list is normally executed. Due to the elimination of the color display, reading of a large amount of metadata information, such as the metadata information of a large number of MDSs, can be avoided at this time, thereby greatly reducing the access of the MDSs.

Optionally, in order to reduce the amount of computation and to determine whether to perform limitation processing on the metadata reading operation associated with the data operation command more quickly, please refer to fig. 3b, which is a flowchart illustrating another metadata reading limitation method provided in this embodiment of the present application. In this embodiment, a distributed storage system is described as an example of Ceph. As shown in FIG. 3b, the metadata read restriction method may include steps 3201-3206.

3201. And receiving an input command for acquiring the file list.

3202. Whether the target path is a Ceph internal path. If so, step 3203 may be performed, otherwise, step 3206 may be performed.

3203. Whether the target path indicates a folder. If so, step 3204 may be performed, otherwise, step 3206 may be performed.

3204. Whether the number of files within the target path exceeds a threshold. If so, step 3205 may be performed, otherwise, step 3206 may be performed.

3205. And limiting the metadata reading operation associated with the command of acquiring the file list.

3206. And executing metadata reading operation according to the command of acquiring the file list.

That is, when determining whether the first condition is satisfied, step 3202 may be executed first, and determine whether the target is a Ceph internal path, if not, step 3206 is directly executed, the metadata reading operation is executed according to the command of obtaining the file list, and the rest of the determination is not performed; if the path is a Ceph internal path, then execute step 3203 to determine whether the target path is a folder, if not, directly execute step 3206 to execute corresponding metadata reading operation according to the command of obtaining the file list; if the file is a folder, executing step 3204 again, determining whether the number of files in the target path exceeds a threshold, if not, executing step 3206, and executing corresponding metadata reading operation according to the command of the acquired file list; if the value exceeds the threshold value, step 3205 is executed to perform a limitation process on the metadata reading operation, such as prohibiting a file color display operation associated with the command to obtain the file list. The calculation amount can be reduced, and the efficiency of determining whether to perform limitation processing on the metadata reading operation can be improved.

Please refer to fig. 4a in conjunction with fig. 2, which is a flowchart illustrating another metadata reading restriction method according to an embodiment of the present application. In this embodiment, the data operation command may be a command to calculate a file size or a command to search for a target file, and the command to calculate the file size may be used to list size information of all files and folders in a target path, for example, a du command of Linux; the command for searching the target file can be used for searching the target file under the target path, such as a find command of Linux, the target file can set the searching condition of the target file when a data operation command is input, and executing the find command searches all file names meeting the condition under the target path. As shown in FIG. 4a, the metadata read restriction method may include steps 4101-4104.

4101. An input command to calculate a file size or a command to find a target file is received.

4102. Whether the path information satisfies a first condition. If so, step 4103 can be performed, otherwise, step 4104 can be performed.

After receiving a command for calculating a file size or a command for searching for a target file, whether to restrict a metadata reading operation corresponding to the command may be determined according to whether path information corresponding to the command satisfies a first condition. The first condition includes: the target path is an internal path of a distributed storage system, such as Ceph, the target path indicates a folder, and the number of files within the target path exceeds a threshold.

4103. And limiting the metadata reading operation associated with the command for calculating the file size or the command for searching the target file.

4104. And executing metadata reading operation according to the command for calculating the file size or the command for searching the target file.

Optionally, when the path information satisfies the first condition, step 4103 is executed to perform limitation processing on a metadata reading operation associated with a command for calculating a file size or a command for searching a target file.

Optionally, the limiting process performed on the metadata reading operation associated with the command for calculating the file size or the command for searching the target file may be: and forbidding executing the command for calculating the file size or the command for searching the target file, and outputting a prompt message for indicating that the data operation command is not executed. For example, the prompt message may be the number of files in the output target path, and prompt the user that the number of files in the output target path exceeds a threshold, and the data operation command is not executed, or may be an execution failure or an execution error of the data operation command, which is not limited herein.

Optionally, when the path information does not satisfy the first condition, step 4104 may be executed to execute a corresponding metadata reading operation according to the received command. Further alternatively, if the command is a command to calculate the file size, the size information of all files and folders under the target path may be listed, or if the command is a command to search for the target file, the file name of the target file under the target path may be listed, and so on.

Optionally, in order to reduce the amount of computation and to more quickly determine whether to perform limitation processing on the metadata reading operation associated with the command for computing the file size or the command for searching the target file, please refer to fig. 4b, which is a flowchart illustrating another metadata reading limitation method provided in the embodiment of the present application. In this embodiment, a distributed storage system is described as an example of Ceph. As shown in FIG. 4b, the metadata reading limitation method may include

steps

4201 and 4206.

4201. An input command to calculate a file size or a command to find a target file is received.

4202. Whether the target path is a Ceph internal path. If so, step 4203 may be performed, otherwise, step 4206 may be performed.

4203. Whether the target path indicates a folder. If so, step 4204 may be performed, otherwise, step 4206 may be performed.

4204. Whether the number of files within the target path exceeds a threshold. If so, step 4205 may be performed, otherwise, step 4206 may be performed.

4205. And limiting the metadata reading operation associated with the command for calculating the file size or the command for searching the target file.

4206. And executing metadata reading operation according to the command for calculating the file size or the command for searching the target file.

That is, when determining whether the first condition is satisfied, step 4202 may be executed first, and determine whether the target is a Ceph internal path (if the target path is in the Ceph fs directory, a large number of short-time Ceph metadata requests may be caused), if not, step 4206 is executed directly, and the corresponding metadata reading operation is executed according to the command for calculating the file size or the command for searching the target file, and the rest of the determination is not performed; if the path is a Ceph internal path, then go to step 4203, determine if the target path is a folder, if not, go to step 4206 directly, and execute the metadata reading operation according to the command for calculating the file size or the command for searching the target file; if the file is a folder, go to step 4204, determine whether the number of files in the target path exceeds a threshold (e.g., 200, 500, or 1000), if not, go to step 4206, execute the metadata reading operation according to the command for calculating the file size or the command for searching the target file; if the value exceeds the threshold value, step 4205 is executed to perform a limitation process on the metadata reading operation, for example, to prohibit the command for calculating the file size or the command for searching the target file, and output a prompt message. Therefore, the calculation amount can be reduced, and the efficiency of determining whether to perform limitation processing on the metadata reading operation is improved.

Please refer to fig. 5a in conjunction with fig. 2, which is a flowchart illustrating another metadata reading restriction method according to an embodiment of the present application. In this embodiment, the data operation command may be a command for searching for specified content, such as a grep command of Linux, where the command for searching for specified content may be used to read all information of all files in the target path and search for specified content in the files, for example, the specified content may be a character string in the files that meets a condition, and may be set when the data operation command is input. For the operation of this command, if the target path is within the CephFS directory, a short, large number of Ceph metadata requests may result. At the same time, however, the grep command may also read data from the standard input, thus disabling the direct disabling of normal use by the user. Thus, additional decisions need to be made for the grep command. As shown in FIG. 5a, the metadata read restriction method may include steps 5101-5108.

5101. An input command to seek specified content is received.

5102. The target path is a path when the command for finding the specified content is input.

5103. The command to find the specified content contains a recursion parameter. If so, step 5104 may be performed, otherwise, step 5105 may be performed.

When the command to search for the specified content is received, if the target path corresponding to the command to search for the specified content is the path when the command to search for the specified content is input, step 5103 may be executed to detect whether the command to search for the specified content includes a recursive parameter (e.g., a recursion parameter). Optionally, the recursive parameter refers to a statement indicating a recursive operation in the command for finding the specified content, for example, a command for finding the specified content: grep-r "import"/home/, which means to search for a file with the string "import" under the target path/home, -r is the recursive parameter.

Further, if the recursive parameter is not included, step 5105 may be executed to execute a metadata reading operation according to the command for searching the specified content; if recursive parameters are included, 5104 may be performed.

5104. Whether the path information satisfies a second condition. If so, step 5108 may be performed, otherwise, step 5105 may be performed.

Wherein the second condition may include: the target path is an internal path of a distributed storage system such as Ceph and the number of files in the target path exceeds a threshold.

When the path information satisfies the second condition, step 5108 is executed to perform limitation processing for the metadata reading operation associated with the command for finding the specified content. For example, the limitation processing may refer to a command for prohibiting execution of the search for the specified content, and outputting a prompt message for indicating that the command for searching for the specified content is not executed. When the path information does not satisfy the second condition, step 5105 may be performed to perform a metadata read operation according to the command to find the specified content.

5105. And executing the metadata reading operation according to the command for searching the specified content.

5106. The target path is the path indicated by the command for finding the specified content.

5107. Whether the path information satisfies a first condition. If so, step 5108 may be performed, otherwise, step 5105 may be performed.

5108. And limiting the metadata reading operation associated with the command for searching the specified content.

When a command for searching for specified content is received, if the target path corresponding to the command for searching for specified content is the path indicated by the command for searching for specified content and the path information thereof satisfies the first condition, step 5108 may be executed to perform limitation processing on the metadata reading operation associated with the command for searching for specified content. Alternatively, the limitation processing may refer to prohibiting the command for searching for the specified content from being executed, and outputting a prompt message indicating that the command for searching for the specified content is not executed. Wherein the first condition comprises: the target path is an internal path of a distributed storage system, such as Ceph, the target path indicates a folder, and the number of files within the target path exceeds a threshold. If the path information does not satisfy the first condition, step 5105 may be executed to perform a metadata read operation according to the command for searching for the specified content.

Optionally, in order to reduce the amount of computation and to quickly determine whether to perform limitation processing on the metadata reading operation associated with the command for searching for the specified content, please refer to fig. 5b in combination with fig. 5a, which is a flowchart illustrating another metadata reading limitation method provided in an embodiment of the present application. In this embodiment, a distributed storage system is described as an example of Ceph. As shown in FIG. 5b, the metadata read restriction method may include steps 5201-5209.

5201. An input command to seek specified content is received.

5202. The target path is a path when the command for finding the specified content is input.

5203. The command to find the specified content contains a recursion parameter. If so, step 5206 may be performed, otherwise, step 5208 may be performed.

5204. The target path is the path indicated by the command for finding the specified content.

5205. Whether the target path indicates a folder. If so, step 5206 may be performed, otherwise, step 5208 may be performed.

5206. Whether the target path is a Ceph internal path. If so, step 5207 may be performed, otherwise, step 5208 may be performed.

5207. Whether the number of files within the target path exceeds a threshold. If so, step 5209 may be performed, otherwise, step 5208 may be performed.

5208. And executing the metadata reading operation according to the command for searching the specified content.

5209. And limiting the metadata reading operation associated with the command for searching the specified content.

That is, the method of indicating the target path may be determined first, and if the target path is the path indicated by the command for searching for the specified content, the path information may be determined with reference to fig. 4b, and it may be determined whether or not to perform the limitation processing on the metadata reading operation associated with the command for searching for the specified content. If the target path is the path when the command for searching the specified content is input, executing step 5203, determining whether the command for searching the specified content includes a recursion parameter, if not, executing step 5208, executing a metadata reading operation according to the command for searching the specified content; if the recursive parameter is included, executing step 5206 to determine whether the target path is an internal path of the distributed storage system, and if not, executing step 5208 to execute a metadata reading operation according to the command for searching the specified content; if the path is an internal path, executing step 5207 to determine whether the number of files in the target path exceeds a threshold, and if not, executing step 5208 to execute a metadata reading operation according to the command for searching the specified content; if the threshold is exceeded, step 5209 is executed to perform a restriction process on the metadata read operation associated with the command for finding the specified content. The calculation amount can be reduced, and the efficiency of determining whether to perform limitation processing on the metadata reading operation can be improved.

According to the embodiment of the application, when a data operation command associated with metadata reading operation is received, the path information of the target path corresponding to the data operation command can be acquired, and whether limitation processing is performed on the metadata reading operation or not can be determined according to the path information, so that the reading operation of limiting part of metadata can be realized.

By adopting the technical scheme of the application, the metadata server such as Ceph MDS is prevented from being overloaded: when more target path storage files in the Ceph reach the million level, a large number of MDS requests can be caused by directly executing massive Ceph metadata operation, so that MDS overload of a Ceph cluster is caused. In addition, the application is also helpful for preventing the Ceph client of the user from crashing: because great pressure is brought to the client while a large number of MDS requests are executed, the Ceph client program is directly crashed, and therefore the user container cannot be normally used. And extra cost and calculation amount are not required to be increased, files are not required to be compressed, the method is not limited by scenes, and user experience is enhanced. In addition, all the restriction programs in the application can judge the target path of the command executed by the user, and can trigger the restriction processing by limiting the target path to be the catalogue in the CephFS, and the number of files in the catalogue reaches a certain threshold, for example, the number of files is returned and the user is prompted not to execute massive Ceph metadata operation on the catalogue, so that the normal use of the user is hardly influenced, and the user experience is good.

Fig. 6 is a flowchart illustrating another metadata reading restriction method according to an embodiment of the present invention. As shown in fig. 6, the metadata read restriction method may include the following steps 601-603:

601. and starting a user container, and acquiring an original command file corresponding to the data operation command in the user container.

The user container is a flexible virtualization processing mode, and a user can run a program through the user container. In this embodiment of the present application, a user may input a data operation command through a user container, where the data operation command may be: a command to acquire a file list, a command to calculate a file size, a command to search for a target file, or a command to search for specified content. Starting the user container may refer to the user submitting a user container creation request to create the user container, or may refer to the user starting the user container that has been created, which is not limited herein. The original command file is used for indicating the original execution logic of the user container for the data operation command, and does not relate to the limiting processing logic of the metadata operation.

602. And replacing the original command file with a target command file corresponding to the data operation command.

The target command file is used to determine whether to perform limitation processing on the metadata reading operation associated with the data operation command, or may be used to indicate an execution logic for the data operation command after an original command file of the user container. Optionally, the original command file is replaced with a target command file corresponding to the data operation command, or an execution logic that determines to perform limitation processing on the metadata reading operation associated with the data operation command according to the judgment of the path information is added on the basis of the original command execution logic. In the target command file corresponding to the data operation command, the execution logic for the data operation command is the execution logic set forth in step 203. For example, if the data operation command is a command for acquiring a file list, the execution logic of the command file is modified to the execution logic shown in fig. 3a or 3b, where the prohibition of the file color display operation in the target path may be implemented by modifying the color parameter setting value in the original command file for the command for acquiring the file list, such as specifying the parameter color as never.

Optionally, after replacing the original command file with the target command file, the attribute of the command file may be modified to prevent the user from modifying the target command file at will, or to bypass a logic for performing restriction processing on a metadata read operation associated with the data operation command in the target command file.

Further, the user container may be delivered after replacing the original command file of the user container, so that the execution logic indicated by the target command file may be executed when receiving the data operation command input by the user in the user container, that is, when detecting that the data operation command satisfies the limitation condition, the metadata reading limitation operation is executed. The delivering of the user container may refer to delivering the user container to workers with different functions, for example, delivering the user container to a tester for testing, delivering the user container to an operation and maintenance worker for deployment, and publishing the user container to a production environment, which is not limited herein.

603. Receiving an input data operation command, acquiring path information of a target path corresponding to the data operation command, and determining whether to limit the metadata reading operation associated with the data operation command according to the path information and a target command file.

For example, the Ceph metadata operation command can be deployed when a user container is started, so that the Ceph metadata operation command is limited during the use process of the user. Specifically, when the container is started, the Ceph metadata operation restriction program, such as the target command file described above, may be installed inside the container to replace the original command file (including the original ls, du, find, and grep commands, etc.), so as to achieve the restriction effect. The specific flow is as follows: a) a user submits a request for creating a container; b) creating a container; c) installing and replacing default ls, du, find and grep command files of the operating system in the container; d) modifying the attributes of the command file after replacement, and preventing the user from modifying or bypassing the command (optional) limited by the metadata; e) delivering the container; f) the user executes a Ceph metadata operation command; g) the Ceph metadata operation restriction procedure is triggered.

Wherein, the step 603 can be performed with reference to the

steps

201 and 203, which are not described herein. Therefore, the data operation commands carried by the system can be replaced inside the container, and specific metadata operations of the user on the volume files in the Ceph are limited, so that the CephFS client crash and the Ceph MDS overload are prevented.

According to the method and the device, the user container can be started, the original command file of the user container is obtained, the original command file corresponding to the data operation command is replaced by the target command file comprising the execution logic for limiting the metadata reading operation, and whether the limitation processing is carried out on the metadata reading operation related to the data operation command is determined according to the path information and the target command file, so that the reading operation of limiting part of metadata can be achieved, and the method and the device are favorable for preventing the metadata server from being overloaded and the client from being crashed.

Based on the description of the above metadata reading limitation method embodiment, the present application further discloses a metadata reading limitation apparatus, which may be configured in the above electronic device, for example, the apparatus may be a computer program (including program code) running in the electronic device. The metadata read restriction means may perform the methods shown in fig. 2-6. Referring to fig. 7, the apparatus may operate as follows:

an obtaining module 701, configured to receive an input data operation command, where the data operation command is associated with a metadata reading operation when executed;

the obtaining module 701 is further configured to obtain path information of a target path corresponding to the data operation command;

a processing module 702, configured to determine whether to perform limitation processing on the metadata reading operation associated with the data operation command according to the path information.

In one embodiment, the processing module 702 is further configured to perform a restriction process on a metadata read operation associated with the data operation command when the path information satisfies a first condition; wherein the first condition comprises: the target path is an internal path of the distributed storage system, the target path indicates a folder, and the number of files in the target path exceeds a threshold value.

In one embodiment, the data operation command is a command for acquiring a file list, and the command for acquiring the file list is associated with a file color display operation; the processing module 702 is further configured to prohibit a file color display operation associated with the command for obtaining the file list, where the file color display operation is used to distinguish different file categories.

In one embodiment, the data operation command is a command for searching for specified content, and the processing module 702 is further configured to detect whether the command for searching for specified content includes a recursion parameter when the target path is a path when the command for searching for specified content is input; if the recursive parameter is not included, the metadata reading operation is executed according to the command for searching the specified content; if the recursive parameter is included, when the path information meets a second condition, performing limitation processing on metadata reading operation associated with the command for searching the specified content; the second condition includes: the target path is an internal path of the distributed storage system and the number of files in the target path exceeds a threshold value.

In one embodiment, the processing module 702 is further configured to, when the target path is a path indicated by the command for finding the specified content and the path information satisfies a first condition, perform restriction processing on a metadata reading operation associated with the command for finding the specified content;

wherein the first condition comprises: the target path is an internal path of the distributed storage system, the target path indicates a folder, and the number of files in the target path exceeds a threshold.

In one embodiment, the data operation command is a command for calculating a file size, a command for searching a target file or a command for searching specified content; the processing module 702 is further configured to prohibit the data operation command from being executed, and output a prompt message, where the prompt message is used to indicate that the data operation command is not executed.

In an embodiment, before the receiving of the input data operation command, the processing module 702 is further configured to start a user container, and obtain an original command file corresponding to the data operation command in the user container; replacing the original command file with a target command file corresponding to the data operation command, wherein the target command file is used for determining whether to limit the metadata reading operation associated with the data operation command;

the processing module 702 determines whether to perform limitation processing on the metadata reading operation associated with the data operation command according to the path information, and specifically is configured to:

and determining whether to limit the metadata reading operation associated with the data operation command or not according to the path information and the target command file.

According to the embodiment of the application, when the data operation command associated with the metadata reading operation is received, the path information of the target path corresponding to the data operation command can be acquired, and whether the limitation processing is carried out on the metadata reading operation or not can be determined according to the path information, so that the reading operation of limiting part of metadata can be realized, and the metadata server overload and the client crash can be prevented.

The functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module. The integrated module may be implemented in a form of hardware, or may be implemented in a form of software functional module, which is not limited in this application.

Referring to fig. 8, the electronic device in the embodiment of the present application is a schematic structural diagram, and the electronic device in the embodiment of the present application includes a power supply module and the like, and includes a processor 801 and a storage device 802. Optionally, the electronic device may also include a network interface 803. Wherein data can be exchanged between the processor 801, the storage device 802 and the network interface 803, the network interface 803 is controlled by the processor for transceiving messages, the memory 802 is used for storing a computer program comprising program instructions, and the processor 801 is used for executing the program instructions stored in the memory 802. Wherein the processor 801 is configured to invoke the program instructions to perform the methods described above.

The storage device 802 may include a volatile memory (volatile memory), such as a random-access memory (RAM); the storage device 802 may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a solid-state drive (SSD), or the like; the storage means 802 may also comprise a combination of memories of the kind described above.

The processor 801 may be a Central Processing Unit (CPU) 801. In one embodiment, the processor 801 may also be a Graphics Processing Unit (GPU) 801. The processor 801 may also be a combination of a CPU and a GPU.

In one embodiment, the storage 802 is used to store program instructions. The processor 801 may invoke the program instructions to perform the steps of:

In an embodiment, the processor 801 determines, according to the path information, whether to perform limitation processing on a metadata read operation associated with the data operation command, specifically to:

when the path information meets a first condition, limiting the metadata reading operation related to the data operation command; wherein the first condition comprises: the target path is an internal path of the distributed storage system, the target path indicates a folder, and the number of files in the target path exceeds a threshold value.

In one embodiment, the data operation command is a command for acquiring a file list, and the command for acquiring the file list is associated with a file color display operation;

the processor 801 performs a restriction process on the metadata reading operation associated with the data operation command, and is specifically configured to perform: and prohibiting a file color display operation associated with the command for acquiring the file list, wherein the file color display operation is used for distinguishing different file types.

In an embodiment, the data operation command is a command for searching for specified content, and the processor 801 determines, according to the path information, whether to perform limitation processing on a metadata reading operation associated with the data operation command, specifically to perform:

when the target path is a path when the command for searching the specified content is input, detecting whether the command for searching the specified content comprises a recursion parameter;

if the recursive parameter is not included, the metadata reading operation is executed according to the command for searching the specified content;

if the recursive parameter is included, when the path information meets a second condition, performing limitation processing on metadata reading operation associated with the command for searching the specified content; the second condition includes: the target path is an internal path of the distributed storage system and the number of files in the target path exceeds a threshold value.

In one embodiment, the processor 801 determines whether to perform limitation processing on the metadata reading operation associated with the data operation command according to the path information, and is further configured to perform:

when the target path is a path indicated by the command for searching the specified content and the path information meets a first condition, performing restriction processing on metadata reading operation associated with the command for searching the specified content;

In one embodiment, the data operation command is a command for calculating a file size, a command for searching a target file or a command for searching specified content; the processor 801 performs a restriction process on the metadata reading operation associated with the data operation command, and is specifically configured to perform:

and prohibiting the data operation command from being executed, and outputting a prompt message, wherein the prompt message is used for indicating that the data operation command is not executed.

In one embodiment, before the receiving the input data operation command, the processor 801 is further configured to:

starting a user container, and acquiring an original command file corresponding to the data operation command in the user container;

replacing the original command file with a target command file corresponding to the data operation command, wherein the target command file is used for determining whether to limit the metadata reading operation associated with the data operation command;

the processor 801 determines, according to the path information, whether to perform limitation processing on the metadata reading operation associated with the data operation command, specifically to perform:

In specific implementation, the apparatuses, the processor 801, the storage apparatus 802 and the like described in the embodiments of the present application may perform the implementation manners described in the above method embodiments, and may also perform the implementation manners described in the embodiments of the present application, which is not described herein again.

Also provided in embodiments of the present application is a computer (readable) storage medium storing a computer program comprising program instructions that, when executed by a processor, perform some or all of the steps performed in the above-described method embodiments. Alternatively, the computer storage media may be volatile or nonvolatile.

Embodiments of the present application also provide a computer program product or computer program comprising program instructions, which may be stored in a computer-readable storage medium. The processor of the computer device reads the program instructions from the computer-readable storage medium, and the processor executes the program instructions, so that the computer executes part or all of the steps executed in the data presentation method, which is not described herein again.

It will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which may be stored in a computer storage medium, and the computer storage medium may be a computer readable storage medium, and when executed, the programs may include the processes of the above embodiments of the methods. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

While the present disclosure has been described with reference to particular embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure.

Claims

1. A metadata read restriction method, comprising:

2. The method of claim 1, wherein the path information comprises: information indicating whether the target path is an internal path of a distributed storage system, information indicating whether the target path indicates a folder, and information indicating whether the number of files in the target path exceeds a threshold;

the determining whether to perform limitation processing on the metadata reading operation associated with the data operation command according to the path information includes:

3. The method according to claim 2, wherein the data operation command is a command to obtain a file list, the command to obtain the file list being associated with a file color display operation;

the limiting processing for the metadata reading operation associated with the data operation command comprises:

and prohibiting a file color display operation associated with the command for acquiring the file list, wherein the file color display operation is used for distinguishing different file types.

4. The method according to claim 1, wherein the data operation command is a command for searching for specified content, and the determining whether to perform limitation processing for the metadata reading operation associated with the data operation command according to the path information comprises:

5. The method of claim 4, wherein determining whether to restrict processing for a metadata read operation associated with the data operation command according to the path information further comprises:

6. The method of claim 1, wherein the data manipulation command is a command to calculate a file size, a command to search for a target file, or a command to search for specified content;

7. The method of claim 1, wherein prior to said receiving an input data manipulation command, the method further comprises:

8. A metadata read restriction apparatus, comprising:

9. An electronic device comprising a processor and a storage device, the processor being interconnected with the storage device, wherein the storage device is configured to store computer program instructions, and the processor is configured to execute the program instructions to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, having stored thereon computer program instructions, which, when executed by a processor, are adapted to perform the method of any one of claims 1-7.