CN115495412A - Query system and device - Google Patents

Query system and device Download PDF

Info

Publication number
CN115495412A
CN115495412A CN202110739776.5A CN202110739776A CN115495412A CN 115495412 A CN115495412 A CN 115495412A CN 202110739776 A CN202110739776 A CN 202110739776A CN 115495412 A CN115495412 A CN 115495412A
Authority
CN
China
Prior art keywords
query
client
target
entry
file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110739776.5A
Other languages
Chinese (zh)
Inventor
阿比吉特霍尔德
李楚
罗先强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN115495412A publication Critical patent/CN115495412A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/14Details of searching files based on file metadata
    • G06F16/144Query formulation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems

Abstract

The application provides an inquiry system and an inquiry device. The query system comprises a client and a storage device, wherein the client is used for sending a query request to the storage device, and the query request is used for requesting to query a target item meeting a query condition under a target path; the storage device is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining the target entry; the target item is any item meeting the query condition under the directory item; sending the target item to the client; the client is used for receiving the target item sent by the storage device. Through the method, the storage device can filter the directory entries based on the query condition, and the number of the target entries obtained after filtering is smaller than that of the complete directory entries, so that the data transmission quantity can be remarkably reduced, and the utilization rate of network bandwidth is improved. Meanwhile, the CPU overhead and the disk I/O can be reduced, the time delay is reduced, and the utilization rate of the CPU and the memory is improved.

Description

Query system and device
Cross Reference to Related Applications
This application claims priority to the filing of indian patent application having the application number 20213102723.4, entitled "directory query method for distributed file system" by indian patent office at 18/06/2021, the entire contents of which are incorporated herein by reference.
Technical Field
The present application relates to the field of computer technologies, and in particular, to an inquiry system and apparatus.
Background
A distributed or shared storage system (e.g., a network-attached storage (NAS) system or cluster) typically includes multiple NAS devices that provide file-based data storage services to other devices (clients, such as application servers) in the network. Generally, a NAS device is a specialized computer system used to store and service files. Therefore, NAS devices have lower capabilities than general-purpose computer systems such as application servers. For example, a NAS device may have a simpler operating system and a less powerful processor, and may also lack a keyboard and display, among other components.
The application server can retrieve data stored in the NAS System via a Network using a data sharing or File sharing protocol such as a Network File System (NFS) or a Common Internet File System (CIFS). After retrieving the data, the application server may analyze the data as mentioned above.
For example, the NAS client may access a file system on the NAS device, such as accessing directory entries in a specified path, and the NAS client may perform some operations on the directory entries, such as filtering the entries based on file attributes, and so on. Conventionally, it may be necessary to perform the above-mentioned purpose by multiple round trips between the NAS client and the NAS device, for example, when the directory entry is large, the NAS device may only return part of entries to the client at a time, and after the NAS client acquires the entries returned by the NAS device, the NAS client acquires file attributes of the entries one by one from the NAS device.
In summary, the existing method consumes a relatively large amount of bandwidth, and the delay is also high.
Disclosure of Invention
The application provides a query system and a query device, which are used for improving the file query efficiency and reducing the network delay.
In a first aspect, an embodiment of the present application provides a query system, where the query system includes a client and a storage device, where the client is configured to send a query request to the storage device, and the query request is used to request to query a target entry meeting a query condition in a target path; the storage device is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; the client is also used for sending the target item to the client; the client is used for receiving the target items sent by the storage device.
In the above manner, the storage device may filter the directory entries based on the query condition, and since the number of the target entries obtained after filtering is smaller than the number of the complete directory entries, the data transmission amount may be significantly reduced, thereby improving the utilization rate of the network bandwidth. Meanwhile, the CPU overhead and the disk I/O can be reduced, the time delay is reduced, and the utilization rate of the CPU and the memory is improved.
In one possible implementation, the query condition includes one or more of the following: specifying a file name and specifying a file attribute;
the file attributes include: file size, creation time, last modification time, file type, access rights.
In a possible implementation manner, the client is further configured to detect query information input by a user, where the query information includes information indicating the target path and information indicating the query condition; generating the query request based on the query information.
By the above manner, the technical scheme of the application can be applied to other query mechanisms, such as the query command triggered by the existing find application program, so that the scheme has better portability and compatibility.
In a second aspect, an embodiment of the present application provides another query system, which includes a client and a storage device, where the client is configured to send a query request to the storage device, where the query request is used to request to query a target entry that satisfies a query condition under a target path, and perform a specified operation on the target entry; the storage device is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; performing the specified operation on the target entry; the storage device is further used for sending a query response to the client, wherein the query response is used for indicating an operation result; the client is further used for receiving the query response sent by the storage device.
In the above manner, the query response returned by the storage device is used for indicating the operation result indicating whether the specified operation is successful, and since the target entry does not need to be transmitted, the data transmission amount can be significantly reduced, thereby improving the utilization rate of the network bandwidth. Meanwhile, the CPU overhead and the disk I/O can be reduced, the time delay is reduced, and the utilization rate of the CPU and the memory is improved.
In one possible implementation, the specified operation includes one or more of the following: delete operation, rename operation, modify access rights.
In a third aspect, an embodiment of the present application further provides a storage apparatus, where the storage apparatus has a function of implementing a behavior of a storage device in the system shown in the first aspect, and for beneficial effects, reference may be made to description of the first aspect and details are not described here again. Or implement the function of storing device behavior in the illustrated system of the second aspect, and the beneficial effects may be referred to the description of the second aspect and will not be described herein again. The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above. In one possible design, the apparatus includes a structure including a receiving module, a processing module, and a transmitting module. The modules may perform corresponding functions of the storage device in the system example in the first aspect or the second aspect, for specific reference, detailed description in the method example is given, and details are not repeated here.
In a fourth aspect, the present application further provides another storage device, where the storage device includes a processor and a memory, and may further include a communication interface, where the processor executes program instructions in the memory to perform operations performed by the storage device provided in the first aspect or any of the possible implementations of the first aspect, or perform operations performed by the storage device provided in the second aspect or any of the possible implementations of the second aspect. The storage device may be a server or the like. The memory is coupled to the processor and holds program instructions and data necessary for the query process. The communication interface is used for communicating with other equipment.
In a fifth aspect, the present application provides a computer-readable storage medium, which, when executed by a storage device, performs the operations performed by the storage device provided in the foregoing first aspect or any possible implementation manner of the first aspect, or performs the operations performed by the storage device provided in the foregoing second aspect or any possible implementation manner of the second aspect. The storage medium stores a program therein. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard Disk Drive (HDD), and Solid State Drive (SSD).
In a sixth aspect, the present application provides a computing device program product comprising computer instructions that, when executed by a computer, perform the operations of the storage device provided in the foregoing first aspect or any possible implementation manner of the first aspect, or perform the operations performed by the storage device provided in the foregoing second aspect or any possible implementation manner of the second aspect. The computer program product may be a software installation package, which may be downloaded and executed on a computing device in case it is required to use the functionality of the storage device as provided in any of the possible implementations of the first or second aspect described above.
In a seventh aspect, the present application further provides a computer chip, where the chip is connected to a memory, and the chip is configured to read and execute a software program stored in the memory, to perform operations of a storage device in each possible implementation manner of the first aspect and the first aspect, or to perform operations of a storage device in each possible implementation manner of the second aspect and the second aspect.
For the advantageous effects achieved by the third aspect to the seventh aspect, please refer to the description of the first aspect or the second aspect about the advantageous effects of the storage system, which is not repeated herein.
Drawings
FIG. 1A is a schematic diagram of a system architecture according to an embodiment of the present application;
fig. 1B is a schematic diagram of a file system on a NAS device according to an embodiment of the present application;
fig. 2 is an interaction diagram of a NAS client and a NAS device according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a query method according to an embodiment of the present application;
fig. 4A is a schematic diagram illustrating a format of a query request according to an embodiment of the present application;
FIG. 4B is a diagram illustrating another format of a query request according to an embodiment of the present application;
fig. 5 is a schematic view of a query process provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of another query process provided in the embodiments of the present application;
fig. 7 is a schematic structural diagram of a NAS device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of another NAS device according to an embodiment of the present application.
Detailed Description
Fig. 1A is a schematic diagram of another system architecture according to an embodiment of the present disclosure. The system includes a NAS client 10 and a NAS cluster. Therein, a NAS cluster typically includes a plurality of NAS devices 20 (three NAS devices 20 are shown in fig. 1A, but not limited to NAS devices 20).
The NAS client 10 is a computing device on the user side, and the NAS client 10 may be a physical machine or a virtual machine. Physical machines include, but are not limited to, desktop computers, servers, notebook computers, and mobile devices. The NAS client 10 may communicate with any NAS device 20 within the NAS cluster over a network, which typically represents any telecommunications or computer network, including, for example, an intranet, a Wide Area Network (WAN), a Local Area Network (LAN), a Personal Area Network (PAN), or the internet.
On a hardware level, the NAS client 10 is provided with a processor, a memory, and a hard disk (hardware components are not shown in fig. 1A). On the software level, the NAS client 10 runs an operating system (not shown in fig. 1A), an application program (application) 101 (application for short), and a client program 102 (client for short). Operating systems include unix, linux, windows, hong meng system (harmony os), and the like. Application 101 is a generic term for various applications presented by a user, and may be, for example, a query application (Find). A client 102, configured to receive a data access request triggered by the application 101, and interact with the NAS device 20 to send the data access request to the NAS device 20. The client 102 is also used to receive data from the NAS device 20 and forward the data to the application 101. It is understood that when the client 102 is a software program, the functionality of the client 102 is implemented by a processor contained within the NAS client 10 running the program in memory. The client 102 may also be implemented by a hardware component, such as a network card, located inside the NAS client 10. It should be noted that fig. 1A only shows 1 client 102, and in practical applications, there may be a plurality of clients 102, for example, including an FS client and an RPC client.
The NAS device 20 may be a device having a data storage function and connected to a network. The various NAS devices 20 in the NAS cluster may communicate with each other. The NAS device 20 has a file system deployed thereon, and the NAS client 10 can access the file system of the NAS device 20 through various protocols.
FIG. 1B is a schematic diagram of the file system within the NAS device 20.
Wherein the file system is a structured data file storage and organization form. As we know, all data in a computer are 0 and 1, and a series of 01 combinations stored on a hardware medium is completely impossible for us to distinguish and manage. Therefore, the concept of 'files' is used for organizing the data, the data used for the same purpose are composed into different types of files according to the structural mode required by different application programs. Different suffixes are often used to refer to different types, and then we start a name for each document that is convenient for understanding the memory. When a large number of files exist, the files are grouped according to a certain division mode, and each group of files are placed in the same directory (or called a folder). Besides, there may be a next level directory (called sub-directory or sub-folder) under the directory, and all the files and directories form a tree structure. This tree structure has a special name: file System (File System). The file system has many types, and common FAT/FAT32/NTFS of Windows, EXT2/EXT3/EXT4/XFS/BtrFS of Linux and the like. For the convenience of searching, the names of the directories, sub-directories and files are spliced together by using special characters (such as "\" for Windows/DOS and "/" -for Unix-like systems) from the root node to the files themselves, and such a string of characters is called a file path, such as "/etc/System/system.conf" in Linux or "C:" Windows System32\ taskmtr.exe "in Windows. A path is a unique identification of access to a particular file. For example, D: \ data \ file.exe under Windows is a path of a file, which represents the file.exe file under the data directory under the D partition.
The file system is built on the block device, and not only records the file path, but also records which blocks form a file, and which blocks record directory/subdirectory information. Different file systems have different organizational structures. For convenience of management, a block device such as a hard disk may be generally divided into a plurality of logical block devices, that is, hard disk partitions (partitions). In turn, a single medium has limited capacity and performance, and multiple physical block devices may be combined into a single logical block device by some means, such as various levels of RAID, JBOD, etc. A file system may also be built on top of these logical block devices. In any case, the application server does not need to care about the specific location of the block device at the bottom layer of the file to be accessed, and only needs to send the file name/ID of the file to the file system, and the file system queries the file path according to the file name/ID.
The NAS client 10 can access the file system by the NAS device 20 via a network using a file access protocol. The more common file access protocol is NFS, CIFS, or SMB, and the like, and this embodiment does not limit this. Taking NFS as an example, NFS is a distributed file system, and is also a network protocol, and is used for accessing and sharing files among devices on the same local area network. The system and its protocol are typically used to support Network Attached Storage (NAS). Network file systems are a low cost network file sharing option that enables users and applications to access, store, and update files on remote computers as if using direct attached storage. NFS uses the Remote Procedure Call (RPC) protocol to route requests between NAS client 10 and NAS device 20. NFS uses the Remote Procedure Call (RPC) protocol to route requests between NAS client 10 and NAS device 20.
FIG. 2 shows a flow diagram of the NAS client 10 accessing the NAS device 20. The NAS client 10 may call RPC send instructions (e.g., NFS requests) onto the NAS device 20 to perform operations on the file system on the NAS device 20, such as accessing a shared directory, opening a file, reading and writing a file, etc.
For ease of understanding, the following application scenarios are presented.
Taking application 101 in fig. 1A as a find application, the find application is one of UNIX and UNIX-like operating systems, and can search for files in a local or remote file system according to specified conditions input by a user, and can apply some requested operations to each entry matching the specified conditions. The present application focuses on a query method of a remote file system, and introduces the query flow as follows by taking the NAS client 10 and the NAS device 20 as examples:
the find application may provide a query interface for a user, where the user may enter find commands to perform a conditional query. For example, the find command entered by the user is a directory entry under the query target path. As another example, find command 1 entered by the user is an entry under the query target path that specifies a name. For another example, the find command 2 is a file for querying a specified name under a target path, and for another example, the find command 3 is a file for finding a specified file name under a target path and specifying file attributes. Where file attributes such as file size, last modified time, etc., for example, specify file attributes as all files that match a given size and/or all files that are older than some timestamp.
Taking the find command 3 as an example, in a current query method, the query process of the find command 3 includes: first, a "find" command reads a directory entry under the target path using the readdir () POSIX API, and assuming the target path is within the NAS device 20, the find application first calls a PRC to send an instruction (or request) 210 to the NAS device 20 to read the directory contained in the target path. Correspondingly, the NAS device 20, in response to the instruction 210, obtains a directory entry (e.g., referred to as a target directory entry) under the target path, and then the NAS device returns the target directory entry to the NAS client 10.
The NAS client 10 may invoke read calls iteratively, the number of calls depending on the size of the target directory entry and the NFS configuration. For example, the target directory entries include 1000 entries, and the NFS is configured to return 100 entries each time, then for the first read call of the NAS client 10, the NAS device first returns the 1 st to 100 th entry entries in the 1000 directory entries to the NAS client 10, and then the NAS client 10 sends the second read call. The NAS device 20 returns the 101 th-200 th entry record entry to the NAS client 10 in response to the second read call, and so on.
The Find application, after obtaining the complete list of target directory entries, applies filters to select some of the entries that satisfy the user-specified conditions. The filter is a functional module, configured to perform filtering based on a specified condition of a user, to obtain an object that meets the filtering condition, for example, obtain an entry with a specified name, and filter the object by using a regular expression (regular expression/regex) mode.
The filtering process comprises the following steps: (1) The NAS client 10 generates an instruction 211 for obtaining the file name, and sends the instruction 211 to the NAS device 20 to obtain the name of each target directory entry one by one, and it should be noted that the NAS client 10 sends a read instruction 211 for each target directory entry to obtain the file name of the entry. Thus, this process may require multiple round trips between the NAS client 10 and the NAS device 20 to be implemented. (2) After obtaining the file name of each entry in the target directory, the NAS client 10 filters the complete target directory entry using a filter, and obtains an entry (e.g., referred to as a first entry) whose file name matches the name specified by the user. (3) Thereafter, the NAS client 10 generates an instruction 212 for acquiring file attributes, and acquires file attributes of each entry matching the specified name one by one from the NAS apparatus. Note that this also requires one round trip per entry.
Another implementation manner is that NFSv3 provides an extension to the standard readdir API, which is called readdirplus, and based on the readdirplus call, when the NAS device 20 returns the target directory entry, it actively returns the attribute of the target directory entry to the NAS client 10. This approach may optimize round trip time, but in fact, readdirplus may even have worse performance than readdir performance because the NAS client 10 may only need to load file attributes for entries matching the specified name, but readdirplus may get the file attributes of all entries to the NAS client 10, and some file attributes may not be needed.
Sometimes, the user may also perform some operation, such as deletion, on the entry that matches the specified file. For example, the find command specifies a file name and a file attribute for the target path, and performs a delete operation on the files. Then after getting an entry (e.g., referred to as a target entry) that matches the specified name, the specified attribute, the NAS client 10 sends an instruction 213 to perform a delete operation to the NAS device 20 for each second entry, and the NAS device 20 performs the delete operation on the target entry in response to the instruction 213, which also requires a round trip per entry.
The query method is acceptable when applied to a local file system, and then frequent calls and round trips may occur in a remote file system, so that the data transmission amount is large and the network delay is high.
Therefore, the embodiment of the application provides a query method, which transfers the operation of searching for the file meeting the user specified condition from the NAS client to the NAS device, so as to reduce the overall network round trip, and the method can significantly reduce the network delay.
Referring to fig. 3, a query method provided in the present embodiment is applied to the system architecture shown in fig. 1A as an example. The NAS client referred to in the method may be the NAS client 10 (or a component of the NAS client 10 such as a processor) in fig. 1A, and the NAS device may be the NAS device 20 (or a component of the NAS device 20 such as a processor).
Fig. 3 is a schematic flowchart corresponding to a query method according to an embodiment of the present disclosure. As shown in fig. 3, the method comprises the steps of:
in step 301, the NAS client sends a directory pathname to the NAS device. Correspondingly, the NAS device receives the directory pathname sent by the NAS client.
Illustratively, the directory pathname may be derived based on a find command entered by a user.
In step 302, the NAS device sends the file handle for the target directory object to the NAS client. Correspondingly, the NAS client receives the file handle sent by the NAS device.
The target directory object refers to a directory object indicated by a directory path sent by the NAS client, that is, a directory object to be queried.
Those skilled in the art will appreciate that in the NFS protocol, a computer device addresses a file or directory object via an opaque file handle. In a file system, each file or directory object has a respective file handle that is a unique identification for the file or directory object.
In fact, the NAS client, based on the find command, first queries the local cache, and if the local cache misses, sends the directory path to the NAS device. The NAS device searches for a file handle corresponding to the directory path and returns the file handle to the NAS client. In step 303, the NAS client sends a query request to the NAS device. Correspondingly, the NAS device receives the query request sent by the NAS client.
In an embodiment of the present application, the query request includes a filter condition, which may be obtained based on a user input, such as a find command, for filtering the entries in the target directory object. Specifically, the filtration conditions include, but are not limited to: the filter condition and the file attribute filter condition of the regular expression are supported, and an operation chain can be further included. By operation chain is meant one or more operations that need to be performed on the filtered target entry, such as the one or more operations comprising: delete, modify access rights, rename. Such as regular expressions used to filter file names. File attributes such as file size, creation time, last modified time, file type, etc.
For example, take the find command as an example, the find command input by the user is a file with name abc and file size larger than 10KB in the directory object under the search path a. Based on the find command, the query request may include the following filter conditions: (1) Named abc, the filter condition supports representation in a regular expression pattern. (2) The file size is larger than 10KB, which is a property filter condition.
For another example, the find command is to find a file with name abc and file size larger than 10KB in the directory object under path a, and perform a delete operation on the directory entry matching the filter condition. The query request for the find command may include an operation specified by the user, i.e., a delete operation, in addition to the aforementioned filter condition.
Referring to fig. 4A, fig. 4A is a schematic diagram of a frame structure of a query request according to an embodiment of the present disclosure.
The frame structure includes a header portion and a data portion. The header is used for placing some description information about filtering the query request, and the data part is used for storing information such as filtering conditions and operation chains.
Each of the frame structures is specifically described as follows:
1. a head;
1, version: is an input parameter to the NAS device that represents the version of the filter applied on the NAS client. Wherein the filter can be a functional module such as an application or plug-in that can be installed on the NAS client and the NAS device for filtering the directory entries based on the user-specified filtering criteria.
The NAS device determines based on the parameter whether the filter used by itself is compatible with the filter used by the NAS client, and if the NAS device does not support the version of the filter being used by the NAS client, the NAS device will return an error indication to the NAS client indicating a compatibility issue.
2, size of storage area: the parameter is an input parameter of the NAS device, and is used to indicate the size of the storage area allocated by the NAS client for the current read call. The NAS device returns a corresponding data amount according to the size of the storage area, for example, if the size of the storage area allocated by the NAS client is 8Kb, the NAS device returns data not greater than 8k to the NAS client.
The storage area may be a buffer (buffer), and in the following description, the buffer will be taken as an example.
3, data offset: is an input parameter to the NAS device that indicates the amount of the buffer in which the NAS device can begin filling out the directory entry for the target directory object at the offset indicated by the parameter. For example, as shown in FIG. 4A, the data offset may be an offset between the start location of the query request to the start location of the list of entries. After placing the data query request in its own buffer, the NAS device starts to fill the read directory entry based on the buffer location indicated by the parameter.
4, cookie: both input parameters and output parameters of the NAS device. The Cookie is opaque data for the NAS client, the NAS client does not need to parse the data, and the NAS client does not care about the content of the data, and is used for marking the position of a returned directory entry when the NAS device cannot return a complete directory entry of a target directory object once, where usually the Cookie contains a start address (such as offset information of the start address) of a next directory entry of the returned directory entry, and the NAS client returns the Cookie to the NAS device after receiving the Cookie, and the Cookie is parsed by the NAS device to indicate where the NAS device starts to read next time.
The NAS client may use a cookie-iterated invoke query request to obtain the full directory entry. For example, the directory entries include 1000 entries, the NAS device returns 100 entries each time, and the NAS device first sends 1 st to 100 th entries in the directory entries to the NAS client and carries a cookie for indicating a start address of the 101 th entry. Then, the NAS client invokes the query request (carrying the cookie returned by the NAS device) again to the NAS device, and the NAS device reads the 101 th entry based on the cookie, and returns the 101 th to 200 th entries and a new cookie (indicating the start address of the 201 st entry) to the NAS client, and so on. It should be understood that the first query request sent by the NAS client is that the cookie is not included or is null.
5, end flag: is an output parameter of the NAS device to indicate whether the directory entry of the target directory object is completely returned. When the NAS device completes reading the directory entry and sends all directory entries of the target directory object to the NAS client, this field is set, and the field of the end flag may include 1 bit, where a different value of the bit indicates whether the returned directory entry reaches the end of the directory entry, for example, when the value is 0, it indicates that the returned directory entry does not reach, and when the value is 1, it indicates that the end of the directory entry has been reached.
It will be appreciated that typically the NAS device will set this field in the query response that returns the last entry of the target directory entries. After receiving the query response, the NAS client does not initiate a read call of the same directory object to the NAS device.
6, effective data length: the output parameter is a length of valid data that is a directory entry returned by the NAS device when the directory entry is queried.
7, number of entries: is an output parameter of the NAS apparatus, which indicates the number of directory entries contained in the valid data, i.e., the number of entries.
It should be noted that the above-mentioned parameters are merely examples, and the embodiments of the present application are not limited to these parameters, and these parameters are optional, for example, the query request may not carry versions, storage area sizes, and the like, or these parameters may be configured in other ways, and are not necessarily carried in the query request.
2. A filter;
1, regular expression filter: the filtering condition including the regular expression pattern is an input parameter of the NAS device, and is used for the NAS device to filter the entries of the target directory object based on the set filtering condition of the regular expression pattern, where the filtering condition of the regular expression pattern includes, but is not limited to, a name of a regular expression.
2, attribute filter: the attribute filter condition is included and is an input parameter of the NAS device, and is used by the NAS device to filter the entry of the target directory object based on the set attribute filter condition. Attributes of a file include, but are not limited to: file size, creation time, last modified time, file type, user group, access rights, readable/writable, etc.
Illustratively, as shown in FIG. 4A, each row in the list is an attribute filter term, each attribute filter term comprising an attribute type, an attribute value, and an opcode, e.g., the attribute filter term is a file size (attribute type) greater than (opcode) 10kb (attribute value). As another example, an attribute filter condition is that the creation time (attribute type) is earlier (opcode, i.e., less) than 1991.01.01 (attribute value), and so on. Illustratively, the last row of the property filter list may be null, which marks the termination point of the property filter list.
The query conditions and the file attributes which partially support the regular expression mode are listed above, but the embodiments of the present application are not limited to this, and any query conditions which can use the regular expression and the file attributes are all applicable to the embodiments of the present application.
It should be noted that although the frame structure shows a regular expression filter and an attribute filter, in practical applications, a query request may include both, may include only a regular expression filter, or may include only an attribute filter.
3. List of operations: the operation list may include a command for instructing a specified operation, and may optionally include a corresponding parameter of the operation, a parameter length, and the like. The operations that may be performed include, but are not limited to: delete, copy, view attributes, statistics, and the like. The parameter is used to indicate an execution manner of the specified operation, for example, the specified operation is deletion, and the deletion manner includes forced deletion, demo deletion, and the like.
Optionally, the query request may further include an operation entry number, where the operation entry number is used to indicate the number of operations included in the operation list, so that the NAS device determines an end position of the operation list when parsing the operation list.
4. List of items: the valid data portion output by the NAS device may be different in data in the entry list based on whether the query request includes an operation specified by the user, for example, if the query request does not include the specified operation, the NAS device may store an entry in the target directory object that matches the filter in the entry list, that is, the entry list includes the filtered entry name. Alternatively, if the query request includes a user-specified operation, the NAS device may perform the indicated operation on the filtered entry and return an operation status, such as successful or unsuccessful operation.
It is noted that the specified operation may be one or more, such as renaming a file and modifying access rights, etc. Correspondingly, the NAS device needs to perform the one or more operations on the file corresponding to each filtered entry. If there are multiple operations specified, the order of execution may be performed in the order of the commands in the operation list.
In the above frame format, because the input parameter and the output parameter are in the same message, that is, the query request and the query response are in the same format, where the input parameter may be filled or forwarded by the NAS client, the output parameter may be filled by the NAS device, and a field is reserved for the output parameter when the NAS client generates the query request. After the NAS device fills the output parameters, the output parameters and the input parameters are returned to the NAS client side, and the NAS client side can conveniently and repeatedly call the query request.
In the present application, the query request and the query response may also be in two different frame formats, please refer to fig. 4B. Fig. 4B is a frame structure diagram of another query request and query response provided in the embodiment of the present application. Fig. 4B is different from fig. 4A in that, in fig. 4A, the query request transmitted by the NAS client and the query response transmitted by the NAS device are in the same format, that is, the frame structure shown in fig. 4A. In fig. 4B, as shown in fig. 4B (a), the query request transmitted by the NAS client includes only the input parameters (including both the input parameters and the output parameters) belonging to the NAS device. Referring to fig. 4B (B), the query response transmitted by the NAS device includes only the items belonging to the output parameters of the NAS device. As for the meaning and function of each parameter shown in fig. 4B, please refer to the description of each parameter in fig. 4A, which is not described herein again.
However, the above two frame structures are examples, and the format of the query request and the query response is not limited in the embodiment of the present application.
After generating the query request, the NAS client transmits the query request to the NAS device together with the file handle of the target directory object.
In step 304, the NAS device filters the entries of the target directory in response to the query request to obtain target entries.
After receiving the query request and the file handle of the target directory object, the NAS device locates the target directory according to the file handle, which is actually a list of the target directory, where the list includes the name of each entry in the target directory. After obtaining the list, the NAS device starts traversing the entries in the target directory one by one.
During the traversal, the NAS device applies a filter to each entry to obtain an entry that matches the filter (the entry that matches the filter is referred to as a target entry as follows). The filter herein may include a regular expression filter and/or an attribute filter, including in particular which types of filters are determined from the query request, i.e., specified by the client.
In one scenario, the query request includes a regular expression filter and an attribute filter for filtering file names. One filtering method in this scenario is: in traversing the target directory, the NAS device may first filter each directory entry using a regular expression filter to obtain one or more entries whose file names match the regular expression filter (to distinguish from the following entries that match the attribute filter, the entries that match the regular expression filter are referred to herein as first entries). Then, the NAS device obtains the attribute of each first entry, and filters each first entry one by one using the attribute filter to obtain one or more entries, that is, target entries, whose file attributes match the attribute filter.
Usually, the information of the file attribute of the entry is written into a disk for persistent storage, and when the NAS device needs to use the file attribute of the entry, the NAS device migrates the file attribute from the disk to the memory. Illustratively, the migration process may include: in the NAS device, the processor sends a read I/O for reading a file attribute to the disk, and the disk executes the read I/O to obtain a file attribute of an entry and migrate the file attribute of the entry to the memory. Because of disk execution, read I/O may also be referred to herein as disk I/O.
For example, the query request includes a regular expression filter (filter term ab), and an attribute filter (filter term 1 is file size greater than 10kb; filter term 2 is the most recent modification time after a specified time, such as 2021.06.26.11. The filtering process of the NAS device based on the query request may include: traversing the entries in the target directory object, performing filtering on each entry in the target directory using a regular expression filter, getting entries with names beginning with ab, then loading file attributes of the entries with names beginning with ab, filtering the entries with names beginning with ab using an attribute filter, and further getting entries with file sizes larger than 10kb and with latest modification times after 2021.06.26.11.
In the filtering manner, the NAS device only needs to load the file attribute for the entry matched with the regular expression, so that the disk I/O for reading the file attribute can be significantly reduced, and the reduction amount depends on the percentage of the entry matched with the regular expression filter.
In another scenario, if the query request only includes the attribute filter, the NAS device needs to load file attributes of all entries of the target directory, and then the NAS device filters each entry of the target directory one by one using the attribute filter to obtain an entry that matches the attribute filter, that is, the target entry.
This filtering method reduces the CPU overhead because it does not need to perform regular expression matching, but it requires loading the file attributes of all directory entries from the disk, which results in more disk I/O compared to the aforementioned filtering method.
Therefore, the embodiment of the application provides several management strategies for file attributes: one management strategy is that the NAS device determines file attributes to be prefetched based on a prefetching algorithm and migrates the file attributes from a disk to a memory in advance. For example, the NAS device is acquiring file attributes for an entry for directory 1, then it is likely that the file attributes for an entry for a next level directory of directory 1 will be acquired next. The embodiments of the present application do not limit the prefetching algorithm. Another management strategy is for the NAS device to determine file attributes to be prefetched based on a neural network model that can predict file attributes that are likely to be prefetched in the future based on the read records of the file attributes over a continuous time dimension. Both prefetching modes can shorten the loading time and reduce the response delay when the file attributes of a large number of entries need to be loaded. The third management strategy is that the NAS device caches the file attribute with the high heat value in the memory based on the heat value of each file attribute, and similarly, the NAS device may cache the file attribute with the high heat value in the memory based on the heat value of the entry, so as to improve the memory hit rate and reduce the latency. The hot value of the file attribute may be the access frequency of the file attribute, and the hot value may also be determined according to the access time and the access frequency of the file attribute, and any way of determining the hot value is applicable to the embodiments of the present application.
In order to further improve the query efficiency, the embodiment of the application further provides a file management policy. For example, the NAS device maintains data structures for the file system based on file names or file attributes that allow early termination when traversing directory entries or filtering directory entries, thereby improving query efficiency.
For example, a data structure, such as an LRU list, is first enumerated, the LRU list including the names of the entries and the time parameters of the entries (e.g., most recent modification time or creation time, etc.), and the entries are sorted in descending or ascending chronological order in the LRU list based on the time parameters of the entries, for example, as shown in list 1 below.
TABLE 1
Most recent modification time Item name
2019.02.01 08:03 a
2019.02.01 08:19 b
2019.02.01 08:29 c
2019.02.01 09:15 de
2019.02.01 09:20 df
…… ……
Table 1 above may be entries of the same directory, with the entries in table 1 sorted in descending order by parameter-most recently modified time. The entry names shown in table 1 above are merely illustrative to keep brevity and should not be used as a limitation on the practical application of the present application.
The NAS device may apply this table to improve efficiency when filtering entries based on time parameters. For example, if the attribute filter condition specified by the client is that the "most recent modification time" is after the specified timestamp, such as 2019.02.01. As shown in Table 1, the traversal may stop after the NAS device traverses to entry b, which matches the attribute filter criteria.
The method can be realized and applied on the NAS device side on the basis of the query method provided by the application, and can help the NAS device stop traversing before reading the whole directory entry, so that the query efficiency is improved.
From the above, if the query request contains a regular expression filter and an attribute filter, the target entry is an entry in the target directory that matches both the regular expression filter and the attribute filter. Similarly, if the query request contains only regular expression filters, the target entry is an entry that matches the regular expression filter. If the query request contains only an attribute filter, then the target entry is the entry that matches the attribute filter.
The NAS device sends a query response to the client, the query response including the target entry, step 305a. Correspondingly, the client receives the query response sent by the NAS device.
Further, if the client does not specify any operation, i.e. the query request does not carry an operation, the NAS device may directly encapsulate the target entry into a query response and send the query response to the client (see example one in fig. 5).
At step 305b, the NAS device performs one or more specified operations on the target entry.
In step 306b, the NAS device sends a query response to the client, the query response including the operation results of the one or more specified operations. Correspondingly, the client receives the query response sent by the NAS device.
If the specified operation is carried in the query request, the NAS device further performs the specified operation on the target entry, and encapsulates the operation state as a query response to be sent to the client (see example two in fig. 5).
As previously described, the operation status is used to indicate whether the specified operation was performed successfully on the target entry. Since there may be a plurality of designated operations, there may also be one or more operation states, and the operation states correspond to the designated operations one to one. Alternatively, if the designated operations all succeed or all fail, the operation status may return one indicating that the designated operations all executed successfully or all executed failed, and so on. If the specified operation fails to be executed, the client may also initiate a new query request, which is not described herein.
Thus, a file query initiated by the NAS client is completed.
In this way, in an example one, the NAS device may filter entries of the target directory, and since the number of the filtered target entries is smaller than the number of the full directory entries, the amount of data transmission may be significantly reduced, so as to improve the utilization rate of the network bandwidth. Meanwhile, the CPU overhead and disk I/O of the NAS device and the client can be reduced, the time delay is reduced, and the CPU and memory utilization rate of the NAS device and the client is improved. In example two, since the NAS device does not need to return any entry to the client, the data transfer amount can be further reduced on the basis of example one.
Through experiments, a person skilled in the art can obtain that, in a small-scale system cluster, the performance of the query method provided by the embodiment of the present application can be improved by nearly 1000 times compared with the query method introduced in the background art by applying a simulation code to operate the query method. The performance improvement is higher as the cluster is larger.
The query method provided by the embodiment of the present application is introduced by taking the find application in the UNIX and UNIX-like operating systems as an example, and the performance of directory query in the UNIX and UNIX-like operating systems can be significantly improved by the above design. Of course, the embodiments of the present application may also be applied to other operating systems, such as a windows system, a Linux system, an Android system, an ios system, a dammon system (harmony os), and a future possible operating system, and any application that can provide a directory or file query function and an operating system running the application are all suitable for the embodiments of the present application.
In order to implement the technical scheme, an ioctl interface provided by a file system can be used. The ioctl interface is an interface provided by many file systems for implementing some custom behavior.
Taking the find application as an example, the find application triggers an open () system call to read the directory entry for the user-specified pathname, and the NAS device returns a file handle to the NAS client in response to the open () system call. The FS client generates a file descriptor for the returned file handle, maintains the corresponding relation between the file handle and the file descriptor of each process, and then returns the file descriptor corresponding to the file handle to the application program. Then, the NAS client sends out an ioctl () system call to perform directory query, wherein the ioctl system call carries a query condition specified by a user and can also carry a specified operation input by the user. And then, returning the ioctl system call and the file descriptor returned by the FS client to the FS client through the VFS interface, retrieving the corresponding file handle by the FS client based on the correspondence between the file handle and the file descriptor generated before, and sending the file handle and the ioctl message to the NAS device.
As shown in fig. 5, the Find application may run one or more instances on the NAS client 10, and the Find application communicates with the file system local to the NAS client 10 via a VFS interface provided by the kernel, which provides an interface that can interact with the common file system. The Find application triggers the ioctl system call, the VFS responds to the ioctl system call and converts the ioctl system call into an ioctl command in a format corresponding to a file system in the FS client, then the FS client communicates with the RPC client, and the RPC client communicates with an RPC server running on a remote server node, so that the ioctl command is sent to the NAS device 20. The RPC server on the remote server node of the NAS device 20 communicates with the FS server, which has an implementation to execute user-specified ioctl commands. This ioctl command is the query request above. The query flow executed by the NAS device 20 based on the ioctl command is described above, and is not described here again.
In order to enable the technical scheme of the application to have better compatibility and portability, the embodiment of the application can convert a conventional command triggered by a Find application into a specific ioctl command (including a query condition and also including a specified operation), for example, the readdir and standard libc of all similar APIs are implemented to cover the existing behavior of a readdir POSIX interface. In one embodiment, the NAS client 10 may detect the instructions triggered by the application programs, and generate the ioctl command based on the instructions.
Since most existing applications use certain POSIX interfaces, such as readdir () to read directory entries, these interfaces need to be intercepted. In order to build a filter from the parameters of an application, a new library (such as libhook in fig. 6) provided by the embodiment of the present application is also needed to intercept __ libc _ start _ main () function, which is a common entry point of any application. In libc _ start _ main (), the command line parameters passed by the user can be copied to the global buffer or a filter can be built from the command line input and a global reference maintained so that it can be accessed from inside the interceptor library. When an application later calls the readdir () API, it intercepts the readdir () API, copies the filter into its buffer, and triggers the ioctl system call to pass the buffer readdir () obtained from the user. The following process can refer to the related description of fig. 5 above, and is not described here again.
In the above manner, if the local file system of the NAS client 10 does not have the function of implementing a specific ioctl command, the FS client returns an error code indicating that the specific ioctl command is not implemented in the file system, and returns to the conventional implementation of the API-based libc library to implement the directory query.
In the above, the NAS client interacts with one NAS device as an example, and the query method in the embodiment of the present application is shown, however, for a distributed NAS cluster, the embodiment of the present application further provides a query method applied to the NAS client and multiple NAS devices.
Taking fig. 1A as an example, in the query method, each directory corresponds to one master node, the master nodes corresponding to different directories may be different or the same, and the master nodes are used for recording metadata, fragment information, and the like of the directories. The master node is a NAS device 20 in the NAS cluster.
The fragmentation information includes the fragmentation of the directory and the mapping relationship between the NAS devices 20 where the fragmentation is located. For example, the mapping relationship includes an identification of the target, such as a file handle, a shard identification, an identification of the NAS device 20 in which the shard is located. The size of each fragment may be the same, and each fragment includes a partial entry under the directory, for example, a fragment may include one or a data block of the directory in fig. 1B, and may also be of other granularities, which is not limited in this application. In addition, some check information and the like can be included in the fragment.
For example, all the fragments included in one directory may be stored in multiple NAS devices, and the number of fragments of the directory stored in each NAS device may be the same or different, which is not limited in this application.
In this case, the query flow may include: the NAS client sends the path name to the master node of the target path indicated by the path name. Wherein the NAS client and the NAS device may determine the master node of the directory based on the same consistent HASH algorithm. This method is merely an example, and the embodiment of the present application is not limited to the method for determining the master node, and may also be specified or determined by querying a NAS device in the NAS cluster.
In one embodiment, the master node queries the file handle of the target directory indicated by the path name and sends the file handle of the target directory to the NAS client. The master node may also determine a number of NAS devices (e.g., referred to as fragmentation nodes) where fragments of the target directory reside, and send information of these fragmentation nodes together to the NAS client. Then, the NAS client sends the query request and the file handle to multiple sharded nodes, and each sharded node executes subsequent query processes in parallel, see steps 304 to 307.
In another embodiment, the master node may send only the file handle to the NAS client, and the NAS client sends the query request and the file handle to the master node, and the master node distributes the query request to different sharded nodes.
By the method, the query efficiency can be further improved on the basis of the query method shown in fig. 3.
The embodiment of the application also provides another data query method. Taking fig. 1A as an example, in the method, the NAS cluster includes a management node, and the management node is configured to create and manage fragmentation information of all directories of the NAS cluster, that is, in the above method, the master nodes corresponding to all directories are the same, and this node may be referred to as a management node. The management node may be any NAS device 20 in the NAS cluster.
In this case, the management node executes the operation executed by the master node, which is specifically referred to the above description, and is not described herein again.
Based on the same concept as the method embodiment, an embodiment of the present application further provides a storage device, where the storage device is configured to execute the method executed by the NAS device in the foregoing method embodiment. As shown in fig. 7, the storage device 700 includes a receiving module 701, a processing module 702, and a transmitting module 703. Specifically, in the storage device 700, the modules are connected to each other through a communication path.
In one possible embodiment, the receiving module 701 is configured to receive a query request sent by a client (e.g., a NAS client), where the query request is used to request to query a target entry satisfying a query condition under a target path;
a processing module 702, configured to respond to the query request, obtain a directory entry corresponding to the target path, and determine a target entry, where the target entry is any entry that meets the query condition under the directory entry; the client is also used for sending the target item to the client;
a sending module 703, configured to send a query response to the client, where the query response includes the target entry.
In one possible embodiment, the query condition includes one or more of the following:
specifying a file name and specifying a file attribute;
the file attributes include: file size, creation time, last modification time, file type, access rights.
In another possible embodiment, the receiving module 701 is configured to receive a query request sent by a client (e.g., a NAS client), where the query request is used to request to query a target entry satisfying a query condition under a target path;
a processing module 702, configured to respond to the query request, obtain a directory entry corresponding to the target path, and determine a target entry, where the target entry is any entry that meets the query condition under the directory entry; the client is also used for sending the target item to the client; performing the specified operation on the target entry;
a sending module 703 is configured to send a query response to the client, where the query response is used to indicate an operation result.
In one possible embodiment, the query condition includes one or more of the following:
specifying a file name and specifying a file attribute;
the file attributes include: file size, creation time, last modification time, file type, access rights.
In one possible embodiment, the specifying operation comprises one or more of:
delete operation, rename operation, modify access rights.
Similar to the above concept, as shown in fig. 8, the present application provides another storage apparatus 800, which apparatus 800 can be used to perform the steps performed by the NAS apparatus in the method shown in fig. 3.
The apparatus 800 may include a processor 801 and a memory 802. Further, the apparatus may further include a communication interface 804, which may be a transceiver, or a network card. Further, the apparatus 800 may also include a bus system 803.
The processor 801, the memory 802 and the communication interface 804 may be connected by a bus system 803, the memory 802 may store instructions, and the processor 801 may be configured to execute the instructions stored in the memory 802 to control the communication interface 804 to receive or transmit signals, so as to complete the steps executed by the NAS device in the method shown in fig. 3.
The processor 801 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), an Artificial Intelligence (AI) chip, a system on chip (SoC) or a Complex Programmable Logic Device (CPLD), a Graphics Processing Unit (GPU), or the like.
The storage 802 includes a memory and may also include a hard disk. The memory is an internal memory that directly exchanges data with the processor 801, can read and write data at any time, and is fast enough to be used as a temporary data storage for an operating system or other programs running on the processor 112. The Memory includes a volatile Memory (volatile Memory), such as a Random Access Memory (RAM), a Dynamic Random Access Memory (DRAM), and the like, and may also include a non-volatile Memory (non-volatile Memory), such as a Storage Class Memory (SCM), and the like, or a combination of a volatile Memory and a non-volatile Memory, and the like.
Hard disks, hard disks are used to provide storage resources, such as for storing data. Unlike memory, hard disks read and write data slower than memory, and are typically used to store data persistently. The hard disk may be a magnetic disk or other types of storage media, such as a solid-state drive (SSD), a mechanical hard disk (HDD), a shingled magnetic recording hard disk, or a flash memory (flash) memory.
The memory 802 may be integrated in the processor 801 or may be a different physical entity than the processor 801.
As an implementation manner, the function of the communication interface 804 may be realized by a transceiver circuit or a dedicated chip for transceiving. The processor 801 may be considered to be implemented by a dedicated processing chip, processing circuit, processor, or general purpose chip.
As another implementation manner, a manner of using a computer may be considered to implement the function of the NAS device in the embodiment shown in fig. 3 of the present application. Program code that implements the functions of processor 801 and communication interface 804 is stored in memory 802, and a general-purpose processor may implement the functions of processor 801 and communication interface 804 by executing the code in memory.
For the concepts, explanations, and detailed descriptions related to the technical solutions provided in the present application and other steps related to the apparatus 800, reference may be made to the descriptions of the foregoing methods or other embodiments, which are not repeated herein.
In an example of the present application, the apparatus 800 may be configured to perform the steps of the NAS apparatus in the flow shown in fig. 3. For example, a query request sent by a client is received through the communication interface 804, where the query request is used to request to query a target entry satisfying a query condition under a target path; the processor 801 responds to the query request, acquires the directory entry corresponding to the target path from the memory, and determines a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; then, the query response is sent to the client through the communication interface 804; the query response includes the target entry.
For example, a query request sent by a client is received through the communication interface 804, and the query request is used for requesting to query a target item meeting a query condition under a target path, and executing a specified operation on the target item; the processor 801 responds to the query request, acquires the directory entry corresponding to the target path from the memory, determines a target entry from the directory entries, and executes the specified operation on the target entry; the target entry is any entry meeting the query condition under the directory entry; then, the query response is sent to the client through the communication interface 804; the query response is used for indicating operation results, and the operation results comprise specified operation execution success or specified operation execution failure.
For descriptions of the processor 801 and the communication interface 804, reference may be made to the description of the flow illustrated in fig. 3, which is not repeated herein.
In an example of the present application, the apparatus 800 may be configured to perform the steps of the NAS client in the flow shown in fig. 3. For example, the processor generates a query request and sends the query request to the NAS device via the communication interface 804. In one implementation, the processor 801 detects query information input by a user, where the query information includes information indicating the target path and information indicating the query condition; the query request is generated based on query information.
Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application program codes, which are not specifically limited in the embodiments of the present application.
An embodiment of the present application further provides a computer storage medium, where a computer instruction is stored in the computer storage medium, and when the computer instruction runs on a storage device, the storage device executes the relevant method steps to implement the method executed by the NAS device in the foregoing embodiment, see the description of step 303 to step 308b in fig. 3, which is not described herein again, or executes the relevant method steps to implement the method executed by the NAS client in the foregoing embodiment, see the description of step 301 to step 303, step 307a, and step 308b in fig. 3, which is not described herein again.
An embodiment of the present application further provides a computer program product, and when the computer program product runs on a computer, the computer executes the relevant steps to implement the method performed by the NAS device in the foregoing embodiment, see the description of step 303 to step 308b in fig. 3, which is not described herein again, or executes the relevant method steps to implement the method performed by the NAS client in the foregoing embodiment, see the description of step 301 to step 303, step 307a, and step 308b in fig. 3, which is not described herein again.
In addition, embodiments of the present application also provide an apparatus, which may be specifically a chip, a component or a module, and may include a processor and a memory connected to each other; the memory is configured to store a computer execution instruction, and when the apparatus runs, the processor may execute the computer execution instruction stored in the memory, so as to enable the chip to execute the method executed by the NAS apparatus in each method embodiment, see the description of step 303 to step 308b in fig. 3, which is not described herein again, or execute the related method steps to implement the method executed by the NAS client in the above embodiment, see the description of step 301 to step 303, step 307a, and step 308b in fig. 3, which is not described herein again.
In addition, the storage device, the computer storage medium, the computer program product, or the chip provided in the embodiment of the present application are all used to execute the method corresponding to the NAS client or the NAS device provided above, and therefore, the beneficial effects achieved by the method may refer to the beneficial effects in the corresponding method provided above, and are not described herein again.
Through the description of the above embodiments, those skilled in the art will understand that, for convenience and simplicity of description, only the division of the above functional modules is used as an example, and in practical applications, the above function distribution may be completed by different functional modules as needed, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another apparatus, or some features may be omitted, or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units (or modules) in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributed to by the prior art, or all or part of the technical solutions may be embodied in the form of a software product, where the software product is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application program codes, which are not specifically limited in the embodiments of the present application.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device including one or more available media integrated servers, data centers, and the like. The usable medium may be a magnetic medium (e.g., a floppy Disk, a hard Disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.
The various illustrative logical units and circuits described in this application may be implemented or operated upon by design of a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in the embodiments herein may be embodied directly in hardware, in a software element executed by a processor, or in a combination of the two. The software cells may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Although the present application has been described in conjunction with specific features and embodiments thereof, it will be evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely illustrative of the present application as defined in the appended claims and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include such modifications and variations.

Claims (11)

1. A query system, comprising a client and a storage device, wherein,
the client is used for sending a query request to the storage device, wherein the query request is used for requesting to query target items meeting query conditions under a target path;
the storage device is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; the client is also used for sending the target item to the client;
the client is used for receiving the target item sent by the storage device.
2. The system of claim 1, wherein the query condition comprises one or more of:
specifying a file name and specifying a file attribute;
the file attributes include: file size, creation time, last modification time, file type, access rights.
3. The system of claim 1 or 2, wherein the method further comprises:
the client is further used for detecting query information input by a user, wherein the query information comprises information used for indicating the target path and information used for indicating the query condition; generating the query request based on the query information.
4. A query system, comprising a client and a storage device, wherein,
the client is used for sending a query request to the storage device, wherein the query request is used for requesting to query a target item meeting a query condition under a target path and executing a specified operation on the target item;
the storage device is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; performing the specified operation on the target entry;
the storage device is further used for sending a query response to the client, wherein the query response is used for indicating an operation result;
the client is further used for receiving the query response sent by the storage device.
5. The system of claim 1, wherein the specified operation comprises one or more of:
delete operation, rename operation, modify access rights.
6. A storage device, the device comprising:
the system comprises a receiving module, a query module and a query module, wherein the receiving module is used for receiving a query request sent by a client, and the query request is used for requesting to query a target item meeting a query condition under a target path;
the processing module is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; the client is also used for sending the target item to the client;
and the sending module is used for sending a query response to the client, wherein the query response comprises the target item.
7. The apparatus of claim 6, wherein the query condition comprises one or more of:
specifying a file name and specifying a file attribute;
the file attributes include: file size, creation time, last modification time, file type, access rights.
8. A memory device, the device comprising:
the system comprises a receiving module, a query module and a processing module, wherein the receiving module is used for receiving a query request sent by a client, the query request is used for requesting to query a target item meeting a query condition under a target path, and executing a specified operation on the target item;
the processing module is used for responding to the query request, acquiring the directory entry corresponding to the target path and determining a target entry, wherein the target entry is any entry meeting the query condition under the directory entry; performing the specified operation on the target entry;
and the sending module is used for sending a query response to the client, wherein the query response is used for indicating an operation result.
9. The apparatus of claim 8, wherein the specified operation comprises one or more of:
delete operation, rename operation, modify access rights.
10. A storage device, wherein the storage device comprises a processor and a memory;
the memory to store computer program instructions;
the processor executing instructions that invoke the computer program in the memory performs a method performed by the storage device of any of claims 1 to 3, or for performing a method performed by the storage device of claim 4 or 5.
11. A computer-readable storage medium, wherein the computer-readable storage medium, when executed by a computing device, performs a method performed by the storage device of any of claims 1 to 3, or for performing a method performed by the storage device of claim 4 or 5.
CN202110739776.5A 2021-06-18 2021-06-30 Query system and device Pending CN115495412A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN202131027234 2021-06-18
IN202131027234 2021-06-18

Publications (1)

Publication Number Publication Date
CN115495412A true CN115495412A (en) 2022-12-20

Family

ID=84464084

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110739776.5A Pending CN115495412A (en) 2021-06-18 2021-06-30 Query system and device

Country Status (1)

Country Link
CN (1) CN115495412A (en)

Similar Documents

Publication Publication Date Title
US10296494B2 (en) Managing a global namespace for a distributed filesystem
US10430397B2 (en) Peer to peer ownership negotiation
US9348842B2 (en) Virtualized data storage system optimizations
US9087066B2 (en) Virtual disk from network shares and file servers
US9811532B2 (en) Executing a cloud command for a distributed filesystem
US9811662B2 (en) Performing anti-virus checks for a distributed filesystem
US9804928B2 (en) Restoring an archived file in a distributed filesystem
US9852150B2 (en) Avoiding client timeouts in a distributed filesystem
US7818287B2 (en) Storage management system and method and program
JP2010097359A (en) File management method and hierarchy management file system
US10831719B2 (en) File consistency in shared storage using partial-edit files
Liu et al. Cfs: A distributed file system for large scale container platforms
CN113032335A (en) File access method, device, equipment and storage medium
CN115525602A (en) Data processing method and related device
US10387384B1 (en) Method and system for semantic metadata compression in a two-tier storage system using copy-on-write
WO2023040305A1 (en) Data backup system and apparatus
WO2023045492A1 (en) Data pre-fetching method, and computing node and storage system
CN115495412A (en) Query system and device
US10628391B1 (en) Method and system for reducing metadata overhead in a two-tier storage architecture
WO2008029146A1 (en) A distributed file system operable with a plurality of different operating systems
US20210286765A1 (en) Computer system, file storage and data transfer method
CN115878580A (en) Log management method and device
US20240119029A1 (en) Data processing method and related apparatus
US11853319B1 (en) Caching updates appended to an immutable log for handling reads to the immutable log
WO2023087231A1 (en) Directory reading system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication