CN115774695A

CN115774695A - File searching method, storage device and storage equipment

Info

Publication number: CN115774695A
Application number: CN202111038496.8A
Authority: CN
Inventors: 余仕琦
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-09-06
Filing date: 2021-09-06
Publication date: 2023-03-10

Abstract

The embodiment of the application provides a file searching method, a storage device and storage equipment. In the method, a storage node receives a first lookup request. And the storage node searches a storage state record of the target file in a memory of the storage node according to the identifier of the target file in the first search request, wherein the storage state record comprises a state identifier used for indicating that the target file is not stored in the storage system. After the storage state record of the target file is found, the storage node judges whether the storage state record is valid or not; and when the storage state record is valid, the storage node returns first indication information, and the first indication information is used for indicating that the target file does not exist. By the method, the storage node can inquire the storage state record after receiving the search request, ensure the validity of the storage state record and quickly confirm whether the file does not exist, so that the file does not need to traverse the memory of the storage node and the hard disk of the storage system, the absence of the file can be determined, and the efficiency of searching the file is improved.

Description

File searching method, storage device and storage equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a file searching method, a storage apparatus, and a storage device.

Background

With the development of transmission technology, the way of reading and writing data by an operating system is not limited to acquiring locally stored data through an Input/Output (I/O) technology, but also supports acquiring data stored in other devices through a transmission control protocol/internet protocol (TCP/IP) remote communication way. That is, data acquired by the TCP/IP technology can be regarded as an I/O technology for long-distance transmission, so that the distributed storage devices and the user operating system can be combined together in a network manner, and a storage system with larger capacity and easier expansion and contraction is formed.

In the storage system for acquiring data based on the TCP/IP communication method, at least one storage node is connected to at least one client, and the at least one storage node may be configured to store data and manage the stored data. In a storage system, after a storage node receives a file search request sent by a client, the storage node first searches metadata corresponding to a file in a memory. When the metadata does not exist in the memory, the storage node searches the metadata in the hard disk through a lookup operation (lookup), and if the metadata is not found in the file metadata stored in the hard disk of the traversal storage system, it is determined that the file does not exist.

It can be seen from the above contents that, when a file is searched, it is necessary to traverse the memory of the storage node and the metadata stored in the hard disk to determine that the file does not exist, and the efficiency is low.

Disclosure of Invention

The embodiment of the application provides a file searching method, a storage device and storage equipment, which are used for improving the efficiency of searching files.

In a first aspect, an embodiment of the present application provides a file lookup method, where the method is performed by a storage node, and in the method, the storage node receives a first lookup request, where the first lookup request includes an identifier of a target file. And searching a storage state record of the target file in a memory of the storage node by the storage node according to the identifier of the target file, wherein the storage state record comprises a state identifier for indicating that the target file is not stored in a storage system. After the storage state record of the target file is found, the storage node judges whether the storage state record is valid or not; and when the storage state record is valid, the storage node returns first indication information, wherein the first indication information is used for indicating that the target file does not exist.

In the above method, after receiving a search request sent by a client device for searching for a target file, a storage node may search in a memory for a storage status record indicating whether the target file is stored, where the storage status record may include a status identifier indicating that the target file is not stored in the storage system, and when the storage node determines that the storage status record is valid, the storage node may return first indication information indicating that the target file does not exist to the client. By the method, the storage node can inquire the storage state record stored in the memory of the storage node after receiving the search request, ensure the validity of the storage state record and quickly confirm whether the file requested by the client is a nonexistent file, so that the nonexistent file can be determined without traversing the memory of the storage node and a hard disk of a storage system, and the efficiency of searching the file is improved.

In one possible design, determining whether the storage status record is valid includes: and judging whether the storage state record is valid according to the operation time of the parent directory of the target file.

Through the design, after the storage node acquires the storage state record of the target file, whether the storage state record is effective or not can be judged according to the operation time of the parent directory of the target file. Wherein the storage status record effectively indicates that the information that the target file indicated by the storage status record is not stored in the storage system is accurate. Therefore, the accuracy of the determined result that the target file does not exist can be ensured by judging whether the storage state record is valid or not.

In one possible design, the storage status record further includes first time information of a parent directory of the target file, where the first time information is a latest access time and/or a latest modification time of the parent directory of the target file before the storage status record is generated; judging whether the storage state record is valid according to the operation time of the parent directory of the target file, wherein the judging step comprises the following steps: acquiring second time information of the parent directory of the target file, wherein the second time information is latest access time and/or latest modification time of the parent directory of the target file before the storage state record is searched; if the first time information is the same as the second time information, determining that the storage state record is valid; and if the first time information is different from the second time information, determining that the storage state record is invalid.

Through the design, the storage state record can further comprise first time information of a parent directory of the target file, and the first time information can be used for determining the validity of the storage state record, so that the accuracy of the storage state record is guaranteed.

In one possible design, before returning the first indication, the method further includes: and when the storage state record is invalid, deleting the storage state record.

Through the design, when the storage node determines that the storage state record is invalid, the storage state record can be deleted, the invalid storage state record is prevented from indicating the wrong file storage state, and the accuracy of the file query result is further ensured.

In one possible design, before the receiving the first lookup request, the method further includes: and when determining that the storage system where the storage node is located does not store the metadata of the target file, generating the storage state record in the memory.

By means of the design, the storage node generates the storage state record of the target file when the storage node determines that the target file is not stored in the storage system. That is to say, the state identifier of the target file in the storage state machine record of the target file is used to indicate that the system does not store the target file, and may indicate that the target file is not stored in both the memory of the storage node and the hard disk of the storage system. The storage state record of the target file can accurately record the non-existence state of the target file.

In one possible design, the method further includes: receiving a file creation request, wherein the file creation request is used for requesting to create the target file; and updating the second time information of the parent directory of the target file to be the time for creating the target file.

Through the design, after the storage node receives the file creation request, the second time information of the parent directory of the target file can be updated, and timeliness of the second time information for verifying the storage state record is guaranteed.

In a second aspect, an embodiment of the present application provides a storage apparatus, where the apparatus is applied to a storage node, and the apparatus includes a communication unit, a processing unit, and a storage unit;

the communication unit is used for receiving and sending data;

the storage unit is used for storing data;

the processing unit is configured to receive a first search request through the communication unit, where the first search request includes an identifier of a target file; searching a storage state record of the target file in the storage unit according to the identifier of the target file, wherein the storage state record comprises a state identifier used for indicating that the target file is not stored in a storage system; judging whether the storage state record is valid or not; and when the storage state record is valid, returning first indication information through the communication unit, wherein the first indication information is used for indicating that the target file does not exist.

In one possible design, the processing unit is specifically configured to: and judging whether the storage state record is valid according to the operation time of the parent directory of the target file.

In one possible design, the storage status record further includes first time information of a parent directory of the target file, where the first time information is a latest access time and/or a latest modification time of the parent directory of the target file before the storage status record is generated;

the processing unit is specifically configured to: acquiring second time information of the parent directory of the target file, wherein the second time information is latest access time and/or latest modification time of the parent directory of the target file before the storage state record is searched; if the first time information is the same as the second time information, determining that the storage state record is valid; and if the first time information is different from the second time information, determining that the storage state record is invalid.

In one possible design, the processing unit is further configured to: deleting the storage status record when the storage status record is invalid before returning the first indication information through the communication unit.

In one possible design, the processing unit is further configured to: generating the storage status record in the storage unit when it is determined that the storage node does not store the metadata of the target file in the storage system before receiving the first search request through the communication unit.

In one possible design, the processing unit is further configured to: receiving a file creation request for requesting creation of the target file through the communication unit; and updating the second time information of the parent directory of the target file to be the time for creating the target file.

In a third aspect, an embodiment of the present application provides a storage device, where the storage device includes: a communication module, at least one processor coupled with the at least one memory, and at least one memory for reading a computer program stored by the at least one memory to perform the steps of:

receiving a first search request through the communication module, wherein the first search request comprises an identifier of a target file; searching a storage state record of the target file in the memory according to the identifier of the target file, wherein the storage state record comprises a state identifier used for indicating that the target file is not stored in a storage system; judging whether the storage state record is valid; and when the storage state record is valid, returning first indication information through the communication module, wherein the first indication information is used for indicating that the target file does not exist.

In an optional embodiment, the processor is specifically configured to determine whether the storage status record is valid according to an operation time of the parent directory of the target file.

In an optional embodiment, the storage status record further includes first time information of the parent directory of the target file, where the first time information is a latest access time and/or a latest modification time of the parent directory of the target file before the storage status record is generated;

the processor is specifically configured to obtain second time information of the parent directory of the target file, where the second time information is latest access time and/or latest modification time of the parent directory of the target file before searching for the storage status record; if the first time information is the same as the second time information, determining that the storage state record is valid; and if the first time information is different from the second time information, determining that the storage state record is invalid.

In an alternative embodiment, the processor is further configured to delete the storage status record when the storage status record is invalid before returning the first indication information through the communication module.

In an optional embodiment, the processor is further configured to generate the storage status record in the memory when it is determined that the storage device is not in the storage system storing the metadata of the target file before receiving the first search request through the communication module.

In an optional embodiment, the processor is further configured to receive a file creation request through the communication module, where the file creation request is used to request creation of the target file; and updating the second time information of the parent directory of the target file to be the time for creating the target file.

In a fourth aspect, embodiments of the present application further provide a computer program, which, when run on a computer, causes the computer to perform the method provided in any one of the above aspects.

In a fifth aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a computer, the computer is caused to execute the method provided in any one of the above aspects.

Drawings

Fig. 1 is a schematic diagram of a distributed storage system according to an embodiment of the present application;

fig. 2 is a schematic diagram of a distributed storage system with a fully converged architecture according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a distributed storage system of a computing integrated architecture according to an embodiment of the present application;

FIG. 4 is a diagram illustrating a centralized storage system with a disk-controlled separate architecture according to an embodiment of the present application;

FIG. 5 is a diagram illustrating a centralized storage system with a disk-control integrated architecture according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a file system according to an embodiment of the present application;

fig. 7 is a schematic flowchart of a file searching method according to an embodiment of the present application;

FIG. 8 is a schematic block diagram of a memory device provided by an embodiment of the present application;

fig. 9 is a schematic structural diagram of a memory device according to an embodiment of the present disclosure.

Detailed Description

To facilitate understanding of embodiments of the present application, terms related to embodiments of the present application are described below:

metadata: the data (data about data) is mainly information describing data attributes and is used for supporting functions such as indicating storage positions, historical data, resource searching, file recording and the like. Metadata can be understood as an electronic catalog that describes and stores the content or attributes of data for the purpose of cataloguing to facilitate data retrieval.

It should be noted that, in the present application, unless otherwise stated, metadata generally refers to file metadata, and the file metadata may include a directory entry (entry) and an index node (inode). The entry includes semantic information of the file, such as file identifier, file name, and the like. The inode includes attribute information of the file, such as file size, file authority, and the like.

It is to be understood that in this application, "/" indicates an OR meaning, e.g., A/B may indicate either A or B; in the present application, "and/or" is only an association relationship describing an association object, and means that there may be three relationships, for example, a and/or B, and may mean: a exists alone, A and B exist simultaneously, and B exists alone. "at least one" means one or more, "a plurality" means two or more.

In this application, "exemplary," "in some embodiments," "in other embodiments," and the like are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the word using examples is intended to present concepts in a concrete fashion.

Furthermore, the terms "first," "second," and the like, as used herein, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicit to any number of technical features indicated, or to indicate or imply an order.

The embodiments of the present application will be described in detail below with reference to the drawings attached hereto. It is to be understood that the terminology used in the description of the embodiments section of the present application is for the purpose of describing particular embodiments of the present application only and is not intended to be limiting of the present application.

Fig. 1 is a schematic diagram of a distributed storage system according to an embodiment of the present application. As shown in fig. 1, the distributed storage system refers to a system that stores data in a distributed manner on a plurality of independent storage nodes. The traditional network storage system adopts a centralized storage array to store all data, the performance of the storage array is not only the bottleneck of the system performance, but also the focus of reliability and safety, and the requirement of large-scale storage application cannot be met. The distributed network storage system adopts an expandable system structure, and utilizes a plurality of storage nodes to share the storage load, thereby not only improving the reliability, the availability and the access efficiency of the system, but also being easy to expand.

Referring to fig. 1, a distributed storage system includes a plurality of clients and a plurality of storage nodes. For example, fig. 1 shows a client a to a client E and a storage node 1 to a storage node 4 as examples. The client a can access the storage node 1 or the storage node 3 to read the stored data, the client B can access the storage node 2 and the storage node 3 to read the stored data, the client C can access the storage node 1 or the storage node 4 to read the stored data, the client D can access the storage node 2 to read the stored data, and the client E can access the storage node 4 to read the stored data.

Among them, the client may be a terminal device such as a mobile phone, a tablet computer, a wearable device (e.g., a watch, a bracelet, a helmet, an earphone, etc.), an in-vehicle device, an Augmented Reality (AR)/Virtual Reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), a smart home device (e.g., a smart television, a smart speaker, a smart camera, etc.).

Fig. 2 is a schematic diagram of a distributed storage system with a fully converged architecture according to an embodiment of the present application. In the storage system, each server may serve as a storage node in the distributed storage system shown in fig. 1, and a plurality of servers form a server cluster. Referring to fig. 2, the server cluster includes one or more application servers 110 (three servers 110 are shown in fig. 2, but not limited to two servers 110), and the servers 110 may communicate with each other. Server 110 is a device, such as a server, desktop computer, etc., that has both computing and storage capabilities. In software, each server 110 has an operating system thereon. The virtual machine 107 may be created on the server 110, and the computing resources required by the virtual machine 107 may be sourced from the processor 112 and the memory 113 local to the server 110, while the storage resources required by the virtual machine 107 may be sourced from the hard disk 105 local to the server 110, or may be sourced from the hard disk 105 in another server 110. In addition, various applications may be running in the virtual machine 107, and a user may trigger a read/write data request through the applications in the virtual machine 107.

In hardware, as shown in fig. 2, the server 110 includes at least a processor 112, a memory 113, a network card 114, and a hard disk 105. The processor 112, the memory 113, the network card 114 and the hard disk 105 are connected through a bus. The processor 112 and the memory 113 are used to provide computing resources, among other things. Specifically, processor 112 is a Central Processing Unit (CPU) for processing data access requests from outside server 110 or requests generated internally within server 110. For example, when the processor 112 receives a write data request sent by a user, the data in the write data request is temporarily stored in the memory 113. When the total amount of data in the memory 113 reaches a certain threshold, the processor 112 sends the data stored in the memory 113 to the hard disk 105 for persistent storage. In addition, the processor 112 is used for data calculation or processing, such as metadata management, data de-duplication, data compression, data verification, virtualized storage space, address translation, and the like. Only one CPU 112 is shown in fig. 2, and in practical applications, the number of CPUs 112 is often multiple, wherein one CPU 112 has one or more CPU cores. The number of CPUs and the number of CPU cores are not limited in this embodiment.

The memory 113 is an internal memory for directly exchanging data with the processor, and it can read and write data at any time, and it is fast, and it is used as a temporary data storage for an operating system or other programs in operation. The Memory includes at least two types of Memory, for example, the Memory may be a random access Memory (ram) or a Read Only Memory (ROM). The Random Access Memory is, for example, a Dynamic Random Access Memory (DRAM) or a Storage Class Memory (SCM). DRAM is a semiconductor Memory, and belongs to a volatile Memory (volatile Memory) device, like most Random Access Memories (RAMs). SCM is a hybrid storage technology that combines the features of both traditional storage devices and memory, memory-class memory providing faster read and write speeds than hard disks, but slower access speeds and lower cost than DRAM. However, the DRAM and the SCM are only exemplary in this embodiment, and the Memory may also include other Random Access memories, such as Static Random Access Memory (SRAM), and the like. As the rom, for example, a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), and the like may be used. In addition, the Memory 113 may also be a Dual In-line Memory Module (Dual In-line Memory Module, DIMM for short), that is, a Module composed of a Dynamic Random Access Memory (DRAM), or a Solid State Disk (SSD). In practical applications, the server 110 may be configured with a plurality of memories 113 and different types of memories 113. The number and type of the memories 113 are not limited in this embodiment. In addition, the memory 113 may be configured to have a power conservation function. The power saving function means that when the system is powered off and powered on again, the data stored in the memory 113 will not be lost. A memory having a power saving function is called a nonvolatile memory.

The hard disk 105 is used to provide storage resources, such as storing data. It may be a magnetic disk or other type of storage medium such as a solid state disk or a shingled magnetic recording disk. The network card 114 is used to communicate with other application servers 110.

Fig. 3 is a schematic diagram of a distributed storage system of a storage and computation integrated architecture according to an embodiment of the present application. Referring to fig. 3, in the storage system, each server 110 may serve as a storage node in the distributed storage system shown in fig. 1, and a plurality of servers 110 constitute a server cluster. Server 110 includes at least processor 112, memory 113, network card 114, and hard disk 105. The processor 112, the memory 113, the network card 114 and the hard disk 105 are connected through a bus.

It is to be understood that, in the embodiment of the present application, the distributed storage system may also be a storage separation architecture, that is, the computing function and the storage function of each storage node may be implemented by different devices, and a device for implementing the computing function may access any device for implementing the storage function through a network.

In addition, the embodiment of the application also provides a centralized storage system. The centralized storage system is characterized in that one or more main devices form a central storage node, data is stored in the central storage node in a centralized manner, and all data processing services of the whole system are deployed on the central storage node in a centralized manner. In other words, in the centralized storage system, the client is only responsible for the input and output of data, and the storage and control processing of data are completely finished by the central storage node. The centralized system has the greatest characteristic of simple deployment structure, and does not need to consider how to deploy a plurality of nodes for the service, namely, the problem of distributed cooperation among the nodes.

For example, fig. 4 is a schematic diagram of a centralized storage system with a disk control separated architecture according to an embodiment of the present application. In a centralized storage system, clients may access data through applications, and the computers that support the operation of the applications may be referred to as "application servers". Referring to fig. 4, the application server 400 may be a physical machine or a virtual machine. Physical application servers include, but are not limited to, desktop computers, servers, laptops, and mobile devices. The application servers access the storage system 420 through the fabric switch 410 to access data. However, the switch 410 is only an optional device, and the application server 400 may also communicate directly with the storage system 420 through a network. Alternatively, the fabric switch 410 may be replaced with an Ethernet switch, an InfiniBand switch, a RoCE (RDMA over switched Ethernet) switch, or the like.

Referring to fig. 4, the centralized storage system is characterized by a uniform entry through which all data from external devices pass, which is the engine 421 of the centralized storage system. The engine 421 is the most central component in a centralized storage system, in which the high-level functions of many storage systems are implemented.

As shown in fig. 4, there are one or more controllers in the engine 421, and fig. 4 illustrates an example where the engine includes two controllers. If a mirror channel is provided between controller 0 and controller 1, then after controller 0 writes a copy of data into its memory 424, controller 1 may send a copy of the data to controller 1 via the mirror channel, and controller 1 stores the copy in its local memory 424. Therefore, the controller 0 and the controller 1 are backup to each other, when the controller 0 fails, the controller 1 can manage the service of the controller 0, and when the controller 1 fails, the controller 0 can manage the service of the controller 1, thereby avoiding the unavailability of the whole storage system 420 caused by hardware failure. When 4 controllers are deployed in the engine 421, any two controllers have mirror channels therebetween, so that any two controllers are backup to each other.

The engine 421 also contains a front-end interface 425 and a back-end interface 426, where the front-end interface 425 is used to communicate with the application server 400 to provide storage services for the application server 400. And a back-end interface 426 for communicating with the hard disk 434 to expand the capacity of the storage system. Through the backend interface 426, the engine 421 can connect more hard disks 434, thereby forming a very large pool of storage resources.

In hardware, as shown in fig. 4, the controller 0 includes at least a processor 423 and a memory 424. Processor 423 is a Central Processing Unit (CPU) that processes data access requests from outside the storage system (server or other storage system) as well as requests generated internally within the storage system. Illustratively, when the processor 423 receives a write data request from the application server 400 through the front port 425, the data in the write data request is temporarily stored in the memory 424. When the total amount of data in the memory 424 reaches a certain threshold, the processor 423 sends the data stored in the memory 424 to the hard disk 434 through the back-end port for persistent storage.

The memory 424 is internal storage that exchanges data directly with the processor, can read and write data at any time, and is fast enough to serve as temporary data storage for an operating system or other programs that are running. The Memory includes at least two types of Memory, for example, the Memory may be a random access Memory (ram) or a Read Only Memory (ROM). For example, the Random Access Memory is a Dynamic Random Access Memory (DRAM) or a Storage Class Memory (SCM). DRAM is a semiconductor Memory, and belongs to a volatile Memory (volatile Memory) device, like most Random Access Memories (RAMs). SCM is a hybrid storage technology that combines the features of both traditional storage devices and memory, memory-class memory providing faster read and write speeds than hard disks, but slower access speeds and lower cost than DRAM. However, the DRAM and the SCM are only exemplary in this embodiment, and the Memory may also include other Random Access memories, such as Static Random Access Memory (SRAM), and the like. As the rom, for example, a Programmable Read Only Memory (PROM), an Erasable Programmable Read Only Memory (EPROM), and the like can be used. In addition, the Memory 424 may also be a Dual In-line Memory Module (DIMM), a Module composed of Dynamic Random Access Memory (DRAM), or a Solid State Disk (SSD). In practical applications, a plurality of memories 424 and different types of memories 424 may be configured in the controller 0. The number and type of the memories 424 are not limited in this embodiment. In addition, the memory 424 may be configured to have power conservation. The power conservation function means that when the system is powered down and powered up again, the data stored in the memory 424 will not be lost. A memory having a power saving function is called a nonvolatile memory.

The memory 424 stores software programs, and the processor 423 runs the software programs in the memory 424 to manage the hard disk. For example, the hard disk is abstracted into a storage resource pool, and then divided into LUNs for use by the server. The LUN here is in fact the hard disk seen on the server. Of course, some centralized storage systems are themselves file servers, and may provide shared file services for the servers.

The hardware components and software structure of controller 1 (and other controllers not shown in fig. 1) are similar to controller 0 and will not be described again.

In the centralized storage system with separate disk control shown in fig. 4, the engine 421 may not have a hard disk slot, the hard disk 434 needs to be placed in the hard disk frame 430, and the backend interface 426 communicates with the hard disk frame 430. The backend interface 426 is in the form of an adapter card in the engine 421, and two or more backend interfaces 426 can be used simultaneously on one engine 421 to connect multiple hard disk frames. Alternatively, the adapter card may be integrated on the motherboard, and in this case, the adapter card may communicate with the engine 421 through the PCIE bus.

It should be noted that only one engine 421 is shown in fig. 4, however, in practical applications, the storage system may include two or more engines 421, and redundancy or load balancing is performed among the engines 421.

The hard disk frame 430 includes a control unit 431 and several hard disks 434. The control unit 431 may have various forms. There is no affiliation between the network card 404 and the hard disk 434 in the hard disk frame 430, and the network card 404 can access any hard disk 434 in the hard disk frame 430, so it is convenient to expand the hard disk when the storage space is insufficient.

Hard disk frame 430 may be a SAS hard disk frame, or may be a NVMe hard disk frame, an IP hard disk frame, and other types of hard disk frames, depending on the type of communication protocol between engine 421 and hard disk frame 430. SAS hard disk frames adopt SAS3.0 protocol, and each frame supports 25 SAS hard disks. Engine 421 interfaces with hard disk frame 430 through an onboard SAS interface or SAS interface module. The NVMe hard disk frame is more like a complete computer system, and the NVMe hard disk is inserted into the NVMe hard disk frame. The NVMe hard disk box is in turn connected to the engine 421 through an RDMA port.

In other embodiments, the present application further provides a centralized storage system with a disk control integrated architecture. Fig. 5 is a schematic diagram of a centralized storage system with a unified disk control architecture according to an embodiment of the present disclosure. Referring to fig. 5, when the backend interface 426 of the engine 421 in the storage system 420 is directly connected to the plurality of hard disks 434, the storage system 420 in the integrated disk control architecture does not include a control unit, compared to the storage system 420 in the separate disk control architecture shown in fig. 4.

In addition, in the centralized storage system with integrated disk control shown in fig. 5, the engine 421 has a hard disk slot, the hard disk 434 may be directly disposed in the engine 421, the backend interface 426 belongs to an optional configuration, and when the storage space of the system is insufficient, more hard disks or hard disk frames may be connected through the backend interface 426.

The storage systems shown in fig. 2-5 have file systems. For example, fig. 2-3 illustrate a distributed storage system, which provides a uniform access interface to the outside, and a host can use the distributed storage system illustrated in fig. 2-3 as accessing a normal file system. . Likewise, the centralized storage system shown in FIG. 4 or FIG. 5 may also include a file system. The file system can provide metadata service processing for the outside. Optionally, the metadata service processing provided externally may include, for example, updating content of the metadata cache, receiving a file search request or a metadata request sent by a client, performing a metadata operation on metadata according to a file operation command, and the like. The following describes a file system provided in an embodiment of the present application.

Fig. 6 is a schematic diagram of a file system according to an embodiment of the present application. The file system is a structured data file storage and organization. As we know, all data in a computer are 0 and 1, and a series of 01 combinations stored on a hardware medium is completely impossible for us to distinguish and manage. Therefore, the concept of 'files' is used for organizing the data, the data used for the same purpose are composed into different types of files according to the structural mode required by different application programs. Different types are often referred to with different suffixes, and then each file is given a name that is convenient for understanding the memory. When there are many files, the files can be grouped according to some dividing method, and each group of files is placed in the same directory (or called folder). Besides, the following directories may also have next level directories (called sub-directories or sub-folders) besides files, all files and directories form a tree structure, which can be shown in fig. 6, and the tree structure has a specific name: file System (File System). The file system has many types, and common FAT/FAT32/NTFS of Windows, EXT2/EXT3/EXT4/XFS/BtrFS of Linux and the like. For the convenience of searching, the names of the directories, sub-directories and files are spliced together by using special characters (such as "\" for Windows/DOS and "/" -for Unix-like systems) from the root node to the files themselves, and such a string of characters is called a file path, such as "/etc/System/system.conf" in Linux or "C:" Windows System32\ taskmtr.exe "in Windows. A path is a unique identification for accessing a particular file. For example, D: \ data \ file.exe under Windows is a path of a file, which represents the file.exe file under the data directory under the D partition.

Optionally, the file system may also store metadata of directories or files. In one possible design, the metadata of the directory or file is stored using a directory entry (dentry) and an index node (inode), where the dentry is used to store information such as a directory name or a file name, and the inode is used to store information such as a basic attribute and a permission attribute of the directory or file. The basic attribute refers to information such as file size, access time, and file type of the file, and the permission attribute refers to Access Control List (ACL) permission, user group, and the like.

For convenience of description, in the following of the present application, the storage system may be any distributed storage system or centralized storage system having a file system. The storage system may include at least one storage node, which may be used to implement the functionality of a file system. When the storage system is the distributed storage system shown in fig. 2 or fig. 3, the storage node may be any server 110 in fig. 2 or fig. 3. When the storage system is a centralized storage system as shown in fig. 4 or fig. 5, the storage node may be the storage system 420 in fig. 4 or fig. 5.

In the storage system, any client may send a file search request to the storage node, where the file search request is used to request to search for a target file. After receiving the file searching request, the storage node may search the storage system for the metadata of the target file. Specifically, the storage node may first search for whether the metadata of the target file exists in the metadata cached in the memory. And if the storage node does not find the metadata of the target file in the memory, the storage node finds the metadata of the target file from a hard disk of the storage system. If the metadata of the target file is not found by traversing the metadata stored in the hard disk, the target file can be determined to be absent. At this time, the storage node may send indication information indicating that the target file is not searched to the client.

As can be seen from the above, the storage node needs to traverse the metadata stored in the memory and the hard disk to determine that the file does not exist, which is inefficient.

In another implementation, the storage node may cache the metadata of all files in the memory of the storage node. After receiving a file search request sent by a client, the storage node can traverse metadata of files cached in a memory. If the metadata of the target file is not found, it can be determined that the target file does not exist, but the memory overhead of the storage node in this way is large.

Therefore, the current file searching method has low efficiency or large expense of the memory of the storage node.

Based on the above problems, embodiments of the present application provide a file searching method, so as to improve the efficiency of searching for a file.

Fig. 7 is a flowchart illustrating a file searching method according to an embodiment of the present application. The method may be performed by a storage node. The storage node may be any of the servers 110 in the distributed storage system shown in fig. 2 or fig. 3, or the storage node may be the storage system 420 in the centralized storage system shown in fig. 4 or fig. 5. Referring to fig. 7, the method includes the steps of:

s701: the client sends a search request to the storage node.

The searching request is used for searching the target file, and the searching request comprises the identification of the target file. Alternatively, the identification of the target file may be the dentry in the metadata of the target file.

S702: and the storage node searches the metadata of the target file in the storage system according to the identifier of the target file in the search request, and determines that the metadata of the target file is not stored in the storage system.

In an optional implementation manner, when the storage node receives a search request sent by a client, the storage node may search the storage system for the metadata of the target file. Specifically, the storage node may first search for whether metadata of the target file exists in metadata cached in a memory of the storage node, and when the metadata of the target file is not stored in the memory, the storage node may search for the metadata of the target file in a hard disk of the storage system through a lookup (lookup) operation. If the storage node does not find the metadata of the target file in the memory and the hard disk of the storage node, it may be determined that the target file is not stored in the current storage system, that is, the target file does not exist.

It can be understood that, if the storage system stores the target file, the storage node may find the metadata of the target file when searching for the target file according to the identifier of the target file, where the metadata of the target file includes the identifier of the target file. For example, the metadata of the target file includes dentry and inode, wherein dentry can be used as the identification of the target file.

S703: and the storage node generates a storage state record of the target file.

When determining that the target file does not exist, the storage node may generate a storage status record of the target file, and record, in the storage status record, a status identifier that the target file is not stored in the storage system. After the storage node generates the storage state record of the target file, the storage node may store the storage state record of the target file in the memory of the storage node.

Optionally, the storage status record may further include an identifier of the target file and first time information of a parent directory of the target file. The first time information is the latest access time and/or latest modification time of the parent directory of the target file before the storage state record is generated. When the storage node uses the storage state record of the target file, whether the first time information is the same as the access time and/or the modification time of the current parent directory of the target file or not can be checked, if the first time information is the same as the access time and/or the modification time of the current parent directory of the target file, the parent directory of the target file is not changed, and the storage state record is effective at the moment, so that the absence of the target file can be accurately indicated; if the target file is different from the target file, the parent directory of the target file is changed, and the storage state record of the target file is invalid and cannot be used for indicating that the target file does not exist. That is to say, the first time information in the storage status record can be used to verify the validity of the storage status record, and the accuracy of the storage status record is ensured.

It should be noted that the parent directory of the target file is the last directory in the path for searching the target file. For example, when the client searches for the file C, the client first sends a search request to search for the directory B under the directory a, and the storage node searches for the directory B according to the search request and feeds back the directory B to the client. The client side continues to send a search request for searching the file C under the directory B, and the directory B can be regarded as a parent directory of the file C.

As can be seen, in the file searching method provided in the embodiment of the present application, after receiving the search request, if the target file exists, the storage node may search the metadata of the target file in the memory or the hard disk of the storage node; if the target file does not exist, the storage state record of the target file can be found in the memory of the storage node, and the storage state record comprises a state identifier for indicating that the target file is not stored in the storage system.

In this way, the storage node generates a storage status record of the target file when determining that the target file is not stored in the storage system. That is to say, the state identifier of the target file in the storage state machine record of the target file is used to indicate that the system does not store the target file, which indicates that neither the memory of the storage node nor the hard disk of the storage system stores the target file. The storage state record of the target file can accurately record the non-existence state of the target file. In addition, the storage state record in the embodiment of the application may further include first time information of a parent directory of the target file, which is used to determine validity of the storage state record, and further ensure accuracy of the storage state record.

S704: and the storage node sends indication information to the client.

Wherein the indication information is used for indicating that the target file does not exist.

S705: the client sends a first search request to the storage node.

The first search request is used for searching for the target file, and the first search request comprises an identifier of the target file and an identifier of a parent directory of the target file.

S706: and the storage node searches the storage state record of the target file in the memory of the storage node according to the identifier of the target file in the first search request.

As can be seen from S703, the storage status record generated by the storage node may include an identifier of the target file, a status identifier of the target file, and first time information of the parent directory of the target file.

S707: the storage node judges whether the storage state record of the target file is valid or not; if yes, entering S709; if not, the process proceeds to S710.

In an alternative embodiment, the storage node may determine whether the storage status record is valid according to the operation time of the parent directory of the target file.

In some embodiments, the storage node may obtain second time information of the parent directory of the target file, where the second time information is a latest access time and/or a latest modification time of the parent directory of the target file before searching for the storage status record. If the first time information is the same as the second time information and indicates that the file in the parent directory of the target file is not changed after the storage state record of the target file is generated, the storage node can determine that the storage state record is valid and can use the storage state record; conversely, if the first time information is different from the second time information and indicates that the file in the parent directory of the target file has changed and the target file may already exist after the storage status record of the target file is generated, the storage node may determine that the storage status record is invalid.

S708: the storage node sends first indication information to the client.

Wherein the first indication information is used for indicating that the target file does not exist.

S709: and the storage node deletes the storage state record of the target file from the memory.

In an optional implementation manner, if the storage node determines that the storage status record of the target file is invalid, the storage status record of the target file cannot indicate that the target file is not stored in the storage system, and the storage node needs to delete the storage status record of the target file from the memory.

Further, after the storage node deletes the storage state record of the target file, the storage node may search for the metadata of the target file in the storage system, and when the storage node determines that the metadata of the target file is not stored in the storage system, it may determine that the target file does not exist, and at this time, the storage node may send the first indication information to the client.

S710: the client sends a file creation request to the storage node.

Wherein the file creation request is for requesting creation of a target file.

S711: and the storage node creates the target file according to the file creation request, and updates the second time information of the parent directory of the target file to the time for creating the target file.

It is to be understood that, when the storage node creates the target file, the access time/modification time of the parent directory of the target file may be changed, and the second time information may be updated to the time when the target file is created.

In some embodiments, the second time information is updated after the storage node creates the target file according to the file creation request. If the storage node receives the search request sent by the client at this time, and after the storage node searches the storage state record of the target file, it can be determined that the first time information in the storage state record is different from the second time information of the parent directory of the target file, and then the storage node can delete the storage state record of the target file. Further, the storage node may find the target file in the storage system, and then the storage node may send the target file or the metadata of the target file to the client.

By the file searching method provided by the embodiment of the application, the memory of the storage node can store the storage state records of the files which are not stored in the system, that is, the memory of the storage node is not only used for storing the metadata of the currently existing files, but also can store the storage state records of the currently unavailable files. After receiving a search request sent by a client device for searching for a target file, a storage node may search in a memory whether a storage state record of the target file is stored, where the storage state record may include a state identifier for indicating that the target file is not stored in the storage system, and when the storage node determines that the storage state record is valid, the storage node may return first indication information for indicating that the target file does not exist to the client. By the method, the storage node can quickly inquire whether the file requested by the client is a nonexistent file or not after receiving the search request, so that the nonexistent file can be determined without traversing the memory of the storage node and the hard disk of the storage system, and the efficiency of searching the file is improved.

It should be noted that, in the embodiment of the present application, the search request sent by the client may also be used to search directories, as can be seen from the foregoing description of the file system shown in fig. 6, the directories are used to store files, and one directory may have a next-level directory in addition to the files. Similar to the method for searching for a target file provided in the embodiment of the present application, the storage node may also store a storage status record of the directory, which may be specifically implemented with reference to the file searching method shown in fig. 7, and details are not described here.

The following describes an apparatus for implementing the above method in the embodiment of the present application with reference to the drawings. Therefore, the above contents can be used in the following embodiments, and the repeated contents are not described again.

Fig. 8 is a schematic block diagram of a memory device 800 according to an embodiment of the present application. The storage apparatus 800 may be used to implement the functions of any server 110 in the distributed storage system shown in fig. 2 or fig. 3, or the storage apparatus 800 may be used to implement the functions of the storage system 420 in the centralized storage system shown in fig. 4 or fig. 5. Referring to fig. 8, the storage apparatus 800 includes a communication unit 801, a processing unit 802, and a storage unit 803.

For example, processing unit 802 may be configured to perform all operations performed by a storage node in the embodiment shown in FIG. 7, except for transceiving operations, such as operations to look up a storage status record of a target file, and/or other processes to support the techniques described herein. The communication unit 801 may be used to perform all transceiving operations performed by a storage node in the embodiment illustrated in fig. 7, e.g., S701, S704, and/or other processes for supporting the techniques described herein. The storage unit 803 may be used to store files, metadata of the files, and storage status records of the files.

Optionally, the communication unit 801 may include a receiving unit and a sending unit, where the receiving unit is configured to execute all receiving operations executed by the storage node in the embodiment of the present application, and the sending unit is configured to execute all sending operations executed by the storage node in the embodiment of the present application.

Optionally, the processing unit is configured to receive a first search request through the communication unit 801, where the first search request includes an identifier of a target file; according to the identifier of the target file, searching a storage status record of the target file in the storage unit 803, where the storage status record includes a status identifier used to indicate that the target file is not stored in the storage system; judging whether the storage state record is valid or not; when the storage status record is valid, first indication information is returned through the communication unit 801, where the first indication information is used to indicate that the target file does not exist.

In an optional implementation manner, the processing unit 802 is specifically configured to determine whether the storage status record is valid according to an operation time of the parent directory of the target file.

the processing unit 802 is specifically configured to obtain second time information of the parent directory of the target file, where the second time information is latest access time and/or latest modification time of the parent directory of the target file before the storage status record is searched; if the first time information is the same as the second time information, determining that the storage state record is valid; and if the first time information is different from the second time information, determining that the storage state record is invalid.

In an alternative embodiment, the processing unit 802 is further configured to delete the storage status record when the storage status record is invalid before the first indication information is returned through the communication unit 801.

In an alternative embodiment, the processing unit 802 is further configured to, before receiving the first search request through the communication unit 801, generate the storage status record in the storage unit 803 when determining that the storage node is in the storage system without storing the metadata of the target file.

In an optional embodiment, the processing unit 802 is further configured to receive a file creation request through the communication unit 801, where the file creation request is used to request creation of the target file; and updating the second time information of the parent directory of the target file to be the time for creating the target file.

The embodiment of the present application further provides a storage device 900, and the storage device 900 may be used to implement the functions of the storage node in the embodiment shown in fig. 7. Fig. 9 is a schematic structural diagram of a storage device 900 according to an embodiment of the present application. Referring to fig. 9, the storage device 900 includes: a communication module 901, a processor 902 and a memory 903.

The communication module 901, the processor 902 and the memory 903 are coupled to each other. In fig. 9, the communication module 901, the processor 902, and the memory 903 are connected by a bus 904, which is indicated by a thick line in fig. 9. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 9, but this does not indicate only one bus or one type of bus.

The communication module 901 is configured to receive and send data, so as to implement communication interaction with other devices. For example, when the communication apparatus 900 is applied to a storage node in the embodiment shown in fig. 7, the communication module 901 may be implemented by a physical interface, a communication interface, and an input/output interface.

The memory 903 is used to store instructions and the processor 902 is used to execute the instructions stored by the memory 903. The memory 903 stores program code, and the processor 902 may call the program code stored in the memory 903 to perform the following operations:

receiving a first search request through the communication module 901, where the first search request includes an identifier of a target file; searching a storage state record of the target file in the memory 903 according to the identifier of the target file, wherein the storage state record comprises a state identifier for indicating that the target file is not stored in a storage system; judging whether the storage state record is valid or not; when the storage status record is valid, first indication information is returned through the communication module 901, where the first indication information is used to indicate that the target file does not exist.

In an alternative embodiment, the processor 902 is specifically configured to determine whether the storage status record is valid according to an operation time of the parent directory of the target file.

the processor 902 is specifically configured to obtain second time information of the parent directory of the target file, where the second time information is latest access time and/or latest modification time of the parent directory of the target file before the storage status record is searched; if the first time information is the same as the second time information, determining that the storage state record is valid; and if the first time information is different from the second time information, determining that the storage state record is invalid.

In an alternative embodiment, the processor 902 is further configured to delete the storage status record when the storage status record is invalid before the first indication information is returned through the communication module 901.

In an optional embodiment, the processor 902 is further configured to generate the storage status record in the memory 903 when it is determined that the storage node is located in the storage system without storing the metadata of the target file before receiving the first search request through the communication module 901.

In an optional embodiment, the processor 902 is further configured to receive a file creation request through the communication module 901, where the file creation request is used to request creation of the target file; and updating the second time information of the parent directory of the target file to be the time for creating the target file.

It will be appreciated that the memory 903 in FIG. 9 can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double Data Rate Synchronous Dynamic random access memory (DDR SDRAM), enhanced Synchronous SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Also provided in embodiments of the present application is a computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method of the above-described embodiments.

Also provided in embodiments of the present application is a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the above-described embodiments.

The embodiment of the application also provides a chip, and the logic in the chip is used for executing the method of the embodiment.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by instructions. These instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A file lookup method performed by a storage node, the method comprising:

receiving a first search request, wherein the first search request comprises an identifier of a target file;

searching a storage state record of the target file in a memory of the storage node according to the identifier of the target file, wherein the storage state record comprises a state identifier used for indicating that the target file is not stored in a storage system;

judging whether the storage state record is valid or not;

and returning first indication information when the storage state record is valid, wherein the first indication information is used for indicating that the target file does not exist.

2. The method of claim 1, wherein determining whether the storage status record is valid comprises:

and judging whether the storage state record is valid according to the operation time of the parent directory of the target file.

3. The method of claim 2, further comprising first time information of a parent directory of the target file in the storage status record, the first time information being a latest access time and/or a latest modification time of the parent directory of the target file before the storage status record is generated;

judging whether the storage state record is valid according to the operation time of the parent directory of the target file, wherein the judging step comprises the following steps:

acquiring second time information of the parent directory of the target file, wherein the second time information is latest access time and/or latest modification time of the parent directory of the target file before searching the storage state record;

if the first time information is the same as the second time information, determining that the storage state record is valid;

and if the first time information is different from the second time information, determining that the storage state record is invalid.

4. The method of claim 3, prior to returning the first indication, further comprising:

and when the storage state record is invalid, deleting the storage state record.

5. The method of any of claims 1 to 4, wherein prior to said receiving the first lookup request, the method further comprises:

and when determining that the storage system where the storage node is located does not store the metadata of the target file, generating the storage state record in the memory.

6. The method of any of claims 1 to 5, further comprising:

receiving a file creation request, wherein the file creation request is used for requesting to create the target file;

and updating the second time information of the parent directory of the target file to be the time for creating the target file.

7. A storage device is applied to a storage node and comprises a communication unit, a processing unit and a storage unit;

the communication unit is used for receiving and sending data;

the storage unit is used for storing data;

the processing unit is configured to:

receiving a first search request through the communication unit, wherein the first search request comprises an identifier of a target file; searching a storage state record of the target file in the storage unit according to the identifier of the target file, wherein the storage state record comprises a state identifier used for indicating that the target file is not stored in a storage system;

judging whether the storage state record is valid;

and when the storage state record is valid, returning first indication information through the communication unit, wherein the first indication information is used for indicating that the target file does not exist.

8. The apparatus as claimed in claim 7, wherein said processing unit is specifically configured to:

9. The apparatus of claim 8, wherein the storage status record further includes first time information of a parent directory of the target file, the first time information being a latest access time and/or a latest modification time of the parent directory of the target file before the storage status record is generated;

the processing unit is specifically configured to:

acquiring second time information of the parent directory of the target file, wherein the second time information is latest access time and/or latest modification time of the parent directory of the target file before the storage state record is searched;

10. The apparatus as recited in claim 9, said processing unit to further:

deleting the storage status record when the storage status record is invalid before returning the first indication information through the communication unit.

11. The apparatus of any of claims 7 to 10, wherein the processing unit is further to:

generating the storage status record in the storage unit when it is determined that the storage node does not store the metadata of the target file in the storage system before receiving the first search request through the communication unit.

12. The apparatus of any of claims 7 to 11, wherein the processing unit is further to:

receiving a file creation request through the communication unit, the file creation request requesting creation of the target file;

13. A storage device, comprising a communication module, at least one processor coupled with the at least one memory, and at least one memory to read a computer program stored by the at least one memory to perform the method of any one of claims 1-6.

14. A computer-readable storage medium having stored therein instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1-6.