CN112579528B - Method for efficiently accessing files at server side of embedded network file system - Google Patents
Method for efficiently accessing files at server side of embedded network file system Download PDFInfo
- Publication number
- CN112579528B CN112579528B CN202011363341.7A CN202011363341A CN112579528B CN 112579528 B CN112579528 B CN 112579528B CN 202011363341 A CN202011363341 A CN 202011363341A CN 112579528 B CN112579528 B CN 112579528B
- Authority
- CN
- China
- Prior art keywords
- file
- client
- node
- server
- access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/137—Hash-based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/148—File search processing
- G06F16/152—File search processing using file content signatures, e.g. hash values
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for efficiently accessing files at a server side of an embedded network file system, which can remarkably reduce the consumption of a server on running resources, effectively improve the access efficiency of files at the server side and overcome the defects of slow data request response, low concurrent access efficiency and high occupation of processing resources of the traditional file system under an embedded architecture by dynamically monitoring the overtime state of a client side and the timely release of resources while ensuring that the file indexing and management are faster and more efficient. The method for rapidly indexing and dynamically managing the server disclosed by the invention optimizes and designs file indexing, file reading and writing and management of the mounted nodes.
Description
Technical Field
The invention belongs to the technical field of electronics, and relates to a method for efficiently accessing files at a server side of an embedded network file system.
Background
In a commonly used network system, access to a remote storage device node is usually implemented by using a conventional Network File System (NFS) method. The network file system uses a client/server architecture, uses a Remote Procedure Call (RPC) mechanism to map a remote storage medium into a local storage device of the client, and realizes the access operation of the client to a server file through a network, thereby realizing the file sharing and the unified management and maintenance under the network system.
The traditional network file system is widely oriented to a generalized application platform, the processing resources and the cache resources of a server terminal are sufficient, and no clear requirement is made on the real-time performance of file retrieval access. Under the embedded system, the traditional server-side implementation mode has obvious defects:
1. the server has slow response speed to the data request: the server program is limited by a file retrieval mode of the server and an organization management mode of the mounted nodes, and much time is consumed for responding to a client request and indexing a target file;
2. the processing efficiency is low when the multi-node accesses in a concurrent way: when a plurality of client nodes access a certain file simultaneously under a network file system, the organization structure of a client service program determines that the index efficiency is low due to the overlarge cache pool, and the processing stability of concurrent access is poor due to the access management mode of the file;
3. the occupied amount of processor resources is high: the processor resources in the embedded system are often relatively limited, and the network file system service does not monitor and manage the mounted nodes and occupied resources, so that a large amount of processors and cache resources are occupied when the mounted client nodes change dynamically and frequently, and the load of the processors is increased.
The above disadvantages are that the traditional network file system server cannot meet the embedded system with high real-time requirement, and a method for realizing fast indexing and dynamic management of the server-side file with high efficiency needs to be designed to improve the service performance of the whole network file system.
Disclosure of Invention
Technical problem to be solved
In order to avoid the defects of the prior art, the invention provides a method for efficiently accessing files at a server side of an embedded network file system, which is characterized in that a file which is accessed recently is placed at the front end of a linked list by using a form organization of a fast Hash and a cross linked list according to a locality principle of file access, so that the request of a client side can be indexed fast, the response speed of the server side to the file access is improved, the fast retrieval of the file at the server side is realized by using the minimum processor and memory resources, and the service performance of the whole file system is improved.
Technical scheme
A method for efficiently accessing files at a server side of an embedded network file system is characterized by comprising the following steps: the method comprises the following steps:
when a client requests to open a server file, the server quickly indexes the target file of the mounted node and generates dynamic management data, and the process is as follows:
step a 1: the client side is used as a remote node to be mounted to the server side as a mounting point, and the server generates and initializes a file access linked list of the mounting point; then, according to the file path information mounted by the client, generating a hash table of a target path, completing hash value calculation of all files under the mounting point and generating a one-to-one mapping relation between the mounting point and the files;
step a 2: when a client requests to open a certain file under a mounting path, a server analyzes the name of a target file, calculates a corresponding hash value and then indexes a hash table to position the target file;
step a 3: indexing the access node linked list of the target file to confirm whether the file can be opened with the authority requested by the client, if not, jumping to the step a6, and if so, executing the step a 4;
step a 4: opening a target file, and generating an NFS handle of the file and a mapping table of a file name and the NFS handle;
step a 5: updating the file access bidirectional linked list information of the client node, and updating the mounted client linked list information of the file;
step a 6: and replying a file opening request result to the client node:
when a client requests to read and write a server file, the server accesses a bidirectional linked list of the file from the client to quickly index a target file, and adjusts the sequence of the linked list, wherein the process comprises the following steps:
step b 1: analyzing the NFS handle of the client request file;
step b 2: indexing an access file bidirectional linked list of the client node, positioning a target file according to a handle value, and obtaining current access state information of the file;
step b 3: executing read-write operation according to the request content of the client to obtain an operation result;
step b 4: adjusting the sequence of file access bidirectional linked lists of the client, and setting the currently accessed file nodes to the head positions of the linked lists;
step b 5: and replying a file read-write operation request result to the client node.
When the server dynamically manages the client, the activity state of the mounting point is monitored in real time, and the overtime client node is released in time, so that the processing and cache resources of the server are saved, and the process is as follows:
step c 1: the server program creates a client overtime monitoring task and maintains and monitors a client node queue in the initialization execution stage;
step c 2: after the client is mounted successfully, adding the node into a monitoring queue by taking the timestamp of the RPC request as the creation time and the latest access time;
step c 3: updating the latest access time of the corresponding node in the monitoring queue every time the client initiates an access request;
step c 4: the overtime monitoring task traverses the latest access time of each node in the monitoring queue by taking 5s as a period, if the node does not perform file access operation in six continuous periods, the node access is judged to be overtime, and the step c5 is executed;
step c 5: and releasing the overtime client node, deleting the data access doubly linked list of the node, and updating the information related to the node under the file access linked list.
Advantageous effects
The method for efficiently accessing the files at the server side of the embedded network file system ensures that file indexing and management are faster and more efficient, remarkably reduces the consumption of the server on operating resources by dynamically monitoring the overtime state of the client side and the timely release of resources, effectively improves the access efficiency of the files at the server side, and overcomes the defects of slow data request response, low concurrent access efficiency and high occupation of processing resources of the traditional file system under an embedded architecture. The method for rapidly indexing and dynamically managing the server disclosed by the invention optimizes and designs file indexing, file reading and writing and management of the mounted nodes.
Drawings
FIG. 1 is a basic schematic diagram of server-side efficient file access;
FIG. 2 is a flowchart of a server processing a client file open request;
FIG. 3 is a flowchart of a server processing a client file read-write application;
FIG. 4 is a flowchart of server-side timeout monitoring;
Detailed Description
The invention will now be further described with reference to the following examples and drawings:
the technical scheme of the invention is realized as follows:
a method for quick index and dynamic management of embedded network file system server includes executing embedded client end device of client end program and storing shared file and executing embedded server end device of server end program. Wherein:
and the client device is mainly responsible for mapping a target storage path of the remote server to local equipment, initiating access operation of a server file and carrying out dynamic heartbeat information interaction with the server. After the client finishes mounting to the server, the client becomes a mounting node of the server;
the server-side equipment is mainly responsible for storing file data shared to the client and executing a server program, so that the mounted node and related files thereof are dynamically managed, and the operations of mounting, heartbeat, read-write access and the like of the client are efficiently and quickly responded;
the basic principle of dynamic management in the invention is shown in fig. 1, firstly, a fast mapping mechanism of file indexes is generated through hash operation, and a group of mount client information tables alpha are managed and maintained based on each shared file; meanwhile, a set of two-way linked list beta of the accessed files is maintained based on the mounted client, and the linked list set alpha and the linked list set beta form a cross chain. The chain table group alpha is a client information table; and the chain table group is a beta file information table.
The details of the invention are as follows:
when the client requests to open the server file, the server quickly indexes the target file of the mounted node and generates dynamic management data. The flow is shown in fig. 2 and described as follows:
1) when a client side is used as a remote node to be mounted to a server side, the server generates and initializes a file access linked list of the node; then, according to the file path information mounted by the client, generating a hash table of a target path, completing hash value calculation of all files under the target path and generating a one-to-one mapping relation between each file and the hash value thereof;
2) when a client requests to open a certain file under a mounting path, a server analyzes the name of a target file, calculates a corresponding hash value and then indexes a hash table to position the target file;
3) indexing an access node linked list of the target file to confirm whether the file can be opened with the authority requested by the client, if not, jumping to step 6), and if so, executing step 4);
4) opening a target file, and generating an NFS handle of the file and a mapping table of a file name and the NFS handle;
5) updating a bidirectional linked list of a file accessed by the client node, and updating a mounted client linked list of the file;
6) and replying a file opening request result to the client node.
When the client requests to read and write the server file, the server accesses the bidirectional linked list of the file from the client to quickly index the target file, and adjusts the sequence of the linked list. The flow is shown in fig. 3 and described as follows:
1) analyzing the NFS handle of the client request file;
2) indexing an access file bidirectional linked list of the client node, positioning a target file according to a handle value, and setting a file access state according to current client information of the target file;
3) according to the request content of the client, executing file operations such as reading and writing, obtaining an operation result, and updating the access state of the current client;
4) adjusting the sequence of file access bidirectional linked lists of the client, and setting the currently accessed file nodes to the head positions of the linked lists;
5) and replying a file read-write operation request result to the client node.
The server dynamically manages the client, monitors the activity state of the mounted node in real time, and releases the overtime client node in time so as to save the processing and cache resources of the server. The flow is shown in fig. 4 and described as follows:
1) the server program creates a client overtime monitoring task and maintains and monitors a client node queue in the initialization execution stage;
2) after the client is mounted successfully, adding the node into a monitoring queue by taking the timestamp of the RPC request as the creation time and the latest access time;
3) updating the latest access time of the corresponding node in the monitoring queue every time the client initiates an access request;
4) traversing the latest access time of each node in the monitoring queue by taking 5s as a period for the overtime monitoring task, if the node has no file access operation within six continuous periods, judging that the node access is overtime, and executing the step 5);
5) and releasing the overtime client node, deleting the data access doubly linked list of the node, and updating the information related to the node under the file access linked list.
The specific embodiment is as follows:
constructing a multi-node network system comprising:
8 client nodes, used for realizing remote mounting and accessing operations of network file system, wherein the client nodes adopt a software running minimum system composed of a processor based on Power series P2020, a wind river VxWorks6.9 operating system matched with the processor, and a DDR chip of a Microsmi company W3J128M72G-800LBI model, and are configured with embedded hardware modules of an RS232 debugging serial port and an Ethernet communication port;
1 server end node, is used for operating and realizing the service function of the server end of the network file system, the server end node uses and forms the minimum system that the software operates based on the very small Karl Power series P2020 processor, supporting wind river VxWorks6.9 operating system, and Microsmi W3J128M72G-800LBI model DDR chip, and dispose RS232 and debug the embedded hardware module of serial port and Ethernet communication port;
the FLASH is used as a storage medium of a server side shared file, a magnesium light 128MB FLASH chip is adopted, the FLASH is hung on a LocalBus bus of a server side module P2020 processor, the address space is 0xF 8000000-0 xFFFFFFFF, a FAT32 local file system with the capacity of 32MB is hung on the FLASH, and service is provided for local access of file data in the FLASH.
Claims (1)
1. A method for efficiently accessing files at a server side of an embedded network file system is characterized by comprising the following steps: the method comprises the following steps:
when a client requests to open a server file, the server quickly indexes the target file of the mounted node and generates dynamic management data, and the process comprises the following steps:
step a 1: the client side is used as a remote node to be mounted to the server side as a mounting point, and the server generates and initializes a file access linked list of the mounting point; then, according to the file path information mounted by the client, generating a hash table of a target path, completing hash value calculation of all files under the mounting point and generating a one-to-one mapping relation between the mounting point and the files;
step a 2: when a client requests to open a certain file under a mounting path, a server analyzes the name of a target file, calculates a corresponding hash value and then indexes a hash table to position the target file;
step a 3: indexing the access node linked list of the target file to confirm whether the file can be opened with the authority requested by the client, if not, jumping to the step a6, and if so, executing the step a 4;
step a 4: opening a target file, and generating an NFS handle of the file and a mapping table of a file name and the NFS handle;
step a 5: updating the file access bidirectional linked list information of the client node, and updating the mounted client linked list information of the file;
step a 6: replying a file opening request result to the client node;
when a client requests to read and write a server file, the server accesses a bidirectional linked list of the file from the client to quickly index a target file, and adjusts the sequence of the linked list, wherein the process comprises the following steps:
step b 1: analyzing the NFS handle of the client request file;
step b 2: indexing an access file bidirectional linked list of the client node, positioning a target file according to the handle value, and obtaining the current access state information of the file;
step b 3: executing read-write operation according to the request content of the client to obtain an operation result;
step b 4: adjusting the sequence of file access bidirectional linked lists of the client, and setting the currently accessed file nodes to the head positions of the linked lists;
step b 5: replying a file read-write operation request result to the client node;
when the server dynamically manages the client, the activity state of the mounting point is monitored in real time, and the overtime client node is released in time, so that the processing and cache resources of the server are saved, and the process is as follows:
step c 1: the server program creates a client overtime monitoring task and maintains and monitors a client node queue in an initialization execution stage;
step c 2: after the client is mounted successfully, adding the node into a client node queue by taking a timestamp of an RPC request as creation time and latest access time;
step c 3: updating the latest access time of the corresponding node in the client node queue every time the client initiates an access request;
step c 4: the overtime monitoring task traverses the latest access time of each node in the client node queue by taking 5s as a period, if the node has no file access operation within six continuous periods, the node access is judged to be overtime, and step c5 is executed;
step c 5: and releasing the overtime client node, deleting the data access doubly linked list of the node, and updating the information related to the node under the file access linked list.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011363341.7A CN112579528B (en) | 2020-11-28 | 2020-11-28 | Method for efficiently accessing files at server side of embedded network file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011363341.7A CN112579528B (en) | 2020-11-28 | 2020-11-28 | Method for efficiently accessing files at server side of embedded network file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112579528A CN112579528A (en) | 2021-03-30 |
| CN112579528B true CN112579528B (en) | 2022-09-02 |
Family
ID=75123966
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011363341.7A Active CN112579528B (en) | 2020-11-28 | 2020-11-28 | Method for efficiently accessing files at server side of embedded network file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579528B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116150090A (en) * | 2021-11-22 | 2023-05-23 | 华为技术有限公司 | File verification method and related equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101158965A (en) * | 2007-10-25 | 2008-04-09 | 中国科学院计算技术研究所 | File reading system and method of distributed file systems |
| CN108959660A (en) * | 2018-08-15 | 2018-12-07 | 东北大学 | A kind of storage method and application method based on HDFS distributed file system |
| CN111221776A (en) * | 2019-12-30 | 2020-06-02 | 上海交通大学 | Method, system and medium for implementing file system facing nonvolatile memory |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020161860A1 (en) * | 2001-02-28 | 2002-10-31 | Benjamin Godlin | Method and system for differential distributed data file storage, management and access |
| US10803013B2 (en) * | 2017-02-10 | 2020-10-13 | Smugmug, Inc. | Efficient similarity detection |
-
2020
- 2020-11-28 CN CN202011363341.7A patent/CN112579528B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101158965A (en) * | 2007-10-25 | 2008-04-09 | 中国科学院计算技术研究所 | File reading system and method of distributed file systems |
| CN108959660A (en) * | 2018-08-15 | 2018-12-07 | 东北大学 | A kind of storage method and application method based on HDFS distributed file system |
| CN111221776A (en) * | 2019-12-30 | 2020-06-02 | 上海交通大学 | Method, system and medium for implementing file system facing nonvolatile memory |
Non-Patent Citations (2)
| Title |
|---|
| Scalable Distributed Meta-data Management In Parallel NFS;Rajesh Vellore Arumugam等;《International Congress on Advanced Applied Informatics》;20160731;全文 * |
| 网络文件系统实时性性能改进浅析;孙志颖等;《信息通信》;20181231;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112579528A (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11741053B2 (en) | Data management system, method, terminal and medium based on hybrid storage | |
| US8352517B2 (en) | Infrastructure for spilling pages to a persistent store | |
| Feeley et al. | Implementing global memory management in a workstation cluster | |
| CN101493826B (en) | Database system based on WEB application and data management method thereof | |
| CN111427844B (en) | Data migration system and method for file hierarchical storage | |
| JP2006155623A (en) | Method and apparatus for recovering database cluster | |
| US11080207B2 (en) | Caching framework for big-data engines in the cloud | |
| CN102945251A (en) | Method for optimizing performance of disk database by memory database technology | |
| CN113553346B (en) | Large-scale real-time data stream integrated processing, forwarding and storing method and system | |
| CN111984191A (en) | Multi-client caching method and system supporting distributed storage | |
| CN111782612A (en) | File data edge caching method in cross-domain virtual data space | |
| CN112579528B (en) | Method for efficiently accessing files at server side of embedded network file system | |
| WO2023125524A1 (en) | Data storage method and system, storage access configuration method and related device | |
| CN106202459A (en) | Relevant database storage performance optimization method under virtualized environment and system | |
| CN104281673A (en) | Cache building system and method for database | |
| US11921683B2 (en) | Use of time to live value during database compaction | |
| CN116455972B (en) | Method and system for realizing simulation middleware based on message center communication | |
| CN113311994A (en) | Data caching method based on high concurrency | |
| Otoo et al. | Accurate modeling of cache replacement policies in a data grid | |
| US9069821B2 (en) | Method of processing files in storage system and data server using the method | |
| Duan et al. | Gengar: an RDMA-based distributed hybrid memory pool | |
| CN106997304B (en) | Input and output event processing method and device | |
| CN103714022A (en) | Mixed storage system based on data block | |
| CN112540954A (en) | Multi-level storage construction and online migration method in directory unit | |
| US10452546B2 (en) | Cache utility modeling for automated cache configuration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |