CN113792023B - Multi-channel supporting embedded file system and control method thereof - Google Patents
Multi-channel supporting embedded file system and control method thereof Download PDFInfo
- Publication number
- CN113792023B CN113792023B CN202111093259.1A CN202111093259A CN113792023B CN 113792023 B CN113792023 B CN 113792023B CN 202111093259 A CN202111093259 A CN 202111093259A CN 113792023 B CN113792023 B CN 113792023B
- Authority
- CN
- China
- Prior art keywords
- data
- index
- file
- block
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000003491 array Methods 0.000 claims description 5
- 239000012634 fragment Substances 0.000 description 6
- 230000007547 defect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses an embedded file system supporting multiple channels and a control method thereof, wherein the embedded file system comprises: a file system writing unit, configured to configure an available index table for each available channel, where the available index table includes storage medium address information, write file data of each available channel into the storage medium according to the information of the available index table, and update the data index and the file index; the file system reading unit is used for matching file index information of the file to be read, finding out data indexes of all the file data to be read according to the file index information, and outputting the file data to be read according to the data indexes and the channels corresponding to the corresponding available index tables. The invention solves the problem of simultaneous reading and writing of multi-channel files, and the file reading and writing channel is dynamically adjusted, so that the stability is good.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an embedded file system supporting multiple channels and a control method thereof.
Background
With the rapid development of information technology and the internet, the bandwidth requirements of military large-capacity equipment, civil large-capacity equipment and high-speed data recording systems on collected signals are further improved, and the requirements of channel number are further increased, so that the requirements of the system on the real-time data transmission bandwidth of the recording equipment are higher and higher, and the existing embedded file system is difficult to meet the requirements.
Currently, the mainstream high-speed file system on the market includes a tree-type data structure file system, a linked list-type data structure file system and an array-type data structure file system.
For a tree-type data structure file system, the file system can realize the functions of simplicity, high efficiency and power-off reliability, but has the defects that files cannot be written simultaneously by multiple channels and the like. For example, in the Tree data structure file system using the phase Tree algorithm disclosed in the patent CN101162459a, the three sub data structures are separated under the root file system, the whole system has atomicity, no file system scanning and repairing operation is required for starting up, but simultaneous writing of multiple files is not mentioned, the structure of the Tree needs to be repeatedly modified for writing a single file, and under the condition of multiple channels, the file system is inefficient.
For a linked list type data structure file system, the file system can realize the functions of simplicity, high efficiency, power-off reliability and expansibility, but has the defects of low file access efficiency, more single file fragments, discontinuous data and the like when files are simultaneously written by multiple channels.
For an array type data structure file system, although the file system can realize the functions of simplicity, inextensibility, high efficiency, power-off reliability, the file system has the defects of more single file fragments, discontinuous data and the like when files are written in multiple channels simultaneously. For example, patent CN110716904a discloses a design method of an embedded file system, which divides a flash into three blocks, a spatial information block, a file description block and a data block, wherein the file description block is an array type data structure file system, and each file has a file size, a data size and a data storage start page. There are disadvantages such as a large number of fragments and discontinuity in the case of writing a plurality of files simultaneously.
In summary, the problems of the prior art are: when the multichannel file is written at the same time, the efficiency is low, the file fragments are more, the data are discontinuous, and the high-speed information storage application is difficult to meet.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: aiming at the technical problems existing in the prior art, the invention provides an embedded file system supporting multiple channels and a control method thereof, which meet the requirements of high-speed information storage for the embedded file system under the conditions of high speed, multiple channels, large capacity, large single file and large number of files.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
an embedded file system supporting multiple channels, comprising:
a file system writing unit, configured to generate an idle index table according to a preset maximum data index value and a preset minimum data index value, where the idle index table includes information of at least one data block, the information of the data block includes a start address in a storage medium and a length in the storage medium, segment the idle index table according to the number of available channels, obtain an available index table corresponding to each available channel one by one, synchronously write file data of each available channel into the storage medium according to the information of the data block of the available index table, and update a data index block area, a file index block area and a file system block area in a storage data structure of the embedded file system;
A file system reading unit, configured to find file index information of a file to be read in a file index block area in a storage data structure of the embedded file system, match corresponding data indexes and data masks in the data index block area according to a start data index value in the file index information, where the data indexes, the data masks and the data blocks are in one-to-one correspondence, find file data to be read in a storage medium according to the data masks, and output the data of the file to be read through an available channel corresponding to the data indexes;
the file deleting unit is used for finding file index information of a file to be deleted in a file index block area in a storage data structure of the embedded file system, matching corresponding data indexes and data masks in the data index block area according to a start data index value in the file index information, finding file data to be deleted in a storage medium according to the data masks, and then deleting the file data to be deleted;
and the defragmentation unit is used for traversing all file index information in a file index block area in a storage data structure of the embedded file system, deleting files which are not checked after checking, traversing all data indexes in a data index block area in the storage data structure of the embedded file system, and deleting data indexes which are not occupied by the files but are not empty.
Further, in the storage data structure of the embedded file system:
the file system block area comprises a file system version, a file number, a total space, a residual space, a minimum data index, a maximum data index, a used file index, an unused file index, a mask data block byte number and a redundancy check bit, and the data block comprises 32 mask data blocks;
the file index block area comprises file index information of each file, wherein the file index information comprises a file start data index value, a file time, a file name and a file size of the corresponding file;
the data index block area comprises a two-dimensional array formed by data index values and corresponding data masks, and the length of the two-dimensional array is the number of all data indexes of the data block;
the storage data structure of the embedded file system further comprises a data block area, the data block area comprises data blocks with the same size, the two-dimensional arrays in the data block and the data index block area are in one-to-one correspondence, and the data blocks are numbered according to the data index numbers in the corresponding two-dimensional arrays respectively so as to support parallel recording of the multi-channel file.
Further, the method further comprises a file initialization unit, wherein the file initialization unit is used for identifying the size of a storage medium and the number of bytes supporting block operation, setting the size of a data block according to the number of bytes supporting block operation, calculating the ratio of the size of a storage medium to the size of the data block to obtain the maximum data index value in the number of data indexes, respectively setting the number of reserved data indexes in the file system block area, the file index block area and the data index block area, and summing the reserved data index numbers to obtain the minimum data index value in the number of data indexes.
The invention also provides any control method of the embedded file system supporting multiple channels, which comprises the following steps:
s1) generating an idle index table according to a preset maximum data index value and a preset minimum data index value, wherein the idle index table comprises information of at least one data block, and the information of the data block comprises a starting address in a storage medium and a length in the storage medium;
s2) dividing the idle index table into available index tables corresponding to available channels one by one, synchronously writing files in each available channel into a storage medium according to the information of each data block in the available index tables, and updating the values of data indexes and data masks corresponding to the data blocks and file index information corresponding to the files;
S3) updating the number of files, the residual space, the used file index, the unused file index and the redundancy check bit according to the updated data index and the file index information.
Further, step S2) specifically includes:
s21) according to the number of the available channels, the idle index table is segmented averagely, and the available index table corresponding to each available channel one by one is obtained;
s22) synchronously writing the file data to be written in each available channel into corresponding data blocks according to the information of the data blocks in the corresponding available index table, updating the data index corresponding to each data block to point to the data index corresponding to the next data block according to the information of the data blocks in the available index table, wherein the value of the data index corresponding to the last data block is an end mark, and updating the content of the data mask corresponding to each data block to be the information of the data block;
s23) generating file index information of the written file, and pointing the value of the start data index in the file index information to the data index of the first data block in the available index table, where the file index information in this embodiment includes the start data index, file time, file name and file size, and the value of the start data index is the sequence number of the data index of the first data block in the corresponding available index table in the data index block area.
Further, step S22) specifically includes: and for the data of the file to be written in the current channel, sequentially writing the corresponding data blocks according to the information of the data blocks in the corresponding available index table, continuously writing the data into the next data block according to the information of the next data block in the available index table after each data block is fully written, and directing the data index corresponding to the current data block to the data index corresponding to the next data block, updating the content of the data mask of the current data block pair to the information of the current data block, and updating the value of the corresponding data mask to the end mark and updating the value of the corresponding data mask to the information of the data block if no data is to be written.
Further, the step S1) further includes a pre-configuration step, specifically including: identifying the space size of a storage medium and the byte number of a support block operation, setting the size of a data block according to the byte number of the support block operation, calculating the ratio of the space size of the storage medium to the size of the data block to obtain a maximum data index value, respectively setting the reserved data index numbers of a file system block area, a file index block area and a data index block area, and summing the reserved data index numbers to obtain a minimum data index value.
Further, step S3) further includes a step of reading the file, specifically including:
SA 1) traversing all file index information, matching one or more of file time, file name or file size of a file to be read, if a matching result is obtained, finding a data index and a data mask corresponding to the first data block of the file to be read in a data index block area according to the corresponding value of the starting data index, and sequentially finding a data index and a data mask corresponding to the subsequent data block from the value of the data index;
SA 2) acquiring data of the file to be read stored in a storage medium according to the value of a data mask corresponding to each data block of the file to be read, averagely segmenting the idle index table according to the number of the current available channels to obtain a current available index table corresponding to each current available channel one by one, determining the current available index table corresponding to each data block according to the information of each data block, and outputting the data of the file to be read through the current available channels corresponding to the current available index table.
Further, step S3) further includes a step of deleting the file, specifically including:
SB 1) traversing all file index information, matching one or more of file time, file name or file size of the file to be deleted, if a matching result is obtained, according to the value of the corresponding starting data index, finding the data index and the data mask corresponding to the first data block of the file to be deleted in a data index block area, and sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index;
SB 2) deleting the data of the file to be deleted stored in the storage medium according to the value of the data mask corresponding to each data block of the file to be read, and filling the data mask and the value of the corresponding data index to be empty.
Further, step S3) further includes a step of defragmentation, specifically including:
firstly traversing all file index information, respectively checking the file index information, if the checking is not passed, marking the file corresponding to the file index information as a file to be deleted, and according to the value of the starting data index in the file index information, finding the data index and the data mask corresponding to the first data block of the file to be deleted in a data index block area, and sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index;
deleting the data of the file to be deleted stored in the storage medium according to the value of the data mask corresponding to each data block of the file to be read, and filling the data mask and the value of the corresponding data index into the space;
and traversing all the data indexes of the data index block area, backtracking the data indexes with non-null values, filling the data indexes and the values of the corresponding data masks to be null if no backtracking result exists in the file index block area, and deleting the file data stored in the storage medium corresponding to the data masks.
Compared with the prior art, the invention has the advantages that:
1. the invention segments the idle index table according to the number of the available channels, and stores the data according to the information in the corresponding available index table when the data of each channel is written, thereby solving the problem of simultaneous writing of multi-channel files.
2. Aiming at the situation that file fragments are generated when the power is suddenly cut off during file writing or file deleting, the invention configures the defragmentation unit to defragment, and solves the problem of data redundancy of the storage medium.
Drawings
Fig. 1 is a schematic diagram of a storage data structure according to a first embodiment of the invention.
Fig. 2 is a schematic diagram of a free index table or an available index table according to a first embodiment of the present invention.
Fig. 3 is a schematic structural diagram of an embedded file system according to a first embodiment of the present invention.
Detailed Description
The invention is further described below in connection with the drawings and the specific preferred embodiments, but the scope of protection of the invention is not limited thereby.
Example 1
The embodiment proposes an embedded file system supporting multiple channels, as shown in fig. 1, where a storage data structure of the embedded file system includes four block areas, which are a file system block area, a file index block area, a data index block area, and a data block area, respectively, where:
the file system block area comprises a file system version, a file number, a total space, a residual space, a minimum data index, a maximum data index, a used file index, an unused file index, a mask data block byte number and a redundancy check bit; wherein the minimum data index and the maximum data index provide corresponding data index values for managing the locations of the data blocks in the data block area in the storage medium; the byte number and the total space of the mask data block determine the minimum unit of the space occupied by the file in the storage medium and also determine the basis of multi-channel data recording; the redundancy check bit provides security guarantee for the multichannel file system; the file system block area not only manages information parameters of the whole embedded file system, but also has the functions of flexibly adjusting the data index length corresponding to the data blocks of the data block area and verifying the security of the embedded file system.
The file index block region includes file index information of each file, the file index information including a file start data index value, a file time, a file name, and a file size of the corresponding file, the start data index value being a sequence number of a data index in the data index block region.
The data index block area is composed of a plurality of two-dimensional arrays, each two-dimensional array is composed of a data index and a corresponding data mask, the value of the data index can have three state values, namely an unused state (initial state), an ending state (representing that the data index is a file ending index) and a next data index number (data blocks corresponding to the data indexes possibly need to be stored when the file is large); the data mask more precisely specifies the length of the actual data in the data block corresponding to the current data index.
The data block area is composed of a plurality of data blocks with the same size, each data block corresponds to the data index and the data mask one by one, the data blocks are numbered from 0 to the largest data index number-1, each data block is composed of 32 mask data blocks, and the number of the data blocks is fixed, so that parallel recording of multi-channel file data is supported, each data block is composed of 32 mask data blocks from bit0 to bit31, and the actual file space utilization rate of a file system can be accurately acquired.
In this embodiment a 32-bit mask is added to the data block. Compared with the fixed file size array file system, the file index block area is doubled compared with the traditional file index block area, but the minimum file storage space of the data block area is changed, and the file size is not fixed. Compared with any file size linked list file system, the file system has the advantage of small minimum file storage space, but is also a key factor influencing the occupancy rate.
In order to implement multi-channel file reading and writing, as shown in fig. 2, this embodiment generates a free index table for each available channel, where the index table includes a combination of information of data blocks corresponding to 1 or more data indexes of the file in the storage medium, each data block is represented in the multi-channel by a corresponding data index number, and finally converted into a physical address and a length of the storage medium expressed by data, so that the information of the data block includes a data index number corresponding to the data block, a start address in the storage medium, and a length in the storage medium. The available channels in this embodiment may be idle channels or designated channels.
As shown in fig. 3, the embedded file system of the present embodiment includes a file system initializing unit, a file system writing unit, a file system reading unit, a file deleting unit, and a defragmentation unit, where:
The file initialization unit identifies a byte number range of a storage medium supporting block operation, sets a default mask data block byte number as a lowest byte number of the supporting block operation, calculates a sum of byte numbers of a file system block area, a file index block area, a data index block area, and a required data index number, thereby calculating a minimum data index value and a maximum data index value of the data block, calculating a remaining space and a total space, filling a file index used, a file index unused, a file number, and calculating redundancy check bits, filling the file system block area, and initializing the data index block area to be empty.
For example, for a 1GByte storage space, the number of bytes supporting block operations is 512, the number of bytes of the mask data block is set to be a multiple of 512 by the file initialization unit, and is set to be 512 by default, and in this embodiment, 32 bits are masked, so that each mask data block has a size of 512 x 32 bytes, which represents that each file occupies 16 kbytes at minimum, and a single file occupies the whole disk at maximum, so that 32 data indexes are reserved in the file system block area. The maximum number of files is 65536, each file index occupies 128Byte, the whole file index block area occupies 8MByte, so the file index block area reserves 512 data indexes. The maximum data index value is 1 Gbyte/(512 x 32) byte equal to 65536, each data index occupies 8 bytes, the whole file index block occupies 512 kbytes, so the data index block area reserves 32 data indexes, and the minimum data index value is 32+512+32. The remaining space is the data index space of the maximum index value minus the minimum index value, and the total space is the data index space of the maximum index. The calculated data is filled into the file system block area. And filling data of all 1 bits in each data index block area.
The file system writing unit calculates a complete idle space index table according to files to be written in N available channels, divides the idle space index table into N available index tables according to the number N of the available channels, and each available index table is also an independent index table and can represent the storable position of each file in the corresponding channel. In the writing process, the index table is updated, the data index and the data mask value corresponding to the data blocks in the index table are updated, and finally, when the writing is finished, the file index block area and the file system block area are updated, the idle index table is segmented according to the number of available channels, and data storage is carried out on the data of each channel according to the information in the corresponding available index table when the data of each channel is written, so that the problem of simultaneous writing of the multi-channel file is solved.
For example, a file may be currently created by 4 available channels, and for a new 1GByte storage hard disk, the file system writing unit calculates the complete free space index table to be the starting address 0 value as the minimum index 576, the length 0 value as the maximum index (65536) minus the minimum index (576) equal to 64960, the starting address 1 is 0, the length 1 is also 0, and sequentially filling to be the last starting address 65536 and the length 65536 of 0. The free space index table is divided into 4 segments, the first segment initial starting address has the value of index 576, the corresponding length is 16240, the second segment initial starting address has the value of index 161686, the corresponding length is 16240, the third segment initial starting address has the value of index 33056, the corresponding length is 16240, the fourth segment initial starting address has the value of index 49296, and the corresponding length is 16240. And for each available index table, after the current data block is written into the storage medium, continuing writing data into the next data block according to the information of the next data block of the available index table, pointing the data index corresponding to the current data block to the data index corresponding to the next data block, updating the content of the data mask of the current data block pair to the information of the current data block, and repeating the process until the writing of the data block into the available index table is finished and the value of the corresponding data index is set as an end mark. Updating file index information of the written file of the file index block area, pointing the value of the start data index in the file index information to the data index of the first data block in the available index table, and updating the number of files, the residual space, the used file index, the unused file index and the redundancy check bit of the file system block area.
The file system reading unit finds a file to be read in a file index block area, finds a corresponding data index and a data mask in a data index block according to a start data index of the file to be read, finds contents of a corresponding data block according to the data mask, sequentially finds contents of a data block corresponding to a subsequent data index according to the data index until a value of the subsequent data index is an end mark, and then outputs the contents of the data block through a channel corresponding to an available index table according to the relation between the data block and the available index table, so that multi-channel file reading can be realized under the condition that a plurality of files need to be read.
For example, 100 files are currently saved, the file system reading unit looks at the file index used in the file system block area, and traverses to find the file name to be read, so as to find the start data index (value 1024), the file time, the file name and the file size (1 MByte) of the corresponding file, so as to find the 1024 th two-dimensional array (data index 1024 and data mask 1024) of the data index block area, wherein the value of the data mask 1024 is equal to 1025, the value of the data mask 1024 is 0, and the 1025 th two-dimensional array of the data index block area is continuously found until the value of the data index in the next two-dimensional array is the end mark (greater than 65536, and the check value is increased by 65536) but not blank (not all 1), and if the two-dimensional array is the 1088 th two-dimensional array, the file data is in the actual area represented by the data indexes 1024 to 1088, that is the positions 1024×512×32 bytes to 1088×512×32 bytes in the storage medium.
The file deleting unit finds a file to be deleted in a file index block area, finds a corresponding data index and a data mask in a data index block according to a start data index of the file to be deleted, finds a corresponding data block and clears content according to the data index, and sequentially finds a corresponding data block and clears content of a subsequent data index according to the data mask until the value of the subsequent data index is an end mark.
For example, 100 files are currently saved, the file index used in the file system block is checked, the deleted file name is traversed, so that the next data index (value 1024), the file time, the file name and the file size (1 MByte) are found, the 1024 th data (next data index 1024 and data mask 1024) of the data index block is equal to 1025, the next data index 1024 value is equal to 0, the 1024 th data is filled to be empty, the 1025 th data of the data index block is continued, the 1025 th data is filled to be empty until the next data index value is an end mark (greater than 65536, and the check value is added to 65536) but not to be empty (not all 1), the file index block is updated, and the used file index, the unused file index and the rest space in the file system are updated.
When the file writing is interrupted, file fragments are generated, some non-empty data indexes appear in the data index block area, but the data indexes not pointed to in the file index block area, or when the file is deleted, the power is interrupted, and unknown files are generated in the file index block area and the file system block area. The embedded file system of this embodiment further provides a defragmentation unit, where the defragmentation unit traverses file indexes used in the embedded file system, or traverses all file index information in a file index block area, deletes files that are not checked after checking, traverses all data indexes in a data index block area, deletes data indexes in the data index block area that are not occupied by files but are not empty, and updates the file index block area and the file system block area.
Example two
The embodiment proposes a control method for supporting a multi-channel embedded file system according to the first embodiment, including:
s1) generating an idle index table according to a preset maximum data index value and a preset minimum data index value, wherein the idle index table comprises information of at least one data block, the data block corresponds to all data indexes between the maximum data index value and the minimum data index value one by one, the data block comprises 32 mask data blocks, and the information of the data block comprises a start address and a length of a data segment formed by the corresponding 32 mask data blocks in a storage medium;
S2) dividing the idle index table into available index tables corresponding to available channels one by one, synchronously writing files in each available channel into a storage medium according to the information of each data block in the available index tables, and updating the values of data indexes and data masks corresponding to the data blocks and file index information corresponding to the files;
s3) updating information such as the number of files, the residual space, the used file indexes, the unused file indexes, redundancy check bits and the like according to the updated data indexes and file index information.
Specifically, for the content described in step S2), the specific steps thereof include:
s21) according to the number of the available channels, the idle index table is segmented averagely, and the available index table corresponding to each available channel one by one is obtained;
s22) synchronously writing the file data to be written in each available channel into corresponding data blocks according to the information of the data blocks in the corresponding available index table, updating the data index corresponding to each data block to point to the data index corresponding to the next data block according to the information of the data blocks in the available index table, wherein the value of the data index corresponding to the last data block is an end mark, and updating the content of the data mask corresponding to each data block to be the information of the data block;
S23) generating file index information of the written file, and pointing the value of the start data index in the file index information to the data index of the first data block in the available index table, where the file index information in this embodiment includes the start data index, file time, file name and file size, and the value of the start data index is the sequence number of the data index of the first data block in the corresponding available index table in the data index block area.
The content of step S22) of the present embodiment specifically includes: and for the data of the file to be written in the current channel, sequentially writing the corresponding data blocks according to the information of the data blocks in the corresponding available index table, continuously writing the data into the next data block according to the information of the next data block in the available index table after each data block is fully written, and directing the data index corresponding to the current data block to the data index corresponding to the next data block, updating the content of the data mask of the current data block pair to the information of the current data block, and updating the value of the corresponding data mask to the end mark and updating the value of the corresponding data mask to the information of the data block if no data is to be written.
Before step S1), the method of this embodiment further includes a preconfigured step, specifically including:
identifying the space size of a storage medium and the byte number of a support block operation, setting the size of a data block according to the byte number of the support block operation, calculating the ratio of the space size of the storage medium to the size of the data block to obtain a maximum data index value, respectively setting the reserved data index numbers of a file system block area, a file index block area and a data index block area, and summing the reserved data index numbers to obtain a minimum data index value.
In step S1), the idle index table is generated according to the preset number of data indexes, that is, the information of each data block in the idle index table, that is, the value of the start address and the value of the length are calculated according to the minimum data index value and the maximum data index value. In this embodiment, for a new 1GByte storage hard disk, the file system writing unit calculates that the complete free space index table is that the starting address 0 is the minimum index value 576, the length 0 is the maximum index value (65536) minus the minimum index value (576) is equal to 64960, the starting address 1 is 0, the length 1 is also 0, and the last starting address 65536 and the length 65536 are all 0 in sequence.
After step S3), the method of this embodiment further includes a step of reading the file, specifically including:
SA 1) traversing all file index information, matching one or more of file time, file name or file size of a file to be read, if a matching result is obtained, finding a data index and a data mask corresponding to the first data block of the file to be read in a data index block area according to the corresponding value of the starting data index, and sequentially finding a data index and a data mask corresponding to the subsequent data block from the value of the data index;
SA 2) acquiring data of the file to be read stored in a storage medium according to the value of a data mask corresponding to each data block of the file to be read, determining an available index table corresponding to each data block according to the information of each data block, and outputting the data of the file to be read through an available channel corresponding to the available index table.
In step SA 1) of this embodiment, sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index specifically includes: if the current data index value is the sequence number of the next data index value, the two-dimensional array where the next data index is located is found, so that a corresponding data mask can be obtained, and if the current data index value is an end mark, it is indicated that the corresponding data block stores the last data of the file to be read, and therefore the searching is ended.
In step SA 2) of the present embodiment, the available index table and the available channels may not be the same as in step S2), because the free channels are not fixed during the system operation, the available channels and the available index table may be redetermined before step SA 2) of the present embodiment, that is: according to the number of the current available channels, the idle index table is segmented averagely to obtain a current available index table corresponding to each current available channel one by one, and for the current available index table, as the current available index table is only the index table obtained by segmenting the idle index table, only the combination mode of the information of the data blocks in the table is changed, for example, the original available index table contains the information of the data blocks corresponding to the data indexes 1 to 4, and the current available index table obtained by re-segmentation possibly contains the information of the data blocks corresponding to the data indexes 1 to 8, namely the number of the available channels is reduced by half. At the moment, the file to be read is subjected to multi-channel output through the reassigned available channels, so that dynamic multi-channel file reading and writing are realized.
After step S3), the method of this embodiment further includes a step of deleting the file, specifically including:
SB 1) traversing all file index information, matching one or more of file time, file name or file size of the file to be deleted, if a matching result is obtained, according to the value of the corresponding starting data index, finding the data index and the data mask corresponding to the first data block of the file to be deleted in a data index block area, and sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index;
SB 2) deleting the data of the file to be deleted stored in the storage medium according to the value of the data mask corresponding to each data block of the file to be read, and filling the data mask and the value of the corresponding data index to be empty.
Similarly, in step SB 1) of this embodiment, sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index specifically includes: if the current data index value is the sequence number of the next data index value, the two-dimensional array where the next data index is located is found, so that a corresponding data mask can be obtained, and if the current data index value is an end mark, it is indicated that the corresponding data block stores the last data of the file to be read, and therefore the searching is ended.
After step S3) of the present embodiment, the method further includes a defragmentation step, specifically including:
firstly traversing all file index information, respectively checking the file index information, wherein the checking modes in the embodiment include but are not limited to hash value checking, odd-even check, crc check and the like, if the checking is not passed, marking the file corresponding to the file index information as a file to be deleted, and according to the value of a starting data index in the file index information, finding a data index and a data mask corresponding to the first data block of the file to be deleted in a data index block area, and sequentially finding a data index and a data mask corresponding to the subsequent data block from the value of the data index; and deleting the data of the file to be deleted stored in the storage medium according to the value of the data mask corresponding to each data block of the file to be read, and filling the data mask and the value of the corresponding data index into the space.
And traversing all the data indexes of the data index block area, backtracking the data indexes with non-null values, filling the data indexes and the values of the corresponding data masks to be null if no backtracking result exists in the file index block area, and deleting the file data stored in the storage medium corresponding to the data masks.
According to the first embodiment, in this embodiment, for each functional unit of the embedded file system, the following is adopted:
the file system initial unit completes the following tasks:
checking redundancy check bits of a file system block area, and if the redundancy check bits are correct, successfully initializing;
initializing the number of files to 0;
initializing a total space;
the number of bytes of the initialization mask data block is 512 x 32byte
Initializing a used file index and an unused file index;
initializing a minimum data index and a maximum data index;
initializing a residual space;
redundancy check bits are initialized.
The file system write unit accomplishes the following tasks:
traversing from the minimum data index to the maximum data index, and dividing the idle space index table corresponding to each channel into available index tables corresponding to files to be written in the channels one by one;
Writing data into each data block in the available index table in turn, and updating the corresponding data mask and the value of the data index in the data index block area until the writing is finished and an end mark is set;
and updating the written files of the file index block area and the number of files of the file system block area, the residual space, the used file index, the unused file index and the redundancy check bit according to the information of the data index block area.
The file system read unit accomplishes the following tasks:
traversing the file index block area, finding file information of the file name guide, and finding a corresponding data index in the data index block according to the file start data index;
and sequentially finding the subsequent data indexes according to the data indexes until the value of the subsequent data indexes is an end mark.
The file deleting unit completes the following tasks:
traversing the file index, finding file information of the file name guide, and finding the next data index;
deleting the mask and clearing the current data index to be empty until an end mark is found;
updating the file information and updating the file system.
The defragmentation unit accomplishes the following tasks:
throughout the file index, errors are checked, files are deleted and the file system is updated.
Traversing the data index and the file index, finding the data index that is not empty but the file index is not found, making it clear that the data index is not empty, and updating the file system.
The foregoing is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. While the invention has been described with reference to preferred embodiments, it is not intended to be limiting. Therefore, any simple modification, equivalent variation and modification of the above embodiments according to the technical substance of the present invention shall fall within the scope of the technical solution of the present invention.
Claims (9)
1. An embedded file system supporting multiple channels, comprising:
a file system writing unit, configured to generate an idle index table according to a preset maximum data index value and a preset minimum data index value, where the idle index table includes information of at least one data block, the information of the data block includes a start address in a storage medium and a length in the storage medium, segment the idle index table according to the number of available channels, obtain an available index table corresponding to each available channel one by one, synchronously write file data of each available channel into the storage medium according to the information of the data block of the available index table, and update a data index block area, a file index block area and a file system block area in a storage data structure of the embedded file system;
A file system reading unit, configured to find file index information of a file to be read in a file index block area in a storage data structure of the embedded file system, match corresponding data indexes and data masks in the data index block area according to a start data index value in the file index information, where the data indexes, the data masks and the data blocks are in one-to-one correspondence, find file data to be read in a storage medium according to the data masks, and output the data of the file to be read through an available channel corresponding to the data indexes;
the file deleting unit is used for finding file index information of a file to be deleted in a file index block area in a storage data structure of the embedded file system, matching corresponding data indexes and data masks in the data index block area according to a start data index value in the file index information, finding file data to be deleted in a storage medium according to the data masks, and then deleting the file data to be deleted;
the defragmentation unit is used for traversing all file index information in a file index block area in a storage data structure of the embedded file system, deleting files which are not checked after checking, traversing all data indexes in a data index block area in the storage data structure of the embedded file system, and deleting data indexes which are not occupied by the files but are not empty;
The storage data structure of the embedded file system comprises:
the file system block area comprises a file system version, a file number, a total space, a residual space, a minimum data index, a maximum data index, a used file index, an unused file index, a mask data block byte number and a redundancy check bit, and the data block comprises 32 mask data blocks;
the file index block area comprises file index information of each file, wherein the file index information comprises a file start data index value, a file time, a file name and a file size of the corresponding file;
the data index block area comprises a two-dimensional array formed by data index values and corresponding data masks, and the length of the two-dimensional array is the number of all data indexes of the data block;
the storage data structure of the embedded file system further comprises a data block area, the data block area comprises data blocks with the same size, the two-dimensional arrays in the data block and the data index block area are in one-to-one correspondence, and the data blocks are numbered according to the data index numbers in the corresponding two-dimensional arrays respectively so as to support parallel recording of the multi-channel file.
2. The multi-channel-supported embedded file system according to claim 1, further comprising a file initialization unit, wherein the file initialization unit is configured to identify a storage medium size and a byte number supporting a block operation, set a size of a data block according to the byte number supporting the block operation, calculate a ratio of a storage medium space size to the data block size to obtain a maximum data index value in the data index number, set the data index number reserved in the file system block area, the file index block area, and the data index block area, respectively, and sum the reserved data index number to obtain a minimum data index value in the data index number.
3. The control method of an embedded file system supporting multiple channels according to claim 1 or 2, comprising the steps of:
s1) generating an idle index table according to a preset maximum data index value and a preset minimum data index value, wherein the idle index table comprises information of at least one data block, and the information of the data block comprises a starting address in a storage medium and a length in the storage medium;
s2) dividing the idle index table into available index tables corresponding to available channels one by one, synchronously writing files in each available channel into a storage medium according to the information of each data block in the available index tables, and updating the values of data indexes and data masks corresponding to the data blocks and file index information corresponding to the files;
s3) updating the number of files, the residual space, the used file index, the unused file index and the redundancy check bit according to the updated data index and the file index information.
4. A method of controlling an embedded file system supporting multiple channels according to claim 3, wherein step S2) specifically comprises:
s21) according to the number of the available channels, the idle index table is segmented averagely, and the available index table corresponding to each available channel one by one is obtained;
S22) synchronously writing the file data to be written in each available channel into corresponding data blocks according to the information of the data blocks in the corresponding available index table, updating the data index corresponding to each data block to point to the data index corresponding to the next data block according to the information of the data blocks in the available index table, wherein the value of the data index corresponding to the last data block is an end mark, and updating the content of the data mask corresponding to each data block to be the information of the data block;
s23) generating file index information of the written file, and pointing the value of the start data index in the file index information to the data index of the first data block in the available index table, where the file index information in this embodiment includes the start data index, file time, file name and file size, and the value of the start data index is the sequence number of the data index of the first data block in the corresponding available index table in the data index block area.
5. The method for controlling an embedded file system supporting multiple channels according to claim 4, wherein the step S22) specifically comprises: and for the data of the file to be written in the current channel, sequentially writing the corresponding data blocks according to the information of the data blocks in the corresponding available index table, continuously writing the data into the next data block according to the information of the next data block in the available index table after each data block is fully written, and directing the data index corresponding to the current data block to the data index corresponding to the next data block, updating the content of the data mask of the current data block pair to the information of the current data block, and updating the value of the corresponding data mask to the end mark and updating the value of the corresponding data mask to the information of the data block if no data is to be written.
6. A method of controlling an embedded file system supporting multiple channels according to claim 3, wherein step S1) is preceded by the step of pre-configuring, specifically comprising: identifying the space size of a storage medium and the byte number of a support block operation, setting the size of a data block according to the byte number of the support block operation, calculating the ratio of the space size of the storage medium to the size of the data block to obtain a maximum data index value, respectively setting the reserved data index numbers of a file system block area, a file index block area and a data index block area, and summing the reserved data index numbers to obtain a minimum data index value.
7. The method for controlling an embedded file system supporting multiple channels according to claim 3, wherein step S3) further comprises a step of reading a file, specifically comprising:
SA 1) traversing all file index information, matching one or more of file time, file name or file size of a file to be read, if a matching result is obtained, finding a data index and a data mask corresponding to the first data block of the file to be read in a data index block area according to the corresponding value of the starting data index, and sequentially finding a data index and a data mask corresponding to the subsequent data block from the value of the data index;
SA 2) acquiring data of the file to be read stored in a storage medium according to the value of a data mask corresponding to each data block of the file to be read, averagely segmenting the idle index table according to the number of the current available channels to obtain a current available index table corresponding to each current available channel one by one, determining the current available index table corresponding to each data block according to the information of each data block, and outputting the data of the file to be read through the current available channels corresponding to the current available index table.
8. The method for controlling an embedded file system supporting multiple channels according to claim 3, wherein step S3) further comprises a step of deleting the file, specifically comprising:
SB 1) traversing all file index information, matching one or more of file time, file name or file size of the file to be deleted, if a matching result is obtained, according to the value of the corresponding starting data index, finding the data index and the data mask corresponding to the first data block of the file to be deleted in a data index block area, and sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index;
SB 2) deleting the data of the file to be deleted stored in the storage medium according to the value of the data mask corresponding to each data block of the file to be read, and filling the data mask and the value of the corresponding data index to be empty.
9. The method for controlling a multi-channel enabled embedded file system according to claim 8, wherein step S3) further comprises a defragmentation step, specifically comprising:
firstly traversing all file index information, respectively checking the file index information, if the checking is not passed, marking the file corresponding to the file index information as a file to be deleted, and according to the value of the starting data index in the file index information, finding the data index and the data mask corresponding to the first data block of the file to be deleted in a data index block area, and sequentially finding the data index and the data mask corresponding to the subsequent data block from the value of the data index;
deleting the data of the file to be deleted stored in the storage medium according to the value of the data mask corresponding to each data block of the file to be read, and filling the data mask and the value of the corresponding data index into the space;
and traversing all the data indexes of the data index block area, backtracking the data indexes with non-null values, filling the data indexes and the values of the corresponding data masks to be null if no backtracking result exists in the file index block area, and deleting the file data stored in the storage medium corresponding to the data masks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111093259.1A CN113792023B (en) | 2021-09-17 | 2021-09-17 | Multi-channel supporting embedded file system and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111093259.1A CN113792023B (en) | 2021-09-17 | 2021-09-17 | Multi-channel supporting embedded file system and control method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113792023A CN113792023A (en) | 2021-12-14 |
| CN113792023B true CN113792023B (en) | 2023-11-21 |
Family
ID=78878778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111093259.1A Active CN113792023B (en) | 2021-09-17 | 2021-09-17 | Multi-channel supporting embedded file system and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113792023B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6571261B1 (en) * | 2000-07-13 | 2003-05-27 | International Business Machines Corporation | Defragmentation utility for a shared disk parallel file system across a storage area network |
| CN102129443A (en) * | 2010-12-30 | 2011-07-20 | 中国民航信息网络股份有限公司 | Real-time data transmission channel and method based on USAS (Univac Standard Airline Systems) host |
| CN103077200A (en) * | 2012-12-27 | 2013-05-01 | 深圳市赛为智能股份有限公司 | Audio/video index storage method and device |
| CN103885770A (en) * | 2014-01-21 | 2014-06-25 | 唐道成 | Implementation method for retrieving assembly files from executable files for single chip microcomputer |
| CN104731921A (en) * | 2015-03-26 | 2015-06-24 | 江苏物联网研究发展中心 | Method for storing and processing small log type files in Hadoop distributed file system |
| CN107729558A (en) * | 2017-11-08 | 2018-02-23 | 郑州云海信息技术有限公司 | Method, system, device and the computer-readable storage medium that file system fragmentation arranges |
| CN110647497A (en) * | 2019-07-19 | 2020-01-03 | 广东工业大学 | HDFS-based high-performance file storage and management system |
| CN111142780A (en) * | 2018-11-06 | 2020-05-12 | 深圳市茁壮网络股份有限公司 | Large file storage file system and large file processing method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110153972A1 (en) * | 2009-12-23 | 2011-06-23 | Quantum Corporation | Free space defragmention in extent based file system |
| GB2516864A (en) * | 2013-08-02 | 2015-02-11 | Ibm | Increased instruction issue rate and latency reduction for out-of-order processing by instruction chaining and collision avoidance |
-
2021
- 2021-09-17 CN CN202111093259.1A patent/CN113792023B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6571261B1 (en) * | 2000-07-13 | 2003-05-27 | International Business Machines Corporation | Defragmentation utility for a shared disk parallel file system across a storage area network |
| CN102129443A (en) * | 2010-12-30 | 2011-07-20 | 中国民航信息网络股份有限公司 | Real-time data transmission channel and method based on USAS (Univac Standard Airline Systems) host |
| CN103077200A (en) * | 2012-12-27 | 2013-05-01 | 深圳市赛为智能股份有限公司 | Audio/video index storage method and device |
| CN103885770A (en) * | 2014-01-21 | 2014-06-25 | 唐道成 | Implementation method for retrieving assembly files from executable files for single chip microcomputer |
| CN104731921A (en) * | 2015-03-26 | 2015-06-24 | 江苏物联网研究发展中心 | Method for storing and processing small log type files in Hadoop distributed file system |
| CN107729558A (en) * | 2017-11-08 | 2018-02-23 | 郑州云海信息技术有限公司 | Method, system, device and the computer-readable storage medium that file system fragmentation arranges |
| CN111142780A (en) * | 2018-11-06 | 2020-05-12 | 深圳市茁壮网络股份有限公司 | Large file storage file system and large file processing method |
| CN110647497A (en) * | 2019-07-19 | 2020-01-03 | 广东工业大学 | HDFS-based high-performance file storage and management system |
Non-Patent Citations (4)
| Title |
|---|
| Backup to the Future: How Workload and Hardware Changes Continually Redefine Data Domain File Systems;Yamini Allu 等;《Computer》;第50卷(第7期);64-72 * |
| Concurrent log-structured memory for many-core key-value stores;Alexander Merritt 等;《Proceedings of the VLDB Endowment》;第11卷(第4期);458-471 * |
| 基于存储测试的大容量数据处理技术研究;孟庆;《中国优秀硕士学位论文全文数据库 信息科技辑》(第08(2017)期);I137-17 * |
| 虚拟化环境中资源管理机制的优化研究;郭帆;《中国博士学位论文全文数据库 信息科技辑》(第08(2019)期);I137-3 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113792023A (en) | 2021-12-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111506269B (en) | Disk storage space allocation method, device, equipment and storage medium | |
| US8285757B2 (en) | File system for a storage device, methods of allocating storage, searching data and optimising performance of a storage device file system | |
| WO2012168960A1 (en) | Semiconductor storage apparatus and method of controlling semiconductor storage apparatus | |
| CN103581331B (en) | The online moving method of virtual machine and system | |
| WO2012056491A1 (en) | Storage apparatus and data control method | |
| CN107766374B (en) | Optimization method and system for storage and reading of massive small files | |
| US6636941B1 (en) | Enhanced stable disk storage | |
| EP3963853B1 (en) | Optimizing storage and retrieval of compressed data | |
| CN108228479B (en) | Embedded FLASH data storage method and system | |
| CN111007990B (en) | Positioning method for quickly positioning data block references in snapshot system | |
| CN115639970B (en) | Storage virtualization method, system, device and equipment based on disk array card | |
| CN105243027A (en) | Method for storing data in storage device and memory controller | |
| CN109407985A (en) | A kind of method and relevant apparatus of data management | |
| CN104572492A (en) | Method and apparatus of burning data to FAT (File Allocation Table)32 partition | |
| DE102021131101A1 (en) | Storage facility with data deduplication, operation method of the storage facility and operation method of a storage server | |
| KR101805828B1 (en) | Method for translating address of storage system and memory device controller for storage system | |
| CN113792023B (en) | Multi-channel supporting embedded file system and control method thereof | |
| CN103530067A (en) | Data operation method and device | |
| CN116662019B (en) | Request distribution method and device, storage medium and electronic device | |
| CN116069685B (en) | Storage system write control method, device, equipment and readable storage medium | |
| US20100217787A1 (en) | Controller, information recording device, access device, information recording system, and information recording method | |
| EP4130970A1 (en) | Data processing method and apparatus in storage system, and computer readable storage medium | |
| CN115328859A (en) | Metadata access method, device, equipment and medium | |
| CN112035380B (en) | Data processing method, device and equipment and readable storage medium | |
| CN113886350A (en) | Data processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |