CN117271440A - File information storage method, reading method and related equipment based on freeRTOS - Google Patents
File information storage method, reading method and related equipment based on freeRTOS Download PDFInfo
- Publication number
- CN117271440A CN117271440A CN202311554890.6A CN202311554890A CN117271440A CN 117271440 A CN117271440 A CN 117271440A CN 202311554890 A CN202311554890 A CN 202311554890A CN 117271440 A CN117271440 A CN 117271440A
- Authority
- CN
- China
- Prior art keywords
- file
- information
- sub
- storage area
- offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000012216 screening Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000012795 verification Methods 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 239000002699 waste material Substances 0.000 abstract description 14
- 230000006870 function Effects 0.000 description 8
- 238000013500 data storage Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a file information storage method, a reading method and related equipment based on a freeRTOS, which relate to the field of file information storage and are used for determining the size of a target file according to the size and the number of a sub-file set; the target file is divided into a file header storage area, a file name index storage area, a file information storage area and a text storage area, and text offset and offset of the sub-file information storage area are determined according to information of each sub-file, so that the whole file structure is compact, and space waste caused by using a LittleFS file system is avoided. Further, the storage sequence of each subfile is determined by summing the hash value of the subfile name and the number of subfiles, the file structure is concise by storing the offset of the subfile information with collision in the file information storage area, and the storage and the reading are more convenient and rapid by linking the file name index area and the file information storage area, so that the storage and the reading efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of file information storage, and in particular, to a file information storage method, a file information reading method and related devices based on freeRTOS.
Background
With the miniaturization of devices, there is an increasing demand for operating systems. Embedded real-time operating systems are becoming more and more widely used in the embedded field. The adoption of the embedded real-time operating system (RTOS) can more reasonably and effectively utilize the resources of the CPU, simplify the design of application software, shorten the development time of the system and better ensure the real-time performance and the reliability of the system.
However, under the freeRTOS operating system, the data is directly stored in flash without a file system by default, and in order to facilitate application development, a third-party file system can be transplanted, and more commonly, littleFs is used. However, under littleFS, the block size is usually a fixed size, usually 4KB, and even if the file content is only 1KB, the file system allocates 4KB of space, resulting in 3KB of space waste, which is greater if there are more files to be stored. For example, when the stored files are all 1KB, it causes a space waste of 3kb×the number of files, which is fatal to a device having a small storage space.
There is also a need for a non-embedded system or server to merge files. However, the reading speed is more important in these systems, and the problem of storage space is not considered. Usually, an index is established independently, and when the data needs to be read, the storage position of the target file is searched through the index, the data is returned to the system, and the system acquires corresponding data from the data pool according to the storage position.
Therefore, there is an urgent need for a method for storing file information and a combined file structure to solve the problems of low space utilization and low read/write efficiency of the existing littleFS.
Disclosure of Invention
The embodiment of the application provides a file information storage method, a file information reading method and related equipment based on a freeRTOS, which can combine and store a plurality of files in the combined file, and quickly and accurately locate and read file contents through file names.
In one aspect, an embodiment of the present application provides a method for storing file information based on freeRTOS, where the method includes:
screening and determining a plurality of subfiles to obtain a subfile set to be processed;
determining the size of the target file according to the size of the sub-file set and the number of the sub-files;
dividing the storage space of the target file into a file header storage area, a file name index storage area, a file information storage area and a text storage area;
determining storage areas of the file header storage area and the file name index storage area;
writing file header information in the file header storage area;
initializing the file name index storage area and the file information storage area;
sequentially determining file information storage offset of each sub-file of the sub-file set, and updating the file name index storage area information;
writing and updating the file information storage area and writing each sub-file into the text storage area.
Further, the header information includes a target file identifier, a number of subfiles, a body offset, and a check bit, and the writing the header information in the header storage area includes:
writing target file identifier information and sub-file quantity information;
calculating to obtain text offset according to the number of subfiles and the length of each subfile name;
writing text offset information;
and writing verification information of the target file.
Further, determining the file information storage offset of each sub-file of the sub-file set in turn, and updating the file name index storage area information, including:
extracting the file name of the current sub-file, and carrying out hash calculation on the file name of each sub-file to obtain hash values of each sub-file name;
the hash value of the current sub-file name is calculated and calculated with the number of the sub-files;
determining the position of the index area of the current sub-file name and the offset of the information storage area of the current sub-file according to the residual result;
and writing the offset of the current sub-file information storage area into and updating the corresponding file name index area information.
Further, writing and updating the file information storage area and writing each subfile into the body storage area includes:
determining and obtaining the text offset of the current subfile according to the residual result of the current subfile;
determining a storage area of the current sub-file information according to the offset of the current sub-file information storage area;
writing the sub-file name size, the sub-file text offset and the sub-file name into a current sub-file information storage area;
and moving the current sub-file to the text offset area of the current sub-file, and writing the current sub-file.
Further, the file information storage area further includes an offset of hash collision next sub-file information, and the method further includes:
judging whether collision exists in the residual result of the current subfile;
if the sub file exists, the text offset of the current sub file information storage area is redetermined;
and updating the file information offset of the current sub-file, and writing the file information offset of the current sub-file into the hash of the previous file to collide with the offset of the next sub-file information.
Further, the method comprises:
extracting the file names of all the subfiles, and carrying out hash calculation on the file names of all the subfiles to obtain hash values of all the subfiles;
the hash value of each sub-file name is calculated and calculated with the quantity of the sub-files;
sorting the residual results corresponding to the subfiles;
obtaining the storage sequence of each sub-file according to the sequencing result;
and updating the file name index storage area information one by one according to the storage sequence of each sub-file, and writing and updating the file information storage area and the text storage area one by one.
A second aspect of the present application provides a method for reading file information based on freeRTOS, where the method includes:
acquiring a sub-file name to be read;
calculating a hash value according to the subfolder to obtain the position of the file in a file name index area;
obtaining the storage offset of the corresponding file information area according to the position of the file name index area;
acquiring the text offset and the file size of the current sub-file according to the offset of the file information area;
and positioning to the text position of the subfile according to the text offset and the file size of the current subfile, and reading the text information of the subfile according to the file size.
Further, the obtaining the text offset and the file size of the current sub-file according to the storage offset of the file information area includes:
positioning according to the storage offset of the file information area;
judging whether the file names are the same, if so, reading text offset information and file size information;
if the information areas are different, obtaining the offset information of the next sub-file information of the hash collision, and circularly searching for the information of the next file information area until the text offset and the file size of the current sub-file are found.
A third aspect of the present application provides a computer device comprising a memory and a processor, said memory storing a computer program, characterized in that said processor implements the aforementioned method steps when executing said computer program.
A fourth aspect of the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the aforementioned method steps.
In the embodiments of the application, a plurality of files are merged and stored in the merged file, and file contents are rapidly and accurately positioned and read out through file names, so that the waste of storage space is reduced, and the reading and writing efficiency is improved. Specifically, the file structure is changed, the storage and reading methods of the file information are changed, and the size of the target file is determined according to the size and the number of the sub-file sets; and the target file is divided into a file header storage area, a file name index storage area, a file information storage area and a text storage area, and text offset and offset of the sub-file information storage area are determined according to information of each sub-file, so that the whole file structure is compact, the files are sequentially arranged, and more space waste is avoided under the condition of using a LittleFS file system. Further, in the embodiments of the present application, the hash value of the subfile name and the number of subfiles are calculated to determine the storage sequence of each subfile, and the offset of the subfile information with collision is stored in the file information storage area, so that the file structure is concise, the index table of the relevant information of the file name and the storage position is not required to be stored separately, and the storage and the reading are more convenient through the linkage of the file name index area and the file information storage area. The target file uses the hash value of the file name to calculate the position, so that the target file can be quickly positioned to the specific position of the subfile, the searching times are reduced, and the speed of accessing the subfile is increased.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for storing information based on a freeRTOS file according to an embodiment of the present application;
fig. 2 is a flowchart of a method for reading file information based on freeRTOS according to an embodiment of the present application.
Detailed Description
The embodiment of the application provides a file information storage method based on a freeRTOS, which is used for merging and storing a plurality of files in the merged file, and rapidly and accurately positioning the file content and reading the file content through file names, so that the waste of storage space is reduced, and the reading and writing efficiency is improved. Specifically, the file structure is changed, the storage and reading methods of the file information are changed, and the size of the target file is determined according to the size and the number of the sub-file sets; and the target file is divided into a file header storage area, a file name index storage area, a file information storage area and a text storage area, and text offset and offset of the sub-file information storage area are determined according to information of each sub-file, so that the whole file structure is compact, the files are sequentially arranged, and more space waste is avoided under the condition of using a LittleFS file system.
Further, in the embodiments of the present application, the hash value of the subfile name and the number of subfiles are calculated to determine the storage sequence of each subfile, and the offset of the subfile information with collision is stored in the file information storage area, so that the file structure is concise, the index table of the relevant information of the file name and the storage position is not required to be stored separately, and the storage and the reading are more convenient through the linkage of the file name index area and the file information storage area.
The file information storage method provided in the embodiment of the present application will be described below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart of a file information storage method according to an embodiment of the present application. As shown in fig. 1, the data processing method includes the following steps S1 to S8.
S1, screening and determining a plurality of subfiles to obtain a subfile set to be processed;
FreeRTOS is a mini real-time operating system kernel. As a lightweight operating system, the functions include: task management, time management, semaphores, message queues, memory management, recording functions, software timers, coroutines, etc., can substantially meet the needs of smaller systems. In FreeRTOS, a commonly used file system is LittleFS. Although LittleFS has the advantages of power-down recovery, erasure equalization, etc., it also has significant drawbacks. The space utilization rate of LittleFS is limited to a certain extent, and the block size of LittleFS determines the minimum unit of file storage; the larger the block, the more likely it is that space is wasted. For example, a block size of 32K, even if the file content is only 1kb, littlefs allocates 32K space for it, resulting in 31K space waste. Even at present, the block of LittleFS is at least 4K, and when the file content is only 1KB, there is still a great waste.
It will be appreciated that in FreeRTOS, a large number of subfiles are typically stored, and these subfiles may be system subfiles, subfiles stored by a user, and may also be fragment files of an application, etc. The sub-files have different file sizes, and if the existing LittleFS file system is adopted, a large space waste is caused. Therefore, in the present embodiment, by adopting the file storing method of the present application, the above-described drawbacks can be overcome.
The method and the device can obtain the set of the subfiles to be processed by screening the subfiles. It is understood that the present application does not employ the file storage method of the present application for all subfiles. Since the size of the block is specified in LittleFS, when the size of the data file is the same as the size of the block specified in LittleFS, the data file can fully utilize storage resources and does not cause resource waste. Therefore, in this embodiment, a to-be-processed sub-file set is obtained by analyzing and screening a plurality of sub-files, and the to-be-processed sub-file set adopts the file storage method of the application.
In one embodiment, the screening of multiple subfiles may be based on their data size. Specifically, the file sizes of a plurality of subfiles are extracted, the remainder of the file sizes and the block sizes specified in LittleFS is calculated, and the subfiles corresponding to the remainder meet the preset condition and are used as subfiles to be processed through screening. For example, the block size specified in LittleFS may be 4K, and the preset condition may be that the remainder is not equal to 0, and in the process of screening, all subfiles with file sizes not equal to an integer multiple of 4 pass through the screening, so as to form a set of subfiles to be processed. It will be appreciated that the preset condition is that the remainder is not equal to 0, but one implementation, the preset condition may be that the remainder is less than a preset threshold, which may be set according to the block size specified in LittleFS, for example, half or three-quarters of the block size. The screening condition is to screen subfiles which can cause serious waste of storage space to form a to-be-processed subfile set, and the to-be-processed subfile set adopts the file information storage method so as to avoid the waste of the storage space.
S2, determining the size of the target file according to the size of the sub-file set and the number of the sub-files;
the target file is a combined file finally generated by adopting the file information storage method. In this embodiment, the target file stores the related data of the sub-file set, so that the size of the target file is determined according to the size of the sub-file set and the number of the sub-files.
S3, dividing the storage space of the target file into a file header storage area, a file name index storage area, a file information storage area and a text storage area.
The target file is the combined file finally generated by adopting the file information storage method. The system needs to store the related data of the sub-file set, and the target file also needs to contain corresponding identification information and index information so that the system can find the needed sub-file.
Therefore, in one embodiment of the present application, the storage space of the target file is divided into a file header storage area, a file name index storage area, a file information storage area, and a body storage area. The header storage area is a header of the target file, and stores related header information. Specifically, the header information includes: a 4-byte file identifier, a 2-byte number of subfiles, 4-byte body offset data, and 1-byte check bytes. The file identifier is used for identifying the target file, so that the system can quickly identify the target file; the number of subfiles is used for representing the number of subfiles stored in the target file; the text offset data is an offset value of a text storage area in the target file; the check byte is used for checking the target file. It will be appreciated that the various parts of the header information are merely examples, which may be modified as desired and as appropriate.
In one embodiment of the application, the file index function is completed by adopting the file name index storage area and the file information storage area together, so that the coarse-fine file index function is realized, the storage space occupied by the index area is reduced, and the index efficiency is ensured. The file name index storage area is used for storing the offset of the file information storage area, and specific one or more subfiles can be quickly positioned through the offset of the file information storage area. The file information storage area stores detailed information of each sub-file, including: a 1 byte subfolder length, a 4 byte subfolder body offset, a 4 byte subfolder length, and a subfolder name. When the file is required to be read, specific one or more subfiles can be rapidly positioned through the offset of the file information storage area, and the text offset of the subfiles can be obtained through specific file information in the positioned file information storage area, so that the required subfile data can be accurately positioned from the file text storage area. In the application, the file names of the subfiles are not simply extracted, and the subfiles are directly stored and recorded in the storage positions; in this way, although the required subfiles can be found, the efficiency is low. In the method, a file name index storage area and a file information storage area are arranged in a target file, the file index function is completed by using information of the two areas, a coarse-fine file index function is realized, file information is accurately positioned through related information of a file name, accurate file information is obtained from the file information storage area, and complete data of subfiles are obtained from a file text storage area according to the obtained file information; the storage space occupied by the index area is reduced, and the index efficiency is ensured.
The file structure of the target file in this embodiment is shown in the following table, for example:
the file head storage area, the file name index storage area, the file information storage area and the text storage area respectively reserve corresponding storage spaces according to the above table so as to update and write subsequent information.
S4, determining a storage area of the file header storage area and the file name index storage area;
the embodiment is a file information storage method based on a freeRTOS, after screening a to-be-processed sub-file set, determining the size of a target file according to the size of the sub-file set and the number of the sub-files, calculating the size of each region of the target file, and reserving a corresponding space for writing the file into a corresponding region.
In one embodiment of the present application, the header storage area stores the header of the target file. Specifically, the header information includes: a 4-byte file identifier, a 2-byte number of subfiles, 4-byte body offset data, and 1-byte check bytes. And the file name index storage area is used for storing the offset of the file information storage area, and the occupied space size is the number of subfiles multiplied by 4 bytes. The space occupied by the two parts is fixed, and in the case of determining the sub-file set, the storage areas of the file header storage area and the file name index storage area can be determined and reserved. As for the offset values in the two regions, it is possible to fill in after the subsequent determination.
S5, writing the file header information in the file header storage area;
after the storage areas of the file header storage area and the file name index storage area of the target file are determined, the file header information may be written. The file header information comprises a target file identifier, the number of subfiles, a text offset and a check bit, and the writing of the file header information comprises the following steps: writing target file identifier information and sub-file quantity information; calculating to obtain text offset according to the number of subfiles and the length of each subfile name; writing text offset information; and writing verification information of the target file. The file identifier, the number of subfiles, and the verification information in the header information may be written after the subfile set is determined. Therefore, before a large number of files are written, a part of preset data can be written in advance, and the pressure of the data to be written subsequently is relieved.
S6, initializing the file name index storage area and the file information storage area;
after screening the sub-file set to be processed, determining the size of the target file according to the size of the sub-file set and the number of the sub-files, calculating the size of each region of the target file and reserving a corresponding space so as to write the file into the corresponding region subsequently. The space occupied by the file name index storage area and the file information storage area is fixed, and under the condition of determining the sub-file set, the storage areas of the file header storage area and the file name index storage area can be determined and reserved, and in order to prevent other irrelevant data from being stored in the reserved space, the file name index storage area and the file information storage area need to be initialized before the data is written.
S7, sequentially determining file information storage offset of each sub-file of the sub-file set, and updating the file name index storage area information;
analyzing each sub-file in the sub-file set, and determining the storage space required by the file information storage area, the storage position of each sub-file stored in the file name index storage area and the offset of the file information storage area according to the file data size, the content of the file name and the length of the file name; and updating the offset of the file information storage area to the corresponding position of the sub-file name index storage area.
Specifically, extracting the file name of the current sub-file, and carrying out hash calculation on the file name of each sub-file to obtain hash values of each sub-file name; the hash value of the current sub-file name is calculated and calculated with the number of the sub-files; determining the position of the index area of the current sub-file name and the offset of the information storage area of the current sub-file according to the residual result; and writing the offset of the current sub-file information storage area into and updating the corresponding file name index area information. In this embodiment, the storage space of the file name index storage area is reserved according to the number of subfiles, where the reserved space is the number of subfiles×4 bytes, so as to ensure that the related information of the subfiles can be stored in the subfile name index storage area and the file information storage area. The storage position of the sub-file in the file name index storage area and the file information storage area is determined by carrying out hash on the file name of the sub-file and then summing up the number of the sub-file, so that the position of the sub-file can be quickly found and positioned in the file text storage area when the sub-file needs to be searched later.
S8, writing and updating the file information storage area and writing each sub-file into the text storage area.
After determining the position of the index area of the current sub-file name and the offset of the information storage area of the current sub-file according to the residual result, writing the related information of the sub-file, such as the sub-file name length, the sub-file text offset, the sub-file length, the sub-file name and the like, into the corresponding position of the information storage area of the file; and storing the sub-file data in the file body storage area.
Specifically, determining and obtaining the text offset of the current subfile according to the residual result of the current subfile; determining a storage area of the current sub-file information according to the offset of the current sub-file information storage area; writing the sub-file name size, the sub-file text offset and the sub-file name into a current sub-file information storage area; and moving the current sub-file to the text offset area of the current sub-file, and writing the current sub-file. It can be understood that the sub-file remainder result directly determines the storage position thereof, and corresponding information is written into the file name index storage area, the file information storage area and the file text storage area according to the storage position thereof; the storage space of each area is reserved for the sub-file set, so that compact writing of each sub-file can be ensured, and space waste caused by fixing the block size of 4K in the prior art is avoided.
Since the file names of the subfiles are different, hash values after hashing are also different, but in the embodiment of the application, the hash values and the number of subfiles are calculated to be the same, so that the calculation results of two or more subfiles are the same. In view of the above, the present application proposes a further solution.
In one embodiment of the present application, the file information storage area further includes an offset for hash collision of next subfile information, and 4 bytes of space may be allocated for the offset. After the sub-file remainder result is obtained, the result is analyzed, and whether collision exists in the current sub-file remainder result is judged. A collision may be understood as having two or more subfiles with the same result of computation. If collision exists, the text offset of the current sub-file information storage area is redetermined; and updating the file information offset of the current sub-file, and writing the file information offset of the current sub-file into the hash of the previous file to collide with the offset of the next sub-file information. When data is written, the sub-file information is more reasonably written into the target file through judging collision, so that the required file data can be accurately and quickly found through a coarse-to-fine positioning mode when the file is read. It will be understood that in this embodiment, the collided subfiles are stored in the file information storage area and the file body storage area at adjacent positions, and the storage area at one position is shared at the file name index storage area for storing the file information offset value of the first subfile of the collided subfile.
Further, the present application includes another embodiment. In the file writing process, each subfile of the screened to-be-processed subfile set can be analyzed, the file name of each subfile is extracted, and hash calculation is carried out on the file name of each subfile to obtain hash values of each subfile name; the hash value of each sub-file name is calculated and calculated with the quantity of the sub-files; sorting the residual results corresponding to the subfiles; obtaining the storage sequence of each sub-file according to the sequencing result; and updating the file name index storage area information one by one according to the storage sequence of each sub-file, and writing and updating the file information storage area and the text storage area one by one. Further, when the remainder results corresponding to the subfiles are ranked, if there are multiple files with the same remainder results, the information such as file priority or access frequency of the subfiles with the same remainder results can be further obtained, and the ranking is further performed. In this embodiment, by uniformly analyzing each subfile in the subfile set, extracting the file name of each subfile, and calculating the balance between the hash value of the subfile name and the number of the subfiles, the storage positions of each subfile in the target file may be ordered, according to the ordering result, the storage space of each region is reserved, and the corresponding data is written into the target file. In this embodiment, since the target file includes the file name index storage area and the file information storage area, the remainder of the sub-file is used to read the offset value of the file information stored in the file name index storage area, and further the sub-file symptom offset is further located through the file information storage area; however, when there is a collision, there is more than one corresponding file information at the offset value of the file information, if the file information arranged in front is not matched, it is necessary to locate the next file information by hashing the offset of the next sub-file information, and if there are more collision files, reading is slow. In this embodiment, by acquiring the information such as the file priority or the access frequency of the subfiles with the same residual result, the subfiles with important or high-frequency access are further ordered, so that the subfiles with important or high-frequency access are arranged at the front position, and can be preferentially read when the files are read, thereby improving the read-write efficiency.
Further, after the remainder is made for each sub-file, it is determined whether a collision occurs. If the number of collisions meets the preset condition, changing the structure of the file name index storage area, and adopting the mapping relation of remainder-file information offset value, so that only a storage space (the number of uncorrupted subfiles+the number of different remainders of the collided subfiles) x 6 bytes is reserved for the file name index storage area, wherein 4 bytes are used for storing the offset value of the file information, and 2 bytes are used for storing the number of subfiles. The preset condition may be that the number of collisions reaches a certain proportion, for example, 50%, and the like, and if the preset condition is satisfied, it is stated that most collisions occur after the information of the subfiles is found. For example, when there are 100 subfiles in the subfiles to be stored, after the file name is hashed and the number of the subfiles is calculated, the possible result is that 30 subfiles are not collided, 70 subfiles are collided, and the remainder of 40 subfiles in 70 is 2, and the remainder of 30 subfiles is 3. If the storage structure in the foregoing embodiment is adopted, a (100) ×4 byte storage space needs to be reserved for the file name index storage area, and the method of this embodiment is only required to reserve a (30+2) ×6 byte storage space for the file name index storage area. By analyzing the collision situation and combining the analysis result, the specific structure of the file name index storage area is determined so as to reserve the corresponding storage space, so that the storage space occupied by the file name index storage area is minimum.
Further, since each subfile in the subfile set is analyzed in advance in the present embodiment. Thus, the file structure of the target file can be further optimized for the analysis result. For the sub-file which is not collided, the offset information of the next sub-file information of the hash collision in the file information storage area can be deleted, so that more resources are saved.
In one embodiment, referring to fig. 2, the present application further provides a method for reading file information based on freeRTOS, where the method includes:
acquiring a sub-file name to be read;
calculating a hash value according to the subfolder to obtain the position of the file in a file name index area;
obtaining the storage offset of the corresponding file information area according to the position of the file name index area;
acquiring the text offset and the file size of the current sub-file according to the offset of the file information area;
and positioning to the text position of the subfile according to the text offset and the file size of the current subfile, and reading the text information of the subfile according to the file size.
It can be seen that, because the target file adopts the above structure, when the file is read, the hash value of the sub-file name and the number of files in the target file can be calculated to perform a rough positioning on the sub-file, after the position of the sub-file in the file name index area is obtained, the text offset and the file size of the current sub-file are obtained through the storage offset corresponding to the file information area, and then the text position of the sub-file is positioned, and the text information of the sub-file is read according to the file size. Therefore, when the subfiles are required to be read, the required subfile data can be quickly and accurately found.
Further, since there may be a collision, the obtaining the text offset and the file size of the current sub-file according to the storage offset of the file information area includes:
positioning according to the storage offset of the file information area;
judging whether the file names are the same, if so, reading text offset information and file size information;
if the information areas are different, obtaining the offset information of the next sub-file information of the hash collision, and circularly searching for the information of the next file information area until the text offset and the file size of the current sub-file are found. The target file uses the hash value of the file name to calculate the position, so that the target file can be quickly positioned to the specific position of the subfile, the searching times are reduced, and the speed of accessing the subfile is increased.
Accordingly, an exemplary embodiment of the present application also provides a computer device, including: a processor, a memory, a network interface, an input device, and an output device; the processor is connected to the memory, the network interface for providing network communication functions, the memory for storing program code, the input device for receiving input instructions to generate signal inputs related to settings and function control of the computer device, the output device for outputting data information, and the processor for invoking the program code to perform the data storage or reading methods in embodiments of the present application.
In addition, it should be noted that, in an exemplary embodiment of the present application, a computer readable storage medium is further provided, where a computer program is stored in the storage medium, where the computer program includes program instructions, and when one or more processors loads and executes the program instructions, the foregoing data storage or reading method may be described in the embodiment, which is not repeated herein, and a description of beneficial effects of using the same method is not repeated herein. It will be appreciated that the program instructions may be executed on one or more computer devices that are capable of communicating with each other.
The computer readable storage medium may be an internal storage unit of the data storage or reading method provided in any of the foregoing embodiments, for example, a hard disk or a memory of a computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card (flash card) or the like, which are provided on the computer device. Further, the computer-readable storage medium may also include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
The steps in the method of the embodiment of the application can be sequentially adjusted, combined and deleted according to actual needs.
The modules in the device of the embodiment of the application can be combined, divided and deleted according to actual needs.
The above disclosure is only a few examples of the present application, and it is not intended to limit the scope of the claims, and those of ordinary skill in the art will understand that all or a portion of the above-described embodiments may be implemented and that equivalent changes may be made in the claims of the present application while still falling within the scope of the present invention.
Claims (10)
1. A freeRTOS file information storage method, the method comprising:
screening and determining a plurality of subfiles to obtain a subfile set to be processed;
determining the size of the target file according to the size of the sub-file set and the number of the sub-files;
dividing the storage space of the target file into a file header storage area, a file name index storage area, a file information storage area and a text storage area;
determining storage areas of the file header storage area and the file name index storage area;
writing file header information in the file header storage area;
initializing the file name index storage area and the file information storage area;
sequentially determining file information storage offset of each sub-file of the sub-file set, and updating the file name index storage area information;
writing and updating the file information storage area and writing each sub-file into the text storage area.
2. The method of claim 1, wherein the header information includes a target file identifier, a number of subfiles, a body offset, and a check bit, and wherein writing the header information in the header storage area includes:
writing target file identifier information and sub-file quantity information;
calculating to obtain text offset according to the number of subfiles and the length of each subfile name;
writing text offset information;
and writing verification information of the target file.
3. The method of claim 1, wherein sequentially determining file information storage offsets for each subfile of the set of subfiles and updating the file name index storage area information comprises:
extracting the file name of the current sub-file, and carrying out hash calculation on the file name of each sub-file to obtain hash values of each sub-file name;
the hash value of the current sub-file name is calculated and calculated with the number of the sub-files;
determining the position of the index area of the current sub-file name and the offset of the information storage area of the current sub-file according to the residual result;
and writing the offset of the current sub-file information storage area into and updating the corresponding file name index area information.
4. A method according to claim 3, wherein writing the updated file information storage area and writing each subfile to the body storage area comprises:
determining and obtaining the text offset of the current subfile according to the residual result of the current subfile;
determining a storage area of the current sub-file information according to the offset of the current sub-file information storage area;
writing the sub-file name size, the sub-file text offset and the sub-file name into a current sub-file information storage area;
and moving the current sub-file to the text offset area of the current sub-file, and writing the current sub-file.
5. The method of claim 4, wherein the file information storage area further includes an offset for hash collision of next subfile information, the method further comprising:
judging whether collision exists in the residual result of the current subfile;
if the sub file exists, the text offset of the current sub file information storage area is redetermined;
and updating the file information offset of the current sub-file, and writing the file information offset of the current sub-file into the hash of the previous file to collide with the offset of the next sub-file information.
6. The method according to any one of claims 1-5, characterized in that the method comprises:
extracting the file names of all the subfiles, and carrying out hash calculation on the file names of all the subfiles to obtain hash values of all the subfiles;
the hash value of each sub-file name is calculated and calculated with the quantity of the sub-files;
sorting the residual results corresponding to the subfiles;
obtaining the storage sequence of each sub-file according to the sequencing result;
and updating the file name index storage area information one by one according to the storage sequence of each sub-file, and writing and updating the file information storage area and the text storage area one by one.
7. A freeRTOS-based file information reading method, the method comprising:
acquiring a sub-file name to be read;
calculating a hash value according to the subfolder to obtain the position of the file in a file name index area;
obtaining the storage offset of the corresponding file information area according to the position of the file name index area;
acquiring the text offset and the file size of the current sub-file according to the offset of the file information area;
and positioning to the text position of the subfile according to the text offset and the file size of the current subfile, and reading the text information of the subfile according to the file size.
8. The method of claim 7, wherein the obtaining the text offset and the file size of the current sub-file according to the storage offset of the file information area comprises:
positioning according to the storage offset of the file information area;
judging whether the file names are the same, if so, reading text offset information and file size information;
if the information areas are different, obtaining the offset information of the next sub-file information of the hash collision, and circularly searching for the information of the next file information area until the text offset and the file size of the current sub-file are found.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the method steps of any of claims 1 to 8 when the computer program is executed.
10. A computer readable storage medium storing a computer program, which when executed by a processor causes the processor to perform the method steps of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311554890.6A CN117271440B (en) | 2023-11-21 | 2023-11-21 | File information storage method, reading method and related equipment based on freeRTOS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311554890.6A CN117271440B (en) | 2023-11-21 | 2023-11-21 | File information storage method, reading method and related equipment based on freeRTOS |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117271440A true CN117271440A (en) | 2023-12-22 |
| CN117271440B CN117271440B (en) | 2024-02-06 |
Family
ID=89212799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311554890.6A Active CN117271440B (en) | 2023-11-21 | 2023-11-21 | File information storage method, reading method and related equipment based on freeRTOS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117271440B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5754844A (en) * | 1995-12-14 | 1998-05-19 | Sun Microsystems, Inc. | Method and system for accessing chunks of data using matching of an access tab and hashing code to generate a suggested storage location |
| US20030233381A1 (en) * | 2002-06-13 | 2003-12-18 | Hitachi, Ltd. | File management method |
| US20080228843A1 (en) * | 2006-08-25 | 2008-09-18 | Dan Dodge | Filesystem having a filename cache |
| US20140095458A1 (en) * | 2012-09-28 | 2014-04-03 | Samsung Electronics Co., Ltd. | Storage device for storing directory entries, directory entry lookup apparatus and method, and storage medium storing directory entry lookup program |
| CN104978355A (en) * | 2014-04-10 | 2015-10-14 | 炬芯（珠海）科技有限公司 | Operation method of CPIO file and operation device of CPIO file |
| CN105069048A (en) * | 2015-07-23 | 2015-11-18 | 东方网力科技股份有限公司 | Small file storage method, query method and device |
| US20150331619A1 (en) * | 2012-12-14 | 2015-11-19 | Tencent Technology (Shenzhen) Company Limited | Data storage method and apparatus |
| WO2016199230A1 (en) * | 2015-06-10 | 2016-12-15 | 株式会社日立製作所 | Storage system |
| CN107679182A (en) * | 2017-09-29 | 2018-02-09 | 华为技术有限公司 | A kind of catalogue collocation method and device |
| WO2018121430A1 (en) * | 2016-12-26 | 2018-07-05 | 贵州白山云科技有限公司 | File storage and indexing method, apparatus, media, device and method for reading files |
| CN111258499A (en) * | 2020-01-08 | 2020-06-09 | 杭州电子科技大学 | Small file system with variable sectors |
| CN111984591A (en) * | 2019-05-21 | 2020-11-24 | 北京京东尚科信息技术有限公司 | File storage method, file reading method, file storage device, file reading device, equipment and computer readable storage medium |
| CN113220644A (en) * | 2021-05-28 | 2021-08-06 | 北京微纳星空科技有限公司 | File processing method, device, equipment and storage medium |
| CN115017098A (en) * | 2022-06-09 | 2022-09-06 | 重庆川仪速达机电有限公司 | File system management method and device |
| CN115525605A (en) * | 2022-08-11 | 2022-12-27 | 上海睿赛德电子科技有限公司 | Transmission system and method of file system |
-
2023
- 2023-11-21 CN CN202311554890.6A patent/CN117271440B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5754844A (en) * | 1995-12-14 | 1998-05-19 | Sun Microsystems, Inc. | Method and system for accessing chunks of data using matching of an access tab and hashing code to generate a suggested storage location |
| US20030233381A1 (en) * | 2002-06-13 | 2003-12-18 | Hitachi, Ltd. | File management method |
| US20080228843A1 (en) * | 2006-08-25 | 2008-09-18 | Dan Dodge | Filesystem having a filename cache |
| US20140095458A1 (en) * | 2012-09-28 | 2014-04-03 | Samsung Electronics Co., Ltd. | Storage device for storing directory entries, directory entry lookup apparatus and method, and storage medium storing directory entry lookup program |
| US20150331619A1 (en) * | 2012-12-14 | 2015-11-19 | Tencent Technology (Shenzhen) Company Limited | Data storage method and apparatus |
| CN104978355A (en) * | 2014-04-10 | 2015-10-14 | 炬芯（珠海）科技有限公司 | Operation method of CPIO file and operation device of CPIO file |
| WO2016199230A1 (en) * | 2015-06-10 | 2016-12-15 | 株式会社日立製作所 | Storage system |
| CN105069048A (en) * | 2015-07-23 | 2015-11-18 | 东方网力科技股份有限公司 | Small file storage method, query method and device |
| WO2018121430A1 (en) * | 2016-12-26 | 2018-07-05 | 贵州白山云科技有限公司 | File storage and indexing method, apparatus, media, device and method for reading files |
| CN107679182A (en) * | 2017-09-29 | 2018-02-09 | 华为技术有限公司 | A kind of catalogue collocation method and device |
| CN111984591A (en) * | 2019-05-21 | 2020-11-24 | 北京京东尚科信息技术有限公司 | File storage method, file reading method, file storage device, file reading device, equipment and computer readable storage medium |
| CN111258499A (en) * | 2020-01-08 | 2020-06-09 | 杭州电子科技大学 | Small file system with variable sectors |
| CN113220644A (en) * | 2021-05-28 | 2021-08-06 | 北京微纳星空科技有限公司 | File processing method, device, equipment and storage medium |
| CN115017098A (en) * | 2022-06-09 | 2022-09-06 | 重庆川仪速达机电有限公司 | File system management method and device |
| CN115525605A (en) * | 2022-08-11 | 2022-12-27 | 上海睿赛德电子科技有限公司 | Transmission system and method of file system |
Non-Patent Citations (1)
| Title |
|---|
| 曹海涛;余永红;: "基于multimap映射的动态内存分配算法探究", 电脑知识与技术, no. 30 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117271440B (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3678346B1 (en) | Blockchain smart contract verification method and apparatus, and storage medium | |
| CN110287044B (en) | Lock-free shared memory processing method and device, electronic equipment and readable storage medium | |
| US9377959B2 (en) | Data storage method and apparatus | |
| US10628200B2 (en) | Base state for thin-provisioned volumes | |
| CN110765076A (en) | Data storage method and device, electronic equipment and storage medium | |
| CN113010116B (en) | Data processing method, device, terminal equipment and readable storage medium | |
| CN111694992A (en) | Data processing method and device | |
| CN108776665B (en) | Data processing method and device | |
| CN116880908B (en) | Instruction processing method and device, electronic equipment and readable storage medium | |
| CN117271440B (en) | File information storage method, reading method and related equipment based on freeRTOS | |
| CN112286968A (en) | Service identification method, equipment, medium and electronic equipment | |
| CN111857600A (en) | Data reading and writing method and device | |
| CN114816772B (en) | Debugging method, debugging system and computing device for application running based on compatible layer | |
| CN115221360A (en) | Tree structure configuration method and system | |
| CN112732649A (en) | OFD document signing method, electronic device and medium | |
| CN111143418A (en) | Data reading method, device and equipment for database and storage medium | |
| CN111400342A (en) | Database updating method, device, equipment and storage medium | |
| CN115037799B (en) | Current limiting method, device, equipment and medium | |
| CN113535660B (en) | Android log storage method and device | |
| CN111966295B (en) | Multi-journ recording method, device and medium based on ceph | |
| CN116431066B (en) | Data storage method, device, electronic equipment and storage medium | |
| CN110134691B (en) | Data verification method, device, equipment and medium | |
| CN112527745B (en) | Embedded file system multi-partition analysis method, terminal device and storage medium | |
| CN113031908B (en) | Ordered data processing method and computing device | |
| CN112181301A (en) | Data export method and device of cloud hard disk and server equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |