CN111427848B - Method for realizing high-speed spaceborne file system based on JSON - Google Patents
Method for realizing high-speed spaceborne file system based on JSON Download PDFInfo
- Publication number
- CN111427848B CN111427848B CN202010292816.1A CN202010292816A CN111427848B CN 111427848 B CN111427848 B CN 111427848B CN 202010292816 A CN202010292816 A CN 202010292816A CN 111427848 B CN111427848 B CN 111427848B
- Authority
- CN
- China
- Prior art keywords
- file system
- file
- speed
- json
- storage device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a method for realizing a high-speed satellite-borne file system based on JSON, which comprises the following steps: s1: transplanting a cJSON library for the current satellite-borne embedded operating system, wherein the cJSON adopts a linked list to store management key/value pairs; s2: building a structure body composed of a plurality of fields, wherein the fields form node attributes of a file system so as to provide attribute interfaces for application program development; s3: dividing the number and the address of the logic units of the storage device according to the capacity and the load data size of the storage device; s4: establishing a function library; s5: the logical unit address information of the storage device of the file is recorded by using a Vector container of C++. The invention realizes the management of the fixed-size high-speed load data based on the Vector by utilizing the cJSON library and the C++ container, can improve the read-write speed of the fixed-length load data aiming at the satellite-borne embedded system, and provides a technical approach for realizing the high-speed access and storage of the load data.
Description
Technical Field
The invention relates to the technical field of satellite-borne file systems, in particular to a method for realizing a high-speed satellite-borne file system based on JSON.
Background
With the rapid development of satellite industry in China, the application load data volume is larger and larger. The space-borne computer is required to greatly improve the storage and read-write performance. After the spaceborne computer adopts an operating system, a user can customize service more conveniently. The complex file system of the operating system can be utilized to facilitate and facilitate the development of the integrated electronic system software, but the complex file system is utilized, so that the read-write speed can not meet the requirement of high-speed load.
Due to the large volume of applications of satellite payload data, there is an urgent need to increase the file system rate of on-board computers. Based on the fixed-length load data, a high-speed satellite-borne file system is designed, so that the requirement of the speed on the satellite is met; the satellite-borne computer can only improve the overall performance index of the satellite by improving the performance of a file system.
The file system based on the operating system has wide application in space navigation satellite-borne computers, and in the existing technology for realizing the lifting rate, the following two implementation modes are mainly adopted:
1. the simple file system is realized by utilizing the FPGA to control the memory, but the simple file system is not compatible with the existing embedded operating system interface, so that the development difficulty of application software is high;
2. the file system of the operating system is utilized, but the file system is too complex to meet the performance requirements.
Disclosure of Invention
The invention aims to provide a method for realizing a high-speed satellite-borne file system based on JSON, which aims to solve the problems of high software development difficulty and difficulty in meeting system performance requirements of the existing satellite-borne file system.
In order to achieve the above purpose, the present invention provides a method for implementing a high-speed satellite-borne file system based on JSON, comprising the following steps:
s1: transplanting a cJSON library for a current satellite-borne embedded operating system, wherein the cJSON adopts a linked list to store management key/value pairs;
s2: building a structure body composed of a plurality of fields, wherein the fields form node attributes of a file system so as to provide attribute interfaces for application program development;
s3: dividing the number and the address of the logic units of the storage device according to the capacity and the load data size of the storage device;
s4: establishing a function library, wherein the function library comprises: creating a file, writing the file, closing the file, inquiring the data length of the load type, inquiring the file number, reading the file, deleting the file, storing the file system, initializing the file system, emptying the file system and acquiring a function library of the size of the file system;
s5: the logical unit address information of the storage device of the file is recorded by using a Vector container of C++.
Preferably, the cJSON is a JSON parser implemented by pure C, and the cJSON records the service condition of the storage device and releases the memory after the use.
Preferably, in the step S2, the field includes: file number, payload type, task number, start time, end time, file size, and node queue.
Preferably, in the step S2, the structure body forms a system attribute of the file system, and is used for maintaining attribute information of the load data in the memory.
Preferably, in the step S3, when the address and number information of the logic units are established according to the capacity and the load data size of the storage device, the storage device is divided into logic units with fixed lengths.
Preferably, in the step S3, the addresses of the logical units of the file system node are: current node number payload data size.
Preferably, the step S4 further includes an initialization process and a save process for designing the high-speed on-board file system, where the initialization process includes: analyzing the file information and node information which are recorded currently and whether the address of the initialization logic unit is recorded or not from the JSON data; the preservation flow comprises the following steps: the node information is stored in RAM to speed up the access of the node attribute of the storage device during the data recording process, and is stored in the hard disk in JSON format after the recording is finished.
Preferably, the step S4 further includes: the function library is a programming interface of a file system, and the load application program realizes a high-speed file system according to the programming interface.
Preferably, the step S5 further includes: and recording the logical unit address of the file by using a Vector container, wherein the logical unit address is a virtual address, and when the virtual address is converted into a physical address, the controller determines the mapping relation between the logical address and the physical address.
Preferably, the file system manages the usage of the individual logical units.
Compared with the prior art, the method for improving the read-write rate of the satellite-borne embedded fixed-length satellite-borne file system is provided by aiming at the satellite-borne file system and using the technologies such as cJSON, vector and logic address, and solves the problem that the storage performance and the read-write performance of the current satellite-borne computer system are limited by a complex embedded file system, so that the IO performance index of the satellite-borne computer system is greatly improved.
The invention uses the Vector to record whether the node has data, the time complexity is O (n), the algorithm advantage is exerted, and the processor utilization rate of the satellite-borne embedded system is reduced. Aiming at the satellite-borne file system, the invention can effectively improve the read-write speed of the load data and provides an effective technical approach for realizing high-speed access and storage of the data.
Drawings
FIG. 1 is a flow chart of a method for implementing a high-speed on-satellite file system based on JSON provided by a preferred embodiment of the invention;
FIG. 2 is a schematic diagram of a high-speed fixed length satellite-borne file system according to a preferred embodiment of the present invention;
FIG. 3 is a flow chart of recording data provided by a preferred embodiment of the present invention;
FIG. 4 is a flow chart of data saving according to a preferred embodiment of the present invention.
Detailed Description
The following description and the discussion of the embodiments of the present invention will be made more complete and less in view of the accompanying drawings, in which it is to be understood that the invention is not limited to the embodiments of the invention disclosed and that it is intended to cover all such modifications as fall within the scope of the invention.
For the purpose of facilitating an understanding of the embodiments of the present invention, reference will now be made to the drawings, by way of example, of specific embodiments, and the various embodiments should not be construed to limit the embodiments of the invention.
As shown in fig. 1, the present embodiment provides a method for implementing a high-speed on-satellite file system based on JSON, which includes the following steps:
s1: transplanting a cJSON library for a current satellite-borne embedded operating system, wherein the cJSON adopts a linked list to store management key/value pairs;
s2: building a structure body composed of a plurality of fields, wherein the fields form node attributes of a file system so as to provide attribute interfaces for application program development;
s3: dividing the number and the address of the logic units of the storage device according to the capacity and the load data size of the storage device;
s4: establishing a function library, wherein the function library comprises: creating a file, writing the file, closing the file, inquiring the data length of the load type, inquiring the file number, reading the file, deleting the file, storing the file system, initializing the file system, emptying the file system and acquiring a function library of the size of the file system;
s5: the logical unit address information of the storage device of the file is recorded by using a Vector container of C++. Here, the Vector container easily adds and deletes node location information, providing a good programming interface for applications.
Referring to fig. 2, in the high-speed fixed-length satellite-borne file system implemented by the method of the present invention, the load data and the load type are respectively resolved and stored in the corresponding logical unit addresses of the corresponding storage devices of the SSD (storage device) corresponding to the Vector container in the Vector node location table by using the csson.
The cJSON in this embodiment is a JSON parser implemented by pure C (ANSI C), and the use condition of the storage device is recorded through the cJSON, and the memory is released after the use is completed. The cJSON library compiling and transplanting method used in the embodiment has good universality and can be well compatible with the current embedded operating system.
Preferably, in the above step S2, the fields constituting the structure include: file number (int), payload type (char), task number (char), start time (uint64_t), end time (uint64_t), file size (int), node queue (deque < uint64_t >), etc. These fields constitute node properties of the file system, which are advantageous for developing applications. After the structural body formed by the fields is established, the structural body forms the system attribute of the file system and is used for maintaining the attribute information of the load data in the memory, so that the efficiency of searching the logical unit address is improved.
Further preferably, in the step S3, the storage device is divided into logical units of a fixed length when address and number information of the logical units are established according to the capacity (M) and the load data size (S) of the storage device. After division, the address of each logic unit of the storage device can be designed, wherein:
number of storage device logical units = storage device capacity (M)/payload data size(s);
address of logical unit of file system node = current node number (i) payload data size(s).
Further preferably, the step S4 further includes an initialization process and a save process for designing the high-speed on-board file system. Referring to fig. 3, the initialization process includes: and analyzing the file information and the node information which are recorded currently from the JSON data, and initializing whether the logical unit address is recorded. Specifically, firstly, recording the load data type N, then judging whether the data type is created, if not, directly ending the process, if yes, entering the next step, judging whether the current node position is larger than or equal to the total node number in the next step, if yes, recording the load data type N, if not, recording the data to the corresponding Bitmap position, and then ending the current initialization flow.
With further reference to fig. 4, the save process includes: the node information is stored in RAM to speed up the access of the node attribute of the storage device during the data recording process, and is stored in the hard disk in JSON format after the recording is finished. Specifically, firstly traversing the file queue to judge whether the file is ended, if so, directly storing the character string into the file and ending, and if not, entering the next step. In the next step, firstly, converting the file number, the data type and the like into cJSON attributes, then traversing the logic unit queue of the file and judging whether the file is finished, if not, converting into the cJSON attributes, adding the cJSON queue, then returning to continue traversing the logic unit queue of the file and judging whether the file is finished, if yes, converting into a character string format, and if the conversion is finished, storing the character string into the file and finishing, otherwise, returning to continue traversing the file queue and judging whether the file is finished.
Wherein, step S4 further comprises: the function library is a programming interface of a file system, and the load application program realizes a high-speed file system according to the programming interface.
Further preferably, the step S5 further includes: the logical unit address of the file is recorded by the Vector container, and the logical unit address is a virtual address, wherein when the virtual address is converted into a physical address, the mapping relation between the logical address and the physical address is determined by the controller. The file system manages the usage of the various logical units.
Aiming at a satellite-borne embedded operating system, the invention utilizes a cJSON library and a C++ container to form a structural body by fields such as a file number, a load type, a task number, a start time, an end time, a file size, a node queue and the like, and realizes the management of fixed-size high-speed load data based on a Vector. The method can provide environment for the satellite-borne embedded operation system, further improves the processing capacity of the satellite-borne load data, has positive reference significance for improving the read-write performance of the satellite-borne embedded operation system, and has good practical application value in the field of aerospace engineering.
Specifically, the following two parts of the invention realize a high-speed spaceborne file system, which is specifically as follows:
(1) Recording the node attributes of file numbers, load types, task numbers, starting time, ending time, file sizes, node queues and the like of the files for each load data type by utilizing a JSON lightweight data exchange format;
(2) The initialization flow and the storage flow of the load data are designed, the memory is directly operated in the running process, the information of the storage equipment is not read, the attribute speed of accessing files is improved, the read-write data are directly DMA according to the virtual address, and the utilization rate of the processor is reduced.
In summary, the method and the device remarkably improve the file reading and writing speed of the satellite-borne embedded system in a mode of recording node information in the memory, thereby improving the system performance of the satellite-borne file. Compared with the traditional on-board software development mode which uses a complex file system for reading and writing, the method can greatly improve the speed of reading and writing files by on-board software and has stronger engineering practical value.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. The method for realizing the high-speed spaceborne file system based on JSON is characterized by comprising the following steps of:
s1: transplanting a cJSON library for a current satellite-borne embedded operating system, wherein the cJSON adopts a linked list to store management key/value pairs;
s2: building a structure body composed of a plurality of fields, wherein the fields form node attributes of a file system so as to provide attribute interfaces for application program development;
s3: dividing the number and the address of the logic units of the storage device according to the capacity and the load data size of the storage device;
s4: establishing a function library, wherein the function library comprises: creating a file, writing the file, closing the file, inquiring the data length of the load type, inquiring the file number, reading the file, deleting the file, storing the file system, initializing the file system, emptying the file system and acquiring a function library of the size of the file system;
s5: recording logical unit address information of a storage device of a file by adopting a Vector container of C++;
the load data and the load type are respectively analyzed and stored at a logic unit address of the storage device corresponding to the Vector container in a Vector node position table by adopting cJSON;
the step S4 further includes an initialization process and a save process for designing the high-speed on-satellite file system, where the initialization process includes: analyzing the file information and node information which are recorded currently and whether the address of the initialization logic unit is recorded or not from the JSON data; the preservation flow comprises the following steps: the node information is stored in RAM to speed up the access of the node attribute of the storage device during the data recording process, and is stored in the hard disk in JSON format after the recording is finished.
2. The method for realizing a high-speed on-board file system based on JSON according to claim 1, wherein the cJSON is a JSON parser implemented by using pure C, and the cJSON records the use condition of the storage device and performs memory release after use.
3. The method for implementing a high-speed on-board file system based on JSON according to claim 1, wherein in step S2, the fields include: file number, payload type, task number, start time, end time, file size, and node queue.
4. The method for implementing a high-speed on-satellite file system based on JSON according to claim 1, wherein in step S2, the structure body forms a system attribute of a file system, and is used for maintaining attribute information of load data in a memory.
5. The method for implementing a high-speed on-board file system based on JSON according to claim 1, wherein in step S3, the storage device is divided into fixed-length logical units when establishing address and number information of the logical units according to capacity and load data size of the storage device.
6. The method for implementing a high-speed on-board file system based on JSON according to claim 1 or 5, wherein in step S3, the addresses of the logical units of the file system node are: current node number payload data size.
7. The method for implementing a high-speed on-board file system based on JSON according to claim 1, wherein said step S4 further comprises: the function library is a programming interface of a file system, and the load application program realizes a high-speed file system according to the programming interface.
8. The method for implementing a high-speed on-board file system based on JSON according to claim 1, wherein said step S5 further comprises: and recording the logical unit address of the file by using a Vector container, wherein the logical unit address is a virtual address, and when the virtual address is converted into a physical address, the controller determines the mapping relation between the logical address and the physical address.
9. The method for implementing a high-speed on-board file system based on JSON of claim 1, wherein the file system manages the usage of each logical unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010292816.1A CN111427848B (en) | 2020-04-14 | 2020-04-14 | Method for realizing high-speed spaceborne file system based on JSON |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010292816.1A CN111427848B (en) | 2020-04-14 | 2020-04-14 | Method for realizing high-speed spaceborne file system based on JSON |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111427848A CN111427848A (en) | 2020-07-17 |
| CN111427848B true CN111427848B (en) | 2023-09-05 |
Family
ID=71556533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010292816.1A Active CN111427848B (en) | 2020-04-14 | 2020-04-14 | Method for realizing high-speed spaceborne file system based on JSON |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111427848B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103646063A (en) * | 2013-11-27 | 2014-03-19 | 中国航天科技集团公司第五研究院第五一三研究所 | Satellite-borne high-speed file management system |
| CN104915206A (en) * | 2015-06-15 | 2015-09-16 | 南京阿凡达机器人科技有限公司 | Method for managing attributes and data on DSP based on text analysis |
| CN109815466A (en) * | 2018-12-19 | 2019-05-28 | 广州金十信息科技有限公司 | A kind of dynamic data graph table generating method |
| CN110659083A (en) * | 2019-09-19 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Method and system for recovering memory data after program fault restart |
-
2020
- 2020-04-14 CN CN202010292816.1A patent/CN111427848B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103646063A (en) * | 2013-11-27 | 2014-03-19 | 中国航天科技集团公司第五研究院第五一三研究所 | Satellite-borne high-speed file management system |
| CN104915206A (en) * | 2015-06-15 | 2015-09-16 | 南京阿凡达机器人科技有限公司 | Method for managing attributes and data on DSP based on text analysis |
| CN109815466A (en) * | 2018-12-19 | 2019-05-28 | 广州金十信息科技有限公司 | A kind of dynamic data graph table generating method |
| CN110659083A (en) * | 2019-09-19 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Method and system for recovering memory data after program fault restart |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111427848A (en) | 2020-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107168657B (en) | Virtual disk hierarchical cache design method based on distributed block storage | |
| CN106201350B (en) | Method, memory and the computer system of storing data | |
| CN106570113B (en) | Mass vector slice data cloud storage method and system | |
| CN102521419A (en) | Hierarchical storage realization method and system | |
| CN104239134A (en) | Method and device for managing tasks of many-core system | |
| CN113111131B (en) | Method and system for achieving Neo4j data synchronization based on Flink, and integration method and device | |
| WO2023040399A1 (en) | Service persistence method and apparatus | |
| WO2020192710A1 (en) | Method for processing garbage based on lsm database, solid state hard disk, and storage apparatus | |
| CN108664577A (en) | A kind of file management method and system based on the free areas FLASH | |
| CN102542036B (en) | Use the table optimized | |
| KR102471966B1 (en) | Data input and output method using storage node based key-value srotre | |
| CN111427848B (en) | Method for realizing high-speed spaceborne file system based on JSON | |
| CN109189724B (en) | Method and device for improving audio and video data storage efficiency of video monitoring system | |
| CN102609486A (en) | Data reading/writing acceleration method of Linux file system | |
| CN109446258A (en) | A kind of distributed data storage method and system | |
| CN104239231A (en) | Method and device for accelerating preheating of second-level cache | |
| CN111444154B (en) | Method for efficiently storing log files in single chip microcomputer system | |
| CN109376165A (en) | The implementation method and device and computer readable storage medium of memory database lock | |
| CN116185892B (en) | Flash memory controller and method for accessing flash memory particles | |
| CN116578234B (en) | Flash memory access system and method | |
| CN102270179B (en) | For the data Storage and Processing method based on sheet unit and the system thereof of CAD system | |
| CN112540954B (en) | Multi-level storage construction and online migration method in directory unit | |
| CN104408054A (en) | Database management system | |
| CN109669878A (en) | A kind of most simplified data cached operating method | |
| CN116185891B (en) | Descriptor management method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |