CN115687228A - Satellite-borne solid-state storage system and method based on PCIe bus - Google Patents
Satellite-borne solid-state storage system and method based on PCIe bus Download PDFInfo
- Publication number
- CN115687228A CN115687228A CN202310001279.4A CN202310001279A CN115687228A CN 115687228 A CN115687228 A CN 115687228A CN 202310001279 A CN202310001279 A CN 202310001279A CN 115687228 A CN115687228 A CN 115687228A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- fifo
- module
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003860 storage Methods 0.000 title claims abstract description 235
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000007726 management method Methods 0.000 claims abstract description 109
- 238000004891 communication Methods 0.000 claims abstract description 56
- 238000005192 partition Methods 0.000 claims abstract description 51
- 238000013500 data storage Methods 0.000 claims abstract description 45
- 230000005540 biological transmission Effects 0.000 claims abstract description 32
- 230000015654 memory Effects 0.000 claims abstract description 10
- 238000004458 analytical method Methods 0.000 claims description 31
- 239000000872 buffer Substances 0.000 claims description 26
- 238000012937 correction Methods 0.000 claims description 17
- 230000003111 delayed effect Effects 0.000 claims description 17
- 230000007246 mechanism Effects 0.000 claims description 15
- 238000013461 design Methods 0.000 claims description 13
- 230000001360 synchronised effect Effects 0.000 claims description 13
- 239000007787 solid Substances 0.000 claims description 10
- 230000008520 organization Effects 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 9
- 238000012423 maintenance Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 5
- 238000004140 cleaning Methods 0.000 claims description 4
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 239000013078 crystal Substances 0.000 claims description 3
- 230000000977 initiatory effect Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- 230000003139 buffering effect Effects 0.000 claims description 2
- 230000006870 function Effects 0.000 abstract description 15
- 238000010586 diagram Methods 0.000 description 14
- 230000006872 improvement Effects 0.000 description 8
- 238000011161 development Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000012827 research and development Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 241000252233 Cyprinus carpio Species 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention belongs to the technical field of space-borne solid-state memories and space storage and transmission, and particularly relates to a space-borne solid-state storage system and method based on a PCIe bus. The system of the invention comprises: the data receiving module receives engineering data and application data sent by external main control module CPU software; the data storage module receives the data analyzed by the data receiving module and stores the data into two fixed partitions of the storage array according to the categories; the data sending module plays back the delay data in the storage module according to the instruction of the main control module CPU software and sends the delay data to the main control module CPU software; the communication control module is communicated with the CPU software of the main control module through a serial bus to complete storage related instruction control and state feedback; and the clock management module generates clocks with different frequencies required by the interior of the FPGA from the clocks input from the outside through the DCM and generates a global reset signal. The invention has the advantages of high modularization, clear logic, clear function division, strong reusability and strong expandability.
Description
Technical Field
The invention belongs to the technical field of space-borne solid-state memories and space storage and transmission, and particularly relates to a space-borne solid-state storage system and method based on a PCIe bus.
Background
The satellite-borne bus technology is very important in satellite-borne data acquisition and transmission, and the buses or networks widely applied in the field of aerospace at present mainly comprise RS422, RS485, CAN bus, 1553 bus, LVDS and the like. Although these buses have a wide range of applications, with the complication and difficulty of the aerospace detection task and the application of more advanced data acquisition devices such as synthetic aperture radar and multispectral imager in the aerospace field, the on-board buses have an increasing data volume, and these buses have disadvantages in terms of transmission rate, communication distance, protocol coordination and power consumption. In order to meet the requirements of mass data transmission and storage, it is necessary to apply a faster, more stable and more efficient bus protocol.
PCIe (Peripheral Component Interconnect express) is a high-speed serial computer expansion bus standard, proposed by intel in 2001, and is intended to replace the old PCI (Peripheral Component Interconnect), PCI-X, and AGP (accessed graphics Port) bus standards. PCIe belongs to high-speed serial point-to-point double-channel high-bandwidth transmission, connected devices share the channel bandwidth independently and do not share the bus bandwidth, and the PCIe mainly supports the functions of active power management, error reporting, end-to-end reliable transmission, hot plug, quality of service (QOS) and the like. The PCIe wireless local area network interface has the main advantages that the data transmission rate is high, the PCIe 3.0 speed can reach 8GT/s, and the PCIe wireless local area network interface has considerable development potential and is suitable for data transmission in an aerospace scene.
The satellite-borne solid-state storage system is one of key systems of a satellite platform and is used as a data hub for supporting the development and implementation of a satellite task. The satellite-borne solid-state storage system mainly has the functions of receiving system engineering data (reflecting working state data of each main control module and an interface of a scientific experimental system) and application data (a general name of scientific data, video data and image data generated in an on-orbit mode) from a main control module computer unit CPU software, and storing the system engineering data and the application data in a classified mode according to two fixed partitions, namely a work parameter area and a load area; reading out and sending the data to the computer unit after receiving a playback instruction of the computer unit; and complete the instruction and state interaction with the computer unit.
Most of the current satellite-borne solid-state storage systems are designed in a customized manner, namely scheme design needs to be carried out again on requirements such as effective load data types, storage capacity and storage rate according to different satellite model tasks, so that the research and development efficiency is low, the reusability is poor, and the like, therefore, the system needs to be subjected to architectural design such as layering, modularization and regularization, and the reconfigurability and universality of the satellite-borne solid-state storage system are improved.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a satellite-borne solid-state memory system based on a PCIe bus and also discloses a satellite-borne solid-state memory method based on the PCIe bus.
In order to achieve the above purpose, the present invention is realized by the following technical scheme.
The invention provides a PCIe bus-based satellite-borne solid-state storage system, which is applied to storage hardware and comprises: the data receiving module and the data storage module; wherein,
the data receiving module is used for receiving engineering data and application data sent by the external main control module, performing PCIe physical layer protocol analysis and data application layer protocol analysis, generating corresponding identification signals and ping-pong buffering the data into corresponding FIFOs;
and the data storage module adopts a fixed partition storage mode, respectively and independently stores the data circularly according to the data types, and takes out the delayed data after receiving the playback instruction and transmits the delayed data to the CPU software of the computer unit.
As an improvement of the foregoing technical solution, the data receiving module includes: PCIe protocol analysis receiving unit, application layer protocol analysis unit, FIFO unit before scientific data coding and FIFO unit before engineering data coding; wherein,
the PCIe protocol analysis receiving unit is used for receiving engineering data and application data sent by an external main control module, adopting a PCIe special IP to check the received data to complete PCIe physical layer protocol analysis, converting the analyzed data stream and writing the converted data stream into an AXI bus;
the application layer protocol analysis unit is used for receiving the engineering data and the application data analyzed by the PCIe protocol analysis receiving unit, performing data packet format discrimination on the received data, and caching the data in the corresponding engineering data FIFO or application data FIFO according to the discriminated data type, and specifically includes: if the synchronous word is 0x55AA and the identification field is 0xED1D, judging that the data is an engineering data packet, starting one-time data receiving, and writing the received data into an engineering data FIFO for caching; if the synchronous word is 0x55AA and the identification field is 0x6D1D to 0x6DFD, determining that the application data packet is received, starting one-time data receiving, and writing the received data into an application data FIFO for caching; if the identification field is invalid when the synchronous word is not 0x55AA or the synchronous word is 0x55AA, the data is not received;
the scientific data coding pre-FIFO unit is used for setting relevant parameters of application data FIFO and providing a prog _ full signal to the data storage module as a read FIFO triggering condition;
the engineering data pre-coding FIFO unit is used for setting relevant parameters of the engineering data FIFO and providing prog _ full signals to the data storage module as read FIFO triggering conditions.
As an improvement of the above technical solution, the data storage module includes: the system comprises an RS code management unit, a data cache unit, a FLASH control management unit and a stored data output unit; wherein,
the RS coding management unit is used for receiving the engineering data and the application data cached by the data receiving module, performing RS error correction coding on the effective data according to the corresponding identification signal, and performing ping-pong caching on the data to the data caching unit;
the data caching unit is used for fixedly partitioning an SDRAM storage space, wherein a partition I is used for caching engineering data; the partition II is used for caching application data and then waiting for storage scheduling to be written into the FLASH chip; the data buffer area is also used for generating a signal reflecting the state of each data buffer area;
the FLASH control management unit is used for finishing storage control of engineering data and application data and finishing logic realization of FLASH chip bottom layer drive;
and the storage data output unit is used for carrying out RS decoding on the played-back delayed storage data and writing the delayed storage data into the FIFO cache for carrying out output preprocessing operation.
As an improvement of the above technical solution, the data caching unit performs fixed partitioning on the SDRAM storage space, where the partition I is used to cache engineering data; the partition II is used to cache application data, and then waits for storage scheduling to be written into the FLASH chip, and specifically includes:
when the data amount of any one of the engineering data cache FIFO or the application data cache FIFO reaches a threshold value, starting the reading operation of the corresponding cache FIFO; at the moment, if the RS coding enabling signal is valid, RS coding is carried out on the read data and the read data are written into the asynchronous ping-pong FIFO cache, and if the RS coding enabling signal is invalid, the read data are directly written into the asynchronous ping-pong FIFO cache; when data is written into the asynchronous ping-pong FIFO, the corresponding channel number is recorded.
When the data amount of any asynchronous FIFO buffer reaches 256 × 128bits, the RS coding management unit outputs an effective corresponding asynchronous ping-pong FIFO half-full signal, and outputs corresponding FIFO buffer data and a channel number after receiving an RS coded asynchronous ping-pong FIFO read enable signal;
when the asynchronous ping-pong FIFO half-full signal is effective, starting corresponding asynchronous ping-pong FIFO read operation and SDRAM write operation, and writing the coded data source packet into the corresponding partition cache of SDRAM according to the channel number from the RS coding management module;
when the cache data volume of any partition of the SDRAM engineering data partition or the application data partition is full of 4 clusters, starting 1-time FLASH writing operation, reading out target data from the SDRAM according to clusters and writing the target data into a rear-end asynchronous cache FIFO by a data cache unit according to SDRAM read request signals and read addresses from a FLASH control management unit; and the FLASH control management unit finishes reading the data of the asynchronous cache FIFO cluster according to the signal sent by the data cache unit.
As an improvement of the above technical solution, the logic implementation of the FLASH chip bottom driver is completed, which specifically includes:
generating a drive signal for the operation of the FLASH memory device and meeting the requirement of the operation time sequence; operations include reset, read, write and erase;
automatically loading cluster mark information and system time code information in each free area of each used cluster in a storage area;
starting the operation aiming at the appointed FLASH storage area through a software command;
automatically detecting error information in the storage area, marking the clusters and blocks with errors, and timely notifying equipment software;
maintaining relevant information representing the hardware working state, wherein the information can be read by software from a specific address;
after each power-on, automatically generating BAT reflecting the use condition of all blocks in the storage area, and reading the BAT from the specified address by software;
and automatically managing the storage block address sent by the CPU software of the computer unit, and finishing automatic taking, checking and forwarding of the unused block address, the block address to be played back, the block address to be erased, the invalid block address to be marked and the CPU cluster reading address cached in the communication control module according to the storage task scheduling.
As an improvement of the above technical solution, the system further includes a data transmission module; the data sending module comprises: SCI receiving FIFO unit, ENG receiving FIFO unit, data scheduling unit and PCIe data transmitting unit; wherein,
the SCI receiving FIFO unit is used for receiving the application data played back by the data storage module and performing clock domain crossing processing; setting SCI receiving FIFO related parameters and providing prog _ full signals to the data scheduling unit as reading FIFO triggering conditions;
the ENG receiving FIFO unit is used for receiving the engineering data played back by the data storage module and performing clock domain crossing processing; setting related parameters of an ENG receiving FIFO, and providing a prog _ full signal to a data scheduling unit as a read FIFO triggering condition;
the data scheduling unit is used for monitoring the prog _ full signal states of the SCI receiving FIFO unit and the ENG receiving FIFO unit, setting the GPIO signal to be effective if the prog _ full signal is '1', and informing CPU software to read a PCIe bus, otherwise, outputting a high level by the GPIO to indicate that the PCIe is unreadable; when the prog _ full signals of the SCI receiving FIFO unit and the ENG receiving FIFO unit are both effective, the ENG receiving FIFO, namely engineering data, is read preferentially;
the PCIe data sending unit is used for time division multiplexing data receiving modules in a physical layer to form the same link, and a PCIe IP core autonomously arbitrates and dispatches a transmission layer; and after the CPU software receives the valid GPIO signal, initiating a request for reading the PCIe bus, reading a corresponding SCI receiving FIFO or ENG receiving FIFO according to scheduling by the PCIe bus-based satellite-borne solid-state storage system, placing data on the AXI bus, reading the data by the PCIe IP core, sending the data to the CPU software, and completing data transmission.
As an improvement of the above technical solution, the system further includes a communication control module; the communication control module includes: the UART communication management unit, the BAT cache management unit and the storage address management unit; wherein,
the BAT cache management unit is used for starting the reset configuration operation of the FLASH chips, and after the reset configuration of all the FLASH chips of the storage array is finished, the BAT organization operation is started by a CPU software instruction;
the UART communication management unit is used for monitoring the communication state of the UART bus and receiving control information according to a UART protocol; analyzing received serial input data based on communication constraints of command frames and data frames between CPU software and a PCIe bus-based satellite-borne solid-state storage system, if three continuous bytes are 0xEB90A1, determining that a command frame header is detected, and starting receiving and analyzing parameters in the command frame, otherwise, judging whether the received data is 0xEB90A1 all the time; judging whether the accumulated sum is correct after receiving the complete command frame, if so, executing a corresponding command and returning a correct response; if not, the command is not executed and corresponding response is returned; forwarding the analyzed command or state including storage soft reset, an external data input switch, an RS coding switch, an RS decoding switch, storage initialization end, FLASH chip reset, data transmission state, storage starting block counting, storage mark invalidation and storage time code to other internal functional modules; the device is used for respectively finishing data query including hardware status words, BAT information and designated clusters according to instructions including status query, BAT reading and data point reading, and feeding back the data query to CPU software; the CPU is also used for accumulating and checking the received command frame, if the accumulated sum is correct, performing instruction analysis or starting a storage block address receiving cache, and if the accumulated sum is incorrect, not performing instruction analysis, namely not performing instruction forwarding or storage block address and other information receiving caches, and directly feeding back frame error information to the CPU software;
and the storage address management unit is used for writing the analyzed new storage block addresses, playback block addresses and erasure block addresses into the DPRAM partition for management, and performing autonomous maintenance on the internal storage addresses according to the storage state information.
As an improvement of the above technical solution, the system further comprises a clock management module; the clock management module comprises: the master control clock management unit and the reset logic management unit;
the master control clock management unit comprises CSU5.1.1 DCM0, inputs a clock provided for an external crystal oscillator and outputs a clock used by related logic of SDRAM and a clock used by related logic of NAND FLASH;
the reset logic management unit is used for generating reset signals used by the modules, wherein the LOCKED signal output by the DCM0 generates uart _ rst signal output after being logically bonded, and is only used for resetting the communication control module; DCM0 outputs the generated UART _ rst reset signal and UART instruction reset cmd _ FPGA _ rst signal phase from the UART communication management unit, and then generates a global reset signal sys _ rst through BUFG for resetting other modules except the communication control module.
The invention also provides a PCIe bus-based satellite-borne solid-state storage method, which is realized based on the system, and comprises the following steps:
after the storage hardware is powered on, the satellite-borne solid-state storage system based on the PCIe bus automatically starts the FLASH chip reset configuration operation, and after the reset configuration of all the FLASH chips of the storage array is completed, the BAT organization operation is started by the instruction of CPU software; during traversing data of all the cluster vacant areas of the storage array, performing corresponding Hamming decoding operation, and after BAT information of the NAND FLASH of the storage array is read by the CPU, entering a conventional task management state after receiving an initialization ending command by the storage system;
the data storage module monitors the state of each data cache region from an internal data cache unit, when the cache data volume of any partition of the SDRAM is full of 4 clusters, the data storage module starts 1 FLASH write operation according to task scheduling, sends an SDRAM read request signal to the data cache module, and starts four-level flow data writing;
after each stage of the pipeline is loaded with the effective data, cleaning the corresponding SDRAM space use identifier; detecting the internal programming state of the storage area after programming is finished, marking the erroneous clusters and blocks, and informing storage management software; if storage failure occurs, automatically rewriting failure cluster data into the replacement block;
when the data storage module receives a data playback instruction, starting FLASH reading operation according to a playback address sent by the communication control module, reading corresponding data according to a cluster, and sending the read data to the stored data output unit;
and when the data storage module receives the data erasing instruction, starting the erasing operation of the FLASH block according to four-level flow according to the erasing address sent by the communication control module.
As an improvement of the above technical solution, the method further designs an error detection and correction coding protection mechanism and a bad block management mechanism: wherein,
the error detection and correction coding protection mechanism specifically comprises:
for the data stored in the NAND FLASH main storage area, an RS coding technology with higher performance is selected, namely RS coding is firstly carried out before the engineering data and the application data are stored, then the RS coding is written into the main storage area, and during data playback, RS decoding error correction is firstly carried out, and then the RS coding error correction is transmitted to CPU software;
for the auxiliary information storage of each page of spare area of the NAND FLASH, a simple and reliable Hamming coding technology is selected, namely Hamming coding protection is carried out when file information is stored, and Hamming decoding error correction is carried out when the file information is read;
the bad block management mechanism specifically comprises:
processing invalid blocks existing when the NAND FLASH chip leaves a factory as static bad blocks, wherein the invalid blocks are not used any more;
for newly added invalid blocks in the using process of a chip, the invalid blocks are taken as dynamic bad blocks and managed by software and hardware together, and the specific method comprises the following steps: for the case of programming failure, performing normal read operation on a page which is normally programmed in the block, but rewriting the data of the page which is failed to program in another effective block, and marking the block to avoid further writing or erasing the block later; for an erasure failure block, marking the block invalid in a PCIe bus-based satellite-borne solid-state storage system to avoid writing or erasing the block later; the dynamic bad block can be selected not to be used any more in the future, and can also be selected to be used again after the storage area maintenance is carried out.
Compared with the prior art, the invention has the advantages that:
1. the data transmission rate is high, and a PCIe bus with high transmission rate is used for load data interaction;
2. the system interface is simple and highly uniform, the external data interface only adopts PCIe bus, the types of the used bus are reduced, various bus protocols are not required to be configured in the development, and the design complexity is reduced;
3. the system is highly modularized, clear in logic, clear in function division, strong in reusability and strong in expandability.
Drawings
FIG. 1 is a functional block diagram of a PCIe bus based on-board solid state storage system;
FIG. 2 is a structure diagram of a PCIe bus based satellite borne solid state storage system;
FIG. 3 is a data store path data flow and control flow graph;
FIG. 4 is a data receiving module call relationship diagram
FIG. 5 is a diagram of a data storage module call relationship;
FIG. 6 is a diagram of a RS code management module call relationship;
FIG. 7 is a diagram of a data caching module call relationship;
FIG. 8 is a calling relationship diagram of FLASH control management module
FIG. 9 is a stored data output module call relationship diagram;
FIG. 10 is a schematic diagram of four-level pipelined FLASH operation;
FIG. 11 is a data sending module call relationship diagram;
FIG. 12 is a diagram of computer unit communication module call relationships;
FIG. 13 is a clock management module call relationship diagram.
Detailed Description
The invention aims to solve the problems of the conventional satellite-borne solid-state storage system, and provides a design of a satellite-borne solid-state storage system based on a PCIe bus, wherein the PCIe bus is used as a uniform data interface of the satellite-borne solid-state storage system to perform high-speed data transmission with an upper computer CPU, so that the data transmission rate is greatly improved. AXI4 bus communication is uniformly used among the internal modules, and the speed of the communication can be matched with that of a PCIe high-speed interface. In addition, aiming at the technical problems of redundancy, complexity, poor universality, difficult upgrading, difficult maintenance, high cost and the like of the conventional satellite-borne solid-state storage system, the invention provides a hierarchical, modular and regularized design method, which has the following specific meanings:
1) Layering: the system is divided into several modules and then each module is further divided until the functions of the modules are sufficiently well defined.
2) Modularization: all modules have well-defined interfaces and functions so that they can be easily interconnected without side effects.
3) Regularization: when in design, the consistency of all modules is pursued on the premise of not influencing the functions, and the universal module can be reused so as to reduce the number of different modules in design.
Therefore, the standardization, the reconfigurability and the universality of the satellite-borne solid-state storage system are realized, and the development cost is favorably reduced.
The technical solution of the present invention is described in detail below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 1, it is a functional block diagram of a PCIe bus based satellite borne solid state storage system; as shown in fig. 2, the structure diagram of the PCIe bus based satellite-borne solid-state storage system is shown. The function of the PCIe bus-based satellite-borne solid-state storage system mainly comprises:
1) A data receiving function; receiving engineering data and application data sent by external main control module CPU software;
2) A data storage function; receiving data analyzed by the data receiving module, and storing the data into two fixed partitions of the storage array according to the categories;
3) A data transmission function; the time delay data in the storage module is played back according to the instruction of the main control module CPU software and is sent to the main control module CPU software;
4) A communication control function; the control system is communicated with the CPU software of the main control module through a serial bus to complete storage related instruction control and state feedback;
5) A clock management function; the externally input clock is used for generating clocks with different frequencies required in an FPGA (Field Programmable Gate Array) from a DCM (digital clock modulator), and generating a global reset signal.
System data flow and control flow
The data flow and control flow of the data memory path are shown in fig. 3, in which the curved part is the data flow and the dotted part is the control flow.
Data flow:
the method comprises the steps that a satellite-borne solid-state storage system based on a PCIe bus receives system engineering data and application data from external CPU software through a 1-path x4 Lane PCIe interface; and the PCIe special IP core provided by the Xilinx is adopted to complete the PCIe physical layer protocol analysis.
Carrying out data packet format judgment on received data by a satellite-borne solid-state storage system based on a PCIe bus, if a synchronous word is 0x55AA and an identification field is 0xED1D, judging that the data is an engineering data packet and starting primary data receiving, wherein the primary data receiving length is 2024 bytes, and writing the received data into an engineering data FIFO (first in first out) for caching; if the synchronous word is 0x55AA and the identification field is 0x6D1D to 0x6DFD, determining to apply the data packet and starting one-time data reception, wherein the one-time data reception length is 2024 bytes, and writing the received data into an application data FIFO for caching; if the identification field is invalid when the sync word is not 0x55AA or when the sync word is 0x55AA, no data is received.
After the initialization of the system is finished, if detecting that the data volume of any one of the engineering data cache FIFO or the application data cache FIFO reaches a threshold value (2016 × 16bits), starting the reading operation of the corresponding cache FIFO; at this time, if the RS (252, 256) encoding enabling signal is valid (the default encoding enabling signal is valid), the read data is RS encoded and written into the asynchronous ping-pong FIFO buffer, and if the RS (252, 256) encoding enabling signal is invalid, the read data is directly written into the asynchronous ping-pong FIFO buffer; and recording the corresponding channel number when data is written into the asynchronous ping-pong FIFO.
When the data amount of any asynchronous FIFO reaches 256 × 128bits, an effective corresponding asynchronous ping-pong FIFO half-full signal is output, and after an asynchronous ping-pong FIFO read enable signal after RS encoding is received, corresponding FIFO cache data and a channel number are output.
The PCIe bus-based satellite-borne solid-state storage system monitors an asynchronous ping-pong FIFO half-full signal from an RS coding management module, starts corresponding asynchronous ping-pong FIFO read operation and SDRAM write operation when the half-full signal is effective, and writes an encoded data source packet into a corresponding partition cache of an SDRAM according to a channel number from the RS coding management module.
The data cache module is designed with signals reflecting the state of each data cache region for the data storage module to inquire; when the data amount of any partition cache of the SDRAM platform engineering parameter partition or the load science data partition is full of 4 clusters (4 x 4096 x 128bits), the data storage module starts 1 FLASH write operation, and the data cache module reads out target data from the SDRAM according to clusters and writes the target data into a rear-end asynchronous cache FIFO according to an SDRAM read request signal and a read address from a FLASH control management module. And the FLASH control management module finishes the reading of the cluster data of the asynchronous cache FIFO according to signals of half full of the cache FIFO, the reading end of the cluster data and the like sent by the data cache module.
The FLASH control management module monitors the state of each data cache region from the internal data cache module, when the cache data volume of any partition of the SDRAM is full of 4 clusters (4 x 4096 x 128bits), the data storage module starts 1 time of FLASH write operation according to task scheduling, sends SDRAM read request signals and read addresses to the data cache module, and starts four-level streaming data writing.
After each stage of the pipeline is loaded with the effective data, writing cluster marking information (including time codes) into a spare area of the FLASH cluster, and cleaning a corresponding SDRAM space use identifier; detecting the internal programming state of the storage area after programming is finished, marking the erroneous clusters and blocks, and informing CPU software; and if the storage fails, automatically rewriting the failed cluster data into the replacement block.
( FLASH page, cluster, block: the FLASH storage medium uses a Spare area (Spare area) of 448 bytes added in each Main storage area (Main area) of 8K bytes as a Page (Page), 128 pages form a Block (Block), 8192 blocks form a chip Device (Device), and the selected FLASH module is formed by stacking 4 chips. In the design, the chips with the same serial number in the 16 stacked modules are regarded as a whole to simultaneously perform various operations, namely parallel expansion operation. The page with the same address of the 16 parallel FLASH chips is regarded as a basic unit, namely a cluster, so that the basic storage space of 1 cluster is 8K multiplied by 128bits. )
When the satellite-borne solid-state storage system based on the PCIe bus is in a 'start data transmission' working mode, the storage data output module receives playback data from the FLASH control management module; when the RS (252, 256) decoding switch is turned on (the default decoding switch is turned on), the storage data output module receives data and simultaneously performs RS decoding, and writes the decoded data into a corresponding first-level cache FIFO according to the playback channel identifier; when the RS (252, 256) decoding switch is closed, the effective data is directly written into the corresponding first-level cache FIFO according to the playback channel identifier; both first level buffer FIFOs are set to 4096 x 128bits size.
When the data volume in any one first-level cache FIFO reaches a threshold (504 × 128bits) and the corresponding second-level cache FIFO is effective, starting to read the first-level cache FIFO and write the corresponding second-level cache FIFO; both level two cache FIFOs are set to 512 x 128bits in size. And when the data amount in any secondary cache FIFO reaches a threshold (126 × 128bits), transmitting the engineering data or the application data to the data transmitting module. When the satellite-borne solid-state storage system based on the PCIe bus is in a 'data transmission stop' working mode, the first-level cache FIFO and the second-level cache FIFO are in a reset state.
The data sending module is provided with 2 data receiving FIFOs for respectively caching and storing the delayed engineering data and the delayed application data sent by the data output module, when the data volume of any cache FIFO reaches 2024 bytes, GPIO signals are triggered to inform CPU software to read PCIe, and the data sending module sends 1 packet of delayed engineering data or delayed application data to a PCIe bus according to arbitration scheduling to finish playback and sending of 1 packet of data.
Control flow:
the communication module controls and monitors the communication state of the UART bus and receives bus data according to the UART protocol.
The communication control module analyzes the received serial input data, if the three continuous bytes are 0xEB90A1, the communication control module considers that a command frame head is detected, and starts to receive and analyze the parameters in the command frame, otherwise, the communication control module judges whether the received data is 0xEB90A1 all the time.
Judging whether the accumulated sum is correct after receiving the complete command frame, if so, executing the corresponding command and returning a correct response; if not, the command is not executed and a corresponding response is returned.
The communication control module transmits the analyzed commands or states of storage soft reset, an external data input switch, an RS coding switch, an RS decoding switch, storage initialization ending, flash chip reset, data transmission state, storage starting block counting, storage mark invalidation, storage time code and the like to other internal functional modules.
Writing the analyzed new storage block addresses, playback block addresses and erasure block addresses (at most 64 blocks) into a DPRAM partition for management, and performing autonomous maintenance on internal storage addresses according to storage state information fed back by a storage control module; and the storage management software completes automatic management and distribution of the new storage block address, the playback block address and the erase address according to the hardware state word.
And respectively finishing the inquiry feedback of hardware state words, BAT information and appointed cluster data according to the inquiry of the storage state, the reading of BAT (Block Assignment Table) and the reading of data point instructions.
According to the invention, the PCIe bus is successfully applied to the satellite-borne solid-state storage system, so that the effective transmission of high-speed data is realized, the problem of supporting the high-speed data by the traditional data bus is solved, and the domestic advanced level is reached;
example 2
The invention provides a hierarchical, modular and regularized design method of a satellite-borne solid-state storage system, which realizes the rapid construction and project implementation of the satellite-borne solid-state storage system, and the universality of the scheme effectively solves the problems of low system research and development efficiency, poor reusability and the like.
The CSC1 data receiving module receives the engineering data and the application data from the CPU software through the 1-way x4 Lane PCIe data receiving interface, performs PCIe physical layer protocol analysis and data application layer protocol analysis, generates a corresponding identification signal, and ping-pong buffers the data into a corresponding FIFO, where a call relationship is shown in fig. 4.
The data receiving module comprises a PCIe protocol analysis receiving module, an application layer protocol analysis module, a FIFO before scientific data (SCI) coding module, an FIFO before engineering data (ENG) coding module and the like.
The PCIe protocol analysis receiving module receives engineering data and application data from external CPU software through a 1-path x4 Lane PCIe interface, a DMA/Bridge Subsystem for PCI Express IP core provided by the instantiated Xilinx completes the PCIe physical layer protocol analysis, and the analyzed data stream is converted into 64-bit data and written into an AXI bus.
The application layer protocol analysis module judges the packet format of the received data:
1) If the synchronous word is 0x55AA and the identification field is 0xED1D, judging that the data is an engineering data packet and starting primary data receiving, wherein the primary data receiving length is 2024 bytes, and writing the received data into an engineering data FIFO for caching;
2) If the synchronous word is 0x55AA and the identification field is 0x6D1D to 0x6DFD, determining to apply the data packet and starting one-time data reception, wherein the one-time data reception length is 2024 bytes, and writing the received data into an application data FIFO for caching;
3) If the identification field is invalid when the sync word is not 0x55AA or when the sync word is 0x55AA, no data is received.
The FIFO module before scientific data (SCI) coding sets the write clock of the application data FIFO to be 250MHz, the write depth to be 32768 × 64, the read clock to be 32MHz, the read depth to be 16384 × 128, the threshold value of the prog_full signal to be 252 × 128bit, and provides the prog _ full signal to the data storage module as the trigger condition of the read FIFO.
The FIFO module before engineering data (ENG) coding sets an engineering data FIFO write clock of 250MHz, a write depth of 32768 × 64, a read clock of 32MHz, a read depth of 16384 × 128, a prog _fullsignal threshold of 252 × 128bit, and provides prog _ full signal to the data storage module as a read FIFO trigger condition.
The CSC2 data storage module mainly realizes the coding and caching of the engineering data and the application data received by the data receiving module, then stores the data packet into a storage array, and takes out the delay data after receiving a playback instruction and transmits the delay data to the CPU software of the computer unit. The CSC2 data storage module mainly includes four parts of RS code management, data cache module, FLASH control management and stored data output, and the calling relationship is as shown in fig. 5.
The CSU2.1 RS coding management mainly completes RS (256, 252) coding protection on received engineering data and application data. The RS coding management module receives the engineering data and the application data cached by the data receiving module and transmits the data to the RS (256, 252) coding module according to the corresponding identification signal; after RS (256, 252) error correction coding is carried out on the effective data, the data is ping-pong buffered into a data buffer FIFO. The call relationship is shown in fig. 6.
And the CSU2.2 data caching module caches the platform data and the scientific data after RS encoding through an external SDRAM chip, and then waits for storage scheduling to be written into a FLASH chip. The data caching module supports fixed partition of an SDRAM storage space, wherein a partition I is used for caching engineering data; partition II is used to cache application data; after the system initialization is completed, monitoring the state of a data cache FIFO (first in first out) based on a PCIe (peripheral component interconnect express) bus satellite-borne solid-state storage system, and writing data into a corresponding partition of an SDRAM (synchronous dynamic random access memory) for respective caching according to the data category of engineering data or application data when the cache data volume reaches 256 x 128bits; when the buffer memory data volume of any partition of the SDRAM engineering data partition or the application data partition is full of 4 clusters, 1 time of FLASH writing operation is started, and target data are read from the SDRAM according to the address and written into a back-end SDRAM output buffer memory FIFO. The calling relationship is shown in fig. 7.
The CSU2.3 FLASH control management module mainly completes the storage control of engineering data and application data and the realization of the bottom layer drive logic of the NAND FLASH chip, and comprises the following steps: 1) Generating drive signals for operations such as resetting, reading, writing, erasing and the like of the FLASH memory device and meeting the time sequence requirements of the FLASH memory device; 2) Automatically loading cluster mark information and system time code information in each free area of each used cluster in a storage area; 3) Operations such as resetting, writing, replaying, erasing and the like aiming at a specified FLASH storage area can be started through software commands; 4) Automatically detecting error information such as programming and erasing in the storage area, marking the error clusters and blocks, and timely notifying equipment software; 5) The method comprises the following steps of providing relevant information for representing the working state of hardware, wherein the information can be read by software from a specific address; 6) After each power-on, automatically generating a 'block allocation table' (BAT) reflecting the use condition of all blocks in the storage area, and reading the 'BAT' from a specified address by software; 7) The method supports automatic management of the storage block address sent by the CPU software of the computer unit, and completes the functions of automatic taking, checking, forwarding and the like of the unused block address, the block address to be played back, the block address to be erased, the invalid block address to be marked, the CPU cluster reading address and the like cached in the communication control function according to the storage task scheduling. The calling relationship is shown in fig. 8.
The CSU2.4 stored data output module performs RS decoding on the played-back delayed stored data, and writes the decoded delayed stored data into the FIFO buffer to perform output preprocessing operation, and the call relationship is as shown in fig. 9.
The data storage module comprises an RS code management module, a data cache module, a FLASH control management module and the like.
After the system initialization is finished, if the RS code management module detects that any cache data volume of an application data cache FIFO (CSU 1.3 SCI pre-coding FIFO) or an engineering data cache FIFO (CSU 1.4 ENG pre-coding FIFO) reaches a threshold value (252 x 128bits), starting the reading operation of the corresponding cache FIFO; at this time, if the RS (252, 256) encoding enable signal is valid (the default encoding enable signal is valid), RS encoding is performed on the read data (csu 2.2.1 RS encoder 0 to csu2.2.16 RS encoder 15) and the read data are written into an asynchronous ping-pong FIFO buffer (scu 2.2.17 RS encoded FIFO _ a or scu2.2.18 RS encoded FIFO _ B), and if the RS (252, 256) encoding enable signal is invalid, the read data are directly written into the asynchronous FIFO buffer (scu 2.2.17 RS encoded FIFO _ a or scu2.2.18 RS encoded FIFO _ B); and recording the corresponding channel identification when the data is written into the asynchronous ping-pong FIFO.
When the buffer data volume of any asynchronous FIFO (FIFO _ A after SCU2.2.17 RS coding or FIFO _ B after SCU2.2.18 RS coding) reaches 256 x 128bits, an effective corresponding asynchronous ping-pong FIFO half-full signal is output, and after receiving an asynchronous ping-pong FIFO read enable signal after RS coding, corresponding FIFO buffer data and a channel number are output.
The data caching module supports fixed partition of an SDRAM storage space, wherein the partition I is used for caching engineering data; partition II is used to cache application data. The design SDRAM supports 4 groups of scientific data and engineering parameter partitions, each group comprises 4 clusters, and the data volume of each cluster is 4K x 128bits.
After the system initialization is completed, the PCIe bus-based satellite-borne solid-state storage system monitors the state of a data receiving cache FIFO (FIFO _ A after SCU2.2.17 RS coding or FIFO _ B after SCU2.2.18 RS coding) from a CSU2.1 RS coding management module, when the cache data volume of any FIFO reaches 256 x 128bits, corresponding asynchronous ping-pong FIFO read operation and SDRAM write operation are started, and data are written into corresponding partitions of the SDRAM to be cached respectively according to the data category of engineering data or application data.
The data cache module is designed with signals reflecting the states of the data cache regions and used for the CSU2.3 FLASH control management module to inquire; when the data volume of any partition cache of the SDRAM platform engineering data partition or the application data partition is full of 4 clusters (4 x 4096 x 128bits), the data storage module starts 1 time of FLASH write operation, and the data cache module reads out target data from SDRAM according to the cluster and writes the target data into a rear-end asynchronous cache FIFO (CSU2.2.2 SDRAM output FIFO) according to an SDRAM read request signal and a read address from the CSU2.3 FLASH control management module. And the FLASH control management module finishes reading the cluster data of the asynchronous cache FIFO according to signals of half full of the cache FIFO, end of reading the cluster data and the like sent by the data cache module.
The PCIe bus-based satellite-borne solid-state storage system completes SDRAM bottom layer logic drive (CSU2.2.1 SDRAM bottom layer drive) according to DDR3 SDRAM related IP cores provided by Xilinx, and completes operation scheduling control of SDRAM read and write and the like.
The FLASH control management module comprises a FLASH reset configuration module, an organization BAT module, a FLASH writing module, a FLASH reading module, a FLASH erasing module and the like.
(1) Storage scheme design
In order to improve the storage throughput rate, the data storage module is designed by adopting a four-stage pipeline operation scheme, namely, the writing and erasing operation of the NAND FLASH array is carried out according to four-stage pipeline, and the reading operation of the NAND FLASH data is carried out according to clusters. The four-stage pipeline operation is schematically shown in FIG. 10.
The storage area management mechanism adopts a fixed partition storage mode: setting two storage partitions according to task requirements, respectively storing according to data types, storing engineering data to an engineering data area, storing application data to an application data partition, independently and circularly storing data of the two storage partitions (the engineering data area and the application data area), and automatically erasing the oldest part of data and covering the oldest part of data with new data after any storage area is full; during playback, the engineering data area data is played back first, and then the application data area data is played back.
(2) Storage mode of operation
1) Write only mode
And recording the application data and the engineering data into a storage area, and not downloading and playing back.
2) Read-only mode
And returning the delayed application data and the delayed engineering data in the storage area to the next place for transmission without storing new data.
3) Write while erase mode
When the storage area is full, the mode of writing and erasing is entered, namely, the data stored firstly is erased when new data is stored, so as to realize the rolling storage of the storage area.
4) Read while write mode
The data playback operation is carried out simultaneously when the storage and playback operation mode is supported, namely, new data is stored.
5) Read-while-write-while-erase mode
After the storage area is full, the working mode of erasing while writing and replaying is supported.
(3) Storing workflows
After the storage hardware is powered on, the satellite-borne solid-state storage system based on the PCIe bus automatically starts FLASH chip reset configuration operation (CSU2.3.1 FLASH reset configuration), and after the reset configuration of all FLASH chips of the storage array is completed, BAT organization operation (CSU2.3.2 BAT organization) is started by a CPU software instruction. During traversing data of all the free areas of the storage array, corresponding Hamming decoding operations (CSU2.3.2.1 Hamming decoding 0-CSU2.3.2.4 Hamming decoding 3) are carried out, and after BAT information of the NAND FLASH of the storage array is read by the CPU, the storage system enters a conventional task management state after receiving an initialization ending command.
The data storage module monitors the state of each data cache region from the internal CSU2.2 data cache module, when the cache data volume of any partition of the SDRAM is full of 4 clusters (4 x 4096 x 128bits), the data storage module starts 1 FLASH write operation (CSU 2.3.3 FLASH write) according to task scheduling, sends SDRAM read request signals to the data cache module, and starts four-level flow data write.
After each stage of the pipeline is loaded with the effective data, cleaning the corresponding SDRAM space use identifier; detecting the internal programming state of the storage area after the programming is finished, marking the erroneous clusters and blocks and informing the storage management software; and if the storage fails, automatically rewriting the failed cluster data into the replacement block.
When the data storage module receives a data playback instruction, according to the playback address sent by the CSC4 communication control module, a FLASH reading operation (CSU2.3.4 FLASH reading) is started, corresponding data are read out according to clusters and then sent to the CSU2.4 storage data output module.
When the data storage module receives a data erasing instruction, according to an erasing address sent by the CSC4 communication control module, the erasing operation (CSU2.3.5 FLASH erasing) on the FLASH block is started according to four-level flow.
The data storage module supports data point reading operation (CSU2.3.7 FLASH point reading) on any cluster of main storage area and spare area of FLASH in the debugging process.
Reliability measures taken by the data storage module:
1) Error detection and correction coding protection mechanism
Aiming at the problems of NAND FLASH bit upset and single event upset, an error detection and correction coding protection mechanism is designed. For the data stored in the main storage area of the NAND FLASH, an RS (256, 252) coding technology with higher performance is selected, namely, the RS (256, 252) coding is firstly carried out before the engineering data and the application data are stored, then the data are written into the main storage area, and when the data are played back, the RS (256, 252) decoding error correction is firstly carried out, and then the data are transmitted to CPU software; for the auxiliary information storage of each page of spare area of the NAND FLASH, a simple and reliable Hamming coding technology is selected, namely Hamming coding protection is carried out when file information is stored, and Hamming decoding error correction is carried out when the file information is read. By designing RS (256, 252) coding and Hamming coding protection mechanisms, the reliability of data storage is improved.
2) Bad block management mechanism
One of the characteristics of NAND FLASH chips is that there is a certain percentage of invalid blocks at the time of factory shipment, and the read operation for the invalid blocks is allowed, but the write and erase operations for the invalid blocks should be avoided as much as possible. And for the management of the factory invalid blocks, the factory invalid blocks are comprehensively treated as static bad blocks according to a test manual provided by a manufacturer and a secondary test result of a user, and are not used in the future.
During the use of the chip, a new invalid block may be generated, which is particularly indicated as a situation that programming failure or erasing failure occurs in the programming or erasing process. The new invalid block is taken as a dynamic bad block and is managed by software and hardware together, and the specific method is that for the condition of programming failure, a page which is programmed normally in the block can be read normally, but the data of the page which is programmed failure is rewritten in another valid block, and the block is marked at the same time, so that the block is prevented from being further written or erased later. For an erase failure block, the block is marked in the system (csu 2.3.6 FLASH flag invalid) to avoid subsequent writes or erases to the block. The dynamic bad block can be selected not to be used any more in the future, and can also be selected to be used again after the storage area maintenance is carried out.
By designing a bad block management mechanism with cooperative software and hardware, the storage correctness and effectiveness of data can be ensured. Meanwhile, a wear leveling mechanism is designed for storage management so as to reduce the probability of generating dynamic bad blocks as much as possible.
When the FPGA is in a 'start data transmission' working mode, the CSU2.4 storage data output module receives delay data played back by the CSU2.3 FLASH control management module; when the RS (252, 256) decoding switch is turned on (the default decoding switch is turned on), the data output sub-module receives data and simultaneously performs RS decoding, and writes the decoded data into a ping-pong FIFO buffer; when the RS (252, 256) decoding switch is closed, the valid data is directly written into the ping-pong FIFO buffer; two FIFOs of the ping-pong cache are set to be 8192 × 128bits; when any ping-pong FIFO is not empty and the receiving FIFO corresponding to the type of the back-end CSC3 data transmitting module is not full, the receiving FIFO corresponding to the CSU2.4 storage data output module is read, and then the data is transmitted to the CSC3 data transmitting module.
The data sending module is mainly used for sending the delayed data played back by the data storage module to the CARP CPU software. The data sending module arbitrates the same 1-path x4 Lane PCIe link in the physical layer time division multiplexing data receiving module according to the monitored buffer states of the playback engineering data FIFO and the playback application data FIFO, reads data in the corresponding FIFO after arbitration, and transmits the playback data to computer unit CPU software through a PCIe sending interface, so that the transmission of the playback data is completed. The calling relationship is shown in fig. 11.
The data sending module comprises an SCI receiving FIFO, an ENG receiving FIFO, a data scheduling module, a PCIe data sending module and the like.
The SCI receiving FIFO module is used for receiving the application data played back by the CSC2 data storage module and performing cross-clock domain processing.
The SCI receives a FIFO write clock of 32MHz, the write depth is 8192 × 128, the read clock is 250MHz, the read depth is 16384 × 64, the threshold value of the prog _fullsignal is 253 × 64bit (1 packet data), and provides the prog _ full signal to the CSU3.3 data scheduling module as a read FIFO trigger condition.
And the ENG receiving FIFO module is used for receiving the engineering data played back by the CSC2 data storage module and performing clock domain crossing processing.
An ENG receiving FIFO write clock is set to be 32MHz, the write depth is 8192 × 128, a read clock is 250MHz, the read depth is 16384 × 64, a prog full signal threshold is 253 × 64bit (1 packet data), and a prog full signal is provided to a CSU3.3 data scheduling module to serve as a read FIFO triggering condition.
And the data scheduling module monitors the prog _ full signal states of the CSU3.1 SCI receiving FIFO and the CSU3.2 ENG receiving FIFO, sets the GPIO signal to be effective if the prog _ full signal is '1', and informs CPU software to read the PCIe bus, otherwise, outputs a high level to indicate that the PCIe is unreadable.
When the prog _ full signals of both the CSU3.1 SCI receive FIFO and the CSU3.2 ENG receive FIFO are asserted, the ENG receive FIFO, i.e., the engineering data, is read preferentially.
The PCIe data sending module time division multiplexes 1 route of x4 Lane PCIe link which is the same with the CSC1 data receiving module on a physical layer, and a PCIe IP core autonomously arbitrates and dispatches the data on a transmission layer.
And after the CPU software receives the valid GPIO signal, initiating a request for reading the PCIe bus, reading a corresponding SCI receiving FIFO or ENG receiving FIFO according to scheduling by the PCIe bus-based satellite-borne solid-state storage system, placing data on the AXI bus, reading the data by the PCIe IP core, sending the data to the CPU software, and completing data transmission.
The CSC4 communication control module receives UART protocol data from CPU software of the computer unit through a UART bus and completes analysis and command forwarding of the UART protocol; caching storage block addresses sent by CPU software, and caching unused block addresses, block addresses to be played back, block addresses to be erased and other addresses into different storage spaces of the DPRAM respectively; the method comprises the steps of latching control information such as time code information, an RS coding opening/closing instruction, an RS decoding opening/closing instruction, a storage soft reset instruction and the like sent by a computer unit; and organizing and storing state information such as hardware state word information, BAT table information and the like, organizing data frames according to a UART protocol, and sending the data frames to CPU software to complete data interaction. The calling relationship is shown in fig. 12.
The communication control module comprises a UART communication management module, a BAT cache management module, a memory address management module and the like.
After the power-on of the storage hardware, the BAT cache management module automatically starts a FLASH chip reset configuration operation (CSU2.3.1 FLASH reset configuration) based on the PCIe bus satellite-borne solid-state storage system, and after the reset configuration of all the FLASH chips of the storage array is completed, a CPU software instruction starts a BAT organization operation (CSU2.3.2 BAT organization).
The UART communication management module monitors the communication state of the UART bus, receives control information according to a UART protocol, and restricts the communication of command frames and data frames between CPU software and the satellite-borne solid-state storage system based on the PCIe bus in detail in CSCI data in section 6.
The UART communication management module analyzes the received serial input data, if three continuous bytes are 0xEB90A1, the UART communication management module considers that a command frame head is detected, and starts to receive and analyze parameters in the command frame, otherwise, the UART communication management module judges whether the received data is 0xEB90A1 or not; judging whether the accumulated sum is correct after receiving the complete command frame, if so, executing the corresponding command and returning a correct response; if not, the command is not executed and a corresponding response is returned.
The UART communication management module forwards the analyzed commands or states of storage soft reset, an external data input switch, an RS coding switch, an RS decoding switch, storage initialization end, flash chip reset, data transmission state, storage starting block counting, storage mark invalidation, storage time code and the like to other internal functional modules.
And the UART communication management module respectively completes data query such as hardware status words, BAT information, designated clusters and the like and feeds back the data query to the CPU software according to the instructions such as status query, BAT reading, data point reading and the like.
When a satellite-borne solid-state storage system based on a PCIe bus organizes BAT in a scanning storage area, 1 BAT item is started to write CSU4.1.1 BAT cache DPRAM operation for 1 time when 1 BAT item is organized. After the BAT items with the required length are organized according to the CPU software instructions, the CPU software starts the reading operation of the BAT items, the BAT cache management module reads out the corresponding BAT items in the CSU4.1.1 BAT cache DPRAM according to the CPU software instructions and sends the BAT items to the CPU software, and the operation is repeated until all the BAT items are read out, and then the hardware initialization operation of the storage area is completed.
The storage address management module writes a plurality of analyzed storage new block addresses, playback block addresses and erasure block addresses (at most 64 blocks) into the DPRAM partition for management, and performs autonomous maintenance on internal storage addresses according to the storage state information fed back by the storage control module; and the computer unit CPU software completes automatic management and issuing of a new storage block address, a playback block address and an erasure address of the PCIe bus-based satellite-borne solid-state storage system according to the storage hardware state word.
The UART communication management module accumulates and checks the received command frame, if the command frame is accumulated and correct, the command analysis is carried out or a storage block address receiving cache is started, if the command frame is accumulated and incorrect, the command analysis is not carried out, namely, the command forwarding or the storage block address and other information receiving caches are not carried out, and frame error information is directly fed back to the CPU software.
After the CSU4.2 UART communication management module receives a "0x33 storage address management" command frame, the state machine jumps to a state of receiving a data frame, a waiting time threshold is set to 20ms in the waiting process (if the host does not receive response data sent by the slave within a specified time (between 20us and 15 ms), the signal on the channel is considered to fail this transmission), and if a valid data frame is not received within a time-out period, the receiving process of this data frame is automatically received, and frame error information is fed back to the CPU software. If a valid data frame is received, respective DPRAM pointers of the current two channel writing, returning and erasing areas are latched, then the received storage block address information is written into the DPRAM, and after the data reception is finished, the data frame is accumulated and verified. If the verification is passed, the data frame receiving process is ended, and if the verification is not passed, the DPRAM is restored to the state before the data frame receiving process by using the pointer latched in advance, and invalid data which are just received are equivalently erased.
The CSC5 clock management module generates an input 100MHz clock signal into a 200MHz clock signal as an SDRAM logic clock, and generates a 32MHz clock signal as a storage logic clock; processing a Locked signal generated by the DCM to be used as a system global reset signal; and supports setting a CPU state register as a system soft reset signal. The calling relationship is shown in fig. 13.
The clock management module comprises a master clock management module, a reset logic management module and the like.
The master control clock management module mainly comprises CSU5.1.1 DCM0, inputs a 100MHz clock provided for an external crystal oscillator, and outputs a 200MHz clock used for related logic of SDRAM and a 32MHz clock used for related logic of NAND FLASH.
The reset logic management module generates a reset signal for use by the communication control module and a reset signal for use by the other modules.
After logic bonding, the LOCKED signal output by the DCM0 generates uart _ rst signal output for the reset use of the communication control module.
The DCM0 outputs the generated UART _ rst reset signal, and the UART command reset cmd _ FPGA _ rst signal from the communication management module are subjected to AND operation, and then a global reset signal sys _ rst is generated through BUFG and is used for resetting other modules except the communication control module.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limited. Although the present invention has been described in detail with reference to the embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A PCIe bus-based satellite-borne solid-state storage system applied to storage hardware is characterized in that the system comprises: the data receiving module and the data storage module; wherein,
the data receiving module is used for receiving engineering data and application data sent by the external main control module, performing PCIe physical layer protocol analysis and data application layer protocol analysis, generating corresponding identification signals and ping-pong buffering the data into corresponding FIFOs;
and the data storage module adopts a fixed partition storage mode, respectively and independently stores the data circularly according to the data types, and takes out the delayed data after receiving the playback instruction and transmits the delayed data to the CPU software of the computer unit.
2. The PCIe bus based on-board solid state storage system of claim 1, the data receiving module comprising: PCIe protocol analysis receiving unit, application layer protocol analysis unit, scientific data coding pre-FIFO unit and engineering data coding pre-FIFO unit; wherein,
the PCIe protocol analysis receiving unit is used for receiving engineering data and application data sent by an external main control module, finishing PCIe physical layer protocol analysis on the received data by adopting PCIe special IP (Internet protocol) check, converting the analyzed data stream and writing the converted data stream into an AXI (advanced extensible interface) bus;
the application layer protocol analysis unit is used for receiving the engineering data and the application data analyzed by the PCIe protocol analysis receiving unit, performing data packet format discrimination on the received data, and caching the data in the corresponding engineering data FIFO or application data FIFO according to the discriminated data type, and specifically includes: if the synchronous word is 0x55AA and the identification field is 0xED1D, judging that the data is an engineering data packet, starting one-time data receiving, and writing the received data into an engineering data FIFO for caching; if the synchronous word is 0x55AA and the identification field is 0x6D1D to 0x6DFD, determining that the application data packet is received, starting one-time data receiving, and writing the received data into an application data FIFO for caching; if the identification field is invalid when the synchronous word is not 0x55AA or the synchronous word is 0x55AA, the data is not received;
the scientific data pre-coding FIFO unit is used for setting relevant parameters of the application data FIFO and providing prog _ full signals to the data storage module as read FIFO triggering conditions;
the engineering data pre-coding FIFO unit is used for setting relevant parameters of the engineering data FIFO and providing prog _ full signals to the data storage module as read FIFO triggering conditions.
3. The PCIe bus based spaceborne solid state storage system as claimed in claim 1, wherein said data storage module comprises: the system comprises an RS code management unit, a data cache unit, a FLASH control management unit and a stored data output unit; wherein,
the RS coding management unit is used for receiving the engineering data and the application data cached by the data receiving module, performing RS error correction coding on the effective data according to the corresponding identification signal, and then ping-pong caching the data to the data caching unit;
the data caching unit is used for fixedly partitioning the SDRAM storage space, wherein the partition I is used for caching engineering data; the partition II is used for caching application data and then waiting for storage scheduling to be written into the FLASH chip; the data buffer area is also used for generating a signal reflecting the state of each data buffer area;
the FLASH control management unit is used for completing the storage control of engineering data and application data and the logic realization of FLASH chip bottom layer driving;
and the storage data output unit is used for carrying out RS decoding on the played-back delayed storage data and writing the delayed storage data into the FIFO cache for carrying out output preprocessing operation.
4. The PCIe bus based satellite-borne solid-state storage system according to claim 3, wherein the data caching unit is used for caching engineering data by fixedly partitioning an SDRAM storage space; the partition II is used to cache application data, and then waits for storage scheduling to be written into the FLASH chip, and specifically includes:
when the data volume of any one of the engineering data cache FIFO or the application data cache FIFO reaches a threshold value, starting the reading operation of the corresponding cache FIFO; at the moment, if the RS coding enabling signal is valid, RS coding is carried out on the read data and the read data are written into the asynchronous ping-pong FIFO cache, and if the RS coding enabling signal is invalid, the read data are directly written into the asynchronous ping-pong FIFO cache; recording the corresponding channel number when data is written into the asynchronous ping-pong FIFO;
when the data amount of any asynchronous FIFO buffer reaches 256 × 128bits, the RS coding management unit outputs an effective corresponding asynchronous ping-pong FIFO half-full signal, and outputs corresponding FIFO buffer data and a channel number after receiving an RS coded asynchronous ping-pong FIFO read enable signal;
when the asynchronous ping-pong FIFO half-full signal is effective, starting corresponding asynchronous ping-pong FIFO read operation and SDRAM write operation, and writing the coded data source packet into the corresponding partition cache of SDRAM according to the channel number from the RS coding management module;
when the buffer memory data volume of any partition of the SDRAM engineering data partition or the application data partition is full of 4 clusters, starting 1 time of FLASH writing operation, reading target data from the SDRAM by clusters and writing the target data into a rear-end asynchronous buffer FIFO by a data buffer unit according to SDRAM read request signals and read addresses from a FLASH control management unit; and the FLASH control management unit finishes reading the data of the asynchronous cache FIFO cluster according to the signal sent by the data cache unit.
5. The PCIe bus-based satellite-borne solid-state storage system according to claim 3, wherein the completing the logic implementation of the FLASH chip underlying driver specifically comprises:
generating a drive signal for the operation of the FLASH memory device and meeting the requirement of the operation time sequence; operations include reset, read, write, and erase;
automatically loading cluster mark information and system time code information in a free area of each used cluster in a storage area;
starting the operation aiming at the appointed FLASH storage area through a software command;
automatically detecting error information in the storage area, marking the clusters and blocks with errors, and timely notifying equipment software;
maintaining relevant information representing the hardware working state, wherein the information can be read by software from a specific address;
after each power-on, automatically generating BAT reflecting the use condition of all blocks in the storage area, and reading the BAT from the specified address by software;
and automatically managing the storage block address sent by the CPU software of the computer unit, and completing automatic fetching, checking and forwarding of an unused block address, a block address to be replayed, a block address to be erased, an invalid block address to be marked and a CPU cluster reading address cached in the communication control module according to the storage task scheduling.
6. The PCIe bus based on satellite borne solid state storage system of claim 1, the system further comprising a data sending module; the data sending module comprises: SCI receives FIFO unit, ENG receives FIFO unit, data scheduling unit and PCIe data sending unit; wherein,
the SCI receiving FIFO unit is used for receiving the application data played back by the data storage module and performing clock domain crossing processing; setting SCI receiving FIFO related parameters and providing prog _ full signals to the data scheduling unit as reading FIFO triggering conditions;
the ENG receiving FIFO unit is used for receiving the engineering data played back by the data storage module and performing clock domain crossing processing; setting related parameters of an ENG receiving FIFO, and providing a prog _ full signal to a data scheduling unit as a read FIFO triggering condition;
the data scheduling unit is used for monitoring the prog _ full signal states of the SCI receiving FIFO unit and the ENG receiving FIFO unit, if the prog _ full signal is '1', the GPIO signal is set to be effective, CPU software is informed to read a PCIe bus, and otherwise, the GPIO outputs high level to indicate that PCIe is unreadable; when the prog _ full signals of the SCI receiving FIFO unit and the ENG receiving FIFO unit are both effective, the ENG receiving FIFO, namely engineering data, is read preferentially;
the PCIe data sending unit is used for multiplexing the same link of the data receiving module in time division on a physical layer, and the transmission layer is autonomously arbitrated and scheduled by a PCIe IP core; and after the CPU software receives the GPIO signal to be valid, initiating a request for reading a PCIe bus, reading a corresponding SCI receiving FIFO or an ENG receiving FIFO according to scheduling by the PCIe bus-based satellite-borne solid-state storage system, placing data on an AXI bus, reading the data by a PCIe IP core, sending the data to the CPU software, and completing data transmission.
7. The PCIe bus based spaceborne solid state storage system as claimed in claim 1, further comprising a communication control module; the communication control module includes: the UART communication management unit, the BAT cache management unit and the memory address management unit are arranged in the storage area; wherein,
the BAT cache management unit is used for starting the reset configuration operation of the FLASH chips, and after the reset configuration of all the FLASH chips of the storage array is completed, the BAT organization operation is started by the instruction of CPU software;
the UART communication management unit is used for monitoring the communication state of the UART bus and receiving control information according to a UART protocol; analyzing received serial input data based on communication constraints of command frames and data frames between CPU software and a PCIe bus-based satellite-borne solid-state storage system, if three continuous bytes are 0xEB90A1, determining that a command frame header is detected, and starting receiving and analyzing parameters in the command frame, otherwise, judging whether the received data is 0xEB90A1 all the time; judging whether the accumulated sum is correct after receiving the complete command frame, if so, executing a corresponding command and returning a correct response; if not, the command is not executed and corresponding response is returned; forwarding the analyzed command or state including storage soft reset, an external data input switch, an RS coding switch, an RS decoding switch, storage initialization end, FLASH chip reset, data transmission state, storage starting block counting, storage mark invalidation and storage time code to other internal functional modules; the system is used for respectively finishing data query including hardware status words, BAT information and designated clusters according to instructions including status query, BAT read and data point read, and feeding back the data query to CPU software; the CPU is also used for accumulating and checking the received command frame, if the accumulated sum is correct, performing instruction analysis or starting a storage block address receiving cache, and if the accumulated sum is incorrect, not performing instruction analysis, namely not performing instruction forwarding or storage block address and other information receiving caches, and directly feeding back frame error information to the CPU software;
and the storage address management unit is used for writing the analyzed new storage block addresses, playback block addresses and erasure block addresses into the DPRAM partition for management, and performing autonomous maintenance on the internal storage addresses according to the storage state information.
8. The PCIe bus based spaceborne solid state storage system as claimed in claim 7, further comprising a clock management module; the clock management module comprises: the master control clock management unit and the reset logic management unit;
the master control clock management unit comprises a CSU5.1.1 DCM0, inputs a clock provided for an external crystal oscillator and outputs a clock used by related logic of SDRAM and a clock used by related logic of FLASH;
the reset logic management unit is used for generating reset signals used by the modules, wherein the LOCKED signal output by the DCM0 generates uart _ rst signal output after being logically bonded, and is only used for resetting the communication control module; DCM0 outputs the generated UART _ rst reset signal and UART instruction reset cmd _ FPGA _ rst signal phase from the UART communication management unit, and then generates a global reset signal sys _ rst through BUFG for resetting other modules except the communication control module.
9. A PCIe bus based on-board solid state storage method, which is implemented based on the system of any one of claims 1 to 8, and the method comprises:
after the storage hardware is powered on, the satellite-borne solid-state storage system based on the PCIe bus automatically starts the FLASH chip reset configuration operation, and after the reset configuration of all the FLASH chips of the storage array is completed, the BAT organization operation is started by the instruction of CPU software; during traversing data of all cluster vacant areas of the storage array, performing corresponding Hamming decoding operation, and after BAT information of the storage array FLASH is read by the CPU, entering a conventional task management state after the storage system receives an initialization ending command;
the data storage module monitors the state of each data cache region from an internal data cache unit, when the cache data volume of any partition of the SDRAM is full of 4 clusters, the data storage module starts 1 time of FLASH write operation according to task scheduling, sends an SDRAM read request signal to the data cache module, and starts four-level flow data writing;
after each stage of pipelining is loaded with valid data, cleaning corresponding SDRAM space use identification; detecting the internal programming state of the storage area after programming is finished, marking the erroneous clusters and blocks, and informing storage management software; if storage failure occurs, automatically rewriting failure cluster data into the replacement block;
when the data storage module receives a data playback instruction, starting FLASH reading operation according to a playback address sent by the communication control module, reading corresponding data according to a cluster, and sending the read data to the stored data output unit;
and when the data storage module receives a data erasing instruction, according to an erasing address sent by the communication control module, starting the erasing operation of the FLASH block according to the four-level flow.
10. The PCIe bus based spaceborne solid state storage method according to claim 9 wherein the method further designs an error detection and correction coding protection mechanism and a bad block management mechanism: wherein,
the error detection and correction coding protection mechanism specifically comprises:
for the data stored in the FLASH main storage area, an RS coding technology with higher performance is selected, namely RS coding is firstly carried out before the engineering data and the application data are stored, then the RS coding is written into the main storage area, and during data playback, RS decoding error correction is firstly carried out, and then the RS coding error correction is transmitted to CPU software;
for the auxiliary information storage of each page of spare area of FLASH, a simple and reliable Hamming coding technology is selected, namely Hamming coding protection is carried out when file information is stored, and Hamming decoding error correction is carried out when the file information is read;
the bad block management mechanism specifically comprises:
processing invalid blocks existing when the FLASH chip leaves a factory as static bad blocks, and not using the invalid blocks;
for an invalid block newly added in the use process of a chip, the invalid block is taken as a dynamic bad block and managed by software and hardware together, and the specific method comprises the following steps: for the condition of programming failure, carrying out normal reading operation on the page which is normally programmed in the block, but rewriting the data of the page which is failed in programming in another effective block, and marking the block at the same time to avoid further writing or erasing the block later; for an erasure failure block, marking the block invalid in a PCIe bus-based satellite-borne solid-state storage system to avoid writing or erasing the block later; the dynamic bad block can be selected not to be used any more in the future, and can also be selected to be used again after the storage area maintenance is carried out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310001279.4A CN115687228B (en) | 2023-01-03 | 2023-01-03 | PCIe bus-based satellite-borne solid-state storage system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310001279.4A CN115687228B (en) | 2023-01-03 | 2023-01-03 | PCIe bus-based satellite-borne solid-state storage system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115687228A true CN115687228A (en) | 2023-02-03 |
| CN115687228B CN115687228B (en) | 2023-05-02 |
Family
ID=85057597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310001279.4A Active CN115687228B (en) | 2023-01-03 | 2023-01-03 | PCIe bus-based satellite-borne solid-state storage system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115687228B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116301266A (en) * | 2023-03-03 | 2023-06-23 | 无锡众星微系统技术有限公司 | PCIe (peripheral component interconnect express) equipment in-band resetting method and device based on security authentication |
| CN116299463A (en) * | 2023-05-16 | 2023-06-23 | 四川天府新区北理工创新装备研究院 | Small sar imaging system and method based on rear end of general computing device |
| CN116501268A (en) * | 2023-06-28 | 2023-07-28 | 牛芯半导体(深圳)有限公司 | Data reading method applied to DDR PHY |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718219A (en) * | 2016-01-19 | 2016-06-29 | 浙江大学 | Method and module for high-speed data transmission and storage based on USB3.0 |
| CN110334040A (en) * | 2019-07-09 | 2019-10-15 | 中国科学院国家空间科学中心 | A kind of spaceborne solid-state memory system |
| US20200074288A1 (en) * | 2017-12-06 | 2020-03-05 | Tencent Technology (Shenzhen) Company Limited | Convolution operation processing method and related product |
| CN112181304A (en) * | 2020-09-30 | 2021-01-05 | 浙江大学 | Satellite-borne NAND Flash storage management system |
-
2023
- 2023-01-03 CN CN202310001279.4A patent/CN115687228B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718219A (en) * | 2016-01-19 | 2016-06-29 | 浙江大学 | Method and module for high-speed data transmission and storage based on USB3.0 |
| US20200074288A1 (en) * | 2017-12-06 | 2020-03-05 | Tencent Technology (Shenzhen) Company Limited | Convolution operation processing method and related product |
| CN110334040A (en) * | 2019-07-09 | 2019-10-15 | 中国科学院国家空间科学中心 | A kind of spaceborne solid-state memory system |
| CN112181304A (en) * | 2020-09-30 | 2021-01-05 | 浙江大学 | Satellite-borne NAND Flash storage management system |
Non-Patent Citations (4)
| Title |
|---|
| JIA LANXU 等: ""Design of PCIe-Gigabit Ethernet High-speed Data Interaction System Based on FPGA"" * |
| 周立国 等: ""基于PCI Express总线的SAR回波数据实时记录系统"" * |
| 董振兴: ""基于FPGA平台的深度学习应用研究"" * |
| 黄丹萍: ""基于PCIe的高速数据采集模块的软件设计与实现"" * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116301266A (en) * | 2023-03-03 | 2023-06-23 | 无锡众星微系统技术有限公司 | PCIe (peripheral component interconnect express) equipment in-band resetting method and device based on security authentication |
| CN116301266B (en) * | 2023-03-03 | 2023-11-17 | 无锡众星微系统技术有限公司 | PCIe (peripheral component interconnect express) equipment in-band resetting method and device based on security authentication |
| CN116299463A (en) * | 2023-05-16 | 2023-06-23 | 四川天府新区北理工创新装备研究院 | Small sar imaging system and method based on rear end of general computing device |
| CN116299463B (en) * | 2023-05-16 | 2023-08-08 | 四川天府新区北理工创新装备研究院 | Small sar imaging system and method based on rear end of general computing device |
| CN116501268A (en) * | 2023-06-28 | 2023-07-28 | 牛芯半导体(深圳)有限公司 | Data reading method applied to DDR PHY |
| CN116501268B (en) * | 2023-06-28 | 2024-02-27 | 牛芯半导体(深圳)有限公司 | Data reading method applied to DDR PHY |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115687228B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115687228B (en) | PCIe bus-based satellite-borne solid-state storage system and method | |
| KR100962769B1 (en) | Supercharge message exchanger | |
| CN101681323B (en) | Device directed memory barriers | |
| CN100524252C (en) | Embedded system chip and data read-write processing method | |
| JP4477906B2 (en) | Storage system | |
| CN101965559B (en) | Flash memory controller having crosspoint switch connecting processer and internal storage | |
| CN103282887A (en) | Controller and method for performing background operations | |
| US10552047B2 (en) | Memory system | |
| CN103034454B (en) | Flexible flash command | |
| CN103092782A (en) | Flash controller hardware architecture for flash devices | |
| US20080147969A1 (en) | Separate Handling of Read and Write of Read-Modify-Write | |
| CN110334040B (en) | Satellite-borne solid-state storage system | |
| CN101364346A (en) | Embedded real-time intelligent traffic monitoring system based on video stream | |
| CN107491267B (en) | High-speed image data storage device based on L VDS interface | |
| US7484030B2 (en) | Storage controller and methods for using the same | |
| CN103092781A (en) | Effective utilization of flash interface | |
| KR102645983B1 (en) | Open channel vector command execution | |
| CN110737618B (en) | Method, device and storage medium for embedded processor to carry out rapid data communication | |
| CN101777030B (en) | Device and method for verifying data transmission system | |
| US20140101339A1 (en) | Efficient Scheduling of Read and Write Transactions in Dynamic Memory Controllers | |
| JPH11175311A (en) | Flash memory system | |
| CN114860158A (en) | High-speed data acquisition and recording method | |
| JP2002157091A (en) | Storage sub-system, and memory used therefor | |
| US8402233B2 (en) | Method and apparatus for high throughput mass storage device interface in a microprocessor for handheld systems | |
| CN117251117B (en) | Satellite-borne solid-state storage system based on serial bus polling management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |