CN106126442A - A kind of data store organisation and remote sensing satellite data-storage system - Google Patents
A kind of data store organisation and remote sensing satellite data-storage system Download PDFInfo
- Publication number
- CN106126442A CN106126442A CN201610439984.2A CN201610439984A CN106126442A CN 106126442 A CN106126442 A CN 106126442A CN 201610439984 A CN201610439984 A CN 201610439984A CN 106126442 A CN106126442 A CN 106126442A
- Authority
- CN
- China
- Prior art keywords
- data
- address
- submodule
- unit
- remote sensing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
- G06F12/10—Address translation
- G06F12/109—Address translation for multiple virtual address spaces, e.g. segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A kind of data store organisation and remote sensing satellite data-storage system, data store organisation includes address module and data module；Wherein data module includes N2Individual data submodule；Address module includes N1Individual address submodule, each address submodule includes M1Individual address location, wherein before (M11) in individual address location, each address location stores the first physical address of a data submodule, M respectively1The first physical address of individual address location storage next address submodule；Data-storage system includes data receipt unit, data uploading unit and user interface section；Utilizing this data store organisation and data-storage system, can realize real time record and the access process of magnanimity remote sensing satellite data, the data drastically increasing ground system process ageing with record.
Description
Technical field
The present invention relates to space remote sensing field of data storage, be specifically related to a kind of remotely-sensed data storage organization and data storage
System, a kind of for realizing data store organisation and the data-storage system that remote sensing satellite high speed mass storage accesses.
Background technology
Existing remote sensing satellite data-storage system uses TCP/IP (Transmission Control Protocol/
Internet Protocol, transmits interconnection protocol between control protocol/network) agreement carries out data transmission, and will receive data
It is stored on the internal memory of operating system, the most again these data is carried out back-end processing.There is handling process and postpone long, real in the program
The defects such as time property difference, specific as follows:
1) due to ICP/IP protocol framework level complicated (include physical layer, link layer, Internet, transport layer, session layer,
Represent layer, application layer), therefore use this agreement to carry out data transmission there is uncertain time delay；Additionally, should in this protocol infrastructure
Need software support (such as transmission control protocol TCP) with layer, thus processor can be caused certain expense；ICP/IP protocol is not
Support the error correction retransmission of hardware；
2) being to store receiving data on the internal memory of operating system due to existing scheme, and this internal memory is based on actual thing
The virtual address of reason medium, virtual address needs to be carried out maintenance management by operating system with the mapping management relation of physical address,
Therefore after internal memory use reaches a certain size, processor can be caused certain expense；Further, since the behaviour that server uses
It is non-real time operating system as system, there is the hierarchical relational of client layer and inner nuclear layer, therefore reception data are stored in thing
Need the participation of processor on reason medium, certain computing cost and storage time delay can be caused.
Summary of the invention
The technology of the present invention solves problem: overcome the deficiencies in the prior art, it is provided that a kind of data store organisation and distant
Sense satellite data storage system, uses the data store organisation that data separate with address, it is achieved the quick reading of data, Jin Erke
To realize real time record and the access process of remote sensing satellite mass data, meet remote sensing satellite mass data and process in real time and remember
The demand of record.
The technical solution of the present invention is: a kind of data store organisation, including address module and data module；
Described data module is used for storing data, including N2Individual data submodule；
Described address module is for storing the address information of data, including N1Individual address submodule, each address submodule
Including M1Individual address location, wherein before (M1-1) in individual address location, each address location stores a data submodule respectively
First physical address, M1The first physical address of individual address location storage next address submodule.
Described data submodule is the most incoherent discrete data submodule, has continuously inside each data submodule
Physical address.
Described address submodule is the most incoherent discrete address submodule, has continuously inside the submodule of each address
Address.
Described N1The M of individual address submodule1Individual address location stores the first physical address of the 1st address submodule.
A kind of remote sensing satellite data-storage system, including data receipt unit, data uploading unit and user interface section；
Described data receipt unit is for receiving the data from front end delivery；
Described data uploading unit is for receiving the output data from data receipt unit, and data are uploaded to user
Specified memory,
Described user interface section opens up data store organisation in remote sensing satellite host computer, it is achieved to data uploading unit
Uploading the management of data, described data store organisation is data store organisation described above.
Described data receipt unit is RapidIO data receipt unit, and described RapidIO data receipt unit is used for receiving
The data that Rapidio agreement is transmitted are used from front end.
Described data uploading unit is PCIE data uploading unit, and described PCIE data uploading unit receives and connects from data
Receive the output data of unit, and pack the data to PCIE protocol format and be uploaded to user's specified memory.
Described data uploading unit uses real-time DMA technology (Direct Memory Access, direct memory access (DMA))
Data are uploaded to user's specified memory.
Described PCIE data uploading unit uses real-time DMA technology that data are uploaded to user's specified memory.
The present invention compared with prior art provides the benefit that:
The present invention arranges the data store organisation including data module and address module, forms address link list in address module
The realization access to data module, and then real time record and the access process of remote sensing satellite mass data can be realized, meet
The demand that remote sensing satellite mass data processes in real time and records.
Accompanying drawing explanation
Fig. 1 is data store organisation schematic diagram；
Fig. 2 is remote sensing satellite data storage system architecture schematic diagram in the present invention；
Fig. 3 is RapidIO data receipt unit schematic flow sheet；
Fig. 4 is PCIE data uploading unit schematic flow sheet；
Fig. 5 is user interface section schematic flow sheet.
Detailed description of the invention
In order to make technical problem solved by the invention, technical scheme and beneficial effect clearer, below in conjunction with
Drawings and Examples, are further elaborated to the present invention, it will be appreciated that specific embodiment described herein only in order to
Explain the present invention, be not intended to limit the present invention.
It is illustrated in figure 1 data store organisation schematic diagram in the present invention, from fig. 1, it can be seen that a kind of data that the present invention proposes
Storage organization, including address module and data module；
Described data module is used for storing data, including N2Individual data submodule；
Described address module is for storing the address information of data, including N1Individual address submodule, each address submodule
Including M1Individual address location, wherein before (M1-1) in individual address location, each address location stores a data submodule respectively
First physical address, M1The first physical address of individual address location storage next address submodule.
Further, in order to preferably realize the access to data, prevent data or address overlap to data storage, access
With the interference processed, in actual applications, need to ensure N2Individual data submodule is the most incoherent discrete data submodule,
Inside each data submodule, there is continuation address.N1Individual address submodule is the most incoherent discrete data submodule, often
Inside the submodule of individual address, there is continuation address.Each data submodule of the above discrete finger and each address submodule it
Between there is no overlap or continuous print address.
Further, described N1The M of individual address submodule1Individual address location stores the first thing of the 1st address submodule
Reason address, so can form a data link table, such that it is able to realize proceeding by data from any one address submodule
Read, to the address information finally having read all address modules.
Number N of the above data submodule2By being actually needed the data volume carrying out storing or process and every number
Determine according to the amount of storage of submodule, need to ensure number N of data submodule2Product with the amount of storage of each data submodule
More than or equal to being actually needed the data volume carrying out storing or process.
Based on above-mentioned data store organisation, present invention further propose that a kind of remote sensing satellite data-storage system, system
As in figure 2 it is shown, as can be seen from Figure 2, the remote sensing satellite data-storage system in the present invention includes in data receipt unit, data block diagram
Leaflet unit and user interface section；
Wherein data receipt unit is for receiving the data from front end delivery；In actual applications, above-mentioned data connect
Receiving unit and be preferably RapidIO data receipt unit, described RapidIO data receipt unit uses from front end for receiving
The data that Rapidio agreement is transmitted, due to RapidIO protocol infrastructure level the simplest (include physical layer, transport layer,
Logical layer), therefore compared to use ICP/IP protocol, use RapidIO agreement time data transmission delay the lowest negligible not
Meter；And this agreement is simple, flow-control mechanism and software complexity low so that error correction retransmission is machine-processed or even whole protocol stack is prone to adopt
Realize with hardware, processor will not be brought expense；Additionally, use this agreement to there is also advantages below: high efficiency, low system become
This, support the point-to-point or communication of point-to-multipoint, supports dma operation, supports messaging mode exchange data, supports dispersion
Process and many master control systems, support multiple topological structure.
Data uploading unit is for receiving the output data from data receipt unit, and data are uploaded to user's appointment
Internal memory；
Further, above-mentioned data uploading unit is preferably PCIE data uploading unit, described PCIE data uploading unit
Receive from the output data of data receipt unit, and pack the data to PCIE protocol format and be uploaded to user's specified memory.
Owing to using PCIE agreement so that receive the intervention without processor when data store internal memory, and utilize PCIE agreement to pass
Transmission of data is to use actual physical address, and therefore transmission delay is the lowest.It addition, PCIE agreement has many advantages itself: association
View framework is simple, be prone to hardware realizes, packing efficiency is high, transmission delay is low.
Further, in order to improve the speed of digital independent further, data uploading unit uses real-time DMA technology by number
According to being uploaded to user's specified memory.
User interface section opens up data store organisation in remote sensing satellite host computer, it is achieved upload data uploading unit
The management of data, described data store organisation is arbitrary above-mentioned data store organisation.
Specific embodiment
Open up two kinds of discrete memory modules at host computer, be respectively used to store data module and address module.Data module
In have N2Individual size is 4KB, incoherent discrete address, but is continuous print in 4KB.Address module is to have n/512 the size to be
2056byte (address of 64,257*8byte), incoherent discrete address, but address in it should be noted that 2056byte
It it is continuous print.Each address module 257 address locations of storage, the most front Unit 256 are the first thing of the storage every 4KB of data module
Reason address, the 257th is the first physical address of next address module, and the 257th memory element of the n-th/512 address is first
The first physical address of individual address module.Thus form an address link list.
FPGA (Field-Programmable Gate Array, field-programmable gate array) program is according to the ground issued
First physical address in the module of location, reads 257 addresses (size of the most each address module) every time, when 256 addresses are finished,
Further according to the content of the 257th address, read the content in next address module, the most repeatedly.
RapidIO data receipt unit in the present embodiment, major function uses Rapidio agreement for receiving from front end
Carry out the data of high-speed transfer；The interface of this part is 1 4XRapidio interface, and physical form is QSFP (Quad Small
Form-factor Pluggable, the four-way signal of telecommunication is converted to optical signal interface).This part by data receiver main control module,
Receive buffer control module, Rapdio protocol conversion module composition.
First the handling process of this unit as it is shown on figure 3, particularly as follows: initialize RapidIO data receipt unit；
After completing to front end data source send reset instruction bag, and etc. reset instruction respond packet to be feedback；When receiving reset instruction
Request data respond packet is sent to data source after respond packet；The respond packet to be received such as then, and the respond packet received is carried out
Identify, it is judged that this respond packet, for having data response or no data response, if there being data to respond, then waits front end data source
The data sended over, respond if no data, start request process next time or end.After receiving data and wait always
Determine whether data volume size during request, meet and require start transmission next time or terminate.
PCIE data uploading unit in the present embodiment, major function is to receive the output of RapidIO data receipt unit
Data, and pack the data to PCIE protocol format, use DMA to be automatically uploaded to user according to the parameter of initial configuration and specify
Internal memory in.This part is transmitted master control by PCIE, address controls, data buffer storage controls, PCIE protocol conversion forms.
The handling process of this unit as shown in Figure 4, particularly as follows: first receive the reset signal issued from host computer, and
PCIE board is resetted；Receive the register parameters from host computer configuration, and configure according to relevant variable；Read
Data in the address FIFO (First Input First Output, first in first out) that host computer is set up, and judge data
Whether FIFO there are data, if had, transferring data in the FIFO of address in the physics first address indicated by data.
User interface section in the present embodiment, (size is adjustable in order to open up appointment size in management host computer for major function
Joint, and no maximum) internal memory, it is achieved to the configuration management of hardware program and data pipe that PCIE data uploading unit is uploaded
Reason.This part is managed by data buffer storage, address caching manages, board initializes, hardware state monitoring forms.
First the handling process of this unit as it is shown in figure 5, particularly as follows: initialize the discrete interior poke required for internal memory generates
According to structure and the hardware specified that resets, and issue initial configuration parameter to hardware；Initialization starts to start in data after completing
Spreading journey, data judge whether to meet outputting standard after uploading to relief area, if it is satisfied, then replicate number in units of Frame
According in the internal memory specified, wait that user processes further.
The content not being described in detail in description of the invention belongs to the known technology of professional and technical personnel in the field.
Describe in detail the present invention above by concrete with preferred embodiment, but those skilled in the art should be bright
In vain, the invention is not limited in embodiment described above, all within the spirit and principles in the present invention, changed, etc.
With replacement etc., should be included within the scope of the present invention.
Claims (9)
1. a data store organisation, it is characterised in that: include address module and data module；
Described data module is used for storing data, including N2Individual data submodule；
Described address module is for storing the address information of data, including N1Individual address submodule, each address submodule includes M1
Individual address location, wherein before (M1-1) in individual address location, each address location stores the first physics of a data submodule respectively
Address, M1The first physical address of individual address location storage next address submodule.
A kind of data store organisation the most according to claim 1, it is characterised in that: described data submodule is the most not phase
The discrete data submodule closed, has continuous physical address inside each data submodule.
A kind of data store organisation the most according to claim 1, it is characterised in that: described address submodule is the most not phase
The discrete address submodule closed, has continuation address inside the submodule of each address.
A kind of data store organisation the most according to claim 1, it is characterised in that: described N1The of individual address submodule
M1Individual address location stores the first physical address of the 1st address submodule.
5. a remote sensing satellite data-storage system, it is characterised in that: include data receipt unit, data uploading unit and user
Interface unit；
Described data receipt unit is for receiving the data from front end delivery；
Described data uploading unit is for receiving the output data from data receipt unit, and data are uploaded to user's appointment
Internal memory,
Described user interface section opens up data store organisation in remote sensing satellite host computer, it is achieved upload data uploading unit
The management of data, described data store organisation is arbitrary described data store organisation in Claims 1 to 4.
A kind of remote sensing satellite data-storage system the most according to claim 5, it is characterised in that: described data receipt unit
For RapidIO data receipt unit, described RapidIO data receipt unit uses Rapidio agreement for receiving from front end
The data being transmitted.
A kind of remote sensing satellite data-storage system the most according to claim 5, it is characterised in that: described data uploading unit
For PCIE data uploading unit, described PCIE data uploading unit receives the output data from data receipt unit, and by number
It is uploaded to user's specified memory according to being packaged into PCIE protocol format.
A kind of remote sensing satellite data-storage system the most according to claim 5, it is characterised in that: described data uploading unit
Use real-time DMA technology that data are uploaded to user's specified memory.
A kind of remote sensing satellite data-storage system the most according to claim 5, it is characterised in that: described PCIE data are uploaded
Unit uses real-time DMA technology that data are uploaded to user's specified memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610439984.2A CN106126442A (en) | 2016-06-17 | 2016-06-17 | A kind of data store organisation and remote sensing satellite data-storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610439984.2A CN106126442A (en) | 2016-06-17 | 2016-06-17 | A kind of data store organisation and remote sensing satellite data-storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106126442A true CN106126442A (en) | 2016-11-16 |
Family
ID=57470115
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610439984.2A Pending CN106126442A (en) | 2016-06-17 | 2016-06-17 | A kind of data store organisation and remote sensing satellite data-storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106126442A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1517906A (en) * | 2003-01-14 | 2004-08-04 | 联想(北京)有限公司 | Document system and document management method |
| CN1757202A (en) * | 2002-08-13 | 2006-04-05 | 英特尔公司 | Free list and ring data structure management |
| CN1797379A (en) * | 2004-12-24 | 2006-07-05 | 华为技术有限公司 | Method of data transmission by using mode of direct memory access |
| CN101582092A (en) * | 2009-06-12 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for realizing the store of date in memory |
| CN101887398A (en) * | 2010-06-25 | 2010-11-17 | 浪潮(北京)电子信息产业有限公司 | Method and system for dynamically enhancing input/output (I/O) throughput of server |
| CN102832988A (en) * | 2012-08-22 | 2012-12-19 | 中国科学院对地观测与数字地球科学中心 | System and method for acquiring and distributing baseband data of remote sensing satellite |
| CN103176907A (en) * | 2011-12-22 | 2013-06-26 | 北京新媒传信科技有限公司 | Memory management method and memory management system |
| CN103401846A (en) * | 2013-07-15 | 2013-11-20 | 杭州华为数字技术有限公司 | Data processing method, protocol conversion equipment and Internet |
| US20140047553A1 (en) * | 2011-04-29 | 2014-02-13 | Beijing Zhongtian Antai Technology Co., Ltd. | Method and apparatus for data security reading |
| CN103646063A (en) * | 2013-11-27 | 2014-03-19 | 中国航天科技集团公司第五研究院第五一三研究所 | Satellite-borne high-speed file management system |
| CN104780019A (en) * | 2014-12-03 | 2015-07-15 | 中国航天科工集团第三研究院第八三五七研究所 | Method for storing optical fiber communication data |
| CN104933001A (en) * | 2015-06-15 | 2015-09-23 | 山东超越数控电子有限公司 | Double-controller data communication method based on RapidIO technology |
-
2016
- 2016-06-17 CN CN201610439984.2A patent/CN106126442A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1757202A (en) * | 2002-08-13 | 2006-04-05 | 英特尔公司 | Free list and ring data structure management |
| CN1517906A (en) * | 2003-01-14 | 2004-08-04 | 联想(北京)有限公司 | Document system and document management method |
| CN1797379A (en) * | 2004-12-24 | 2006-07-05 | 华为技术有限公司 | Method of data transmission by using mode of direct memory access |
| CN101582092A (en) * | 2009-06-12 | 2009-11-18 | 中兴通讯股份有限公司 | Method and device for realizing the store of date in memory |
| CN101887398A (en) * | 2010-06-25 | 2010-11-17 | 浪潮(北京)电子信息产业有限公司 | Method and system for dynamically enhancing input/output (I/O) throughput of server |
| US20140047553A1 (en) * | 2011-04-29 | 2014-02-13 | Beijing Zhongtian Antai Technology Co., Ltd. | Method and apparatus for data security reading |
| CN103176907A (en) * | 2011-12-22 | 2013-06-26 | 北京新媒传信科技有限公司 | Memory management method and memory management system |
| CN102832988A (en) * | 2012-08-22 | 2012-12-19 | 中国科学院对地观测与数字地球科学中心 | System and method for acquiring and distributing baseband data of remote sensing satellite |
| CN103401846A (en) * | 2013-07-15 | 2013-11-20 | 杭州华为数字技术有限公司 | Data processing method, protocol conversion equipment and Internet |
| CN103646063A (en) * | 2013-11-27 | 2014-03-19 | 中国航天科技集团公司第五研究院第五一三研究所 | Satellite-borne high-speed file management system |
| CN104780019A (en) * | 2014-12-03 | 2015-07-15 | 中国航天科工集团第三研究院第八三五七研究所 | Method for storing optical fiber communication data |
| CN104933001A (en) * | 2015-06-15 | 2015-09-23 | 山东超越数控电子有限公司 | Double-controller data communication method based on RapidIO technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104572574B (en) | Ethernet controller IP kernel and method based on gigabit Ethernet vision agreement | |
| CN103946828B (en) | Data processing system and method | |
| CN101841470B (en) | High-speed capturing method of bottom-layer data packet based on Linux | |
| CN111061425B (en) | Host, nonvolatile memory flash solid state drive and method of storing services | |
| CN103902486B (en) | System, device and method for implementation of remote direct memory access | |
| CN108701004A (en) | A kind of system of data processing, method and corresponding intrument | |
| CN102065569B (en) | Ethernet MAC sublayer controller suitable for WLAN | |
| CN103959255A (en) | System and method to reduce memory access latencies using selective replication across multiple memory ports | |
| CN102314400B (en) | Method and device for dispersing converged DMA (Direct Memory Access) | |
| CN102014150A (en) | Distributed small file storage system based on UDP (User Datagram Protocol) and data processing method thereof | |
| CN102185833B (en) | Fiber channel (FC) input/output (I/O) parallel processing method based on field programmable gate array (FPGA) | |
| CN110442532A (en) | The whole world of equipment for being linked with host can store memory | |
| CN107171978A (en) | Multi-channel data circulation acquisition system and method for gigabit Ethernet | |
| CN110109852A (en) | System and method for realizing TCP _ IP protocol by hardware | |
| CN106662895A (en) | Computer device and data read-write method for computer device | |
| CN110532208A (en) | A kind of data processing method, interface conversion structure and equipment | |
| CN102033818A (en) | Buffering in media and pipelined processing components | |
| CN102567278A (en) | On-chip multi-core data transmission method and device | |
| CN107038134A (en) | A kind of SRIO interface solid hard disks system and its implementation based on FPGA | |
| CN103517085B (en) | Method for implementing remote server management based on video decoding design | |
| CN101562544B (en) | Data packet generator and data packet generating method | |
| CN106292518A (en) | Remote PLC monitoring and debugging system and method | |
| CN105518610B (en) | Storage system, method and apparatus for processing operation request | |
| CN108090018A (en) | Method for interchanging data and system | |
| CN104317747A (en) | Data caching and sending device and method of network receiver |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100085 room A515, building 79, Shuang Qing Lu, Haidian District, Beijing. Applicant after: Beijing yuatu Remote Sensing Technology Development Co., Ltd. Address before: 100085 Beijing Haidian District Haidian District 79 Road, the enlightening Star Room 515 Applicant before: Beijing Beijing Kun Yi Yi science and Technology Development Co., Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161116 |