CN110557369A - high-speed data processing platform based on domestic operating system kernel mode - Google Patents
high-speed data processing platform based on domestic operating system kernel mode Download PDFInfo
- Publication number
- CN110557369A CN110557369A CN201910678731.4A CN201910678731A CN110557369A CN 110557369 A CN110557369 A CN 110557369A CN 201910678731 A CN201910678731 A CN 201910678731A CN 110557369 A CN110557369 A CN 110557369A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- forwarding
- protocol
- speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/165—Combined use of TCP and UDP protocols; selection criteria therefor
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The high-speed data processing platform based on the kernel state of the domestic operating system comprises a high-speed data receiving module, a data storage module and an escape processing module, wherein the high-speed data receiving module captures a network data packet through a callback function and stores the network data packet into a data receiving buffer area, the data storage module is notified through soft interrupt, the data storage module notifies a user state data storage module after reading the network data packet in the data receiving buffer area, a file transfer protocol is adopted to realize the storage of an original data file in a remote shared file server, the escape processing module reads the network data packet in the data receiving buffer area, performs escape processing and classified temporary storage, and notifies a data forwarding module through soft interrupt to forward, and the data forwarding module forwards data to different users according to forwarding rules.
Description
Technical Field
The invention relates to a receiving, storing and forwarding processing platform for high-speed data, belonging to the technical field of computer networks.
Background
The existing satellite ground computer application system mainly comprises two parts, namely plan management, data distribution and operation control, wherein a plan management and data distribution subsystem comprises a network front-end computer, a data distribution server, SAN storage and the like, and the core task is to receive, distribute and store return user data and assist a ground monitoring subsystem to establish a forward spatial link.
The existing data distribution equipment (network front-end for short) is responsible for receiving high-speed UDP protocol data, respectively forwarding the high-speed UDP protocol data to a user and a data distribution server according to a configurable forwarding strategy, and supporting functions of UDF and a user direction PDXP protocol, data depth detection, calling, generating and forwarding of a status report and the like.
With the development of subsequent services, the existing data transmission equipment cannot meet the requirement of data forwarding performance of 2 x 640Mbps of tasks such as a later-stage space station and the like, and the existing network front-end machine only supports UDP multicast and cannot realize functions such as UDP unicast, TCP protocol, and delimited transcoding processing, so that the requirement of constructing a high-speed data processing platform based on an internal core state of a domestic operating system is provided, the application layer data forwarding performance with the highest processing rate not lower than 2 x 640Mbps when the minimum packet length is 801 bytes is realized, and the functions of medium-low speed forward backward distribution and backward high-speed methods are integrated to further reduce the number of equipment sets.
Disclosure of Invention
the technical problem solved by the invention is as follows: the data receiving and forwarding integrated processing platform overcomes the performance defects of the existing data receiving and forwarding equipment, and provides a high-speed data receiving, storing, code swapping and forwarding integrated processing platform based on the Linux kernel technology.
The technical solution of the invention is as follows: the high-speed data processing platform based on the domestic operating system kernel state comprises a high-speed data receiving module, a code conversion processing module, a data storage module and a data forwarding module, wherein:
the high-speed data receiving module is used for capturing a network data packet by an NF _ INET _ LOCAL _ IN mounting point IN an IP protocol stack by adopting an embedded protocol stack mode and providing a HOOK HOOK function as a callback function through a netfilter, storing the captured network data packet into a data receiving Buffer area Buffer-A, and then notifying a data storage module and a code conversion processing module through tasklet soft interruption;
The data storage module is used for reading the network data packet in the Buffer-C of the data receiving Buffer area, informing the user mode data storage module in a mmap mode, and realizing the storage of an original data file in a remote shared file server by adopting an FTP (file transfer protocol);
The code conversion processing module is used for reading the network data packet in the data receiving Buffer area Buffer-B, then carrying out code conversion processing according to a code conversion rule, and storing the data after code conversion processing in a data processing Buffer-D; classifying the Data after the code conversion processing according to the Link ID and temporarily storing the Data into a Link-Data buffer area; performing code conversion processing on the Data in the Link-Data according to an encoding rule, storing the processed Data in a Buffer-E, and then notifying a Data forwarding module to forward through a tasklet soft interrupt;
and the data forwarding module is used for reading the data in the Buffer-E of the sending Buffer area and then forwarding the data to different users according to the forwarding rule.
the high-speed data receiving module and the data forwarding module are realized by adopting a Linux2.6 kernel module programming mode, and comprise the functions of parallel multitask data receiving, multi-protocol support, legality judgment, code conversion processing, forwarding according to rules, data storage and state statistics.
The parallel multitask data receiving comprises receiving backward user data sent by a ground station, performing validity check, then distributing the backward user data to a corresponding user center in real time according to a distribution rule, and sending forward data sent by the user center to a ground terminal station data transmission terminal in real time after performing validity check.
The high-speed data receiving module and the data forwarding module support transmission layer protocols comprising a TCP protocol, a UDP unicast protocol and a UDP multicast protocol, the system application layer protocols comprise a UDP protocol and a PDXP protocol and support conversion between the UDF protocol and the PDXP protocol, and when the application layer protocol is the PDXP protocol and the transmission layer protocol is the TCP protocol, the data needs to be subjected to code conversion by the code conversion processing module.
The data receiving and forwarding software has a function of storing received forward and backward data transmission related data, the data storage is realized on the basis of a file server, a storage directory is configurable, and the storage format is a binary file;
And the data receiving and forwarding software carries out state statistics on each ground station and each user center according to the task and link statistical mode.
The high-speed data receiving module and the data forwarding module exchange data in a shared memory mode and report task statistical information.
the system also comprises a task management module which realizes multitask configuration management, dispatching management of distribution equipment, statistical information reporting, system timing, double-machine management, log reporting, multitask chain monitoring information processing, multitask state report transmission and information exchange with a user center.
when the task management module starts a task, the task management module reads a task configuration file and informs the high-speed data receiving module and the data forwarding module to start a data forwarding task, and the high-speed data receiving module and the data forwarding module respectively create a corresponding TCP/UDP unicast/UDP multicast communication socket according to configuration information to complete data receiving and sending.
The system also comprises a monitoring management module which is used for completing the real-time statistics of the number of forward return data receiving frames, the number of forwarding frames and the number of error frames, the abnormal statistics of packet loss, error codes and disorder rate, the hot standby switching control of the double computers and the log audit.
When a data source corresponding to the network data packet sends data, 1 byte unsigned number is respectively added at the head and the tail of a data frame to be used as a data domain head HD and a data domain tail TA, the data domain head and the data domain tail are both filled with a 16-system number 7EH, and all data between the data domain head and the data domain tail are transformed when the data is sent, so that all valid data with the value of 7EH are replaced by 7D5EH, and all valid data with the value of 7DH are replaced by 7D5 DH; and when the data is received, restoring, searching a data field head and a data field tail delimiter 7EH, and converting all data between the data field head and the data field tail to ensure that all 7D5EH are replaced by the 7EH and all 7D5DH are replaced by the 7DH, thereby finishing the code conversion processing.
the data storage module realizes the storage of the original data file in the remote shared file server by adopting an FTP transmission protocol to realize the remote file transmission with a Sun file server, the Sun file server finishes the falling storage of the data, and then the QFS provides file access service for other servers.
compared with the prior art, the invention has the advantages that:
(1) The invention overcomes the defect that the network front-end machine in the existing satellite data forwarding system only supports high-speed backward data service and UDP multicast protocol, can simultaneously support high-speed backward and medium-low speed forward backward data service, and can support three communication protocols of TCP, UDP unicast and UDP multicast among the ground station data transmission terminal, the network front-end machine and the user center;
(2) The invention overcomes the defect that the network front-end machine in the prior art can only realize the data forwarding performance of the return 2X 340Mbps without the transcoding processing, and can realize the data forwarding performance index of the high-speed return 2X 680Mbps + the medium-low speed forward return 20Mbps under the condition of supporting the transcoding processing;
(3) Compared with the prior art, the high-speed data processing platform based on the kernel mode of the domestic operating system can realize the real-time storage of the received high-speed data, the storage space supports the dynamic online capacity expansion, and the cross-platform sharing of data files can be met.
drawings
FIG. 1 is a flow chart of the parallel data processing of the multi-stage pipeline of the high-speed data processing platform based on the kernel mode of the domestic operating system.
Detailed Description
Aiming at the defects of the prior art, the invention provides a high-speed data processing platform based on the kernel state of a domestic kylin Linux operating system, which comprises a high-speed data receiving module, a code conversion processing module, a data storage module and a data forwarding module, wherein the high-speed data receiving module, the code conversion processing module, the data storage module and the data forwarding module are arranged in the high-speed data processing platform
The high-speed data receiving module adopts an embedded protocol stack mode, provides a specific callback function (also called a HOOK HOOK function) for a kernel program through a netfilter, and realizes the interception processing of a network data packet on an NF _ INET _ LOCAL _ IN mounting point IN an IP protocol stack; as shown in fig. 1, the captured data is stored in a data receiving Buffer-a, and then the data storage module and the escape processing module are notified to perform subsequent processing through a tasklet soft interrupt;
After the data storage module reads the data in the Buffer-C of the data receiving Buffer area, the user mode data storage module is informed in a mmap mode, and the storage of an original data file in a remote shared file server is realized by adopting an FTP (file transfer protocol);
After reading the data in the data receiving Buffer area Buffer-B, the code conversion processing module performs code conversion processing on the data according to a code conversion rule, and the data after code conversion processing is firstly stored in a data processing Buffer area Buffer-D; then classifying the Data according to the Link ID and temporarily storing the Data into a Link-Data buffer area; then, carrying out code conversion processing on the data to be sent according to a coding rule, and storing the data subjected to code conversion processing into a Buffer area Buffer-E; finally, informing a data forwarding module to forward through tasklet soft interruption;
after reading the data in the Buffer-E, the data forwarding module forwards the data to different users according to the forwarding rules, and the method of the present invention is explained and explained in detail below with reference to the accompanying drawings.
the invention provides a high-speed data processing platform based on a domestic operating system kernel state, which is used for realizing high-speed data receiving, storing, code swapping and forwarding processing in the field of satellite data forwarding.
(1) High-speed data processing platform overall design based on domestic operating system kernel mode
the platform of the invention adopts a C/S architecture and consists of three parts, namely data receiving and forwarding software, task management software and monitoring management software, wherein the C/S architecture comprises the following steps:
The data receiving and forwarding software is realized by adopting a Linux2.6 kernel module programming mode, and the main realization functions are as follows: the method comprises the following functions of receiving high-speed (the speed is more than 150Mbps) return data/medium-low speed (the speed is lower than 20Mbps) forward return data of parallel multi-tasks, supporting multi-protocol, judging legality, processing escape, forwarding according to rules, storing data, counting states and the like;
The task management software is realized by adopting a Linux2.6 kernel module programming mode, and the main realization functions are as follows: the system has the functions of multitask configuration management, dispatching management of distribution equipment, reporting of statistical information, system timing, double-machine management, log reporting, multitask chain monitoring information processing, multitask state report transmission, information exchange with a user center and the like;
the monitoring management software is divided into two installation versions of an kylin and a Windows operating system, is developed and realized by adopting a Qt language, mainly realizes the multitask remote management and monitoring functions of a data processing platform, and can be operated in any kylin or Windows computer in a management subnet.
(2) data receiving and forwarding software
The data receiving and forwarding software is the most core component of the whole platform, receives return user data sent by the ground station in real time, and distributes the return user data to the corresponding user center in real time according to a distribution rule after carrying out validity check; meanwhile, forward data transmission data sent by a user center is received in real time, and is sent to a ground terminal station data transmission terminal in real time after validity check is carried out;
The transmission layer protocol supported by the data receiving and forwarding software comprises the following steps: a TCP protocol, a UDP unicast protocol, a UDP multicast protocol (including any source multicast and designated source multicast); the system application layer supports a UDP protocol and a PDXP protocol and supports conversion between the UDF protocol and the PDXP protocol;
the data receiving and forwarding software has a data escape processing function, and when the application layer protocol is a PDXP protocol and the transmission layer protocol is a TCP protocol, the data content needs to be subjected to escape processing;
The data receiving and forwarding software has a function of storing received forward and backward data transmission related data, the data storage is realized on the basis of a file server, a storage directory is configurable, and the storage format is a binary file;
And the data receiving and forwarding software carries out state statistics on each ground station and each user center according to the task and link statistical mode.
(3) task management software
The task management software is used for completing configuration management of data transmission tasks, statistical information reporting, multitask chain monitoring information processing, multitask state report transmission, system time correction, dual-computer hot standby management and information exchange with a user center.
The method comprises the steps that a high-speed data processing platform based on a domestic operating system kernel mode manages a data link according to tasks and can support local and remote control of the tasks; the task management software and the data receiving and forwarding software exchange data in a shared memory mode, so that the task statistical information is reported;
In order to avoid that a switch automatically releases a multicast link due to the fact that data are not sent for a long time when UDP multicast communication is carried out, task management software sends link monitoring information according to a 1 second/packet cycle when no data are sent; meanwhile, in order to facilitate users to know the transmission state of the task in time, the task management software supports the transmission function of the overall multitask report, and automatically sends forward/return transmission state report information to the task management software after receiving a forward/return transmission state report calling command;
in order to avoid time deviation in the working process of the data transmission platform, the task management software realizes time synchronization through the timing module;
In order to ensure the working reliability of the data transmission platform, the task management software realizes the control of the main machine and the standby machine through the dual-machine hot standby management module, and the main machine works and the standby machine is in a standby mode under normal conditions.
(4) monitoring management software
The monitoring management software is graphical management software of a high-speed data processing platform based on a domestic operating system kernel state, and mainly completes configuration management of relevant parameters of a user data relay task, real-time statistics of forward and backward data receiving frame number, forwarding frame number, error frame number and the like, abnormal statistics of packet loss, error codes and disorder rate, dual-computer hot standby switching control and log audit functions.
(5) System performance index
a) the actual test return high-speed data distribution performance reaches 2 × 725 Mbps;
b) After the storage function is added, the processing speed of the whole machine reaches 2 × 680Mbps +20 Mbps. A
The realization process of the high-speed network IO kernel multi-protocol support comprises the following steps:
the high-speed data processing platform based on the kernel state of the domestic operating system can support high-speed backward data services and medium-low speed forward backward data services, three communication protocols of TCP, UDP unicast and UDP multicast are simultaneously supported among the ground station data transmission terminal, the data receiving and forwarding software of the data processing platform and the user center, and the specific mode is set through a configuration file so as to meet the requirements of later data forwarding services. When the task management software starts a task, reading a task configuration file and informing the data receiving and forwarding software to start a data forwarding task, wherein a high-speed data receiving module and a data forwarding module of the data receiving and forwarding software respectively create a corresponding TCP/UDP unicast/UDP multicast communication socket according to configuration information, and data receiving and sending processing is completed.
The TCP protocol guarantees reliable transmission of data mainly by the following mechanisms:
in terms of data correctness and legality, TCP uses a checksum function to check whether data has errors, and the checksum is calculated during sending and receiving;
In the aspect of ensuring reliability, an overtime retransmission and piggybacking confirmation mechanism is adopted;
In flow control, a sliding window protocol is used, which specifies that for unacknowledged packets within a window, retransmissions are required.
The UDP protocol is divided into a unicast mode and a multicast mode, wherein the unicast mode refers to that a data source and a data target are both unicast addresses (a host IP), and point-to-point data transmission is realized similar to a TCP protocol; multicast is a host IP as a data source, where a group of hosts receives data at a logical group address, and point-to-multipoint network connection is implemented between a sender and a receiver, and is applied more in a scenario where the master and the slave receive data simultaneously.
According to the requirement of service development, when the PDXP data frame is forwarded by a subsequent task through a TCP protocol, the bound transcoding processing of a data packet is required. The index is not realized in the high-speed distribution of 2 × 150Mbit/s at present, and the main reason is that the data packet sticking phenomenon exists when the TCP protocol is adopted for data transmission. In order to solve or avoid the problems of TCP package sticking and unpacking, the escape processing module of the data receiving and forwarding software adopts a 'data forwarding delimiting escape' technology.
the realization process of improving the transcoding performance of data forwarding delimitation is as follows:
When the data source of the invention transmits data, 1 byte unsigned number is respectively added at the head and the tail of a data frame, which are called as a data domain Head (HD) and a data domain Tail (TA), the data domain head and the data domain tail are both filled with a 16-system number 7EH, all data between the data domain head and the data domain tail are required to be transformed during transmission, all effective data with the value of 7EH are replaced by 7D5EH, and all effective data with the value of 7DH are replaced by 7D5DH, which is the encoding of the data. When the data transmission device receives the data, the data transmission device carries out reduction processing on the data, firstly searches for a data field head and a data field tail delimiter of '7 EH', and then converts all data between the data field head and the data field tail, namely all '7D 5 EH' are replaced by '7 EH', all '7D 5 DH' are replaced by '7 DH', and the decoding is carried out on the data. After the transcoding processing is adopted, even if the phenomena of packet sticking and unpacking occur in a TCP transmission layer, a complete data can be analyzed out by accurately utilizing the characteristic marks of the frame head and the frame tail in an application layer, so that the problem of data disorder during multi-user data forwarding is solved.
because the consumption of the CPU in the process of the escape processing is very large, the current program implementation mode is difficult to reach the data transmission index after the escape processing is finished. Through analysis, the main reason for improving the escape performance is that the processing links are all executed in series, and the platform is supposed to realize 2 × 640Mbit/s escape processing in a multi-stage pipeline parallel processing mode.
After a high-speed data processing platform based on a domestic operating system kernel state receives a return network data packet sent by a ground station data transmission terminal, all data needs to be stored in a specified data storage directory so as to facilitate subsequent analysis and processing; in order to ensure that other servers can implement business operations such as post-data analysis, archiving, file class retransmission and pushing and the like based on received return data, a shared file system is adopted by a data storage module of data receiving and forwarding software (provided with a Linux operating system), and all data is stored through a file storage server (provided with a Solaris operating system).
the Linux & Solaris cross-platform data file sharing is realized by the following steps:
the high-speed data processing platform adopts a domestic kylin Linux operating system to store data, adopts an FTP (file transfer protocol) to realize remote file transmission with a Sun file server, finishes the falling storage of the data by the Sun file server, and provides file access service for other servers through a QFS (quad Flat server) file system.
the invention is not described in detail and is within the knowledge of a person skilled in the art.
Claims (10)
1. The high-speed data processing platform based on the domestic operating system kernel state is characterized by comprising a high-speed data receiving module, an escape processing module, a data storage module and a data forwarding module, wherein:
The high-speed data receiving module captures a network data packet by using an NF _ INET _ LOCAL _ IN mounting point IN an IP protocol stack by adopting an embedded protocol stack mode through a HOOK HOOK function provided by a netfilter as a callback function, stores the captured network data packet into a data receiving Buffer area Buffer-A, and then informs the data storage module and the code conversion processing module through tasklet soft interruption;
The data storage module is used for reading the network data packet in the Buffer-C of the data receiving Buffer area, informing the user mode data storage module in a mmap mode, and realizing the storage of an original data file in a remote shared file server by adopting an FTP (file transfer protocol);
The code conversion processing module is used for reading the network data packet in the data receiving Buffer area Buffer-B, then carrying out code conversion processing according to a code conversion rule, and storing the data after code conversion processing in a data processing Buffer-D; classifying the Data after the code conversion processing according to the Link ID and temporarily storing the Data into a Link-Data buffer area; performing code conversion processing on the Data in the Link-Data according to an encoding rule, storing the processed Data in a Buffer-E, and then notifying a Data forwarding module to forward through a tasklet soft interrupt;
and the data forwarding module is used for reading the data in the Buffer-E of the sending Buffer area and then forwarding the data to different users according to the forwarding rule.
2. The home operating system kernel mode-based high-speed data processing platform according to claim 1, wherein: the high-speed data receiving module and the data forwarding module are realized by adopting a Linux2.6 kernel module programming mode and comprise the functions of parallel multitask data receiving, multi-protocol support, legality judgment, code conversion processing, forwarding according to rules, data storage and state statistics.
3. The home operating system kernel mode-based high-speed data processing platform according to claim 2, wherein: the parallel multitask data receiving comprises receiving backward user data sent by a ground station, performing validity check, then distributing the backward user data to a corresponding user center in real time according to a distribution rule, and sending forward data sent by the user center to a ground terminal station data transmission terminal in real time after performing validity check.
4. The home operating system kernel mode-based high-speed data processing platform according to claim 3, wherein: the high-speed data receiving module and the data forwarding module support transmission layer protocols comprising a TCP protocol, a UDP unicast protocol and a UDP multicast protocol, the system application layer protocols comprise a UDP protocol and a PDXP protocol and support conversion between the UDF protocol and the PDXP protocol, and when the application layer protocol is the PDXP protocol and the transmission layer protocol is the TCP protocol, the data needs to be subjected to code conversion by the code conversion processing module.
The data receiving and forwarding software has a function of storing received forward and backward data transmission related data, the data storage is realized on the basis of a file server, a storage directory is configurable, and the storage format is a binary file;
and the data receiving and forwarding software carries out state statistics on each ground station and each user center according to the task and link statistical mode.
5. The home operating system kernel mode-based high-speed data processing platform according to claim 4, wherein: the high-speed data receiving module and the data forwarding module exchange data in a shared memory mode and report task statistical information.
6. The home operating system kernel mode-based high-speed data processing platform according to claim 5, wherein: the system also comprises a task management module which realizes multitask configuration management, dispatching management of distribution equipment, statistical information reporting, system timing, double-machine management, log reporting, multitask chain monitoring information processing, multitask state report transmission and information exchange with a user center.
7. The home operating system kernel mode-based high-speed data processing platform according to claim 6, wherein: when the task management module starts a task, the task management module reads a task configuration file and informs the high-speed data receiving module and the data forwarding module to start a data forwarding task, and the high-speed data receiving module and the data forwarding module respectively create a corresponding TCP/UDP unicast/UDP multicast communication socket according to configuration information to complete data receiving and sending.
8. The home operating system kernel mode-based high-speed data processing platform according to claim 7, wherein: the system also comprises a monitoring management module which is used for completing the real-time statistics of the number of forward return data receiving frames, the number of forwarding frames and the number of error frames, the abnormal statistics of packet loss, error codes and disorder rate, the hot standby switching control of the double computers and the log audit.
9. The home operating system kernel mode-based high-speed data processing platform according to claim 8, wherein: when a data source corresponding to the network data packet sends data, 1 byte unsigned number is respectively added at the head and the tail of a data frame to be used as a data domain head HD and a data domain tail TA, the data domain head and the data domain tail are both filled with a 16-system number 7EH, and all data between the data domain head and the data domain tail are transformed when the data is sent, so that all valid data with the value of 7EH are replaced by 7D5EH, and all valid data with the value of 7DH are replaced by 7D5 DH; and when the data is received, restoring, searching a data field head and a data field tail delimiter 7EH, and converting all data between the data field head and the data field tail to ensure that all 7D5EH are replaced by the 7EH and all 7D5DH are replaced by the 7DH, thereby finishing the code conversion processing.
10. The home operating system kernel mode-based high-speed data processing platform according to claim 9, wherein: the data storage module realizes the storage of the original data file in the remote shared file server by adopting an FTP transmission protocol to realize the remote file transmission with a Sun file server, the Sun file server finishes the falling storage of the data, and then the QFS provides file access service for other servers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910678731.4A CN110557369A (en) | 2019-07-25 | 2019-07-25 | high-speed data processing platform based on domestic operating system kernel mode |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910678731.4A CN110557369A (en) | 2019-07-25 | 2019-07-25 | high-speed data processing platform based on domestic operating system kernel mode |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110557369A true CN110557369A (en) | 2019-12-10 |
Family
ID=68735776
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910678731.4A Pending CN110557369A (en) | 2019-07-25 | 2019-07-25 | high-speed data processing platform based on domestic operating system kernel mode |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110557369A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112147924A (en) * | 2020-08-27 | 2020-12-29 | 航天东方红卫星有限公司 | High-precision program-controlled task management system |
| CN114938237A (en) * | 2022-05-25 | 2022-08-23 | 中国电子科技集团公司第十研究所 | Method and system for receiving and transmitting high-speed data transmission data of satellite |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101340569A (en) * | 2007-07-06 | 2009-01-07 | 扬智科技股份有限公司 | High-speed cache and data processing method thereof |
| CN102932261A (en) * | 2012-11-20 | 2013-02-13 | 航天东方红卫星有限公司 | Management system for payload information of satellite based on intelligent high-speed router |
| CN104965700A (en) * | 2015-06-09 | 2015-10-07 | 航天科工深圳（集团）有限公司 | Method and system for implementing driving on SPI equipment under VxWorks operating system |
| US20160077872A1 (en) * | 2013-05-09 | 2016-03-17 | Huawei Technologies Co., Ltd. | Data processing apparatus and data processing method |
| CN106502934A (en) * | 2016-11-09 | 2017-03-15 | 上海微小卫星工程中心 | The spaceborne data management system of high-speed integratedization |
| CN106850565A (en) * | 2016-12-29 | 2017-06-13 | 河北远东通信系统工程有限公司 | A kind of network data transmission method of high speed |
-
2019
- 2019-07-25 CN CN201910678731.4A patent/CN110557369A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101340569A (en) * | 2007-07-06 | 2009-01-07 | 扬智科技股份有限公司 | High-speed cache and data processing method thereof |
| CN102932261A (en) * | 2012-11-20 | 2013-02-13 | 航天东方红卫星有限公司 | Management system for payload information of satellite based on intelligent high-speed router |
| US20160077872A1 (en) * | 2013-05-09 | 2016-03-17 | Huawei Technologies Co., Ltd. | Data processing apparatus and data processing method |
| CN104965700A (en) * | 2015-06-09 | 2015-10-07 | 航天科工深圳（集团）有限公司 | Method and system for implementing driving on SPI equipment under VxWorks operating system |
| CN106502934A (en) * | 2016-11-09 | 2017-03-15 | 上海微小卫星工程中心 | The spaceborne data management system of high-speed integratedization |
| CN106850565A (en) * | 2016-12-29 | 2017-06-13 | 河北远东通信系统工程有限公司 | A kind of network data transmission method of high speed |
Non-Patent Citations (2)
| Title |
|---|
| 李望；杨昊苏；吴宏；徐会忠: "中继卫星地面高速数据分发设备设计与性能分析", 《电讯技术》 * |
| 杨德伟;王华: "基于FPGA实现的以太网和E1的协议转换", 《2005年海峡两岸三地无线科技学术会论文集》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112147924A (en) * | 2020-08-27 | 2020-12-29 | 航天东方红卫星有限公司 | High-precision program-controlled task management system |
| CN114938237A (en) * | 2022-05-25 | 2022-08-23 | 中国电子科技集团公司第十研究所 | Method and system for receiving and transmitting high-speed data transmission data of satellite |
| CN114938237B (en) * | 2022-05-25 | 2023-06-16 | 中国电子科技集团公司第十研究所 | Method and system for receiving and transmitting satellite high-speed data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11522734B2 (en) | Method for controlling a remote service access path and relevant device | |
| CN107959715B (en) | Remote terminal information identification software method based on wireless telecommunications | |
| CN103916405B (en) | Method for guiding flow of TCP/UDP of App on IOS | |
| CN102299899B (en) | Method for accelerating TCP (Transmission Control Protocol) under severe channel | |
| US11303737B2 (en) | Method and device for data transmission | |
| CN102394925B (en) | Communication method and device for remote monitoring diagnosis center and area regulation center | |
| US8682288B2 (en) | Call list transmission processing method and system of charging system | |
| CN106330414A (en) | Message transmission method and device | |
| WO2021056793A1 (en) | Power transmission line distributed fault diagnosis system and method | |
| CN111711680A (en) | File breakpoint continuous transmission method and device based on UDP (user Datagram protocol) | |
| CN110557369A (en) | high-speed data processing platform based on domestic operating system kernel mode | |
| CN102769520A (en) | Wireless network congestion control method based on stream control transmission protocol (SCTP) | |
| CN105933453A (en) | Data transmission method and system | |
| US20160204904A1 (en) | Service Message Transmitting Method and Device | |
| CN100405783C (en) | Ethernet group broadcasting method | |
| CN107995315B (en) | Method and device for synchronizing information between service boards, storage medium and computer equipment | |
| WO2020259277A1 (en) | Parameter optimization method and apparatus, base station, server and storage medium | |
| CN201312363Y (en) | Wireless remote monitoring system for safety production emergency command | |
| WO2013075513A1 (en) | Short message buffering method and system | |
| CN101848165B (en) | The method recovered after controlling interrupted communication link and interface board | |
| CN103685178A (en) | Industrial wireless network oriented online debugging method and system | |
| CN100576185C (en) | Signal conditioning package and information processing method | |
| CN1625292A (en) | Method for transforing base station environment monitoring information | |
| CN114727324A (en) | Network disaster tolerance processing method and device, storage medium and electronic equipment | |
| CN100341345C (en) | Short-message central multi-mode data dispatch processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191210 |
|
| RJ01 | Rejection of invention patent application after publication |