CN116582889A - High-throughput data transmission and storage method, electronic equipment and storage medium - Google Patents
High-throughput data transmission and storage method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116582889A CN116582889A CN202310861415.7A CN202310861415A CN116582889A CN 116582889 A CN116582889 A CN 116582889A CN 202310861415 A CN202310861415 A CN 202310861415A CN 116582889 A CN116582889 A CN 116582889A
- Authority
- CN
- China
- Prior art keywords
- satellite
- data
- compressed data
- compressed
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 74
- 230000005540 biological transmission Effects 0.000 title claims abstract description 50
- 238000007906 compression Methods 0.000 claims description 49
- 230000006835 compression Effects 0.000 claims description 49
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 12
- 238000013500 data storage Methods 0.000 abstract description 6
- 238000004422 calculation algorithm Methods 0.000 description 25
- 238000005516 engineering process Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000000903 blocking effect Effects 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/06—Optimizing the usage of the radio link, e.g. header compression, information sizing, discarding information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0015—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the adaptation strategy
- H04L1/0016—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the adaptation strategy involving special memory structures, e.g. look-up tables
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0023—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the signalling
- H04L1/0028—Formatting
- H04L1/0029—Reduction of the amount of signalling, e.g. retention of useful signalling or differential signalling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
- H04L1/0056—Systems characterized by the type of code used
- H04L1/0057—Block codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
- H04W84/04—Large scale networks; Deep hierarchical networks
- H04W84/06—Airborne or Satellite Networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention relates to the technical field of data storage and application, in particular to a high-throughput data transmission and storage method, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring satellite data, and determining whether the satellite data is a preset basic value, wherein the satellite data is stored in a serialization manner; under the condition that satellite data is not a preset basic value, carrying out differential coding on the satellite data to obtain first satellite compressed data; performing first canonical Huffman encoding on the first satellite compressed data to obtain second satellite compressed data; the second satellite compressed data is stored and transmitted to the first device. The method can improve the data processing capacity and the transmission efficiency of the satellite data on the premise of ensuring that the satellite data is not lost.
Description
Technical Field
The present invention relates to the field of data storage and application technologies, and in particular, to a high throughput data transmission and storage method, an electronic device, and a storage medium.
Background
In the related art, the transmission efficiency of satellite data can be improved by using a lossless compression algorithm or a lossy compression algorithm. However, the complexity of the existing lossless compression algorithm is high, the data processing capability of the embedded platform of the small-sized and low-cost satellite is weak, and when the embedded platform of the small-sized and low-cost satellite utilizes the lossless compression algorithm to transmit satellite data, the compression of the satellite data cannot be realized at a higher speed, so that the requirement of high-speed data transmission cannot be met even if the data compression algorithm is used; although the lossy compression algorithm has a higher compression speed, partial satellite data can be lost during compression, and the lost satellite data cannot be recovered, so that the lossy compression algorithm cannot be applied to industries with high accuracy requirements.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a high-throughput data transmission and storage method, electronic equipment and a storage medium, which can improve the data processing capacity and transmission efficiency of satellite data on the premise of ensuring that the satellite data is not lost.
In a first aspect, an embodiment of the present invention provides a high throughput data transmission and storage method, applied to a satellite, where the high throughput data transmission and storage method includes:
acquiring satellite data, and determining whether the satellite data is a preset basic value, wherein the satellite data is stored in a serialization manner;
carrying out differential coding on the satellite data under the condition that the satellite data is not a preset basic value, so as to obtain first satellite compressed data;
performing first canonical Huffman encoding on the first satellite compressed data to obtain second satellite compressed data;
and storing and transmitting the second satellite compressed data to the first device.
According to some embodiments of the first aspect of the present invention, the performing a first canonical huffman encoding on the first satellite compressed data to obtain second satellite compressed data includes:
performing compression coding on high-frequency characters in the first satellite compressed data to obtain initial coded data;
and simplifying the initial coding data to obtain second satellite compression data.
According to some embodiments of the first aspect of the present invention, the second satellite compressed data includes encoded data and a length table, and after the first canonical huffman encoding is performed on the first satellite compressed data, the second satellite compressed data is obtained, the second satellite compressed data has:
determining whether the memory device of the satellite supports executing the second canonical Huffman coding according to a preset memory judgment criterion;
and under the condition that the storage equipment of the satellite supports to execute the second standard Huffman coding, carrying out the second standard Huffman coding on the length table to obtain third satellite compressed data.
According to some embodiments of the first aspect of the present invention, after performing the second canonical huffman encoding on the length table to obtain the third satellite compressed data, the method comprises:
and storing and transmitting the third satellite compressed data to the first device.
According to some embodiments of the first aspect of the invention, the method further has:
under the condition that the satellite data is a preset basic value, carrying out first canonical Huffman coding on the satellite data to obtain fourth satellite compressed data;
and storing and transmitting the fourth satellite compressed data to the first device.
According to some embodiments of the first aspect of the invention, the field in the first satellite compressed data is used to indicate at least one of:
whether the satellite data is a preset basic value or not;
differentially encoded sample times;
a difference sign;
the difference length.
According to some embodiments of the first aspect of the present invention, the differentially encoding the satellite data to obtain first satellite compressed data includes:
performing difference operation on the satellite data and each adjacent data of the satellite data to obtain a difference data set;
and determining the first satellite compressed data according to the preset basic value and the difference data set.
According to some embodiments of the first aspect of the present invention, the storing and transmitting the second satellite compressed data to the first device has:
performing signal conversion on the second satellite compressed data to obtain a target transmission signal;
transmitting the target transmission signal to a first device.
In a second aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing when executing the computer program: the method as described in the first aspect above.
In a third aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for performing the method according to the first aspect.
The method has the beneficial effects that under the condition that satellite data are acquired and the satellite data are not determined to be the preset basic value, differential encoding and first standard Huffman encoding are sequentially carried out on the satellite data, second satellite compressed data are obtained, and then the second satellite compressed data are stored and transmitted to the first equipment. The lossless compression algorithm for carrying out differential encoding and first-specification Huffman encoding on satellite data sequentially has small algorithm complexity, can have better execution effect on an embedded platform of a small-sized and low-cost satellite, and improves the data processing capacity and transmission efficiency of the satellite data on the premise of ensuring that the satellite data is not lost.
Drawings
Fig. 1 is a flow chart of a high throughput data transmission and storage method according to an embodiment of the first aspect of the present invention;
FIG. 2 is a flow chart of another method for high throughput data transmission and storage according to an embodiment of the first aspect of the present invention;
FIG. 3 is a flow chart of another method for high throughput data transmission and storage according to an embodiment of the first aspect of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the second aspect of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following describes in detail a high throughput data transmission and storage method, an electronic device and a storage medium provided by the embodiments of the present invention through specific embodiments and application scenarios thereof with reference to the accompanying drawings.
Example 1:
referring to fig. 1, fig. 1 illustrates a high throughput data transmission and storage method according to an embodiment of a first aspect of the present invention, where the method is applied to a satellite and executed by the satellite. In other words, the method may be performed by software or hardware installed in a satellite, the method comprising the steps of:
in step S110, satellite data is acquired, and it is determined whether the satellite data is a preset base value.
Wherein the satellite data is stored in a serialization. And serializing the stored satellite data, and judging whether the current satellite data is a preset base value, wherein the preset base value is the first satellite data in a preset interval time. The satellite is an important resource, and can provide key data and services for human beings, such as weather prediction, resource exploration, navigation and the like; the satellite data is the satellite signal data after channel coding.
In step S120, differential encoding is performed on the satellite data to obtain first satellite compressed data if the satellite data is not the preset base value.
In this step, the satellite data is divided into a preset base value and a non-preset base value, and the non-preset base value is differentially encoded, so that the satellite data is represented by the non-preset base value and the difference value, and further, the fact that more data information is represented by fewer bytes is realized, the storage space is saved, and more satellite data is stored in the limited storage space.
Step S130, performing first canonical Huffman coding on the first satellite compressed data to obtain second satellite compressed data;
in this step, the differentially encoded first satellite compressed data is further compressed by the first canonical huffman encoding, and the characters with higher frequencies in the first satellite compressed data are represented by shorter bit lengths, thereby reducing the size of the first satellite compressed data; however, since mapping characters to the code table will take up a large memory space, especially for small-sized satellite data, only the length of the code needs to be recorded during the encoding process, and an exemplary 8-bit character has 256 characters in total.
Step S140, the second satellite compressed data is stored and transmitted to the first device.
In this step, the second satellite compressed data, which is the compressed data subjected to differential encoding and first canonical huffman encoding, is stored in the storage device of the satellite, and a signal corresponding to the second satellite compressed data to be transmitted is read in the storage device and transmitted to the first device.
It should be noted that the first device may be a ground station, a co-satellite, or other devices, and embodiments of the present invention are not limited herein.
In the related art, one of multiplexing, blocking and compression techniques is used for high throughput data transmission, where the multiplexing technique is to combine multiple data streams into one stream for transmission, so as to improve the data transmission efficiency; the blocking technology is to divide a large data block into small blocks for transmission, so that the data transmission efficiency is improved; the compression technique is to reduce the transmission amount of data by compressing the data, thereby improving the data transmission efficiency. However, a small satellite is used as a small device, and its resources are limited, so that if different data streams are transmitted by using multiplexing technology at the same time, the calculation and communication resources of the satellite may be limited, and the operation stability of the satellite may be affected; the blocking technology needs to carry out blocking processing on the data, so that occupation of storage space of the small satellite is increased, and sufficient storage space planning and resource management are needed; when a lossless compression algorithm or a lossy compression algorithm in a compression technology is used for transmitting data of a small-sized and low-cost satellite, the data processing capacity of an embedded platform of the small-sized and low-cost satellite is weak due to the high complexity of the conventional lossless compression algorithm, and when the embedded platform of the small-sized and low-cost satellite transmits satellite data by using the lossless compression algorithm, the compression of the satellite data cannot be realized at a high speed; although the lossy compression algorithm has a relatively high compression speed, part of satellite data is lost during compression, and the lost satellite data cannot be recovered, so that the lossy compression algorithm cannot be applied to industries with high accuracy requirements, for example, in some special fields such as: the satellite data information loss can cause great influence in industries such as weather, exploration and the like.
Therefore, the high throughput data transmission and storage method provided by the embodiment of the invention determines whether the satellite data is a preset basic value or not by acquiring the satellite data, wherein the satellite data is stored in a serialization manner; under the condition that satellite data is not a preset basic value, carrying out differential coding on the satellite data to obtain first satellite compressed data; performing first canonical Huffman encoding on the first satellite compressed data to obtain second satellite compressed data; the second satellite compressed data is stored and transmitted to the first device. By the method, the lossless compression algorithm of differential encoding and first-specification Huffman encoding is sequentially carried out on satellite data, the algorithm has small complexity, and the method can achieve a better execution effect on an embedded platform of a small-sized and low-cost satellite, and improves the data processing capacity and transmission efficiency of the satellite data on the premise of ensuring that the satellite data is not lost.
Example 2:
referring to fig. 2, fig. 2 illustrates another high throughput data transmission and storage method provided by an embodiment of the first aspect of the present invention, where the method is applied to a satellite and executed by the satellite. In other words, the method may be performed by software or hardware installed in a satellite, the method comprising the steps of:
in step S210, satellite data is acquired, and it is determined whether the satellite data is a preset base value.
This step may be described in step S110 in the embodiment of fig. 1, and will not be described herein.
In step S220, differential encoding is performed on the satellite data to obtain first satellite compressed data if the satellite data is not the preset base value.
This step may be described in step S120 in the embodiment of fig. 1, and will not be described herein.
Step S230, compression encoding is performed on the high-frequency characters in the first satellite compressed data to obtain initial encoded data.
In this step, the character having the higher frequency in the first satellite compressed data will be represented by a shorter bit length, and although the higher frequency character can reduce the size of the first satellite compressed data, it will occupy a considerable memory space for mapping the character to the encoding table, especially for small-sized data. Therefore, the initial encoded data is simplified in the encoding process, and only the length of the code, for example, 8-bit characters, has to be recorded for a total of 256 characters.
Step S240, the initial coding data is simplified to obtain second satellite compression data.
For example, for initial encoded data in which the encoded lengths are recorded sequentially, the characters in the encoded table may be discarded, and the initial encoded data may be represented as encoded data and a length table of 256 bytes.
Step S250, determining whether the memory device of the satellite supports executing the second canonical Huffman encoding according to the preset memory judgment criterion.
The second satellite compressed data includes encoded data and a length table. Although the length table occupied by the canonical huffman code is already stored relatively small, when the embedded platform memory of the satellite is sufficient, the second canonical huffman code may be performed to obtain a larger compression ratio. The preset memory judgment criteria are as follows: whether the embedded platform memory of the satellite is sufficient.
In step S260, in the case that the storage device of the satellite supports performing the second canonical huffman encoding, the length table is subjected to the second canonical huffman encoding to obtain the third satellite compressed data.
It should be noted that, when the memory of the embedded platform of the satellite is sufficient, the second canonical huffman encoding may be performed, and the length table may be encoded into the canonical huffman encoding form, so as to obtain a larger compression ratio; and when the embedded platform memory of the satellite is insufficient, directly storing and transmitting the second satellite compressed data to the first device.
In step S270, the third satellite compressed data is stored and transmitted to the first device.
In this step, the third satellite compressed data, which is the compressed data after differential encoding, first canonical huffman encoding, and second canonical huffman encoding are sequentially stored in the storage device of the satellite, and a signal corresponding to the third satellite compressed data to be transmitted is read in the storage device and transmitted to the first device.
Due to the large amount of satellite data, data transmission and storage have severe requirements for satellites. Particularly for small-sized and low-cost satellites, most of the satellites use embedded processors, the data processing capacity of the satellites is weak, meanwhile, the internal space of the devices is limited, large-scale storage devices cannot be arranged, and the storage capacity of the devices is weak.
In the related art, the technology for high-throughput data storage is one of a distributed storage architecture, a RAID technology and a compression technology. The distributed storage architecture stores data on a plurality of nodes, so that the data reliability is improved, and the concurrent reading and writing capacity of the data can be improved; the RAID technology comprises the steps of forming a disk array by using a plurality of hard disks, and respectively storing data in the plurality of hard disks, so that the reliability and fault tolerance of data storage are improved; compression techniques reduce the volume of data by compressing the data, thereby enabling the storage device to store more data information in a limited space. However, both distributed storage architecture and RAID technology require multiple storage devices, and small satellites do not have the required internal space; when a lossless compression algorithm or a lossy compression algorithm in a compression technology is used for transmitting data of a small-sized and low-cost satellite, the data processing capacity of an embedded platform of the small-sized and low-cost satellite is weak due to the high complexity of the conventional lossless compression algorithm, and when the embedded platform of the small-sized and low-cost satellite transmits satellite data by using the lossless compression algorithm, the compression of the satellite data cannot be realized at a high speed; although the lossy compression algorithm has a relatively high compression speed, part of satellite data is lost during compression, and the lost satellite data cannot be recovered, so that the lossy compression algorithm cannot be applied to industries with high accuracy requirements, for example, in some special fields such as: the satellite data information loss can cause great influence in industries such as weather, exploration and the like.
The method for providing the service provided by the embodiment of the invention determines whether the satellite data is a preset basic value or not by acquiring the satellite data; under the condition that satellite data is not a preset basic value, carrying out differential coding on the satellite data to obtain first satellite compressed data; performing compression coding on high-frequency characters in the first satellite compressed data to obtain initial coded data; simplifying the initial coded data to obtain second satellite compressed data; determining whether the memory equipment of the satellite supports executing the second canonical Huffman coding according to a preset memory judgment criterion; under the condition that the second standard Huffman coding is supported by the storage equipment of the satellite, the second standard Huffman coding is carried out on the length table, and third satellite compressed data is obtained; the third satellite compressed data is stored and transmitted to the first device. By the method, the canonical Huffman coding is selectively executed according to the memory size of the embedded platform used by the satellite so as to obtain a better compression ratio, and the method has certain expansibility; meanwhile, the lossless compression algorithm of differential encoding and first and second canonical Huffman encoding is sequentially carried out on satellite data, the algorithm has small complexity, and the method can improve the data processing capacity and the transmission efficiency of the satellite data on the premise of ensuring that the satellite data is not lost due to good execution effect of an embedded platform of a small-sized low-cost satellite. In one implementation, a field in the first satellite compressed data is used to indicate at least one of:
whether the satellite data is a preset basic value or not; for example, a field value of 0 in the first satellite compressed data indicates that the satellite data corresponding to the first satellite compressed data is not a preset base value;
differentially encoded sample times; for example, a first field of 0 in the first satellite compressed data indicates that the satellite data corresponding to the first satellite compressed data is not a preset base value, and a second field of 2 indicates that the service restriction time is 15 seconds. Alternatively, the first satellite compressed data may include only whether the sampling time does not include whether the first satellite compressed data is a preset base value, that is, the first field is 2, which is used to indicate that the satellite data corresponding to the first satellite compressed data is not the preset base value and the sampling time of the differential encoding is 15 seconds, without retransmitting the data used to indicate that the satellite data is the preset base value.
A difference sign; for example, a first field in the first satellite compressed data is 0, which indicates that the satellite data is not a preset base value, a second field is 0, which indicates that the difference sign of the first satellite compressed data is negative, and a second field is 1, which indicates that the difference sign of the first satellite compressed data is positive; or, the first satellite compressed data may include only the difference symbol, and does not include whether the satellite data is a preset base value, that is, the first field is 1, which is used to indicate that the satellite data is not the preset base value, and the difference symbol of the first satellite compressed data is a positive sign, and is not required to be sent to indicate whether the satellite data is the preset base value;
the difference length. For example, a first field in the first satellite compressed data is 0, which indicates that the satellite data is not a preset base value, a second field is 00, which indicates that the difference length of the first satellite compressed data is 1, and a sum to a second field is 11, which indicates that the difference length of the first satellite compressed data is 4. Alternatively, only the difference length of the first satellite compressed data may not include whether the satellite data is a preset base value, that is, the first field is 00, which is used to indicate that the satellite data is not a preset base value and the difference length of the first satellite compressed data is 1, without transmitting any signal to indicate whether the satellite data is a preset base value.
Illustratively, positive and negative values are distinguished in decoding the first satellite compressed data, the difference value is replaced with its absolute value, and an additional field is used as a sign bit. In addition, in order to determine the length of the difference value, an additional two-bit field is required to represent the length of the byte. "00" means the difference length is 1, and accumulation of "11" means the difference length is 4. The first WeChat data of the plurality of satellite data may be represented as a base value for a predetermined period of time, and each subsequent value may be represented as a difference value derived from the previous value; each difference value additionally uses three bits to represent the difference sign and the difference length. To mark the differentially encoded sample times, 4 byte integers and 3 byte integers are used to represent the number of seconds and microseconds elapsed since epoch time; the number of seconds after differential operation can be represented by a field, and the number of microseconds is kept unchanged, because the effect of the differential operation is not obvious, and a redundant design is reserved for a lower sampling rate; and a redundant field representing time is used to represent the difference symbol and the difference length to minimize the satellite data length.
Example 3:
referring to fig. 3, fig. 3 illustrates another high throughput data transmission and storage method provided by an embodiment of the first aspect of the present invention, where the method is applied to a satellite and executed by the satellite. In other words, the method may be performed by software or hardware installed in a satellite, the method comprising the steps of:
in step S310, satellite data is acquired, and it is determined whether the satellite data is a preset base value.
This step may be described in step S110 in the embodiment of fig. 1, and will not be described herein.
In step S320, in case that the satellite data is not the preset base value, a difference operation is performed on the satellite data and each neighboring data of the satellite data to obtain a difference data set.
The satellite signal data after the channel coding may be regarded as not only text data but also numerals. Most channel encoded adjacent data is consistent in the high bytes and different in the low bytes. In a short time, the difference between the data will be small, so that a median signal can be used as a base value, the remaining signals being represented by the sum of the difference and a preset base value. The calculation formula of the difference between adjacent satellite data is as follows:
，
wherein, the liquid crystal display device comprises a liquid crystal display device,indicate satellite +.>First->Personal satellite data>
Is thatPersonal satellite data>Representing the difference between adjacent satellite data.
Step S330, determining first satellite compression data according to the preset basic value and the difference data set.
For example, the first satellite compressed data may be summed with the base value using the difference data set as follows:
wherein->Representing a preset base value.
Step S340, carrying out first canonical Huffman coding on the first satellite compressed data to obtain second satellite compressed data;
this step may be described in step S130 in the embodiment of fig. 1, and will not be described herein.
In step S350, signal conversion is performed on the second satellite compressed data to obtain the target transmission signal.
The second satellite compressed data is subjected to signal conversion so that a signal to be transmitted corresponding to the second satellite compressed data can be read in the storage device.
In step S360, the target transmission signal is transmitted to the first device.
In the related art, the technology for high-throughput data storage is one of a distributed storage architecture, a RAID technology and a compression technology. The distributed storage architecture stores data on a plurality of nodes, so that the data reliability is improved, and the concurrent reading and writing capacity of the data can be improved; the RAID technology uses a plurality of hard disks to form a disk array, and the data are respectively stored in the plurality of hard disks, so that the reliability and fault tolerance of data storage are improved. However, both distributed storage architecture and RAID technology require multiple storage devices and small satellites do not have the required internal space.
The method for providing the service provided by the embodiment of the invention determines whether the satellite data is a preset basic value or not by acquiring the satellite data; under the condition that satellite data is not a preset basic value, carrying out difference operation on the satellite data and each adjacent data of the satellite data to obtain a difference data set; determining first satellite compression data according to a preset basic value and a difference data set; performing first canonical Huffman encoding on the first satellite compressed data to obtain second satellite compressed data; performing signal conversion on the second satellite compressed data to obtain a target transmission signal; the target transmission signal is transmitted to the first device. By the method, satellite data are divided into the basic value and the difference value, more satellite data information is represented by fewer fields, the storage space of the satellite is saved, and more satellite data can be stored in the limited storage space of the small-sized low-cost satellite.
Optionally, as shown in fig. 4, the second embodiment of the present invention further provides an electronic device 700, including a processor 710 and a memory 720, where the memory 720 stores a program or an instruction that can be executed on the processor 710, and the program or the instruction implements each process of the first embodiment of the high throughput data transmission and storage method when executed by the processor 710, and the process can achieve the same technical effects, and is not repeated herein.
It should be noted that, the electronic device in the embodiment of the present invention includes: a server, a terminal, or other devices besides a terminal.
The above electronic device structure does not constitute a limitation of the electronic device, and the electronic device may include more or less components than illustrated, or may combine some components, or may be different in arrangement of components, for example, an input unit, may include a graphics processor (Graphics Processing Unit, GPU) and a microphone, and a display unit may configure a display panel in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit includes at least one of a touch panel and other input devices. Touch panels are also known as touch screens. Other input devices may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.
The memory may be used to store software programs as well as various data. The memory may mainly include a first memory area storing programs or instructions and a second memory area storing data, wherein the first memory area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory may include volatile memory or nonvolatile memory, or the memory may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (ddr SDRAM), enhanced SDRAM (Enhanced SDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM).
The processor may include one or more processing units; optionally, the processor integrates an application processor that primarily processes operations involving an operating system, user interface, application programs, and the like, and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor.
The embodiment of the present invention further provides a readable storage medium, where a program or an instruction is stored, where the program or the instruction realizes each process of the embodiment of the high throughput data transmission and storage method of the first aspect when executed by a processor, and the process can achieve the same technical effect, so that repetition is avoided, and no detailed description is given here.
The processor is a processor in the electronic device in the above embodiment. A readable storage medium includes a computer readable storage medium such as ROM, RAM, magnetic or optical disk, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present invention is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, regional, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part in the form of a computer software product stored on a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) including instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method of the embodiments of the present invention.
In the description of embodiments of the present invention, the terms "first," "second," "third," "fourth" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", "a third" and a fourth "may explicitly or implicitly include one or more such feature. In the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more.
In the description of embodiments of the invention, a particular feature, structure, material, or characteristic may be combined in any suitable manner in one or more embodiments or examples.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. A high throughput data transmission and storage method, applied to a satellite, characterized by comprising:
acquiring satellite data, and determining whether the satellite data is a preset basic value, wherein the satellite data is stored in a serialization manner;
carrying out differential coding on the satellite data under the condition that the satellite data is not a preset basic value, so as to obtain first satellite compressed data;
performing first canonical Huffman encoding on the first satellite compressed data to obtain second satellite compressed data;
and storing and transmitting the second satellite compressed data to the first device.
2. The method for high throughput data transmission and storage according to claim 1, wherein said performing a first canonical huffman encoding on said first satellite compressed data to obtain a second satellite compressed data comprises:
performing compression coding on high-frequency characters in the first satellite compressed data to obtain initial coded data;
and simplifying the initial coding data to obtain second satellite compression data.
3. The method for high throughput data transmission and storage according to any one of claims 1 or 2, wherein the second satellite compressed data includes encoded data and a length table, and after the first standard huffman encoding is performed on the first satellite compressed data, the second satellite compressed data is obtained, comprising:
determining whether the memory device of the satellite supports executing the second canonical Huffman coding according to a preset memory judgment criterion;
and under the condition that the storage equipment of the satellite supports to execute the second standard Huffman coding, carrying out the second standard Huffman coding on the length table to obtain third satellite compressed data.
4. The high throughput data transmission and storage method according to claim 3, wherein after performing the second canonical huffman encoding on the length table, there is:
and storing and transmitting the third satellite compressed data to the first device.
5. The high throughput data transmission and storage method of claim 1, wherein said method further has:
under the condition that the satellite data is a preset basic value, carrying out first canonical Huffman coding on the satellite data to obtain fourth satellite compressed data;
and storing and transmitting the fourth satellite compressed data to the first device.
6. The high throughput data transmission and storage method of claim 1, wherein a field in said first satellite compressed data is used to indicate at least one of:
whether the satellite data is a preset basic value or not;
differentially encoded sample times;
a difference sign;
the difference length.
7. The high throughput data transmission and storage method of claim 1, wherein said differentially encoding said satellite data to obtain first satellite compressed data, comprising:
performing difference operation on the satellite data and each adjacent data of the satellite data to obtain a difference data set;
and determining the first satellite compressed data according to the preset basic value and the difference data set.
8. The high throughput data transmission and storage method of claim 1, wherein said storing and transmitting said second satellite compressed data to a first device comprises:
performing signal conversion on the second satellite compressed data to obtain a target transmission signal;
transmitting the target transmission signal to a first device.
9. An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing when executing the computer program: the high throughput data transmission and storage method of any one of claims 1 to 8.
10. A computer-readable storage medium storing computer-executable instructions for: the high throughput data transmission and storage method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310861415.7A CN116582889B (en) | 2023-07-14 | 2023-07-14 | High-throughput data transmission and storage method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310861415.7A CN116582889B (en) | 2023-07-14 | 2023-07-14 | High-throughput data transmission and storage method, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116582889A true CN116582889A (en) | 2023-08-11 |
| CN116582889B CN116582889B (en) | 2023-11-21 |
Family
ID=87543562
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310861415.7A Active CN116582889B (en) | 2023-07-14 | 2023-07-14 | High-throughput data transmission and storage method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116582889B (en) |
Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5748121A (en) * | 1995-12-06 | 1998-05-05 | Intel Corporation | Generation of huffman tables for signal encoding |
| CN1520641A (en) * | 2001-04-04 | 2004-08-11 | 霍尼韦尔国际公司 | Canonical huffman encoded data decompression algorithm |
| CN101715132A (en) * | 2008-09-30 | 2010-05-26 | 雅马哈株式会社 | Lossless compression-encoding device |
| CN101915929A (en) * | 2010-07-13 | 2010-12-15 | 武汉大学 | GNSS observation value compression and decompression method based on sexagesimal system |
| CN103227920A (en) * | 2013-04-03 | 2013-07-31 | 南京信息工程大学 | Lossless compression method of multichannel satellite images |
| CN103618555A (en) * | 2013-11-27 | 2014-03-05 | 北京科技大学 | Compression method for ocean lane monitoring data |
| CN106170921A (en) * | 2014-02-20 | 2016-11-30 | 古鲁洛吉克微系统公司 | Relate to source code and the decoding method and device of the data of sign compression |
| US20170090776A1 (en) * | 2015-09-25 | 2017-03-30 | Seagate Technology Llc | Compression sampling in tiered storage |
| CN106788679A (en) * | 2016-12-21 | 2017-05-31 | 福建星海通信科技有限公司 | A kind of Beidou satellite communication device and method with speech identifying function |
| CN108464005A (en) * | 2015-12-22 | 2018-08-28 | 真实网络公司 | Real transform and reverse transformation system in Video coding and method |
| CN108989817A (en) * | 2018-09-03 | 2018-12-11 | 西安交通大学 | A kind of radar data compression method based on reference frame dislocation prediction |
| CN110191345A (en) * | 2019-04-16 | 2019-08-30 | 浙江大学 | A kind of increment type lossless compression method of prospect based on huffman coding, background separation |
| US20200128307A1 (en) * | 2012-06-26 | 2020-04-23 | BTS Software Solutions, LLC | Realtime Multimodel Lossless Data Compression System and Method |
| CN111818086A (en) * | 2020-07-24 | 2020-10-23 | 北京星途探索科技有限公司 | Frame differential compression transmission method suitable for telemetering digital quantity |
| US20210168131A1 (en) * | 2013-05-14 | 2021-06-03 | Kara Partners Llc | Systems and methods for variable-length encoding and decoding for enhancing computer systems |
| CN113242264A (en) * | 2021-07-09 | 2021-08-10 | 中国人民解放军国防科技大学 | Method and system for compressing and storing large-capacity data |
| CN114614829A (en) * | 2020-12-09 | 2022-06-10 | 千寻位置网络有限公司 | Satellite data frame processing method and device, electronic equipment and readable storage medium |
| CN115278777A (en) * | 2022-09-26 | 2022-11-01 | 江苏骏利精密制造科技有限公司 | Data transmission method for vehicle-mounted purifier |
-
2023
- 2023-07-14 CN CN202310861415.7A patent/CN116582889B/en active Active
Patent Citations (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5748121A (en) * | 1995-12-06 | 1998-05-05 | Intel Corporation | Generation of huffman tables for signal encoding |
| CN1520641A (en) * | 2001-04-04 | 2004-08-11 | 霍尼韦尔国际公司 | Canonical huffman encoded data decompression algorithm |
| CN101715132A (en) * | 2008-09-30 | 2010-05-26 | 雅马哈株式会社 | Lossless compression-encoding device |
| CN101915929A (en) * | 2010-07-13 | 2010-12-15 | 武汉大学 | GNSS observation value compression and decompression method based on sexagesimal system |
| US20200128307A1 (en) * | 2012-06-26 | 2020-04-23 | BTS Software Solutions, LLC | Realtime Multimodel Lossless Data Compression System and Method |
| CN103227920A (en) * | 2013-04-03 | 2013-07-31 | 南京信息工程大学 | Lossless compression method of multichannel satellite images |
| US20210168131A1 (en) * | 2013-05-14 | 2021-06-03 | Kara Partners Llc | Systems and methods for variable-length encoding and decoding for enhancing computer systems |
| CN103618555A (en) * | 2013-11-27 | 2014-03-05 | 北京科技大学 | Compression method for ocean lane monitoring data |
| CN106170921A (en) * | 2014-02-20 | 2016-11-30 | 古鲁洛吉克微系统公司 | Relate to source code and the decoding method and device of the data of sign compression |
| US20170090776A1 (en) * | 2015-09-25 | 2017-03-30 | Seagate Technology Llc | Compression sampling in tiered storage |
| CN108464005A (en) * | 2015-12-22 | 2018-08-28 | 真实网络公司 | Real transform and reverse transformation system in Video coding and method |
| CN106788679A (en) * | 2016-12-21 | 2017-05-31 | 福建星海通信科技有限公司 | A kind of Beidou satellite communication device and method with speech identifying function |
| CN108989817A (en) * | 2018-09-03 | 2018-12-11 | 西安交通大学 | A kind of radar data compression method based on reference frame dislocation prediction |
| CN110191345A (en) * | 2019-04-16 | 2019-08-30 | 浙江大学 | A kind of increment type lossless compression method of prospect based on huffman coding, background separation |
| CN111818086A (en) * | 2020-07-24 | 2020-10-23 | 北京星途探索科技有限公司 | Frame differential compression transmission method suitable for telemetering digital quantity |
| CN114614829A (en) * | 2020-12-09 | 2022-06-10 | 千寻位置网络有限公司 | Satellite data frame processing method and device, electronic equipment and readable storage medium |
| CN113242264A (en) * | 2021-07-09 | 2021-08-10 | 中国人民解放军国防科技大学 | Method and system for compressing and storing large-capacity data |
| CN115278777A (en) * | 2022-09-26 | 2022-11-01 | 江苏骏利精密制造科技有限公司 | Data transmission method for vehicle-mounted purifier |
Non-Patent Citations (3)
| Title |
|---|
| ZHIJUN WU: "ECDSA-Based Message Authentication Scheme for BeiDou-II Navigation Satellite System", 《IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS》 * |
| 徐冰;: "国外星上数据压缩算法发展研究", 国际太空, no. 01 * |
| 许展青, 陈桂林: "静止气象卫星的星上数据压缩技术", 红外, no. 02 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116582889B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108347302B (en) | Coding and decoding method and terminal | |
| EP3057255B1 (en) | Rate matching method and apparatus for polar codes, and wireless communication device | |
| US8265407B2 (en) | Method for coding and decoding 3D data implemented as a mesh model | |
| US11955992B2 (en) | Rate matching method and apparatus for polar code | |
| CN113327599A (en) | Voice recognition method, device, medium and electronic equipment | |
| CN116582889B (en) | High-throughput data transmission and storage method, electronic equipment and storage medium | |
| CN102811113B (en) | Character-type message compression method | |
| EP3893419A1 (en) | Data retransmission decoding method, apparatus and system, and communication device | |
| CN110226290B (en) | Encoding method and encoder | |
| CN113220651A (en) | Operation data compression method and device, terminal equipment and storage medium | |
| CN108880559B (en) | Data compression method, data decompression method, compression equipment and decompression equipment | |
| US20200133583A1 (en) | Method, apparatus and computer program product for storing data | |
| CN1748244A (en) | Pitch quantization for distributed speech recognition | |
| CN114638308A (en) | Method and device for acquiring object relationship, electronic equipment and storage medium | |
| US20060125660A1 (en) | Digital data compression robust relative to transmission noise | |
| CN102007786A (en) | Systems and methods for providing unequal message protection | |
| CN115833847B (en) | Polar code decoding method, polar code decoding device, communication equipment and storage medium | |
| CN116827682B (en) | Data processing method and device and computer equipment | |
| CN114567706B (en) | Public network intercom device jitter removal method and public network intercom system | |
| US20220108480A1 (en) | Bit plane decoding method and apparatus | |
| Rein | Low Complexity Text and Image Compression for Wireless Devices and Sensors | |
| CN113556548A (en) | Data processing method and device | |
| CN117195993A (en) | Address model training method, device, equipment and storage medium | |
| CN115802057A (en) | Data processing method, readable medium and electronic device | |
| CN117076799A (en) | Front-end resource caching method, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |