CN115883545A - High-frequency data transmission method and system - Google Patents
High-frequency data transmission method and system Download PDFInfo
- Publication number
- CN115883545A CN115883545A CN202310113701.5A CN202310113701A CN115883545A CN 115883545 A CN115883545 A CN 115883545A CN 202310113701 A CN202310113701 A CN 202310113701A CN 115883545 A CN115883545 A CN 115883545A
- Authority
- CN
- China
- Prior art keywords
- file
- frequency data
- zip
- compressed data
- csv
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Arrangements For Transmission Of Measured Signals (AREA)
Abstract
The invention provides a high-frequency data transmission method and a high-frequency data transmission system, wherein the method comprises the steps of collecting high-frequency data, carrying out aggregation calculation on the high-frequency data, cleaning the high-frequency data after the aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating the CSV file, generating a ZIP (zero-crossing sequence) compressed data file by the CSV file through a prefabricated tool, carrying out real-time transmission on the high-frequency data after the aggregation calculation, carrying out local transmission on the ZIP compressed data file, storing the high-frequency data after the aggregation calculation of the real-time transmission into a real-time database, and storing the ZIP compressed data file after the local transmission into a local database. The invention can reduce the bandwidth occupation in the transmission process and improve the data processing throughput.
Description
Technical Field
The present invention relates to the field of data transmission technologies, and in particular, to a high frequency data transmission method and system.
Background
With the development of internet technology, the work and life of people are greatly changed. Among them, communication technologies such as communication are becoming more and more popular, and thus, there is an increasing demand for data transmission capable of transmitting general characters, pictures, voices, images, videos, and the like.
In the prior art, data transmission of the Internet of things is carried out through protocols such as MQTT/COAP/HTTP and the like, and data are processed, compressed and stored on a platform side. The disadvantages are the pressure on the platform server and the high transmission bandwidth cost when processing high-frequency data.
Disclosure of Invention
Accordingly, the present invention is directed to a method and system for high frequency data transmission, which at least overcome the above-mentioned deficiencies in the prior art.
In one aspect, the present invention provides the following technical solutions, a high frequency data transmission method, including:
collecting high-frequency data;
performing aggregation calculation on the high-frequency data, cleaning the high-frequency data after the aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating a CSV file, and generating a ZIP (zero-crossing detection) compressed data file from the CSV file by a prefabricated tool, wherein the CSV file comprises a sensor ID (identity), a data timestamp and a collection value;
performing real-time transmission on the high-frequency data subjected to aggregation calculation, and performing local transmission on the ZIP compressed data file;
and storing the high-frequency data subjected to the aggregation calculation in real-time transmission into a real-time database, and storing the locally transmitted ZIP compressed data file into a local database.
Compared with the prior art, the invention has the beneficial effects that: in the process of transmitting high-frequency data, the high-frequency data are subjected to aggregation calculation and transmitted to a real-time database in real time, then the high-frequency data subjected to aggregation calculation are cleaned, the high-frequency data are generated into a ZIP (zero-crossing detection) compressed data file through a prefabricated tool and transmitted to a local database in a local mode, and through two transmission modes of real-time transmission and local transmission, the time consumed by a server for processing and storing the high-frequency data can be effectively reduced, the bandwidth occupation in the transmission process can be reduced, and the data processing throughput is improved.
Further, the step of storing the aggregation-calculated high-frequency data transmitted in real time into a real-time database includes:
and transmitting the high-frequency data after the aggregation calculation to a real-time database through an MQTT protocol, and storing the high-frequency data in the real-time database.
Further, the step of generating the CSV file into a ZIP compressed data file by using a prefabricated tool, where the CSV file includes a sensor ID, a data timestamp, and a collection value includes:
partitioning the CSV file according to the data timestamp and the sensor ID, wherein the partitioning is partitioning according to days;
using the prefabricated tool to import the CSV file into a temporary table, and segmenting the CSV file in the temporary table to obtain a segmented CSV file;
and scanning the fragmented CSV file, and performing ZIP compression on the fragmented CSV file to obtain the ZIP compressed data file.
Further, the step of scanning the fragmented CSV file and performing ZIP compression on the fragmented CSV file to obtain the ZIP compressed data file includes:
processing the ZIP compressed data file through a user-defined file transmission protocol so as to correctly perform sub-packaging, verification and retransmission on the ZIP compressed data file;
receiving the ZIP compressed data file, and decompressing and checking the ZIP compressed data file;
and moving the decompressed and verified ZIP compressed data file into a designated folder.
Further, the step of moving the decompressed and verified ZIP compressed data file into the designated folder includes:
and loading the decompressed and verified ZIP compressed data file into the local database in a TableAttach mode to finish the storage of the decompressed and verified ZIP compressed data file.
In another aspect, the present invention further provides a high frequency data transmission system, including:
the acquisition module is used for acquiring high-frequency data;
the calculation generation module is used for carrying out aggregation calculation on the high-frequency data, cleaning the high-frequency data after aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating a CSV file, and generating a ZIP (zero-crossing detection) compressed data file from the CSV file through a prefabricated tool, wherein the CSV file comprises a sensor ID (identity), a data timestamp and a collection value;
the transmission module comprises a real-time transmission unit and a local transmission unit, the real-time transmission unit is used for transmitting the high-frequency data subjected to aggregation calculation in real time, and the local transmission unit is used for locally transmitting the ZIP compressed data file;
and the storage module is used for storing the high-frequency data subjected to the aggregation calculation and transmitted in real time into a real-time database and storing the locally transmitted ZIP compressed data file into a local database.
Further, the storage module includes:
and the storage unit is used for transmitting the high-frequency data after the aggregation calculation to a real-time database through an MQTT protocol and storing the high-frequency data in the real-time database.
Further, the calculation generation module includes:
a partitioning unit, configured to partition the CSV file according to the data timestamp and the sensor ID, where the partitioning is a partition by number of days;
the fragmentation unit is used for importing the CSV file into a temporary table by using a prefabricated tool and fragmenting the CSV file in the temporary table to obtain a fragmented CSV file;
and the scanning unit is used for scanning the fragmented CSV file and carrying out ZIP compression on the fragmented CSV file to obtain a ZIP compressed data file.
Further, the calculation generation module further includes:
the processing unit is used for processing the ZIP compressed data file through a user-defined file transmission protocol so as to correctly perform sub-packaging, verification and retransmission on the ZIP compressed data file;
the decompression and verification unit is used for receiving the ZIP compressed data file and decompressing and verifying the ZIP compressed data file;
and the moving unit is used for moving the decompressed and verified ZIP compressed data file into a specified folder.
Further, the storage module further includes:
and the loading unit is used for loading the decompressed and verified ZIP compressed data file into the local database in a Table Attach mode to finish the storage of the decompressed and verified ZIP compressed data file.
Drawings
FIG. 1 is a flow chart of a high frequency data transmission method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a high frequency data transmission method according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a high frequency data transmission system according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a calculation generation module in a high-frequency data transmission system according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of a transmission module in a high-frequency data transmission system according to a third embodiment of the present invention;
fig. 6 is a schematic structural diagram of a memory module in a high frequency data transmission system according to a third embodiment of the present invention.
Description of the main element symbols:
| |
10 | |
20 |
| |
21 | |
22 |
| |
23 | |
24 |
| |
25 | |
26 |
| |
30 | Real- |
31 |
| |
32 | |
40 |
| |
41 | Loading |
42 |
| Real- |
51 | |
52 |
The following detailed description will further illustrate the invention in conjunction with the above-described figures.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully hereinafter with reference to the accompanying drawings. Several embodiments of the invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and the like as used herein are for illustrative purposes only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Example one
Referring to fig. 1, a high frequency data transmission method according to a first embodiment of the present invention is shown, and the method specifically includes steps S101 to S104;
s101, collecting high-frequency data;
when the intelligent gateway is specifically implemented, the intelligent gateway controls the collection of the dynamic high-frequency sensor equipment so as to enable the dynamic high-frequency sensor equipment to collect data, the high-frequency data refers to the fact that the sampling interval time is short, and the sampling frequency is larger than the sampling frequency researched at ordinary times, so that the intelligent gateway controls the dynamic high-frequency sensor equipment to collect the data according to the sampling requirement.
S102, performing aggregation calculation on the high-frequency data, cleaning the high-frequency data subjected to aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating a CSV file, and generating a ZIP (ZIP) compressed data file from the CSV file by a prefabricated tool, wherein the CSV file comprises a sensor ID (identity), a data timestamp and a collection value;
in specific implementation, in the process of performing aggregation calculation on the high-frequency data, the high-frequency data can be aggregated according to the requirement in seconds, minutes or hours, then the high-frequency data after aggregation calculation is cleaned, then the cleaned high-frequency data is written into a CSV data file, and a CSV file is generated every minute, wherein the CSV file data format comprises a sensor ID, a data timestamp and a collection value, the sensor ID represents a dynamic high-frequency sensor device, and the data timestamp represents the generation time of the CSV file.
S103, performing real-time transmission on the high-frequency data subjected to aggregation calculation, and performing local transmission on the ZIP compressed data file;
and S104, storing the high-frequency data subjected to aggregation calculation and transmitted in real time into a real-time database, and storing the locally transmitted ZIP compressed data file into a local database.
In summary, in the high-frequency data transmission method in the above embodiment of the present invention, the collected high-frequency data is subjected to aggregation calculation, the high-frequency data after the aggregation calculation is cleaned, the cleaned high-frequency data is written into a CSV data file, a CSV file is periodically generated, the CSV file is generated into a compressed data file by using a prefabricated tool, and then the compressed data file is transmitted by using two transmission modes, namely real-time transmission and local transmission and is stored respectively.
Example two
Referring to fig. 2, a high frequency data transmission method according to a second embodiment of the invention is shown, which includes steps S201 to S210;
s201, collecting high-frequency data;
when the intelligent gateway is specifically implemented, the intelligent gateway controls the collection of the dynamic high-frequency sensor equipment so as to enable the dynamic high-frequency sensor equipment to collect data, the high-frequency data refers to the fact that the sampling interval time is short, and the sampling frequency is larger than the sampling frequency researched at ordinary times, so that the intelligent gateway controls the dynamic high-frequency sensor equipment to collect the data according to the sampling requirement.
S202, performing aggregation calculation on the high-frequency data, cleaning the high-frequency data after the aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating a CSV file, and generating a ZIP (ZIP) compressed data file from the CSV file by a prefabricated tool, wherein the CSV file comprises a sensor ID (identity), a data timestamp and a collection value;
in specific implementation, in the process of performing aggregation calculation on the high-frequency data, the high-frequency data can be aggregated according to the requirement in seconds, minutes or hours, then the high-frequency data after aggregation calculation is cleaned, then the cleaned high-frequency data is written into a CSV data file, and a CSV file is generated every minute, wherein the CSV file data format comprises a sensor ID, a data timestamp and a collection value, the sensor ID represents a dynamic high-frequency sensor device, and the data timestamp represents the generation time of the CSV file.
It is worth noting that in this embodiment, the prefabricated tool is a clickhouse-local component.
S203, transmitting the high-frequency data after the aggregation calculation to a real-time database through an MQTT protocol, and storing the high-frequency data in the real-time database;
in specific implementation, in the process of aggregation calculation, aggregation calculation can be performed on the high-frequency data according to the requirements in seconds, minutes or hours, and the high-frequency data after the aggregation calculation is completed is transmitted to a real-time database through an MQTT protocol and stored in the real-time database.
S204, partitioning the CSV file according to the data timestamp and the sensor ID, wherein the partitioning is partitioning according to days;
in specific implementation, the server creates a data time partition table, and uses a MergeTree engine in Clickhouse, data is partitioned according to the day, and a sorting rule depends on a sensor ID and a data time stamp.
S205, using a prefabricated tool to import the CSV file into a temporary table, and segmenting the CSV file in the temporary table to obtain a segmented CSV file;
in specific implementation, a clickhouse-local tool is installed in an Ubuntu gateway system of an ARM architecture, and the generated CSV file calls the tool to perform data import and is inserted into a temp table consistent with a server structure, so that a corresponding fragmented CSV file is generated under a file directory specified in a gateway.
S206, scanning the fragmented CSV file, and carrying out ZIP compression on the fragmented CSV file to obtain a ZIP compressed data file;
s207, processing the ZIP compressed data file through a user-defined file transmission protocol so as to correctly perform sub-packaging, verification and retransmission on the ZIP compressed data file;
in specific implementation, after a newly added fragment CSV file is scanned, ZIP compression is carried out on the newly added fragment CSV file, and then file transmission is carried out with a platform server based on an MQTT protocol. And the user-defined file transmission protocol is used for processing, so that the correct sub-packaging, checking and retransmission are ensured.
S208, receiving the ZIP compressed data file, and decompressing and checking the ZIP compressed data file;
s209, moving the decompressed and verified ZIP compressed data file into a specified folder;
s210, loading the decompressed and verified ZIP compressed data file into the local database in a TableAttach mode, and finishing storage of the decompressed and verified ZIP compressed data file;
in specific implementation, the Server side correctly receives a ZIP compressed data file, decompresses and checks the ZIP compressed data file, moves the decompressed file into a cached folder under a Clickhouse-Server data directory, and then executes a Table Attach operation to load the file into a local database to finish data storage.
In summary, in the high-frequency data transmission method in the above embodiment of the present invention, in the process of local transmission, aggregation calculation is performed on high-frequency data, the high-frequency data after aggregation calculation is cleaned, the cleaned high-frequency data is written into a CSV data file, a CSV file is periodically generated, the CSV file is divided by days and is fragmented to obtain fragmented CSV files, the fragmented CSV files are compressed and transmitted, after receiving the compressed CSV file at the server, the compressed CSV file is moved into a cached folder, and the file is loaded into a local database through Table Attach operation, so that a platform user can view data of a collection granularity in real time, and through local transmission and real-time transmission, time consumption of processing and storage of the high-frequency data at the server can be effectively reduced, bandwidth occupation in the transmission process is reduced, and data processing throughput is improved.
EXAMPLE III
Referring to fig. 3 to 6, a high frequency data transmission system according to a third embodiment of the present invention is provided, and the system includes:
the acquisition module 10 is used for acquiring high-frequency data;
the calculation generation module 20 is configured to perform aggregation calculation on the high-frequency data, clean the high-frequency data after the aggregation calculation, write the cleaned high-frequency data into a CSV data file, periodically generate a CSV file, and generate a ZIP compressed data file from the CSV file through a prefabricated tool, where the CSV file includes a sensor ID, a data timestamp, and a collection value;
the transmission module 30 includes a real-time transmission unit 31 and a local transmission unit 32, where the real-time transmission unit 31 is configured to transmit the high-frequency data after aggregation calculation in real time, and the local transmission unit 32 performs local transmission on the ZIP compressed data file;
and the storage module 40 is configured to store the high-frequency data after aggregation calculation in real-time transmission into the real-time database 51, and store the ZIP compressed data file in local transmission into the local database 52.
In some optional embodiments, the system further comprises:
the storage unit 41 is configured to transmit the high-frequency data after aggregation calculation to the real-time database 51 through an MQTT protocol, and store the high-frequency data into the real-time database 51.
In some optional embodiments, the system further comprises:
a partitioning unit 21, configured to partition the CSV file according to the data timestamp and the sensor ID, where the partitioning is a partition by days;
the fragmentation unit 22 is configured to import the CSV file into a temporary table by using a prefabricated tool, and fragment the CSV file in the temporary table to obtain a fragmented CSV file;
the scanning unit 23 is configured to scan the fragmented CSV file, and perform ZIP compression on the fragmented CSV file to obtain a ZIP compressed data file.
In some optional embodiments, the system further comprises:
the processing unit 24 is configured to process the ZIP compressed data file through a custom file transfer protocol, so that the ZIP compressed data file is correctly subjected to packetization, verification and retransmission;
the decompression and verification unit 25 is configured to receive the ZIP compressed data file, and decompress and verify the ZIP compressed data file;
and the moving unit 26 is used for moving the decompressed and verified ZIP compressed data file into a specified folder.
In some optional embodiments, the system further comprises:
and the loading unit 42 is configured to load the decompressed and verified ZIP compressed data file into the local database in a Table Attach manner, so as to complete storage of the decompressed and verified ZIP compressed data file.
The high frequency data transmission system provided by the embodiment of the present invention has the same implementation principle and the same technical effects as those of the method embodiments, and for brief description, reference may be made to corresponding contents in the method embodiments for the parts that are not mentioned in the system embodiments.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the present invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of high frequency data transmission, the method comprising:
collecting high-frequency data;
performing aggregation calculation on the high-frequency data, cleaning the high-frequency data after the aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating a CSV file, and generating a ZIP (ZIP) compressed data file from the CSV file by using a prefabricated tool, wherein the CSV file comprises a sensor ID (identity), a data timestamp and a collection value;
performing real-time transmission on the high-frequency data subjected to aggregation calculation, and performing local transmission on the ZIP compressed data file;
and storing the high-frequency data subjected to the aggregation calculation in real-time transmission into a real-time database, and storing the locally transmitted ZIP compressed data file into a local database.
2. A high frequency data transmission method according to claim 1, wherein the step of storing the aggregation calculated high frequency data transmitted in real time into a real time database comprises:
and transmitting the high-frequency data after the aggregation calculation to a real-time database through an MQTT protocol, and storing the high-frequency data in the real-time database.
3. The high frequency data transmission method according to claim 1, wherein the step of generating the CSV file by a prefabrication tool into a ZIP compressed data file, the CSV file including a sensor ID, a data timestamp and a collection value comprises:
partitioning the CSV file according to the data timestamp and the sensor ID, wherein the partitioning is partitioning according to days;
the CSV file is imported into a temporary table by using the prefabricated tool, and the CSV file is fragmented in the temporary table to obtain a fragmented CSV file;
and scanning the fragmented CSV file, and carrying out ZIP compression on the fragmented CSV file to obtain the ZIP compressed data file.
4. The high-frequency data transmission method according to claim 3, wherein the step of scanning the fragmented CSV file and performing ZIP compression on the fragmented CSV file to obtain the ZIP compressed data file comprises:
processing the ZIP compressed data file through a user-defined file transmission protocol so as to correctly perform sub-packaging, verification and retransmission on the ZIP compressed data file;
receiving the ZIP compressed data file, and decompressing and checking the ZIP compressed data file;
and moving the decompressed and verified ZIP compressed data file into a specified folder.
5. The high frequency data transmission method according to claim 4, wherein the step of moving the decompressed and verified ZIP compressed data file into a designated folder comprises:
and loading the decompressed and verified ZIP compressed data file into the local database in a TableAttach mode to finish the storage of the decompressed and verified ZIP compressed data file.
6. A high frequency data transmission system, the system comprising:
the acquisition module is used for acquiring high-frequency data;
the calculation generation module is used for carrying out aggregation calculation on the high-frequency data, cleaning the high-frequency data after aggregation calculation, writing the cleaned high-frequency data into a CSV data file, periodically generating a CSV file, and generating a ZIP (zero-crossing detection) compressed data file from the CSV file through a prefabricated tool, wherein the CSV file comprises a sensor ID (identity), a data timestamp and a collection value;
the transmission module comprises a real-time transmission unit and a local transmission unit, the real-time transmission unit is used for transmitting the high-frequency data subjected to the aggregation calculation in real time, and the local transmission unit is used for locally transmitting the ZIP compressed data file;
and the storage module is used for storing the high-frequency data subjected to the aggregation calculation and transmitted in real time into a real-time database and storing the locally transmitted ZIP compressed data file into a local database.
7. The high frequency data transmission system according to claim 6, wherein the storage module comprises:
and the storage unit is used for transmitting the high-frequency data after the aggregation calculation to a real-time database through an MQTT protocol and storing the high-frequency data in the real-time database.
8. The high frequency data transmission system according to claim 6, wherein the calculation generation module comprises:
a partitioning unit, configured to partition the CSV file according to the data timestamp and the sensor ID, where the partitioning is a partition by number of days;
the fragmentation unit is used for importing the CSV file into a temporary table by using a prefabricated tool and fragmenting the CSV file in the temporary table to obtain a fragmented CSV file;
and the scanning unit is used for scanning the fragmented CSV file and carrying out ZIP compression on the fragmented CSV file to obtain a ZIP compressed data file.
9. The high frequency data transmission system of claim 8, wherein the calculation generation module further comprises:
the processing unit is used for processing the ZIP compressed data file through a user-defined file transmission protocol so as to correctly perform sub-packaging, verification and retransmission on the ZIP compressed data file;
the decompression and verification unit is used for receiving the ZIP compressed data file and decompressing and verifying the ZIP compressed data file;
and the moving unit is used for moving the decompressed and verified ZIP compressed data file into a specified folder.
10. The high frequency data transmission system of claim 9, wherein the memory module further comprises:
and the loading unit is used for loading the decompressed and verified ZIP compressed data file into the local database in a Table Attach mode to finish the storage of the decompressed and verified ZIP compressed data file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310113701.5A CN115883545B (en) | 2023-02-15 | 2023-02-15 | High-frequency data transmission method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310113701.5A CN115883545B (en) | 2023-02-15 | 2023-02-15 | High-frequency data transmission method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115883545A true CN115883545A (en) | 2023-03-31 |
| CN115883545B CN115883545B (en) | 2023-05-30 |
Family
ID=85761129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310113701.5A Active CN115883545B (en) | 2023-02-15 | 2023-02-15 | High-frequency data transmission method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115883545B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110246460A1 (en) * | 2010-03-31 | 2011-10-06 | Cloudera, Inc. | Collecting and aggregating datasets for analysis |
| US20130312043A1 (en) * | 2012-05-20 | 2013-11-21 | Transportation Security Enterprises, Inc. (Tse) | System and method for security data acquisition and aggregation on mobile platforms |
| CN106790511A (en) * | 2016-12-16 | 2017-05-31 | 酷伴科技（北京）有限责任公司 | A kind of data uploading method and system |
| CN109739818A (en) * | 2018-12-28 | 2019-05-10 | 浪潮软件股份有限公司 | A kind of portable high-throughput big data acquisition method and system |
| US20200252210A1 (en) * | 2019-01-31 | 2020-08-06 | Re Formsnet, Llc | Systems and methods for encryption and authentication |
| CN113342813A (en) * | 2021-06-09 | 2021-09-03 | 南京冰鉴信息科技有限公司 | Key value data processing method and device, computer equipment and readable storage medium |
| CN113377726A (en) * | 2021-06-02 | 2021-09-10 | 浪潮软件股份有限公司 | High-reliability distributed mass data transmission method and tool |
| CN115001148A (en) * | 2022-08-03 | 2022-09-02 | 杭州轻舟科技有限公司 | Energy storage power station data full-scale high-frequency real-time acquisition method and system |
-
2023
- 2023-02-15 CN CN202310113701.5A patent/CN115883545B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110246460A1 (en) * | 2010-03-31 | 2011-10-06 | Cloudera, Inc. | Collecting and aggregating datasets for analysis |
| US20130312043A1 (en) * | 2012-05-20 | 2013-11-21 | Transportation Security Enterprises, Inc. (Tse) | System and method for security data acquisition and aggregation on mobile platforms |
| CN106790511A (en) * | 2016-12-16 | 2017-05-31 | 酷伴科技（北京）有限责任公司 | A kind of data uploading method and system |
| CN109739818A (en) * | 2018-12-28 | 2019-05-10 | 浪潮软件股份有限公司 | A kind of portable high-throughput big data acquisition method and system |
| US20200252210A1 (en) * | 2019-01-31 | 2020-08-06 | Re Formsnet, Llc | Systems and methods for encryption and authentication |
| CN113377726A (en) * | 2021-06-02 | 2021-09-10 | 浪潮软件股份有限公司 | High-reliability distributed mass data transmission method and tool |
| CN113342813A (en) * | 2021-06-09 | 2021-09-03 | 南京冰鉴信息科技有限公司 | Key value data processing method and device, computer equipment and readable storage medium |
| CN115001148A (en) * | 2022-08-03 | 2022-09-02 | 杭州轻舟科技有限公司 | Energy storage power station data full-scale high-frequency real-time acquisition method and system |
Non-Patent Citations (2)
| Title |
|---|
| DONGCHAO MA: "A Lossless Convergence Method for Reducing Data Fragments on WSN", 《IEEE ACCESS》 * |
| 李娜;陈晰;吴帆;李祥珍;: "面向智能电网的物联网信息聚合技术", 信息通信技术 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115883545B (en) | 2023-05-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104424229B (en) | A kind of calculation method and system that various dimensions are split | |
| CN102571966B (en) | Network transmission method for large extensible markup language (XML) document | |
| CN102246165B (en) | Method and apparatus for representing and identifying feature descriptors utilizing a compressed histogram of gradients | |
| CN101552652A (en) | A document transmission method and transmission device | |
| CN113569117B (en) | Cloud platform system and method based on industrial internet big data service | |
| CN101345776B (en) | Content adapting implementing method and content adapting server | |
| CN103023796B (en) | network data compression method and system | |
| CN102447778A (en) | Message display method and device | |
| CN103944744A (en) | Method and system for log acquisition | |
| Hossain et al. | A data compression and storage optimization framework for iot sensor data in cloud storage | |
| CN106850761A (en) | Journal file storage method and device | |
| CN101616100B (en) | Methods for pushing, receiving and transmitting mail, mail server and mail client | |
| CN111835836A (en) | Intelligent Internet of things terminal data processing device and method | |
| CN104753922A (en) | Method for pre-loading, server side, client and system | |
| CN104753856A (en) | File receiving and sending method and system | |
| CN115883545B (en) | High-frequency data transmission method and system | |
| CN112335203A (en) | Processing local area network diagnostic data | |
| CN112667620A (en) | Data processing method and device, computer equipment and storage medium | |
| CN116805940A (en) | Data acquisition system and method based on extensible edge calculation | |
| CN111447229A (en) | Large-scale data acquisition method and device based on compressed sensing theory | |
| CN110855755B (en) | Message pushing method | |
| CN107688978A (en) | The method and device of sequence information is repeated for detecting | |
| CN102495816B (en) | Quick interrupt graded processing device and method | |
| US20060222256A1 (en) | Device, method, and program for compressing image data | |
| CN112000749A (en) | Server, data storage method and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |