CN111770177A - Data processing method and data acquisition terminal - Google Patents
Data processing method and data acquisition terminal Download PDFInfo
- Publication number
- CN111770177A CN111770177A CN202010607217.4A CN202010607217A CN111770177A CN 111770177 A CN111770177 A CN 111770177A CN 202010607217 A CN202010607217 A CN 202010607217A CN 111770177 A CN111770177 A CN 111770177A
- Authority
- CN
- China
- Prior art keywords
- data
- processed
- cloud server
- currently acquired
- format
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 230000005540 biological transmission Effects 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000007906 compression Methods 0.000 claims abstract description 17
- 230000006835 compression Effects 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims description 9
- 238000012790 confirmation Methods 0.000 claims description 6
- 238000013480 data collection Methods 0.000 claims description 3
- 230000003111 delayed effect Effects 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000007405 data analysis Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013144 data compression Methods 0.000 description 2
- 239000002131 composite material Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/70—Type of the data to be coded, other than image and sound
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/18—Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a data processing method and a data acquisition terminal, wherein the method comprises the following steps: periodically acquiring data to be processed of the field equipment according to a preset period and a preset transmission protocol; comparing whether the currently acquired data to be processed is consistent with the data to be processed acquired last time; if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time, compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data, and storing the compressed data into an internal memory; and uploading the compressed data in the memory to the cloud server according to the subscription information of the cloud server. In the scheme, the data to be processed obtained by compression and the compressed data are uploaded to the cloud server in a delayed sending mode so as to adapt to the low-bandwidth and high-delay field environment and improve the stability of data transmission.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data processing method and a data acquisition terminal.
Background
With the development of industrialization, in order to ensure the normal operation of each field device, the data of the field device needs to be collected and uploaded to the cloud server, so that the requirements of data collection and remote data interaction are met.
At present, data of field equipment acquired by uploading is generally uploaded based on an intranet environment, but a satellite communication network is needed to be used for data transmission in areas such as ships, remote areas and the like, and due to the fact that the satellite communication network has the characteristics of low bandwidth and high delay, in the data transmission process, the data loss phenomenon is easily caused by the existing data uploading mode, and the data transmission stability is poor.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data processing method and a data acquisition terminal, so as to solve the problem of poor data transmission stability in the current data uploading manner of a field device.
In order to achieve the above purpose, the embodiments of the present invention provide the following technical solutions:
the first aspect of the embodiment of the invention discloses a data processing method, which is applied to a data acquisition terminal and comprises the following steps:
periodically acquiring data to be processed of the field equipment according to a preset period and a preset transmission protocol;
comparing whether the currently acquired data to be processed is consistent with the data to be processed acquired last time;
if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time, compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data, and storing the compressed data into an internal memory;
and uploading the compressed data in the memory to a cloud server according to subscription information of the cloud server.
Preferably, the type of the data to be processed is real-time data or historical data, the data format of the data to be processed is Int format, boot format or float format, and the step of compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data and storing the compressed data in a memory includes:
if the currently acquired data to be processed is historical data, compressing the currently acquired data to be processed by using a revolving door compression algorithm to obtain compressed data, and storing the compressed data in an internal memory;
if the currently acquired data to be processed is real-time data and in a Bool format, compressing the currently acquired data to be processed with the size of each Byte into compressed data with the size of 1bit, and storing the compressed data into a memory;
if the currently acquired data to be processed is real-time data and the currently acquired data to be processed is in an Int format or a float format, compressing the currently acquired data to be processed of each Byte size into compressed data of a preset size, and storing the compressed data into a memory.
Preferably, the field device includes a plurality of bit number values, and periodically acquires data to be processed of the field device according to a preset period and a preset transmission protocol, including:
and periodically acquiring data to be processed corresponding to each bit number value of the field equipment according to a preset period and a preset transmission protocol.
Preferably, the uploading the compressed data in the memory to the cloud server according to the subscription information of the cloud server includes:
according to subscription information of a cloud server, determining a bit number value of the field device subscribed by the cloud server;
and uploading the compressed data corresponding to the bit number value subscribed by the cloud server in the memory to the cloud server.
Preferably, the method further comprises the following steps:
and if an instruction which is sent by the cloud server and used for instructing to subscribe the real-time data is received, suspending uploading the compressed data in the memory to the cloud server, and uploading the data to be processed with the data format of the real-time data to the cloud server.
Preferably, after the compressed data in the memory is uploaded to the cloud server, the method further includes:
and receiving confirmation information fed back by the cloud server and used for indicating that the cloud server has received the compressed data.
A second aspect of the embodiments of the present invention discloses a data acquisition terminal, including:
the acquisition unit is used for periodically acquiring data to be processed of the field equipment according to a preset period and a preset transmission protocol;
the comparison unit is used for comparing whether the currently acquired data to be processed is consistent with the data to be processed acquired last time;
the processing unit is used for compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data and storing the compressed data into an internal memory if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time;
and the uploading unit is used for uploading the compressed data in the memory to the cloud server according to the subscription information of the cloud server.
Preferably, the type of the data to be processed is real-time data or historical data, the data format of the data to be processed is Int format, boot format, or float format, and the processing unit is specifically configured to: if the currently acquired data to be processed is historical data, compressing the currently acquired data to be processed by using a revolving door compression algorithm to obtain compressed data, and storing the compressed data in an internal memory; if the currently acquired data to be processed is real-time data and in a Bool format, compressing the currently acquired data to be processed with the size of each Byte into compressed data with the size of 1bit, and storing the compressed data into a memory; if the currently acquired data to be processed is real-time data and the currently acquired data to be processed is in an Int format or a float format, compressing the currently acquired data to be processed of each Byte size into compressed data of a preset size, and storing the compressed data into a memory.
Preferably, the field device comprises a plurality of bit number values, the acquisition unit being specifically configured to: and periodically acquiring data to be processed corresponding to each bit number value of the field equipment according to a preset period and a preset transmission protocol.
Preferably, the uploading unit is specifically configured to: according to subscription information of a cloud server, determining a bit number value of the field device subscribed by the cloud server, and uploading the compressed data corresponding to the bit number value subscribed by the cloud server in the memory to the cloud server.
Based on the data processing method and the data acquisition terminal provided by the embodiment of the invention, the method comprises the following steps: periodically acquiring data to be processed of the field equipment according to a preset period and a preset transmission protocol; comparing whether the currently acquired data to be processed is consistent with the data to be processed acquired last time; if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time, compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data, and storing the compressed data into an internal memory; and uploading the compressed data in the memory to the cloud server according to the subscription information of the cloud server. According to the scheme, when the acquired to-be-processed data changes, the currently acquired to-be-processed data is compressed to obtain compressed data, the compressed data is stored in the memory, and the compressed data in the memory is uploaded to the cloud server according to subscription information of the cloud server. The data to be processed obtained through compression and acquisition and the compressed data are uploaded to a cloud server in a delayed sending mode so as to adapt to the low-bandwidth and high-delay field environment and improve the stability of data transmission.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic view of communication among a field device, a data acquisition terminal, and a cloud server according to an embodiment of the present invention;
fig. 2 is a flowchart of a data processing method according to an embodiment of the present invention;
fig. 3 is another flowchart of a data processing method according to an embodiment of the present invention;
fig. 4 is another schematic communication diagram among the field device, the data acquisition terminal, and the cloud server according to the embodiment of the present invention;
fig. 5 is a block diagram of a data acquisition terminal according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
As can be seen from the background art, currently, the collected data of the field device is usually uploaded based on the intranet environment, but in the areas such as ships and remote areas, the data loss phenomenon is easily generated by adopting the existing data uploading mode, and the stability of data transmission is poor.
Therefore, the embodiment of the invention provides a data processing method and a data acquisition terminal, and the data to be processed obtained by compression and the compressed data are uploaded to a cloud server in a delayed sending mode, so that the method and the terminal are suitable for a low-bandwidth and high-delay field environment, and the stability of data transmission is improved.
In order to more conveniently understand the following content of a data processing method according to an embodiment of the present invention, a communication relationship among a field device, a data acquisition terminal, and a cloud server according to an embodiment of the present invention is explained with reference to fig. 1.
Referring to fig. 1, a schematic diagram of communication among a field device, a data acquisition terminal, and a cloud server according to an embodiment of the present invention is shown.
In fig. 1, the local end includes a field device and a data acquisition terminal, and the cloud server at least includes a data analysis service.
The data acquisition terminal acquires data corresponding to the field device and uploads the acquired data to the cloud server.
It can be understood that, because the data format of the data (the data acquired by the data acquisition terminal) at the local end is different from the data format required by the cloud server, and the cloud server has different display requirements for the received data, the cloud server analyzes the data acquired by the data acquisition terminal by using the data analysis service of the cloud server, and converts the data acquired by the data acquisition terminal into the data required by the cloud server.
Referring to fig. 2, a data processing method provided by an embodiment of the present invention is shown, where the data processing method is applied to a data acquisition terminal, and the data processing method includes the following steps:
step S201: the data acquisition terminal periodically acquires data to be processed of the field device according to a preset period and a preset transmission protocol.
In the process of implementing step S201 specifically, the data acquisition terminal is interconnected with the field device through a preset transmission protocol, for example, the data acquisition terminal is interconnected with the field device through a Modbus protocol.
The data acquisition terminal periodically acquires data of the field device interconnected with the data acquisition terminal, namely, to-be-processed data of the field device, according to a preset period, for example, the to-be-processed data of the field device is acquired every 1 second.
It can be understood that the field device includes a plurality of bit number values, each bit number value of the field device has data corresponding to it, and when the data to be processed of the field device is collected, the data to be processed (including the data of the register in the field device) corresponding to each bit number value of the field device is collected periodically according to a preset period and a preset transmission protocol.
Step S202: and the data acquisition terminal compares whether the currently acquired data to be processed is consistent with the data to be processed acquired last time. If the currently acquired to-be-processed data is not consistent with the last acquired to-be-processed data, executing step S203, and if the currently acquired to-be-processed data is consistent with the last acquired to-be-processed data, not compressing the currently acquired to-be-processed data.
In the process of implementing step S202 specifically, when the data acquisition terminal acquires the to-be-processed data of the field device each time, the currently acquired to-be-processed data is compared with the to-be-processed data acquired last time, and whether the acquired to-be-processed data changes is determined, that is, whether the currently acquired to-be-processed data is consistent with the previously acquired to-be-processed data is determined.
Such as: and the data acquisition terminal compares whether the currently acquired data to be processed is consistent with the data to be processed acquired last time by using a filtering algorithm.
And if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time, the data acquisition terminal compresses the currently acquired data to be processed and stores the compressed data into a memory (local storage).
It can be understood that, when comparing the currently acquired to-be-processed data with the last acquired to-be-processed data, the currently acquired to-be-processed data and the last acquired to-be-processed data corresponding to each bit number value are compared.
That is, for each bit number value, the currently acquired to-be-processed data corresponding to the bit number value is compared with the previously acquired to-be-processed data, and if the currently acquired to-be-processed data is inconsistent with the previously acquired to-be-processed data, the data acquisition terminal compresses the currently acquired to-be-processed data (data corresponding to the bit number value) and stores the compressed data in the memory.
Step S203: and the data acquisition terminal compresses the currently acquired data to be processed according to the data format and the type of the currently acquired data to be processed to obtain compressed data, and stores the compressed data in the memory.
It should be noted that, for the data to be processed with different data formats and types, in order to ensure the controllability of data compression, the data to be processed with different data formats and types is controlled by performing specialized data compression processing.
The type of the data to be processed is real-time data or historical data, and the data format of the data to be processed is Int format, Bool format or float format.
In the process of implementing step S203 specifically, if the currently acquired to-be-processed data is historical data, the data acquisition terminal compresses the currently acquired to-be-processed data by using a revolving door compression algorithm to obtain compressed data, and stores the compressed data in the memory.
It can be understood that, if the currently acquired to-be-processed data is historical data (data with a bit number value at a past time), the encoding of the to-be-processed data is configured as follows: the serial number of the bit number is 2Byte, the type of the bit number is 1Byte and the starting time is 4Byte, and the value time offset and the composite set of the value are formed through the encoding. The time offset (4Byte) is compared with the start time, and the value is encoded according to the format of the data to be processed, specifically: bool format is 1Byte, Int format is 4Byte and Float format is 4 Byte.
Historical data is compressed by the revolving door compression algorithm, thereby maintaining a higher compression ratio.
If the currently acquired data to be processed is real-time data and in a Bool format, the data acquisition terminal compresses the currently acquired data to be processed with the size of each Byte into compressed data with the size of 1bit, and stores the compressed data into the memory. For the real-time data (Bool format) of the same Byte, the real-time data of the Byte is compressed into 1bit, and the other 7 bits are respectively used for representing the data format and the data length of the real-time data.
If the currently acquired data to be processed is real-time data and the currently acquired data to be processed is in an Int format or a float format, the data acquisition terminal compresses the currently acquired data to be processed with the size of each Byte into compressed data with a preset size and stores the compressed data into the memory.
It can be understood that, if the currently acquired to-be-processed data is real-time data and the currently acquired to-be-processed data is in an Int format or a float format, the real-time data of each Byte is compressed into compressed data of a preset size according to the range definition, wherein the first 4 bits of the same Byte of the real-time data are used for representing the data format, and the last 4 bits of the same Byte of the real-time data are used for representing the data length.
It should be noted that the above compression method for the data to be processed is also only used for illustration, and other compression methods may be used to compress the data to be processed, which is not limited herein.
Step S204: and the data acquisition terminal uploads the compressed data in the memory to the cloud server according to the subscription information of the cloud server.
It should be noted that, the cloud server specifies, through a subscription operation, which bit number of the field device is to acquire the to-be-processed data generated by the data acquisition terminal, for example: the cloud server subscribes the bit number value A, and the data acquisition terminal periodically acquires data to be processed generated by the bit number value A.
When the data to be processed subscribed by the cloud server changes, the data acquisition terminal compresses and uploads the data to be processed to the cloud server, for example: the cloud server subscribes the bit number value A, and when the data to be processed of the bit number value A changes, the data acquisition terminal compresses and uploads the data to be processed of the bit number value A to the cloud server.
It should be further noted that the data acquisition terminal is accessed to the cloud server in a digital certificate manner, so that safe and reliable communication is ensured, and the binding of the analysis service in the cloud server and the identification of the data acquisition terminal is realized in a digital certificate issuing manner, so that a one-to-one correspondence relationship is formed.
The CommonName field in the digital certificate is used as the signing and issuing of the field device identity, namely the cloud server can identify the identity accessed by the data acquisition terminal through the serial number (unique identification) of the field device, so that the difference of the field devices is distinguished.
In the specific implementation process of step S204, the data acquisition terminal determines, according to the subscription information of the cloud server, a bit number value of the field device subscribed by the cloud server, and uploads compressed data in the memory, which corresponds to the bit number value subscribed by the cloud server, to the cloud server.
It should be noted that, the data acquisition terminal uploads the compressed data to the cloud server in a preset uploading manner, for example: the compressed data is uploaded to the cloud server by means of an Application Programming Interface (API).
After the cloud server receives the compressed data uploaded by the data acquisition terminal, the cloud server feeds back confirmation information to the data acquisition terminal to indicate that the cloud server has received the compressed data, so that whether the compressed data can be reached or not is determined, and the compressed data can be reached.
Such as: the cloud server relies on a Message Queue Telemetry Transport (MQTT) mechanism, realizes reliable piece data transmission by using a QOS (quality of service) > level 1, and sends the confirmation information to the data acquisition terminal.
Preferably, after the data acquisition terminal uploads the compressed data to the cloud server, the data acquisition terminal receives confirmation information fed back by the cloud server and used for indicating that the cloud server has received the compressed data.
Preferably, the cloud server can directly subscribe the real-time data of the field device, that is, the cloud server can check the real-time data of the field device in real time, in specific implementation, if the data acquisition terminal receives an instruction for instructing to subscribe the real-time data sent by the cloud server, the compressed data in the memory is suspended to be uploaded to the cloud server, and the data to be processed with the data format of the real-time data is uploaded to the cloud server, for example, the real-time data is uploaded to the cloud server through an API.
The cloud server analyzes the received compressed data (or real-time data), and takes an analysis result as data used for subsequent operation.
After the cloud server receives the compressed data (or real-time data) uploaded by the data acquisition terminal, detailed operation records are recorded in the system.
The cloud server uses the MQTT communication mode, and the uniform Modbus write command is encapsulated, so that the mode of controlling the Modbus write command downwards is realized, and the field device is controlled.
In the embodiment of the invention, when the acquired to-be-processed data changes, the currently acquired to-be-processed data is compressed to obtain compressed data, the compressed data is stored in the memory, and the compressed data in the memory is uploaded to the cloud server according to the subscription information of the cloud server. The data to be processed obtained through compression and acquisition and the compressed data are uploaded to a cloud server in a delayed sending mode so as to adapt to the low-bandwidth and high-delay field environment and improve the stability of data transmission.
To better explain the contents shown in fig. 2 of the above embodiment of the present invention, it is illustrated by fig. 3 and 4, and it should be noted that fig. 3 and 4 are only for illustration.
Referring to fig. 3, another flowchart of a data processing method provided in an embodiment of the present invention is shown, where the data processing method is applied to a data acquisition terminal, and the data processing method includes the following steps:
step S301: and the data acquisition terminal periodically acquires the data to be processed of the field equipment according to the data subscription service.
In the process of implementing step S301, the data acquisition terminal periodically acquires data to be processed of the field device through the Modbus data parsing service and the data subscription service.
It should be noted that the cloud server performs data subscription operation on the data acquisition terminal in an MQTT client/certificate manner.
Step S302: and if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time, the data acquisition terminal compresses the currently acquired data to be processed by using a compression algorithm and uploads the compressed data to the cloud server in an MQTT client/certificate mode.
It should be noted that, the content of each step in the above embodiment of fig. 2 can be referred to as an execution principle of step S301 and step S302, and details are not repeated here.
Referring to fig. 4, another communication diagram among the field device, the data acquisition terminal, and the cloud server according to the embodiment of the present invention is shown.
In fig. 4, the cloud server writes a Modbus command through a Modbus protocol control and write program. The cloud server processes the Modbus instruction (the instruction of the field device to be operated) by using the MQTT protocol, and sends the Modbus instruction wrapped by the MQTT protocol to the data acquisition terminal through the data analysis service.
The data acquisition terminal analyzes the Modbus instruction wrapped by the MQTT protocol to obtain a Modbus instruction, and sends the Modbus instruction to the field device, so that the cloud server can control the field device.
Corresponding to the data processing method provided by the embodiment of the present invention, referring to fig. 5, an embodiment of the present invention further provides a structural block diagram of a data acquisition terminal, where the data acquisition terminal includes: the device comprises an acquisition unit 501, a comparison unit 502, a processing unit 503 and an uploading unit 504;
the acquiring unit 501 is configured to periodically acquire data to be processed of the field device according to a preset period and a preset transmission protocol.
In a specific implementation, the field device includes a plurality of bit number values, and the acquisition unit 501 is specifically configured to: and periodically acquiring data to be processed corresponding to each bit number value of the field equipment according to a preset period and a preset transmission protocol.
The comparing unit 502 is configured to compare whether the currently acquired to-be-processed data is consistent with the previously acquired to-be-processed data.
The processing unit 503 is configured to, if the currently acquired to-be-processed data is inconsistent with the previously acquired to-be-processed data, compress the currently acquired to-be-processed data according to the data format and the type of the currently acquired to-be-processed data to obtain compressed data, and store the compressed data in the memory.
In a specific implementation, the type of the data to be processed is real-time data or historical data, the data format of the data to be processed is Int format, boot format, or float format, and the processing unit 503 is specifically configured to: if the currently acquired data to be processed is historical data, compressing the currently acquired data to be processed by using a revolving door compression algorithm to obtain compressed data, and storing the compressed data in an internal memory; if the currently acquired data to be processed is real-time data and in a Bool format, compressing the currently acquired data to be processed with the size of each Byte into compressed data with the size of 1bit, and storing the compressed data into an internal memory; if the currently acquired data to be processed is real-time data and the currently acquired data to be processed is in an Int format or a float format, compressing the currently acquired data to be processed with the size of each Byte into compressed data with a preset size, and storing the compressed data into a memory.
The uploading unit 504 is configured to upload the compressed data in the memory to the cloud server according to the subscription information of the cloud server.
In a specific implementation, the uploading unit 504 is specifically configured to: according to the subscription information of the cloud server, the bit number value of the field device subscribed by the cloud server is determined, and compressed data corresponding to the bit number value subscribed by the cloud server in the memory is uploaded to the cloud server.
In the embodiment of the invention, when the acquired to-be-processed data changes, the currently acquired to-be-processed data is compressed to obtain compressed data, the compressed data is stored in the memory, and the compressed data in the memory is uploaded to the cloud server according to the subscription information of the cloud server. The data to be processed obtained through compression and acquisition and the compressed data are uploaded to a cloud server in a delayed sending mode so as to adapt to the low-bandwidth and high-delay field environment and improve the stability of data transmission.
Preferably, in combination with the content shown in fig. 5, the data acquisition terminal further includes:
and the real-time subscription unit is used for suspending uploading the compressed data in the memory to the cloud server and uploading the data to be processed with the data format of real-time data to the cloud server if receiving an instruction which is sent by the cloud server and used for indicating subscription of the real-time data.
Preferably, in combination with the content shown in fig. 5, the data acquisition terminal further includes:
and the receiving unit is used for receiving confirmation information fed back by the cloud server and used for indicating the cloud server to receive the compressed data.
In summary, embodiments of the present invention provide a data processing method and a data acquisition terminal, which periodically acquire data to be processed of a field device according to a preset period and a preset transmission protocol, compress the data to be processed, which is acquired currently, to obtain compressed data when the acquired data to be processed changes, store the compressed data in an internal memory, and upload the compressed data in the internal memory to a cloud server according to subscription information of the cloud server. The data to be processed obtained through compression and acquisition and the compressed data are uploaded to a cloud server in a delayed sending mode so as to adapt to the low-bandwidth and high-delay field environment and improve the stability of data transmission.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data processing method is applied to a data acquisition terminal, and comprises the following steps:
periodically acquiring data to be processed of the field equipment according to a preset period and a preset transmission protocol;
comparing whether the currently acquired data to be processed is consistent with the data to be processed acquired last time;
if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time, compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data, and storing the compressed data into an internal memory;
and uploading the compressed data in the memory to a cloud server according to subscription information of the cloud server.
2. The method according to claim 1, wherein the type of the to-be-processed data is real-time data or historical data, the data format of the to-be-processed data is Int format, boot format or float format, and the compressing the currently acquired to-be-processed data according to the data format and type of the currently acquired to-be-processed data to obtain compressed data, and storing the compressed data in a memory comprises:
if the currently acquired data to be processed is historical data, compressing the currently acquired data to be processed by using a revolving door compression algorithm to obtain compressed data, and storing the compressed data in an internal memory;
if the currently acquired data to be processed is real-time data and in a Bool format, compressing the currently acquired data to be processed with the size of each Byte into compressed data with the size of 1bit, and storing the compressed data into a memory;
if the currently acquired data to be processed is real-time data and the currently acquired data to be processed is in an Int format or a float format, compressing the currently acquired data to be processed of each Byte size into compressed data of a preset size, and storing the compressed data into a memory.
3. The method of claim 1, wherein the field device includes a plurality of bit number values, and wherein periodically collecting data to be processed for the field device according to a predetermined period and a predetermined transmission protocol comprises:
and periodically acquiring data to be processed corresponding to each bit number value of the field equipment according to a preset period and a preset transmission protocol.
4. The method of claim 3, wherein the uploading the compressed data in the memory to the cloud server according to subscription information of the cloud server comprises:
according to subscription information of a cloud server, determining a bit number value of the field device subscribed by the cloud server;
and uploading the compressed data corresponding to the bit number value subscribed by the cloud server in the memory to the cloud server.
5. The method of claim 1, further comprising:
and if an instruction which is sent by the cloud server and used for instructing to subscribe the real-time data is received, suspending uploading the compressed data in the memory to the cloud server, and uploading the data to be processed with the data format of the real-time data to the cloud server.
6. The method of claim 1, wherein after uploading the compressed data in the memory to the cloud server, further comprising:
and receiving confirmation information fed back by the cloud server and used for indicating that the cloud server has received the compressed data.
7. A data collection terminal, characterized in that the data collection terminal comprises:
the acquisition unit is used for periodically acquiring data to be processed of the field equipment according to a preset period and a preset transmission protocol;
the comparison unit is used for comparing whether the currently acquired data to be processed is consistent with the data to be processed acquired last time;
the processing unit is used for compressing the currently acquired data to be processed according to the data format and type of the currently acquired data to be processed to obtain compressed data and storing the compressed data into an internal memory if the currently acquired data to be processed is inconsistent with the data to be processed acquired last time;
and the uploading unit is used for uploading the compressed data in the memory to the cloud server according to the subscription information of the cloud server.
8. The data acquisition terminal according to claim 7, wherein the type of the data to be processed is real-time data or historical data, the data format of the data to be processed is Int format, boot format, or float format, and the processing unit is specifically configured to: if the currently acquired data to be processed is historical data, compressing the currently acquired data to be processed by using a revolving door compression algorithm to obtain compressed data, and storing the compressed data in an internal memory; if the currently acquired data to be processed is real-time data and in a Bool format, compressing the currently acquired data to be processed with the size of each Byte into compressed data with the size of 1bit, and storing the compressed data into a memory; if the currently acquired data to be processed is real-time data and the currently acquired data to be processed is in an Int format or a float format, compressing the currently acquired data to be processed of each Byte size into compressed data of a preset size, and storing the compressed data into a memory.
9. The data acquisition terminal of claim 7, wherein the field device comprises a plurality of bit number values, and wherein the acquisition unit is specifically configured to: and periodically acquiring data to be processed corresponding to each bit number value of the field equipment according to a preset period and a preset transmission protocol.
10. The data acquisition terminal of claim 9, wherein the upload unit is specifically configured to: according to subscription information of a cloud server, determining a bit number value of the field device subscribed by the cloud server, and uploading the compressed data corresponding to the bit number value subscribed by the cloud server in the memory to the cloud server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607217.4A CN111770177A (en) | 2020-06-29 | 2020-06-29 | Data processing method and data acquisition terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010607217.4A CN111770177A (en) | 2020-06-29 | 2020-06-29 | Data processing method and data acquisition terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111770177A true CN111770177A (en) | 2020-10-13 |
Family
ID=72723291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010607217.4A Pending CN111770177A (en) | 2020-06-29 | 2020-06-29 | Data processing method and data acquisition terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111770177A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112256771A (en) * | 2020-10-22 | 2021-01-22 | 衡阳大河文化发展有限公司 | Exhibition platform system for expo |
| CN112804682A (en) * | 2020-12-31 | 2021-05-14 | 新奥数能科技有限公司 | Data transmission method and device, readable medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010083936A1 (en) * | 2009-01-23 | 2010-07-29 | Nortel Networks Limited | Method for packet communication |
| US20140201233A1 (en) * | 2013-01-15 | 2014-07-17 | Tencent Technology (Shenzhen) Company Limited | Method, device, and system for uploading data |
| CN108667463A (en) * | 2018-03-27 | 2018-10-16 | 江苏中科羿链通信技术有限公司 | Monitoring data compression method |
| JP2019106640A (en) * | 2017-12-13 | 2019-06-27 | コニカミノルタ株式会社 | Image forming apparatus and program |
| CN110481250A (en) * | 2019-08-16 | 2019-11-22 | 上海能塔智能科技有限公司 | Vehicular remote real-time monitoring data transmission method and system |
-
2020
- 2020-06-29 CN CN202010607217.4A patent/CN111770177A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010083936A1 (en) * | 2009-01-23 | 2010-07-29 | Nortel Networks Limited | Method for packet communication |
| US20140201233A1 (en) * | 2013-01-15 | 2014-07-17 | Tencent Technology (Shenzhen) Company Limited | Method, device, and system for uploading data |
| JP2019106640A (en) * | 2017-12-13 | 2019-06-27 | コニカミノルタ株式会社 | Image forming apparatus and program |
| CN108667463A (en) * | 2018-03-27 | 2018-10-16 | 江苏中科羿链通信技术有限公司 | Monitoring data compression method |
| CN110481250A (en) * | 2019-08-16 | 2019-11-22 | 上海能塔智能科技有限公司 | Vehicular remote real-time monitoring data transmission method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112256771A (en) * | 2020-10-22 | 2021-01-22 | 衡阳大河文化发展有限公司 | Exhibition platform system for expo |
| CN112804682A (en) * | 2020-12-31 | 2021-05-14 | 新奥数能科技有限公司 | Data transmission method and device, readable medium and electronic equipment |
| CN112804682B (en) * | 2020-12-31 | 2023-02-17 | 新奥数能科技有限公司 | Data transmission method and device, readable medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210350546A1 (en) | Adaptive video streaming | |
| CN108848060B (en) | Multimedia file processing method, processing system and computer readable storage medium | |
| CN111770177A (en) | Data processing method and data acquisition terminal | |
| CN100359919C (en) | Network camera | |
| CN110620890B (en) | File uploading method and system | |
| IL123819A (en) | Network media streaming | |
| CN112335202B (en) | Processing local area network diagnostic data | |
| CN102412999A (en) | Packet capturing based remote fault location method, system and device | |
| US20170085340A1 (en) | Method and apparatus for data communication of vehicle | |
| CN111954028B (en) | Screen projection method, device and equipment of audio data and storage medium | |
| CN105357302B (en) | A kind of network packet breaker point continuous transmission method and device | |
| CN110324681B (en) | Slicing method, device, storage medium and transmission system of multimedia data | |
| CN113286002A (en) | Power data transmission method based on multi-priority queue | |
| CN112335203B (en) | Processing local area network diagnostic data | |
| CN109788251B (en) | Video processing method, device and storage medium | |
| CN109584889B (en) | Audio transmission method and device and storage medium | |
| CN104754038A (en) | Monitoring method and device for grain in distributed grain storehouse | |
| CN105677885A (en) | Receiving and sending device and method and synchronizing system for achieving shipping industry data synchronization | |
| CN101635716B (en) | Method and system for remotely controlling network audio/video coder | |
| CN103763401A (en) | Distributed management system of devices in semiconductor plant | |
| CN113395324A (en) | Power distribution room data acquisition system and method, gateway and storage medium | |
| CN219536083U (en) | Sensor automatic test system | |
| CN110049348A (en) | A kind of video analytic method, system and video resolution server | |
| KR20210011243A (en) | Method and apparatus for monitoring HTTP/2 header compressed packet | |
| CN115941717B (en) | Method and system for data acquisition and transmission of energy storage power station |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201013 |