CN109542059B

CN109542059B - Historical data compression device and method

Info

Publication number: CN109542059B
Application number: CN201811376820.5A
Authority: CN
Inventors: 史旭明
Original assignee: State Nuclear Power Automation System Engineering Co Ltd
Current assignee: State Nuclear Power Automation System Engineering Co Ltd
Priority date: 2018-11-19
Filing date: 2018-11-19
Publication date: 2022-04-01
Anticipated expiration: 2038-11-19
Also published as: CN109542059A

Abstract

The invention relates to a historical data compression device, comprising: a controller configured to add a time stamp to a tag point data packet of the process quantity; a processor coupled to the controller and configured to perform the steps of: splitting a label point data packet of the process quantity into one or more label points, and adding the time mark to each label point; respectively performing first-stage compression on analog quantity label points and switching value label points in the label points; respectively performing second-stage compression on the analog quantity label points and the switching value label points which are subjected to the first-stage compression; and storing the analog quantity label points and the switching quantity label points which are compressed by the second stage.

Description

Historical data compression device and method

Technical Field

The invention relates to the field of industrial control, in particular to a distributed control device in the field of industrial control

Background

With the rapid development of upper computers and network technologies, the scale of industrial control systems is larger and larger, and the real-time performance is higher and higher. Tag points for a large number of process quantities in an industrial control system require their process values to be recorded, forming historical data, and therefore techniques for processing these process values are required. In the processing process value, core problems such as data acquisition, data processing, data transmission, data compression, data organization and the like need to be solved.

Disclosure of Invention

The invention aims to provide a historical data compression device and method, which have better clock precision and compression rate.

The present invention provides a historical data compression apparatus for solving the above-mentioned problems, comprising: a controller configured to add a time stamp to a tag point data packet of the process quantity; a processor coupled to the controller and configured to perform the steps of: splitting a label point data packet of the process quantity into one or more label points, and adding the time mark to each label point; respectively performing first-stage compression on analog quantity label points and switching value label points in the label points; respectively performing second-stage compression on the analog quantity label points and the switching value label points which are subjected to the first-stage compression; and storing the analog quantity label points and the switching quantity label points which are compressed by the second stage.

In an embodiment of the present invention, after performing the first-stage compression on the analog quantity tag point and the switching quantity tag point in the tag points, the method further includes: and sending the analog quantity label point and the switching value label point which are subjected to the first-stage compression into a data queue to wait for the second-stage compression.

In one embodiment of the invention, the analog quantity label point and the switching value label point which are compressed in the first stage are sent into the same data queue.

In an embodiment of the invention, the first stage compression and the second stage compression of the analog quantity label point are different.

In an embodiment of the present invention, the first stage compression and the second stage compression of the switching value tag point have different compression rates.

In an embodiment of the present invention, the first stage of compression of the analog label points is run-length compression, and the second stage of compression of the analog label points is slope compression or frequency compression.

In an embodiment of the present invention, the method for the first stage compression and the second stage compression of the switching value tag point includes: the tag points where the value of the switch amount does not change are merged.

In an embodiment of the invention, the processor is further configured to detect mutations in the analog tag points and directly record the mutation values without compression.

In an embodiment of the present invention, the controller is disposed in a distributed control device, and the processor is disposed in an upper computer.

Another aspect of the invention provides a method for compressing historical data. The method comprises the following steps: adding a time mark to a tag point data packet of the process quantity; splitting a label point data packet of the process quantity into one or more label points, and adding the time mark to each label point; respectively performing first-stage compression on analog quantity label points and switching value label points in the label points; respectively performing second-stage compression on the analog quantity label points and the switching value label points which are subjected to the first-stage compression; and storing the analog quantity label points and the switching quantity label points which are compressed by the second stage.

By adopting the technical scheme, compared with the prior art, the invention can efficiently and reliably realize the acquisition, compression and later storage of data, improves the time precision and the data compression rate and solves the problem of data loss.

Drawings

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below, wherein:

fig. 1 is a schematic diagram of a history data compression apparatus according to an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of a distributed control device of the history data compression apparatus according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of a structure of an upper computer of the historical data compression apparatus according to an embodiment of the present invention.

FIG. 4 is a diagram illustrating a process of compressing historical data according to an embodiment of the invention.

FIG. 5 is a flow chart of a method of historical data compression according to an embodiment of the invention.

Element numbering in the figures:

100: historical data compression device

110: distributed Control equipment (Distributed Control Device)

111: controller

112: read-only memory (ROM)

113: random Access Memory (RAM)

114: communication port

115: collection equipment

116: internal communication bus

120: upper computer

121: processor with a memory having a plurality of memory cells

122: read-only memory (ROM)

123: random Access Memory (RAM)

123 a: data queue

123 b: memory data block

124: communication port

125: input/output device

126: hard disk

127: user interface

128: internal communication bus

130: network

201: label point data packet (with time mark)

211: splitting data

212: judging whether the analog quantity is

213: analog run-length compression

214: switching variable bit compression

215: analog slope compression

216: switching variable bit compression

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described herein, and thus the present invention is not limited to the specific embodiments disclosed below.

As used in this application and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.

The following embodiments of the present invention describe a history data compression apparatus. The device is favorable for efficiently and reliably realizing the acquisition, compression and later-period storage of data, improves the time precision and the data compression rate, and solves the problem of data loss.

Fig. 1 is a schematic diagram of a history data compression apparatus according to an embodiment of the present invention. Referring to fig. 1, the history data compression apparatus 100 may include a distributed control device 110 and an upper computer 120, which are connected through a network 130. The network 130 may be any known wired network (e.g., ethernet) or wireless network, and is not expanded herein. The distributed control apparatus 110 and the upper computer 120 cooperate to implement the devices described in the following embodiments or variations thereof. A controller, a collection device, a communication port, and the like may be provided in the distributed control device 110. The upper computer 120 may be equipped with a processor, a user interface, input/output devices, and the like. The processor is responsible for executing instructions, the user interface may present various interfaces to the user, and the input and output devices may receive or transmit user inputs and outputs.

Fig. 2 is a schematic structural diagram of a distributed control device of the history data compression apparatus according to an embodiment of the present invention. Referring to fig. 2, the distributed control apparatus 110 may include a controller 111, a Read Only Memory (ROM)112, a Random Access Memory (RAM)113, a communication port 114, an acquisition apparatus 115, and an internal communication bus 116.

In some embodiments, the controller 111 may be comprised of one or more controllers. The distributed control apparatus 110 includes various forms of program storage units and data storage units, such as a Read Only Memory (ROM)112 and a Random Access Memory (RAM) 113. The memory unit can store various data files used by the distributed control apparatus 110 for processing and/or communication, as well as possible program instructions executed by the controller 111. The communication port 114 may enable data communication between the distributed control apparatus 110 and other components (not shown in the figures). In some embodiments, the distributed control apparatus 110 may send and receive information and data from the network 130 through the communication port 114. The collection device 115 collects various physical quantities, which may be temperature, pressure, vibration, etc., and converts them into analog electrical signals through corresponding sensors. The acquisition device 115 converts these analog electrical signals to digital signals for storage and pre-processing. An internal communication bus 116 may enable data communication between the various components of the distributed control apparatus 110.

Fig. 3 is a schematic diagram of a structure of an upper computer of the historical data compression apparatus according to an embodiment of the present invention. Referring to FIG. 3, the upper computer 120 may include a processor 121, a Read Only Memory (ROM)122, a Random Access Memory (RAM)123, a communication port 124, an input/output device 125, a hard disk 126, a user interface 127, and an internal communication bus 128.

The processor 121 is coupled to the controller 111 in fig. 2. In some embodiments, processor 121 may be comprised of one or more processors. The upper computer 120 includes various types of program storage units and data storage units, such as a Read Only Memory (ROM)122, a Random Access Memory (RAM)123, and a hard disk 126. The memory unit is capable of storing various data files for processing and/or communication by the upper computer 120, as well as possible program instructions for execution by the processor 121. The communication port 124 may enable data communication between the host computer 120 and other components (not shown). In some embodiments, the upper computer 120 may send and receive information and data from the network 130 through the communication port 124. The input/output device 125 supports input/output data flow between the host computer 120 and other components. By way of example, the input/output device 125 may include one or more of the following components: a mouse, a trackball, a keyboard, a touch-sensitive component, a sound receiver, etc. The user interface 127 may enable interaction and information exchange between the host computer 120 and a user. The internal communication bus 128 may enable data communication between components of the upper computer 120.

It is to be understood that the history data compression apparatus of the present application is not limited to be implemented by one upper computer, but may be cooperatively implemented by a plurality of online upper computers. The online upper computer can be connected and communicated through a local area network or a wide area network.

For example, the history data compression apparatus 100 of the present application may be implemented as a host computer program, stored in the hard disk 126, and recorded in the processor 121 for execution, so as to implement the method of the present application.

When the history data compression device is implemented as a host computer program, it may be stored in a readable storage medium of the host computer as a product. For example, a host computer-readable storage medium may include, but is not limited to, magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips), optical disks (e.g., Compact Disk (CD), Digital Versatile Disk (DVD)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memory (EPROM), card, stick, key drive). In addition, various storage media described herein can represent one or more devices and/or other machine-readable media for storing information. The term "machine-readable medium" can include, without being limited to, wireless channels and various other media (and/or storage media) capable of storing, containing, and/or carrying code and/or instructions and/or data.

It should be understood that the above-described embodiments are illustrative only. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and/or other electronic units designed to perform the functions described herein, or a combination thereof.

FIG. 4 is a diagram illustrating a process of compressing historical data according to an embodiment of the invention. The apparatus of the present embodiment may be implemented in the history data compression apparatus of fig. 1 or a variation thereof. In some embodiments, one or more of the following steps in the apparatus may be performed manually. The history data compression process of the present embodiment is described below with reference to fig. 4.

Referring to fig. 2 and 3 in combination, the controller 111 in the distributed control apparatus 110 adds a time stamp to the tag point data packet of the process quantity. Because the data is uploaded from the distributed control equipment to the upper computer and time is needed, the controller in the distributed control equipment adds the time mark to the process quantity label point data packet, so that the time in the time mark is more accurate.

The processor 121 in the upper computer 120 is coupled to the controller 111, and configured to perform the following steps: a process quantity tag point data packet (with time stamp) 201 is split into one or more tag points, this process being split data 211. Each label point after splitting is a detected quantity, and may be a temperature value or a pressure value, for example. The processor 121 adds the time stamp in the controller 111 for each label point after splitting. The processor 121 determines whether the tag point is an analog tag point or a switch tag point, and this process is an analog determination 212. The processor 121 performs a first stage of compression on the analog quantity tag points and the switching quantity tag points, respectively. The processor 121 performs a second stage of compression on the analog quantity tag points and the switching quantity tag points, respectively, which are subjected to the first stage of compression. The processor 121 stores the analog quantity tag points and the switching quantity tag points subjected to the second stage compression. In some embodiments of the present invention, the processor 121 puts the analog tag dots and the switching tag dots forming data blocks after the second stage of compression into the memory data block 123b and stores the data blocks on the hard disk 126.

In one example, the compressed values of the analog quantity label point and the switching value label point are formed into a data block by historical data software in the upper computer and stored in a hard disk of the computer. Other programs may retrieve the historical data in the historical data software.

In an embodiment of the present invention, the processor 121 of the historical data compression apparatus 100 performs a first stage of compression on the analog quantity tag point and the switching quantity tag point of the tag points, and then sends the compressed analog quantity tag point and switching quantity tag point to a data queue to wait for the second stage of compression. The data queue may be, for example, two buffers located in the memory RAM123 of fig. 3, and the buffers may play a role of buffering data to prevent the processor from having time to process the data.

In another embodiment of the present invention, the processor 121 of the historical data compressing apparatus 100 performs a first stage of compression on the analog quantity tag point and the switching value tag point of the tag points, sends the compressed analog quantity tag point and switching value tag point into the same data queue 123a, and waits for the second stage of compression, as shown in fig. 4.

In one example, the data queue 123a is located in a buffer area in the memory RAM123 of the upper computer 120, and the length of the buffer area can be defined in advance to be enough to store all the recorded values for 10 seconds or more. And storing the record values of the compressed analog quantity label point and the compressed switching value label point in a queue mode according to the time sequence.

In an embodiment of the invention, the historical data compression apparatus 100 performs a first stage compression and a second stage compression on the analog label points differently. The compression rate can be improved using different compression methods. Preferably, as shown in fig. 4, the processor 121 compresses the analog tag points to a first level of analog run compression 213 and compresses the analog tag points to a second level of analog slope compression 215. The second stage of compression of the analog quantity by processor 121 may also be frequency compression (not shown). The run-length compression of the analog quantity label point is lossless compression, the error of the compression can be defined in advance, the minimum error can be 0, and is usually 0.1% -1% of the range, and the time period of the compression can also be defined in advance. When the analog label point is slope-compressed and placed in the memory data block 123b, the time accuracy is guaranteed to be 50 milliseconds.

In an embodiment of the present invention, the historical data compression apparatus 100 has different compression rates for the first stage and the second stage of the switching value tag point. In some embodiments of the present invention, referring to fig. 4, the processor 121 performs a first stage of compressing the tag points of the switching values into a switching value shift compression 214, and performs a second stage of compressing the switching values into a switching value shift compression 216, wherein the compression rates of the two shift compressions are different. Using different compression ratios can improve the overall compression ratio. The type of compression of the first and second stages of compression of the switch tag points is typically time-dimensional compression. When the switching value label point is subjected to displacement compression and placed in the memory data block 123b, the time accuracy is ensured to be 1 millisecond.

In one example, the way of the history data compression apparatus 100 compressing the first stage and the second stage of the tag point of the switching value includes merging the tag points having no change in the value of the switching value. When the history data compression apparatus 100 performs the first-stage compression on the switching value label points, only the values of the switching values that have changed are recorded, and the values of the switching values having the same time are combined. If the value of the switching quantity is not changed, a recorded value is generated for a maximum of 1-5 minutes. Similarly, when the above-described history data compression apparatus 100 performs the second-stage compression on the switching amount tag points, only the values of the switching amounts that have changed are recorded, and the values of the switching amounts that have the same time are combined. If the value of the switching value has not changed, the previous value of the switching value is recorded. In the case of a long-term unchanged value of the switching variable, a value of the switching variable is recorded every 1 hour or so. Only the changed switching values are stored, so that the occupied memory space can be saved and the processing efficiency of the processor can be improved on the premise of ensuring that all the switching value change values are recorded. The compression rate can be further improved by specially processing the switching value which does not change for a long time.

In an embodiment of the invention, the processor 121 of the historical data compression apparatus 100 is further configured to detect a mutation of the analog tag point, and directly record a mutation value without compression. All compression results can be ensured to be within a predefined error range by specially processing the mutation of the analog quantity label point.

Another aspect of the invention provides a method for compressing historical data. The method is beneficial to efficiently and reliably realizing the acquisition, compression and later storage of data, improves the time precision and the data compression rate, and solves the problem of data loss.

A flowchart of a historical data compression method according to an embodiment of the present invention is shown in fig. 5. The history data compression method is explained below with reference to fig. 5.

Step 302, add time stamp to the tag point data packet of the process quantity.

Step 304, the tag point data packet of the process quantity is split into one or more tag points, and the time stamp is added to each tag point.

And step 306, respectively performing first-stage compression on the analog quantity label points and the switching quantity label points in the label points.

And 308, respectively performing second-stage compression on the analog quantity label points and the switching value label points which are subjected to the first-stage compression.

And step 310, storing the analog quantity label points and the switching quantity label points which are compressed in the second stage.

The invention provides a historical data compression device and a method. The device and the method are beneficial to efficiently and reliably realizing the acquisition, compression and later storage of data, improve the time precision and the data compression rate and solve the problem of data loss.

Having thus described the basic concept, it will be apparent to those skilled in the art that the foregoing disclosure is by way of example only, and is not intended to limit the present application. Various modifications, improvements and adaptations to the present application may occur to those skilled in the art, although not explicitly described herein. Such modifications, improvements and adaptations are proposed in the present application and thus fall within the spirit and scope of the exemplary embodiments of the present application.

Also, this application uses specific language to describe embodiments of the application. Reference throughout this specification to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic described in connection with at least one embodiment of the present application is included in at least one embodiment of the present application. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the present application may be combined as appropriate.

The upper computer program code required for the operation of the various parts of the present application may be written in any one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C + +, C #, vb. net, Python, etc., a conventional programming language such as C, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, a dynamic programming language such as Python, Ruby and Groovy, or other programming languages, etc. The program code can be completely run on the user upper computer, or run on the user upper computer as an independent software package, or run partially on the user upper computer and partially run on a remote upper computer, or run completely on the remote upper computer or a server. In the latter case, the remote host computer may be connected to the user host computer via any form of network, such as a Local Area Network (LAN) or a Wide Area Network (WAN), or to an external host computer (e.g., via the internet), or in a cloud computing environment, or as a service, such as a software as a service (SaaS).

Additionally, the order in which elements and sequences of the processes described herein are processed, the use of alphanumeric characters, or the use of other designations, is not intended to limit the order of the processes and methods described herein, unless explicitly claimed. While various presently contemplated embodiments of the invention have been discussed in the foregoing disclosure by way of example, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements that are within the spirit and scope of the embodiments herein. For example, although the system components described above may be implemented by hardware devices, they may also be implemented by software-only solutions, such as installing the described system on an existing server or mobile device.

Similarly, it should be noted that in the preceding description of embodiments of the application, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the embodiments. This method of disclosure, however, is not intended to require more features than are expressly recited in the claims. Indeed, the embodiments may be characterized as having less than all of the features of a single embodiment disclosed above.

Although the present invention has been described with reference to the present specific embodiments, it will be appreciated by those skilled in the art that the above embodiments are merely illustrative of the present invention, and various equivalent changes and substitutions may be made without departing from the spirit of the invention, and therefore, it is intended that all changes and modifications to the above embodiments within the spirit and scope of the present invention be covered by the appended claims.

Claims

1. A historical data compression apparatus, comprising:

the collection equipment is used for collecting the variable physical quantities including temperature, pressure and vibration and carrying out pretreatment;

the controller is used for receiving the data generated by the acquisition equipment and is configured to add a time mark to a tag point data packet of the process quantity;

a processor coupled to the controller and configured to perform the steps of:

splitting a label point data packet of the process quantity into one or more label points, and adding the time scale to each label point, wherein each split label point is a detected quantity; respectively performing first-stage compression on analog quantity label points and switching value label points in the label points;

respectively performing second-stage compression on the analog quantity label points and the switching value label points which are subjected to the first-stage compression; and

storing the analog quantity label points and the switching value label points which are compressed in the second stage;

the controller is arranged in the distributed control equipment, and the processor is arranged in the upper computer.

2. The apparatus of claim 1, wherein after performing a first stage of compression on the analog tag points and the switching tag points of the tag points, the apparatus further comprises:

and sending the analog quantity label point and the switching value label point which are subjected to the first-stage compression into a data queue to wait for the second-stage compression.

3. The history data compression apparatus according to claim 2, wherein the analog quantity tag point and the switching quantity tag point subjected to the first stage compression are fed into the same data queue.

4. The historical data compression apparatus of claim 1, wherein the first stage of compression of the analog quantity tag points is different from the second stage of compression.

5. The history data compression apparatus according to claim 1, wherein a compression rate of the first stage compression of the switching value tag point is different from that of the second stage compression.

6. The history data compression apparatus as claimed in claim 4, wherein the first stage compression of the analog quantity tag points is run-length compression, and the second stage compression of the analog quantity tag points is slope compression or frequency compression.

7. The apparatus of claim 1, wherein the first and second stages of compression of the switch amount tag points comprise: the tag points where the value of the switch amount does not change are merged.

8. The historical data compression apparatus of claim 1, wherein the processor is further configured to detect abrupt changes in the analog tag points and directly record abrupt change values without compression.