CN101622584A - Optimized storage and access method for a historian server of an automated system - Google Patents
Optimized storage and access method for a historian server of an automated system Download PDFInfo
- Publication number
- CN101622584A CN101622584A CN200780042118A CN200780042118A CN101622584A CN 101622584 A CN101622584 A CN 101622584A CN 200780042118 A CN200780042118 A CN 200780042118A CN 200780042118 A CN200780042118 A CN 200780042118A CN 101622584 A CN101622584 A CN 101622584A
- Authority
- CN
- China
- Prior art keywords
- value
- storer
- accumulator
- original value
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Exemplary embodiments of the disclosure relate to a method and a system for the storage and processing of process values of a technical process or a technical installation in a historian server which can be integrated into a process control system of a technical installation or of a technical process. In such a method and system a dedicated signal cache can be associated with every process value for its raw values provided by the process. The raw values provided by the process are stored in the respective signal caches and additional internal signal-individual aggregate accumulators for the desired aggregates are associated with every raw value. The aggregate accumulators are incrementally modified for the defined interval when a new raw value is received.
Description
Technical field
The present invention relates to a kind of best storage and access method that is used for historical record server (historian server), this method preferably is incorporated in the Process Control System of technical equipment or technical process.The method according to this invention is particularly suitable for the demand in the automated environment, and has obtained than the remarkable more performance of existing standard method.The invention still further relates to a kind of system that is used to carry out this method.
Background technology
For the long term archival of process values (being also referred to as process measurement value or process data), with the form of measured value (being also referred to as measured value historical record or process values historical record) sequence with data storage in so-called historical record server.
This historical record server is the special database with real-time function, be used for collection process data or measurement data, message and the report (file) relevant with filing, collected data and finished report is stored in protocol server with the time interval that defines hard disk with the longer-term storage of collected data.
In this case, because the automated system of technical equipment or technical process provide mass data, the operation that data in the historical record server are filed is to expend very much storer, promptly when read data, must move very many data, even only read single mean value (for example annual mean of process variable), also need through a very long time.
This historical record server is continuously inquired about so-called rendezvous value (aggregate value) (being also referred to as the ultimate value or the limit value of process values) usually in very long a period of time.For this purpose, must read whole raw data storages of desired signal, and determine corresponding set.Another shortcoming of presently used method is the rewriting of historical data.In this case, the rendezvous value that is pre-created will become unavailable suddenly, and only just available once more in the new set computation cycles that will be manually booted.
Method of stating below and the current method that is used for automatic technology are used simultaneously, and are intended to store the process values (signal that is also referred to as below) that is used to file.
Use the method for relational approach to operate according to following two modification stating by the mode of relational database.
In modification 1, all signals and signal index, timestamp, value and the state mode with " event-driven " is stored in the table, signal index and/or timestamp are as base index.In this case, triggering the incident of storing is near each change that measured value takes place the special-purpose tolerance range of the signal of determining.In the mode that time interval does not wait all signals are filed in a table.
In modification 2, all signals with the pattern storage of set time in a table, for example per 5 seconds.In this case, row have formed signal value and status information.
In fact, find that all there is following shortcoming in these two modification: if very a large amount of signal (for example more than 1000 signals) is stored in this table, reading the operation of independent signal time sequence so can carry out very slowly, this is because because the typical SQL visit of relational database, in order to search the desired signal index subsequently and to extract, at first must read all concordance lists of all data or all signals.In addition, in modification 2, for reducing scope, only with all signals of identical speed record, for example 5 seconds.Therefore will change slow value record several times, and even lose the value of variation in 5 seconds.
When using relational approach, because relational database technology may have been determined desirable value in advance circularly, and with its storage.For example, so-called OLAP (on-line analysis processing) solution usually every night operation once and determine and store required set.Because must read all not packed datas (being also referred to as raw data) from hard disk once more, so this operation is very consuming time and need high data throughput.In addition, must plan and create the OLAP operation, this is accompanied by considerable configuration.In addition, the compression script in the database has guaranteed that (for example every hour) forms set (for example set in a hour) and it is stored from all signals once more.
Although will shorten query time in theory like this, and but must plan and dispose this program to the annual mean in the report.
But, because the usual method of relational database, optimization described above reaches its restriction very soon, and complete reason owing to a large amount of storage signals, cause relational database only to handle " himself " very fast, this is because search history record for continuous productive set and over and over promptly for example must be read nearest one day or nearest one hour, and must calculate all values that is read.
Under the re-wrote history data conditions, another shortcoming has appearred.Under the situation of prognoses system, for example, created the new prediction that circulation is determined, i.e. Wei Lai historical time sequence, and rewrite old prediction during the course continuously.In this case, also can only be owing to produced other values now in service reusing next time, the rendezvous value that is pre-created will become suddenly and not be worth.In addition, two kinds concern that modification is not so feasible.The data of database model may must comprise the special extension of future value.
Describe another kind of method below by PGIM (generating information management) server that ABB uses, this method is used the storage means of high performance proprietary data-oriented stream with the form of proprietary database in database.In this case, memory period does not use relational data model.On the contrary, each signal all is counted as an independent database.The program of the above-mentioned relation method that this principle is evaded is because signal is separate, so needn't predefine during writing about the fixed mode of data rate.When reading the time series of signal, the subdata base of the signal that directly address is searched for, and need from the overall data storehouse, not filter in the mode of complexity.This method causes the operating personnel of technical equipment can write down all data as much as possible, and therefore all data of occurring with not packed data or raw data form of storage.
As a result, in recent years, archiving process data desired data storage and required hard-disk capacity increase greatly.Current, data storage to be stored is in 0.5 scope to about 5 terabytes, but according to data storage to be stored, trend is to continue to increase.
In principle, with regard to possible storage and reading rate, the use of the relational approach by the relational database mode described above efficient aspect several is lower than with the use to the storage means of high performance proprietary data-oriented stream of the form of proprietary database.
But, because described optimization is faster and therefore enough fast response can be provided aspect several under the situation of long inquiry, thereby and usually do not allow any independent recomputating because it only is designed for original data storage, therefore the storage means of the high performance proprietary data-oriented stream by using the proprietary data library format can not realize set optimization.
In addition, described above according to following problem having occurred in relational approach and the two kinds of methods according to proprietary method.The historical record server is query set value continuously in very long one period usually, for example so that create report.For this purpose, must read whole raw data storages of desired signal, and determine corresponding set.For example, if the about per second of putative signal (with one second resolution) provides a new value, and with timestamp, value and state storage in 18 bytes, then needed the storage space of 365*24*3600*18=528 megabyte in 1 year.
Therefore, in order to obtain a single annual mean, in example described above, must read and handle the data storage of 528 megabyte.
Summary of the invention
Therefore, the objective of the invention is to specify best storage and disposal route and a kind of system that carries out this method of the process values (being also referred to as process measurement value or process data) of technical process in a kind of historical record server or technical equipment, thereby overcome the shortcoming of above-described prior art.Specifically, the method for best storage by being used for process values in the historical record server of the present invention and processing improves the speed when reading rendezvous value.
According to the present invention, realized this purpose by method with the specified feature of claim 1.According to the preferred embodiment and improvement of the inventive method and the system that carries out this method, all in other claim and instructions, specified.
The present invention relates to a kind of storage and access method, this method is based on such fact: automatically carry out set storer that the oneself keeps online for each signal creation and keep set to determine, and need not with the extra historical data storage that reads once more.Simultaneously, automatically the rendezvous value of being created is used for client's inquiry, addressing is carried out in client's application pair set value specially.
Hereinafter when, where will be described in and how to form set.For this purpose,, be independent RAM buffer memory of each signal allocation (being also referred to as independent signal buffer memory) at the corresponding original signal that process provided.When data arrive, it is stored in independent signal buffer memory.
In addition, each signal has the proprietary signal set accumulator in inside that is used for desired set (for example minimum value separately, maximal value and integrated value) extraly.When the new value of new value, value and state with timestamp arrives, correspondingly incrementally changed the set accumulator at interval at defining.For this purpose, accept one than the current little new value of minimal set accumulator value (below be also referred to as minimum accumulator value) separately, as a new minimum accumulator value.
For maximum set accumulator value (being also referred to as cumulative maximum device value) and set accumulator integration (being also referred to as the integration accumulator), program is the same, has increased between additional zone for the integration accumulator.If the timestamp of now new value has exceeded predefined interval constraint, for example one hour, under the situation of the further operation that need not to read historical data, can form all set from accumulator immediately so, and it can be stored in proprietary signal set storer (4) (being preferably hard disk).Simultaneously at next interval initialization accumulator.
By disposing the proprietary signal set storer on the hard disk serviceable life separately, and therefore organize collective data with so-called ring form of memory.For this purpose, the always automatic set historical record that has disposed for the corresponding signal maintenance together.
With the form of ring advantageously config memory mean that owing to only change the pointer of indication beginning and ending, and directly rewriting occurs on the defined position thus, so the data on the hard disk are not to carry out displacement.
If necessary re-wrote history data, predicted value or correction in for example fixed historical record are then determined set once more for the time range that will be rewritten equally immediately, and at once correction are deposited in the storer.Because be present in the storer in order to carry out definite and required raw data, so this operation is carried out in very fast mode equally.
The set storer is the form of cascade.For example, this makes and set in five minutes is converted to set in a hour becomes possibility.For this purpose, once more according to the principle at raw data described above (each set buffer memory has a RAM buffer memory and a set accumulator again), in the higher set storer of the next one, determine set, for example set in a hour, specifically at be higher level.
In a preferred embodiment of the present invention, the concurrent process desired set of (for example every day) monitoring circularly in the historical record server, be not stored in the set storer for very slow signal (normally mean change is slower than per 20 seconds signal), this be because, because the small amount of data that produces, they are not favourable in the storage of set storer.In this case, automatically make decision, to determine with which signal storage which signal no longer needs to gather storer in gathering storer.
Do not follow the change of the overall performance others of historical record server in order when reading rendezvous value, can greatly to improve performance, it is favourable that the method according to this invention is provided, promptly compare, but inquiry can not make server be carried to the degree of real-time response selectively with method according to prior art.
Another advantage of the present invention is based on rendezvous value is effectively true immediately afterwards at interval surpassing time predefined.Further database operation such as do not need, for example in the relational approach that utilizes OLAP, usually at night.Therefore, all collective datas are consistent.
Owing to no longer must reading historical record in order to determine set, so compare with there being solution, with regard to the reading rate of the possible memory rate of original value and rendezvous value, performance has great raising.
In addition, the database performance of advantageously having avoided additional set to determine that operation is caused descends, and this is because saved this operation here fully.Therefore, all computing power is always available to client's inquiry.
Because each signal has an independent set buffer memory, on database, carry out and reshuffle duration of work, the needed real-time performance of historical record server can be guaranteed, and any problem can be do not occurred.Be with known standard SQL database difference, do not carry out the inner reorganization of the not true database of hiding, thereby can predict the time of each action.
Rendezvous value seamlessly is integrated in the enquiry module (being also referred to as query engine), and has therefore simplified the inquiry of using from the client.From client's viewpoint, only there is original signal.From client's viewpoint, dynamically and according to circumstances inquire about all set.
For example, gather and client's query set after 15 minutes, then automatically form set in 15 minutes, and can as previous habitually practising, not form set in 15 minutes from each value that is recorded in 15 minutes from set in 5 minutes if the set storer comprises 5 minutes.Therefore from client's viewpoint, only there is original signal, and dynamically and according to circumstances inquires about all set.
By current available value, each signal has its oneself timing, as this true result, also can favourablely keep predicted value easily extraly, except that the normal historical record that arrives in real time, wherein also comprise future value, promptly have signal usually, and do not need complex configurations or even the more special database design in SQL or OLAP method usually as the current time of nearest measured value as nearest value.
Can accessory rights require 5 systems that obtain carrying out the method according to this invention.
System of the present invention is used for the process values of the technical process of historical record server or technical equipment is carried out best storage and processing, be incorporated in the Process Control System of technical equipment or technical process described optimum system choosing, this system comprises: the independent signal buffer memory separately of original value distribution for process values is the corresponding original value distributing signal buffer memory that process provided.When data arrive, it is stored in independent signal buffer memory.
When corresponding original value arrives the historical record server, it is stored in independent signal buffer memory.In addition, can be used in the proprietary signal set accumulator of desired interior of a set for each original value branch extraly.When new value arrives, for defining the corresponding at interval set accumulator that incrementally changes.
Can be each original value and distribute timestamp, value and state.When the predetermined space of the timestamp that has exceeded new value limits, can form all set and need not further to read historical record data from accumulator immediately, and be stored in the proprietary signal set storer (being preferably hard disk), and can be simultaneously at next interval initialization accumulator.
In the favourable preferred embodiment of the present invention, can be by disposing the proprietary signal set storer on the hard disk independent serviceable life, and can therefore organize collective data with the form of ring storer.
Will be by exemplary embodiments illustrated in fig. 1, describe in more detail and explain the present invention and favourable preferred embodiment of the present invention and improvement.
Description of drawings
Fig. 1 shows the embodiment of invention system, is used for the technical process of historical record server 1 or the process values of technical equipment are carried out best storage and processing, and this system is incorporated in Process Control System, and shows data stream and set composition sequence.
Embodiment
System according to the present invention comprises the independent signal buffer memory 2 separately that is used for each signal, the corresponding original value that the described signal buffer memory process that is assigned to provides.For this purpose, the original value that will arrive by Scan Manager 3 (below the value of being also referred to as or data) is stored in the independent signal buffer memory 2.Can be each original value in this case and distribute timestamp, value and state.
The data that provide for the form collection process control system that reaches with packet also provide it to the purpose of signal buffer memory 2 in the historical record server 1 in real time, thereby Scan Manager 3 is provided.
Each signal has extraly and is used for the proprietary signal set accumulator 11 of desired interior of a set.When new value arriving signal buffer memory 2, correspondingly change set accumulator 11.
When exceeding the set interval aspect the time, set accumulator 11 is sent to accumulated value writes storage manager 5, writes storage manager 5 and then it is stored in the set storer 4.
By using so-called user API (application programming interfaces) 7.3, realize the inquiry of historical data of the set of using about original value and/or from the client, user API7.3 transfers to inquiry and reads storage manager 6.Read storage manager 6 decision whether directly from signal buffer memory 2 or from raw data file 10 or the specified data of setting a file from set storer 4 simply.
The client uses same by inquiry being transferred to the historical record of the user API7.3 rewriting signal of writing storage manager 5, for example uses forecasting sequence.Storage manager 5 is carried out in raw data file 10 directly and is rewritten, definite immediately new rendezvous value 11 that is produced under the situation that does not further read RAM, and similarly in set storer 4, rewrite these values.Advantageously, raw data and relative set are therefore always synchronous.
Also have refresh memory 8 according to system of the present invention, refresh memory 8 uses so-called cache flush process to control " hard disk with open arms " and empties buffer memory separately.
For this reason, the cache flush process makes that cache contents is write in the manager 5 fully, writes manager 5 and then the content of signal buffer memory is write in the raw data file 10, and rendezvous value 11 is write in the set storer 4.Empty signal buffer memory 1 can be used for the record of ensuing value subsequently once more now.
The cache flush process is also controlled each dynamic buffering size based on signal speed.For example, each time quantum is that so-called fast signal distributes a plurality of values.Be the bigger short-term buffer memory of these signal allocation.By contrast, slow signal little by little discharges its buffer memory to fast signal.
Therefore, the cache flush process with volume controlled of time control is operated in such a way: need seldom data access operation or data access operation to distribute as far as possible equably as much as possible.
Organize proprietary signal set storer 4 with the form of what is called ring storer, consequently the set historical record for the corresponding signal configuration always keeps automatically together in an advantageous manner.
Automatically the while (for example every night once) is determined and determines whether signal plans to receive the set buffer memory to refresh management process 8.
The security manager 9 that is used to the different user of historical record server to manage different rights of using (as read right, write permission and/or configuration authority) also is incorporated in the historical record server 1.
Claims (8)
1. method that the process values that is used for the technical process of historical record server (1) or technical equipment carries out best storage and processing, described method preferably is incorporated in the Process Control System of technical equipment or technical process,
It is characterized in that:
-at the original value that process provided, for each process values distributes an independent signal buffer memory (2),
-original value that process provided is stored in each signal buffer memory (2),
-for each original value divide extraly be used in the proprietary signal set accumulator of desired interior of a set (11) and
-when new original value arrives, correspondingly incrementally change set accumulator (11) at interval at defining.
2. the method for claim 1 is characterized in that, for original value distributes timestamp, value and state.
3. as one of above claim described method, it is characterized in that, when the predetermined space of the timestamp that has exceeded new value limits, under the situation of the further operation that need not to read historical data, form all set from accumulator immediately, and all set are stored in proprietary signal set storer (4), preferably be stored in the proprietary signal set storer (4) on the hard disk, and simultaneously at next interval initialization accumulator.
4. method as claimed in claim 3 is characterized in that, by disposing proprietary signal set storer (4) on the hard disk serviceable life separately, and therefore organizes collective data with the form of ring storer.
5. the system that the process values that is used for the technical process of historical record server (1) or technical equipment carries out best storage and processing is incorporated in the Process Control System of technical equipment or technical process described optimum system choosing, it is characterized in that:
-at the original value that process provided, can be the independent signal buffer memory (2) that each process values distributes a separation,
-the original value that provided can be stored in the independent signal buffer memory (2),
-can be each original value divide extraly be used in the proprietary signal set accumulator of desired interior of a set (11) and
-when new original value arrives, can correspondingly incrementally change set accumulator (11) at interval at defining.
6. system as claimed in claim 5 is characterized in that, can be original value and distributes timestamp, value and state.
7. as the described system in one of claim 5 and 6, it is characterized in that, when the predetermined space of the timestamp that has exceeded new value limits, under the situation of the further operation that need not to read historical data, can form all set from accumulator immediately, and all set can be stored in proprietary signal set storer (4), preferably be stored in the proprietary signal set storer (4) on the hard disk, and can be simultaneously at next interval initialization accumulator.
8. system as claimed in claim 7 is characterized in that, can be by disposing the proprietary signal set storer (4) on the hard disk serviceable life separately, and can therefore organize collective data with the form of ring storer.
Applications Claiming Priority (7)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102006053700.9 | 2006-11-13 | ||
| DE102006053700 | 2006-11-13 | ||
| DE102006053699 | 2006-11-13 | ||
| DE102006053699.1 | 2006-11-13 | ||
| DE102007040676.4 | 2007-08-29 | ||
| DE102007040676A DE102007040676A1 (en) | 2006-11-13 | 2007-08-29 | Optimized storage and access procedure in a history memory of an automation system |
| PCT/EP2007/009554 WO2008058647A1 (en) | 2006-11-13 | 2007-11-05 | Optimized storage and access method for a historian server of an automated system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101622584A true CN101622584A (en) | 2010-01-06 |
| CN101622584B CN101622584B (en) | 2012-11-07 |
Family
ID=41514929
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007800421189A Active CN101622584B (en) | 2006-11-13 | 2007-11-05 | Optimized storage and access method for a historian server of an automated system |
| CN2007800461025A Active CN101657768B (en) | 2006-11-13 | 2007-11-05 | System and method for the loss-free processing of process values of a technical installation or a technical process |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007800461025A Active CN101657768B (en) | 2006-11-13 | 2007-11-05 | System and method for the loss-free processing of process values of a technical installation or a technical process |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN101622584B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104756030A (en) * | 2012-10-31 | 2015-07-01 | 通用电气公司 | Method, system, and device for monitoring operations of a system asset |
-
2007
- 2007-11-05 CN CN2007800421189A patent/CN101622584B/en active Active
- 2007-11-05 CN CN2007800461025A patent/CN101657768B/en active Active
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104756030A (en) * | 2012-10-31 | 2015-07-01 | 通用电气公司 | Method, system, and device for monitoring operations of a system asset |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101657768B (en) | 2013-01-02 |
| CN101622584B (en) | 2012-11-07 |
| CN101657768A (en) | 2010-02-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105653591B (en) | A kind of industrial real-time data classification storage and moving method | |
| CN104965850B (en) | A kind of database high availability implementation method based on open source technology | |
| US9355112B1 (en) | Optimizing compression based on data activity | |
| CN101137981A (en) | Methods and apparatus for managing the storage of content in a file system | |
| CN107239382A (en) | The log processing method and system of a kind of container application | |
| NO326041B1 (en) | Procedure for managing data storage in a system for searching and retrieving information | |
| US8266273B2 (en) | Realtime process history server | |
| CN101496012A (en) | Data processing over very large databases | |
| CN101673192B (en) | Method for time-sequence data processing, device and system therefor | |
| CN101288074A (en) | Moving data from file on storage volume to alternate location to free space | |
| CN102163231A (en) | Method and device for data collection | |
| CN102637142A (en) | Computer system and method for realizing log management | |
| CN106021585A (en) | Traffic incident video access method and system based on time-space characteristics | |
| CN104111898A (en) | Hybrid storage system based on multidimensional data similarity and data management method | |
| CN109391629A (en) | Track traffic synthetic monitoring system data processing method | |
| CN110287152A (en) | A kind of method and relevant apparatus of data management | |
| CN110858210A (en) | Data query method and device | |
| CN108139904A (en) | Writing commands filter | |
| CN1996305A (en) | Data saving and reading method and device therefore, and data transmission system | |
| CN116842055A (en) | System and method for integrated processing of internet of things data batch flow | |
| CN109542339B (en) | Data layered access method and device, multilayer storage equipment and storage medium | |
| CN101622584B (en) | Optimized storage and access method for a historian server of an automated system | |
| CN107480281B (en) | Method for constructing unified view among heterogeneous file systems | |
| US5713023A (en) | Data management device providing increased storage and record access efficiency of clustered data | |
| US9104198B2 (en) | Optimized storage and access method for a historian server of an automated system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180521 Address after: Baden, Switzerland Patentee after: ABB TECHNOLOGY LTD. Address before: Zurich Patentee before: ABB Technology AG |