CN101622584B - Optimized storage and access method for a historian server of an automated system - Google Patents
Optimized storage and access method for a historian server of an automated system Download PDFInfo
- Publication number
- CN101622584B CN101622584B CN2007800421189A CN200780042118A CN101622584B CN 101622584 B CN101622584 B CN 101622584B CN 2007800421189 A CN2007800421189 A CN 2007800421189A CN 200780042118 A CN200780042118 A CN 200780042118A CN 101622584 B CN101622584 B CN 101622584B
- Authority
- CN
- China
- Prior art keywords
- value
- storer
- signal
- original value
- accumulator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
Exemplary embodiments of the disclosure relate to a method and a system for the storage and processing of process values of a technical process or a technical installation in a historian server which can be integrated into a process control system of a technical installation or of a technical process. In such a method and system a dedicated signal cache can be associated with every process value for its raw values provided by the process. The raw values provided by the process are stored in the respective signal caches and additional internal signal-individual aggregate accumulators for the desired aggregates are associated with every raw value. The aggregate accumulators are incrementally modified for the defined interval when a new raw value is received.
Description
Technical field
The present invention relates to a kind of best storage and access method that is used for historical record server (historian server), this method preferably is incorporated in the Process Control System of technical equipment or technical process.Be particularly suitable for the demand in the automated environment according to the method for the invention, and obtained than the remarkable more performance of existing standard method.The invention still further relates to a kind of system that is used to carry out this method.
Background technology
For the long term archival of process values (being also referred to as process measurement value or process data), with the form of measured value (being also referred to as measured value historical record or process values historical record) sequence with data storage in so-called historical record server.
This historical record server is the special database with real-time function; Be used for collection process data or measurement data, message and the report (file) relevant with filing, collected data and accomplished report is stored in protocol server with the time interval that defines hard disk with the longer-term storage of collected data.
In this case; Because the automated system of technical equipment or technical process provide mass data; The operation that data in the historical record server are filed is to expend very much storer, promptly when read data, must move very many data; Even only read single mean value (the for example annual mean of process variable), also need pass through a very long time.
This historical record server is continuously inquired about so-called rendezvous value (aggregate value) (being also referred to as the ultimate value or the limit value of process values) usually in very long a period of time.For this purpose, must read whole raw data storages of desired signal, and confirm corresponding set.Another shortcoming of presently used method is the rewriting of historical data.In this case, the rendezvous value of creating in advance will become unavailable suddenly, and only just available once more in the new set computation cycles that will be manually booted.
The method of following stated and the current method that is used for automatic technology are used simultaneously, and are intended to store the process values that is used to file (below be also referred to as signal).
Use the method for relational approach to operate according to following two modification stating through the mode of relational database.
In modification 1, all signals and signal index, timestamp, value and the state mode with " event-driven " is stored in the table, signal index and/or timestamp are as base index.In this case, triggering the incident of storing is near each change that measured value takes place the special-purpose tolerance range of the signal of confirming.Mode so that time interval does not wait is filed all signals in a table.
In modification 2, all signals with the pattern storage of set time in a table, for example per 5 seconds.In this case, row have formed signal value and status information.
In fact; Find that all there is following shortcoming in these two modification: if very a large amount of signal (for example more than 1000 signals) is stored in this table; Reading the operation of independent signal time sequence so can carry out very slowly; This is because because the typical SQL visit of relational database in order to search the desired signal index subsequently and to extract, at first must be read all concordance lists of all data or all signals.In addition, in modification 2, for reducing scope, only with all signals of identical speed record, for example 5 seconds.Therefore will change slow value record several times, and even lose the value of variation in 5 seconds.
When using relational approach, because relational database technology possibly confirmed desirable value in advance circularly, and with its storage.For example, so-called OLAP (on-line analysis processing) solution usually every night operation once and confirm and store required set.Because must read all not packed datas (being also referred to as raw data) from hard disk once more, so this operation is very consuming time and need high data throughput.In addition, must plan and create the OLAP operation, this is accompanied by considerable configuration.In addition, the compression script in the database has guaranteed that (for example every at a distance from hour) forms set (for example set in a hour) and it is stored from all signals once more.
Although will shorten query time in theory like this, and but must plan and dispose this program to the annual mean in the report.
But; Because the usual method of relational database, the optimization of more than describing reach its restriction very soon, and complete reason owing to a large amount of storage signals; Cause relational database only to handle " himself " very fast; This is because search history record for continuous productive set and over and over promptly for example must be read nearest one day or nearest one hour, and must calculate all values that is read.
Under the re-wrote history data conditions, another shortcoming has appearred.Under the situation of prognoses system, for example, created the new prediction that circulation is confirmed, the promptly following historical time sequence, and rewrite old prediction during the course continuously.In this case, also can only be owing to produced other values now in service reusing next time, the rendezvous value of creating in advance will become suddenly and not be worth.In addition, two kinds concern that modification is not so feasible.The data of database model possibly must comprise the special extension of future value.
PGIM (generating information management) server that uses through ABB is below described another kind of method, and this method is used the storage means of high performance proprietary data-oriented stream with the form of proprietary database in database.In this case, memory period does not use relational data model.On the contrary, each signal all is counted as an independent database.The program of the above-mentioned relation method that this principle is evaded is because signal is separate, so needn't predefine during writing about the fixed mode of data rate.When reading the time series of signal, the subdata base of the signal that directly address is searched for, and need from the overall data storehouse, not filter with the mode of complicacy.This method causes the operating personnel of technical equipment can write down all data as much as possible, and therefore all data of occurring with not packed data or raw data form of storage.
As a result, in recent years, archiving process data desired data storage and required hard-disk capacity increase greatly.Current, with stored data storage 0.5 to the scope of about 5 terabytes, but according to stored data storage, trend is to continue to increase.
In principle, with regard to possible storage and reading rate, the use of the relational approach of more than describing of passing through relational database mode efficient aspect several is lower than with the use to the storage means of high performance proprietary data-oriented stream of the form of proprietary database.
But; Because described optimization is faster and therefore enough fast response can be provided aspect several under the situation of long inquiry; Thereby and usually do not allow any independent recomputating because it only is designed for original data storage, therefore the storage means of the high performance proprietary data-oriented stream through using the proprietary data library format can not realize set optimization.
In addition, more than describe according to following problem having occurred in relational approach and the two kinds of methods according to proprietary method.The historical record server is query set value continuously in very long one period usually, for example so that create report.For this purpose, must read whole raw data storages of desired signal, and confirm corresponding set.For example, if the about per second of putative signal (with one second resolution) provides a new value, and with timestamp, value and state storage in 18 bytes, then needed the storage space of 365*24*3600*18=528 megabyte in 1 year.
Therefore, in order to obtain a single annual mean, in the example of above description, must read and handle the data storage of 528 megabyte.
Summary of the invention
Therefore; The objective of the invention is to specify best storage and disposal route and a kind of system that carries out this method of the process values (being also referred to as process measurement value or process data) of technical process or technical equipment in a kind of historical record server, thereby overcome the shortcoming of above-described prior art.Specifically, the method for best storage through being used for process values in the historical record server of the present invention and processing improves the speed when reading rendezvous value.
According to the present invention, realized this purpose through method with the specified characteristic of claim 1.According to the preferred embodiment and improvement of the inventive method and the system that carries out this method, all in other claim and instructions, specify.
The present invention relates to a kind of storage and access method, this method is based on such fact: automatically carry out set storer that the oneself keeps online for each signal creation and keep set to confirm, and need not with the extra historical data storage that reads once more.Simultaneously, automatically the rendezvous value of being created is used for client's inquiry, addressing is carried out in client's application pair set value specially.
When, where, hereinafter will be described in and how to form set.For this purpose,, be independent RAM buffer memory of each signal allocation (being also referred to as independent signal buffer memory) to the corresponding original signal that process provided.When data arrives, it is stored in independent signal buffer memory.
In addition, each signal has the proprietary signal set accumulator in inside that is used for desired set (for example separately minimum value, maximal value and integrated value) extraly.When the new value of new value, value and state with timestamp arrives, correspondingly incrementally changed the set accumulator at interval to defining.For this purpose, accept one than the current little new value of minimal set accumulator value (below be also referred to as minimal accumulated device value) separately, as a new minimal accumulated device value.
For maximum set accumulator value (being also referred to as cumulative maximum device value) and set accumulator integration (being also referred to as the integration accumulator), program is the same, has increased between additional zone for the integration accumulator.If the timestamp of now new value has exceeded predefined interval constraint; For example one hour; Under the situation of the further operation that need not to read historical data, can form all set from accumulator immediately so, and can it be stored in proprietary signal set storer (4) (being preferably hard disk).Simultaneously to next interval initialization accumulator.
Through disposing the proprietary signal set storer on the hard disk serviceable life separately, and therefore organize collective data with so-called ring form of memory.For this purpose, the set historical record that has always disposed for the corresponding signal maintenance together automatically.
With the form of ring advantageously config memory mean that owing to only change the pointer of indication beginning and ending, and directly rewriting occurs on the defined position thus, so the data on the hard disk are not to carry out displacement.
If necessary re-wrote history data, predicted value or correction in the for example fixed historical record are then confirmed set once more for the time range that will be rewritten equally immediately, and at once correction are deposited in the storer.Because be present in the storer in order to carry out definite and required raw data, so this operation is carried out with very fast mode equally.
The set storer is the form of cascade.For example, this makes and converts set in five minutes into set in a hour and become possibility.For this purpose; Once more according to above description to the principle of raw data (each set buffer memory has a RAM buffer memory and a set accumulator again); In the higher set storer of the next one, confirm set, for example set in a hour, what be directed against specifically is higher level.
In a preferred embodiment of the present invention; The concurrent process desired set of (for example every day) monitoring circularly in the historical record server; Be not stored in the set storer for very slow signal (normally mean change is slower than per 20 seconds signal); This is because because the small amount of data that produces, they are not favourable in the storage of set storer.In this case, automatically make decision, to confirm with which signal storage which signal no longer need be gathered storer in gathering storer.
Do not follow the change of the overall performance others of historical record server in order when reading rendezvous value, can greatly to improve performance; Provide is favourable according to the method for the invention; Promptly compare, but inquiry can not make server be carried to the degree of real-time response selectively with method according to prior art.
Another advantage of the present invention is based on effectively true immediately above rendezvous value after the time predefined interval.Further database operation such as do not need, for example in the relational approach that utilizes OLAP, usually at night.Therefore, all collective datas are consistent.
Owing to no longer must reading historical record in order to confirm set, so compare with there being solution, with regard to the reading rate of the possible memory rate of original value and rendezvous value, performance has great raising.
In addition, the database performance of advantageously having avoided additional set to confirm that operation is caused descends, and this is because saved this operation here fully.Therefore, all computing power is always available to client's inquiry.
Because each signal has an independent set buffer memory, on database, carry out and reshuffle duration of work, the needed real-time performance of historical record server can be guaranteed, and any problem can be do not occurred.Be with known standard SQL database difference, do not carry out the inner reorganization of the not true database of hiding, thereby can foresee the time of each action.
Rendezvous value seamlessly is integrated in the enquiry module (being also referred to as query engine), and has therefore simplified the inquiry of using from the client.From client's viewpoint, only there is original signal.From client's viewpoint, dynamically and according to circumstances inquire about all set.
For example, gather and client's query set after 15 minutes, then automatically form set in 15 minutes, and can as previous habitually practising, not form set in 15 minutes from each value that is recorded in 15 minutes from set in 5 minutes if the set storer comprises 5 minutes.Therefore from client's viewpoint, only there is original signal, and dynamically and according to circumstances inquires about all set.
By current available value; Each signal has its oneself timing, as this true result, also can favourablely keep predicted value easily extraly; Except that the normal historical record that arrives in real time; Wherein also comprise future value, promptly have signal usually, and do not need usually complex configurations in SQL or OLAP method or even more special database design as the current time of nearest measured value as nearest value.
Can accessory rights require 5 systems that obtain carrying out according to the method for the invention.
System of the present invention is used for the process values of the technical process of historical record server or technical equipment is carried out best storage and processing; Be incorporated in the Process Control System of technical equipment or technical process said optimum system choosing; This system comprises: the independent signal buffer memory separately of original value distribution for process values is the corresponding original value distributing signal buffer memory that process provided.When data arrives, it is stored in independent signal buffer memory.
When corresponding original value arrives the historical record server, it is stored in independent signal buffer memory.In addition, can be used in the proprietary signal set accumulator of desired interior of a set for each original value branch extraly.When new value arrives, for defining the corresponding at interval set accumulator that incrementally changes.
Can be each original value and distribute timestamp, value and state.When the predetermined space of the timestamp that has exceeded new value limits; Can form all set and need not further to read historical record data from accumulator immediately; And be stored in the proprietary signal set storer (being preferably hard disk), and can be simultaneously to next interval initialization accumulator.
In the favourable preferred embodiment of the present invention, can be through disposing the proprietary signal set storer on the hard disk independent serviceable life, and can therefore organize collective data with the form of ring storer.
Will be through exemplary embodiments illustrated in fig. 1, describe in more detail and explain the present invention and favourable preferred embodiment of the present invention and improvement.
Description of drawings
Fig. 1 shows invention system implementation example, is used for the technical process of historical record server 1 or the process values of technical equipment are carried out best storage and processing, and this system is incorporated in Process Control System, and shows data stream and set composition sequence.
Embodiment
System according to the present invention comprises the independent signal buffer memory 2 separately that is used for each signal, the corresponding original value that the said signal buffer memory process that is assigned to provides.For this purpose, the original value that will arrive through Scan Manager 3 (below the value of being also referred to as or data) is stored in the independent signal buffer memory 2.Can be each original value in this case and distribute timestamp, value and state.
The data that provide for the form collection process control system that reaches with packet also offer it purpose of signal buffer memory 2 in the historical record server 1 in real time, thereby Scan Manager 3 is provided.
Each signal has extraly and is used for the proprietary signal set accumulator 11 of desired interior of a set.When new value arriving signal buffer memory 2, correspondingly change set accumulator 11.
When aspect the time, exceeding the set interval, set accumulator 11 is sent to accumulated value writes storage manager 5, writes storage manager 5 and then it is stored in the set storer 4.
Through using so-called user API (application programming interfaces) 7.3, the inquiry of historical data of the set that realizes using about original value and/or from the client, user API7.3 transfers to inquiry and reads storage manager 6.Read storage manager 6 decision whether directly from signal buffer memory 2 or from raw data file 10 or the specified data of setting a file from set storer 4 simply.
The client uses same through inquiry being transferred to the historical record of the user API7.3 rewriting signal of writing storage manager 5, for example uses forecasting sequence.Storage manager 5 is carried out in raw data file 10 directly and is rewritten, definite immediately new rendezvous value 11 that is produced under the situation that does not further read RAM, and likewise in set storer 4, rewrite these values.Advantageously, raw data and relative set are therefore always synchronous.
Also have refresh memory 8 according to system of the present invention, refresh memory 8 uses so-called cache flush process to control " hard disk with open arms " and empties buffer memory separately.
For this reason, the cache flush process makes that cache contents is write in the manager 5 fully, writes manager 5 and then the content of signal buffer memory is write in the raw data file 10, and rendezvous value 11 is write in the set storer 4.Empty signal buffer memory 1 can be used for the record of ensuing value subsequently once more now.
The cache flush process is also controlled each dynamic buffering size based on signal speed.For example, each time quantum is that so-called fast is distributed a plurality of values.Be the bigger short-term buffer memory of these signal allocation.By contrast, slow signal little by little discharges its buffer memory to fast.
Therefore, the cache flush process with volume controlled of time control is operated in such a way: need seldom data access operation or data access operation to distribute as far as possible equably as much as possible.
Organize proprietary signal set storer 4 with the form of what is called ring storer, consequently the set historical record for the corresponding signal configuration always keeps automatically together in an advantageous manner.
Automatically the while (for example every night once) is confirmed and determines whether signal plans to receive the set buffer memory to refresh management process 8.
The security manager 9 that the different user that is used to the historical record server is managed different rights of using (like read right, write permission and/or configuration authority) also is incorporated in the historical record server 1.
Claims (4)
1. method that the process values that is used for the technical process of historical record server (1) or technical equipment carries out best storage and processing, said method is incorporated in the Process Control System of technical equipment or technical process,
It is characterized in that:
-to the original value that process provided, for each process values distributes an independent signal buffer memory (2),
-original value that process provided is stored in each signal buffer memory (2),
-for dividing extraly, each original value is used in the proprietary signal set accumulator of desired interior of a set (11),
-for original value distributes timestamp, value and state,
-when new original value arrives, correspondingly to predetermined space incrementally change set accumulator (11) and
-when the predetermined space of the timestamp that has exceeded new value limits; Under the situation of the further operation that need not to read historical data, form all set from the set accumulator immediately; And all set are stored in proprietary signal set storer (4), and simultaneously to next initialization set accumulator at interval.
2. the method for claim 1; It is characterized in that, all set are stored in the proprietary signal set storer (4) on the hard disk, wherein; Through disposing the proprietary signal set storer (4) on the hard disk serviceable life separately, and therefore organize collective data with the form of ring storer.
3. system that the process values that is used for the technical process of historical record server (1) or technical equipment carries out best storage and processing, said system is incorporated in the Process Control System of technical equipment or technical process, it is characterized in that:
-said system is included as the independent signal buffer memory (2) of a separation of each process values distribution, the corresponding original value that the said signal buffer memory process that is assigned to provides,
-the original value that provided is stored in the independent signal buffer memory (2),
-for dividing extraly, each original value is used in the proprietary signal set accumulator of desired interior of a set (11), for original value distributes timestamp, value and state,
-when new original value arrives, correspondingly to predetermined space incrementally change set accumulator (11) and
-when the predetermined space of the timestamp that has exceeded new value limits; Under the situation of the further operation that need not to read historical data, form all set from the set accumulator immediately; And all set are sent to write storage manager (5); Write storage manager (5) all set are stored in proprietary signal set storer (4), and simultaneously to next interval initialization set accumulator.
4. system as claimed in claim 3; It is characterized in that all set are stored in the proprietary signal set storer (4) on the hard disk, wherein; Through disposing the proprietary signal set storer (4) on the hard disk serviceable life separately, and therefore organize collective data with the form of ring storer.
Applications Claiming Priority (7)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102006053699.1 | 2006-11-13 | ||
| DE102006053700.9 | 2006-11-13 | ||
| DE102006053699 | 2006-11-13 | ||
| DE102006053700 | 2006-11-13 | ||
| DE102007040676.4 | 2007-08-29 | ||
| DE102007040676A DE102007040676A1 (en) | 2006-11-13 | 2007-08-29 | Optimized storage and access procedure in a history memory of an automation system |
| PCT/EP2007/009554 WO2008058647A1 (en) | 2006-11-13 | 2007-11-05 | Optimized storage and access method for a historian server of an automated system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101622584A CN101622584A (en) | 2010-01-06 |
| CN101622584B true CN101622584B (en) | 2012-11-07 |
Family
ID=41514929
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007800421189A Active CN101622584B (en) | 2006-11-13 | 2007-11-05 | Optimized storage and access method for a historian server of an automated system |
| CN2007800461025A Active CN101657768B (en) | 2006-11-13 | 2007-11-05 | System and method for the loss-free processing of process values of a technical installation or a technical process |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007800461025A Active CN101657768B (en) | 2006-11-13 | 2007-11-05 | System and method for the loss-free processing of process values of a technical installation or a technical process |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN101622584B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9958863B2 (en) * | 2012-10-31 | 2018-05-01 | General Electric Company | Method, system, and device for monitoring operations of a system asset |
-
2007
- 2007-11-05 CN CN2007800421189A patent/CN101622584B/en active Active
- 2007-11-05 CN CN2007800461025A patent/CN101657768B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN101622584A (en) | 2010-01-06 |
| CN101657768A (en) | 2010-02-24 |
| CN101657768B (en) | 2013-01-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104965850B (en) | A kind of database high availability implementation method based on open source technology | |
| CN100571281C (en) | Great magnitude of data hierarchical storage method | |
| US20160292255A1 (en) | Hybrid data management system and method for managing large, varying datasets | |
| CN106502587B (en) | Hard disk data management method and hard disk control device | |
| CN101183264B (en) | Real-time process history server | |
| CN101673192B (en) | Method for time-sequence data processing, device and system therefor | |
| NO326041B1 (en) | Procedure for managing data storage in a system for searching and retrieving information | |
| CN101496012A (en) | Data processing over very large databases | |
| CN101288074A (en) | Moving data from file on storage volume to alternate location to free space | |
| CN101535964B (en) | Ghosted file | |
| CN105005617A (en) | Storage method and device of time sequence data | |
| CN102346695A (en) | Scalable segment-based data de-duplication system and method for incremental backups | |
| CN102197372A (en) | Partition management in a partitioned, scalable, and available structured storage | |
| CN104067281A (en) | Clustering event data by multiple time dimensions | |
| CN105144142A (en) | Object storage using multiple dimensions of object information | |
| CN102637142A (en) | Computer system and method for realizing log management | |
| CN109391629A (en) | Track traffic synthetic monitoring system data processing method | |
| US10360189B2 (en) | Data object storage across multiple storage nodes | |
| CN102163231A (en) | Method and device for data collection | |
| CN110336863A (en) | A kind of data reporting method and system | |
| CN108139904A (en) | Writing commands filter | |
| CN1996305A (en) | Data saving and reading method and device therefore, and data transmission system | |
| CN110287152A (en) | A kind of method and relevant apparatus of data management | |
| CN116842055A (en) | System and method for integrated processing of internet of things data batch flow | |
| CN110858210A (en) | Data query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20180521 Address after: Baden, Switzerland Patentee after: ABB TECHNOLOGY LTD. Address before: Zurich Patentee before: ABB Technology AG |