A kind of transfer control method of real-time monitoring system data processing
Technical field
The present invention relates to a kind of transfer control method of data processing, especially relate to the transfer control method that a kind of real-time monitoring system interface data is handled, belong to the computer control field.
Background technology
In real-time monitoring system, the interface data processing module is key and the parts that may produce bottleneck as the bridge of contact inside and outside data.Along with the continuous development of the communication technology and computer monitoring technology, the integrated level of real-time monitoring system is more and more higher in the every profession and trade, and scale constantly enlarges, and it is complicated that real-time requires also more to be tending towards.As data uplink descending must be by the path, the pressure of data processing module also rises with geometric progression.Traditional real-time monitoring system data processing module, do not consider the difference of disposal ability between data acquisition unit and the application processing unit, and do not require different data to treat with a certain discrimination to real-time, blocking when taking place does not have rational measure to remedy yet, when emergency case occurring, stability and reliability all can not guarantee.
Summary of the invention
Technical problem to be solved by this invention is under the situation that does not increase the resource input, provide a kind of and can effectively avoid using processing unit and take place congestedly, take into account the transfer control method of the real-time monitoring system data processing of the real-time of data and reliability simultaneously.
For solving the problems of the technologies described above, the invention provides a kind of transfer control method of real-time monitoring system data processing, it is characterized in that: may further comprise the steps
1) sets up the priority transmit queue, limit the transmission per unit of time upper limit;
2) dynamically adjust the column rank preface by data time-delays different in the formations at different levels;
3) when the information snowslide causes formation to be blocked, preserve significant data by the mode that data are temporary, and retransfer at one's leisure.
Set up the priority transmit queue and can adopt following method:
At first, need all kinds of different pieces of informations of real-time monitoring system collection are carried out unified planning, dissimilar delta data cell abstracts is unified " delta data object ", be designated as OC, OC comprises following attribute at least as can be known: priority tag (PRI_LEVEL), time-delay be sign (DT_AVAIL), system-key (KEY_ID), data type (TYPE), codomain (VALUE), status field (STATUS), transformation period (CHG_TIME) etc. effectively.Wherein OC.PRI_LEVEL and OC.DT_AVAIL are the distinctive controlled flag of the present invention, and all the other attributes are general real-time monitoring system platform data description.
Secondly, by the real-time of image data, the priority that the definition image data is accepted processing is as follows according to system:
PRI_LEVEL_CRUCIAL, most important level is used to transmit the data that must meet with a response immediately, and real-time requires the highest, in real-time monitoring system, is called the insertion transport level again;
PRI_LEVEL_IMPORTANT, importance level, real-time is had relatively high expectations, and data can not lose, but allows of short duration time-delay, and the size by time-delay can be subdivided into I, II, III level etc. again;
PRI_LEVEL_NORMAL, regular grade, real-time requires minimum, allows certain time-delay.
According to the specific requirement of above-described priority and real-time monitoring system image data, set up one group of fifo queue, be generally the 3-5 bar, be designated as S, A[, B, C], N.Wherein the S formation is called insertion transmission queue, and the A formation is called one-level significant data formation (the B formation is called the formation of secondary significant data, and the C formation is called three grades of significant data formations), and the N formation is called the general data formation.
Congested unit interval limit value flow F may occur according to system scale, operation characteristic and application end for process, estimate the reasonable length Li of formations at different levels.By the real-time monitoring system characteristics, system does not produce congested maximum data flow F and is directly proportional with systematic function P in unit interval, be inversely proportional to integrated scale M and the busy extent B of system at that time, it is as follows to get formula: F=(λ P)/(μ MB), λ, μ are proportionality coefficient.After putting into operation in the real-time monitoring system scene, its system configuration and integrated scale are also definite substantially, so P, M can be similar to and think constant value.And under the situation of cpu load less than threshold values (by general design of Real-time Monitor System requirement), B is desirable to be worth 1 approximately, and F is a constant in the system that determines as can be known.
As from the foregoing, the maximum allowable delay of establishing delta data object among the formation i is Δ i, the reasonable length Li evaluation method Li=F* Δ i of formations at different levels is arranged, wherein i=S, A[, B, C], N.
The order of falling out of priority queries at different levels is described below:
At first allow most important level data to fall out.Being located at the maximum data flow that can fall out in the unit interval is F, and the data volume of insertion transmission queue current cache is FC, if FC>F then gets F to fall out, and returns and waits for next blanking time, otherwise calculate the residue amount of falling out F Δ=F-FC;
Next allows the importance level data to fall out.If have 3 grades of segmentation priority altogether, be that it distributes different transmission weight, β, γ, make alpha+beta+γ=1.If the data volume of each subfractionation significant data formation current cache is respectively FI, FII, FIII, if (FI>=α * F Δ) ﹠amp is arranged; ﹠amp; (FII>=β * F Δ) ﹠amp; ﹠amp; (FIII>=γ * F Δ) then falls out by maximum, and returns and wait for next blanking time; Otherwise the height according to segmentation priority is got the space-variant data object successively, and reaching F Δ or each subfractionation significant data formation up to summation all be sky; If the amount of falling out is full, then returns and wait for next blanking time, otherwise calculate the residue amount of falling out F Δ ';
Allow the regular grade data to fall out at last, the data volume of note general data formation current cache is FN, if FN>F Δ ', then get the F Δ ' fall out, and return and wait for next blanking time; Fall out otherwise get FN, simultaneously to idle condition message of system feedback, and idle amount S Δ=(the F Δ '-FN), return and wait for next blanking time.
Use the token mechanism in the processing unit end processing mode,, determined when load arrives to a certain degree, can trigger the downslide significantly of treatment effeciency, thereby produced congested as the locking and de-locking of real-time storehouse memory table etc.By above step real-time monitoring system is operated in below the critical condition, prevent to use the congested generation of processing unit.Hierarchical transmission has guaranteed the prioritised transmission power of significant data simultaneously, has improved data validity.But when the accident duration causes taking place the transmit queue obstruction too greatly than long or bursty data amount,, must carry out load and come off, diplomatic abandoning or temporal data for guaranteeing the reliability of system.
In real-time monitoring system, some data are extremely important in its term of validity, and it is also high that real-time requires, but because its refreshing frequency is very fast, change more frequently, and inefficacy speed is also fast; Some data itself have the markers when taking place, though the real-time requirement is not necessarily very harsh, its information relationship must all keep to the integrality of system data.Last class data are called " milk " data (MILK_DATA), and back one class data are called " grape wine " data (WINE_DATA).The feature of MILK_DATA is to get that it is fresh, expired rottenly then abandon; The feature of WINE_DATA is permanently effective, more Chen Yuexiang.
The method that load provided by the invention comes off is as follows:
At first, when each delta data object is fallen in lines, need prejudge corresponding priority query and whether overflow,, then think to block and take place if be true;
When blocking generation, stop the regular grade data and fall in lines, most important level and importance level data can not continue to fall in lines under overflowing situation in priority query at the corresponding levels;
When blocking generation, need insert the delta data object and the regular grade data object of the formation of having overflowed and should keep in or abandon according to definite its of the effective sign of time-delay, MILK_DATA or WINE_DATA feature by data when time-delay effectively is identified at system modelling identify;
The temporary library of temporary delta data object has following characteristics: can adopt temporary file mode or lightweight database (as sqlite etc.) mode to realize; Can divide file or table to store according to the difference of delta data object priority, also can cancel data priority difference storage and uniform; Have suitable memory space, guarantee in the system design capacity, can not overflow.
In the obstruction process, judge at any time whether system has entered idle condition, if be judged as very, then from temporary library, take out the delta data object and directly send according to idle amount.Judge that the method whether system has entered idle condition is: obtain idle condition message and idle amount S Δ that the process of falling out sends, and S Δ>0 is arranged, then be judged as true.
The beneficial effect that the present invention reached:
The transfer control method of real-time monitoring system data processing of the present invention is avoided using end for process by the qualification transmission upper limit and is produced congested; By the priority query and the control strategy of falling out, guarantee real-time property; By the temporary technology of data and the control strategy of falling in lines, guarantee the integrality of data, highly versatile is not subjected to the restriction of operating system version and implementation tool, and independence is strong, but is added on easily after the modularization in the existing real-time monitoring system.
Fig. 1 is the data flow diagram of real-time monitoring system data processing of the present invention;
Fig. 2 is transmitting control data flow figure of the present invention;
Fig. 3 is delta data object structure figure of the present invention;
Fig. 4 is a priority query of the present invention schematic diagram;
Fig. 5 is the control flow chart of falling in lines of the present invention;
Fig. 6 is the control flow chart of falling out of the present invention.
Embodiment
Also in conjunction with the embodiments technical scheme of the present invention is further elaborated according to Figure of description below.
Fig. 1 is the data flow diagram of real-time monitoring system data processing, and signal the present invention has increased transmission control unit on the basis of traditional data processing module structure, be the output of the constant upper limit with at random input shaper.
Fig. 2 is transmitting control data flow figure of the present invention, illustrates the basic structure and the data flow of transfer control method of the present invention.This method is made of two kinds of basic data structures and two control strategies, is respectively delta data object, priority query and the control strategy of falling in lines, the control strategy of falling out.
Fig. 3 is delta data object structure figure of the present invention, and the attribute of having described the delta data object constitutes.The delta data object comprises following attribute at least: priority tag (PRI_LEVEL), time-delay be sign (DT_AVAIL), system-key (KEY_ID), data type (TYPE), codomain (VALUE), status field (STATUS), transformation period (CHG_TIME) etc. effectively.Wherein priority tag and time-delay effectively identify and are called controlled attribute, and example value derives from the system model parameter; All the other attributes are called data attribute, and wherein system-key and data type example value derive from model parameter, the remaining field apparatus that derives from.
Fig. 4 is priority query's schematic diagram of the present invention's definition, and its grade classification is as follows:
■ PRI_LEVEL_CRUCIAL, most important level is used to transmit the data that must meet with a response immediately, and real-time requires the highest, in real-time monitoring system, is called the insertion transport level again;
■ PRI_LEVEL_IMPORTANT, importance level, real-time is had relatively high expectations, and data can not lose, but allows of short duration time-delay, and the size by time-delay can be subdivided into I, II, III level etc. again;
■ PRI_LEVEL_NORMAL, regular grade, real-time requires minimum, allows certain time-delay.
Below be that the preferred embodiments of the present invention are illustrated with the track traffic synthetic monitoring system.The data of its collection can be divided by following rule:
■ and driving and the relevant data of passenger's personal safety guarantee as fire alarm and pattern startup, electric power system dc circuit breaker fault or important protection action, the important fault of signal system etc., must be incorporated into the pass importance level;
■ and monitoring of tools and safeguard relevant data as the fault-signal that returns letter point, critical system equipment of some controls etc., can be inserted importance level I level;
Digital quantity, analog quantity input that ■ is real-time can be divided by its importance, insert importance level II, III level respectively;
■ is to some data quasi real time, and collection period is above at tens of seconds, as the information such as humiture that have nothing to do with pattern control, can insert regular grade.
As from the foregoing, this system is fit to set up 5 grades priority query, and the data type of its collection point mainly is analog quantity and digital quantity, can use the delta data object factory.By formula F=(λ P)/(μ MB) as can be known, according to system configuration and integrated scale, and the characteristics of track traffic synthetic monitoring system, constituting the stage in system can estimate and not produce congested maximum application data processing amount F, and derive rational queue length Li=F* Δ i thus, wherein Δ i is the maximum allowable delay of delta data object among the formation i, i=S, A[, B, C], N.And the data structure of element is formed the three elements of priority query just in priority division, queue length and the formation.
Fig. 5 is the realization flow figure of the control strategy of falling in lines of the present invention.The control strategy module of falling in lines is by independently thread or process realization, and its handling process is described below:
1.. the periodic scan passage, check whether there is the delta data unit;
2.. if exist, then obtain a delta data unit,, from model, get parms and organize delta data object OC according to its system-key;
3.. by system congestion plug sign, judge current whether being in the obstruction, if, carry out 4., otherwise, carry out 5.;
4.. judge whether OC.PRI_LEVEL equals PRI_LEVEL_NORMAL, if, carry out 8., otherwise, carry out 5.;
5.. by interface interchange, judge whether the formation of OC.PRI_LEVEL level overflows, if, carry out 7., otherwise, carry out 6.;
6.. OC is inserted in the formation of OC.PRI_LEVEL level, return 1.;
7.. the system congestion sign is changed to very, carries out 8.;
8.. judge whether OC.DT_AVAIL is true, if, carry out 10., otherwise, carry out 9.;
9.. OC is abandoned, return 1.;
10.. OC is preserved in the temporary library, return 1..
Fig. 6 is the realization flow figure of the control strategy of falling out of the present invention.The control strategy module of falling out by two independently thread or process realize, realize respectively from formation, reading and from temporary library read functions, intercom mutually by the platform messaging bus between the two.Its handling process is described below:
Reading and sending from formation:
1.. read and insert transmission queue's current cache delta data amount, be designated as FC;
2.. judge whether FC does not produce congested maximum data flow F greater than system, if, carry out 3., otherwise, carry out 4.;
3.. from insert transmission queue, take out F and send to the application processing unit, return and 1. wait for next time slot;
4.. from insert transmission queue, take out FC and send to the application processing unit, but calculate residue traffic volume F Δ=F-FC, carry out 5.;
5.. read respectively from significant data formations at different levels by preset ratio α, β, γ and to be limited to α * F Δ, β * F Δ, γ * F Δ, judge whether read total amount reaches the F Δ.If, carry out 6., otherwise, carry out 7.;
6.. the F Δ that takes out is sent to the application processing unit, return and 1. wait for next time slot;
7.. read sky one-level, secondary, three grades of significant data formations successively and reach the F Δ until summation, if, carry out 6., otherwise, carry out 8.;
8.. read the formation of sky general data and reach the F Δ until summation, if, carry out 6., otherwise, carry out 9.;
9.. the data of taking out are sent to the application processing unit, and computation-free amount S Δ has S Δ>0, carries out 10.;
10.. transmitting system idle message in dedicated tunnel, return and 1. wait for next time slot;
Reading and sending from temporary library:
1.. receiving system idle message from dedicated tunnel;
2.. judged whether that new information produces, if, carry out 3., otherwise, return 1.;
3.. whether the message that verification receives is effective, if, carry out 4., otherwise, return 1.;
4.. according to system's idle message, judge whether correspondence table (file) lining in the temporary library has record, if, carry out 5., otherwise, return 1.;
5.. reading and recording and send to the application processing unit from temporary library, return 1.;
The present invention is illustrated according to the preferred embodiment, should be appreciated that the foregoing description does not limit the present invention in any form, and all employings are equal to the technical scheme that form obtained of replacement or equivalent transformation, all drop within protection scope of the present invention.