CN103777950A - Gridding method for resolving AOS (Advanced Orbiting System) telemetering data - Google Patents

Abstract

The invention discloses a gridding method for resolving AOS (Advanced Orbiting System) telemetering data. In the method, a centralized data resolving process is split into independent service flows, and data of different virtual channels are transmitted to different VCDU (Virtual Channel Data Unit) resolving server nodes according to the virtual channel characteristic in an AOS, so that a serial process is converted into a parallel processing process. The method comprises the following steps: I, designing a service interface by using an ICE (Internet Communications Engine); II, implementing the interface by using C++; III, dynamically allocating a scheduling algorithm; IV, loading a balance algorithm. By adopting the method, parallel processing of the AOS telemetering data can be realized, and dynamic task distribution scheduling and load balance of can be performed according to the characteristic of a VCDU data format.

Description

A kind of gridding method that AOS telemetry is resolved

Technical field

The present invention relates to a kind of gridding method that AOS telemetry is resolved, it is a kind of method that spacecraft AOS telemetry is carried out to grid processing.Belong to grid computing and satellite telemetering data process field.

Background technology

Modern Satellite develops constantly to future developments such as deep space, two-forty and high reliability.In order to adapt to the development of space technology, it is stronger that consultative committee for space data system (CCSDS) has proposed again dirigibility on the basis that proposes conventional Packet TC & TM system, more diversified data processing business---high data system in-orbit, is called for short high system in-orbit (AOS).Payload data and the unification of spacecraft platform data are formed uniform data stream (VCDU of virtual channel data unit unit) by this system.AOS Telemetering Data Processing adopts centralized configuration traditionally.In the time that remote measurement downstream rate is very fast, there is the bottleneck in performance in this structure, there will be the phenomenon such as frame losing, obstruction.

Grid is an integrated calculating and resource environment, in other words conj.or perhaps a computational resource pond.Grid can fully receive various computational resources, and when they are converted into a kind of that be available anywhere, reliable, standard or economic computing power.Grid computing not simply refers to different computer equipments to couple together by physical network, but a kind of organic combination of all computational resources.

For AOS telemetry format characteristic, design a kind of remote measurement resolution system based on grid, can realize AOS remote measurement and resolve parallelization, count pipe test macro for spacecraft a set of New methods in working is provided.There is significant contribution for promoting China's aerospace industry intellectuality and synthesization development!

Summary of the invention

(1) technical matters that will solve

Traditional AOS telemetry is resolved and is adopted centralized configuration, resolves software by a computer run, and this software is responsible for all parsing tasks.The object of the invention is to overcome the limitation of traditional centralized analytic method, solve to process when high downstream rate data and block, packet loss, takies the problems such as computer resource is excessive.

(2) technical scheme of dealing with problems

In order to realize described object, the present invention has designed a kind of grid processing method for AOS telemetry feature.AOS telemetry resolving comprises Channel Access Data Unit unit (CADU), virtual channel data unit unit (VCDU), multichannel protocol Data Unit (MPDU) and source bag (EPDU) four kinds of forms.CADU is the most original data that receive; VCDU, is resolved by CADU, and VCDU can have at most 64 pseudo channels; MPDU is resolved by VCDU; EPDU is resolved by MPDU.The core of the method is that concentrated Data Analysis process is split into independently service procedure, according to pseudo channel characteristic in AOS, thereby send to different VCDU analysis service node processing that series process is converted to parallel processing process the data of different virtual channel.The structure of the method is as Fig. 1.

A kind of gridding method that AOS telemetry is resolved of the present invention, the method concrete steps are as follows:

Step 1: utilize ICE design service interface: ICE(Internet Communications Engine) be the OO middleware of ZeroC company and GNU GPL and patent grant mandate, it provides OO far call, grid computing and publish/subscribe mechanism; ICE may operate in most of mainstream operation systems, as Linux, and Solaris, Windows and Mac OS X support C++ simultaneously, Java .NET language (as C# or Visual Basic), Objective-C, Python, PHP and Ruby; ICE application program can be deployed under multiple environment, and client and service end can be used different programming languages to realize; Simultaneously it may operate on different operating system platforms and different hardware structure, effectively solves C/S model poor compatibility, the problems such as cost of development height; Adopt the flow process of ICE developing network communications applications as Fig. 2;

The present invention is in order to process the telemetry of high-speed downstream, adopt producer consumer model to realize a kind of asynchronous communication mode, analysis service all comprises a buffer queue with the service that distributes and is responsible for from queue, taking out data and processing with reception data and a processing threads; The OO feature of general character based on all services and ICE, design service base class interface BaseDecoder, it thes contents are as follows:

CADU analysis service interface is as follows:

It is as follows that VCDU distributes service interface:

VCDU analysis service interface is as follows:

；

Step 2: utilize C++ to realize interface: the slice compiler providing by ICE can be compiled as interface document the interface document of specific development language, consider the factors such as efficiency performance, the present invention adopts C++ as development language, C++ file after the compiling of slice interface document is abstract class, therefore needs to write the subclass of realization;

Step 3: dynamic assignment dispatching algorithm: task assignment is the core technology of grid computing, is also the basis of grid application, and Task Scheduling Model and algorithm can make full use of the hardware and software resource of grid system efficiently, thereby improves the performance of grid system; Gridding task scheduling problem has been proved to be a np complete problem, has now become a research focus in current grid computing field; For distributed environment, the change of node is common scene, such as cause recovery, the interpolation of node disappearance or node due to reasons such as mechanical disorders, this has caused certain difficulty to the assignment of distributed task scheduling, must be able to realize a kind of reliable and efficient dynamic task dispatching algorithm, the variation of node in adaptive mess automatically; Conventional a kind of technology is called DHT(Distributed Hash Table, distributed hash table); DHT technology is mainly used in P2P scene, is mainly the communication between peer nodes in this scene, and the scene that this and the present invention study has larger difference; The transformation from serial to parallel that CCSDS remote measurement is resolved is the work that is converted to this one deck of VCDU at CADU, according to different vcid, data being sent to different computer nodes resolves, but do not have communication requirement between the node of operation VCDU analysis service, data transmission is mainly to carry out between allocator and VCDU analysis service, is the model of one-to-many; But DHT still has certain reference, and under distributed environment, if the variation of dynamically adapting number of nodes, Hash mapping is undoubtedly a kind of desirable way; Simultaneously in the communication aspects of processing one-to-many model, the selection that Observer Pattern can yet be regarded as, therefore in order to solve dynamic assignment problem, the present invention has taked Hash delivery and the way that observer's Design Mode combines, and distributes the uml diagram of service and VCDU analysis service as Fig. 3;

Step 4: load-balancing algorithm: the flow control that distributes the load balancing control principle of service to VCDU analysis service and Transmission Control Protocol is similar, adopts the mode of moving window; VCDU analysis service is inherited from BaseDecoder interface, the service of distributing can be called the GetCacheSize of analysis service and the maximum amount of data that GetCacheMaxSize function obtains the current data amount in VCDU analysis service buffer zone and can hold, and determines to send to the data volume of analysis service according to the difference of the two.

(3) advantage of the present invention and effect:

Adopt method of the present invention, can realize the parallel processing of AOS telemetry, and can according to the feature of VCDU data layout carry out dynamic task distribute scheduling and load balancing.

Accompanying drawing explanation:

Figure 1A OS telemetry is resolved grid system overall architecture

Fig. 2 adopts ICE middle unit development web application process flow diagram

Fig. 3 distributes service and VCDU analysis service uml diagram

Fig. 4 registers analysis service function works process flow diagram

Fig. 5 function process flow diagram that remaps

Fig. 6 removes analysis service function process flow diagram

Fig. 7 distributes service load balancing schematic diagram of mechanism

Embodiment

A kind of gridding method that AOS telemetry is resolved of the present invention, as shown in Figure 1, the method concrete steps are as follows:

1 utilizes ICE design service interface

ICE(Internet Communications Engine) be the OO middleware of ZeroC company and GNU GPL and patent grant mandate.It provides OO far call, grid computing and publish/subscribe mechanism.ICE may operate in most of mainstream operation systems, as Linux, Solaris, Windows and Mac OS X support C++ simultaneously, Java .NET language (as C# or Visual Basic), Objective ?C, Python, PHP and Ruby.ICE application program can be deployed under multiple environment, and client and service end can be used different programming languages to realize; Simultaneously it may operate on different operating system platforms and different hardware structure, effectively solves C/S model poor compatibility, the problems such as cost of development height.

Adopt the flow process of ICE developing network communications applications as Fig. 2

The present invention, in order to process the telemetry of high-speed downstream, adopts producer consumer model to realize a kind of asynchronous communication mode.Analysis service all comprises a buffer queue with the service that distributes and is responsible for from queue, taking out data and processing with reception data and a processing threads.The OO feature of general character based on all services and ICE, design service base class interface BaseDecoder, it thes contents are as follows:

CADU analysis service interface is as follows:

It is as follows that VCDU distributes service interface:

VCDU analysis service interface is as follows:

2 utilize C++ to realize interface

The slice compiler providing by ICE can be compiled as interface document the interface document of specific development language.Consider the factors such as efficiency performance, the present invention adopts C++ as development language, and the C++ file after the compiling of slice interface document is abstract class, therefore needs to write the subclass of realization.

3 dynamic assignment dispatching algorithms

Task assignment is the core technology of grid computing, is also the basis of grid application.Task Scheduling Model and algorithm can make full use of the hardware and software resource of grid system efficiently, thereby improve the performance of grid system.Gridding task scheduling problem has been proved to be a np complete problem, has now become a research focus in current grid computing field.For distributed environment, the change of node is common scene, such as cause recovery, the interpolation of node disappearance or node due to reasons such as mechanical disorders, this has caused certain difficulty to the assignment of distributed task scheduling, must be able to realize a kind of reliable and efficient dynamic task dispatching algorithm, the variation of node in adaptive mess automatically.Conventional a kind of technology is called DHT(Distributed Hash Table, distributed hash table).DHT technology is mainly used in P2P scene.In this scene, be mainly the communication between peer nodes, the scene that this and the present invention study has larger difference.The transformation from serial to parallel that CCSDS remote measurement is resolved is the work that is converted to this one deck of VCDU at CADU, according to different vcid, data being sent to different computer nodes resolves, but do not have communication requirement between the node of operation VCDU analysis service, data transmission is mainly to carry out between allocator and VCDU analysis service, is the model of one-to-many.But DHT still has certain reference, and under distributed environment, if the variation of dynamically adapting number of nodes, Hash mapping is undoubtedly a kind of desirable way.Simultaneously in the communication aspects of processing one-to-many model, the selection that Observer Pattern can yet be regarded as.Therefore take Hash delivery and the way that observer's Design Mode combines in order to solve the present invention of dynamic assignment problem, distributed the uml diagram of service and VCDU analysis service as Fig. 3.

Dispatcher and VcduDecoder adopt the mode of the combination of class to realize coupling.First, VcduDecoder member variable and member function are described:

NodeId: integer value, represent the unique identification of VcduDecoder, be responsible for distribution by Dispatcher.

LoadNum: integer value, represents that VcduDecoder is responsible for the quantity of the vcid resolving

Dispatcher:Dispatcher type.For each VcduDecoder example, what Dispatcher played is mainly the effect that distributes task, and therefore variable is named as dispatcher.

Void decode (ByteArray vcdu): the main interface function of VcduDecoder, be responsible for the parsing of vcdu data.

Void setId (int id): for nodeId is set, be responsible for calling by Dispatcher

Int getId (): for obtaining nodeId, provide Dispatcher to obtain the interface of nodeId.

Secondly, member variable and member function to Dispatcher describe:

VdMap is stl::map<int, VcduDecoder> type, and its key is integer value, is worth the type for VcduDecoder, key table shows the nodeId of vcduDecoder object.

NvMap is stl::map<int, vector<int>> type, and its key represents the id of VcduDecoder, value represents all No. vcid that this VcduDecoder is responsible for resolving.

Vid2nid: be simple integer array, length is 64, represents 64 virtual channel identifiers.Value in array represents the nodeId of corresponding VcduDecoder.Utilize these mapping relations can within the time of O (1), locate No. id of analysis service that each pseudo channel is corresponding.

VdCount: integer value, records current just in the quantity of operating VcduDecoder.

VdMaxCount: integer value, the quantity of the VcduDecoder maximum that record occurred so far.

Void registDecoder (VcduDecoder vd): VcduDecoder is registered in vdMap, is called by the constructed fuction of VcduDecoder.

Void removeDecoder (VcduDecoder vd): VcduDecoder is removed from vdMap, and it is retained to get up for corresponding No. nodeId, add after coming in and redistribute this id at new node.

Void rehash (): in the time that VcduDecoder node increases or reduce, recalculate nvMap and vidnid, guarantee that different virtual channel data can be evenly distributed to each node and guarantee that original structure is not destroyed as far as possible.

When each VCDU analysis service starts, the registDecoder function that can call the member variable of its inner Dispatcher type will oneself be registered in the mapping relations of Dispatcher.The workflow of registDecoder function is as Fig. 4.

In node ID pond, preserve deleted VCDU analysis service No. ID.In the time that a new service node is registered into, preferentially in pond, choose No. ID and distribute to new service.If Chi Weikong, produces in order and new distribute to new service node No. ID, vdMaxCount value adds one simultaneously, represents that analysis service number of nodes has increased, and is that the ID that new node is set is the value of new generation afterwards equally.Next new ID and service object's corresponding relation to be joined in vdMap mapping.Finally call rehash function and recalculate mapping relations.

Rehash function mainly upgrades the mapping relations of vid2nid and nvMap preservation according to member variables such as vdMap, vdMaxCount, vdCount.The flow process of Rehash function can represent as Fig. 5.

This function is only called in the time of new node registration or old endpoint unregistration, and therefore variation has necessarily occurred vdMap.Function compares the key of vdMap and nvMap in the time starting, and the quantity that can find node is to increase or reduce.If node increase, recalculates the mapping relations of vid2nid according to the value of vdMaxCount, if node reduces, from nvMap, take out the relation pair of the vcid list that the node that is removed is corresponding with it, vcid in list is hashed on other service nodes again, recalculate vid2nid.Finally upgrade the content of nvMap according to vid2nid.

RemoveDecoder function VcduDecoder service analyse structure process in call, can feed back in time with the revocation information that guarantees service node the service of distributing.Its overall flow and registDecoder function class seemingly, as Fig. 6.

Dispatch function is responsible for distributing of VCDU, first obtain VCID, find node ID by vid2nid array, find out corresponding VcduDecoder service object by nvMap again, the benefit of locating by two-stage mapping relations be both can dynamic adaptation service node quantity variation, also can guarantee that original mapping relations are not destroyed as far as possible.Primary mapping time complexity is O (1), and the STL Map data structure that mapping adopts is for the second time RBTree, and therefore time complexity is O (logn), and wherein n is service node quantity.Therefore total time complexity is O (logn).

4 load-balancing algorithms

The flow control that distributes the load balancing control principle of service to VCDU analysis service and Transmission Control Protocol is similar, adopts the mode of moving window.VCDU analysis service is inherited from BaseDecoder interface, the service of distributing can be called the GetCacheSize of analysis service and the maximum amount of data that GetCacheMaxSize function obtains the current data amount in VCDU analysis service buffer zone and can hold, and determines to send to the data volume of analysis service according to the difference of the two.Distributing service load balancing mechanism can represent as Fig. 7.

Test result

Respectively 1 to 6 VCDU analysis service node is tested in the situation of different remote measurement downstream rates, the performance index of test comprise throughput, CPU usage and memory usage.The wherein ratio of the frame number of VCDU analysis service processing per second and the frame number of reception for throughput.Test result is as shown in table 1, and the performance parameter value in table is the mean value of each node in grid.

Under phase same rate, the quantity of VCDU analysis service node is more as can be seen from Table 1, and the performance performance of each node is better, and in the time that VCDU analysis service number of nodes is identical with pseudo channel VCID quantity, performance reaches optimum condition.When transfer rate is during lower than 1MB/s, various number of nodes situations all can normally be worked, but in the time that speed is greater than 1MB/s, single node is difficult to competent at a job, it is ultimate attainment that throughput and memory usage have all reached, and occurred comparatively significantly choking phenomenon, and the situation that takies of CPU is also comparatively obvious simultaneously.Throughput is higher, and memory usage is lower, and CPU usage is also corresponding lower, and this is because high-throughput explanation node can efficiently be processed the data that receive in real time, there will not be internal memory to overstock phenomenon, and CPU also can have surplus energy to be located in understanding to analyse task.

Invention conclusion

Grid computing is applied to AOS telemetry parsing aspect, can solves the performance issue that traditional centralized processing method faces in the situation that telling downstream rate.Along with the increase of number of nodes, the handling capacity of individual node strengthens, and cpu utilization rate and memory usage decline to some extent, and occupation condition is optimized.Meanwhile, the extensibility and the reliability that dynamically distribute dispatching algorithm and load-balancing algorithm and can strengthen grid system for AOS telemetry feature of the present invention design.Therefore, the method has feasibility and validity.

Claims (1)

1. the gridding method that AOS telemetry is resolved, it is characterized in that: the method is that concentrated Data Analysis process is split into independently service procedure, according to pseudo channel characteristic in AOS, thereby send to different VCDU analysis service node processing that series process is converted to parallel processing process the data of different virtual channel, its concrete steps are as follows:

Step 1: utilize ICE design service interface: this ICE is OO middleware, and it provides OO far call, grid computing and publish/subscribe mechanism; ICE can operate in most of mainstream operation systems, supports C++ simultaneously, Java, and .NET language, objective-C, python, pHPwith ruby; ICE application program can be deployed under multiple environment, and client can be used different programming languages to realize with service end; Simultaneously it can operate on different operating system platforms and different hardware structure, effectively solves C/S model poor compatibility, the high problem of cost of development;

In order to process the telemetry of high-speed downstream, adopt producer consumer model to realize a kind of asynchronous communication mode, analysis service all comprises a buffer queue with the service that distributes and is responsible for from queue, taking out data and processing with reception data and a processing threads; The OO feature of general character based on all services and ICE, design service base class interface BaseDecoder, it thes contents are as follows:

CADU analysis service interface is as follows:

It is as follows that VCDU distributes service interface:

VCDU analysis service interface is as follows:

；

Step 2: utilize C++ to realize interface: the slice compiler providing by ICE can be compiled as interface document the interface document of specific development language, consider the factors such as efficiency performance, adopt C++ as development language, C++ file after the compiling of slice interface document is abstract class, therefore needs to write the subclass of realization;

Step 3: dynamic assignment dispatching algorithm: task assignment is the core technology of grid computing, is also the basis of grid application, and Task Scheduling Model and algorithm can make full use of the hardware and software resource of grid system efficiently, thereby improves the performance of grid system; Gridding task scheduling problem has been proved to be a np complete problem, has now become a research focus in current grid computing field; For distributed environment, the change of node is common scene, such as cause recovery, the interpolation of node disappearance or node due to mechanical disorder reason, this has caused certain difficulty to the assignment of distributed task scheduling, must be able to realize a kind of reliable and efficient dynamic task dispatching algorithm, the variation of node in adaptive mess automatically; Conventional a kind of technology is called DHT distributed hash table; This DHT technology is applied to P2P scene, is mainly the communication between peer nodes in this scene, and the scene of this and this research has larger difference; The transformation from serial to parallel that CCSDS remote measurement is resolved is the work that is converted to this one deck of VCDU at CADU, according to different vcid, data being sent to different computer nodes resolves, but do not have communication requirement between the node of operation VCDU analysis service, data transmission is mainly to carry out between allocator and VCDU analysis service, is the model of one-to-many; But DHT still has reference, and under distributed environment, if the variation of dynamically adapting number of nodes, Hash mapping is a kind of desirable way; In the communication aspects of processing one-to-many model, the selection that Observer Pattern has been, therefore in order to solve dynamic assignment problem, has taked Hash delivery and the way that observer's Design Mode combines simultaneously, distributes the uml diagram of service and VCDU analysis service;

Step 4: load-balancing algorithm: the flow control that distributes the load balancing control principle of service to VCDU analysis service and Transmission Control Protocol is similar, adopts the mode of moving window; VCDU analysis service is inherited from BaseDecoder interface, the service of distributing can be called the GetCacheSize of analysis service and the maximum amount of data that GetCacheMaxSize function obtains the current data amount in VCDU analysis service buffer zone and can hold, and determines to send to the data volume of analysis service according to the difference of the two.

Images