CN101515863B - Network data acquiring method, acquisition machine and acquisition system - Google Patents
Network data acquiring method, acquisition machine and acquisition system Download PDFInfo
- Publication number
- CN101515863B CN101515863B CN2008101008240A CN200810100824A CN101515863B CN 101515863 B CN101515863 B CN 101515863B CN 2008101008240 A CN2008101008240 A CN 2008101008240A CN 200810100824 A CN200810100824 A CN 200810100824A CN 101515863 B CN101515863 B CN 101515863B
- Authority
- CN
- China
- Prior art keywords
- data
- layer
- acquisition
- logic control
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Transfer Between Computers (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a network data acquiring method, which comprises the following steps that: after receiving an acquisition policy, a north orientation interface layer of an acquisition machine sends the acquisition policy to a logic control and processing layer in an instruction mode; the acquisition policy is a interface language used for reaching a purpose of controlling and influencing the behavior of acquisition machine at the time of acquiring and managing the data; the logic control and processing layer stores the instruction into a database of the acquisition machine in a task mode; the logic control and processing layer reads the task and performs dispatching according to the task; and a data adaptor layer acquires data according to the dispatching of the logic control and processing layer, and feeds back the acquired data. The invention discloses a network data acquisition machine and a network data acquisition system at the same time. The invention can achieve the aim of acquisition according to need, and can customize acquisition range, acquisition mode, acquisition content, acquisition granularity, and the like of the acquisition machine to control the acquisition action of the acquisition machine and acquire the most needed data so as to reduce the working load of the system.
Description
Technical field
The present invention relates to network technology, particularly a kind of network data acquisition method, harvester and acquisition system.
Background technology
Along with the fast development of information technology and communications industry, the computer networking technology that is based upon on the computer and network technologies basis has obtained swift and violent development, and computer network has entered into the various aspects of people's life.For guaranteeing that network provides better service for people's life, just need the ruuning situation that a cover NMS is obtained network in time, accurately, and occur giving to repair timely and solve when unusual at network.
Network management technology has developed for many years, especially aspect telecommunication network management.Along with the develop rapidly of telecommunication industry, the telecommunication network management technology has also obtained fast development in recent years.The system architecture of telecommunication network management is general divides three layers: acquisition layer, data analysis layer and data exhibiting layer.Acquisition layer is responsible for collection and the preliminary treatment to data; Data analysis layer is responsible for pretreated data are more suitable for using and the various dimensions that represent are handled; The data exhibiting layer is responsible for the demand according to the user, and needed function and appearing the multiple mode of data are provided.
As shown in Figure 1, in the existing technical scheme, acquisition layer generally is divided into three layers: data adaptation layer, data pretreatment layer, data transfer layer.Such acquisition layer implementation can be good at after good by the data collection of guard system and preliminary treatment, being delivered to data analysis layer.
The inventor is through analyzing; Find that present data acquisition is on harvester, to carry out this locality to be configured to accomplish data acquisition; The acquisition layer implementation of prior art; Though can satisfy the basic demand to data collection, preliminary treatment and transmission, program that it can only be through acquisition layer or configuration file are accomplished by the function of the data acquisition of guard system and preliminary treatment and transmission, and the realization of this harvester can not be supported from application interface harvester to be carried out the function to realize gathering alternately; Might gather simultaneously and numerously not be sought after or even unwanted data the live load that the increase system is unnecessary.The concrete deficiency as follows that exists:
1, can not in time change the function or the operational mode of data collection layer, it is said that change etc. like the destination that the preprocessing function to data is made amendment, logarithm send.
2, can not in time check the situation of current data collection, preliminary treatment and transmission.
3, for timely, the of short duration supervision of needs by the situation of guard system index; Present implementation can only be configured or move a module at harvester; When need not keep watch on, stop again or delete configuration, this has brought great inconvenience for the use to system.
4, for the distributed capture system, the current mode of controlling acquisition layer data collection, preliminary treatment and transmission through the local configuration of harvester equally also is inconvenient to use.
Summary of the invention
The embodiment of the invention provides a kind of network data acquisition method, in order to realization network data is gathered as required, and this method comprises:
Harvester northbound interface layer is sent to logic control and processing layer with said acquisition strategies with instruction type after receiving acquisition strategies; Said acquisition strategies is when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used;
Said logic control and processing layer deposit said instruction in the harvester database with the task form;
Said logic control and processing layer read task and dispatch according to task;
The data adaptation layer is carried out data acquisition according to the scheduling of said logic control and processing layer, and the data of feedback collection.
The embodiment of the invention also provides a kind of network data acquisition machine, in order to realization network data is gathered as required, and this network data acquisition machine comprises northbound interface layer, logic control and processing layer, data adaptation layer, also comprises:
The acquisition strategies processing module links to each other with the northbound interface layer, is used for the acquisition strategies that the northbound interface layer receives is sent to logic control and processing layer with instruction type; Said acquisition strategies is when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used;
Command process module links to each other with processing layer with logic control, is used for depositing the said instruction that receives in the harvester database with the task form;
Said logic control and processing layer are dispatched according to the task in the harvester database;
The data adaptation layer is carried out data acquisition according to the task of said logic control and processing layer scheduling, and the data of feedback collection.
The embodiment of the invention also provides a kind of network data acquisition system, network data is gathered as required in order to realizing, comprises the harvester that is made up of northbound interface layer, logic control and processing layer, data adaptation layer, also comprises:
The acquisition management module is used for acquisition strategies is issued to the northbound interface layer of harvester; Said acquisition strategies is when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used;
The acquisition strategies processing module links to each other with the northbound interface layer, is used for the acquisition strategies that the northbound interface layer receives is sent to logic control and processing layer with instruction type;
Command process module links to each other with processing layer with logic control, is used for depositing the said instruction that receives in the harvester database with the task form;
Said logic control and processing layer read task and dispatch according to task;
The data adaptation layer is carried out data acquisition according to the scheduling of said logic control and processing layer, and the data of feedback collection;
The south orientation adaptation module is used to receive the image data that said data adaptation layer is fed back.
In the embodiment of the invention, harvester northbound interface layer is sent to logic control and processing layer with said acquisition strategies with instruction type after receiving acquisition strategies; Said logic control and processing layer deposit said instruction in the harvester database with the task form; Said logic control and processing layer read task and dispatch according to task; The data adaptation layer is carried out data acquisition according to the scheduling of said logic control and processing layer, and the data of feedback collection, has realized a mode and a function through data collection, preliminary treatment and the transmission that represents layer control harvester with this; Abstract with defined data collection, preliminary treatment and load mode and the function of harvester; And realize above-mentioned functions through the interface strategy that definition represents between layer, data analysis layer and the acquisition layer; Reach the purpose of gathering as required, can acquisition range, acquisition mode, collection content and the acquisition granularity etc. of harvester be customized, with the collection behavior of control harvester; The data that acquisition needs most, thereby the live load of minimizing system.
Description of drawings
Fig. 1 is the structural representation of institute's acquisition layer in the background technology;
Fig. 2 concerns sketch map between strategy, task and the adapter in the embodiment of the invention;
Fig. 3 is the implementing procedure figure of network data acquisition method in the embodiment of the invention;
Fig. 4 is for being sent to acquisition strategies the implementing procedure figure of logic control and processing layer in the embodiment of the invention with instruction type;
Fig. 5 will be for instructing the implementing procedure figure that deposits the harvester database with the task form in the embodiment of the invention;
Fig. 6 is the structural representation of network data acquisition machine in the embodiment of the invention;
Fig. 7 is the structural representation of acquisition strategies processing module in the embodiment of the invention;
Fig. 8 is the pictorial diagram of strategy in the embodiment of the invention, container, instruction, this four class object of task;
Fig. 9 is the structural representation of command process module in the embodiment of the invention;
Figure 10 is the structural representation of network data acquisition system in the embodiment of the invention;
Figure 11 is the implementing procedure figure of policy distribution in the embodiment of the invention.
Embodiment
Below in conjunction with Figure of description embodiment of the invention method is elaborated.
At first the notion that relates in the embodiment of the invention is explained as follows:
Acquisition strategies: be when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used.Control the behavior different through different acquisition strategies in the enforcement with influencing harvester; When converting acquisition strategies into acquisition tasks; A strategy can be broken down into a plurality of tasks; A plurality of strategies also can be merged into a task, and the back also will be elaborated to the formulation of strategy, enforcement etc.
Adapter (adapter): being positioned at the data adaptation layer, is the minimum material of carrying out a collecting work in the harvester.General, each adapter is responsible for a concrete collecting work.
Task: handle by logic control and processing layer, be harvester to strategy resolve with handle after, can by the identification of the task scheduling modules of harvester, with the related entity of a concrete task execution module (being adapter).Each task is all related with at least one adapter, and when task of the collaborative completion of a plurality of adapters of needs, a task need be carried out related with a plurality of adapters.Task scheduling modules according to the precedence relationship of association in order or parallel each adapter of scheduling.When task was related with adapter, the task distribution module of harvester can find suitable adapter and scheduling according to " type " field in the task object.
More than three notions be that acquisition strategies, task, adapter all exist with different object form in harvester, the model of oneself is all arranged; In addition, all there is corresponding model conversion module to be responsible for the mappings work between strategy, task, the adaptor model in northbound interface layer and logic control and processing layer.Fig. 2 is the sketch map that concerns between strategy, task and the adapter, and the triadic relation is as shown in the figure.
After understanding above-mentioned notion, implement to be elaborated in the face of the collection of network data down.
As shown in Figure 3, in the embodiment of the invention, can comprise the steps: network data collection
After step 31, harvester northbound interface layer receive acquisition strategies, acquisition strategies is sent to logic control and processing layer with instruction type;
The instruction that step 32, logic control and processing layer will receive deposits the harvester database in the task form;
As shown in Figure 4, among the embodiment, the interface protocol adaptation module of the harvester northbound interface layer of step 31 receives and said acquisition strategies to be sent to logic control with instruction type after the acquisition strategies and processing layer specifically can comprise the steps:
The acquisition strategies that step 311, parsing receive.
Among the embodiment, the acquisition strategies that the interface protocol adaptation module of harvester northbound interface layer receives can be formulated the demand of data according to network management system.Strategy is converted to instruction and issues through instruction path in the process of command analysis module of logic control and processing layer; Clear and definite behavior of an instruction expression; The promoter of this behavior is a harvester; Behavior be object of action by the implementer, act of execution can be had ready conditions and time-constrain, these attributes have constituted and have instructed the fundamental characteristics of this object.We can say that the conversion of instruction translates out with instruction type with these information exactly from tactful container.
As shown in Figure 5, among the embodiment, the logic control of step 32 and processing layer deposit said instruction in the harvester database with the task form and specifically can comprise the steps:
The instruction that step 321, parsing receive.
In fact instruction transformation is become the task form is exactly to generate task object, and each task object can identify its type through distributing a task identification (ID), and task identification is follow-up can be used for selecting adapter.
Among the embodiment, in step 34, logic control and processing layer can also be transferred to said data adaptation layer and carry out after the task of scheduling decomposed earlier and/or data replenish.
In step 34, the data that feedback is gathered have two kinds of forms, and a kind ofly upload, a kind ofly upload for file data for message data, wherein:
The image data of feedback is when being message data, the feedback image data can for:
The data adaptation layer will be uploaded to logic control and processing layer through message channel with the data that form of message is uploaded; Logic control and processing layer are passed to the northbound interface layer with message data; The purpose that the northbound interface layer will be uploaded according to message data sends a message to the south orientation adaptation module.
The image data of feedback is when being file data, the feedback image data can for:
Be kept at this locality behind the data spanned file that the data adaptation layer will be uploaded with document form, and notification logic control and processing layer; Logic control and processing layer receive that through the file uploading channel file being uploaded message after the documentary information is sent to the northbound interface layer; The northbound interface layer receives after file uploads message, and the destination that will upload according to file is with file transfer to south orientation adaptation module.
Based on same inventive concept; The embodiment of the invention also provides a kind of network data acquisition machine; Its structure is as shown in Figure 6; Can comprise northbound interface layer 61, logic control and processing layer 62, data adaptation layer 63, acquisition strategies processing module 64, command process module 65, wherein: acquisition strategies processing module 64 links to each other with northbound interface layer 61, is used for the acquisition strategies that the northbound interface layer receives is sent to logic control and processing layer with instruction type; Command process module 65 links to each other with processing layer 62 with logic control, is used for depositing the instruction that receives in the harvester database with the task form; Logic control and processing layer 62 are dispatched according to the task in the harvester database; Data adaptation layer 63 is carried out data acquisition according to the task of logic control and processing layer 62 scheduling, and the data of feedback collection.
As shown in Figure 7, among the embodiment, acquisition strategies processing module 64 can comprise: policy resolution unit 641 is used for the acquisition strategies that receives resolving; Policy data model conversion unit 642 is used for converting the acquisition strategies that parses to instruction type after instruction path 643 is sent to logic control and processing layer; Instruction path 643 is used to transmit mutual instruction between northbound interface layer and the logical AND processing layer.
After acquisition strategies processing module 64 receives strategies, can be transmitted to policy resolution unit 641 and resolve, and convert strategy the data model of internal unity to, be transmitted to policy data model conversion unit 642.
Uniform data model can be a container object, loads required strategy with this container object, and one of suction parameter of policy data model conversion unit 642 can be exactly a container object.Policy data model conversion unit 642 receives this object and it is carried out model conversion.Fig. 8 is the pictorial diagram of strategy, container, instruction, this four class object of task, can find out, in fact, tactful container is equivalent on the basis of strategy, add the policy name that is used to identify strategy, and other part promptly is the content of policy object.Policy data model conversion unit 642 converts strategy instruction to and issues logic control and processing layer carries out instruction transformation through instruction path 643, from tactful container, the relevant information of policy object is translated out.
As shown in Figure 9, among the embodiment, command process module 65 can comprise: instruction resolution unit 651 is used to resolve the instruction of reception; Director data model conversion unit 652 is used for the instruction transformation that parses is become the task form; Task deployment unit 653 is used for depositing task in the harvester database.
Among the embodiment, can also comprise the data complementary module in the data gather computer, be used for the task of scheduling is decomposed and/or after data replenish, and transfer to the data adaptation layer and carry out.
The data complementary module is used for task is carried out necessary decomposition and additional, and dispatches the collection that each adapter carries out data.Data are replenished pointer to different data acquisition modes, and harvester need replenish necessary information, and under the mode like ftp, harvester also needs the information such as file directory/filename of replenish user name, password, ftp; The task decomposition is meant that in some task be when needing the collaborative completion of a plurality of adapters; Need the parallel perhaps a plurality of adapters of serial scheduling to accomplish a task; Just need decompose task this moment, thereby the parallel perhaps a plurality of adapters of serial scheduling are finished the work.
Based on same inventive concept; The embodiment of the invention also provides a kind of network data acquisition system; Its structure is shown in figure 10, can comprise the harvester that is made up of northbound interface layer 61, logic control and processing layer 62, data adaptation layer 63 in the system, can also comprise:
Acquisition strategies processing module 64 links to each other with the northbound interface layer, is used for the acquisition strategies that the northbound interface layer receives is sent to logic control and processing layer with instruction type;
Logic control and processing layer read task and dispatch according to task;
The data adaptation layer is carried out data acquisition according to the scheduling of logic control and processing layer, and the data of feedback collection;
South orientation adaptation module 67 is used to receive the image data that the data adaptation layer is fed back.
Among the embodiment, after collecting network data, the director data model conversion unit of logic control and processing layer receives task deployment feedback, translates into message and issues the policy data model conversion unit of northbound interface layer through the interlayer passage.
Policy data model conversion cell translation message is the outlet message of northbound interface and through the interface adaptation module result of policy distribution is fed back to the acquisition management module.
In task scheduling and as a result in the process of feedback, task deployment is instruction data model converting unit is carried out application for registration to task a process.Task scheduling is the task that task scheduling modules is disposed according to success, according to dispatching the dispatching cycle of each task.Task needs first deployment success, follow-up could be by normal consistency.
Task scheduling modules obtain mission bit stream and task scheduling time point initiate task scheduling through the task passage and instruct the data complementary module to the data adaptation layer.The task scheduling time point can be meant when begin to have transferred this task in implementing, such as periodic task, begins the time point transferred at every turn, all can according to the cycle in order recursion obtain; Begin focus poll etc. such as the time point of confirming as required to call.
When feedback data, if the image data of feedback is a message data, the data adaptation layer can comprise:
Message sending unit is used for the data of uploading with form of message are uploaded to logic control and processing layer through message channel; After logic control and processing layer were passed to said northbound interface layer with message data, the purpose that the northbound interface layer will be uploaded according to message data sent a message to the south orientation adaptation module.
Perhaps, if the image data of feedback is a file data, the data adaptation layer can comprise:
Preserve the unit, be used for being kept at this locality behind the data spanned file of uploading with document form, and notification logic control and processing layer; After logic control and processing layer are received after the documentary information and through the file uploading channel message to be sent to the northbound interface layer; The destination that the northbound interface layer will be uploaded according to file is with file transfer to south orientation adaptation module.
Wherein: the acquisition management module is that application layer is responsible for carrying out the mutual module of strategy with harvester.The south orientation adaptation module is responsible for accepting the data from acquisition layer.In the enforcement, before the interface and strategy between definition acquisition management module, south orientation adaptation module and the harvester, can first defining interface and tactful transmission means.As: can be the instant message interface between acquisition management module and the harvester, and the data between south orientation adaptation module and the harvester can have file mode and non-file mode, non-file mode also can be to transmit through instant message.In the enforcement, do not have special qualification for message mode, so long as can support the network service under the heterogeneous system situation technology or middleware can, such as RMI, JMS, Socket, WebService, HTTP, CORBA or the like.
There are data passes relation from bottom to top in acquisition management module and south orientation adaptation module and harvester, also have call relation from the top down.The acquisition management module has guaranteed that the strategy that harvester execution collection is relied on has had the source; The interpolation of south orientation adaptation module has guaranteed that the data that harvester is gathered have had the destination; Can call from top to bottom between them; After supporting northbound interface to receive strategy, progressively convert harvester internal task and scheduled for executing into, thereby accomplish collection behavior, realized webmaster harvester based on policy-driven by policy control.
Describe in the face of implementation of strategies in the foregoing description down.Figure 11 is the flow implementation sketch map of policy distribution, and is as shown in the figure, comprising issuing of strategies such as fault, configuration, performance:
Step 111, user carry out policy configurations, perhaps the configuration of strategy are upgraded.
Step 112, the strategy of needs being realized through the interface of acquisition management carry out preliminary treatment.
Step 113, acquisition strategies is issued to the acquisition management service through the acquisition management interface.
Step 114, acquisition management service are carried out analyzing and processing to strategy.
Step 115-116, acquisition management service are passed through message with the policy distribution harvester through equipment such as DOWN FROM CMM and UP TO CMM.
Step 117, harvester are analyzed, handle, are carried out acquisition strategies.
Step 118-119, harvester return the response message of acquisition strategies to the acquisition management service.
Step 1110, acquisition management service feed back to the result application interface of acquisition management.
Step 1111, the feedback message of acquisition strategies is presented to the user through the application interface of acquisition management.
The strategy that relates among Figure 11 in order to example is respectively: MSG_FM_POLICY (warning strategies), MSG_FM_FILTER_POLICY (alarm filter), MSG_PM_POLICY (resource distribution collection), MSG_LOG_POLICY (log collection), MSG_PM_POLICY (performance collection), MSG_SRV_POLICY (service data acquisition), MSG_CF_POLICY (equipment disposition collection), MSG_PROBE_CONF_UPDATE (system configuration renewal).Implement to describe in the face of the partial strategy among Figure 11 down.
1, fault management interface.
(1) warning information is uploaded:
For guaranteeing real-time, this paper adopts message-oriented middleware to upload warning information.Need on MOM (MessageOriented Midware, message based middleware) server, to create the Topic of a T_FM_DATA by name.
(2), warning strategies issues interface:
Definition strategy message is MSG_FM_POLICY among the embodiment, and the flow process reference policy issues the interface section.
(3), alarm filter policy distribution interface:
Definition strategy message is MSG_FM_FILTER_POLICY among the embodiment, and the flow process reference policy issues the interface section.
2, configuration management interface.
(1), resource distribution data upload interface:
The resource distribution data are divided resource allocation information and device configuration file.
The collection frequency of resource allocation information is lower, but the data content collection of gathering is comparatively complicated, and the amount of information of once gathering is bigger.Adopt message-oriented middleware to upload resource allocation information in the example.On the MOM server, create formation Q_RM_DATA and be used to transmit the resource distribution data.
Definition MSG_CF_UL_NOTIFY is that data collection layer finishes in order to notify south orientation adaptation module configuration file to upload in each control messages.
(2), resource distribution data acquisition policy distribution interface:
Definition strategy message is MSG_RM_POLICY in the example, and the flow process reference policy issues the interface section.
(3), office data issues interface:
Will make office data between application management layer group, acquisition management need be obtained the ftp file between data management layer group.The defined declaration of each control messages is following in the example:
Office data file is downloaded notice (SSD_SET_DL_NOTIFY): application management layer notification data acquisition layer obtains office data file on it.
Office data file is downloaded and responded (SSD_SET_DL_ACK): data collection layer is confirmed the validity of office data file.
The office data making destination file is uploaded the notice (SSD_SET_UL_NOTIFY) that finishes: data collection layer notification applications management level office data making destination file has been uploaded and has been finished.
(4), office data is uploaded interface:
The explanation of each control messages is following in the example:
Office data acquisition instructions file is downloaded logical (SSD_GET_DL_NOTIFY): application management layer notification data acquisition layer obtains office data acquisition instructions file from it.
Office data acquisition instructions file is downloaded and responded (SSD_GET_DL_ACK): data collection layer is confirmed the validity of office data acquisition instructions file.
Office data collection result file is uploaded the notice (SSD_GET_UL_NOTIFY) that finishes: data collection layer notification applications management level office data collection result file has been uploaded and has been finished.
(5), patrol task issues interface:
Definition strategy message is AP_DL_NOTIFY in the example, and flow process issues the interface section with reference to office data.
(6), patrol task Status Change notification interface:
Require to adopt message-oriented middleware to upload patrol task Status Change notice in the enforcement; Therefore can on the MOM server, create a TOPIC; Be named T_AP_UPMSG in the example, be used for sending patrol task Status Change notice to application management layer and data management layer respectively.
(7), patrol task control interface:
Definition strategy message is AP_TASK_CONTROL in the example, and the flow process reference policy issues the interface section.
(8), patrol and examine the baseline results file and upload interface:
Data collection layer will be patrolled and examined baseline results file FTP and uploaded to the data management layer, and transmission can be encoded into message MSG_AP_RESULTFILE_UL_NOTIFY with notification message after finishing, and is published to UP_TO_SA.The data management layer can obtain to upload end notification through subscribing to this TOPIC.
The defined declaration of each control messages is following in the example:
MSG_AP_RESULTFILE_UL_NOTIFY: the data acquisition layer assembly is used for the notification data management layer component and successfully uploads and patrol and examine the baseline results file.
(9), log collection policy distribution interface:
Definition strategy message is MSG_LOG_POLICY in the example, and the flow process reference policy issues the interface section
(10), interface is uploaded in daily record:
Data collection layer uploads to the data management layer with journal file FTP, and transmission can be encoded into message MSG_LOG_UL_NOTIFY with notification message after finishing, and is published to UP_TO_SA.The data management layer assembly obtains to upload end notification through subscribing to this TOPIC.
The defined declaration of each control messages is following in the example:
MSG_LOG_UL_NOTIFY: the data acquisition layer assembly is used for the notification data management layer component and successfully uploads journal file.
3, performance management interface.
(1), performance collection policy distribution interface:
Definition strategy message is MSG_PM_POLICY in the example, and the flow process reference policy issues the interface section.
(2), performance data is uploaded interface:
Performance data is uploaded and can be adopted the JMS standard interface, needs on the MOM server, to set up the message queue of a Q_PM_DATA by name.
(3), focus polling interface:
Be not limited in the collection in cycle in the enforcement, also can generate strategy and carry out the focus poll, promptly refer to user's the present situation, and need a network management function of use for a managed object of real time inspection.The setting of this interface is based on following consideration: the normal data acquisition granularity of network management system was generally more than 5 minutes; Time granularity too young pathbreaker makes system resources consumption big; Requirement to configuration is high; Therefore can not generally reduce the acquisition time granularity to the whole network system, and hope still can monitor the state of certain object in real time where necessary.
The interface function of focus poll can comprise issuing of poll strategy and uploading of poll result, and focus poll general action requires high in the short time to real-time.
After having formulated focus poll strategy, when the graphic user interface of operating personnel through the application management layer started the focus poll, application management course data collection layer issued the poll strategy, and data collection layer begins the hot spot data collection subsequently; Operating personnel can align operations such as afoot focus polling operation suspends, continues and stops through the focus polling interface when needed.
For the real-time that guarantees that the focus polling data is uploaded, need on the MOM server, create the message queue of a Q_HOT_POLLING_DATA by name separately.
The defined declaration of the message of example is following:
MSG_PM_HOT_CMD: heat generating spot poll control command under the application management course data collection layer comprises enabled instruction, pause instruction, continuation instruction and halt instruction etc.;
MSG_PM_HOT_DATA: data acquisition course application management layer transmits focus poll result.
(4), service data acquisition policy distribution interface:
Tactful message definition is MSG_SRV_POLICY in the example, and the flow process reference policy issues the interface section.
(5), business datum is uploaded interface:
It is the interface with the data management interlayer that business datum in the data collection layer northbound interface is uploaded interface; The bottom communication implementation that business datum is uploaded interface comprises two kinds of FTP and transmission of messages; Wherein, The transmission of service feature data file (being the XML file of original business datum through forming after the preliminary treatment) adopts FTP to accomplish, and control information adopts message to carry.
The defined declaration of each control messages is following in the example:
MSG_SRV_UL_NOTIFY: harvester notification data management level south orientation adaptation module business datum file has been uploaded and has been finished.
Can know that to sum up the embodiment of the invention possesses following beneficial effect:
1, can in time change the function or the operational mode of data collection layer through the data exhibiting layer; It is said that change etc. is done in the destination that the preprocessing function as to data is made an amendment, logarithm send.
2, can in time check the situation of current data collection, preliminary treatment and transmission at the data exhibiting layer, this is necessary for the data exhibiting layer.
3, for timely, the of short duration supervision of needs by the situation of guard system index; Needn't be configured or move a module at harvester; Be supported in when needing to keep watch on and in time gather, this brings great convenience for the use to system, like the focus poll.
4,, can effectively control the mode of a plurality of acquisition layer data collections, preliminary treatment and transmission through representing layer for the distributed capture system.
5, the embodiment of the invention has not only proposed harvester implementation method, thinking and the scheme based on policy-driven, has also defined concrete tactful execution mode simultaneously.
Obviously, those skilled in the art can carry out various changes and modification to the present invention and not break away from the spirit and scope of the present invention.Like this, belong within the scope of claim of the present invention and equivalent technologies thereof if of the present invention these are revised with modification, then the present invention also is intended to comprise these changes and modification interior.
Claims (12)
1. a network data acquisition method is characterized in that, this method comprises:
Harvester northbound interface layer is sent to logic control and processing layer with said acquisition strategies with instruction type after receiving acquisition strategies; Said acquisition strategies is when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used;
Said logic control and processing layer deposit said instruction in the harvester database with the task form;
Said logic control and processing layer read task and dispatch according to task;
The data adaptation layer is carried out data acquisition according to the scheduling of said logic control and processing layer, and the data of feedback collection.
2. the method for claim 1 is characterized in that, said harvester northbound interface layer is sent to logic control and processing layer with said acquisition strategies with instruction type after receiving acquisition strategies, comprises the steps:
Resolve the acquisition strategies of said reception;
Convert the acquisition strategies that parses to instruction type;
Instruction path through interactive instruction between transmission northbound interface layer and logic control and the processing layer is sent to logic control and processing layer with said instruction.
3. the method for claim 1 is characterized in that, said logic control and processing layer deposit said instruction in the harvester database with the task form, comprise the steps:
Resolve the instruction of said reception;
The instruction transformation that parses is become the task form;
Deposit task in the harvester database.
4. the method for claim 1 is characterized in that, said logic control and processing layer further comprise the steps: after dispatching according to task
After said logic control and processing layer decompose the said task of scheduling and/or data replenish, transfer to said data adaptation layer and carry out.
5. the method for claim 1 is characterized in that, when the image data of said feedback was message data, said data adaptation layer feedback image data comprised:
The data adaptation layer will be uploaded to logic control and processing layer through message channel with the data that form of message is uploaded; Logic control and processing layer are passed to the northbound interface layer with message data; The purpose that the northbound interface layer will be uploaded according to message data sends a message to the south orientation adaptation module;
Or when the image data of said feedback was file data, said feedback image data comprised:
Be kept at this locality behind the data spanned file that the data adaptation layer will be uploaded with document form, and notification logic control and processing layer; Logic control and processing layer receive that through the file uploading channel file being uploaded message after the documentary information is sent to the northbound interface layer; The northbound interface layer receives after file uploads message, and the destination that will upload according to said file is with file transfer to south orientation adaptation module.
6. a network data acquisition machine comprises northbound interface layer, logic control and processing layer, data adaptation layer, it is characterized in that, also comprises:
The acquisition strategies processing module links to each other with the northbound interface layer, is used for the acquisition strategies that the northbound interface layer receives is sent to logic control and processing layer with instruction type; Said acquisition strategies is when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used;
Command process module links to each other with processing layer with logic control, is used for depositing the said instruction that receives in the harvester database with the task form;
Said logic control and processing layer are dispatched according to the task in the harvester database;
The data adaptation layer is carried out data acquisition according to the said task of said logic control and processing layer scheduling, and the data of feedback collection.
7. data gather computer as claimed in claim 6 is characterized in that, said acquisition strategies processing module comprises:
The policy resolution unit is used to resolve the acquisition strategies of said reception;
Policy data model conversion unit is used for converting the acquisition strategies that parses to instruction type after instruction path is sent to logic control and processing layer;
Instruction path is used to transmit mutual instruction between northbound interface layer and logic control and the processing layer.
8. data gather computer as claimed in claim 6 is characterized in that, said command process module comprises:
The instruction resolution unit is used to resolve the instruction of said reception;
Director data model conversion unit is used for the instruction transformation that parses is become the task form;
The task deployment unit is used for depositing task in the harvester database.
9. data gather computer as claimed in claim 6 is characterized in that, also comprises:
The data complementary module is used for the task of said scheduling is decomposed and/or after data replenish, transferred to said data adaptation layer and carry out.
10. a network data acquisition system comprises the harvester that is made up of northbound interface layer, logic control and processing layer, data adaptation layer, it is characterized in that, also comprises:
The acquisition management module is used for acquisition strategies is issued to the northbound interface layer of harvester; Said acquisition strategies is when data are carried out acquisition management, for reaching control and influencing a kind of interface language that the purpose of harvester behavior is used;
The acquisition strategies processing module links to each other with the northbound interface layer, is used for the acquisition strategies that the northbound interface layer receives is sent to logic control and processing layer with instruction type;
Command process module links to each other with processing layer with logic control, is used for depositing the said instruction that receives in the harvester database with the task form;
Said logic control and processing layer read task and dispatch according to task;
The data adaptation layer is carried out data acquisition according to the scheduling of said logic control and processing layer, and the data of feedback collection;
The south orientation adaptation module is used to receive the image data that said data adaptation layer is fed back.
11. data acquisition system as claimed in claim 10 is characterized in that, said data adaptation layer comprises during for message data in the image data of feedback:
Message sending unit is used for the data of uploading with form of message are uploaded to logic control and processing layer through message channel;
After said logic control and processing layer were passed to said northbound interface layer with message data, the purpose that said northbound interface layer will be uploaded according to message data sent a message to the south orientation adaptation module.
12. data acquisition system as claimed in claim 10 is characterized in that, said data adaptation layer comprises during for file data in the image data of feedback:
Preserve the unit, be used for being kept at this locality behind the data spanned file of uploading with document form, and notification logic control and processing layer;
After said logic control and processing layer were received after the documentary information and through the file uploading channel file to be uploaded message and is sent to the northbound interface layer, the destination that said northbound interface layer will be uploaded according to said file was with file transfer to south orientation adaptation module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008101008240A CN101515863B (en) | 2008-02-22 | 2008-02-22 | Network data acquiring method, acquisition machine and acquisition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008101008240A CN101515863B (en) | 2008-02-22 | 2008-02-22 | Network data acquiring method, acquisition machine and acquisition system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101515863A CN101515863A (en) | 2009-08-26 |
| CN101515863B true CN101515863B (en) | 2012-07-11 |
Family
ID=41040169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008101008240A Active CN101515863B (en) | 2008-02-22 | 2008-02-22 | Network data acquiring method, acquisition machine and acquisition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101515863B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104301130B (en) * | 2013-07-17 | 2019-11-08 | 腾讯科技（深圳）有限公司 | Monitoring data collection method and system and task the distribution server and acquisition device |
| CN105511958A (en) * | 2014-10-11 | 2016-04-20 | 阿里巴巴集团控股有限公司 | Method and device for task scheduling |
| CN104484849B (en) | 2014-12-26 | 2018-07-27 | 同方威视技术股份有限公司 | Customs tests place system and method |
| CN105827702A (en) * | 2016-03-16 | 2016-08-03 | 上海轻维软件有限公司 | Distributed message queue based FTP data collection method |
| CN109150563A (en) * | 2017-06-16 | 2019-01-04 | 中兴通讯股份有限公司 | A kind of collecting method based on northbound interface, system and north interface system |
| CN108347360B (en) * | 2018-02-07 | 2021-05-25 | 网宿科技股份有限公司 | Network test data acquisition method and network server |
| CN115051979B (en) * | 2022-04-28 | 2023-09-19 | 福思(杭州)智能科技有限公司 | Monitoring data debugging system, method, vehicle and computer readable storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101101699A (en) * | 2007-07-30 | 2008-01-09 | 当代天启技术（北京）有限公司 | Method and apparatus for data collection and transmission |
-
2008
- 2008-02-22 CN CN2008101008240A patent/CN101515863B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101101699A (en) * | 2007-07-30 | 2008-01-09 | 当代天启技术（北京）有限公司 | Method and apparatus for data collection and transmission |
Non-Patent Citations (1)
| Title |
|---|
| 袁俊佳等.移动综合网管监控数据采集.《应用科技》.2006,第33卷(第10期),32-34,41. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101515863A (en) | 2009-08-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101515863B (en) | Network data acquiring method, acquisition machine and acquisition system | |
| CN112799789B (en) | Node cluster management method, device, equipment and storage medium | |
| US8065657B2 (en) | Exchange infrastructure system and method | |
| CN1688950A (en) | Automation platform for information management, condition monitoring and real-time control of distributed industrial systems | |
| CN104967650A (en) | Third-party e-commerce platform unified releasing method | |
| CN1791861A (en) | Business process management for a message-based exchange infrastructure | |
| CN112181648B (en) | Configuration-driven lightweight hybrid infrastructure platform and data processing method | |
| KR102678656B1 (en) | Data processing methods and devices | |
| CN113766026B (en) | Data processing method and system applied to energy industrial network | |
| CN113937894A (en) | Cloud edge cooperation-based electric intelligent terminal management system and method | |
| CN112804362A (en) | Dispersed data micro-service automation operation and maintenance system | |
| CN1433545A (en) | Method and apparatus for anonymous subiject-based addressing | |
| CN1981541A (en) | Integrated management of a wireless network | |
| CN109656518A (en) | A kind of Internet Industry equipment remote monitoring system design of Android intelligence | |
| CA2481099C (en) | Exchange infrastructure system and method | |
| JP2005322222A (en) | Communication function adding method, program, storage medium and communication apparatus | |
| CN114385541B (en) | OPC UA aggregation server oriented to intelligent manufacturing and design method thereof | |
| CN115081679A (en) | CDC-based industrial data dynamic change automatic perception processing system | |
| CN114500530A (en) | Automatic adjustment method for civil edge information system | |
| CN100492982C (en) | A network equipment driving system and method for implementing message management | |
| CN105007266A (en) | Automated information transmission system and method for demand responses of power generation and power consumption of high energy-consuming enterprise | |
| CN112486954B (en) | Operation and maintenance platform database subsystem design method | |
| CN114331113A (en) | Method and system for managing industrial production based on micro-service architecture and process engine | |
| CN118034918A (en) | Lightweight micro-service combination platform architecture based on cloud edge combination | |
| Fernández Sierra | Design, Development and Testing of a Virtualized Database for Monitoring 5G-connected AGVs in an Industry 4.0 environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |