CN109005083A - The method and system of large scale collection base station data - Google Patents
The method and system of large scale collection base station data Download PDFInfo
- Publication number
- CN109005083A CN109005083A CN201810788650.5A CN201810788650A CN109005083A CN 109005083 A CN109005083 A CN 109005083A CN 201810788650 A CN201810788650 A CN 201810788650A CN 109005083 A CN109005083 A CN 109005083A
- Authority
- CN
- China
- Prior art keywords
- task
- order
- base station
- large scale
- machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/0213—Standardised network management protocols, e.g. simple network management protocol [SNMP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/08—Protocols specially adapted for terminal emulation, e.g. Telnet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/08—Testing, supervising or monitoring using real traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W88/00—Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
- H04W88/08—Access point devices
Abstract
The present invention provides a kind of method and systems of large scale collection base station data, the described method comprises the following steps: addition task obtains task into task queue from task queue；Task execution configuration is generated, task is analyzed, by Task Switching at the order that can be executed on machine；Order is executed by multithreading calling；After the completion of executing order, implementing result is obtained, implementing result is parsed, extracts information needed.The present invention is not necessarily to install any software in monitored equipment, different collection modes can be selected according to primary device characteristics.By the way of active probe, the inconvenience of installation client software is avoided.The present invention supports a variety of existing network protocols for monitoring, and height customizable, different orders can be selected to go to execute according to receiver model.The present invention greatly improves the handling capacity of system.
Description
Technical field
The present invention relates to data collection techniques fields, and in particular to a kind of method of large scale collection base station data and is
System.
Background technique
Have and persistently build up the base station with considerable scale at present, high accuracy positioning is used for, in order to monitor these in time
The health status of base station, such as internal network connectivty, if online, just whether the network flow of receiver and its function
Often.These information must send back monitoring center in time, so as to subsequent processing and alarm.
There is no a set of ready-made solution, traditional monitoring system needs to pacify in monitored equipment industry at present
Dress disposes a set of client to send data, and has oneself a set of agreement in itself.The shortcomings that prior art, is specific as follows:
1, traditional monitoring system needs to affix one's name to a set of client in monitored equipment upper mounting portion mostly to send data.
But the equipment of current base station is the network equipment mostly, limited capacity, and system does not also have the condition of installation software.
2, existing monitoring system on the market is all whole set of system, and the monitoring system that can not be had by oneself with company blends,
Such as the data format supporting various protocols (SNMP, TELNET) etc. simultaneously, and generating also with company existing comparative maturity
Monitor control index data are incompatible.Existing monitoring system can not be customized large-scale data collection, such as cannot basis
Receiver model selects different collection modes；When receiver there are multiple IP, it is beyond recognition and finally uses that IP as mesh
Mark；
3, data throughout is insufficient.Existing technology before the deadline, for example can not be completed to collect in one minute, such as
Fruit postpones, and will continue to collect, and data is caused to overstock.
Summary of the invention
The present invention solves the technical issues of quickly collecting base station data, the present invention adopts under the premise of guaranteeing low latency
Technical solution is as follows:
A kind of method of large scale collection base station data, comprising the following steps:
Addition task obtains task into task queue from task queue；
Task execution configuration is generated, task is analyzed, by Task Switching at the order that can be executed on machine；
Order is executed by multithreading calling；
After the completion of executing order, implementing result is obtained, implementing result is parsed, extracts information needed.
Further, the addition task into task queue specifically includes the following steps:
The list of all base station machines is read, the information of base station machine is analyzed；
After obtaining the information of base station machine, judge whether to can be used as task, if it is, being added in task queue；It is no
Then it is not processed.
Further, the information of the base station machine includes that base station name, machine models, machine IP, machine port are wherein appointed
It anticipates a kind of or more.
Further, at least one, the base station machine, all base station machines obtain task simultaneously from task queue.
Further, operation is primary per minute for the step of adding task, if the task of upper one minute is without all processing
It completes, makes its expired failure automatically.
Further, the order that can be executed on machine include the main body of final order, order parameter field and
The corresponding parameter value of the parameter field of order.
Further, the main body of the final order includes PING, TELNET one of which or more, the ginseng of the order
Digital section includes the subparameter name one of which or more of number COUNT, port name PORT, SNMP, and the parameter value includes end
Mouth number.
Further, generation task execution configuration specifically includes the following steps:
Select command determines the type of final order；
Select command parameter judges whether to need to be customized, if so, the parameter field of select command, determines parameter
The corresponding parameter value of field generates final order；If not, directly generating final order.
The present invention also provides a kind of system of large scale collection base station data, the system comprises:
Task addition and reading unit, for the task of adding into task queue, and obtain task from task queue；
Configuration unit is executed, for generating task execution configuration, task is analyzed, by Task Switching at can be in machine
The order of upper execution；
Order unit is executed, for executing order by multithreading calling；
Implementing result acquiring unit obtains implementing result, parses to implementing result after the completion of executing order,
Extract information needed.
Further, the system supports ICMP, TELNET, the order of SNMP network protocol.
The present invention also provides a kind of memory, the memory is stored with computer program, and the computer program is held
Row following steps:
Addition task obtains task into task queue from task queue；
Task execution configuration is generated, task is analyzed, by Task Switching at the order that can be executed on machine；
Order is executed by multithreading calling；
After the completion of executing order, implementing result is obtained, implementing result is parsed, extracts information needed.
Beneficial effects of the present invention are as follows:
1, the present invention can select different without installing any software in monitored equipment according to primary device characteristics
Collection mode.By the way of active probe, the inconvenience of installation client software is avoided.
2, the present invention supports a variety of existing network protocols for monitoring, such as SNMP and TELNET, and is collected
Data directly with the Data Structures metric of existing monitoring system be output, be fully immersed into existing monitor supervision platform.And
And height customizable, different orders can be selected to go to execute according to receiver model.The different IP of same equipment can be led to
Configuration is crossed, is selectively collected respectively.
3, present invention utilizes high concurrent technologies outstanding in open source community, such as Redis to realize producer consumer
Development mode, there are also the thread scheduling that Java programming language is had by oneself, the distribution that completes various tasks is executed, and is greatly improved
The handling capacity of system.
Detailed description of the invention
Fig. 1 is the overall flow figure of large scale collection base station data method of the present invention；
Fig. 2 is present invention addition mission flow diagram；
Fig. 3 is that the present invention generates execution configuration flow figure；
Fig. 4 is the structure chart of large scale collection base station data system of the present invention.
Specific embodiment
Hereinafter, the present invention is further elaborated in conjunction with the accompanying drawings and embodiments.
Embodiment one:
Fig. 1 is base station data collection method flow chart of the present invention, and core of the invention is obtained in a task queue
Task, the task refer to the collection task for executing smallest particles degree, and a certain service for a certain base station is collected, and raw
At configuration is executed, finally executes some and order and return to implementing result.More machines can take task in queue simultaneously.
Addition task is further elucidated below, generates and executes parameter, the practical embodiment executed and obtain implementing result.
1) task is added:
As shown in Fig. 2, the process of addition task is first to take the list of all base station machines, the specific of machine is then analyzed
Information.
Base station information data include station name, and code, IP, device model, port etc., the following are the bases involved in the present invention arrived
The sample for information of standing.
" id ": " 590303cec2a98ae558dd4eea "
" encode ": " AHCF ",
" name ": " Anhui Changfeng ",
" province ": " Anhui ",
" proterty ": " self-built ",
" status ": " ONLINE ",
" receiverIp ": " 172.17.63.26 "
" firewallIpWan ": " 172.17.198.26 ",
" firewallIpLan ": " 172.17.63.25 ",
" modelNo ": " afw.sangfor ",
" receiverBrand ": " China surveys ",
" receiverModel ": " CHC N72 ",
" anrennaType ": " HXCCGX601A HXCS ",
" arpX ": " -2458342.9098 "
" arpY ": " 4791968.9270 ",
" arpZ ": " 3405376.8894 "
" arpB ": " 32.4789539697 ",
" arpL ": " 117.1584185183 "
" arpH ": " 41.6191 "
After obtaining base station machine specifying information, system decides whether to can be used as task, if it can, being so directly added into
To task queue.If because lacking certain information, for example without IP, then mission bit stream will not be added to.
The process of addition task is that operation per minute is primary, and if the task of upper one minute has been handled without whole
At making its expired failure automatically, such benefit is not have task heap product to influence follow-up work processing progress.
2) it generates and executes configuration
After system takes specific tasks, the task can be analyzed, each task is allowed to become really execute in machine
On order.Process is as shown in Figure 3.
Task becomes executable order, includes:
1, final order main body (type), such as ping, telnet etc.；
2, the parameter field ordered is incoming, such as number count, the subparameter name of port name port, SNMP；
3, the corresponding value of each parameter field ordered, such as how many times, specific port number.
The order example ultimately generated is as follows:
snmpbulkwalk-v 2c-c com
192.168.11.131.3.6.1.4.1.25506.2.6.1.1.1.1.12.78
3) order is executed
When the parameter needed for the step of before obtains all execution, executing final order becomes very simple,
Only need to start simultaneously at multiple thread dispatching orders.
4) implementing result is obtained
When ordering execution, using the standard output of system as direct result.After the completion of ordering execution, parsing standard is defeated
Out, useful information is extracted.Such as implementing result are as follows:
2packets transmitted, 2received, 0%packet loss, time 1075ms
System obtains " 0% " this number and is used as operation result.
Embodiment two:
The present invention also provides a kind of systems of large scale collection base station data, as shown in figure 4, the system comprises:
Task addition and reading unit, for the task of adding into task queue, and obtain task from task queue；
Configuration unit is executed, for generating task execution configuration, task is analyzed, by Task Switching at can be in machine
The order of upper execution；
Order unit is executed, for executing order by multithreading calling；
Implementing result acquiring unit obtains implementing result, parses to implementing result after the completion of executing order,
Extract information needed.
The present invention supports to realize ICMP, the order of the network protocols such as TELNET, SNMP, and has high scalability, Ke Yi
More multi-protocols are realized in the case where having no need to change framework, including SSH etc..
Embodiment three:
The present invention also provides a kind of memory, the memory is stored with computer program, the computer program quilt
Processor executes following steps:
Addition task obtains task into task queue from task queue；
Task execution configuration is generated, task is analyzed, by Task Switching at the order that can be executed on machine；
Order is executed by multithreading calling；
After the completion of executing order, implementing result is obtained, implementing result is parsed, extracts information needed.
Present invention preferably employs 8 programming language of Java and GNU/Linux operating systems.
By developing the extremely low client of software and hardware requirement, it can dispose on network devices, can just accomplish in this way
The collection of big data quantity；
Secondary development to tradition monitoring product such as Zabbix, is added configuration schedules mode specific to the present invention, can
Realize identical purpose.
Although the invention has been described by way of example and in terms of the preferred embodiments, but it is not for limiting the present invention, any this field
Technical staff without departing from the spirit and scope of the present invention, may be by the methods and technical content of the disclosure above to this hair
Bright technical solution makes possible variation and modification, therefore, anything that does not depart from the technical scheme of the invention, and according to the present invention
Technical spirit any simple modifications, equivalents, and modifications to the above embodiments, belong to technical solution of the present invention
Protection scope.
Claims (11)
1. a kind of method of large scale collection base station data, which comprises the following steps:
Addition task obtains task into task queue from task queue；
Task execution configuration is generated, task is analyzed, by Task Switching at the order that can be executed on machine；
Order is executed by multithreading calling；
After the completion of executing order, implementing result is obtained, implementing result is parsed, extracts information needed.
2. a kind of method of large scale collection base station data as described in claim 1, which is characterized in that the addition task arrives
In task queue specifically includes the following steps:
The list of all base station machines is read, the information of base station machine is analyzed；
After obtaining the information of base station machine, judge whether to can be used as task, if it is, being added in task queue；Otherwise not
It processes.
3. a kind of method of large scale collection base station data as claimed in claim 2, which is characterized in that the base station machine
Information includes base station name, machine models, machine IP, machine port any of them or more.
4. a kind of method of large scale collection base station data as claimed in claim 3, which is characterized in that the base station machine is extremely
Rare one, all base station machines obtain task simultaneously from task queue.
5. a kind of method of large scale collection base station data as claimed in claim 3, which is characterized in that the step of adding task
Operation is primary per minute, if the task of upper one minute is completed without all processing, makes its expired failure automatically.
6. a kind of method of large scale collection base station data as claimed in claim 3, which is characterized in that it is described can be on machine
Parameter field corresponding parameter value of the order of execution including the main body of final order, the parameter field and order of order.
7. a kind of method of large scale collection base station data as claimed in claim 6, which is characterized in that the final order
Main body includes PING, TELNET one of or more, the parameter field of the order include number COUNT, port name PORT,
Subparameter name one of which of SNMP or more, the parameter value includes port number.
8. a kind of method of large scale collection base station data as claimed in claim 3, which is characterized in that the generation task is held
Row configuration specifically includes the following steps:
Select command determines the type of final order；
Select command parameter judges whether to need to be customized, if so, the parameter field of select command, determines parameter field
Corresponding parameter value generates final order；If not, directly generating final order.
9. a kind of system of large scale collection base station data, which is characterized in that the system comprises:
Task addition and reading unit, for the task of adding into task queue, and obtain task from task queue；
Configuration unit is executed, for generating task execution configuration, task is analyzed, by Task Switching at can be held on machine
Capable order；
Order unit is executed, for executing order by multithreading calling；
Implementing result acquiring unit obtains implementing result, parses to implementing result after the completion of executing order, extracts
Information needed.
10. a kind of system of large scale collection base station data as claimed in claim 9, which is characterized in that the system is supported
The order of ICMP, TELNET, SNMP network protocol.
11. a kind of memory, the memory is stored with computer program, which is characterized in that the computer program executes such as
Lower step:
Addition task obtains task into task queue from task queue；
Task execution configuration is generated, task is analyzed, by Task Switching at the order that can be executed on machine；
Order is executed by multithreading calling；
After the completion of executing order, implementing result is obtained, implementing result is parsed, extracts information needed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810788650.5A CN109005083A (en) | 2018-07-17 | 2018-07-17 | The method and system of large scale collection base station data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810788650.5A CN109005083A (en) | 2018-07-17 | 2018-07-17 | The method and system of large scale collection base station data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109005083A true CN109005083A (en) | 2018-12-14 |
Family
ID=64600469
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810788650.5A Pending CN109005083A (en) | 2018-07-17 | 2018-07-17 | The method and system of large scale collection base station data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109005083A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116737172A (en) * | 2023-08-11 | 2023-09-12 | 杭州初灵信息技术股份有限公司 | Small particle data packet analysis system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2533153A1 (en) * | 2011-06-08 | 2012-12-12 | Vodafone Holding GmbH | Unit for managing messages indicating event situations of monitored objects |
| CN103324552A (en) * | 2013-06-06 | 2013-09-25 | 西安交通大学 | Two-stage single-instance data de-duplication backup method |
| CN103731298A (en) * | 2013-11-15 | 2014-04-16 | 中国航天科工集团第二研究院七〇六所 | Large-scale distributed network safety data acquisition method and system |
| CN103885781A (en) * | 2014-04-02 | 2014-06-25 | 云南电网公司 | Double-queue data collecting and developing method |
| CN106936660A (en) * | 2015-12-31 | 2017-07-07 | 华为软件技术有限公司 | Collecting method and device |
-
2018
- 2018-07-17 CN CN201810788650.5A patent/CN109005083A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2533153A1 (en) * | 2011-06-08 | 2012-12-12 | Vodafone Holding GmbH | Unit for managing messages indicating event situations of monitored objects |
| CN103324552A (en) * | 2013-06-06 | 2013-09-25 | 西安交通大学 | Two-stage single-instance data de-duplication backup method |
| CN103731298A (en) * | 2013-11-15 | 2014-04-16 | 中国航天科工集团第二研究院七〇六所 | Large-scale distributed network safety data acquisition method and system |
| CN103885781A (en) * | 2014-04-02 | 2014-06-25 | 云南电网公司 | Double-queue data collecting and developing method |
| CN106936660A (en) * | 2015-12-31 | 2017-07-07 | 华为软件技术有限公司 | Collecting method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116737172A (en) * | 2023-08-11 | 2023-09-12 | 杭州初灵信息技术股份有限公司 | Small particle data packet analysis system and method |
| CN116737172B (en) * | 2023-08-11 | 2023-12-12 | 杭州初灵信息技术股份有限公司 | Small particle data packet analysis system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104345717B (en) | Intelligent remote data acquisition system based on Internet of Things | |
| CN104765641B (en) | A kind of job scheduling method and system | |
| CN109698787A (en) | A method of realizing Edge position control in gateway of internet of things | |
| CN1784656A (en) | Monitoring operational data in data processing systems | |
| CN109756443A (en) | Smart home device interlock method, equipment and system | |
| US20190361712A1 (en) | Data processing device, data processing method, setting management device, and data processing system | |
| EP3722956A1 (en) | Control system, information processing device, and abnormality factor estimation program | |
| CN103544060B (en) | A kind of service dispatch system and method based on WEBSERVICE | |
| CN103369402A (en) | Method and system for comparing functional consistency of multiple set top boxes | |
| CN110320853A (en) | A kind of plc data capturing analysis method and system | |
| CN110766263A (en) | Automatic acceptance method for secondary equipment of transformer substation and terminal equipment | |
| CN109005083A (en) | The method and system of large scale collection base station data | |
| CN113609048B (en) | Cloud edge service collaborative interaction method for electric power Internet of things | |
| CN109445384B (en) | Multi-device control system | |
| CN111224817A (en) | Distributed micro-acquisition system and method thereof | |
| CN107222589A (en) | A kind of embedded equipment I P address batch modification method and system of identical IP | |
| CN109213091A (en) | A kind of semiconductor chip process equipment method for monitoring state based on document analysis | |
| CN102361355A (en) | Method for concentratedly acquiring data of city and county scheduling power equipment and monitoring operation | |
| CN104217314B (en) | Routing iinformation grasping means and device | |
| KR20150000567A (en) | Apparatus for remote meter reading protocol of binary type and apparatus and method for task scheduling | |
| CN114387123B (en) | Data acquisition management method | |
| CN110531724B (en) | Remote measurement automatic sampling definition method based on smart power grid dispatching control system | |
| CN112859660B (en) | Equipment synchronization control method, device, terminal and system | |
| CN107592286A (en) | A kind of intelligent communications terminal and its implementation for supporting multi-protocols | |
| CN107729136A (en) | A kind of processing priority collocation method and device based on FPGA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181214 |