CN107070890A - Flow data processing device and communication network major clique system in a kind of communication network major clique system - Google Patents
Flow data processing device and communication network major clique system in a kind of communication network major clique system Download PDFInfo
- Publication number
- CN107070890A CN107070890A CN201710142542.6A CN201710142542A CN107070890A CN 107070890 A CN107070890 A CN 107070890A CN 201710142542 A CN201710142542 A CN 201710142542A CN 107070890 A CN107070890 A CN 107070890A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- layer
- nodes
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
- H04L41/044—Network management architectures or arrangements comprising hierarchical management structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0823—Configuration setting characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
- H04W84/04—Large scale networks; Deep hierarchical networks
- H04W84/08—Trunked mobile radio systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides the flow data processing device in a kind of communication network major clique system and communication network major clique system, flow data processing device includes：Stream data generation module, the file data for different data sources to be obtained is loaded as stream data and pushed in Kafka processing modules；Kafka processing modules, for stream data to be equably loaded onto in Kafka clusters, so that Storm processing modules are called；Storm processing modules, including Spout nodes and Bolt nodes, Spout nodes are used to read data from Kafka clusters and push to Bolt nodes, the data that Bolt nodes are used to push Spout nodes according to the application scenarios being pre-configured with accordingly calculate and result of calculation are carried out into System, and periodically store summarized results into Redis clusters；Redis memory modules, for by the data buffer storage collected in internal memory；Data outputting module, for Stream Processing result to be supplied into third party data person with the interface of standard.The present invention enables network optimization system to provide the real-time perception analytic function of user class.
Description
Technical field
The present invention relates to communication technical field, and in particular to flow data processing device in a kind of communication network major clique system and logical
Believe network optimization system.
Background technology
The positioning of network optimization system initially is primarily as off line data analysis system, and the running status to mobile network is carried out
Comprehensive analysis and assessment, such as comprehensive assessment wireless network, the network environment of core net, assess current network covering, interference problem,
Special Handle of Drop Question, assesses quality of service, user and perceives, can show assessment result with modes such as intuitively GIS, chart, forms.On
The realization for stating network optimization function is depended on to basic data, performance data, MR data, supplemental characteristic, call bill data etc. in network no
The collection and parsing of same type data source, and generally above-mentioned data processing be required for periodically generating by producer OMC, data text
Multiple steps such as part remote transmission, data parsing, data summarization, data loading, so occurring from network event, to network optimization system
Show the delay of at least 2-3 hours.
But with network optimization system increasing using department and user of service, usage scenario it is abundant, at network optimization system
Manage the requirement more and more higher of real-time, such as the quick response complained for customer service, the monitoring guarantee of important race scene, single
Signaling tracing of call etc., these usage scenarios are all the standards for needing to reach real-time Treatment Analysis.This just exists to network optimization system
Real time data processing technically proposes higher requirement.
The content of the invention
For defect of the prior art, the invention provides the flow data processing device in a kind of communication network major clique system and
Communication network major clique is united, and the invention enables the real-time perception analytic function that network optimization system can provide user class.
In order to solve the above technical problems, the present invention provides following technical scheme：
In a first aspect, the invention provides the flow data processing device in a kind of communication network major clique system, including：
Stream data generation module, the flow data that the file data for different data sources to be obtained is loaded as a rule draws
The stream data of form required for holding up, and push in Kafka processing modules；
Kafka processing modules, are that a distributed message lines up processing module, for by the stream data generation module
The stream data of generation is equably loaded onto in Kafka clusters, so that Storm processing modules are called；
Storm processing modules, are the computing engines of Stream Processing, including Spout nodes and Bolt nodes, Spout nodes
For reading data from Kafka clusters, and Bolt nodes are pushed to, Bolt nodes are used for the data for pushing Spout nodes
According to the application scenarios being pre-configured with accordingly calculate and result of calculation is subjected to System, and periodically by summarized results
Store into Redis clusters；
Redis memory modules, using distributed type assemblies scheme, for the data that the Storm processing modules collect to be delayed
Exist in internal memory；
Data outputting module, for being supplied to third party data person to use with the interface of standard the result of Stream Processing.
Further, the data outputting module support to carry out in the way of component it is extending transversely, including：HBASE components,
Codis components, Hive components, Oracle components and GP components.
Further, the data outputting module supports user mobile phone APP is called by WebService to be looked into real time
Ask.
Second aspect, present invention also offers a kind of communication network major clique system, including：The data set gradually from bottom to top are adopted
Collection layer, data processing and accumulation layer, technological service component layer and apply represent layer；
The data collection layer, for carrying out distributed deployment to acquisition tasks, supports real-time by cloud acquisition technique
Collection, batch capture and the collection of internet reptile, for being acquired to PB DBMSs with convergence, there is provided flow data processing work
Tool, supports the acquisition process of real time data, realizes data distribution and loading；
The data processing employs flow data processing device as described above with accumulation layer；The data processing is with depositing
Reservoir uses " streaming computing+Hadoop+MPP+RDB " mashed up data processing architecture to support the mashed up calculating of diversified magnanimity to deposit
There is provided the computation module and service of computing and streaming computing outside real-time operation, storehouse for energy storage power；
The technological service component layer is used for form/report, search engine, GIS service and the information for supporting upper layer application
Push Service；
The application represent layer is used for the graphical interfaces displaying for realizing system, including PC editions are applied represent layer and intelligent movable
Terminal version applies represent layer, and figure display form includes GIS and chart.
Further, the data processing is configured with the Storm stream process clusters of N number of node composition, M section with accumulation layer
The Kafka message trunkings of the Redis memory databases cluster of point composition and Q node composition, and each clustered deploy(ment) is 10,000,000,000
In Ethernet.
Further, the N values are that 6, M values are that 4, Q values are 4.
Further, the data of the data collection layer collection include network foundation data, MR data, supplemental characteristic, words
Forms data and signaling data.
Further, the technological service component layer include WebServices components, SPSS components, Solr components and
Struts2 components.
Further, figure is realized using JSP, easyUI, JQuery and ArcGis technology using represent layer for described PC editions
Displaying.
Further, the mobile intelligent terminal version application represent layer uses IOS, Android, Html5 and BaiduMap
Technology realizes that figure is shown.
As shown from the above technical solution, the communication network major clique system that the present invention is provided, using " streaming computing+Hadoop+MPP+
The system architecture technology of the mashed up patterns of RDB ", had both taken into account real-time analysis and the off-line analysis demand of big data processing, had not had again
High concurrent, the demand of on-line analysis of routine data are abandoned, network optimization system is provided the real-time perception analysis of user class
Function.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the accompanying drawing used required in technology description to be briefly described, it should be apparent that, drawings in the following description are the present invention
Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
These accompanying drawings obtain other accompanying drawings.
Fig. 1 is the structural representation of the flow data processing device during the communication network major clique that one embodiment of the invention is provided is united；
Fig. 2 is the streaming computing software architecture schematic diagram that one embodiment of the invention is provided；
Fig. 3 is the structural representation for the communication network major clique system that another embodiment of the present invention is provided；
Fig. 4 is the software architecture schematic diagram for the communication network major clique system that another embodiment of the present invention is provided.
Embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, clear, complete description is carried out to the technical scheme in the embodiment of the present invention, it is clear that described embodiment is
A part of embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.
Compared with the data such as the communication network major clique analyze data used of system and alarm, configuration, the performance in tradition O domains, data
The scale of construction is huge, particularly with data such as MR, ticket, signalings, and often an initial data increment for saving one day can just reach several
Individual or even more than ten of T.For the big data application scenarios, existing more ripe distributed proccessing is to big data progress now
Parsing and storage, but generated from data to analysis presentation, often there is the time delay of some hours.It is big currently invention addresses solving
The real-time performance of data processing, using flow data processing device, the when pressure limiting of the data processing in communication network major clique being united
It is reduced to a second rank.
Fig. 1 shows the structural representation of the flow data processing device in the communication network major clique system that one embodiment of the invention is provided
Figure.Referring to Fig. 1, the flow data processing device in the communication network major clique system that the present invention is provided, including：Stream data generation module
11st, Kafka processing modules 12, Storm processing modules 13, Redis memory modules 14 and data outputting module 15, wherein：
Stream data generation module 11, the file data for different data sources to be obtained is loaded as the flow data of a rule
The stream data of form required for engine, and push in Kafka processing modules；
It is understood that the major function of stream data generation module 11 is the number of files that will be obtained by different data sources
According to being loaded as the stream data of form required for flow data engine one by one, and push in kafka processing modules.Referring to
Streaming computing software architecture schematic diagram shown in Fig. 2, the stream data generation module 11 support FTP, Socket,
The real time data online acquisition access of the common protocols such as WebService, STDP, and common system journal are gathered in real time.It is right
In the most common data file collection of network optimization system, when any change occurs for destination folder, monitoring journey can be all triggered
Sequence is performed, and monitoring programme differentiates whether the change is to have increased newly to need file to be processed automatically, and if it is monitoring programme can be by
File name is added to pending document queue.
Kafka processing modules 12, are that a distributed message lines up processing module, for the stream data to be generated into mould
The stream data of block generation is equably loaded onto in Kafka clusters, so that Storm processing modules are called；
It is understood that module of the Kafka processing modules 12 primarily as a Distributed Message Queue, for inciting somebody to action
The stream data generated in stream data generation module is uniformly loaded onto in Kafka clusters, so that Storm processing modules are carried out
Call, it can be understood as the caching of a data flow.The distributed post subscription that the module uses a kind of high-throughput disappears
Breath system, is to unify Message Processing on line and offline by similar Hadoop distributed parallel load mechanism, supportive
Energy linear expansion, can obtain the lifting of the disposal ability of approximately linear ratio by way of adding hardware.
Storm processing modules 13, are the computing engines of Stream Processing, including Spout nodes and Bolt nodes, Spout sections
Point is used to read data from Kafka clusters, and pushes to Bolt nodes, and Bolt nodes are used for the number for pushing Spout nodes
The application scenarios being pre-configured with according to basis accordingly calculate and result of calculation are carried out into System, and will periodically collect knot
Fruit is stored into Redis clusters；
It is understood that Storm processing modules 13 are the computing engines of Stream Processing, can be according to the applied field of configuration
Scape carries out calculating operation, plays a part of data System, and output results in corresponding storage.Shown in Figure 2
Streaming computing software architecture schematic diagram, Spout nodes are responsible for reading data from Kafka clusters, and push to Bolt nodes,
The data that Bolt nodes are responsible for pushing Spout are classified according to different service applications (such as Subscriber Number), and are carried out
Real time data collects, and timing stores summarized results into Redis clusters.
Redis memory modules 14, using distributed type assemblies scheme, for the data for collecting the Storm processing modules
It is buffered in internal memory；
It is understood that Redis memory modules 14 are for guaranteed efficiency, combined data is buffered in internal memory.
Redis clusters consider that data scale is huge, situations such as concurrency is high are called, using distributed type assemblies scheme, this solution
The function such as scheme has load balancing and standby machine is fault-tolerant, can effectively solve fault-tolerant and concurrent problem.
Data outputting module 15, for being supplied to third party data person to make with the interface of standard the result of Stream Processing
With.
It is understood that data outputting module 15 is mainly responsible for the result of Stream Processing being supplied to the interface of standard
Third party data person uses.Streaming computing software architecture schematic diagram shown in Figure 2, the data outputting module 15 is supported with group
The mode of part carries out extending transversely, including HBASE components, Codis components, Hive components, Oracle components, GP components, also props up
Hold user mobile phone App and carry out real-time query is called by WebService.
Compared with the offline data processing technique of large quantities such as Hadoop, flow data processing device provided in an embodiment of the present invention
More it is good at sudden, wall scroll event real-time processing.By taking XDR class data processings as an example, counted since XDR data acquisitions in place
Calculate, it is only necessary to by the processing of rank time second, you can complete the parsing storage of data.Place Jing Guo background application module again
Reason, the time delay that user behavior and network index are only needed to by second rank can be presented.Such time delay is relative with tradition
The technology time delay of 2-3 hours, is properly termed as processing in real time completely, in important race/meeting monitoring guarantee, real-time signaling tracing etc.
Application scenarios can meet application requirement.
Another embodiment of the present invention is united there is provided a kind of communication network major clique, referring to Fig. 3, and communication network major clique system includes：From
The data collection layer 31, data processing and the accumulation layer 32 that set gradually upwards down, technological service component layer 33 and apply represent layer
34；
The data collection layer 31, for carrying out distributed deployment to acquisition tasks, supports real by cloud acquisition technique
When collection, batch capture and internet reptile collection, for PB DBMSs are acquired with convergence there is provided flow data handle work
Tool, supports the acquisition process of real time data, realizes data distribution and loading；
It is understood that data collection layer 31 is by cloud acquisition technique, distributed deployment can be carried out to acquisition tasks,
Support the acquisition techniques such as collection in real time, batch capture, the collection of internet reptile.PB DBMSs can be acquired and be restrained, carried
For flow data handling implement, the acquisition process of real time data is supported, and realizes data distribution and loading.The data collection layer 31
The data of collection include network foundation data, MR data, supplemental characteristic, call bill data and signaling data.
The data processing employs the flow data processing device as described in above example with accumulation layer 32；The data
Processing uses " streaming computing+Hadoop+MPP+RDB " mashed up data processing architecture to support diversified magnanimity to mix with accumulation layer
Taking calculating storage capacity, there is provided the computation module and service of computing and streaming computing outside real-time operation, storehouse；
It is understood that the technological core of communication network major clique system is the various software handled various big datas
The integrated application of technology.Hadoop technologies possess born distributed data processing advantage, are mainly used in realizing MR, ticket, XDR
Batch storing and resolving and preliminary KPI indexs etc. class big data original document collect, and speed is fast, efficiency high；MPP is column number
According to storehouse cluster, it is adaptable to the storage of long-term magnanimity analyze data (MR combined data, call bill data, traffic statistics data etc.), especially fit
Close ticket, the inquiry of signaling alanysis application, data storage compression than high, be the data storage scheme that uses of network optimization system it
One；RDB is traditional relevant database, with technical characterstics such as high concurrent, low capacities, it is adaptable to the result class of small data quantity
Data, cycle/real-time update class data (base station work parameter evidence, result of appraisal data, work order data, management data etc.) are deposited
Storage；Streaming computing is then the technology occurred to solve big data real-time processing requirement, is mainly used in small time delay class data
Collection, analysis, processing scene, such as important race/meeting monitoring is ensured, real-time signaling tracing.It is understood that at data
Reason and the embodiment that accumulation layer 32 is that network optimization system handles big data characteristic, it is mixed using " streaming computing+Hadoop+MPP+RDB "
The data processing architecture taken, with support diversified mass data it is mashed up calculate storage capacity there is provided computing outside real-time operation, storehouse,
The computation modules such as streaming computing and service, also provide the data quality management functions such as data integrity, accuracy.
Such as, it is necessary to using wireless during being ridden to user in high ferro in the high ferro special project analysis that communication network major clique is united
The quality of network is analyzed, and needs to be monitored and be analyzed by user.To realize that the real-time perception of user class is analyzed, need
Processing scheme is calculated using real-time streaming, to provide alap processing delay.In order to improve the processing of XDR call bill datas
Real-time, reduces processing delay, builds Storm stream process cluster and completes the real-time analytic function of single user.In project practical application
In, 6 node composition Storm stream process clusters of configuration, 4 node composition Redis memory database clusters of configuration configure 4
Node constitutes Kafka message trunkings.Clustered deploy(ment) is in ten thousand mbit ethernets, to ensure network transfer speeds.In actual motion mistake
The processing procedure of wall scroll user bill data in Cheng Zhong, XDR file, from XDR files output be accomplished to the real-time behavior of user and
The KPI achievement datas of perception analysis are output to real-time monitoring analysis platform, and overall process time delay is about 5 seconds.It can be seen that, by using
Stream Processing Clustering, greatly improves the process performance of user class data, and processing delay is reduced to " real-time "
Scope, is that active user behavior monitoring and perception analysis provide effective support.
The technological service component layer 33 is used for form/report, search engine, GIS service and the letter for supporting upper layer application
Cease Push Service；
It is understood that form/report of the support upper layer application of technological service component layer 33, search engine, GIS clothes
The infrastructure components such as business, Information Push Service, also using modularization, modular design, aspect flexibly upgrading and replacement.Referring to Fig. 4
The software architecture schematic diagram of shown communication network major clique system, technological service component layer 33 includes WebServices components, SPSS groups
Part, Solr components and Struts2 components.
The application represent layer 34 is used for the graphical interfaces displaying for realizing system, including PC editions application represent layers and mobile intelligence
Energy terminal version applies represent layer, and figure display form includes GIS and chart.
It is understood that the displaying of system graphical interfaces is mainly realized using represent layer 34, including PC editions (web browsings
Device mode) and mobile intelligent terminal version it is (ios, android, windows phone primary APP, non-protogenous APP, wechat, short
The modes such as letter), showing form includes GIS, chart etc..The software architecture schematic diagram of communication network major clique system shown in Figure 4, PC
Version application represent layer realizes that figure is shown using JSP, easyUI, JQuery and ArcGis technology；Mobile intelligent terminal version application
Represent layer realizes that figure is shown using IOS, Android, Html5 and BaiduMap technology.
Communication network major clique system provided in an embodiment of the present invention, using " streaming computing+Hadoop+MPP+RDB " mashed up pattern
System architecture technology, both taken into account real-time analysis and the off-line analysis demand of big data processing, and and do not abandoned routine data
High concurrent, the demand of on-line analysis, network optimization system is provided the real-time perception analytic function of user class.
Above example is merely to illustrate technical scheme, rather than its limitations；Although with reference to the foregoing embodiments
The present invention is described in detail, it will be understood by those within the art that：It still can be to foregoing each implementation
Technical scheme described in example is modified, or carries out equivalent substitution to which part technical characteristic；And these are changed or replaced
Change, the essence of appropriate technical solution is departed from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (10)
1. the flow data processing device in a kind of communication network major clique system, it is characterised in that including：
Stream data generation module, the file data for different data sources to be obtained is loaded as the flow data engine institute of a rule
The stream data of form is needed, and is pushed in Kafka processing modules；
Kafka processing modules, are that a distributed message lines up processing module, for the stream data generation module to be generated
Stream data be equably loaded onto in Kafka clusters, so that Storm processing modules are called；
Storm processing modules, are the computing engines of Stream Processing, including Spout nodes and Bolt nodes, and Spout nodes are used for
Read data from Kafka clusters, and push to Bolt nodes, Bolt nodes be used for the data that push Spout nodes according to
The application scenarios being pre-configured with accordingly calculate and result of calculation are carried out into System, and periodically store summarized results
Into Redis clusters；
Redis memory modules, using distributed type assemblies scheme, for the data buffer storage that the Storm processing modules collect to be existed
In internal memory；
Data outputting module, for being supplied to third party data person to use with the interface of standard the result of Stream Processing.
2. device according to claim 1, it is characterised in that the data outputting module is supported to carry out in the way of component
It is extending transversely, including：HBASE components, Codis components, Hive components, Oracle components and GP components.
3. device according to claim 1, it is characterised in that the data outputting module supports user mobile phone APP to pass through
WebService calls carry out real-time query.
4. a kind of communication network major clique system, it is characterised in that including：The data collection layer that sets gradually from bottom to top, data processing
With accumulation layer, technological service component layer and applying represent layer；
The data collection layer is by cloud acquisition technique, for carrying out distributed deployment to acquisition tasks, support collection in real time,
Batch capture and the collection of internet reptile, for being acquired to PB DBMSs with convergence, there is provided flow data handling implement, support
The acquisition process of real time data, realizes data distribution and loading；
The data processing employs the flow data processing device as described in any one of claims 1 to 3 with accumulation layer；The number
" streaming computing+Hadoop+MPP+RDB " mashed up data processing architecture is used to support diversified magnanimity according to processing and accumulation layer
There is provided the computation module and service of computing and streaming computing outside real-time operation, storehouse for mashed up calculating storage capacity；
The technological service component layer is used to support form/report, search engine, GIS service and the information of upper layer application to push
Service；
The application represent layer is used for the graphical interfaces displaying for realizing system, including PC editions are applied represent layer and mobile intelligent terminal
Version applies represent layer, and figure display form includes GIS and chart.
5. system according to claim 4, it is characterised in that the data processing is configured with N number of node group with accumulation layer
Into Storm stream process clusters, the Kafka message that the Redis memory databases cluster and Q node of M node composition are constituted
Cluster, and each clustered deploy(ment) is in ten thousand mbit ethernets.
6. system according to claim 5, it is characterised in that the N values are that 6, M values are that 4, Q values are 4.
7. system according to claim 4, it is characterised in that the data of the data collection layer collection include network foundation
Data, MR data, supplemental characteristic, call bill data and signaling data.
8. system according to claim 4, it is characterised in that the technological service component layer includes WebServices groups
Part, SPSS components, Solr components and Struts2 components.
9. system according to claim 4, it is characterised in that the PC editions application represent layer using JSP, easyUI,
JQuery and ArcGis technologies realize that figure is shown.
10. system according to claim 4, it is characterised in that the mobile intelligent terminal version application represent layer is used
IOS, Android, Html5 and BaiduMap technology realize that figure is shown.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710142542.6A CN107070890A (en) | 2017-03-10 | 2017-03-10 | Flow data processing device and communication network major clique system in a kind of communication network major clique system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710142542.6A CN107070890A (en) | 2017-03-10 | 2017-03-10 | Flow data processing device and communication network major clique system in a kind of communication network major clique system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107070890A true CN107070890A (en) | 2017-08-18 |
Family
ID=59622366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710142542.6A Pending CN107070890A (en) | 2017-03-10 | 2017-03-10 | Flow data processing device and communication network major clique system in a kind of communication network major clique system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107070890A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107508888A (en) * | 2017-08-25 | 2017-12-22 | 同方(深圳)云计算技术股份有限公司 | A kind of car networking service platform |
| CN108009257A (en) * | 2017-12-08 | 2018-05-08 | 武汉虹信技术服务有限责任公司 | A kind of wireless RF data screening plant and method based on streaming computing |
| CN108737177A (en) * | 2018-05-21 | 2018-11-02 | 中国联合网络通信有限公司重庆市分公司 | A kind of implementation method mobile Internet real-time streaming data acquisition and analyzed |
| CN109086410A (en) * | 2018-08-02 | 2018-12-25 | 中国联合网络通信集团有限公司 | The processing method and system of streaming mass data |
| CN109558397A (en) * | 2018-10-30 | 2019-04-02 | 平安医疗健康管理股份有限公司 | A kind of data processing method, device, server and computer storage medium |
| CN109686035A (en) * | 2018-12-27 | 2019-04-26 | 福建小电科技有限公司 | A kind of electric motorcar charging station fire-fighting early warning system based on big data |
| CN109766363A (en) * | 2019-01-08 | 2019-05-17 | 北京江融信科技有限公司 | Stream data processing method, system, electronic equipment and storage medium |
| CN110020360A (en) * | 2017-11-09 | 2019-07-16 | 北京京东尚科信息技术有限公司 | The method that user behavior characteristics are extracted, system and server |
| CN110232073A (en) * | 2019-05-10 | 2019-09-13 | 中国联合网络通信集团有限公司 | A kind of Data Management Analysis system and method |
| CN110750562A (en) * | 2018-07-20 | 2020-02-04 | 武汉烽火众智智慧之星科技有限公司 | Storm-based real-time data comparison early warning method and system |
| CN110809050A (en) * | 2019-11-08 | 2020-02-18 | 智者四海（北京）技术有限公司 | Personalized push system and method based on streaming computing |
| CN110971687A (en) * | 2019-11-29 | 2020-04-07 | 浙江邦盛科技有限公司 | Rail transit flow data processing method |
| CN111049898A (en) * | 2019-12-10 | 2020-04-21 | 杭州东方通信软件技术有限公司 | Method and system for realizing cross-domain architecture of computing cluster resources |
| CN111787497A (en) * | 2020-07-01 | 2020-10-16 | 北京长焜科技有限公司 | Method for storing original charging call ticket by using database cluster |
| CN112134846A (en) * | 2020-08-21 | 2020-12-25 | 宜通世纪科技股份有限公司 | Method, system, device and medium for analyzing signaling data of communication network |
| CN112200931A (en) * | 2020-09-02 | 2021-01-08 | 南京知数网络科技有限公司 | Intelligent positioning distribution system and method for eagle eye track |
| CN112506915A (en) * | 2020-10-27 | 2021-03-16 | 百果园技术(新加坡)有限公司 | Application data management system, processing method and device and server |
| CN112995263A (en) * | 2019-12-18 | 2021-06-18 | 中国移动通信集团陕西有限公司 | Network priority data processing system |
| CN113422840A (en) * | 2021-07-13 | 2021-09-21 | 全景智联(武汉)科技有限公司 | Picture processing system and method based on file transfer protocol |
| CN113486063A (en) * | 2021-07-05 | 2021-10-08 | 国网河北省电力有限公司信息通信分公司 | Method and device for processing flow data in power internet of things and terminal equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105681474A (en) * | 2016-03-31 | 2016-06-15 | 浪潮通信信息系统有限公司 | System architecture for supporting upper layer applications based on enterprise-level big data platform |
| CN106168909A (en) * | 2016-06-30 | 2016-11-30 | 北京奇虎科技有限公司 | A kind for the treatment of method and apparatus of daily record |
| US9560119B2 (en) * | 2014-12-23 | 2017-01-31 | Cisco Technology, Inc. | Elastic scale out policy service |
-
2017
- 2017-03-10 CN CN201710142542.6A patent/CN107070890A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9560119B2 (en) * | 2014-12-23 | 2017-01-31 | Cisco Technology, Inc. | Elastic scale out policy service |
| CN105681474A (en) * | 2016-03-31 | 2016-06-15 | 浪潮通信信息系统有限公司 | System architecture for supporting upper layer applications based on enterprise-level big data platform |
| CN106168909A (en) * | 2016-06-30 | 2016-11-30 | 北京奇虎科技有限公司 | A kind for the treatment of method and apparatus of daily record |
Non-Patent Citations (3)
| Title |
|---|
| 丁亦志 等: "大数据在电信行业的应用研究", 《互联网天地》 * |
| 屈国庆: "基于Storm的实时日志分析系统的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 杨华辉: "分布式日志系统的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107508888A (en) * | 2017-08-25 | 2017-12-22 | 同方(深圳)云计算技术股份有限公司 | A kind of car networking service platform |
| CN110020360A (en) * | 2017-11-09 | 2019-07-16 | 北京京东尚科信息技术有限公司 | The method that user behavior characteristics are extracted, system and server |
| CN108009257A (en) * | 2017-12-08 | 2018-05-08 | 武汉虹信技术服务有限责任公司 | A kind of wireless RF data screening plant and method based on streaming computing |
| CN108009257B (en) * | 2017-12-08 | 2020-09-11 | 武汉虹信技术服务有限责任公司 | Wireless radio frequency data screening device and method based on stream computing |
| CN108737177A (en) * | 2018-05-21 | 2018-11-02 | 中国联合网络通信有限公司重庆市分公司 | A kind of implementation method mobile Internet real-time streaming data acquisition and analyzed |
| CN110750562B (en) * | 2018-07-20 | 2023-10-27 | 宿迁市公安局 | Real-time data comparison early warning method and system based on Storm |
| CN110750562A (en) * | 2018-07-20 | 2020-02-04 | 武汉烽火众智智慧之星科技有限公司 | Storm-based real-time data comparison early warning method and system |
| CN109086410A (en) * | 2018-08-02 | 2018-12-25 | 中国联合网络通信集团有限公司 | The processing method and system of streaming mass data |
| CN109558397A (en) * | 2018-10-30 | 2019-04-02 | 平安医疗健康管理股份有限公司 | A kind of data processing method, device, server and computer storage medium |
| CN109558397B (en) * | 2018-10-30 | 2023-08-22 | 深圳平安医疗健康科技服务有限公司 | Data processing method, device, server and computer storage medium |
| CN109686035A (en) * | 2018-12-27 | 2019-04-26 | 福建小电科技有限公司 | A kind of electric motorcar charging station fire-fighting early warning system based on big data |
| CN109766363A (en) * | 2019-01-08 | 2019-05-17 | 北京江融信科技有限公司 | Stream data processing method, system, electronic equipment and storage medium |
| CN110232073A (en) * | 2019-05-10 | 2019-09-13 | 中国联合网络通信集团有限公司 | A kind of Data Management Analysis system and method |
| CN110809050A (en) * | 2019-11-08 | 2020-02-18 | 智者四海（北京）技术有限公司 | Personalized push system and method based on streaming computing |
| CN110971687A (en) * | 2019-11-29 | 2020-04-07 | 浙江邦盛科技有限公司 | Rail transit flow data processing method |
| CN111049898A (en) * | 2019-12-10 | 2020-04-21 | 杭州东方通信软件技术有限公司 | Method and system for realizing cross-domain architecture of computing cluster resources |
| CN112995263A (en) * | 2019-12-18 | 2021-06-18 | 中国移动通信集团陕西有限公司 | Network priority data processing system |
| CN111787497B (en) * | 2020-07-01 | 2021-07-09 | 北京长焜科技有限公司 | Method for storing original charging call ticket by using database cluster |
| CN111787497A (en) * | 2020-07-01 | 2020-10-16 | 北京长焜科技有限公司 | Method for storing original charging call ticket by using database cluster |
| CN112134846B (en) * | 2020-08-21 | 2023-04-18 | 宜通世纪科技股份有限公司 | Method, system, device and medium for analyzing signaling data of communication network |
| CN112134846A (en) * | 2020-08-21 | 2020-12-25 | 宜通世纪科技股份有限公司 | Method, system, device and medium for analyzing signaling data of communication network |
| CN112200931A (en) * | 2020-09-02 | 2021-01-08 | 南京知数网络科技有限公司 | Intelligent positioning distribution system and method for eagle eye track |
| CN112506915A (en) * | 2020-10-27 | 2021-03-16 | 百果园技术(新加坡)有限公司 | Application data management system, processing method and device and server |
| CN112506915B (en) * | 2020-10-27 | 2024-05-10 | 百果园技术(新加坡)有限公司 | Application data management system, processing method and device and server |
| CN113486063A (en) * | 2021-07-05 | 2021-10-08 | 国网河北省电力有限公司信息通信分公司 | Method and device for processing flow data in power internet of things and terminal equipment |
| CN113422840A (en) * | 2021-07-13 | 2021-09-21 | 全景智联(武汉)科技有限公司 | Picture processing system and method based on file transfer protocol |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107070890A (en) | Flow data processing device and communication network major clique system in a kind of communication network major clique system | |
| CN105224445B (en) | Distributed tracking system | |
| CN107103064B (en) | Data statistical method and device | |
| CN106126641A (en) | A kind of real-time recommendation system and method based on Spark | |
| CN106709003A (en) | Hadoop-based mass log data processing method | |
| CN106951552A (en) | A kind of user behavior data processing method based on Hadoop | |
| CN105677842A (en) | Log analysis system based on Hadoop big data processing technique | |
| CN109739919A (en) | A kind of front end processor and acquisition system for electric system | |
| US9374475B1 (en) | System for processing customer records | |
| CN104317942A (en) | Massive data comparison method and system based on hadoop cloud platform | |
| CN108268569A (en) | The acquisition of water resource monitoring data and analysis system and method based on big data technology | |
| CN110083600A (en) | A kind of method, apparatus, calculating equipment and the storage medium of log collection processing | |
| CN115033646A (en) | Method for constructing real-time warehouse system based on Flink and Doris | |
| CN115038083A (en) | Telecom fraud early warning identification method and system applied to AI operator industry | |
| CN112631754A (en) | Data processing method, data processing device, storage medium and electronic device | |
| CN113378219B (en) | Unstructured data processing method and system | |
| CN103220363A (en) | Distributed network training resource management system based on cloud computing and scheduling method | |
| Tudoran et al. | SAGE: Geo-distributed streaming data analysis in clouds | |
| CN113342806A (en) | Big data processing method and device, storage medium and processor | |
| CN105740397A (en) | Big data parallel operation-based voice mail business data analysis method | |
| CN111049898A (en) | Method and system for realizing cross-domain architecture of computing cluster resources | |
| CN108430067A (en) | A kind of Internet service mass analysis method and system based on XDR | |
| Tseng et al. | A successful application of big data storage techniques implemented to criminal investigation for telecom | |
| Jiadi et al. | Research on Data Center Operation and Maintenance Management Based on Big Data | |
| CN116186053A (en) | Data processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170818 |
|
| RJ01 | Rejection of invention patent application after publication |