CN106294580A - LTE network MR data analysing method based on HADOOP platform - Google Patents
LTE network MR data analysing method based on HADOOP platform Download PDFInfo
- Publication number
- CN106294580A CN106294580A CN201610605960.XA CN201610605960A CN106294580A CN 106294580 A CN106294580 A CN 106294580A CN 201610605960 A CN201610605960 A CN 201610605960A CN 106294580 A CN106294580 A CN 106294580A
- Authority
- CN
- China
- Prior art keywords
- data
- platform
- hadoop
- analysis
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Abstract
The present invention provides a kind of LTE network MR data analysing method based on HADOOP platform, collects HADOOP computing Platform native including by the original compression file of MR data；On HADOOP platform, MR data are evenly distributed on each independent disk, start n × M computing process, MR data are decompressed and document analysis, after completing, merge storage；Carry out MR data parsing based on HADOOP platform, MR Network Quality Analysis, start warehouse-in process according to default unit file number, export in relevant database.The present invention relates to be applied in the analysis of LTE network MR data big data platform, by the distributed file system of big data platform, task debugging management process, greatly improve the efficiency of MR data analysis, and support that realizing MR data analysis is transferred to face by point, is converted to time+Spatial Multi-Dimensional degree analysis by one-dimensional degree.
Description
Technical field
The present invention relates to moving communicating field, be specifically related to MR (mobile phone test report) in communication 4G LTE data network
Analyze.
Background technology
Along with operator optimizes the drastically minimizing of expenditure, the day by day increase of network size, network structure increasingly sophisticated, use
Family service and the successive Regression of troxerutine tabtets, traditional network optimization and assessment, customer complaint location and the method processed and think of
Road cannot meet current demand------economical and efficient, three-dimensional assessment, quickly response, precise positioning.
Summary of the invention
It is an object of the invention to LTE network MR analysis platform based on big data, promote data analysis and mining ability, fast
Network is estimated and analyzes by speed accurately, prior to the user discover that network problem and problem hidden danger, and then systematic lifting
Efficiency-cost ratio and user satisfaction.
The present invention provides a kind of LTE network MR data analysing method based on HADOOP platform, comprises the following steps:
1) the original compression file of MR data is collected HADOOP computing Platform native；
2) on HADOOP platform, MR data are evenly distributed on each independent disk, it is achieved mode is, if MR platform by
N platform main frame forms, and every main frame has the disk of M block independence, then the original compression file of MR data is equally assigned into n × M part；
3) on every main frame, start M operational analysis task, start n × M computing process altogether, MR data are carried out as follows
Process,
A) decompressing, including by n × M computing process, the original compression file that will distribute accordingly respectively, decompression is condensed to original
Xml document；
B) document analysis, including by n × M computing process, is converted to txt file by original xml document respectively；
C) each process merges after being parsed；
D) it is stored in HADOOP platform by merging the file generated；
4) according to being stored in the file of HADOOP platform, MR data parsing based on HADOOP platform is carried out, each including searching for
The indices of community, described index is community Rsrp, county Rsrp rsrq index, districts and cities Rsrp rsrq index, RSRP is weak covers
Lid, RSRP count, grid adjacent area；
5) carry out big market demand MR Network Quality Analysis based on HADOOP platform, add up including according to indices；
6), after having added up, according to time, region, network element classification, statistical result is exported HADOOP HDFS file system；
7) on HADOOP platform, according to time, region, network element, according to default unit file number, start warehouse-in process, make
With the JDBC of standard by step 6) in acquired results export in relevant database；
8) MR data analysis process completes.
And, carry out big market demand MR interference based on HADOOP platform and analyze.
And, big market demand MR stain is analyzed.
And, if the unit file number preset is 1000, needing the warehouse-in number of processes started is number of files/1000.
The present invention relates to be applied in the analysis of LTE network MR data big data platform, dividing by big data platform
Cloth file system, task debugging management process, greatly improve the efficiency of MR data analysis.Present invention mainly solves tradition
MR analyze and mainly rely on relevant database, it is impossible to the problem processing in time mass data, it is also possible to by HADOOP's
Multitask-scheduling engine, disturbs analysis in conjunction with the analysis of MR stain, MR, can support that realizing MR data analysis is transferred to face by point,
Time+Spatial Multi-Dimensional degree analysis is converted to by one-dimensional degree.
Accompanying drawing explanation
Fig. 1 is embodiment of the present invention flow chart.
Detailed description of the invention
Technical solution of the present invention is described in detail below in conjunction with drawings and Examples.
The present invention provides the method for LTE network MR data analysis based on HADOOP platform, and MR data are passed through HADOOP
Platform, utilizes the distributed file system of big data platform, Portable Batch System process, greatly improves MR data analysis
Efficiency, it is achieved the covering of MR data Points And lines aspect and disturbed condition analysis.HADOOP is a distributed system architecture,
Achieve a distributed file system (Hadoop Distributed File System), be called for short HDFS.Count in a large number at MR
According to storage in, by HADOOP platform introduce MR analysis in, by will gather conversion XML use HDFS distribution storage, solve
The certainly problem of magnanimity MR data storage.During MR Network Quality Analysis, MR interference analysis, MR stain analysis etc., use
The distributed scheduling managing process of HADOOP, it is achieved the distributed treatment of MR parser, reaches to be rapidly completed MR index analysis
Target.
The flow process of the embodiment of the present invention comprises the following steps:
1) by FTP, the original document of MR data (zip tar compressed file) is collected HADOOP computing Platform native；
2) on the big data platform of HADOOP, according to the method for mean allocation, MR data are evenly distributed to each independent magnetic
On dish.
That is: if MR platform is made up of n platform main frame (computer), every main frame is made up of M block independent disk.Then: MR data
Original compressed file will be equally assigned into n*M part according to quantity of documents, n*M i.e. n × M.
3) on every main frame, start M operational analysis task, start n*M computing process altogether, MR data are carried out as
Under the process of several processes:
A) decompress.I.e. by n*M computing process, respectively by the file that the unprocessed form of offer is zip tar, decompression is condensed to
Original xml document；
B) document analysis, by n*M computing process, is converted to the txt file of standard respectively by original xml document.
C) after being respectively parsed, merge, can merge by concrete network element situation.Such as, base station 1, base station
2 ... the relevant MR data of base station N are merged into file 1-MR data .txt, 2-MR data .txt respectively ... N-MR data .txt.
D) it is stored in HADOOP platform by merging the file generated, it is achieved upload HDFS.So HADOOP platform is introduced
In the analysis of MR, use HDFS distribution storage, the problem that the storage of magnanimity MR data can be solved by the XML that will gather.
The file storage structure of HADOOP platform is:
Time (example: on March 15th, 2016)
Region (example: Wuhan-> development zone, East Lake)
1-MR data .txt
2-MR data .txt
N-MR data .txt
4) according to being stored in the file of HADOOP platform, carry out MR data parsing based on HADOOP platform, put down by big data
The YARN scheduling engine of platform, according to: community Rsrp, county Rsrp rsrq index, districts and cities Rsrp rsrq index, the weak covering of RSRP,
RSRP counts, grid adjacent area totally 7 indexs, with name-value pair, it is thus achieved that the These parameters of each community.Described Rsrp is reference signal
Receive power, described rsrq(RSRP) it is Reference Signal Received Quality.YARN is existing scheduling of resource framework, and bottom is distribution
Formula storage system HDFS, stability and high efficiency, therefore embodiment Selection utilization YARN scheduling engine.
5) big market demand MR Network Quality Analysis is carried out based on HADOOP platform: dispatched by the YARN of big data platform
Engine, by by the data in units of community, carries out computing statistics, those skilled in the art's predeterminable statistics when being embodied as
The contents of a project, generally according to: community Rsrp statistics, county Rsrp rsrq indicator-specific statistics, districts and cities Rsrp rsrq indicator-specific statistics, RSRP
Weak covering is added up, RSRP counts, and statistics, grid adjacent area totally 7 indexs are added up.When being embodied as, the most expansible count greatly
Analyze according to application MR interference, big market demand MR stain is analyzed, and concrete analysis refers to prior art and realizes.Use HADOOP's
Distributed scheduling managing process, it is achieved the distributed treatment of MR parser, can improve processing speed.The example metric of statistics
As follows:
Example metric one:
It is cumulative/current that prefecture-level RSRP/RSRQ/SINR average=to all MR sampled points of current districts and cities carries out level value
Total sampling number of districts and cities
6), after having added up, according to time, region, network element classification, statistical result is exported HADOOPHDFS file system.
The structure of output file is as follows:
Time (example: on March 15th, 2016)
Region (example: Wuhan-> development zone, East Lake)
Network element (example: network element 1)
Community Rsrp adds up .txt
County Rsrp rsrq indicator-specific statistics .txt
City Rsrp rsrq indicator-specific statistics .txt
RSRP is weak covers statistics .txt
RSRP counts and adds up .txt
Grid adjacent area .txt
7) on HADOOP platform, according to time, region, network element, in units of every 1000 files, start one put in storage into
Journey, uses the JDBC (Java Data Base Connectivity, java data base connect) of standard, by step 6) in number
According to, export in relevant database.Therefore the warehouse-in number of processes being actually needed startup is number of files/1000.It is embodied as
Time, the number of files of per unit can be preset by those skilled in the art.
8) complete whole MR and analyze process.
Above-described embodiment flow process describes only for understanding explanation technical solution of the present invention, but the present invention is not limited in above-mentioned
Embodiment；Any simple modification, equivalent variations and the modification that in every technical spirit according to the present invention, embodiment is made, all falls
Within entering the protection domain of technical scheme.
Claims (4)
1. a LTE network MR data analysing method based on HADOOP platform, it is characterised in that comprise the following steps:
1) the original compression file of MR data is collected HADOOP computing Platform native；
2) on HADOOP platform, MR data are evenly distributed on each independent disk, it is achieved mode is, if MR platform by
N platform main frame forms, and every main frame has the disk of M block independence, then the original compression file of MR data is equally assigned into n × M part；
3) on every main frame, start M operational analysis task, start n × M computing process altogether, MR data are carried out as follows
Process,
A) decompressing, including by n × M computing process, the original compression file that will distribute accordingly respectively, decompression is condensed to original
Xml document；
B) document analysis, including by n × M computing process, is converted to txt file by original xml document respectively；
C) each process merges after being parsed；
D) it is stored in HADOOP platform by merging the file generated；
4) according to being stored in the file of HADOOP platform, MR data parsing based on HADOOP platform is carried out, each including searching for
The indices of community, described index is community Rsrp, county Rsrp rsrq index, districts and cities Rsrp rsrq index, RSRP is weak covers
Lid, RSRP count, grid adjacent area；
5) carry out big market demand MR Network Quality Analysis based on HADOOP platform, add up including according to indices；
6), after having added up, according to time, region, network element classification, statistical result is exported HADOOP HDFS file system；
7) on HADOOP platform, according to time, region, network element, according to default unit file number, start warehouse-in process, make
With the JDBC of standard by step 6) in acquired results export in relevant database；
8) MR data analysis process completes.
LTE network MR data analysing method based on HADOOP platform the most according to claim 1, it is characterised in that: based on
HADOOP platform carries out big market demand MR interference and analyzes.
LTE network MR data analysing method based on HADOOP platform the most according to claim 1, it is characterised in that: several
Analyze according to application MR stain.
4., according to LTE network MR data analysing method based on HADOOP platform described in claim 1 or 2 or 3, its feature exists
In: setting default unit file number as 1000, needing the warehouse-in number of processes started is number of files/1000.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610605960.XA CN106294580A (en) | 2016-07-28 | 2016-07-28 | LTE network MR data analysing method based on HADOOP platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610605960.XA CN106294580A (en) | 2016-07-28 | 2016-07-28 | LTE network MR data analysing method based on HADOOP platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106294580A true CN106294580A (en) | 2017-01-04 |
Family
ID=57662970
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610605960.XA Pending CN106294580A (en) | 2016-07-28 | 2016-07-28 | LTE network MR data analysing method based on HADOOP platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106294580A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106804046A (en) * | 2017-02-16 | 2017-06-06 | 广州杰赛科技股份有限公司 | Mobile location method and device based on measurement report |
| CN107071821A (en) * | 2017-03-24 | 2017-08-18 | 广州逸信电子科技有限公司 | Mobile communications network measuring and analysis system and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105787064A (en) * | 2016-03-01 | 2016-07-20 | 广州铭诚计算机科技有限公司 | Mining platform establishment method based on big data |
| US9448851B2 (en) * | 2013-06-19 | 2016-09-20 | International Business Machines Corporation | Smarter big data processing using collaborative map reduce frameworks |
| CN105138661B (en) * | 2015-09-02 | 2018-10-30 | 西北大学 | A kind of network security daily record k-means cluster analysis systems and method based on Hadoop |
-
2016
- 2016-07-28 CN CN201610605960.XA patent/CN106294580A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9448851B2 (en) * | 2013-06-19 | 2016-09-20 | International Business Machines Corporation | Smarter big data processing using collaborative map reduce frameworks |
| CN105138661B (en) * | 2015-09-02 | 2018-10-30 | 西北大学 | A kind of network security daily record k-means cluster analysis systems and method based on Hadoop |
| CN105787064A (en) * | 2016-03-01 | 2016-07-20 | 广州铭诚计算机科技有限公司 | Mining platform establishment method based on big data |
Non-Patent Citations (3)
| Title |
|---|
| 伍洪斌: "基于云计算的移动通信4G网络优化", 《网规网优》 * |
| 何珂，全涛，赵晋，李德强: "基于MR数据的LTE网络射频精细优化的方法研究", 《网规网优》 * |
| 张士蒙: "基于Hadoop的无线网络业务和用户挖掘平台的实现与应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106804046A (en) * | 2017-02-16 | 2017-06-06 | 广州杰赛科技股份有限公司 | Mobile location method and device based on measurement report |
| CN107071821A (en) * | 2017-03-24 | 2017-08-18 | 广州逸信电子科技有限公司 | Mobile communications network measuring and analysis system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106874381B (en) | Radio environment map data processing system based on Hadoop | |
| CN104657396A (en) | Data migration method and device | |
| CN104767795A (en) | LTE MRO data statistical method and system based on HADOOP | |
| CN108632111A (en) | Service link monitoring method based on log | |
| CN106331616A (en) | Heterogeneous environment-based massive video structured analysis system and method | |
| CN106294580A (en) | LTE network MR data analysing method based on HADOOP platform | |
| CN104462456A (en) | Life data processing based big data system | |
| CN104301916B (en) | Test optimization method, apparatus and system based on mobile intelligent terminal universal card | |
| Ghasemi | Predictive modeling of LTE user throughput via crowd-sourced mobile spectrum data | |
| CN110865851A (en) | Automatic Android application data acquisition method and system | |
| CN107506422A (en) | The distributed information log processing system and method for a kind of multi-data source | |
| CN111031550B (en) | Method and device for judging weak coverage area of wireless network | |
| US20190306722A1 (en) | Licensed assisted access based on low utilization of unlicensed channels | |
| CN102595461A (en) | Performance testing device, system and testing method for mobile communication system | |
| US9451482B2 (en) | Method for tracking and utilizing operational run time of a network element | |
| CN108093075A (en) | A kind of implementation method of application system gray scale issue | |
| Pina et al. | Automatic coverage based neighbour estimation system: A cloud-based implementation | |
| CN116056117A (en) | Network anomaly prediction method, device and storage medium | |
| CN109947467B (en) | Method, device and system for acquiring performance flame diagram of remote terminal | |
| CN103714433A (en) | Hidden danger troubleshooting internet-of-things trusted terminal and hidden danger troubleshooting method | |
| Skračić et al. | A big data solution for troubleshooting mobile network performance problems | |
| CN102595442A (en) | Performance statistical data acquisition method and data acquisition unit | |
| CN111144687A (en) | Novel electric power big data analysis system | |
| CN113269397A (en) | Method for checking user variation relation of equipment association characteristics based on atlas technology | |
| CN104348654A (en) | CDMA (Code Division Multiple Access) wireless test data management platform based on Internet of things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170104 |
|
| RJ01 | Rejection of invention patent application after publication |