CN104507107B - A kind of preprocess method of signaling data - Google Patents
A kind of preprocess method of signaling data Download PDFInfo
- Publication number
- CN104507107B CN104507107B CN201410791579.8A CN201410791579A CN104507107B CN 104507107 B CN104507107 B CN 104507107B CN 201410791579 A CN201410791579 A CN 201410791579A CN 104507107 B CN104507107 B CN 104507107B
- Authority
- CN
- China
- Prior art keywords
- signaling
- data
- buffer
- file
- signal collecting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000011664 signaling Effects 0.000 title claims abstract description 118
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000004140 cleaning Methods 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims abstract description 14
- 238000006243 chemical reaction Methods 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 230000009466 transformation Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 abstract description 7
- 238000012545 processing Methods 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 238000002203 pretreatment Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002224 dissection Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
Abstract
The invention discloses a kind of preprocess methods of signaling data, include the following steps：Newly-increased signaling file, the signaling data for monitoring the FTP interface message processor (IMP)s of signal collecting layer are transferred to signaling data cleaning and conversion in Buffer, Buffer of signaling inclusion layer.A kind of preprocess method of signaling data is compared with prior art, signaling data is transferred to signaling inclusion layer from signal collecting layer and data preprocessing operation is carried out before file is written, avoid secondary reading of the signaling file in preprocessing process, the time of entire signaling data processing is effectively saved, it is highly practical.
Description
Technical field
The present invention relates to mobile communications industry monitoring signaling technologies, specifically a kind of highly practical, signaling data
Preprocess method.
Background technology
The continuous speedup of development of mobile communications network, various kinds of equipment mixed networking in network, network structure is increasingly sophisticated, net
Interior type of business is also constantly updated, per minute all to generate a large amount of letter as network size constantly expands, number of users increases
Data are enabled, the acquisition of signaling information, warehousing and treatment effeciency etc. made higher requirement.
In a communication signaling monitoring platform, signal collecting layer mainly obtains original letter from mobile radio communication
It enables data, and to the data needed for convergence device transmission, by shunting, being divided, the modes such as branch, realizes to being transmitted in existing net
The acquisition of original signalling coding, and the transmission of backward signalling inclusion layer is being packaged using defined form, in case dissection process,
Has certain storage capacity.Signaling inclusion layer, major function are to receive the signaling number for being sent to come from acquisition layer convergence device
According to progress data cleansing is put in storage, and full dose signaling data is parsed according to specific format.
According to traditional Technical Architecture, signaling data is mainly aggregated into FTP (File Transfer in signal collecting layer
Protocol) on interface message processor (IMP), whenever there is new when being generated of signaling data file, signaling file will be connect by FTP
Mouth machine is transmitted to data sharing layer by FTP transport protocols, then re-reads the signaling file transmitted again and carries out data
The pretreatments such as cleaning.Due to the growth of mobile subscriber and the extension of communication service, signaling data can be increasing, and conventional method is difficult
To meet the needs of real-time processing, based on this, a kind of pretreatment for the signaling data that can effectively solve above-mentioned problem is now provided
Method, this method avoid secondary reading of the signaling file in preprocessing process, effectively save entire signaling data processing
Time.
Invention content
The technical assignment of the present invention is to be directed to more than shortcoming, provides a kind of highly practical, signaling data pretreatment
Method.
A kind of preprocess method of signaling data, specific implementation process are：
1）Monitor the newly-increased signaling file of the FTP interface message processor (IMP)s of signal collecting layer；
2）Signaling data is transferred to the Buffer of signaling inclusion layer：Signaling data is write first before being transferred to signaling inclusion layer
Enter into Buffer and keep in, then signaling data is transmitted using File Transfer Protocol；
3）Signaling data cleaning and conversion in Buffer：The newline for the signaling data kept in Buffer is matched, is obtained
Data line, field segmentation is carried out to each row of data, and it is clear to each row of data progress data then to obtain field cleaning transformation rule
It washes and conversion operation, the data after cleaning and conversion is then written to signaling inclusion layer local file；
4）Releasing memory Buffer, jumps to step 1）, continue to carry out the FTP interface message processor (IMP)s of signal collecting layer newly-increased file
Monitoring.
The step 1）Detailed process be：Whether new signaling text is had in timing monitoring signal collecting layer FTP interface message processor (IMP)s
Part generates, and if there is then obtaining newly-increased file name, signaling file of the same name, Ran Houqi are created in the storage of signaling inclusion layer
Dynamic data pick-up task.
The step 2）Detailed process be：
First, one piece of memory Buffer is opened up in signaling inclusion layer；
2nd, the newly-increased file in signal collecting layer FTP interface message processor (IMP)s is passed by the form of data flow by FTP transport protocols
In the defeated Buffer to memory, Buffer is filled up, then performs step 3）Data in Buffer are cleaned and converted；If
Buffer is unfilled, illustrates that the signaling file in signal collecting layer has been read and finishes, and then performs step 3）With step 4）；
3rd, it is finished when signal collecting layer signaling file is not read：It is obtained by way of finding newline and is stored in memory
A line signaling data in Buffer empties Buffer if the data in Buffer are discontented with a line, these is discontented with a line
Data be stored in Buffer again, then perform step 1, otherwise perform step 4；
The reading of signal collecting layer signaling file finishes：It is obtained and is stored in memory Buffer by way of finding newline
A line signaling data, if the data in Buffer meet a line, step 4 is performed, otherwise by the signaling data of less than a line
It is written in error log, then branches to step 4）；
4th, the data line of acquisition is divided into field according to field delimiter.
The step 3）Detailed process be：Obtain signaling data cleaning transformation rule, the data after dividing to field
Field cleaning and conversion are carried out, converts and the form of this line number file stream according to this is then successfully write into step 1）Newly-built signaling file
In, convert failed then writes error reason in error log；It finishes when signal collecting layer signaling file is not read, finally redirects
To step 3, step 4 is otherwise jumped to）.A kind of preprocess method of signaling data of the present invention, has the following advantages：
A kind of preprocess method of signaling data of the invention can shorten the flow of entire signaling pretreatment, raising processing
Efficiency；Signaling data is transferred to signaling inclusion layer from signal collecting layer and data preprocessing operation is carried out before file is written,
Secondary reading of the signaling file in preprocessing process is avoided, effectively saves the time of entire signaling data processing, practicability
By force, it is applied widely, it is easy to spread.
Description of the drawings
Attached drawing 1 realizes flow chart for present invention pretreatment.
Specific embodiment
The invention will be further described in the following with reference to the drawings and specific embodiments.
Traditional signaling file is transferred to signaling inclusion layer by the FTP interface message processor (IMP)s of signal collecting layer and then secondary read passes
The defeated signaling file to come, the operation cleaned and converted to signaling, the present invention propose a kind of pretreatment side of signaling data
Method, this method consider that signaling is transferred to signaling inclusion layer meeting new files stream from FTP interface message processor (IMP)s and leads to the data transmitted
The characteristics of file stream is written to file is crossed, is written to first in memory Buffer before file is written in data, to memory
Data in Buffer carry out the pretreatment such as cleaning, and the data in Buffer then are written to data sharing layer by file stream
File, this method can shorten the flow of entire signaling pretreatment, improve treatment effeciency.That is signaling data is from signal collecting layer
It is transferred to signaling inclusion layer and data preprocessing operation is carried out before file is written, avoid signaling file in preprocessing process
Secondary reading, effectively save time of entire signaling data processing.As shown in Figure 1, specific implementation process is：
1）Monitor the newly-increased signaling file of the FTP interface message processor (IMP)s of signal collecting layer；
2）Signaling data is transferred to the Buffer of signaling inclusion layer：Signaling data is write first before being transferred to signaling inclusion layer
Enter into Buffer and keep in, then signaling data is transmitted using File Transfer Protocol；
3）Signaling data cleaning and conversion in Buffer：The newline for the signaling data kept in Buffer is matched, is obtained
Data line, field segmentation is carried out to each row of data, and it is clear to each row of data progress data then to obtain field cleaning transformation rule
It washes and conversion operation, the data after cleaning and conversion is then written to signaling inclusion layer local file；
4）Releasing memory Buffer, jumps to step 1）, continue to carry out the FTP interface message processor (IMP)s of signal collecting layer newly-increased file
Monitoring.
Further, the detailed process of above-mentioned steps is：
1st, whether there is new signaling file generated in timing monitoring signal collecting layer FTP interface message processor (IMP)s, it is new if there is then obtaining
The file name of increasing creates signaling file of the same name in the storage of signaling inclusion layer, and then log-on data extracts task；
2nd, one piece of memory Buffer is opened up in signaling inclusion layer；
3rd, the newly-increased file in signal collecting layer FTP interface message processor (IMP)s is passed by the form of data flow by FTP transport protocols
In the defeated Buffer to memory, Buffer is filled up, step 4-6 is then performed and the data in Buffer is cleaned and converted；Such as
Fruit Buffer is unfilled, illustrates that the signaling file in signal collecting layer has been read and finishes, and then performs step 4-7；
4th, signal collecting layer signaling file, which is not read, finishes：It is obtained by way of finding newline and is stored in memory
A line signaling data in Buffer empties Buffer if the data in Buffer are discontented with a line, these is discontented with a line
Data be stored in Buffer again, then perform step 3, otherwise perform step 5；
The reading of signal collecting layer signaling file finishes：It is obtained and is stored in memory Buffer by way of finding newline
A line signaling data, if the data in Buffer meet a line, step 5 is performed, otherwise by the signaling data of less than a line
It is written in error log, then branches to step 7；
5th, the data line of acquisition is divided into field according to field delimiter；
6th, signaling data cleaning transformation rule is obtained, the data after dividing to field carry out field cleaning and conversion, turn
Work(is changed into then to write the form of this line number file stream according to this in the newly-built signaling file of step 1, convert failed then will mistake it is former
Because writing in error log；
It is finished if signal collecting layer signaling file is not read, finally jumps to step 4, otherwise jump to step 7；
7th, releasing memory Buffer jumps to step 1, continues to carry out the FTP interface message processor (IMP)s of signal collecting layer newly-increased file
Monitoring.
Above-mentioned specific embodiment is only the specific case of the present invention, and scope of patent protection of the invention includes but not limited to
Above-mentioned specific embodiment, it is any meet the present invention a kind of signaling data preprocess method it is claims and any
The appropriate change or replacement that the those of ordinary skill of the technical field does it should all fall into the patent protection model of the present invention
It encloses.
Claims (1)
1. a kind of preprocess method of signaling data, which is characterized in that its specific implementation process is：
Step 1）Monitor the newly-increased signaling file of the FTP interface message processor (IMP)s of signal collecting layer；Timing monitoring signal collecting layer FTP interface message processor (IMP)s
In whether have new signaling file generated, if there is then obtaining newly-increased file name, in the storage of signaling inclusion layer create
Signaling file of the same name, then log-on data extract task；
Step 2）Signaling data is transferred to the Buffer of signaling inclusion layer：Signaling data is write first before being transferred to signaling inclusion layer
Enter into Buffer and keep in, then signaling data is transmitted using File Transfer Protocol, and detailed process is：
Step 1: open up one piece of memory Buffer in signaling inclusion layer；
Step 2: the newly-increased file in signal collecting layer FTP interface message processor (IMP)s is passed by the form of data flow by FTP transport protocols
In the defeated Buffer to memory, Buffer is filled up, then performs step 3）Data in Buffer are cleaned and converted；If
Buffer is unfilled, illustrates that the signaling file in signal collecting layer has been read and finishes, and then performs step 3）With step 4）；
Step 3: it is finished when signal collecting layer signaling file is not read：It is obtained by way of finding newline and is stored in memory
A line signaling data in Buffer empties Buffer if the data in Buffer are discontented with a line, these is discontented with a line
Data be stored in Buffer again, then perform step 1, otherwise perform step 4；
The reading of signal collecting layer signaling file finishes：One be stored in memory Buffer is obtained by way of finding newline
Row signaling data if the data in Buffer meet a line, performs step 4, otherwise the signaling data of less than a line is written
Into error log, step 4 is then branched to）；
Step 4: the data line of acquisition is divided into field according to field delimiter；
Step 3）Signaling data cleaning and conversion in Buffer：The newline for the signaling data kept in Buffer is matched, is obtained
Data line, field segmentation is carried out to each row of data, and it is clear to each row of data progress data then to obtain field cleaning transformation rule
It washes and conversion operation, the data after cleaning and conversion is then written to signaling inclusion layer local file, detailed process is：
Signaling data cleaning transformation rule is obtained, the data after dividing to field carry out field cleaning and conversion, convert and successfully then will
The form of this line number file stream according to this writes step 1）In newly-built signaling file, error reason is then write mistake by convert failed
In daily record；It is finished when signal collecting layer signaling file is not read, finally jumps to step 3, otherwise jump to step 4）；
Step 4）Releasing memory Buffer, jumps to step 1）, continue to carry out the FTP interface message processor (IMP)s of signal collecting layer newly-increased file
Monitoring.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410791579.8A CN104507107B (en) | 2014-12-19 | 2014-12-19 | A kind of preprocess method of signaling data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410791579.8A CN104507107B (en) | 2014-12-19 | 2014-12-19 | A kind of preprocess method of signaling data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104507107A CN104507107A (en) | 2015-04-08 |
| CN104507107B true CN104507107B (en) | 2018-06-19 |
Family
ID=52948808
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410791579.8A Active CN104507107B (en) | 2014-12-19 | 2014-12-19 | A kind of preprocess method of signaling data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104507107B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430534B (en) * | 2015-10-29 | 2019-12-10 | Tcl集团股份有限公司 | method and system for reporting data by intelligent equipment |
| CN109561389A (en) * | 2018-12-28 | 2019-04-02 | 海南中智信信息技术有限公司 | A method of based on flow of the people real-time in mobile phone signaling zoning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769547A (en) * | 2011-05-03 | 2012-11-07 | 中国移动通信集团浙江有限公司 | System and method for providing signaling data |
| CN103346905A (en) * | 2013-06-14 | 2013-10-09 | 吴建进 | Method and device for analyzing signaling |
| CN103607329A (en) * | 2013-12-04 | 2014-02-26 | 北京中创信测科技股份有限公司 | IMS network any end-to-end media QoS monitoring method and system |
| CN203466974U (en) * | 2013-09-04 | 2014-03-05 | 河北烽联信息技术有限公司 | Equipment for mobile communication network signaling monitoring system |
-
2014
- 2014-12-19 CN CN201410791579.8A patent/CN104507107B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102769547A (en) * | 2011-05-03 | 2012-11-07 | 中国移动通信集团浙江有限公司 | System and method for providing signaling data |
| CN103346905A (en) * | 2013-06-14 | 2013-10-09 | 吴建进 | Method and device for analyzing signaling |
| CN203466974U (en) * | 2013-09-04 | 2014-03-05 | 河北烽联信息技术有限公司 | Equipment for mobile communication network signaling monitoring system |
| CN103607329A (en) * | 2013-12-04 | 2014-02-26 | 北京中创信测科技股份有限公司 | IMS network any end-to-end media QoS monitoring method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104507107A (en) | 2015-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102655481B (en) | A kind of instant messaging chat content inspection method based on webpage and system | |
| CN104572411B (en) | A kind of processing method and processing device of daily record data | |
| CN105653484B (en) | A kind of deblocking compression multi-channel transmission method | |
| CN106412861B (en) | A kind of short message distribution method and system | |
| CN106598889A (en) | SATA (Serial Advanced Technology Attachment) master controller based on FPGA (Field Programmable Gate Array) sandwich plate | |
| CN105676023A (en) | Recording method and system of fault recording data | |
| CN107104782A (en) | Data transmission method and device | |
| CN106951521A (en) | The reading/writing method of journal file, apparatus and system | |
| CN104507107B (en) | A kind of preprocess method of signaling data | |
| CN108132868A (en) | A kind of data monitoring method, device, computing device and storage medium | |
| CN103442156A (en) | Image steganography and secret information extraction method based on intelligent mobile terminal | |
| CN107135189A (en) | A kind of file transmitting method and physical machine | |
| JPWO2014178275A1 (en) | Transmitting apparatus, receiving apparatus, communication apparatus, program, transmitting method, and receiving method for wirelessly communicating continuous data by packet | |
| CN105337850A (en) | Internet of Things data processing method and internet of Things gateway | |
| CN106385568A (en) | Video monitoring system | |
| CN103558995B (en) | A kind of storage control chip and disk message transmitting method | |
| CN109213742A (en) | Log collection method and device | |
| CN108055594A (en) | Implementation method, device, computer equipment and the storage medium of edge section | |
| CN102404207B (en) | A kind of processing method and processing device of Ethernet data | |
| CN104410664A (en) | Full-duplex audio transmission system and method based on virtualized desktop environment | |
| CN204906525U (en) | Internet vidicon | |
| CN103442091A (en) | Data transmission method and device | |
| CN108173852A (en) | A kind of way of recording and system based on server | |
| CN105099745A (en) | Tracking method and device of business process | |
| CN108282266A (en) | Audio signal processing method, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |