CN102999602A - Data parsing method and system - Google Patents
Data parsing method and system Download PDFInfo
- Publication number
- CN102999602A CN102999602A CN201210472687XA CN201210472687A CN102999602A CN 102999602 A CN102999602 A CN 102999602A CN 201210472687X A CN201210472687X A CN 201210472687XA CN 201210472687 A CN201210472687 A CN 201210472687A CN 102999602 A CN102999602 A CN 102999602A
- Authority
- CN
- China
- Prior art keywords
- data
- data analysis
- identifier
- parsing
- analytical function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Stored Programmes (AREA)
Abstract
The invention relates to a data parsing method, which comprises the following steps of: 1, defining a basic parsing function for parsing and processing data for each data according to storage rules of the data; 2, defining a data parsing identifier for each basic parsing function; 3, establishing a mapping relation between each data parsing identifier and each basic parsing function which correspond to each other; 4, organizing the basic parsing function, the data parsing identifier and the mapping relation which correspond to each data in a mode of a dynamic link library; and 5, configuring the data parsing identifiers required by data parsing according to the data parsing identifiers and the mapping relations between the parsing identifiers and the basic parsing functions, and calling the basic parsing functions according to information of the configured data parsing identifiers, thus completing the parsing of the data. By adopting the method, the quick and flexible parsing of complex diverse mass telecommunication data can be realized; and the method is quickly supported through simple configuration and can be multiplexed, so that the program is high in flexibility.
Description
Technical field
The present invention relates to the Data Preprocessing Technology field of telecommunications industry, particularly relate to a kind of data analysis method and data analyzing system.
Background technology
Along with the telecommunications industry development, the data storage method of various complexity has appearred in the data volume explosive growth, to becoming increasingly complex that the dissection process of these various data becomes, supports more and more flexibly and fast when requiring to process resolution data.
Carry out summarizes for the available data analysis mode, exist the resolution rules extent for multiplexing not high, configurable degree is low, the defectives such as the processing resolution data is dumb, therefore take out the Data Analysis mode on basis, as the data mode accumulation, the analysis mode set is provided, and convenient, flexible fast newly-increased new analysis mode.
Summary of the invention
Technical matters to be solved by this invention provides a kind of data analysis method and data analyzing system, is used for solving the problems such as not high, the configurable degree of prior art Data Analysis rule reusing degree is low, processing is dumb.
The technical scheme that the present invention solves the problems of the technologies described above is as follows: a kind of data analysis method comprises:
Step 1 according to the data storage rule, is basic analytical function that is used for the dissection process data of each data definition;
Step 2 is resolved data of function definition for each basis and is resolved identifier;
Step 3 is set up corresponding Data Analysis identifier and the mapping relations of basic analytical function;
Step 4 is organized the corresponding basic analytical function of each data, Data Analysis identifier and mapping relations in the mode of dynamic link library;
Step 5, the Data Analysis identifier that establishes according to Data Analysis identifier and the step 4 of step 2 definition and the mapping relations of basic analytical function, configuration data is resolved needed Data Analysis identifier, and according to the configuration the Data Analysis identifier information call basic analytical function, the parsing of complete paired data.
On the basis of technique scheme, the present invention can also do following improvement.
Further, carry out before the described step 1, also comprise: if data have had corresponding basic analytical function, direct execution in step 5 then.
Further, described step 1 also comprises: if a data correspondence has a plurality of storage rules, then its each storage rule is all defined corresponding basic analytical function.
Further, described step 2 also comprises: if there has been the Data Analysis identifier in basic analytical function, and the upright and outspoken execution in step 5 that connects.
Further, configuration data is resolved needed Data Analysis identifier and is comprised in the described step 5: the Data Analysis identifier information of configuration is write in the configuration template of described basic analytical function, and after in a single day configuring identifier defines, can be used as the flexible repeated configuration of a kind of Basic Solutions analysis method uses, namely after the definition once, later in case need to use this to resolve analytical function corresponding to sign, just can not need to define again, enter the analytical function configuration template as long as will resolve identifier configurations.
Further, configuration data is resolved needed Data Analysis identifier and is comprised in the described step 5: to code tag value of each data configuration.
Further, can function call be resolved to the optional position in the basis of its correspondence by the Data Analysis identifier in the described data analysis method.
A kind of data analyzing system comprises:
The analytical function definition module, it is used for according to the data storage rule, is basic analytical function that is used for the dissection process data of each data definition;
Resolve the identifier definition module, it is used to each basis in the described analytical function definition module to resolve a data parsing of function definition identifier;
Mapping relations are set up module, and it is used for setting up corresponding Data Analysis identifier and the mapping relations of basic analytical function;
Link module, it is used for organizing the corresponding basic analytical function of each data, Data Analysis identifier and mapping relations in the mode of dynamic link library;
Information preparation module, it is used for configuration data and resolves identifier;
Data resolution module, it is used for calling basic analytical function, the parsing of complete paired data according to the Data Analysis identifier information of described information collocation module configuration.
The invention has the beneficial effects as follows: the present invention can realize the parsing to the fast and flexible of complicated and diversified telecommunications mass data, supports fast and reusable by easy configuration, makes calling program that very high dirigibility be arranged.
Description of drawings
Fig. 1 is the schematic flow sheet of data analysis method of the present invention;
Fig. 2 is the structural representation of data analyzing system of the present invention.
Embodiment
Below in conjunction with accompanying drawing principle of the present invention and feature are described, institute gives an actual example and only is used for explaining the present invention, is not be used to limiting scope of the present invention.
As shown in Figure 1, embodiment one is a kind of data analysis method, comprising:
Step 1 according to the data storage rule, is basic analytical function that is used for the dissection process data of each data definition.If a data correspondence has a plurality of storage rules, then its each storage rule is all defined corresponding basic analytical function.
Step 2 is resolved data of function definition for each basis and is resolved identifier.
For step 1 and step 2, if data have had corresponding basic analytical function, direct execution in step 5 then; If there has been the Data Analysis identifier in basic analytical function, the upright and outspoken execution in step 5 that connects.
Step 3 is set up corresponding Data Analysis identifier and the mapping relations of basic analytical function;
Step 4 is organized the corresponding basic analytical function of each data, Data Analysis identifier and mapping relations in the mode of dynamic link library;
Step 5, the Data Analysis identifier that establishes according to Data Analysis identifier and the step 4 of step 2 definition and the mapping relations of basic analytical function, configuration data is resolved needed Data Analysis identifier, and according to the configuration the Data Analysis identifier information call basic analytical function, the parsing of complete paired data.Here, configuration data is resolved identifier and specifically comprised: just the Data Analysis identifier information of configuration writes in the configuration template of described basic analytical function, and after in a single day configuring identifier defines, can be used as the flexible repeated configuration of a kind of Basic Solutions analysis method uses, namely after the definition once, this resolves analytical function corresponding to sign in case need use later on, just can not need to define again, enters the analytical function configuration template as long as will resolve identifier configurations.Can function call be resolved to the optional position in addition in the basis of its correspondence by the Data Analysis identifier in the described data analysis method, before configuration data is resolved identifier to code tag value of each data configuration.
Embodiment one has also set up a kind of corresponding data analyzing system according to said method, comprising:
The analytical function definition module, it is used for according to the data storage rule, is basic analytical function that is used for the dissection process data of each data definition;
Resolve the identifier definition module, it is used to each basis in the described analytical function definition module to resolve a data parsing of function definition identifier;
Mapping relations are set up module, and it is used for setting up corresponding Data Analysis identifier and the mapping relations of basic analytical function;
Link module, it is used for organizing the corresponding basic analytical function of each data, Data Analysis identifier and mapping relations in the mode of dynamic link library;
Information preparation module, it is used for configuration data and resolves identifier;
Data resolution module, it is used for calling basic analytical function, the parsing of complete paired data according to the Data Analysis identifier information of described information collocation module configuration.
Embodiment two: adopt data analysis method of the present invention and data analyzing system that string of binary characters is resolved to ascii code character string, as binary one 0101011 being resolved to character string " AB ".
The first step: define basic analytical function: readAsHex, binary data can be resolved to the ASCII character data by input parameter.
Second step: definition data field title: servedMsisdn.
The 3rd step: configuration data code tag value: 9D.
The 4th step: configuration data analytical function sign: decode_func:readAsHex.
The 5th step: according to the Data Analysis function identification of configuration, call basic analytical function by the mode of dynamic link library, be that the data content 10101011 of 9D is resolved to as a result that AB outputs in the field that field name is msisdn with label value in the data, and store in the mode of ASCII character;
Concrete parsing configuration mode is as follows:
<field?init_field_name="servedMsisdn?"tag="9D"decode_func="readAsHex"field_name="msisdn"/>
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (8)
1. a data analysis method is characterized in that, comprising:
Step 1 according to the data storage rule, is basic analytical function that is used for the dissection process data of each data definition;
Step 2 is resolved data of function definition for each basis and is resolved identifier;
Step 3 is set up mutually corresponding Data Analysis identifier and the mapping relations of basic analytical function;
Step 4 is organized the corresponding basic analytical function of each data, Data Analysis identifier and mapping relations in the mode of dynamic link library;
Step 5, the Data Analysis identifier that establishes according to Data Analysis identifier and the step 4 of step 2 definition and the mapping relations of basic analytical function, configuration data is resolved needed Data Analysis identifier, and according to the configuration the Data Analysis identifier information call basic analytical function, the parsing of complete paired data.
2. data analysis method according to claim 1 is characterized in that, carries out before the described step 1, also comprises: if data have had corresponding basic analytical function, and direct execution in step 5 then.
3. data analysis method according to claim 1 is characterized in that, described step 1 also comprises: if a data correspondence has a plurality of storage rules, then its each storage rule is all defined corresponding basic analytical function.
4. data analysis method according to claim 1 is characterized in that, described step 2 also comprises: if there has been the Data Analysis identifier in basic analytical function, and the upright and outspoken execution in step 5 that connects.
5. data analysis method according to claim 1 is characterized in that, configuration data is resolved needed Data Analysis identifier and comprised in the described step 5: the Data Analysis identifier information of configuration is write in the configuration template of described basic analytical function.
6. data analysis method according to claim 1 is characterized in that, configuration data also comprises before resolving needed Data Analysis identifier in the described step 5: to code tag value of each data configuration.
7. data analysis method according to claim 1 is characterized in that, can function call be resolved to the optional position in the basis of its correspondence by the Data Analysis identifier in the described data analysis method.
8. a data analyzing system is characterized in that, comprising:
The analytical function definition module, it is used for according to the data storage rule, is basic analytical function that is used for the dissection process data of each data definition;
Resolve the identifier definition module, it is used to each basis in the described analytical function definition module to resolve a data parsing of function definition identifier;
Mapping relations are set up module, and it is used for setting up corresponding Data Analysis identifier and the mapping relations of basic analytical function;
Link module, it is used for organizing the corresponding basic analytical function of each data, Data Analysis identifier and mapping relations in the mode of dynamic link library;
Information preparation module, it is used for configuration data and resolves needed Data Analysis identifier;
Data resolution module, it is used for calling basic analytical function, the parsing of complete paired data according to the Data Analysis identifier information of described information collocation module configuration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210472687XA CN102999602A (en) | 2012-11-20 | 2012-11-20 | Data parsing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210472687XA CN102999602A (en) | 2012-11-20 | 2012-11-20 | Data parsing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102999602A true CN102999602A (en) | 2013-03-27 |
Family
ID=47928170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210472687XA Pending CN102999602A (en) | 2012-11-20 | 2012-11-20 | Data parsing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102999602A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750758A (en) * | 2013-12-31 | 2015-07-01 | 深圳航天东方红海特卫星有限公司 | Micro satellite test data packet general analysis processing method and system |
| CN109450922A (en) * | 2018-11-29 | 2019-03-08 | 厦门科灿信息技术有限公司 | A kind of communication data analytic method, device and relevant device |
| CN111933270A (en) * | 2020-07-17 | 2020-11-13 | 浙江理工大学 | Mobile medical data acquisition and transmission system based on Internet of things |
| CN112468721A (en) * | 2020-11-19 | 2021-03-09 | 江汉大学 | Visual acquisition method and device with automatic focusing function |
| CN114564253A (en) * | 2022-03-02 | 2022-05-31 | 重庆紫光华山智安科技有限公司 | Task creation method, system, electronic device and readable storage medium |
| CN115203180A (en) * | 2022-05-16 | 2022-10-18 | 北京航空航天大学 | Data blood relationship generation method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1366397A (en) * | 2001-01-19 | 2002-08-28 | 吴江洪 | Method and device for in-line real-time charge |
| CN101616174A (en) * | 2009-07-09 | 2009-12-30 | 浪潮电子信息产业股份有限公司 | A kind of storage system IO handles the method that the path dynamic tracking realizes the optimization system performance |
| CN101710984A (en) * | 2009-12-04 | 2010-05-19 | 深圳创维数字技术股份有限公司 | Digital television program processing method based on PID reconstruction |
| CN102208998A (en) * | 2011-06-08 | 2011-10-05 | 中国人民解放军国防科学技术大学 | Field programmable gate array (FPGA)-based common object request broker architecture (CORBA) communication device |
-
2012
- 2012-11-20 CN CN201210472687XA patent/CN102999602A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1366397A (en) * | 2001-01-19 | 2002-08-28 | 吴江洪 | Method and device for in-line real-time charge |
| CN101616174A (en) * | 2009-07-09 | 2009-12-30 | 浪潮电子信息产业股份有限公司 | A kind of storage system IO handles the method that the path dynamic tracking realizes the optimization system performance |
| CN101710984A (en) * | 2009-12-04 | 2010-05-19 | 深圳创维数字技术股份有限公司 | Digital television program processing method based on PID reconstruction |
| CN102208998A (en) * | 2011-06-08 | 2011-10-05 | 中国人民解放军国防科学技术大学 | Field programmable gate array (FPGA)-based common object request broker architecture (CORBA) communication device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750758A (en) * | 2013-12-31 | 2015-07-01 | 深圳航天东方红海特卫星有限公司 | Micro satellite test data packet general analysis processing method and system |
| CN104750758B (en) * | 2013-12-31 | 2018-07-03 | 深圳航天东方红海特卫星有限公司 | A kind of general parsing processing method and system of microsatellite test data bag |
| CN109450922A (en) * | 2018-11-29 | 2019-03-08 | 厦门科灿信息技术有限公司 | A kind of communication data analytic method, device and relevant device |
| CN109450922B (en) * | 2018-11-29 | 2021-09-21 | 厦门科灿信息技术有限公司 | Communication data analysis method, device and related equipment |
| CN111933270A (en) * | 2020-07-17 | 2020-11-13 | 浙江理工大学 | Mobile medical data acquisition and transmission system based on Internet of things |
| CN112468721A (en) * | 2020-11-19 | 2021-03-09 | 江汉大学 | Visual acquisition method and device with automatic focusing function |
| CN114564253A (en) * | 2022-03-02 | 2022-05-31 | 重庆紫光华山智安科技有限公司 | Task creation method, system, electronic device and readable storage medium |
| CN114564253B (en) * | 2022-03-02 | 2023-06-09 | 重庆紫光华山智安科技有限公司 | Task creation method, system, electronic device and readable storage medium |
| CN115203180A (en) * | 2022-05-16 | 2022-10-18 | 北京航空航天大学 | Data blood relationship generation method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102999602A (en) | Data parsing method and system | |
| CN108664239B (en) | Cross-technology-stack web front-end development system and method based on micro-service | |
| CN104281601B (en) | Build method and device, character display method and the device of super large character library | |
| CN102750268A (en) | Object serializing method as well as object de-serializing method, device and system | |
| CN108647010B (en) | Project engineering initialization method, terminal equipment and storage medium | |
| CN102647414A (en) | Protocol analysis method, protocol analysis device and protocol analysis system | |
| CN106970820A (en) | Code storage method and code storage | |
| CN102043626A (en) | Method and device for encapsulating extjs component, and method and system for generating user interface | |
| CN103309806B (en) | The device and method of a kind of quick development and testing | |
| CN105760184A (en) | Method and device for loading component | |
| CN111241182A (en) | Data processing method and apparatus, storage medium, and electronic apparatus | |
| CN110502520B (en) | Method, system and equipment for data warehousing and computer readable storage medium | |
| CN102622237B (en) | A kind of collocation method of business function procedure and system | |
| CN100585561C (en) | Method for clipping relocatable ELF files in embedded system | |
| CN103714116A (en) | Webpage information extracting method and webpage information extracting equipment | |
| CN105407138A (en) | Configuration file loading method and apparatus, and communication equipment | |
| CN105005543A (en) | Interface generation apparatus and method | |
| CN108664546B (en) | XML data structure conversion method and device | |
| CN108885544B (en) | Front-end page internationalized processing method, application server and computer-readable storage medium | |
| CN102203734A (en) | Conditional processing method and apparatus | |
| CN102880690A (en) | Method for simplifying character library of electronic book, and processing device for electronic book | |
| CN101930362A (en) | Method for integrating TTCN language and data description language | |
| CN105389180A (en) | Android platform-based USB (Universal Serial Bus) port configurable method | |
| US8656371B2 (en) | System and method of report representation | |
| CN104753891A (en) | XML (Extensive Markup Language) message analyzing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130327 |
|
| RJ01 | Rejection of invention patent application after publication |