KR20010053768A - Loading device with reduced memory consumption of row account data from packet network and method thereof - Google Patents

Abstract

PURPOSE: A memory low-consuming type loading apparatus for packet network source billing data and a method thereby are provided to load and cleanse source billing data in accordance with the purpose of a data warehouse. CONSTITUTION: A traffic data warehouse system is comprised of a data collection part(1), a source data table(2), an object table(3), a summary table(4) and a traffic information analysis part(5). The data collection part(1) collects source billing data. The source data table(2) stores the source billing data inputted through the data collection part(1). The object table(3) extracts information suitable for various analysis subjects from the source data table(2). The summary table(4) stores summary information extracted from the object table(3) in order to provide various analyses rapidly and precisely. The traffic information analysis part(5) executes various analyses for information stored in the object table(3) and the summary table(4) and provides an analytic authority to users through a graphic interface.

Description

Loading device with reduced memory consumption of row account data from packet network and method

The present invention relates to an apparatus for low memory consumption of a packet network raw charging data and a method thereof, and more particularly, to extract the information during the process of extracting various information by processing the charging data generated as a result of operating the packet network. An apparatus and method for efficiently loading billing data in order to make it available.

In recent years, as network technology is advanced and comprehensive service management is required, in order to perform comprehensive analysis on the performance and function of owned network, raw data generated from the network is gathered in one place from various perspectives. Data warehouse systems designed to enable processing analysis have emerged.

The technology in this field is to change the existing simple and individually managed individual network database data into a single place and store it together. Each detailed technology for manipulating the database can be used even if the conventional method is used. There was no set of algorithms for integrated management.

On the other hand, since most of the billing data has information on various events generated during the call progress in the unit of a call, a large amount of one record, and also a large amount of the total billing data generated.

Therefore, in order to extract processed information including traffic information from billing data, it is advantageous to obtain various information after loading raw billing data into a database and performing various extraction procedures.

In this process, a huge amount of raw billing data must be loaded into the database. However, as explained earlier, the amount of billing data is so large that a machine with sufficient processing power and storage capacity must be equipped for successful loading into the database, and additional storage space and processing speed to be added during billing data loading. Algorithm that can adjust properly is required.

In order to fit the packet network raw billing data into the contents of the traffic data warehouse system based on the above requirements, a large amount of data is automatically executed while using the DB Loader function provided by the DBMS. At the same time, the first cleansing should be performed to ensure that the data uploaded to the database management system is in the proper format and is within the range of appropriate values.

Looking at the related art similar to the above, there is an example in which billing data is loaded as an object table in a simple database rather than a data warehouse.

In such a case, the loading into the database is performed using the functions provided by the database, but a purification process for checking the suitability of the uploaded data is not provided. Sometimes there is a process of checking the range of data values in the process of checking the raw data format before loading, but it is different from the above-mentioned purification process to check the suitability of the raw table contents of the data warehouse mentioned above.

Accordingly, an object of the present invention is to implement an apparatus and method for loading and purifying raw billing data in accordance with the purpose of the data warehouse in view of the conventional problems and requirements. It provides a device and method that satisfies.

First, in order to become the contents of the raw table of the data warehouse, it is necessary to check the value of each loaded billing data column and the range of values that the data value can have according to the logical relationship with other columns. There is a confirmation process.

In addition, since it processes a large amount of data, there are many cases where the resources of the system including memory are insufficient. Therefore, the loading of the system can be performed while the memory of the system is properly adjusted so that an error occurs during the loading process due to the exhaustion of system resources. An algorithm that can be used is required.

1 is a block diagram of a packet network charging data loading system according to the present invention.

2 is a flowchart illustrating a process of loading packet network charging data according to the present invention.

<Description of Symbols for Major Parts of Drawings>

1: data collector 2: raw table

3: objective table 4: summary table

5: Information Analysis Department

The memory low consumption type loading device for packet network raw billing data of the present invention in view of the above objects and requirements includes: a data collector for collecting raw billing data;

A raw data table for storing raw data input through the data collecting unit;

An object table for extracting and storing information suitable for various analysis subjects including traffic information for each analysis subject from the raw data table;

A summary table for storing summary information extracted from the object table to provide various analyzes quickly and accurately;

And a traffic information analyzer configured to execute various analyzes on the information stored in the object table and the summary table to provide a user with analysis authority through a graphic interface.

In addition, the low-cost memory loading method of the packet network raw billing data of the present invention for achieving the above object comprises a first step of receiving a list of raw billing files from the user to configure a series of automatic execution tasks;

Dividing the input raw billing files into minimum units, converting the input raw billing files into a format that can be read by a database loader, and loading the raw billing tables in a raw billing table;

A third step of recording a loading result and time required for the billing file in a log table;

And a fourth process of removing a record in which a duplication has occurred in the data group loaded in the raw billing table in the raw data table or a logical error in a range of data values.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a packet network charging data loading system according to the present invention. In the following description, the packet network charging data loading system is referred to as a traffic data warehouse system.

As shown in the figure, a data collector 1 for collecting raw billing data;

A raw data table (2) for storing raw data input through the data collecting unit (1);

An objective table (3) for extracting and storing information suitable for various analysis subjects including traffic information for each analysis subject from the raw data table (21);

A summary table (4) for storing summary information extracted from the object table (3) to provide various analyzes quickly and accurately;

A traffic information analyzer 5 is provided for executing various analyzes on the information stored in the object table 3 and the summary table 4 to provide the user with analysis authority through a graphic interface.

In detail, the data collection unit 1 is an operation site of a network and generates and collects traffic statistics data and charging data generated in each operation system.

In addition, the raw data table (2) is a table class and data in the table mounted on the data warehouse system in the form of a table having a series of naming conventions on a single machine the raw data generated by the data collection unit (1) Includes the brackets.

In addition, the destination table (3) is extracted data tables obtained by extracting the traffic information elements to be analyzed from the raw data table (2), the summary table (4) is to abbreviate and store the contents of the destination table (3) Are the tables.

In addition, the traffic information analyzer 5 retrieves the data in the object table 3 or the summary table 4 using an external analysis tool and processes the data into the analysis information requested by the end-user. The result is a graphical interface that allows you to derive back-data for making decisions.

An operation process of the traffic data warehouse system implemented as described above will be described with reference to FIG. 2.

2 is a flowchart illustrating a process of loading raw billing data of a packet network into a data warehouse system according to an embodiment of the present invention. The list of raw billing files to be uploaded from the data collector 1 to the raw data table 2 is shown in FIG. In response to the input, a series of automatic execution tasks are configured (S1).

Subsequently, it repeats in one billing file unit, cuts and reads the contents of the raw billing file to be uploaded in a non-large unit (S2), and checks whether the range of the data value is in the normal range and converts it to the contents of the raw data table part. It stores the data as a temporary data file having a form that can be read and recognized by the database loader (DB Loader) (S3).

The temporary data file is then loaded into a raw billing table (not shown in the figure) of the data warehouse system using a database loader, wherein the raw billing table is a target table of the raw data table 2, and the raw data Since the amount of data to be stored in the table (2) is enormous, a partial value can be set so that the total number of records to be stored are divided and stored at a constant rate. It is to be partitioned to use (S4).

Then, after the steps (S2) to (S4) are completed until the end of the billing file (End Of File), the number of records loaded, the number of data records excluded from data format conversion, the number of records uploaded as normal data and each step The time required is recorded in the log table (S5).

Subsequently, while continuously loading the target raw billing file to be loaded into the target table, the process checks whether the currently loaded billing file is the last file, and if not the last file, repeats from step S2 of checking the data value range of the billing file. (S6).

After all of the input billing files are processed (S2) to (S6), the contents loaded into the raw billing table in the raw data table 2 of the data warehouse system are checked for duplication. The data record and the logically defective record are removed (S7).

As described in detail above, the present invention is to analyze the characteristics and performance of the packet communication network by efficiently loading the underlying data into the data warehouse system so that various analysis information including the traffic data of the packet network can be retrieved. It has advantages that can be applied for the purpose of analyzing resource trends and forecasting.

In addition, preferred embodiments of the present invention are disclosed for the purpose of illustration, those skilled in the art will be able to various modifications, changes, additions, etc. within the spirit and scope of the present invention, such modifications and modifications belong to the following claims You will have to look.

Claims (5)

A data collecting unit collecting raw charging data; A raw data table for storing raw data input through the data collecting unit; An object table for extracting and storing information suitable for various analysis subjects including traffic information for each analysis subject from the raw data table; A summary table for storing summary information extracted from the object table to provide various analyzes quickly and accurately; And a traffic information analyzer configured to execute various analyzes on the information stored in the object table and the summary table to provide a user with analysis authority through a graphic interface. The method of claim 1, The raw data table includes a plurality of raw billing tables, wherein the raw billing table is a target table of a raw data table, and sets a split value for dividing a total of the records to be accommodated in the raw data table by a constant ratio. A low-power memory loading device for packet network raw billing data, characterized in that it is a unit table partitioned with values as delimiters. A first step of receiving a list of raw billing files from a user and configuring a series of automatic execution tasks; Dividing the input raw billing files into minimum units, converting the input raw billing files into a format that can be read by a database loader, and loading the raw billing tables in a raw billing table; A third step of recording a loading result and time required for the loaded billing file in a log table; And a fourth process of removing a record in which a duplication has occurred in the data group loaded in the raw billing table in the raw data table or a logical error in a range of data values. Way. The method of claim 1, In the second process, the raw data table includes a plurality of raw billing tables, wherein the raw billing table is a target table of the raw data table, and sets a split value for dividing records to be totally accommodated in the raw data table by a constant ratio. And a unit table partitioned by using the partition value as a delimiter. The method of claim 1, The information recorded in the log table in the third process includes the number of charge records in each of the loaded raw files, the number of records removed due to an error in the range of the format and data values among all the loaded charge records, and raw data as normal records. And a record number inserted into a billing table, and information on a start time and a end time of each process.