KR101533719B1 - Realtime data analysis apparatus based on streaming - Google Patents

Abstract

The present invention relates to an apparatus and method for analyzing data in real time based on streaming technique, capable of analyzing a large amount of log data related to calls and data services which are collected from a communication network in real time based on a streaming data processing technology, to calculate statistical data for managing the service quality in real time. More particularly, a log data stream, which is transmitted in a streaming scheme, is received and sequentially buffered, and then a plurality of stored log data streams are sequentially loaded and processed through a data workflow consisting of one stream input module, and at least one processing module each for performing a predetermined data processing operation, to calculate statistical data for managing the service quality in real time.

Description

Technical Field [0001] The present invention relates to a streaming-based real-

The present invention relates to a real-time data analysis technique for real-time quality monitoring of a call service and a data service over a communication network, and more particularly, Stream data analysis apparatus and method capable of receiving statistical data by analyzing the received log data stream in real time.

The first generation mobile communication technology such as the Advanced Mobile Phone System (AMPS) of the 1980s has emerged as a mobile communication technology that enables a user to use a communication service and a data communication service, It has been developed to 2.9 generation LTE (Long Term Evolution) and 4th generation LTE-A (LTE-Advanced) through CDMA, GSM, 3G CDMA-2000 and WCDMA.

Recently, LTE and LTE-A, which are recently used, are mobile communication standards of WCDMA series proposed by 3rd Generation Partnership Project (3GPP). They are all-IP based solutions and include ultra-wideband Internet access, And streaming multimedia.

In addition, a high-end user device having a powerful processor, a large memory and a large screen like a smart phone is popularly used, and multimedia services such as VoD, real-time broadcasting, and network-based games are increasingly used, Data traffic volume is growing rapidly.

In particular, as smartphones support standardized interfaces and platforms for application developers, the use of various application programs developed by third party application developers other than mobile communication service providers is increasing rapidly, The amount of data traffic generated by the network is increasing rapidly.

Meanwhile, in providing a call service and a data communication service based on a mobile communication network, monitoring of status and traffic of each equipment in the mobile communication network is required to ensure stable service quality.

For this purpose, in the past, log data is periodically collected and stored from each device constituting a mobile communication network such as an eNB, an MME, a serving gateway and a PDN gateway through a network management system (NMS: Network Management System) (For example, KPI: Key Performance Indicator) for the service quality monitoring and stores it in a relational database (RDB). The manager can inquire the calculated indicators or monitor the service quality through visualized indicators and analyze the cause of the quality degradation.

However, since the existing service quality management system collects log data from network devices such as eNB, MME, serving gateway and PDN gateway, there is a problem that the data collecting part must be changed according to the change of the log format according to the equipment vendor.

In addition, due to the difference in the generation period of the log data generated for each equipment, there may be a delay in collecting and storing the log data and re-analyzing and processing the stored log data, and the load of the centralized relational database A delay can also occur.

As described above, since the conventional service quality management technology necessarily generates a delay in the process of collecting, storing and analyzing log data, it is impossible to calculate statistical data in real time such as indicators for monitoring the quality of service, As a result, there is a problem that it is impossible to analyze and respond in real time to the degradation of service quality.

Particularly, in the existing service quality management system, once the collected log data is stored in the file system, the log data stored for a predetermined period is processed in a batch process in units of files, Real-time processing is impossible.

Korean Patent Laid-Open No. 10-2007-0120257, December 24, 2007 (Name: Quality Analysis System and Analysis Method for Real-Time Service in Next Generation Network)

Accordingly, the present invention provides a streaming service capable of analyzing a large amount of log data related to a call and data service collected in real time from a communication network including a mobile communication network based on streaming data processing technology, Based real-time data analysis apparatus and method.

In particular, the present invention receives a log data stream generated based on a packet transmitted through a communication network rather than a network device and transmitted in a streaming manner, sequentially buffers the stream data stream, and then performs a predetermined data process Based real-time data analysis apparatus and method capable of calculating statistical data for service quality management in real time by successively reading and sequentially processing a plurality of log data streams stored through a data workflow comprising at least one processing module .

As a means for solving the above-mentioned problems, the present invention provides a mobile communication system including at least one data collection unit for receiving a plurality of log data streams generated in real time while providing a call service and a data service through a communication network, in a streaming manner; One or more buffering units for sequentially distributing and storing a plurality of log data streams received by the one or more data collecting units and outputting a log data stream in a stored order in response to a read request; And a controller for sequentially processing the log data streams stored in the one or more buffering units and parallelly processing the log information included in the log data stream to calculate statistical data for the call service and the data service, A streaming-based real-time data analysis apparatus including the above data analysis unit is provided.

According to an embodiment of the present invention, in the real-time data analysis apparatus, the at least one data analysis unit analyzes call related log information included in the log data stream collected in association with the call service, And a second data workflow for analyzing traffic related log information included in the log data stream collected in association with the data service and calculating statistical data related to the data service quality, .

Wherein the first data workflow sequentially reads a log data stream related to a call service stored in a corresponding buffering unit and reads log data included in the log data stream on a predetermined basis from one of a plurality of first data workflows A stream input module for distributing the stream to a grouping processing module; One or more pieces of log data transmitted from any one of the plurality of first data workflows are grouped based on identification information of the user device and transmitted to one of the plurality of first data workflows A grouping processing module; An analysis processing module for analyzing a correlation between log data grouped based on identification information of a specific user apparatus transmitted from any one of the plurality of first data workflows; And at least one statistical processing module for calculating statistical data related to call service quality by integrating the analysis results of the analysis processing module step by step.

The second data workflow sequentially reads log data streams related to the data service stored in the corresponding buffering unit and sequentially reads the log data included in the log data stream on the basis of a predetermined one of the plurality of second data workflows A stream input module for distributing to one statistical processing module; One or more pieces of log data transmitted from any one of the plurality of second data workflows are integrated with the statistical processing modules of the second data workflow and stepwise integrates one or more field values based on one or more field values Statistical processing module; And at least one grouping processing module for grouping the results integrated through the at least one statistical processing module stepwise on the basis of one or more field values and outputting statistical data related to the data service quality.

In addition, the present invention provides, as another solution to the above-mentioned problems, a method for receiving a plurality of log data streams in real time while providing a call service and a data service through a communication network; Sequentially distributing and storing a plurality of received log data streams; A method for processing statistical data for a call service and a data service by processing log information contained in a corresponding log data stream in parallel while sequentially reading the plurality of distributed log data streams in a stored order through one or more data workflows Based real-time data analysis method comprising the steps of:

According to an embodiment of the present invention, in the streaming-based real-time data analysis method, the calculating step may include sequentially reading log data streams related to a call service among a plurality of log data streams distributed and stored, Distributing log data included in the data stream; Comparing the field values of the distributed log data and grouping based on the identification information of the user device; Analyzing a correlation between the grouped log data; And correlating the correlation analysis results step by step to calculate statistical data related to the call service quality.

According to an embodiment of the present invention, in the streaming-based real-time data analysis method, the calculating step includes sequentially reading log data streams related to a data service among a plurality of log data streams distributed and stored, Distributing log data included in the log data stream; Stepwise integrating the distributed log data; And calculating statistical data related to data service quality by grouping the integrated results stepwise based on one or more field values.

In addition, the present invention provides, as still another object of the present invention, to provide a method and apparatus for parallel processing a plurality of log data streams generated in real time while providing a call service through one or more data workflows, Wherein the data workflow comprises: a stream input module for sequentially reading a log data stream related to a call service and distributing log data included in the log data stream to a grouping processing module among a plurality of data work flows on a predetermined basis; A grouping processing module for grouping one or more pieces of log data transmitted from any one of the plurality of data workflows on the basis of identification information of a user apparatus and transmitting the grouped data to any one of the plurality of data processing workflows; ; An analysis processing module for analyzing a correlation between log data grouped on the basis of identification information of a specific user device transmitted from any one of the plurality of data workflows; And at least one statistical processing module for calculating statistical data related to call service quality by integrating analysis results of the analysis processing module step by step.

In addition, the present invention provides, as another solution to the problem, a method for processing a plurality of log data streams generated in real time while providing a data service through one or more data workflows, Wherein the data workflow comprises: a stream input module for sequentially reading a log data stream related to a data service and distributing the log data included in the log data stream to a statistical processing module among a plurality of data workflows on a constant basis; One or more statistical processing modules for integrating one or more pieces of log data transmitted from any one of the plurality of data workflows with the statistical processing modules of other data workflows and integrating them in accordance with one or more field values; And at least one grouping processing module for grouping the results integrated through the at least one statistical processing module stepwise on the basis of one or more field values to output statistical data related to data service quality. Thereby providing an analysis apparatus.

According to the present invention, in real-time quality control of a call service and a data service provided through a predetermined communication network, a plurality of log data generated in real time during a call service and a data service are received in a streaming manner, The received log data stream can be sequentially processed without a large delay and the statistical data can be calculated in real time.

In particular, the present invention relates to a method and apparatus for sequentially buffering received log data streams, and then sequentially storing a plurality of log data streams stored in a data workflow comprising one stream input module and one or more processing modules each performing predetermined data processing And the delay is minimized by processing stepwise, thereby enabling the statistical data to be calculated in real time.

Also, according to the present invention, call service-related log data and data service-related log data are separately divided and processed through different data workflows so that statistical data required by each of the call service and the data service can be calculated.

1 is a diagram illustrating a network structure to which a streaming-based real-time data analysis apparatus according to the present invention is applied.
2 is a block diagram illustrating a streaming-based real-time data analysis apparatus according to the present invention.
3 is a flowchart illustrating a streaming-based real-time data analysis method according to the present invention.
4 is a block diagram illustrating a detailed configuration of a first data processing workflow for calculating statistical data related to call service quality in a streaming-based real-time data analyzing apparatus according to the present invention.
5 is a block diagram illustrating a detailed configuration of a second data processing workflow for calculating statistical data related to data service quality in a streaming-based real-time data analyzing apparatus according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings and the inventor is not limited to the concept of terminology for describing his or her invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

Also, terms including ordinal numbers such as first, second, etc. are used to describe various elements, and are used only for the purpose of distinguishing one element from another, Not used. For example, without departing from the scope of the present invention, the second component may be referred to as a first component, and similarly, the first component may also be referred to as a second component.

In addition, when referring to an element as being "connected" or "connected" to another element, it means that it can be connected or connected logically or physically. In other words, it is to be understood that although an element may be directly connected or connected to another element, there may be other elements in between, or indirectly connected or connected.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. It is also to be understood that the terms such as " comprising "or" having ", as used herein, are intended to specify the presence of stated features, integers, It should be understood that the foregoing does not preclude the presence or addition of other features, numbers, steps, operations, elements, parts, or combinations thereof.

The real-time data analyzing apparatus according to the present invention is a real-time data analyzing apparatus that can connect a call channel between two or more callers to transmit call data including at least one of voice and images between callers in real time, And / or < / RTI >

The communication network to which the real-time data analyzing apparatus according to the present invention can be applied can be any communication network that transmits various data including call data in a packet system. An example of a communication network to which the present invention is applied is a mobile communication network such as LTE or LTE-A.

Hereinafter, the structure and operation of the real-time data analyzing apparatus according to the present invention will be described by taking a mobile communication network as an example for convenience of explanation. However, the real-time data analyzing apparatus according to the present invention is not limited to the mobile communication network, but can be applied to any type of communication network provided that it provides one of a call service and a data service on a packet basis.

1 is a diagram illustrating an example of a communication network to which a streaming-based real-time data analyzing apparatus according to the present invention is applied.

In FIG. 1, reference numeral 10 denotes a communication network to be subjected to quality of service management, and reference numeral 100 denotes data analysis for quality control for at least one of a call service and a data service based on the communication network 10 according to the present invention. Time data analysis device.

The real-time data analysis apparatus 100 receives a log data stream generated in real time with respect to a call service and a data service provided through the communication network 10 in a streaming manner. The log data is the detailed information of various packets related to the call service and the data service transmitted through the communication network 10 and is transmitted through the communication network 10 through a separate log data generation device Lt; RTI ID = 0.0 > packets. ≪ / RTI >

For example, in the case where the communication network 10 is a mobile communication network, the network equipment may include a base station apparatus 12, a mobility management apparatus 13, a serving gateway 14, and a PDN gateway 15 have.

The base station apparatus 12 is a configuration for directly connecting to the user apparatus 11 through radio resources and receiving a packet transmitted from the user apparatus 11 or transmitting a packet to the user apparatus 11. [ Specifically, the base station apparatus 12 detects the user apparatus 11 approaching within the communication coverage, access control and radio resource allocation to the user apparatus 11 requesting the connection, user equipment 11 via the allocated radio resource, Lt; RTI ID = 0.0 > and / or < / RTI > The base station apparatus 12 may be, for example, a Node B, an eNB (eNode B), or the like.

The mobility management apparatus 13 is an element that registers the location of the user equipment 11 and manages and modifies the location of the user equipment 11 according to the movement of the user equipment 11. The mobility management apparatus 13 includes a base station 12, And determines where to route the incoming packet from the user equipment 11. [ The mobility management apparatus 13 is a concept including MME (Mobility Management Entity) proposed in 3GPP.

The serving gateway 14 performs an anchoring function for the movement (handover) of the user equipment 11 and forwards the packet to the PDN gateway 15 to the base station device 12, The packet transmitted from the PDN gateway 15 to the base station apparatus 12 to which the user apparatus 11 is connected.

The PDN gateway 15 is a point of contact with another PDN such as an IP multimedia subsystem (IMS), the Internet, or another mobile communication network, and is responsible for anchoring the movement of the user equipment 11 between other communication networks.

A point S1 for connecting the base station apparatus 12 and the mobility management apparatus 13; a point S2 for connecting the base station apparatus 12 and the serving gateway 14; a serving gateway 13; A point S3 for connecting the mobility management apparatus 13 and a point S4 for connection between the PDN gateway 15 and the other communication network are collected and the log data is generated through the probing of the collected packets have.

In addition, the log data generated in this case can be divided into log data related to the call service and log data related to the data service. Further, bearer connection and operation performed for each user device 11 on the communication network 10 Related bearer log data, HTTP log data, application log data, and DNS log data, which are application level log data made through a connected bearer. Basically, each log data includes collection time information indicating the system in which the log data is collected, user information and related network equipment information, and HTTP information, SCTP information, DNS information, bearer information, Traffic information, call processing information, and call information.

Of course, the type and detailed information of the log data may vary depending on the type of the communication network to be analyzed, the application protocol, and the type of service to be provided.

The real-time data analysis apparatus 100 according to the present invention receives log data generated in connection with a call service and a data service in real time in a streaming manner as described above.

The real-time data analysis apparatus 100 includes a plurality of data processing workflows (RDWs: Real-time (RDW), etc.) comprising one stream input module and one or more processing modules for performing data processing after buffering the received log data stream Data workflow) to sequentially and variably and parallelly process the log data streams to calculate various statistical data related to the quality of the call service and the data service.

Wherein the stream input module reads a plurality of distributed log data streams in a stored order and transfers the stream data to the at least one processing module, wherein the at least one processing module includes a data processing function (e.g., grouping, Correlation analysis, and statistical processing), and statistical data related to the quality of the call service and the data service is calculated by stepwise processing the delivered log data stream.

The detailed configuration and operation of the real-time data analysis apparatus 100 of the present invention will be described with reference to FIGS. 2 and 3. FIG.

FIG. 2 is a block diagram illustrating a configuration of a streaming-based real-time data analysis apparatus 100 according to the present invention, and FIG. 3 is a flowchart illustrating a streaming-based real-time data analysis method according to the present invention.

2, the real-time data analysis apparatus 100 according to the present invention includes one or more data collecting units 110, one or more buffering units 120, and one or more data analyzing units 130 have.

The at least one data collecting unit 110, the at least one buffering unit 120, and the at least one data analyzing unit 130 may process a plurality of log data streams connected in a one-to- And the one or more data analysis units 130 may interrelate and exchange data processing results to calculate statistical data combining the plurality of log data streams.

The real-time data analysis apparatus 100 of the present invention firstly transmits a plurality of logs (not shown) generated in real time on the basis of packets transmitted between network devices constituting the communication network 10 through the one or more data collection units 110 And receives a data stream (S110). For example, the plurality of log data streams may be log data streams generated from points S1, S2, S3, and S4 respectively located on communication network 10 of FIG.

The received plurality of log data is dispersed and buffered in the one or more buffering units 120 (S120).

At this time, the one or more data collecting units 110 and the one or more buffering units 120 are connected to each other in a one-to-one manner, and each data collecting unit 110 sequentially transmits the log data streams to the buffering unit 120, . ≪ / RTI >

The at least one buffering unit 120 performs a buffer function to transmit the log data collected by the at least one data collecting unit 110 to one or more data analysis units 130. First, And may be implemented in a memory structure of a FIFO (First Input First Output) method.

That is, the one or more buffering units 120 sequentially store the log data streams collected by one or more data collecting units 110 and store the stored log data streams in response to a request from the one or more data analyzing units 130 And provides them to the data analysis unit 130 in the stored order.

Accordingly, the real-time data analysis apparatus 100 sequentially reads a plurality of log data streams distributed and stored in the one or more buffering units 120 through the one or more data analysis units 130, , Correlation analysis, and statistical processing are performed in a distributed and parallel manner to calculate call service and data service quality and managed statistical data (S130).

More specifically, the one or more data analysis units 130 may be configured such that a real-time data workflow (RDW), which is a combination of one stream input module and one or more processing modules that perform different data processing, And sequentially reads the plurality of log data streams stored in the one or more buffering units 120 through the set real-time data workflow, performs field grouping, correlation analysis, and statistical processing on statistical data related to the call service and the data service .

The plurality of log data streams processed by the one or more data analysis units 130 at a certain point of time are log data generated at the same time from the plurality of points S1, S2, S3, and S4 of the communication network 10, The one or more data analysis unit 130 calculates statistical data on the call service and the data service occurring in the same time zone. In addition, the log data generated in a predetermined period (for example, one minute) is sequentially processed in a streaming fashion, so that the statistical data of the predetermined period unit can be calculated in real time.

In addition, the at least one data analysis unit 130 analyzes a log-related log information included in the log data stream collected in association with the call service to process the log data stream, A second data workflow for calculating statistical data related to data service quality by analyzing traffic related log information included in the log data stream collected in association with the data service, Can be set.

4 and 5, a detailed structure of the first and second data workflows and an analysis process using the same will be described.

FIG. 4 is a block diagram illustrating a detailed configuration of a first data processing workflow for calculating statistical data related to call service quality in a streaming-based real-time data analyzing apparatus according to the present invention. FIG. 2 is a block diagram showing a detailed configuration of a second data processing workflow for calculating statistical data related to data service quality in a streaming-based real-time data analysis apparatus. FIG.

For reference, FIGS. 4 and 5 illustrate a case where three data analysis units 130-1 to 130-3 and 130-4 to 130-6 are provided for convenience of explanation, but this is merely an example, And are not intended to limit the invention. That is, in the real-time data analysis apparatus 100 according to the present invention, the number of one or more data analysis units 130 can be changed according to the data amount of the log data stream and the number of statistical data to be calculated.

4, a first data workflow for analyzing log data related to a call service includes a stream input module 210, a grouping processing module 220, an analysis processing module 230, and one or more statistical processing modules 240 are sequentially connected.

The stream input module 210 sequentially reads a log data stream related to a call service stored in one of the buffering units 120 and sequentially stores log data included in the log data stream on a predetermined basis, Flow to one of the grouping processing modules 220. Specifically, the stream input module 210 reads log data read from a plurality of log data based on a specific field value, for example, identification information (e.g., IMSI) of the user device, from a plurality of data analysis units 130-1 To 130-3, to the grouping processing module 220. [ The plurality of pieces of log data may be collected in the specific grouping processing module 220 among the plurality of first data workflows according to a certain criterion.

The grouping processing module 220 groups one or more pieces of log data transmitted from any one stream input module 210 of the plurality of first data workflows based on the identification information of the user apparatus, 1 data workflow to one of the analysis processing modules 230. That is, the grouping processing module 220 is for grouping log data generated in the user device 11 at the same time in the log data.

The analysis processing module 230 analyzes the correlation between the log data grouped based on the identification information of the specific user apparatus transmitted from any one of the plurality of first data workflows.

Thereafter, the at least one statistical processing module 240 integrates the analysis results of the analysis processing module 230 step by step to calculate statistical data related to the call service quality. Here, the number of statistical processing modules can be increased or decreased as needed.

In the first data workflow configured as described above, the stream input module 210, the grouping processing module 220, and the analysis processing module 230 interoperate with each other to divide a plurality of call service-related log data into user devices , It is possible to calculate various statistical data related to the call service quality such as the call volume per user device, the number of call attempts, and the ratio of call success rate to call completion call.

5, a second data workflow for analyzing log data related to a data service includes a stream input module 310, one or more statistical processing modules 320, one or more And a grouping processing module 330 may be connected. Here, the number of the statistical processing module 320 and the grouping processing module 330 may vary depending on the amount and type of log data and statistical data.

In the second data workflow, the stream input module 310 sequentially reads a log data stream related to a data service stored in a buffering unit 120 and sequentially reads log data included in the log data stream To the statistical processing module 320 of any one of the plurality of second data workflows. That is, the stream input module 310 appropriately distributes a plurality of pieces of log data for statistical processing.

The one or more statistical processing modules 320 may transmit one or more pieces of log data transmitted from any one of the plurality of second data workflows to the statistical processing module 320 of the second data workflow, And integrates based on one or more field values.

At this time, the one or more statistical processing modules 320 may integrate one or more pieces of log data step by step based on different field values.

Through the one or more statistical processing modules 320, the log data can be integrated based on various field values. Then, the at least one grouping processing module 330 groups the integrated results through the at least one statistical processing module 320 and calculates statistical data related to the data service quality.

As described above, in the second data workflow, a plurality of data service-related log data are integrated on the basis of various field values through one or more statistical processing modules 320, and integrated results are stored in one or more grouping processing modules 330 ), Whereby various traffic-related statistical data related to the data service can be calculated. For example, traffic information can be calculated in real time by device, application, and interval.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be apparent to those skilled in the art. Furthermore, although specific terms are used in this specification and the drawings, they are used in a generic sense only to facilitate the description of the invention and to facilitate understanding of the invention, and are not intended to limit the scope of the invention.

Although the present specification and drawings describe exemplary device configurations, the functional operations and subject matter implementations described herein may be embodied in other types of digital electronic circuitry, or alternatively, of the structures disclosed herein and their structural equivalents May be embodied in computer software, firmware, or hardware, including, or in combination with, one or more of the foregoing. Implementations of the subject matter described herein may be embodied in one or more computer program products, i. E. One for computer program instructions encoded on a program storage medium of the type for < RTI ID = 0.0 & And can be implemented as a module as described above. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

A computer program (also known as a program, software, software application, script, or code) that is loaded on an apparatus according to the present invention and which implements the method according to the present invention may be a compiled or interpreted language, a program containing a priori or procedural language Language, and may be deployed in any form including standalone programs or modules, components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in the file system. The program may be stored in a single file provided to the requested program, or in multiple interactive files (e.g., a file storing one or more modules, subprograms, or portions of code) (E.g., one or more scripts stored in a markup language document). A computer program may be deployed to run on multiple computers or on one computer, located on a single site or distributed across multiple sites and interconnected by a communications network.

Computer-readable media suitable for storing computer program instructions and data include semiconductor memory devices such as, for example, EPROM, EEPROM, and flash memory devices, such as magnetic disks such as internal hard disks or external disks, And DVD-ROM disks, including non-volatile memory, media and memory devices. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

Certain embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in a different order and still achieve desirable results. By way of example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

Therefore, the scope of the present invention should not be limited by the described embodiments but should be defined by the claims.

According to the present invention, in real-time quality control of a call service and a data service provided through a predetermined communication network, a plurality of log data generated in real time during a call service and a data service are received in a streaming manner, The received log data stream can be sequentially processed without a large delay and the statistical data can be calculated in real time.

In particular, the present invention relates to a method and apparatus for sequentially buffering received log data streams, and then sequentially storing a plurality of log data streams stored in a data workflow comprising one stream input module and one or more processing modules each performing predetermined data processing And the delay is minimized by processing stepwise, thereby enabling the statistical data to be calculated in real time.

Also, according to the present invention, call service-related log data and data service-related log data are separately divided and processed through different data workflows so that statistical data required by each of the call service and the data service can be calculated.

100: Real-time data analysis device
110: Data collecting unit
120: buffering unit
130: Data analysis section

Claims (9)

One or more data collecting units for receiving a plurality of log data streams generated in real time while providing a call service and a data service through a communication network in a streaming manner;
One or more buffering units for sequentially distributing and storing a plurality of log data streams received by the one or more data collecting units and outputting a log data stream stored in a first-in first-out manner according to a read request; And
One or more buffering units for sequentially reading log data streams stored in the one or more buffering units and parallelly processing log information included in the input log data stream in cooperation with each other to calculate statistical data for the call service and the data service, And a data analysis unit,
The one or more data analysis units
A first data workflow for analyzing call related log information included in the log data stream collected in connection with the call service to calculate statistical data related to call service quality,
And a second data workflow for analyzing traffic-related log information included in the collected log data stream with respect to the data service and calculating statistical data related to the data service quality. delete 2. The method of claim 1, wherein the first data workflow
A stream input unit for sequentially reading a log data stream related to the call service stored in the corresponding buffering unit and distributing the log data included in the log data stream to a grouping processing module among a plurality of first data workflows on a predetermined basis module;
One or more pieces of log data transmitted from any one of the plurality of first data workflows are grouped based on identification information of the user device and transmitted to one of the plurality of first data workflows A grouping processing module;
An analysis processing module for analyzing a correlation between log data grouped based on identification information of a specific user apparatus transmitted from any one of the plurality of first data workflows;
And one or more statistical processing modules for statistically integrating analysis results of the analysis processing module to calculate statistical data related to call service quality. The method of claim 1, wherein the second data workflow
A stream input unit for sequentially reading a log data stream related to a data service stored in a corresponding buffering unit and distributing log data included in the log data stream to one of a plurality of second data workflows on a predetermined basis module;
One or more pieces of log data transmitted from any one of the plurality of second data workflows are integrated with the statistical processing modules of the second data workflow and stepwise integrates one or more field values based on one or more field values Statistical processing module; And
One or more grouping processing modules for grouping the results integrated through the at least one statistical processing module stepwise on the basis of one or more field values and outputting statistical data related to data service quality;
Based real-time data analysis apparatus. Receiving, in real time, a plurality of log data streams generated in real time while providing a call service and a data service through a communication network;
Distributing the received plurality of log data streams to buffers of first-in first-out method sequentially;
Reading sequentially from the first-in first-out buffers a plurality of distributed log data streams through one or more data workflows;
Processing the log information included in the plurality of log data streams in parallel through interworking of the at least one data workflow to calculate statistical data for the call service and the data service;
, ≪ / RTI >
Wherein the at least one data workflow comprises:
A first data workflow for analyzing call related log information included in the log data stream collected in connection with the call service to calculate statistical data related to call service quality,
And a second data workflow for analyzing traffic-related log information included in the log data stream collected in relation to the data service and calculating statistical data related to the data service quality. Way. 6. The method of claim 5, wherein the calculating step
Sequentially reading log data streams related to a call service among a plurality of log data streams that are distributed and distributed, and distributing log data included in the log data stream on a predetermined basis;
Comparing the field values of the distributed log data and grouping based on the identification information of the user device;
Analyzing a correlation between the grouped log data;
And correlating the correlation analysis results step by step to calculate statistical data related to call service quality. 6. The method of claim 5, wherein the calculating step
Sequentially reading log data streams related to a data service among a plurality of log data streams that are distributed and distributed, and distributing log data included in the log data stream on a predetermined basis;
Stepwise integrating the distributed log data; And
And calculating statistical data related to data service quality by grouping the integrated results stepwise based on one or more field values. A plurality of log data streams generated in real time while providing a call service are processed in parallel through one or more data workflows to calculate statistical data related to call service quality,
The data workflow includes:
A stream input module that sequentially reads the plurality of log data streams and distributes the log data included in the log data stream to any one of the plurality of data workflows on a predetermined basis;
A grouping processing module for grouping one or more pieces of log data transmitted from any one of the plurality of data workflows on the basis of identification information of a user apparatus and transmitting the grouped data to any one of the plurality of data processing workflows; ;
An analysis processing module for analyzing a correlation between log data grouped on the basis of identification information of a specific user device transmitted from any one of the plurality of data workflows;
And one or more statistical processing modules for collecting the analysis results of the analysis processing module step by step and calculating statistical data related to the call service quality. A plurality of log data streams generated in real time during a data service providing process are processed in parallel through one or more data workflows to calculate statistical data related to call service quality,
The data workflow
A stream input module for sequentially reading a log data stream related to a data service and distributing log data included in the log data stream to a statistical processing module among a plurality of data work flows on a constant basis;
One or more statistical processing modules for integrating one or more pieces of log data transmitted from any one of the plurality of data workflows with the statistical processing modules of other data workflows and integrating them in accordance with one or more field values; And
One or more grouping processing modules for grouping the results integrated through the at least one statistical processing module stepwise on the basis of one or more field values and outputting statistical data related to data service quality;
Based real-time data analysis apparatus.

Images