KR102531570B1 - Device, method and computer program for determining quality anomaly score of wireless network - Google Patents

Abstract

A method for determining a quality abnormality score of a wireless communication network is disclosed. A method for determining a quality abnormality score of a wireless communication network according to the present invention includes providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator to train the artificial intelligence model, the artificial intelligence model training After that, obtaining a latent variable output from the encoder by providing quality statistical data for inspection to the encoder, obtaining a Mahalanobis distance of the latent variable, and using the Mahalanobis distance to obtain the latent variable. and determining a quality abnormality score corresponding to the quality statistical data for inspection.

Description

A computer program stored in a medium to perform a device, method, and method for determining a quality abnormality score of a wireless communication network {DEVICE, METHOD AND COMPUTER PROGRAM FOR DETERMINING QUALITY ANOMALY SCORE OF WIRELESS NETWORK}

The present invention performs an apparatus and method for determining a quality anomaly score of a wireless communication network, and a method for determining a quality anomaly score of a wireless communication network, which can automatically detect quality anomaly of a wireless communication network using statistical quality data based on unsupervised learning It relates to a computer program stored on a medium to do so.

In general, for the control of mobile communication wireless networks, major equipment and facility failure events are monitored, and quality KPIs (Key Performance Indicators) are also provided by EMS (Element Management System) or NMS (Network Management System). Quality is monitored through statistical data.

For equipment failure or cable breakage, the criterion for occurrence of failure is clear, but in the case of quality monitoring of a wireless communication network, the criterion for occurrence of significant quality deterioration is not clear.

Therefore, based on the absolute threshold value, the relative quality abnormality is currently being judged by monitoring the cases where the numerical value of the major quality KPI becomes very bad, or checking the average value and maximum value of a specific quality KPI compared to the previous day or the previous week from time to time.

However, specific rules depend on the manager's experience, and separate management for quality abnormality judgment is conducted through the own standards of each regional office.

In addition, each cell (RU) has different KPI probability distribution characteristics according to the geographical characteristics of the corresponding region and the traffic trend of users by time zone. Therefore, it is not appropriate to collectively apply the same rule to all cells (RU) nationwide. In order to detect quality anomalies, it is necessary to monitor whether the quality statistical characteristics of each cell (RU) have significantly deteriorated compared to normal conditions. do. In addition, since the quality statistical characteristics for each cell (RU) may change over time due to changes in the surrounding environment, etc., continuous updating is required.

However, physical limitations inevitably exist for a small number of administrators in each regional office to individually monitor wireless network quality deterioration of all cells within their jurisdiction.

Therefore, currently, the quality deterioration of the wireless communication network is monitored by generating a warning message based on the absolute threshold value for major wireless quality KPIs or by managing only the top few in the order of severity of deterioration for each KPI. exists.

In addition, by monitoring dozens of quality KPIs, it is difficult to recognize quality anomalies through correlations between quality KPIs or other quality raw data that are not classified as KPIs. There is a problem that sometimes occurs without recognizing the quality abnormality. In this case, since it is difficult to take appropriate measures in a timely manner, service quality is adversely affected.

The present invention is to solve the above-mentioned problems, and an object of the present invention is to obtain a quality anomaly score of a wireless communication network, which can automatically detect quality anomalies of a wireless communication network using statistical quality data based on unsupervised learning. It is to provide a computer program stored in a medium to perform a determination device, method, and method for determining a quality abnormality score.

A method for determining a quality abnormality score of a wireless communication network according to the present invention includes providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator to train the artificial intelligence model, the artificial intelligence model training After that, obtaining a latent variable output from the encoder by providing quality statistical data for inspection to the encoder, obtaining a Mahalanobis distance of the latent variable, and using the Mahalanobis distance to obtain the latent variable. and determining a quality abnormality score corresponding to the quality statistical data for inspection.

In this case, the step of training the artificial intelligence model includes providing the quality statistical data for training to the encoder, and outputting latent variables for training from the encoder and result data output from the decoder using the latent variables for training. Obtaining, providing the latent variable for training and the probability distribution sample data to the discriminator, so that the discriminator determines whether the latent variable for training and the probability distribution sample data are real or fake and training the artificial intelligence model using the result data output by the decoder and the result value output by the discriminator.

In this case, the probability distribution sample data may have a normal distribution.

In this case, the step of training the artificial intelligence model using the result data output by the decoder and the result value output by the discriminator is the step of training the artificial intelligence model using the difference between the result data output by the decoder and the quality statistical data for training. Calculating a first loss for allowing the encoder and the decoder to restore the training quality statistical data, and using the first loss to train the encoder and the decoder, using the result value output by the discriminator , calculating a second loss for allowing the discriminator to discriminate between the real and the fake, and training the discriminator using the second loss, and a result value output by the discriminator. Calculating a third loss for the encoder to output the latent variable having the normal distribution using , and training the encoder using the third loss.

Meanwhile, the step of training the artificial intelligence model further includes acquiring a Mahalanobis distance of the latent variable for training, and the step of training the encoder using the third loss includes the step of acquiring the latent variable for training. A tuning value reflecting the difference between the distribution of the Mahalanobis distance of and the standard normal distribution of is added to the third loss, and the encoder may be trained using the third loss to which the tuning value is added.

Meanwhile, in the step of training the artificial intelligence model, when the artificial intelligence model is trained based on the training quality statistical data including a plurality of samples, a plurality of latent potentials output by the encoder based on the plurality of samples. The method may include obtaining vectors and storing an average value of the plurality of latent vectors and an inverse matrix of a covariance matrix.

In this case, in the obtaining of the Mahalanobis distance of the latent variable, the Mahalanobis distance may be obtained using the latent variable, an average value of the plurality of latent vectors, and an inverse matrix of the covariance matrix.

Meanwhile, the statistical quality data for training and the statistical quality data for inspection may include at least one of raw data and KPI data collected from a specific base station.

Meanwhile, the quality statistical data for training is updated, the updated training quality statistical data is provided to the artificial intelligence model to retrain the artificial intelligence model, and the average value of the plurality of latent vectors and the inverse matrix of the covariance matrix are calculated. An update step may be further included.

On the other hand, the device for determining the abnormal quality score of the wireless communication network includes a data collection unit that collects statistical quality data for training and statistical quality data for inspection, and an artificial intelligence model including an encoder, a decoder, and a discriminator for the statistical quality data for training. After the artificial intelligence model is trained, quality statistical data for inspection is provided to the encoder to obtain a latent variable output from the encoder and obtain a Mahalanobis distance of the latent variable. and a control unit for determining a quality abnormality score corresponding to the statistical quality data for inspection using the Mahalanobis distance.

In this case, the control unit provides the quality statistical data for training to the encoder, obtains result data output from the decoder using latent variables for training output by the encoder and latent variables for training, and determines the The latent variable for training and the probability distribution sample data are provided to the discriminator so that the discriminator determines whether the latent variable for training and the probability distribution sample data are real or fake, and the result output by the decoder The artificial intelligence model may be trained using data and result values output by the discriminator.

In this case, the probability distribution sample data may have a normal distribution.

In this case, the control unit calculates a first loss for allowing the encoder and the decoder to restore the statistical quality data for training using a difference between the result data output by the decoder and the statistical quality data for training, The encoder and the decoder are trained using the first loss, and the discriminator calculates the second loss for discriminating the real or the fake using the result value output by the discriminator. and training the discriminator using the second loss, calculating a third loss for causing the encoder to output a latent variable having the normal distribution using a result value output by the discriminator, and calculating the third loss The encoder can be trained using the loss.

Meanwhile, the control unit obtains a Mahalanobis distance of the latent variable for training, and adds a tuning value reflecting a difference between a distribution of the Mahalanobis distance of the latent variable for training and a standard normal distribution to the third loss, The encoder may be trained using the third loss to which the tuning value is added.

Meanwhile, when the artificial intelligence model is trained based on the training quality statistical data including a plurality of samples, the control unit obtains a plurality of latent vectors output by the encoder based on the plurality of samples, and An average value of a plurality of latent vectors and an inverse matrix of a covariance matrix may be stored.

In this case, the control unit may obtain the Mahalanobis distance using the latent variable, an average value of the plurality of latent vectors, and an inverse matrix of the covariance matrix.

Meanwhile, the statistical quality data for training and the statistical quality data for inspection may include at least one of raw data and KPI data collected from a specific base station.

Meanwhile, the control unit updates the statistical quality data for training, provides the updated statistical data for training quality to the artificial intelligence model, re-trains the artificial intelligence model, and calculates the mean value and covariance of the plurality of latent vectors. You can update the inverse of a matrix.

On the other hand, in order to perform the method of determining the quality abnormality score of the wireless communication network according to the present invention, a computer program stored in a medium provides quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator to provide the artificial intelligence model After the artificial intelligence model is trained, providing quality statistical data for inspection to the encoder to obtain a latent variable output from the encoder, obtaining a Mahalanobis distance of the latent variable, and determining a quality abnormality score corresponding to the quality statistical data for inspection using the Mahalanobis distance.

According to the present invention, it is possible to monitor base stations within a jurisdiction using quality statistics data of a plurality of base stations (cells). Therefore, according to the present invention, it is possible to solve the conventional problem that a small number of administrators could not individually monitor quality deterioration of base stations within their jurisdiction.

In addition, according to the present invention, since quality anomalies are detected by additionally using previously unavailable information such as correlation between quality KPIs and raw data not classified as KPIs, there is an advantage in that quality anomalies can be detected more accurately.

In addition, according to the present invention, since the artificial intelligence model corresponding to a specific base station is trained using quality statistical data for training collected from a specific base station, the artificial intelligence model corresponding to a specific base station is trained by reflecting the characteristics of the specific base station. . Therefore, there is an advantage in detecting quality anomalies by reflecting the communication environment unique to the base station or other surrounding environments.

In addition, according to the present invention, the artificial intelligence model is continuously updated by the latest quality statistical data. Therefore, it has the advantage of being able to detect quality anomalies by reflecting environmental changes over time.

In addition, it is impossible to create artificial intelligence models for anomaly detection of tens of thousands or hundreds of thousands of base stations with a supervised learning algorithm that labels correct values. However, in the present invention, by using an unsupervised learning algorithm, there is an advantage in that manpower and costs required for generating artificial intelligence models can be drastically reduced, and artificial intelligence models capable of detecting anomalies of numerous base stations can be easily generated.

In addition, according to the present invention, a plurality of artificial intelligence models respectively corresponding to a plurality of base stations are trained with the same training method, except that only input quality statistical data is different. Therefore, the training method proposed in the present invention is a universal method that can be applied to various national affairs, and according to the present invention, it has the advantage of saving time and cost for generating an artificial intelligence model.

In addition, according to the present invention, since the latest artificial intelligence model is additionally trained by inputting the latest training quality statistical data, the accuracy of the latest artificial intelligence model that has already been trained a lot is used as it is, while the environment It has the advantage of being flexible in adapting to changes.

1 is a diagram for explaining a system for determining a quality abnormality score of a wireless communication network according to the present invention.
2 is a block diagram for explaining the components of a decision device according to the present invention.
3 is a flowchart illustrating a method of determining a quality abnormality score of a wireless communication network.
4 is a flowchart for explaining a method of training an artificial intelligence model.
5 is a diagram for explaining various sub-modules constituting a control unit and operations of various sub-modules.
6 is a diagram for explaining a training method of an artificial intelligence model.
7 is a diagram for explaining the overall operation of the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In implementing the present invention, components may be subdivided for convenience of description, but these components may be implemented in one device or module, or one component may be divided into multiple devices or modules may be implemented in

Meanwhile, in the present invention, an artificial intelligence model is trained based on unsupervised learning. Here, unsupervised learning may be a learning method of learning an artificial neural network to find and classify a pattern in training data itself rather than an association between training data and labels corresponding to the training data.

1 is a diagram for explaining a system for determining a quality abnormality score of a wireless communication network according to the present invention.

The detailed names shown in FIG. 1 are described based on 5G base station equipment, but are not limited thereto, and the present invention can be applied to a mobile communication system including a base station (Radio Unit, RU).

Also, since the components shown in FIG. 1 are not essential components, some components of the wireless communication network quality abnormality score determination system may be omitted or more components may be included.

Referring to FIG. 1, the system 10 for determining a quality abnormality score of a wireless communication network includes one or more Central Units (CUs) 410, one or more Distributed Units (DUs) 310 and 320, and one Abnormal base station (Radio Unit, RU) (210, 220, 230), network equipment management system (Element Management System, EMS) (500), network management system (Network Management System, NMS) (600), and quality abnormality of wireless communication network The score determination device 100 may be included.

First of all, the terminology is summarized, the term “network equipment management system” is the term “EMS”, the term “network management system” is the term “NMS”, and the term “determination device of quality abnormality score of wireless communication network” is the term “determination device” It can be used in combination with “.

In the case of a 5G NR base station, as shown in Figure 1 to maximize network operation efficiency, a three-tier structure of a central unit (CU), a distributed unit (DU), and a base station (Radio Unit, RU), each performed It is divided and configured according to the function to be performed. All of these components are generally separated and built, but it is also possible to configure a central device-dispersion device integrated type (CU-DU integrated type) or a distributed device-base station integrated type (DU-RU integrated type).

A base station (Radio Unit, RU) (210, 220, 230) is a basic unit constituting a cell in a base station system, and quality management of a wireless network is performed in units of each base station (Radio Unit, RU), that is, in units of cells. effective.

Basically, wireless network quality statistical data is collected in units of base stations (Radio Units, RUs), and quality statistical data for each base station (Radio Unit, RU) in the jurisdiction is distributed to the corresponding base station (Radio Unit, RU). As it passes through a Distributed Unit (DU) and a Central Unit (CU), it is converted into a predetermined format and transmitted to an Element Management System (EMS).

The quality statistical data collected by the EMS 500 is transmitted to the NMS 600 as it is or after being reprocessed.

Meanwhile, the apparatus 100 for determining the abnormal quality score of the wireless communication network (hereinafter, referred to as the determination apparatus 100) may periodically collect statistical quality data. In this case, the decision device 100 may receive quality statistical data from the EMS 500, but is not limited thereto, and receives various quality statistical data directly from the base station (Radio Unit, RU) 210, 220, 230. Quality statistical data can be collected in this way.

Also, the decision device 100 may collect quality statistical data in real time.

Meanwhile, the quality statistical data may include at least one of raw data and KPI data.

Here, raw data may mean various parameters collected to measure or manage quality of wireless communication, and KPI data may mean data generated by combining raw data. For example, KPI data may include SINR, BLER, and drop rate.

In addition, the statistical quality data may be statistical data obtained by statisticizing data collected in a certain time period. For example, if a large number of information is collected on a specific parameter from 1:00 to 1:15, the statistical result of a large number of information will be set as a value representing the specific parameter from 1:00 to 1:15. can

Meanwhile, raw data may be collected for each base station. Also, the determination device 100 may collect raw data classified for each base station.

2 is a block diagram for explaining the components of a decision device according to the present invention.

According to FIG. 2 , the decision device 100 may include a data collection unit 110 , a control unit 120 , a memory 130 and an output unit 140 .

Meanwhile, all of the components shown in FIG. 2 are not essential, so the decision device 100 may have fewer or more components than those shown in FIG. 2 .

The data collection unit 110 includes a communication unit and can communicate with an external device through a wired/wireless network. Specifically, the data collector 110 may transmit/receive data by being connected to the EMS 500 or other devices through a wired/wireless network.

Also, the data collection unit 110 may receive quality statistical data from the EMS 500 or other devices.

Also, the data collection unit 110 may periodically receive quality statistical data. Here, the quality statistical data may be data periodically collected or generated by the EMS 500 or other devices.

The controller 120 may control overall operations of the decision device 100 . Here, the control unit may be used interchangeably with terms such as controller, processor, and microprocessor.

The memory 130 may store a program for the operation of the determination device 100, a program for executing a method for determining a quality abnormality score of a wireless communication network, and other data.

Also, the memory 130 may store an artificial intelligence model. Specifically, an artificial intelligence model may be implemented in hardware, software, or a combination of hardware and software. In this case, one or more commands constituting the artificial intelligence model may be stored in the memory 130 .

The output unit 140 may include a display unit that generates an output related to time. In this case, the display unit provides information processed by the decision device 100, for example, execution screen information of an application program driven by the decision device 100, or UI (User Interface) and GUI (Graphic User Interface) according to such execution screen information. information can be displayed. Also, the output unit 140 may include a speaker that generates an output related to hearing.

Also, the output unit 140 may output a quality abnormality score.

3 is a flowchart illustrating a method of determining a quality abnormality score of a wireless communication network.

A method for determining a quality abnormality score of a wireless communication network includes training the artificial intelligence model by providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator (S310), after the artificial intelligence model is trained, Acquiring latent variables output from the encoder by providing quality statistical data for inspection to an encoder (S330), acquiring a Mahalanobis distance of the latent variables (S350), and quality statistics for inspection using the Mahalanobis distance A step of determining an abnormal quality score corresponding to the data (S370) may be included.

First, the step of training the artificial intelligence model by providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator (S310) will be described in detail with reference to FIGS. 4 to 6.

4 is a flowchart for explaining a method of training an artificial intelligence model.

Training the artificial intelligence model by providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator (S310) provides the quality statistical data for training to the encoder and outputs training potentials from the encoder. Acquiring result data output by the decoder using variables and latent variables for training (S312), latent variables for training and Providing probability distribution sample data to the discriminator (S314), training an artificial intelligence model using the result data output by the decoder and the result value output by the discriminator (S316), and for training including a plurality of samples When the artificial intelligence model is trained based on the quality statistical data, obtaining a plurality of latent vectors output by the encoder based on a plurality of samples, and storing an average value of the plurality of latent vectors and an inverse matrix of the covariance matrix (S318). ) may be included.

5 is a diagram for explaining various detailed modules constituting the control unit 120 and operations of the various detailed modules.

The component “-unit” shown in FIG. 5 may be used interchangeably with the term “-module”.

The control unit 120 includes a quality statistical data selection unit 121, an AAE learning model-based latent variable derivation unit 122, a Mahalanobis distance derivation unit 123, an anomaly score derivation unit 124, and an anomaly score characteristic analysis unit ( 125) may be included.

The controller 120 may store quality statistical data corresponding to a plurality of base stations (cells) in the memory 130 to be classified for each base station (cell). For example, the memory 130 may store statistical quality data collected from a first base station (cell) and statistical quality data collected from a second base station (cell) in a distinguishable manner.

Meanwhile, the artificial intelligence model described below corresponds to a specific base station (cell) and is a model for detecting whether or not the quality of the specific base station (cell) is abnormal. Accordingly, the quality statistical data selector 121 may select quality statistical data of a specific base station (cell) to be subjected to quality abnormality detection.

Meanwhile, the controller 120 may store quality statistical data corresponding to a specific base station (cell) in the memory 130 so as to be classified for each time interval.

For example, the memory 130 includes quality statistical data (eg, January 1, 2020 00:00 to January 1, 2020 00:15) of the first time interval collected from the first base station (cell). quality statistical data corresponding to the time interval up to) and quality statistical data of the second time interval collected from the first base station (cell) (eg, from 00:15 on January 1, 2020 to January 1, 2020) quality statistical data corresponding to a time interval up to 00:30) may be stored in a distinguishable manner.

In this case, the quality statistical data selector 121 may select quality statistical data of a preset period from among a plurality of quality statistical data of a specific base station (cell). For example, the quality statistical data selector 121 may select quality statistical data for 100 days of a specific base station (cell).

Also, the quality statistical data selector 121 may select the latest quality statistical data of a preset period.

Specifically, quality statistical data of a specific base station (cell) is continuously collected over time. Also, quality statistical data of a specific base station (cell) may be changed according to changes in the environment inside or around the specific base station.

In addition, the quality statistical data selector 121 may select the latest quality statistical data of a preset period so that the artificial intelligence model can be updated by adapting to the changed environment.

For example, the quality statistical data selector 121 may select quality statistical data of a specific base station (cell) for the most recent 100 days. In this way, in order to train the artificial intelligence model, quality statistical data selected as an object to be provided to the artificial intelligence model may be referred to as quality statistical data for training.

Meanwhile, updating (training) of the artificial intelligence model may be performed according to a predetermined cycle. For example, updating (training) of an AI model can be done every day or every other day.

In addition, the artificial intelligence model may be trained using the latest quality statistical data selected for training the artificial intelligence model. Here, the latest quality statistical data may refer to quality statistical data collected during a period closest to the point at which the update (training) of the artificial intelligence model is performed.

In addition, the latest quality statistical data may be data composed of units such as year, half year, quarter, month, day, hour, and the like.

For example, assume that the current time is 14:30 on April 10, 2020. In addition, when the latest quality statistical data is configured on a daily basis, quality statistical data from January 1, 2020 to April 9, 2020 may be selected. For another example, when the latest quality statistical data is configured by hour, quality statistical data from 14:00 on January 1, 2020 to 14 April 10, 2020 may be selected.

When the quality statistical data for training is selected in this way, the AAE learning model-based latent variable derivation unit 122 provides the quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator to train the artificial intelligence model. Yes (S310).

An object of the present invention is to have an encoder in an artificial intelligence model output latent variables having a normal distribution by training an artificial intelligence model. And the reason why the encoder should output latent variables having a normal distribution is as follows.

First, in the present invention, the quality statistical data is provided by the encoder and the degree of abnormality is determined by obtaining the Mahalanobis distance of the outputted latent variable.

However, it is rare for the distribution characteristics of quality statistical data in wireless communication to follow a normal distribution. On the other hand, the method for calculating the Mahalanobis distance is a calculation method performed under the assumption that each element of the vector used to calculate the distance has a normal distribution. am.

Therefore, when the Mahalanobis distance is calculated using a latent variable that does not have a normal distribution, the calculation is inaccurate or cannot be calculated.

Therefore, in the present invention, in order to secure the reliability and accuracy of the quality abnormality score calculation, the artificial intelligence model is trained so that the latent variables output from the encoder have a normal distribution.

This will be described with reference to FIG. 6 together with FIG. 5 .

6 is a diagram for explaining a training method of an artificial intelligence model.

The term “for training” used below is to be distinguished from the term “for inspection”. For example, a latent variable for training is a latent variable generated during the training process of an artificial intelligence model, and a latent variable for inspection may mean a latent variable generated during a quality monitoring process.

Unsupervised learning used in the present invention is a type of machine learning, and labels for training data are not given.

The artificial intelligence model 800 may include an auto encoder including an encoder 810 and a decoder 820 and a discriminator 830 .

An autoencoder (AE) is an artificial neural network that aims to reproduce the input itself as an output.

An autoencoder (AE) may include an input layer, a plurality of hidden layers, and an output layer.

In addition, the input data input to the input layer of the encoder 810 passes through the hidden layer, and is encoded and decoded based on parameters (weights, biases, etc.) of elements (neural networks, nodes, etc.) constituting the auto-encoder, and the decoder 820 can be output through the output layer of

On the other hand, since the number of nodes in the hidden layer is smaller than the number of nodes in the input layer, the dimensionality of the data is reduced. Accordingly, the input data is compressed or encoded and gathered in the latent space 815. A feature vector encoded and gathered in the latent space 815 in this way can be referred to as a latent variable (z).

In addition, the data collected in the latent space 815 is input to the decoder 820, and since the number of nodes in the output layer is greater than the number of nodes in the hidden layer, the dimensionality of the data increases. It is output from the output layer of 820.

In addition, the auto-encoder compresses input data to generate a latent variable (z) and restores the generated latent variable (z) to generate result data. And the autoencoder is an artificial neural network that finds an optimal latent variable (z) to reproduce the input data. That is, since the input data must be reproduced even though the input data is compressed to a lower dimension, the latent variable (z) compressed and collected in the latent space 815 may represent a feature of the input data. .

Also, in the present invention, quality statistical data is used as input data. Therefore, the latent variable (z) in the present invention may mean data in which features extracted from quality statistical data are gathered in order to reproduce quality statistical data.

Meanwhile, among the auto-encoders, the encoder 810 may operate as a generator in the artificial intelligence model 800 based on an adversarial auto-encoder.

Meanwhile, the artificial intelligence model 800 may further include a discriminator 830 and a sample generator 840 . Here, the discriminator 830 and the sample generator 840 may be referred to as an adversarial network.

In the artificial intelligence model 800, two different artificial intelligences, a generator (encoder 810) and a discriminator 830 compete, and performance can be improved.

Here, the generator (encoder 810) is a model that creates new data, and can play a role of generating latent variables based on input data input to the generator (encoder 810).

In addition, the discriminator 830 is a model that recognizes the probability distribution characteristics of data or latent variables, and plays a role in determining the authenticity of the data based on the data input to the discriminator 830. can

Meanwhile, the sample generator 840 may provide probability distribution sample data to the discriminator 830 .

Here, the probability distribution sample data may be provided to the discriminator 830 as probability distribution sample data having a preset probability distribution. Also, the probability distribution sample data having a preset probability distribution may be data having a normal distribution (the distribution characteristic of the sample data is a normal distribution).

Meanwhile, the discriminator 830 evolves to distinguish between probability distribution sample data provided from the sample generator 840 and latent variables (or latent variable data) output from the encoder 810, and the encoder 810 It evolves to deceive the discriminator 830 as best as possible and can improve each other's performance.

Hereinafter, the step of training the artificial intelligence model by providing quality statistical data for training to the artificial intelligence model including the encoder, decoder, and discriminator (S310) will be described in more detail.

The AAE learning model-based latent variable derivation unit 122 provides quality statistical data for training to the encoder 810, and the decoder 820 uses the latent variable for training output by the encoder 810 and the latent variable for training. Output result data can be acquired (S312).

Here, the quality statistical data for training is composed of a plurality of quality statistical data respectively corresponding to a plurality of time intervals. Also, quality statistical data corresponding to one time interval may be referred to as one sample.

For example, quality statistical data for training is a first sample (eg, quality statistical data corresponding to a time interval from 00:00 on January 1, 2020 to 00:15 on January 1, 2020), The second sample (e.g., quality statistical data corresponding to the time interval from 00:15 on January 1, 2020 to 00:30 on January 1, 2020), the n-th sample (e.g., 4 quality statistical data corresponding to the time interval from 23:45 on the 9th of the month to 24:00 on the 9th of April 2020).

Accordingly, the artificial intelligence model 800 may divide a plurality of samples constituting the quality statistical data for training by various time units or use them as input data in various combinations.

Therefore, in this specification, training the artificial intelligence model 800 by providing quality statistical data for training to the artificial intelligence model 800 means inputting training data and based on the resulting data, the parameters (weight, bias) of the artificial intelligence model parameters (weights, biases, etc.) It may also mean a plurality of processes that regulate.

On the other hand, when the AAE learning model-based latent variable derivation unit 122 provides the encoder 810 with quality statistical data (Input X) for training, the encoder 810 provides the encoder 810 with the latent variable for training based on its parameters. Variable (z) can be output. In addition, when the latent variable (z) for training is input, the decoder 820 may output result data (Output X') based on its parameters.

Then, the AAE learning model-based latent variable derivation unit 122 determines whether the discriminator 830 is real or fake of the training latent variable Z and the probability distribution sample data Z'. To discriminate, a latent variable for training (Z) and probability distribution sample data (Z') may be provided to the discriminator 830 (814).

In the present invention, determining whether the latent variable for training (Z) and the probability distribution sample data (Z') are real or fake means whether the latent variable for training and the probability distribution sample data follow a normal distribution. can be determined.

In detail, determining real may mean determining whether probability distribution sample data provided from the sample generator 840 follows a normal distribution. Also, determining a fake may mean determining whether latent variables output from the encoder 810 do not follow a normal distribution.

In this case, the discriminator 830 may output a resultant value based on its parameters. Here, the result value may mean a result of determining whether the input data follows a normal distribution or a determined probability.

For example, the discriminator 830 includes a result of determining whether the probability distribution sample data provided from the sample generator 840 follows a normal distribution or a probability that the probability distribution sample data provided from the sample generator 840 follows a normal distribution. The resulting value can be output. Also, the probability may be expressed as a score, and thus the discriminator 830 may output a result value including a score indicating a probability that the probability distribution sample data follows a normal distribution.

For another example, the discriminator 830 determines whether the latent variable output from the encoder 810 does not follow a normal distribution or a result value including a probability that the latent variable output from the encoder 810 does not follow a normal distribution. can output Probability can also be expressed as a score, so the discriminator 830 may output a result value including a score indicating the probability that the latent variable output from the encoder 810 does not follow a normal distribution. there is.

Then, the AAE learning model-based latent variable derivation unit 122 derives the artificial intelligence model 800 using the result data (Output X') output by the decoder 820 and the result value output by the discriminator 830. Training can be performed (S316).

Specifically, the AAE learning model-based latent variable derivation unit 122 uses the difference between the result data (Output X′) output by the decoder 820 and the quality statistical data for training (Input X) to generate the encoder 810 and the decoder. In operation 820, a first loss for restoring the statistical quality data for training (Input X) may be calculated.

More specifically, the auto-encoder including the encoder 810 and the decoder 820 extracts a latent variable (feature vector) from quality statistical data for training input to the encoder 810, and then extracts the extracted latent variable (feature vector). ) to restore quality statistical data for training.

Therefore, the AAE learning model-based latent variable derivation unit 122 calculates a first loss based on the difference between the result data (Output X′) output by the decoder 820 and the quality statistical data for training (Input X), and Parameters of the encoder 810 and the parameters of the decoder 820 may be adjusted so that 1 loss is small.

When the mean squared error (MSE) is used as the loss function, the first loss (LR) is the mean squared error (MSE) loss for training the encoder 810 and the decoder 820 (loss) value.

In addition, the AAE learning model-based latent variable derivation unit 122 uses the result values (score for Z, Z') output by the discriminator 830 to determine whether the discriminator 830 is real or fake. 2 losses may be calculated, and the discriminator 830 may be trained using the second losses.

Specifically, the discriminator 830 is used to determine whether the probability distribution sample data provided by the sample generator 840 is real (i.e., whether it has a normal distribution) and whether the latent variable output from the encoder 810 is fake (i.e., It is an artificial neural network for discriminating whether or not it has a normal distribution.

Therefore, the AAE learning model-based latent variable derivation unit 122 determines the result value output by the discriminator 830 (for example, the probability that the probability distribution sample data has a normal distribution or the latent variable output from the encoder 810 is normal). A second loss may be calculated based on the probability of not having a distribution), and a parameter of the discriminator 830 may be adjusted so that the second loss becomes small. Here, the second loss LD may be defined as a loss value for training the discriminator 830 of the adversarial auto encoder.

In addition, the AAE learning model-based latent variable derivation unit 122 uses the result value (score for Z, Z') output by the discriminator 830 to cause the encoder 810 to output a latent variable having a normal distribution. A third loss for the second loss may be calculated, and the encoder 810 may be trained using the third loss.

Specifically, the discriminator 830 is an artificial neural network for discriminating whether the latent variable output from the encoder 810 is fake (ie, does not have a normal distribution).

Therefore, the AAE learning model-based latent variable derivation unit 122, based on the result value output by the discriminator 830 (for example, the probability that the latent variable output from the encoder 810 does not have a normal distribution), third The loss may be calculated, and parameters of the encoder 810 may be adjusted so that the third loss becomes small. Here, the third loss (L _G ) may be defined as a loss value for forcing the distribution characteristics of latent variables output from the encoder 810 into a normal distribution.

In addition, the artificial intelligence model outputs result data and result values using various combinations of a plurality of samples constituting the quality statistical data for training, and the process of training the artificial intelligence model using the output result data and result values is repeated. It can be. Accordingly, the encoder 810, the decoder 820, and the discriminator 830 can be organically trained while each parameter is optimized.

Accordingly, the encoder 810 may set internal parameters based on the first loss (L _R ) to extract a latent variable for outputting result data that is closest to quality statistical data for training. In addition, internal parameters of the encoder 810 may be set to output a latent variable having a probability distribution close to a normal distribution based on the third loss (L _G ).

Meanwhile, the Mahalanobis distance must be calculated in the process of determining the quality abnormality score by inputting quality statistical data for inspection into the artificial intelligence model afterwards. In addition, since the average value of latent variables for training and the inverse matrix of the covariance matrix are required to calculate the Mahalanobis distance, the AAE learning model-based latent variable derivation unit 122 calculates the average value of latent variables for training and the inverse matrix of the covariance matrix. can be stored in memory.

Specifically, when an artificial intelligence model is trained based on quality statistical data for training including a plurality of samples, the AAE learning model-based latent variable derivation unit 122 outputs a plurality of outputs from the encoder 810 based on a plurality of samples. A latent vector of , and an average value of a plurality of latent vectors and an inverse matrix of a covariance matrix may be stored (S318).

More specifically, the quality statistical data for training may consist of a plurality of samples each corresponding to a plurality of time intervals.

For example, it is assumed that quality statistical data for training consists of a plurality of samples divided into 15-minute time units for a total of 100 days. In this case, quality statistical data for training may consist of 9600 samples. In this case, the first sample is quality statistical data corresponding to the time interval from 00:00 on January 1, 2020 to 00:15 on January 1, 2020, and the 9600th sample is April 9, 2020 Quality statistical data corresponding to the time interval from 23:45 to 24:00 on April 9, 2020.

Further, the AAE learning model-based latent variable derivation unit 122 may obtain a plurality of latent vectors output by the encoder 810 based on a plurality of samples.

Specifically, the encoder 810 may sequentially (or in parallel) process a plurality of samples and output a plurality of latent vectors respectively corresponding to the plurality of samples.

For example, when samples 1 to 10 are used for training in an n-th iteration of training, the encoder 810 may output first to tenth latent vectors respectively corresponding to the 1 to 10 samples. In this case, the first to tenth latent vectors may constitute latent variables output in the nth iteration of training.

Meanwhile, in this way, the AAE learning model-based latent variable derivation unit 122 may obtain latent vectors corresponding to all samples constituting quality statistical data for training. In addition, the AAE learning model-based latent variable derivation unit 122 may calculate an average value of the obtained latent vectors and an inverse matrix of the covariance matrix and store them in the memory 130 .

Meanwhile, training of the artificial intelligence model may end after going through the above process. However, in the present invention, an additional tuning element for adjusting the distribution characteristics of the abnormal quality score may be used, and this will be further described.

In the embodiment using additional tuning elements, the training process described above can be applied within a range that does not contradict, and will be described below focusing on different points.

When the AAE learning model-based latent variable derivation unit 122 provides quality statistical data (Input X) for training to the encoder 810, the encoder 810 performs a latent variable for training based on its parameters. (Z) can be output. In this case, the latent variable Z for training may be input to the decoder 820 and discriminator 830.

In addition, the latent variable Z for training may be input to the Mahalanobis distance derivation unit 123 together with the decoder 820 and discriminator 830 .

In this case, the Mahalanobis distance derivation unit 123 may obtain the Mahalanobis distance of the latent variable Z for training.

The Mahalanobis Distance may be a distance index considering correlation between variables.

And the Mahalanobis distance can be calculated by the following formula.

: 잠재 변수 Z의 마할라노비스 거리,

: 잠재 벡터들의 평균 값,

: 잠재 벡터들의 공분산 행렬의 역행렬)(Z: latent variable,

: Mahalanobis distance of latent variable Z,

: average value of latent vectors,

: Inverse matrix of covariance matrix of latent vectors)

That is, the Mahalanobis distance derivation unit 123 uses the latent variable Z for training output from the encoder 810, the mean value of the latent vectors, and the inverse matrix of the covariance matrix to obtain the Mahalanobis distance variable Z for training. The novice distance (MD(Z)) can be calculated.

On the other hand, the ideal score derivation unit 124 may take a logarithm of the Mahalanobis distance (MD(Z)) of the latent variable (Z) for training and standardize it. In this case, the standardized Mahalanobis distance can be expressed as log10(MD(Z)).

Meanwhile, the ideal score characteristic analysis unit 125 may calculate a tuning value based on the similarity between the distribution of the Mahalanobis distance (MD(Z)) of the latent variable for training and the ideal distribution.

Specifically, the ideal score characteristic analysis unit 125 may calculate a tuning value reflecting the difference between the distribution of the Mahalanobis distance (MD(Z)) of the latent variable for training and the normal distribution.

으로 측정될 수 있다.Here, the tuning value (TG) may be determined by the distribution difference between the probability density function of the standard normal distribution and the probability density function of the standardized Mahalanobis distance (log10(MD(Z)). And the tuning value (TG) is The computed value of the Kullback-Leibler Divergence (KLD),

can be measured as

(T _G: tuning value, P(x): probability density function of standard normal distribution, Q(x): probability density function of standardized Mahalanobis distance (log ₁₀ (MD(Z)))

Meanwhile, in order to reflect the tuning value in the training of the artificial intelligence model 800, the AAE learning model-based latent variable derivation unit 122 may add a tuning value (T _G ) to one of the three losses described above. Meanwhile, since it is the encoder 810 that determines the distribution characteristics of the latent variable, the AAE learning model-based latent variable derivation unit 122 converts the tuning value T _G to the third loss L _G used to train the encoder 810. ) can be added.

The third loss to which the tuning value is added can be expressed by the following equation.

(L _{G_total} : 3rd loss with added tuning value, L _G: 3rd loss before adding tuning value, T _G: tuning value, c: parameter for determining the degree to which tuning value is reflected in training)

In this case, the parameter (c) may be appropriately adjusted according to the characteristics of the base station to be monitored.

Further, the learning model-based latent variable derivation unit 122 may train the encoder 810 using the third loss (L _{G_total} ) to which the tuning value is added.

That is, the AAE learning model-based latent variable derivation unit 122 calculates a third loss based on the result value output by the discriminator 830, adds a tuning value to the third loss, and then adds the tuning value. Parameters of the encoder 810 may be adjusted to reduce the third loss.

Here, the tuning value (T _G ) maximizes the distribution characteristics of the Mahalanobis distance (more specifically, the standardized Mahalanobis distance) calculated using latent variables and the distribution characteristics of the quality anomaly score (Anomaly Score, AS). It can be defined as a loss value to maintain a form close to a normal distribution.

In addition, the third loss (L _{G_total} ) to which the tuning value is added is the Mahalanobis distance (more specifically, It may be defined as a loss value for maintaining the distribution characteristics of the standardized Mahalanobis distance) and the distribution characteristics of the quality anomaly score (Anomaly Score, AS) in a form as close to a normal distribution as possible.

In addition, the artificial intelligence model outputs result data and result values using various combinations of a plurality of samples constituting the quality statistical data for training, and the process of training the artificial intelligence model using the output result data and result values is repeated. It can be. Accordingly, the encoder 810, the decoder 820, and the discriminator 830 can be organically trained while each parameter is optimized.

The following describes the process of monitoring quality abnormalities for a specific base station after the AI model is trained.

Returning to FIG. 3 again, after the artificial intelligence model is trained, the control unit 120 may obtain latent variables output from the encoder by providing quality statistical data for inspection to the encoder (S330).

In addition, the control unit 120 may acquire the Mahalanobis distance of the latent variable (S350), and determine a quality abnormality score corresponding to quality statistics for inspection using the Mahalanobis distance (S370).

This will be described with reference to FIGS. 5 and 6 again.

The data collection unit 110 may collect statistical quality data of base stations (cells) to be monitored. In this case, the data collection unit 110 may periodically collect quality statistical data of the base station (cell) to be monitored. In addition, the control unit 120 may monitor the quality abnormality in the base station (cell) to be monitored in real time by processing the periodically collected quality statistical data using an artificial intelligence model.

Meanwhile, the controller 120 may provide quality statistical data collected from a specific base station (cell) to an artificial intelligence model corresponding to the specific base station (cell). Specifically, a plurality of artificial intelligence models corresponding to a plurality of base stations (cells) may be loaded in the decision device 100, and in this case, the control unit 120 transmits quality statistical data collected from a specific base station (cell) to a specific base station (cell). It can be provided to the artificial intelligence model corresponding to (cell).

That is, when a specific artificial intelligence model is trained by quality statistical data for training collected from a specific base station, the quality statistical data for inspection collected from a specific base station may be used for monitoring quality abnormalities using the specific artificial intelligence model.

Meanwhile, the quality statistical data used for training of artificial intelligence models was previously named quality statistical data for training. In addition, the quality statistical data used to determine the abnormal quality score may be referred to as quality statistical data for inspection.

The difference between the statistical quality data for inspection and the statistical quality data for training is that the statistical quality data for training is a combination of a plurality of samples, whereas the statistical quality data for inspection is composed of a single sample. For example, the quality statistical data for inspection may be quality statistical data corresponding to a time interval from 00:30 on April 11, 2020 to 00:45 on April 11, 2020.

Meanwhile, the AAE learning model-based latent variable derivation unit 122 may obtain a quality abnormality score by providing quality statistical data for inspection to the latest artificial intelligence model. For example, AI models can be trained and updated on a daily or two-day basis. In addition, the AAE learning model-based latent variable derivation unit 122 may obtain a quality abnormality score by providing quality statistical data for inspection to the most recently trained artificial intelligence model.

Meanwhile, in the training stage, the artificial intelligence model 800 includes an encoder 810, a decoder 820, and a discriminator 830. However, in the process of monitoring the occurrence of quality abnormalities in the base station (cell), only the decoder 820 and the discriminator 830 and the separate encoder 810 among the three components constituting the artificial intelligence model 800 can be used. .

The AAE learning model-based latent variable derivation unit 122 may obtain latent variables output from the encoder 810 by providing quality statistical data for examination to the encoder 810 .

Specifically, when the AAE learning model-based latent variable derivation unit 122 provides quality statistical data for inspection to the encoder 810, the encoder 810 responds to the quality statistical data for inspection based on parameters set in the encoder 810. It is possible to output a latent variable (Z) that

And since the encoder 810 is trained, the output latent variable Z may have characteristics close to a normal distribution.

Meanwhile, the Mahalanobis distance deriving unit 123 may obtain the Mahalanobis distance of the latent variable Z using the latent variable Z output from the encoder 810 .

Specifically, the Mahalanobis distance deriving unit 123 uses the latent variable Z output from the encoder 810, the average value of a plurality of latent vectors, and the inverse matrix of the covariance matrix to obtain the Mahalanobis distance of the latent variable Z. (MD(Z)) can be calculated (see Equation 1).

Meanwhile, an inverse matrix of an average value and a covariance matrix of a plurality of latent vectors used in the process of monitoring quality anomalies may be an inverse matrix of the latest average value and covariance matrix.

Specifically, as described above, an average value of a plurality of latent vectors and an inverse matrix of a covariance matrix are calculated during training of an artificial intelligence model. In addition, when the average value of the plurality of latent vectors and the inverse matrix of the covariance matrix are calculated, the AAE learning model-based latent variable derivation unit 122 may update the average value of the plurality of latent vectors and the inverse matrix of the covariance matrix.

In addition, the average value of the plurality of latent vectors and the inverse matrix of the covariance matrix updated (that is, generated in the training process of the most recent artificial intelligence model) are used to calculate the Mahalanobis distance (MD(Z)) of the latent variable (Z). can be used

Meanwhile, the abnormal score derivation unit 124 may determine an abnormal quality score corresponding to the statistical quality data for inspection using the Mahalanobis distance MD(Z) of the latent variable Z.

Specifically, the ideal score derivation unit 124 may take a logarithm of the Mahalanobis distance (MD(Z)) of the latent variable (Z) for training and standardize it. In this case, the standardized Mahalanobis distance can be expressed as log ₁₀ (MD(Z)).

In addition, the anomaly score derivation unit 124 may calculate an anomaly score using the normalization conversion function F(MD(Z)) below.

: 표준화된 마할라노비스 거리의 평균,

: 표준화된 마할라노비스 거리의 표준 편차,

: 오차 함수)(AS: quality anomaly score, x: standardized Mahalanobis distance,

: Average of standardized Mahalanobis distances,

: Standard deviation of standardized Mahalanobis distance,

: error function)

That is, the ideal score derivation unit 124 uses the Mahalanobis distance of the latent variables, the average of the Mahalanobis distances of the latent variables, and the standard deviation of the Mahalanobis distances, and the quality having a normalized value between 0 and 1. score can be calculated.

In this way, the quality abnormality score may be normalized to a value between 0 and 1 and derived.

)에는 상수 k가 포함되는데, 상수 k에 의해 품질 이상 점수 산출을 위한 정규 누적 분포 함수의 그래프의 완만도가 조정될 수 있다.Meanwhile, the formula for calculating the quality abnormality score is in the form of a normal cumulative distribution function (Gaussian CDF) modified by the k value. That is, the error function (

) includes a constant k, by which the smoothness of the graph of the normal cumulative distribution function for calculating the quality abnormality score can be adjusted.

Meanwhile, the determination device 100 may include an input unit (not shown), and the input unit (not shown) may receive a user input for setting the constant k. Also, the controller 120 may set a constant k based on a user input. In this case, as the value of the constant k increases, the graph may become smoother.

That is, by adjusting the smoothness of the graph through the input of the k value, the operator can directly set the degree of severity according to the size of the quality abnormality score.

Meanwhile, the control unit 120 may determine a final quality abnormality score using a plurality of quality abnormality scores obtained in time series.

Specifically, it is assumed that there are a first quality abnormality score, a second quality abnormality score, and a third quality abnormality score obtained in time series. Here, the first quality abnormality score is a score obtained based on the quality statistical data of the first time interval from 11:00 to 11:15, and the second quality abnormality score is from 11:15 to 11:30 is a score obtained based on statistical quality data of the second time interval of , and the third quality abnormality score is a score obtained based on statistical quality data of the third time interval from 11:30 to 11:45.

In this case, the control unit 120 may determine the final quality abnormality score using the most recently set number of quality abnormality scores.

For example, the controller 120 may determine an average value of the latest two quality abnormality scores (second quality abnormality score and third quality abnormality score) as the final quality abnormality score.

Alternatively, the controller 120 may apply weights to a plurality of abnormal scores, respectively, and obtain final quality abnormal scores using the plurality of abnormal scores to which the weights are respectively applied. In this case, the highest weight is applied to the most recent quality abnormality score, and the weight applied to the quality abnormality score may decrease as the distance from the current point in time increases.

For example, the controller 120 applies a weight of 1.5 to the third abnormal quality score, a weight of 1 to the second quality abnormal score, and a weight of 0.5 to the first quality abnormal score, and averages the weighted quality abnormal scores to obtain a final quality or higher quality score. score can be determined.

In addition, the controller 120 may display or output the quality abnormality score (or the final quality abnormality score) through sound.

In this way, the control unit 120 may collect statistical quality data in real time and monitor the degree of quality abnormality in real time using the collected statistical quality data.

Also, when an update time arrives according to a predetermined cycle, the controller 120 may retrain the artificial intelligence model by updating the quality statistical data for training and providing the updated statistical data for training quality to the artificial intelligence model.

In addition, when the average value and covariance matrix of a plurality of latent vectors are newly calculated during the retraining process of the artificial intelligence model, the control unit 120 updates the inverse matrix of the existing average value and covariance matrix with the newly calculated average value and covariance matrix. can

7 is a diagram for explaining the overall operation of the present invention.

The control unit 120 may collect quality statistical data (S710). In this case, the controller 120 may collect a plurality of statistical quality data corresponding to a plurality of base stations (cells) in real time, and store the collected statistical quality data in the memory 130 .

Meanwhile, the control unit 120 may pre-process quality statistical data (S720).

Specifically, the control unit 120 may generate KPI data using raw data or normalize quality statistical data to be input to an artificial intelligence model.

Meanwhile, the controller 120 may update (train) the artificial intelligence model according to a predetermined cycle. In this case, the controller 120 may select the latest statistical quality data and train an artificial intelligence model using the selected statistical quality data (S730).

In addition, the controller 120 may train an artificial intelligence model by setting a target distribution of latent variables for training output from the encoder 810 as a Gaussian probability density function (PDF).

Also, the controller 120 may store an average value of a plurality of latent vectors obtained in the training process and an inverse matrix of the covariance matrix.

Meanwhile, the controller 120 may collect statistical quality data for inspection and provide the statistical quality data for inspection to an artificial intelligence model (encoder) to obtain latent variables (Z) output from the artificial intelligence model (encoder) ( S740). In this case, the control unit 120 may acquire the latent variable Z by providing quality statistical data for inspection to the latest artificial intelligence model (the latest encoder).

Then, the control unit 120 may calculate the Mahalanobis distance (MD(Z)) using the latent variable Z output from the latest encoder, the latest average value, and the inverse matrix of the latest covariance matrix (S750). ).

In addition, the control unit 120 calculates a quality abnormality score having a normalized value between 0 and 1 using the Mahalanobis distance of the latent variables, the average of the Mahalanobis distances of the latent variables, and the standard deviation of the Mahalanobis distances. It can be done (S760).

Then, the controller 120 outputs the calculated quality abnormality score, so that the operator can monitor the communication quality in real time (S770).

Meanwhile, the apparatus 100 for determining the abnormal quality score of the wireless communication network may be equipped with a plurality of artificial intelligence models respectively corresponding to a plurality of base stations (cells). Accordingly, it is possible to monitor base stations within a jurisdictional area using quality statistical data of a plurality of base stations (cells). Therefore, according to the present invention, it is possible to solve the conventional problem that a small number of administrators could not individually monitor quality deterioration of base stations within their jurisdiction.

For example, the operator monitors only those base stations (cells) whose quality abnormality score is higher than a certain value among all base stations (cells) within the jurisdiction, or by monitoring in order of higher quality abnormality scores, as before, dozens of types of KPI absolute values are monitored every time. It has the advantage that efficient wireless quality monitoring is possible without the need to check.

In addition, according to the present invention, since quality anomalies are detected by additionally using previously unavailable information such as correlation between quality KPIs and raw data not classified as KPIs, there is an advantage in that quality anomalies can be detected more accurately.

In addition, according to the present invention, since the artificial intelligence model corresponding to a specific base station is trained using quality statistical data for training collected from a specific base station, the artificial intelligence model corresponding to a specific base station is trained by reflecting the characteristics of the specific base station. . Therefore, there is an advantage in detecting quality anomalies by reflecting the communication environment of the base station or other surrounding environments.

In addition, according to the present invention, the artificial intelligence model is continuously updated by the latest quality statistical data. Therefore, it has the advantage of being able to detect quality anomalies by reflecting environmental changes over time.

In addition, carriers have tens of thousands or hundreds of thousands of base stations. In addition, it is impossible to create artificial intelligence models for anomaly detection of tens of thousands or hundreds of thousands of base stations with a supervised learning algorithm that labels correct values. However, in the present invention, by using an unsupervised learning algorithm, there is an advantage in that manpower and costs required for generating artificial intelligence models can be drastically reduced, and artificial intelligence models capable of detecting anomalies of numerous base stations can be easily generated.

In addition, according to the present invention, a plurality of artificial intelligence models respectively corresponding to a plurality of base stations are trained with the same training method, except that only input quality statistical data is different. Therefore, the training method proposed in the present invention is a universal method that can be applied to various national affairs, and according to the present invention, it has the advantage of saving time and cost for generating an artificial intelligence model.

In addition, according to the present invention, since the latest artificial intelligence model is trained by inputting the latest quality statistical data for training, the accuracy of the latest artificial intelligence model that has already been trained is used as it is, and the environment changes has the advantage of being flexible.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is Also, the computer may include a control unit. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: quality abnormality score determination device 110: data collection unit
120: control unit 130: memory
140: output unit

Claims (19)

A method for determining a quality abnormality score of a wireless communication network, each step being performed by a computing device,
training the artificial intelligence model by providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator;
acquiring latent variables output from the encoder by providing quality statistical data for inspection to the encoder after the artificial intelligence model is trained;
obtaining a Mahalanobis distance of the latent variable; and
Determining a quality abnormality score corresponding to the statistical quality data for inspection using the Mahalanobis distance;
A method for determining the quality anomaly score of a wireless communication network. According to claim 1,
The step of training the artificial intelligence model,
providing the quality statistical data for training to the encoder;
outputting, by the encoder, a latent variable for training using the quality statistical data for training;
outputting, by the decoder, result data using the latent variable for training;
determining, by the discriminator, whether the latent variable for training and the probability distribution sample data follow a normal distribution, and outputting a resultant value; and
Training the artificial intelligence model using the result data output by the decoder and the result value output by the discriminator;
A method for determining the quality anomaly score of a wireless communication network. According to claim 2,
The probability distribution sample data has a normal distribution
A method for determining the quality anomaly score of a wireless communication network. According to claim 3,
The step of training the artificial intelligence model using the result data output by the decoder and the result value output by the discriminator,
Calculating a first loss for allowing the encoder and the decoder to restore the statistical quality data for training using a difference between result data output by the decoder and the statistical quality data for training;
training the encoder and the decoder using the first loss;
calculating a second loss for allowing the discriminator to determine whether or not the normal distribution is followed by using a result value output by the discriminator;
training the discriminator using the second loss;
calculating a third loss for allowing the encoder to output a latent variable having the normal distribution, using a result value output by the discriminator; and
Training the encoder using the third loss;
A method for determining the quality anomaly score of a wireless communication network. According to claim 4,
The step of training the artificial intelligence model,
Further comprising: obtaining a Mahalanobis distance of the latent variable for training;
Training the encoder using the third loss,
Adding a tuning value reflecting the difference between the distribution of the Mahalanobis distance of the training latent variable and the standard normal distribution to the third loss, and training the encoder using the third loss to which the tuning value is added
A method for determining the quality anomaly score of a wireless communication network. According to claim 1,
The step of training the artificial intelligence model,
When the artificial intelligence model is trained based on the training quality statistical data including a plurality of samples, a plurality of latent vectors output by the encoder are obtained based on the plurality of samples, and Storing the mean value and the inverse matrix of the covariance matrix; comprising
A method for determining the quality anomaly score of a wireless communication network. According to claim 6,
Obtaining the Mahalanobis distance of the latent variable,
Obtaining the Mahalanobis distance using the latent variable, an average value of the plurality of latent vectors, and an inverse matrix of the covariance matrix
A method for determining the quality anomaly score of a wireless communication network. According to claim 1,
The quality statistical data for training and the quality statistical data for inspection,
Including at least one of raw data and KPI data collected from a specific base station
A method for determining the quality anomaly score of a wireless communication network. According to claim 1,
Updating the training quality statistical data, providing the updated training quality statistical data to the artificial intelligence model to retrain the artificial intelligence model, and updating the average value of a plurality of latent vectors and the inverse matrix of the covariance matrix step; further comprising
A method for determining the quality anomaly score of a wireless communication network. a data collection unit that collects quality statistical data for training and quality statistical data for examination; and
Quality statistical data for training is provided to an artificial intelligence model including an encoder, decoder, and discriminator to train the artificial intelligence model, and after the artificial intelligence model is trained, quality statistical data for inspection is provided to the encoder so that the encoder A controller that obtains an output latent variable, obtains a Mahalanobis distance of the latent variable, and determines a quality abnormality score corresponding to the quality statistical data for inspection using the Mahalanobis distance
A device for determining the quality abnormality score of a wireless communication network. According to claim 10,
The control unit,
Providing the quality statistical data for training to the encoder, obtaining result data output from the decoder using latent variables for training output by the encoder and latent variables for training;
Providing the latent variable for training and the probability distribution sample data to the discriminator so that the discriminator determines whether the latent variable for training and the probability distribution sample data follow a normal distribution;
Training the artificial intelligence model using the result data output by the decoder and the result value output by the discriminator
A device for determining the quality abnormality score of a wireless communication network. According to claim 11,
The probability distribution sample data has a normal distribution
A device for determining the quality abnormality score of a wireless communication network. According to claim 12,
The control unit,
A first loss for allowing the encoder and the decoder to restore the training quality statistical data is calculated using a difference between the result data output by the decoder and the quality statistical data for training, and using the first loss training the encoder and the decoder;
Calculating a second loss for the discriminator to determine whether the discriminator follows the normal distribution using the result value output by the discriminator, and training the discriminator using the second loss;
Calculating a third loss for the encoder to output a latent variable having the normal distribution using the result value output by the discriminator, and training the encoder using the third loss
A device for determining the quality abnormality score of a wireless communication network. According to claim 13,
The control unit,
obtaining the Mahalanobis distance of the latent variable for training;
Adding a tuning value reflecting the difference between the distribution of the Mahalanobis distance of the training latent variable and the standard normal distribution to the third loss, and training the encoder using the third loss to which the tuning value is added
A device for determining the quality abnormality score of a wireless communication network. According to claim 10,
The control unit,
When the artificial intelligence model is trained based on the training quality statistical data including a plurality of samples, a plurality of latent vectors output by the encoder are obtained based on the plurality of samples, and which stores the inverse of the mean value and covariance matrix
A device for determining the quality abnormality score of a wireless communication network. According to claim 15,
The control unit,
Obtaining the Mahalanobis distance using the latent variable, an average value of the plurality of latent vectors, and an inverse matrix of the covariance matrix
A device for determining the quality abnormality score of a wireless communication network. According to claim 10,
The quality statistical data for training and the quality statistical data for inspection,
Including at least one of raw data and KPI data collected from a specific base station
A device for determining the quality abnormality score of a wireless communication network. According to claim 10,
The control unit,
Updating the training quality statistical data, providing the updated training quality statistical data to the artificial intelligence model to retrain the artificial intelligence model, and updating the average value of a plurality of latent vectors and the inverse matrix of the covariance matrix
A device for determining the quality abnormality score of a wireless communication network. A computer program stored in a computer-readable recording medium to perform a method for determining a quality abnormality score of a wireless communication network,
training the artificial intelligence model by providing quality statistical data for training to an artificial intelligence model including an encoder, a decoder, and a discriminator;
acquiring latent variables output from the encoder by providing quality statistical data for inspection to the encoder after the artificial intelligence model is trained;
obtaining a Mahalanobis distance of the latent variable; and
A computer stored in a computer-readable recording medium to perform a method of determining a quality abnormality score of a wireless communication network, including determining a quality abnormality score corresponding to the quality statistical data for inspection using the Mahalanobis distance. program.

Images