KR102661221B1 - A method to detect abnormal symptoms occurring during login using text generated during login - Google Patents

Abstract

A first step of providing an evaluation module that determines whether there is an abnormality in login by using text generated during login to evaluate abnormal signs according to the log; A second step in which the evaluation module is provided with storage for storing a plurality of data, and an input device is connected and arranged to input a plurality of data from the outside; A third step of accumulating learning data input through the input device over a certain period of time and storing it in the storage; A fourth step of determining a threshold value by evaluating the accumulated plurality of learning data in the evaluation module; A fifth step of storing the determined threshold in storage; A sixth step of inputting evaluation data to be evaluated through the input device after the threshold is determined; A seventh step of determining a judgment value by analyzing the input evaluation data during the evaluation period; Provides an artificial intelligence-based abnormality detection method characterized by determining whether there is an abnormality in the evaluation data through an eighth step of generating abnormality by analyzing the relationship between the judgment value and the threshold value. .

Description

A method to detect abnormal symptoms occurring during login using text generated during login}

The present invention relates to a method of detecting abnormal signs that occur during login using a machine learning learning algorithm. The text that occurs during various login procedures is converted to ASCII code, and then the log numerical data converted to ASCII code is converted to ASCII code. It is used as learning data and evaluation data, and the learning data is applied to the machine learning learning algorithm to learn normal patterns, determine the threshold through this, and then analyze the evaluation data by applying the machine learning learning algorithm to generate abnormal signs. It's about something.

In particular, it is about a method of detecting abnormal signs for each login type using AI variable thresholds that vary the threshold values for each login type to generate abnormal symptoms for each login type.

Recently, the access records of tag passes recognized by the tag reader installed in the company's security area room and the web access records of the digital document security room accessed through the wired and wireless communication network from the user terminal (PC, smartphone, tablet PC) are managed by security management. It is stored in the server's database, and document security is necessary to prevent abnormal information leakage by monitoring the access records of the security room and the web access records of the digital document security room.

In a conventional document security system, when a client terminal encrypts digital information, authentication is performed using a key value predefined by the document security server, and after authentication, the digital information created by the client terminal is encrypted and stored in an external storage device. When accessing it again, it is decrypted and the digital information is viewed.

For example, a person who uses a 13.56MHz RFID tag on a pass equipped with an RFID tag reader in a security area room and a person who enters a digital document security room connected through a wired or wireless communication network from a user terminal, for example, a security manager. Entering 10 times, employees will enter 5 times, 3 times, 2 times, or 1 time depending on the work involved. By accumulating the access records of people entering and exiting the digital document security room using the tag on the security room pass, daily/weekly/monthly statistics are calculated for each individual, and the access records and web access records are analyzed and systematically managed to detect any abnormalities. It is necessary to manage abnormal data with .

As prior art 1 related to this, patent registration number 10-0750697 registers “a digital document security system equipped with shared storage with a user access function, and a document processing method using the system.”

A digital document security system equipped with shared storage with a user access function, configured to prevent abnormal leakage of digital information worked on by a computer, allows one of at least one DRM client terminal to connect to the shared storage and share the shared storage. It is configured to encrypt and store digital information in a storage medium and perform an editing function by decrypting the encrypted digital information according to a user access control function. The shared storage includes; It consists of a physical serial number provided so that each DRM client terminal can access and perform registration authentication, and a storage unit in which the digital information is stored.

The DRM client terminal is; An authentication process is performed by inputting the physical serial number of the shared storage, and after performing the authentication process, an encryption/decryption unit that encrypts and decrypts the digital information is linked to the shared storage to provide digital information. It is characterized by being composed of an application tool that provides permission setting functions such as editing.

As prior art 2 related to this, “Anomaly detection method and system using machine learning” is registered in Patent Registration No. 10-2185190.

Figure 1 is an exemplary diagram of an abnormality detection system 100 using conventional machine learning.

The detection method of an anomaly detection system using machine learning includes the steps of storing cost changes between predicted values and actual values for learning data using machine learning; Searching for a neighbor having a similar pattern to the target's cost change among the stored cost changes; and determining whether the target is normal or abnormal based on the difference between the searched neighbor's cost change and the target's cost change.

Storing the cost changes includes determining a threshold value for the cost to distinguish normal from abnormal based on a cost corresponding to test data; determining whether the test data is normal/abnormal based on the threshold; and storing a change in cost for a first time with respect to a normal situation while the cost is greater than the threshold in the test data, wherein the cost is a difference between the predicted value and the actual value.

In addition, text data is generated during login for each login type (e.g. Web Server login, Intrusion Detection System login, Splunk internal login, etc.), but abnormal signs are determined during login. There is a problem that there is no proper way to do.

Patent registration number 10-0750697 (registration date: August 13, 2007), “Digital document security system equipped with shared storage with user access function, and document processing method using the system”, MarkAny Co., Ltd. Patent registration number 10-2185190 (registration date: November 25, 2020), “Anomaly detection method and system using machine learning,” Electronics and Telecommunications Research Institute

The purpose of the present invention to solve the above problems is to provide text generated upon logging in for various login types (e.g., Web Server login, Intrusion Detection System login, Splunk internal login, etc.). We would like to propose a method to use data to analyze abnormal signs and issue warnings when logging in.

In addition, the purpose of the present invention is to propose a method for easily warning of abnormalities during login by converting the text generated during login into numerical values and analyzing the log numerical data.

In addition, the purpose of the present invention is to propose a method to more accurately warn of abnormalities that occur during login by having different thresholds for each login type and having a threshold suited to each characteristic.

In addition, the purpose of the present invention is to propose a method of determining abnormal signs according to the operation of the user terminal that has logged in through detecting abnormal signs that occur during the login process.

In order to achieve the purpose of the present invention, a first step is provided with an evaluation module that determines whether there is an abnormality in login by using text generated during login to evaluate abnormal signs according to the log; A second step in which the evaluation module is provided with storage for storing a plurality of data, and an input device is connected and arranged to input a plurality of data from the outside; A third step of accumulating learning data input through the input device over a certain period of time and storing it in the storage; A fourth step of determining a threshold value by evaluating the accumulated plurality of learning data in the evaluation module; A fifth step of storing the determined threshold in storage; A sixth step of inputting evaluation data to be evaluated through the input device after the threshold is determined; It includes a seventh step of analyzing the input evaluation data during the evaluation period and determining a judgment value.

In addition, the present invention is characterized by determining whether there is an abnormality in the evaluation data through an eighth step of generating the presence or absence of an abnormality by analyzing the relationship between the judgment value and the threshold value.

In addition, the present invention includes a first step of providing an evaluation module that determines whether there is an abnormality in login by using text generated during login to evaluate abnormal signs according to the log; A second step in which the evaluation module is provided with storage for storing a plurality of data, and an input device is connected and arranged to input a plurality of data from the outside; A third step of accumulating learning data input through the input device over a certain period of time and storing it in the storage; and a fourth step of determining a threshold value by evaluating the accumulated plurality of learning data in the evaluation module.

In addition, the present invention includes a fifth step of storing the determined threshold value in storage; a sixth step of inputting evaluation data to be evaluated through the input device after the threshold is determined; a seventh step of analyzing the input evaluation data during an evaluation period to determine a judgment value; An eighth step of analyzing the relationship between the judgment value and the threshold value to determine whether there is an abnormality in the evaluation data and detecting the abnormality of the user terminal through a test that occurs during the login process. It is characterized by judgment.

In addition, the present invention is that the learning data and the evaluation data are text data, text generated when logging in to a web server, text generated when logging in to an intrusion detection system, and Splunk. It is characterized by converting one or more of the internal log texts generated during the login process into a numerical value and applying the numerical log numerical data.

In addition, the present invention is characterized in that the certain period of time in the third step is 1 to 5 weeks.

Additionally, the present invention is characterized in that the evaluation period of the seventh step is on a daily or weekly basis.

In addition, in the present invention, as the evaluation module of the fourth step is equipped with a machine learning algorithm, the accumulated plurality of learning data is input to the machine learning algorithm to secure a plurality of learning samples through learning, and then the learning samples are used. Obtain a normal distribution curve, analyze the standard deviation (σ) from the normal distribution curve, and determine this as the threshold value,

The evaluation data is input into a machine learning (deep learning) algorithm to secure a judgment value.

In addition, the present invention is to secure a plurality of learning samples for the learning data through learning by inputting the accumulated plurality of learning data into the machine learning algorithm as the evaluation module of the fourth step is equipped with a machine learning algorithm. , an average value is obtained from the plurality of learning sample values, a difference value is obtained by subtracting the average value from each of the learning sample values, a threshold value is determined as a certain value among the difference values, and the evaluation data is subjected to machine learning (deep learning). It is characterized by obtaining an evaluation sample value for the evaluation data by inputting it into an algorithm, and securing a judgment value by subtracting the average value from the evaluation sample value.

In addition, the present invention determines the difference value closest to the top 70% of the difference values as the threshold value, and generates an abnormality warning when the judgment value is greater than the threshold value. It is characterized by

In addition, in the present invention, the value when the standard deviation is +1 is called the first upper threshold, the value when the standard deviation is +2 is called the second upper threshold, and the value when the standard deviation is +3 is called the third upper threshold. It is called the upper threshold, the value when the standard deviation is -1 is called the first lower threshold, the value when the standard deviation is -2 is called the second lower threshold, and the value when the standard deviation is -3 is called the lower threshold. The 3rd lower threshold value is characterized in that the 1st to 3rd upper threshold values and the 1st to 3rd lower threshold values are used as a standard for generating an abnormality warning.

In addition, the present invention is such that the evaluation module determines in the eighth step that the judgment value is between the first upper threshold and the second upper threshold, or between the first lower threshold and the second lower threshold. If present, a preliminary abnormality warning is generated, and if the judgment value is between the second upper threshold and the third upper threshold, or between the second lower threshold and the third lower threshold, an intermediate abnormality is generated. A warning is generated, and if the judgment value exceeds the third upper threshold or falls below the third lower threshold, an emergency abnormality warning is generated and transmitted to the warning management terminal.

In addition, according to the present invention, the learning data and evaluation data input to the evaluation module are input separately for all log numerical data subject to evaluation, so that different threshold values are set for all log numerical data. Through this, all log numerical data is characterized by determining whether or not there is an abnormality through different threshold values.

In addition, in the present invention, the learning data is Bij (where i is an index for each log type, i = 1, 2, 3, ..., p-1, p, p+1, ..., m-1, m, and j is the time series index for the log numerical data generated during the p type of login process, j = 1, 2, 3, ..., q-1, q, q+1, ..., n-1 , n, and Bpq is the log numerical data generated when the p type of login is performed for the qth time), and the first upper threshold Xi1 and the second upper threshold, which are the upper thresholds for each log type, are obtained through the learning data Bij. Find Xi2 and the third upper threshold Xi3, and find the lower thresholds, the first lower threshold Yi1, the second lower threshold Yi2, and the third lower threshold Yi3 (where i is the index for each log type, i = 1) , 2, 3, ..., p-1, p, p+1, ..., m-1, m), the evaluation data is Dik (where i is the index for each log type and i = 1 , 2, 3, ..., p-1, p, p+1, ..., m-1, m, and k is the time series index for the logarithmic data generated during the p type of login, where k = 1. , 2, 3, ..., r-1, r, r+1, ..., u-1, u, and Dpr is log numerical data generated when the p type of login is performed for the rth time). , The evaluation value Zi is obtained through the evaluation data Dik, and then the evaluation value is compared with the first to third upper threshold values and the first to third lower threshold values to determine abnormal signs.

In addition, the present invention sets the boundary value that determines the top 5% in the normal distribution curve as the upper threshold, or sets the boundary value that determines the bottom 5% in the normal distribution curve as the lower threshold, or uses the upper threshold and the lower threshold as the upper threshold. It is characterized by applying values simultaneously to determine abnormal signs.

In addition, the present invention is characterized in that in the eighth step, the evaluation module generates an abnormality warning and transmits it to the warning management terminal when the judgment value exceeds the upper threshold or falls below the lower threshold. Do it as

In addition, according to the present invention, the learning data and evaluation data input to the evaluation module are input separately for all log types subject to evaluation, so that different threshold values are set for all log types. All log types are characterized in that abnormal signs are determined through different threshold values.

In addition, in the present invention, the learning data is Bij (where i is an index for each log type, i = 1, 2, 3, ..., p-1, p, p+1, ..., m-1, m, and j is the time series index for the log numerical data generated during the p type of login process, j = 1, 2, 3, ..., q-1, q, q+1, ..., n-1 , n, and Bpq is log numerical data generated when the p type of login is performed for the qth time), and the upper threshold Xi and lower threshold Yi of each security component are obtained through the learning data Bij (where i is an index for each log type, where i = 1, 2, 3, ..., p-1, p, p+1, ..., m-1, m), and the evaluation data is Dik (where i is the index for each log type, where i = 1, 2, 3, ..., p-1, p, p+1, ..., m-1, m, and k is the number that occurred during login of type p. As a time series index for logarithmic numerical data, k = 1, 2, 3, ..., r-1, r, r+1, ..., u-1, u, and Dpr is the rth log of type p. (log numerical data generated during the process), the evaluation value Zi is obtained through the evaluation data Dik, and then the evaluation value is compared with the upper threshold value and the lower threshold value to determine abnormal signs.

In addition, the present invention is characterized in that the evaluation value Zi obtained through the evaluation data Dik is obtained by obtaining a normal distribution curve for the evaluation data Dik and then determining the average value of the normal distribution curve as the evaluation value Zi.

In addition, in the present invention, the learning data and evaluation data include one or more of the access IP, session ID, user identifier, access time, request page status code, web browser used, and byte size, and the access IP, session ID, user identifier, It is characterized by converting text indicating the access time, request page status code, web browser in use, and byte size into ASCII code, digitizing it, and using it as the learning data and the evaluation data.

In addition, the machine learning algorithm of the present invention includes unsupervised-visualization-detection (CNN), unsupervised-advance control-detection (AutoEncoder), unsupervised-memory-detection (LSTM), deep memory-detection (Deep LSTM), It is characterized by applying one or more of unsupervised-bidirectional-memory-detection (Bidirectional LSTM) and unsupervised-visualization-memory-detection (Convolution LSTM).

In addition, the present invention determines abnormal signs according to the log, but is characterized by identifying abnormal signs that occur when the user (user terminal) logs in by identifying the abnormal signs using text that occurs when the user (user terminal) logs in.

The present invention utilizes text data generated when logging in for various login types (e.g., Web Server login, Intrusion Detection System login, Splunk internal login, etc.) to detect abnormal signs during login. It has the effect of analyzing and warning.

In addition, by converting the text generated during the login process of the present invention into numbers and analyzing the log numerical data, there is an effect of easily warning of abnormal signs during the login process.

In addition, the present invention has a different threshold value for each login type, so that each login type has a threshold value suited to each characteristic, which has the effect of warning of abnormal signs that occur during login more accurately.

In addition, the present invention has the effect of determining abnormal signs according to the operation of the user terminal that has logged in through detecting abnormal signs that occur during the login process.

Figure 1 is an exemplary diagram of an abnormality detection system 100 using conventional machine learning.
Figure 2 is a diagram illustrating the configuration of an anomaly detection system using a variable threshold (threshold) according to the present invention.
Figure 3a is a screen showing AI numerical anomaly detection: a machine learning algorithm that detects abnormalities in learning data and detection data by learning normal patterns using a learning algorithm.
Figures 3b and 3c show AI detection options (Sensitivity, Duplication, Accumulated Data, Slope degree, Time Window Unit, Outlier/Inlier). This is the screen shown.
Figure 3d is a Test Data (KPI = 1) query (AI numerical anomaly detection) screen using learning data query (SPL) and detection data query (SPL).
Figure 3e is a Test Data Splunk Image (AI numerical anomaly detection) screen including a training data image (Splunk Image) and a detection data image (Splunk Image). Here, Splunk Image is an image obtained through an unstructured data analysis solution that searches, monitors, and analyzes big data through a web-based interface.
Figure 3f is a screen showing data visualization (KPI = 1) of learning data trends by date (trend graph) and learning data distribution (bar graph) showing frequency by value.
Figure 3g is a visualization of detection data (KPI = 1) including detection data trends by date (trend graph - anomaly pattern (Anomaly - Outlier)) and detection data distribution showing frequency by value (bar graph - anomaly pattern (Outlier)). It's a screen.
Figures 3h and 3i show that after learning a normal pattern for the training data using a learning algorithm, applying a numerical anomaly detection threshold using a machine learning model and outputting the detection data, the detection data (detection in Figure 3h) is shown in Figures 3h and 3i. This is a screen that outputs detection data by calculating the distance between the predicted value and the actual value (see data).
Figure 3j is a screen showing the AI detection sensitivity for daily detection data after the learning algorithm, showing that data with abnormalities (red line in the figure) is accurately detected in the detection data.
Figure 3k is a screen showing JMachine scenario - AI detection sensitivity to detection data and AI detection source data.
Figure 4a is a Test Data (KPI = 3) query (AI numerical anomaly detection) screen using learning data query (SPL) and detection data query (SPL).
Figure 4b is a Test Data Splunk Image (AI numerical anomaly detection) (KPI = 3) screen including a training data image (Splunk Image) and a detection data image (Splunk Image).
Figure 4c is a data visualization screen (KPI = 3) showing the learning data trend (trend graph) by time period for cpu_system_pct, cpu_user_pct, and tot_cpu_pct, and the learning data distribution (bar graph) showing the frequency by value.
Figure 4d shows detection data trends by time period for cpu_system_pct, cpu_user_pct, tot_cpu_pct (trend graph - anomaly pattern (Anomaly-Outlier)), and detection data distribution showing the frequency of cpu_system_pct, cpu_user_pct, tot_cpu_pct (bar graph - anomaly pattern (Outlier)). This is one screen containing detection data visualization (KPI = 3).
Figure 4e shows that after learning normal patterns by time zone/date for the cpu_system_pct, cpu_user_pct, and tot_cpu_pct learning data of the learning algorithm, detection data is output by applying an individual numerical abnormality detection threshold using a machine learning (deep learning) model. , cpu_system_pct, cpu_user_pct, and tot_cpu_pct This is a screen that calculates the distance between the predicted value and the actual value of the detection data by time zone/date and outputs the detection data. Figure 4f is a result of learning normal patterns of cpu_system_pct, cpu_user_pct, tot_cpu_pct learning data using a learning algorithm: A screen showing abnormal patterns after learning normal patterns by time slot/date for cpu_system_pct, cpu_user_pct, tot_cpu_pct learning data.
As shown in Figure 4f, in the visualization of the detection data in Figure 4d, the location of the abnormality as a result of analysis using the numerical anomaly detection threshold (threshold) obtained by learning the data with abnormality with the learning data indicates the location of the abnormality (the blue line in the graph indicates the abnormality). This is the location where the symptom is, and as a result of checking this with a machine learning (deep learning) algorithm, it can be confirmed that the location with the abnormal symptom is accurately detected (see 12, 13, and 14).
Figure 4g is a JMachine scenario - AI detection sensitivity and AI detection source data screen for cpu_system_pct, cpu_user_pct, and tot_cpu_pct detection data.
As shown in Figure 4g, in the visualization of the detection data in Figure 4d, the location of the abnormality is determined through a machine learning (deep learning) algorithm (JMachine) in the data with the abnormality (the red line in the graph is the location of the abnormality). ) as a result of checking with a machine learning (deep learning) algorithm, this screen shows that the location of the abnormality is accurately detected.
Figure 4h is a detection event visualization screen showing detection data showing histograms for each value and frequency for cpu_system_pct, cpu_user_pct, and tot_cpu_pct detection data.
Here, you can see that by analyzing cpu_system_pct as tot_cpu_pct = cpu_system_pct + cpu_user_pct, you can check abnormal signs about the user's CPU usage.
Referring to the explanation above, the same method as above can be applied to log numerical data to apply abnormal signs that occur during login.
In other words, the text generated when the user logs in is converted into log numerical data, and by analyzing the changing cpu_system_pct, cpu_user_pct, and tot_cpu_pct, if a pattern different from usual appears, a warning can be issued as an abnormality.
In this way, if abnormal signs of cpu_system_pct that occur when log numerical data changes are determined, abnormal signs related to login performed by the relevant user can be identified, which has the effect of responding to the abnormal signs.
Figure 5a shows AI log anomaly detection: Transform log text data into numeric data, then detect abnormal patterns in numeric data (learn normal patterns, quantify abnormal patterns and specify threshold) Test Data (KPI=1) Learning data query (SPL) ), Detection data query (SPL) - This is the AI log abnormality detection screen.
Figure 5b is a Test Data Splunk Image (AI log anomaly detection) screen including a training data image (Splunk) and a detection data image (Splunk).
Figure 5c is a screen with embedded text (AI log anomaly detection) of learning data and detection data in time-serial order by time and log_key.
Figure 5d is a screen showing text data generated during login.

Hereinafter, the configuration and operation of the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the present invention, if it is determined that a detailed description of a related known function or a known configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the same drawing number is assigned to different drawings when indicating the same configuration.

In order to prevent abnormal information leakage, the access record of the RFID tag pass of the visitor recognized by the 13.56MHz RFID tag reader installed in the company's security area room where document security is required, and the access record from the user terminal (PC, smartphone, tablet PC) Through the communication network, the web access records of the server's digital document security room are stored and managed in the database of the secure server.

Figure 2 is a diagram illustrating the configuration of an anomaly detection system using an AI variable threshold according to the present invention.

In this embodiment, (Example 1) the access record of the RFID tag pass 130 of the person recognized by the RFID tag reader 110 installed in the company's security area room, and the web access record (log record) of the digital document security room ), Additionally, (Example 2) CPU monitoring data of employee PCs within the company, which are monitored using network equipment having a remote PC CPU monitoring function, are stored in the database of the security server 200.

The anomaly detection system using the AI variable threshold of the present invention is a tag reader that transmits the entry record of the tag reader 110 tagging the tag pass 130 of the visitor when entering or leaving the document security room in the security area. PC (120) connected to (110); Entry and exit records of visitors holding a tag pass 130 tagged by the tag reader 110 on the door of the security area 100, digital documents on the security server, web access records (log records) in the security room, and CPU of the remote PC. Status monitoring data (cpu_system_pct, cpu_user_pct, tot_cpu_pct) is stored cumulatively in the database of a secure server for each individual and normalized according to individual z-score, average, and standard deviation according to statistics by time/day/period, and then AI variable threshold for each individual. Determine the values (upper limit, lower limit).

Optionally, if necessary, learn data including access records, web access records (log records), and CPU status monitoring data (cpu_system_pct, cpu_user_pct, tot_cpu_pct) of remote PCs using deep learning or machine learning learning algorithms to identify normal patterns. Learn and apply different AI variable thresholds (upper and lower limits) for each individual to the detection data in real time to separate normal pattern data (normal within 5 to 95% of normal distribution) and abnormal abnormal pattern data (normal with abnormal abnormal patterns). A security server (200) that separates and extracts (less than 5% and more than 95% of the normal distribution) and displays them on a UI screen through a user terminal, and outputs the final results, including logs, in a list or data visualization; and a user terminal 179 connected to the security server 200 through a wired or wireless communication network.

The digital document security room further includes a file server 230 that stores documents and files.

The system transmits the access record of the tag reader 110 tagging the tag pass 130 of the visitor when entering and exiting the document security room of the security area 100, and a PC connected to the tag reader 110 ( 120) is further included.

The security manager's user terminal is connected to the security server, selects the type of detection data, detection target and period, and access record/web access record/remote PC's CPU status monitoring data (cpu_system_pct) with different AI variable thresholds applied for each individual. , cpu_user_pct, tot_cpu_pct), normal patterns (upper limit, lower limit) of the learning data are displayed, detection data images detected in real time are output to the user terminal connected to the security server 200, and the learning data Visualization, detection data visualization with AI variable thresholds (upper limit, lower limit) and abnormal patterns displayed for each individual, extracting abnormal abnormal pattern data (less than 5%, more than 95%) and final results including log and list or This data is visualized and more than just the numbers are displayed on the screen.

The document storage room (100) in the security area within the company has an RFID tag reader (110) on the door, and the access record of the 13.56 MHz tag pass (130) of the visitor is sent to the security server (200) connected to the middleware through the document storage room PC (120). is transmitted.

The company is equipped with multiple user terminals 170 for each president, executive/department/employee.

The user terminal 170 can use a smartphone or tablet PC in addition to a PC.

The learning data includes Example 1) access records of visitors holding tag passes in the document security room of the security area, and web access records (log records) of the digital document security room of the security server.

Additionally, the learning data includes Example 2) CPU status monitoring (cpu_system_pct, cpu_user_pct, tot_cpu_pct) data of a remote PC using a virtual machine of the network equipment.

The evaluation module learns the normal pattern of the learning data, calculates the Z-score, average, and standard deviation based on the accumulated statistical data, normalizes them, and determines variable threshold values (upper and lower limits) for each individual. , detection data is provided by extracting abnormal abnormal patterns from the detection data according to variable thresholds (upper limit, lower limit) that are different for each individual, and the data is visualized and displayed.

The detection options for the above threshold include Sensitivity of AI detection data, De-duplication, Accumulated Area, Slope degree, Time Window Unit, Outlier/ Inliers (abnormal signs/normal signs) are included.

* Sensitivity refers to the degree of difference between predictions and actual values, and becomes a measure of future event detection.

* Data de-duplication: Excluding data with duplicate values

* Accumulated Area: Apply a filter using the area of the data (e.g. not detect when data has a low value below the standard value)

* Slope degree: Apply a filter using the slope of the data (e.g. do not detect events when there is a sharp decrease)

* Time Window Unit: Number of data rows in the window (e.g. 7 unit: 7 pieces of data considered as one pattern (input))

* Outlier/Inlier: Detected as an event among abnormal/normal signs

The security server 200 includes a WWW server 201 connected to the user terminal 170 through a wired or wireless communication network; A control unit 203 that controls security functions; DB 207 that stores access records, web access records (log records) of the digital document security room, CPU status monitoring (cpu_system_pct, cpu_user_pct, tot_cpu_pct) data of remote PCs, and other monitoring data when necessary; Member management department (209) that receives, stores, and manages member information; a user authentication unit 211 that authenticates the user using ID/Passwd or a personal certificate/universal certificate linked to an authentication server; an AI variable threshold setting unit 213 that sets different AI variable thresholds (upper limit, lower limit) for each individual based on statistics of learning data accumulated for each period; A machine learning unit 215 that learns normal patterns in the normal range from the upper limit to the lower limit by learning the corresponding training data using a deep learning or machine learning learning algorithm; An abnormal pattern detection unit 217 that detects an abnormal pattern of detection data detected in real time based on the AI variable threshold (upper limit, lower limit) that is different for each individual; A learning data/detection data output unit 219 that outputs the learning data and detection data in a learning data and detection data list or learning data visualization/detection data visualization; It includes a statistics processing unit 221 that provides data, lists, or visual data visualization that provides statistical information such as Z-score, average, variance, and standard deviation of individual/departmental/total data.

AI numerical abnormality detection is explained below. The evaluation model here is a machine learning algorithm that detects abnormal patterns in the numerical data of the detection data by learning the normal patterns of the training data through a learning algorithm (machine learning algorithm).

Figure 3a is AI numerical anomaly detection: a screen showing a machine learning algorithm that detects numerical data abnormal patterns by learning normal patterns using a learning algorithm.

The detection properties of the monitoring client of the user terminal connected to the security server are detection type (AI detection, AI numerical anomaly detection), detection basic information (detection target field name, aggregate query), AI machine learning (AI algorithm, machine learning query, machine Equipped with learning execution, detection query) and AI predictive detection (AI detection option).

AI machine learning executes training on training data by selecting deep learning or several types of machine learning algorithms from the “Run machine learning” menu.

The machine learning algorithm used here and its characteristics are as follows.

* Unsupervised-visualization-detection (CNN): Consists of a 1-layer CNN and is efficient for time series and image data

* Unsupervised-ahead control-detection (AutoEncoder): Reduces dimensionality by preserving data characteristics, can be used on various learning data

* Unsupervised-memory-detection (LSTM): Consists of 1-layer LSTM and is efficient for time series/text data

* Deep memory-detection (Deep LSTM): Consists of 3-layer LSTM and is efficient for time series/text data.

* Unsupervised-bidirectional-memory-detection (Bidirectional LSTM): Consists of 1-layer bidirectional LSTM and is efficient for time series/text data.

* Unsupervised-Visualization-Memory-Detection (Convolution LSTM): A structure that applies dimension-reduced data to LSTM, and is efficient for image or video data with spatial characteristics.

* Unsupervised-Bidirectional-Circulation-Detection (Bidirectional GRU): Consists of a 1-layer Bidirectional GRU, a simplified version of LSTM, and is effective for time series/text data.

* Unsupervised-bidirectional-stacked-circulation-detection (Stacked Bidirectional GRU): Consists of 3 layers of Bidirectional GRU and is effective for time series/text data

For reference, as the number of layers of the transformer encoder increases, it is more effective for complex or long sequence data. Data encoded with patch/position embedded data through k Transformer encoders is decoded by k decoders and learned.

Convolutional Neural Networks (CNN) are multilayer neural networks mainly used for character recognition and image analysis of videos. Convolutional Neural Network (CNN) uses multiple convolution layers and pooling layers in the form of pairs (convolution layer, pooling layer, convolution layer, pooling layer,...) , followed by a multilayer perceptron (MLP) with an input layer/hidden layer/output layer composed of several FC layers (fully-connected layers). For example, when a specific image is given as an input to a CNN, the feature map generated at each layer goes through a reconstruction process.

Features are extracted and classified step by step from the input image from each feature map. In the process of creating a feature map, the weights are called filters, and the mask used in the convolution layer, 2x2, 3x3, 4x4 or 2x4 used in the pooling layer. The 5x5 window and the collection of weights used in the FC layer can all be called filters. For down-sampling or sub-sampling in the pooling layer, it is used by selecting the mean function that calculates the average or the max function that selects the maximum value.

In addition, with the rapid development of computing power and computing technology such as CPU, GPU, memory, etc., machine learning algorithms such as RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), and GRU (Gated Recurrent Unit) have been developed into artificial neural networks. It maintains sequential data order and can learn a large amount of training data.

For reference, a Recurrent Neural Network (RNN) is a neural network with a structure in which the output of a specific node is re-input to that node. That is, it derives the result by simultaneously considering current input data and past input data. In addition, LSTM (Long Short-Term Memory) was proposed as a solution to the vanishing gradient problem in the learning of neural networks.

LSTM (Long Short-Term Memory neural network) has a gate structure that can add or delete information in the cell state. The gate can be selected in determining information and consists of a multiplication operation between the sigmoid neural network layer and the elements of the vector.

Bidirectional-LSTM (Bidirectional Long Short-Term Memory, Bi-LSTM, Bidirectional Long Short-Term Neural Network) uses the same input as the unidirectional LSTM model, and unlike the structure of the unidirectional LSTM model, it uses the time series of sentences of the language model in a bidirectional manner. Train using information.

The output of all tokens (size: 512x768) of each layer was used as the input of LSTM. LSTM consists of two layers, each LSTM layer consists of 512 LSTM cells, and the LSTM output of each layer (size: 512x192) ) are combined, then a fully connected layer is used.

Bidirectional GRU (Bidirectional Gated Recurrent Unit, Bi-GRU) is used as a sentence similarity measure.

After preprocessing, each sequence of tokens is embedded into a random vector through the word embedding layer. In the example, Bi-GRU sets the size to 256 and the contained vectors are calculated. In an embodiment, similarity is calculated as sigmoid using a full connect layer (FC) and Euclidean distance. Similarity has a value between 0 and 1, and the closer the similarity value is to 1, the more similar the two sentences are. To measure similarity, the distance from the fully connected layer (FC) is measured, and Euclidean distance, Cosine distance, Manhatten distance, Minkowski distance, and Chebyshev distance can be used.

Figures 3b and 3c are screens showing AI detection options (Sensitivity, Duplication, Accumulated Data, Sloop degree, Time Window Unit, Outlier/Inlier).

AI detection options include sensitivity of AI detection data, data deduplication, accumulated area, slope degree, time window unit, and outlier/inlier. signs/normal signs) are included.

* Sensitivity: Indicates the degree of difference between predictions and actual values, and becomes a measure of future event detection.

* Data de-duplication: Excluding data with duplicate values

* Accumulated Area: Apply filter using the area of the data

Example) Not detected when data is of low value)

* Slope degree: Apply a filter using the slope of the data (e.g. do not detect events when there is a sharp decrease)

* Time Window Unit: Number of data rows in the window

Example) 7 unit: 7 data considered as one pattern (input)

* Outlier/Inlier: Whether to detect abnormal/normal signs as events

Figure 3d is a Test Data (KPI = 1) query (AI numerical anomaly detection) screen using learning data query (SPL) and detection data query (SPL).

Here, KPI (Key Performance Indicator) indicates one type of key performance indicator data.

Figure 3e is a Test Data Splunk Image (AI numerical anomaly detection) screen including a training data image (Splunk Image) and a detection data image (Splunk Image).

Figure 3f is a data visualization screen (KPI = 1) showing the learning data trend by date (trend graph) and the learning data distribution (bar graph) showing the frequency by value according to UserID.

In Figure 3f, since KPI = 1, it can be confirmed that the learning data trend or learning data distribution is one type of data.

Figure 3g is a detection data visualization (KPI) including detection data trends by date (trend graph - anomaly pattern (Anomaly-Outlier)) and detection data distribution showing frequency by value (bar graph - anomaly pattern (Outlier)) according to UserID. = 1) It is one screen.

Figures 3h and 3i show that after learning a normal pattern for the learning data using a learning algorithm, detection data is output by applying an individual numerical abnormality detection threshold using a machine learning (deep learning) model, and the detection data This is a screen that calculates the distance between the predicted value and the actual value and outputs the detection data.

Figure 3j is an AI detection sensitivity screen for daily detection data after the learning algorithm.

Figure 3k is a JMachine scenario - AI detection sensitivity to detection data, AI detection source data screen.

As shown in Figures 3h and 3j, as a result of learning the training data through a machine learning (deep learning) model, the threshold value was determined to be 245.3, and then the distance (difference) between the predicted value and the actual value was calculated, and the sensitivity was 5 ( 2510), 6(2573), 7(3138). 8 (2959) can confirm that the distance is far, showing that abnormal signs can be detected.

Also, as shown in Figure 3k, a graph showing the AI detection source data and the sensitivity obtained through it is shown.

In addition, as shown in Figure 3i, as a result of learning the training data through a deep learning model, the threshold was determined to be 0.191897, and then the distance (difference) between the predicted value and the actual value was calculated, and the sensitivity was 12 (2.869) and 13 (1.975). , 14(1.390) can be confirmed to be far away, showing that abnormal signs can be detected.

Below, an example of processing using learning data and detection data will be described.

Figure 4a is a Test Data (KPI = 3) query (AI numerical anomaly detection) screen using learning data query (SPL) and detection data query (SPL).

Figure 4b is a Test Data Splunk Image (AI numerical anomaly detection) (KPI = 3) screen including a training data image (Splunk Image) and a detection data image (Splunk Image).

Figure 4c is a screen showing data visualization (KPI = 3) of learning data trends (trend graph) by date/time for cpu_system_pct, cpu_user_pct, and tot_cpu_pct learning data, and learning data distribution (bar graph) showing frequency by value.

When monitoring the history of CPU usage using a virtual machine on a network device, it shows CPU system (cpu_system_pct), CPU usage by random users (cpu_user_pct), and total CPU usage (tot_cpu_pct). Here, pct means percent, and abnormal signs of CPU usage used by a random user are determined through the relationship between the CPU usage used by a random user and the total CPU usage (tot_cpu_pct) of the CPU system (cpu_system_pct) and stem_pct. do.

Here, KPI = 3, and the KPI refers to the CPU system (cpu_system_pct), CPU usage of random users (cpu_user_pct), and total CPU usage (tot_cpu_pct).

Figure 4D shows detection data trends by date/time for cpu_system_pct, cpu_user_pct, and tot_cpu_pct (trend graph - anomaly pattern (Anomaly-Outlier)), detection data distribution showing the frequency by value (bar graph - anomaly pattern (Outlier)). Detection data visualization (KPI = 3) is one screen.

Figure 4e shows normal patterns by time/date for the cpu_system_pct, cpu_user_pct, and tot_cpu_pct learning data of the learning algorithm, and then applying an individual numerical anomaly detection threshold using a machine learning (deep learning) model to output detection data. This is a screen that calculates the Euclidean distance between the predicted value and the actual value of the detection data for each time zone/date for cpu_system_pct, cpu_user_pct, and tot_cpu_pct and outputs the detection data.

Figure 4f is a result of learning normal patterns of cpu_system_pct, cpu_user_pct, tot_cpu_pct learning data using a learning algorithm: A screen showing abnormal patterns after learning normal patterns by time slot/date for cpu_system_pct, cpu_user_pct, tot_cpu_pct learning data.

In Figure 4f, abnormal signs of CPU usage used by a random user can be determined through the relationship between the CPU usage used by a random user (user), the CPU system (cpu_system_pct), and the total CPU usage (tot_cpu_pct).

By doing this, it is possible to determine the abnormal symptoms of arbitrary users through the abnormal symptoms of the security components used by arbitrary users.

Figure 4g is a JMachine scenario - AI detection sensitivity and AI detection source data screen regarding cpu_system_pct, cpu_user_pct, and tot_cpu_pct detection data.

Figure 4h is a detection event visualization screen showing detection data showing histograms for each value and frequency for cpu_system_pct, cpu_user_pct, and tot_cpu_pct detection data.

Figure 5a shows AI log anomaly detection: Transforming log text data into numerical data, then detecting abnormal patterns in numerical data (learning normal patterns, quantifying abnormal patterns and specifying thresholds) Test Data (KPI=1) Learning data query (SPL) ), Detection data query (SPL) - AI log anomaly detection screen, and Figure 5b is a Test Data Splunk Image (AI log anomaly detection) screen including a training data image (Splunk) and a detection data image (Splunk). Figure 5c is a screen showing embedded text (AI log anomaly detection) of learning data and detection data in chronological order by time and log_key, and Figure 5d is a screen showing text data generated during login.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Hardware devices configured to store and execute program instructions in optical media and storage media such as ROM, RAM, flash memory, etc. may be included. Examples of program instructions may include those produced by a compiler, machine language code, as well as high-level language code that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention is implemented as a program and can be stored on a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) in a form that can be read using computer software. ) can be stored in .

Although the present invention has been described with reference to specific embodiments of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiments to illustrate the technical idea as described above, and is not limited to the technical idea and scope of the present invention. It can be implemented with various modifications, and the scope of the present invention should be determined by the claims described later.

100: Secure area document storage room 110: Tag reader
120: Document storage room PC 130: Tag pass of visitor
131: tag chip 170: user terminal
190: router 200: security server

Claims (16)

A first step of providing an evaluation module that determines whether there is an abnormality in login by using text generated during login to evaluate abnormal signs according to the log;
A second step in which the evaluation module is provided with storage for storing a plurality of data, and an input device is connected and arranged to input a plurality of data from the outside;
A third step of accumulating learning data input through the input device over a certain period of time and storing it in the storage;
A fourth step of determining a threshold value by evaluating the accumulated plurality of learning data in the evaluation module;
A fifth step of storing the determined threshold in storage;
A sixth step of inputting evaluation data to be evaluated through the input device after the threshold is determined.
A seventh step of determining a judgment value by analyzing the input evaluation data during the evaluation period;
An eighth step of generating abnormality by analyzing the relationship between the judgment value and the threshold value; determining whether there is an abnormality in the evaluation data through
As the learning data and evaluation data input to the evaluation module are input separately for all log numerical data subject to evaluation, different threshold values are set for all log numerical data, and through this, all log numerical data are input separately. Numerical data are each judged abnormal through different threshold values.
The learning data is Bij (where i is the index for each log type, i = 1, 2, 3, ..., p-1, p, p+1, ..., m-1, m, and j is a time series index for the log numerical data generated during the p type of login process, where j = 1, 2, 3, ..., q-1, q, q+1, ..., n-1, n, Bpq is log numerical data generated when the p type of login is performed for the qth time)
Through the learning data Bij, the first upper threshold Xi1, the second upper threshold Xi2, and the third upper threshold Xi3, which are the upper thresholds for each log type, are obtained,
Find the first sub-threshold Yi1, the second sub-threshold Yi2, and the third sub-threshold Yi3 (where i is the index for each log type, i = 1, 2, 3, ..., p- 1, p, p+1, ..., m-1, m)
The evaluation data is Dik (where i is the index for each log type, i = 1, 2, 3, ..., p-1, p, p+1, ..., m-1, m, k is a time series index for the log numerical data generated during the p type of login process, where k = 1, 2, 3, ..., r-1, r, r+1, ..., u-1, u, and Dpr is the log numerical data generated when the p type of login was performed for the rth time).
When logging in, the evaluation value Zi is obtained through the evaluation data Dik, and abnormal symptoms are determined by comparing the evaluation value with the first to third upper threshold values and the first to third lower threshold values. A method of detecting abnormal signs that occur when logging in using the text that occurs. delete In paragraph 1
The learning data and the evaluation data are text data, including text generated when logging into a web server, text generated when logging into an Intrusion Detection System, and text from Splunk's internal log. more than one
A method for detecting abnormal signs that occur during login using text generated during login, characterized in that the text generated during the login process is converted into numbers and the numerical log numerical data is applied. In paragraph 1
A method for detecting abnormal signs that occur during login using text generated during login, characterized in that the certain period of time in the third step is 1 to 5 weeks. In paragraph 1
A method for detecting abnormal signs that occur during login using text generated during login, characterized in that the evaluation period of the seventh step is daily or weekly. In paragraph 3
As the evaluation module of the fourth step is equipped with a machine learning algorithm, the accumulated plurality of learning data are input to the machine learning algorithm to secure a plurality of learning samples through learning, and then a normal distribution curve is created using the learning samples. Obtain and analyze the standard deviation (σ) from the normal distribution curve to determine this as the critical value,
A method of detecting abnormal signs that occur during login using text generated during login, characterized by securing a judgment value by inputting the evaluation data into a machine learning (deep learning) algorithm. In paragraph 3
As the evaluation module of the fourth step is equipped with a machine learning algorithm, the accumulated plurality of learning data are input to the machine learning algorithm to secure a plurality of learning samples for the learning data through learning, and then The average value is obtained from the learning sample values, the difference value is obtained by subtracting the average value from each of the learning sample values, and then a certain value among the difference values is determined as a threshold value,
Login using the text generated during login, characterized by inputting the evaluation data into a machine learning (deep learning) algorithm to obtain an evaluation sample value for the evaluation data, and securing a judgment value by subtracting the average value from the evaluation sample value. Method for detecting abnormal signs that occur when In paragraph 7
A certain value determined as a threshold among the difference values determines the difference value closest to the top 70% of the difference values as the threshold value, and generates an abnormality warning when the judgment value is greater than the threshold value. A method of detecting abnormal signs that occur during login using text generated during login. In paragraph 6
The value when the standard deviation is +1 is called the first upper threshold, the value when the standard deviation is +2 is called the second upper threshold, and the value when the standard deviation is +3 is called the third upper threshold. ,
The value when the standard deviation is -1 is called the 1st lower threshold, the value when the standard deviation is -2 is called the 2nd lower threshold, and the value when the standard deviation is -3 is called the 3rd lower threshold. ,
A method for detecting abnormal signs generated during login using text generated during login, characterized in that the first to third upper threshold values and the first to third lower threshold values are used as a standard for generating an abnormality warning. . In paragraph 9
The evaluation module is performed in step 8.
If the decision value is between the first upper threshold and the second upper threshold, or between the first lower threshold and the second lower threshold, a preliminary abnormality warning is generated.
If the decision value is between the second upper threshold and the third upper threshold, or between the second lower threshold and the third lower threshold, an intermediate abnormality warning is generated,
If the judgment value exceeds the 3rd upper threshold or falls below the 3rd lower threshold, an emergency abnormality warning is generated and transmitted to the warning management terminal. A method of detecting abnormal signs that occur when logging in. delete delete In paragraph 6
The boundary value that determines the top 5% of the normal distribution curve is set as the upper threshold, the boundary value that determines the bottom 5% of the normal distribution curve is set to the lower threshold, or the upper threshold and the lower threshold are applied simultaneously. A method of detecting abnormal signs that occur during login using text that occurs during login, which is characterized by determining abnormal signs. In paragraph 13
The evaluation module is performed in step 8.
If the judgment value deviates from the upper threshold or falls short of the lower threshold, an abnormal warning is generated and delivered to the warning management terminal. Generated during login using text generated during login. Abnormal symptom detection method. In paragraph 14
As the learning data and evaluation data input to the evaluation module are input separately for all log types subject to evaluation, different threshold values are set for all log types, and through this, all log types are determined. A method of detecting abnormal signs that occur during login using text generated during login, characterized in that abnormal symptoms are determined through different threshold values. In paragraph 15
Determine abnormal signs according to the log, but identify abnormal signs using text that occurs when the user (user terminal) logs in. Text that occurs during login is characterized by identifying abnormal signs that occur when the user logs in. A method of detecting abnormal signs that occur when logging in.