KR20240000934A - IT infrastructure failure pre-processing system by machine learning algorithm - Google Patents

Abstract

The IT infrastructure failure pre-processing system using machine learning algorithms extracts raw data for each IT infrastructure and source code development system, recognizes failure-causing factors using a cognitive model, and extracts the source code corresponding to the raw data and failure-causing factors. Microagent that labels extracted source code units; a data collection unit that receives labeled source code from the microagent and outputs first failure data, which is source code likely to cause a failure, through an analysis model; a failure prediction unit that inputs the first failure data into a failure prediction model and outputs second failure data that divides the first failure data into non-failure elements and failure elements to predict failures in the IT infrastructure; A failure prevention unit that, upon receiving the second failure data comprised of the failure elements, processes the failure-causing elements before an IT infrastructure failure occurs according to a failure response model; A visualization unit that visualizes and provides the basis for the process of outputting the first failure data and the second failure data and the results of the process, and the results of processing the failure-causing factors in the failure prevention unit; and a machine learning unit that generates or updates the cognitive model, analysis model, failure prediction model, and failure response model according to a reinforcement learning algorithm.

Description

IT infrastructure failure pre-processing system by machine learning algorithm}

The present invention relates to an IT infrastructure failure pre-processing system using a machine learning algorithm that processes factors causing IT infrastructure failures detected in advance by a machine learning algorithm before an IT infrastructure failure occurs.

IT Infrastructure refers to the systems and structures that form the basis of IT services, including networks, servers, databases, information security, system software, and infrastructure.

With the era of the Fourth Industrial Revolution and the rapid increase in demand for IT services, there is a significant potential risk factor for IT service infrastructure throughout society. In the industry, the frequency of failures is increasing due to the rapid increase in demand for IT infrastructure, and the resulting losses are also significant. It is increasing. Since 2017, the government has been strongly recommending the discovery and action of necessary improvements to effectively prevent and respond to information system failures. Compared to the market environment and demand, the development of failure prediction and prevention technology in the IT field has made significant progress. The reality is that there is no such situation, and the application of AI technology to this field is even more distant.

In addition, the shortage of specialized and experienced technical personnel in the field of IT failure management is becoming increasingly severe, while the occurrence of failures is increasing explosively due to the increase in web/mobile services. Automated failure prevention technology in a short period of time by detecting source code errors and difficult-to-identify causes of failures in advance through AI during the development stage, along with analysis technology for AI's prediction basis and process to prepare for incorrect judgments due to AI errors. The demand for it will increase rapidly not only now but also in the future.

In contrast, the introduction of IT infrastructure failure prediction systems based on domestic and foreign servers still faces difficult problems, such as high cost compared to the expected effect and configuring a solution that is not easy to optimize for expected demand, including most IT startup companies. Small and medium-sized web/mobile service companies are experiencing great difficulties in operating and managing server systems due to their size. It is common for a single server failure to cause a drop in brand value or lower reliability. Due to a lack of professional manpower, immediate recovery is not possible in the event of a server failure, resulting in membership withdrawal, transaction and payment errors, and fatal blows to sales, leading to business closure. The seriousness of the problem is very serious, but realistic countermeasures are lacking.

When it comes to server system-related failures, there are many facts in which the actual cause of the failure is not known or should not be known to the outside world from a company's perspective. Financial institutions and other companies engaged in various profit-making businesses through web/mobile services often postpone system patches because they cannot stop the service. For this reason, even though they are aware of the risks, they are still vulnerable to external attacks and threats until they stop the system and apply the patch. There are many cases where people are exposed to disabilities. In this case, if it is reported or known externally as the cause of the failure, it will be exposed as a fatal weakness in the company's technological capabilities, so not only will it not be disclosed, but it will also be exposed to repeated failures in the future. In addition, the reality in the IT industry is that most small and medium-sized businesses are unable to apply patches and perform system inspections due to a lack of professional manpower.

In addition, a company's complex IT infrastructure environment is a representative reason why patch application and updates are difficult. Databases are often connected to middleware or applications, and patch and upgrade policies can vary from simple recommendations to version upgrades. In particular, when a problem is discovered and the version needs to be upgraded, it is common to have no choice but to postpone the patch due to linkage with overall infrastructure such as DB, OS, applications, and solutions. This is because it is not easy to change middleware that is technically tightly interconnected with other applications to upgrade the DB or OS.

Accordingly, the present applicant has proposed in Korea Patent No. 10-2295868 a network failure prediction system that uses a learning model based on machine learning to predict failures caused by errors in source code at the system development stage.

Republic of Korea Patent No. 10-2295868 Republic of Korea Patent No. 10-2391510

The purpose of the present invention is to provide an IT infrastructure failure pre-processing system using a machine learning algorithm that automatically processes IT infrastructure failure-causing factors recognized using a learning model based on a machine learning algorithm before an IT infrastructure failure occurs.

The IT infrastructure failure pre-processing system based on the machine learning algorithm of the present invention extracts raw data for each IT infrastructure and source code development system, recognizes failure-causing elements using a cognitive model, and source code corresponding to the raw data and failure-causing elements. A microagent that extracts and labels extracted source code units; a data collection unit that receives labeled source code from the microagent and outputs first failure data, which is source code likely to cause a failure, through an analysis model; a failure prediction unit that inputs the first failure data into a failure prediction model and outputs second failure data that divides the first failure data into non-failure elements and failure elements to predict failures in the IT infrastructure; A failure prevention unit that, upon receiving the second failure data comprised of the failure elements, processes the failure-causing elements before an IT infrastructure failure occurs according to a failure response model; A visualization unit that visualizes and provides the basis for the process of outputting the first failure data and the second failure data and the results of the process, and the results of processing the failure-causing factors in the failure prevention unit; And a machine learning unit that generates or updates the cognitive model, analysis model, failure prediction model, and failure response model according to a reinforcement learning algorithm, wherein the failure prevention unit collects information on failures in the actual IT infrastructure that occurred in the past. A failure information management module that creates a database and manages IT infrastructure failure types that can be prevented before IT infrastructure failures occur (hereinafter referred to as “first failure type”) based on past failure information; A first judgment module that receives the second failure data and determines whether it belongs to the first failure type; a failure processing module that inputs the second failure data determined as the first failure type by the first judgment module into the failure response model, processes the failure-causing factors according to the failure response model, and outputs the processing result; and a result transmission module that transmits the processing results output by the error processing module to the machine learning unit.

In addition, the microagent includes a data collection module that extracts and collects the raw data from the IT infrastructure and source code development system; a data filtering unit that identifies and processes sensitive information included in the raw data; a recognition module that receives the raw data from the data filtering unit, inputs it into the recognition model, and recognizes the failure-causing factors by the recognition model; a source code collection module that collects source code related to the failure-causing element from the source code development system; A labeling module that automatically labels the source code collected in the source code collection module according to a labeling algorithm; and a data transmission module that transmits the source code labeled in the labeling module to the data collection unit.

Additionally, the data filtering unit includes a sensitive information identification module that identifies sensitive information included in the raw data; a sensitive information determination module that determines whether the sensitive information identified in the sensitive information identification module is essential sensitive information necessary for the recognition module to recognize the failure-causing element; and a sensitive information processing module that automatically replaces the sensitive information determined to be essential sensitive information with general information and deletes the sensitive information that is not essential sensitive information from raw data.

In addition, the disability prevention department divides the first failure type into 1-1 failure type that requires administrator's approval for failure handling and 1-2 failure type that does not require administrator's approval for failure handling, and then creates a database. A failure prevention type management module that manages and manages; a second judgment module that determines whether the second fault data belonging to the first fault type is a 1-1 fault type or a 1-2 fault type by the first judgment module; When the second failure data is determined to be the first failure type, an approval module reports the first failure type to the manager terminal, requests approval for failure processing, and receives approval from the manager terminal. ; further comprising; wherein the failure processing module inputs the second failure data for the 1-1 failure type and the 1-2 failure type approved for failure handling from the administrator terminal into the failure response model; , the failure-causing factors are processed using the failure response model and the processing results are output.

In addition, the machine learning unit includes a cognitive model learning unit that generates or updates the cognitive model according to a reinforcement learning algorithm; An analysis model learning unit that generates or updates the analysis model according to a reinforcement learning algorithm; A failure prediction model learning unit that generates or updates the failure prediction model according to a reinforcement learning algorithm; and a failure response model learning unit that generates or updates the failure response model according to a reinforcement learning algorithm.

In addition, the microagent generates recorded data divided into labeled source code units and creates a database, and the source code labeled by the microagent is processed by the data collection unit, failure prediction unit, and failure prevention unit. It further includes a record management unit that records and manages the record data, wherein the visualization unit backtracks the content recorded in the record data of the source code for the second failure data and visualizes the cause of the failure and the failure prediction process. Visualization module; and a failure prevention visualization module that visualizes the process and results of processing the failure-causing factors by backtracking the contents recorded in the record data of the source code for the second failure data.

In addition, the failure prediction unit receives the first failure data from the data collection unit and inputs it into each of at least two failure prediction models, and each of the failure prediction models divides the first failure data into a spare failure element and a spare failure element. A preliminary failure element selection unit that outputs second preliminary failure data; And by comparing the second preliminary failure data output from each of the two failure prediction models, the second preliminary failure data output as a preliminary failure element by at least one of the at least two failure prediction models is IT. It includes a failure element selection unit that outputs secondary failure data, which is source code that may cause a failure to the infrastructure.

According to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention, IT infrastructure failure-causing factors recognized before the IT infrastructure failure occurs are processed based on source code, and IT infrastructure failure-causing factors are processed based on the machine learning algorithm. It can provide a basis for the process.

In addition, the present invention can prevent leakage of sensitive information by deleting or automatically replacing sensitive information included in raw data with general information when recognizing factors causing failures in IT infrastructure.

In addition, by extracting failure-causing factors through a distributed artificial intelligence model, not only is it possible to quickly recognize failure-causing factors from raw data, the time required to predict IT infrastructure failures is shortened, and the accuracy of predicting IT infrastructure failures is improved. do.

1 shows machine learning according to an embodiment of the present invention This is a conceptual diagram of an algorithmic IT infrastructure failure pre-processing system.
FIG. 2 is a conceptual diagram of the microagent of FIG. 1.
FIG. 3 is a table showing raw data collected by the microagent of FIG. 1.
FIG. 4 is a conceptual diagram of the data filtering unit of FIG. 2.
Figure 5 is a conceptual diagram of the failure prediction unit of Figure 1.
Figure 6 is a conceptual diagram of the disability prevention department of Figure 1.
Figure 7 is a conceptual diagram of the visualization unit of Figure 1.
Figure 8 is a conceptual diagram of the machine learning unit of Figure 1.
Figure 9 shows machine learning according to an embodiment of the present invention. This is a flowchart of how to predict and deal with IT infrastructure failures based on algorithms.
FIG. 10 is a flowchart illustrating the first step of FIG. 9 in detail.
FIG. 11 is a flow chart illustrating steps 1 and 2 of FIG. 10 in detail.
FIG. 12 is a flowchart illustrating the third step of FIG. 9 in detail.
FIG. 13 is a flowchart illustrating the fourth step of FIG. 9 in detail.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

The present invention relates to an IT infrastructure failure pre-processing system using a machine learning algorithm that can prevent IT infrastructure failures in advance by detecting and processing factors causing IT infrastructure failures in advance using a machine learning algorithm.

Figure 1 is a conceptual diagram of an IT infrastructure failure pre-processing system 100 using a machine learning algorithm according to an embodiment of the present invention.

Referring to FIG. 1, the IT infrastructure failure pre-processing system 100 using a machine learning algorithm according to an embodiment of the present invention includes a microagent 110, a data collection unit 120, a failure prediction unit 130, and visualization. It includes a unit 140, a machine learning unit 150, a record management unit 160, and a failure prevention unit 170.

The IT infrastructure 10 includes an application platform for executing applications including a web server, WAS, and DBMS server of a company or organization, and integrated management of processes or information inside and outside the organization, including a CRM server, ERP server, and SCM server. The IT infrastructure failure pre-processing system 100 using the machine learning algorithm of the present invention includes network equipment including routers, backbones, and switches, mail servers, storage, and storage. ) by collecting, refining, extracting, classifying, processing, and analyzing failure data to predict IT infrastructure failures based on machine learning algorithms through repeated learning and then processing them to prevent IT infrastructure failures, detect and predict IT infrastructure failures , the processed results and the basis for their judgment are tracked and visualized.

The microagent 110 extracts raw data for each IT infrastructure 10 and source code development system 20, recognizes failure-causing elements using a cognitive model, and extracts source code corresponding to the raw data and failure-causing elements. Labeling is performed in units of extracted source code.

FIG. 2 is a conceptual diagram of the microagent 110 of FIG. 1. Referring to Figure 2, the microagent 110 according to an embodiment of the present invention includes a data collection module 111, a data filtering unit 112, a recognition module 113, a source code collection module 114, and a labeling module. (115) and data transmission module 116.

The data collection module 111 extracts and collects the raw data from the IT infrastructure 10 and the source code development system 20. At this time, the raw data collected by the data collection module 111 may be predefined in advance, and FIG. 3 is a table showing the raw data collected by the microagent 110 of FIG. 1.

Referring to FIG. 3, the data collection module 111 collects raw data about servers, networks, DBMS, applications, application solutions, and application platforms.

For example, in the case of a DBMS server, the raw data includes information about the server including CPU, memory, disk, process, file system, and network IO, and information about the DBMS installed on the DBMS server.

At this time, information about the DBMS may include information about the IO performance of the database, tables, and indexes.

The data filtering unit 112 identifies and processes sensitive information included in the raw data. FIG. 4 is a conceptual diagram of the data filtering unit 112 of FIG. 2.

Referring to FIG. 4, the data filtering unit 112 according to an embodiment of the present invention includes a sensitive information identification module 112a, a sensitive information determination module 112b, and a sensitive information processing module 112c.

The sensitive information identification module (112a) identifies sensitive information included in the raw data, and the sensitive information determination module (112b) recognizes the sensitive information identified in the sensitive information identification module (112a) in the recognition module (113). It determines whether it is essential sensitive information necessary to recognize a failure-causing factor, and the sensitive information processing module 112c automatically replaces sensitive information determined as essential sensitive information with general information, and sensitive information that is not essential sensitive information is raw data. Delete from

For example, in the case of a DBMS server, if the table name of information about the DBMS includes the product name while being secretly developed by a company or organization, the DBMS table name may be sensitive information, and the service provided through the IT infrastructure (10) may be classified as sensitive information. If the user's personal information is included in the raw data, the user's personal information becomes sensitive information.

At this time, the user's personal information may not be information that is essential for recognizing failure-causing factors in the recognition module 113. In this case, the user's personal information is primarily used in the sensitive information identification module 112a. It is classified as sensitive information, and is secondarily excluded from essential sensitive information in the sensitive information determination module 112b, and the user's personal information is deleted from the raw data in the sensitive information processing module 112c.

Meanwhile, in the case of the DBMS table included in the product name under development by the company, it is primarily classified as sensitive information in the sensitive information identification module 112a, and secondarily classified as essential sensitive information in the sensitive information identification module 112b. The sensitive information processing module 112c automatically replaces the DBMS table name including the product name under development, which is the essential sensitive information, with a DBMS table name of a general name.

In addition, whether the collected raw data is sensitive information may be defined in advance and inputted and stored in the sensitive information identification module 112a in advance, and whether the raw data is essential sensitive information may be defined in advance and the sensitive information may be stored in advance. It may be input to the processing module 112c or learned through a reinforcement learning algorithm.

The microagent 110 according to an embodiment of the present invention includes a data filtering unit 112 that deletes sensitive information included in raw data or automatically replaces it with general information, thereby creating a failure-causing element for the IT infrastructure 10. By recognizing this, you can prevent sensitive information, such as the name of a product under development by the company or the user's personal information, from being leaked to the outside.

Referring again to FIG. 2, the recognition module 113 receives the raw data from the data filtering unit, inputs it into the recognition model, and recognizes the failure-causing factors by the recognition model, and the recognition model is strengthened. It can be created or updated in the machine learning unit 150 by a learning algorithm, and the cognitive model will be described in detail with reference to FIG. 8.

The source code collection module 114 collects source codes related to the failure-causing factors from the source code development system 20. At this time, the source code collected by the appeal source code collection module 114 is a source code according to a preset value according to the type of each server of the IT infrastructure 10 targeting the small code classified as server system source in FIG. Collect.

For example, in the case of a DBMS server, the source code related to the error-causing factors recognized by the cognitive model can be DBMS area execution information for the corresponding DBMS server.

Meanwhile, the labeling module 115 automatically labels the source code collected by the source code collection module 114 according to a labeling algorithm.

At this time, the labeling algorithm can use the labeling algorithm of the Presudo-labeling technique. When using the labeling algorithm of the Presudo-labeling technique, if the source code cannot be reliably labeled because it has not been sufficiently learned, the labeling algorithm may be used by the administrator or another system. Reinforcement learning can be performed repeatedly by inputting labeled source code.

The labeled source code can be managed by the record management unit 160 and used in the visualization unit 140.

The data transmission module 116 transmits the source code labeled in the labeling module 115 to the data collection unit 120. At this time, the data transmission module 116 uses a security protocol such as Https, SSL, etc., uses RSA2048 for server/client session authentication, an algorithm to avoid DDOS, protocol compression technology to reduce the amount of transmitted data, A timeout processing function can be installed to prevent thread resource consumption. In addition, it includes the necessary communication protocols according to the type of each server of the IT infrastructure 10.

In addition, if the data transmission module 116 uses the TCP/IP protocol to transmit data to the data collection unit 120, the response may be delayed. In order to minimize this, the data transmission module 116 After data to be transmitted to the data collection unit 120 is input into a message queue having a queue-type data structure, the data to be transmitted immediately next is processed, and the data input to the message queue is entered in the order in which it was received. It can be transmitted to the data collection unit 120 using the TCP/IP protocol.

Referring again to FIG. 1, the data collection unit 120 receives labeled source code from the microagent 110 and outputs first failure data, which is source code that is likely to cause a failure, through an analysis model.

The analysis model may be created or updated in the machine learning unit 150 using a reinforcement learning algorithm. The analysis model will be described in detail with reference to FIG. 8.

The failure prediction unit 130 inputs the first failure data into at least two failure prediction models and outputs second failure data that divides the first failure data into non-failure elements and failure elements to form the IT infrastructure 10. Predict failures for

At this time, the non-failure element refers to source code that is unlikely to cause a failure, and the obstacle element refers to source code that is highly likely to cause a failure.

Each of the at least two failure prediction models may be generated or updated in the machine learning unit 150 by a reinforcement learning algorithm like an analysis model. The failure prediction models will be described in detail with reference to FIG. 8.

Figure 5 is a conceptual diagram of the failure prediction unit 130 of Figure 1. Referring to FIG. 5, the failure prediction unit 130 includes a preliminary obstacle selection unit 131 and a failure element selection unit 132.

The preliminary failure element selection unit 131 receives the first failure data from the data collection unit 120 and inputs it into each of at least two failure prediction models, and each of the failure prediction models divides the first failure data into a preliminary failure element and Outputs the second preliminary failure data classified by preliminary failure elements.

The obstacle selection unit 132 compares the second preliminary failure data output by each of the two failure prediction models to determine the preliminary failure element output by at least one failure prediction model among the at least two failure prediction models. The second preliminary failure data is finally output as second failure data, which is source code that has the potential to cause failure to the IT infrastructure 10.

At this time, according to the installation and use environment of each server constituting the IT infrastructure 10, the failure element selection unit 132 compares the second preliminary failure data output from each of the two failure prediction models to determine the two failures. It would also be possible to configure the second preliminary failure data output as a preliminary failure element in each prediction model to be output as second failure data, which is source code that is likely to cause a failure to the IT infrastructure 10.

The record management unit 160 generates and databases recorded data divided into source code units labeled by the microagent 110, and the source code labeled by the microagent 110 is stored in the data collection unit 120. , the process processed by the failure prediction unit 130 is recorded and managed in the record data.

FIG. 6 is a conceptual diagram of the failure prevention unit 170 of FIG. 1. Referring to FIG. 6, the failure prevention unit 170 includes a failure information management module 171, a first judgment module 172, and a failure processing module. (176), including the result transmission module 177, when receiving the second failure data containing the failure element, the failure-causing element is processed before the IT infrastructure failure occurs according to the failure response model.

The failure information management module 171 is an IT infrastructure failure type that can be prevented before an IT infrastructure failure occurs based on past failure information, which is information on failures in the actual IT infrastructure 10 that occurred in the past (hereinafter referred to as the “first failure type”). .) is managed in a database.

The first judgment module 172 receives the second failure data output as a failure element from the failure prediction unit 130 and determines whether it belongs to the first failure type.

The failure processing module 176 inputs the second failure data determined as the first failure type by the first judgment module 172 into the failure response model, and according to the failure response model, before the IT infrastructure failure occurs. Processes failure-causing factors and outputs the processing results.

The failure response model may be created or updated in the machine learning unit 150 using a reinforcement learning algorithm. The failure response model will be described in detail with reference to FIG. 8.

The result transmission module 177 transmits the processing result output by the error processing module 176 to the machine learning unit 150. The processing results transmitted to the machine learning unit 150 through the result transmission module 177 are used to update the cognitive model, analysis model, failure prediction model, and failure response model according to the reinforcement learning algorithm.

As shown in FIG. 6, the failure prevention unit 170 may further include a failure prevention type management module 173, a second judgment module 174, and an approval module 175.

The failure prevention type management module 173 divides the first failure type into a 1-1 failure type that requires administrator approval for failure handling and a 1-2 failure type that does not require administrator approval for failure handling. After classification, it is managed by creating a database.

The second judgment module 174 determines whether the second failure data belonging to the first failure type is the 1-1 failure type or the 1-2 failure type by the first determination module 172.

When the approval module 175 determines that the second failure data is the first failure type, it reports the first failure type to the manager terminal and requests approval for failure processing from the manager terminal. Receive approval.

If the failure prevention unit 170 is configured to further include the failure prevention type management module 173, the second judgment module 174, and the approval module 175, the failure processing module 176 is an administrator terminal. Enter the second failure data for the 1-1 failure type and the 1-2 failure type approved for failure handling from the above into the failure response model, and detect the failure before the IT infrastructure failure occurs by the failure response model. Process triggers and output the processing results.

FIG. 7 is a conceptual diagram of the visualization unit 140 of FIG. 1. Referring to FIG. 7, the visualization unit 140 includes a failure prediction visualization module that visualizes the cause of the failure and the failure prediction process by backtracking the contents recorded in the record data of the source code for the second failure data, and the second failure prediction visualization module. The first failure data and the second failure data are used using the failure prevention visualization module 142, which visualizes the process and results of processing the failure-causing factors by backtracking the contents recorded in the record data of the source code for the failure data. The basis for the output process and the results resulting from the process is extracted and visualized.

At this time, the data structure of the record data can be implemented using a decision tree and managed by the record management unit 160, and the failure prediction visualization module is not defined in the dictionary as a failure-causing factor. In the case of first failure data or second failure data that is not available, the basis for the process of outputting the first failure data and second failure data and the results of that process can be extracted according to model-agnostic, It is possible to visualize the process of outputting the first and second failure data extracted through hierarchical correlation propagation (LRP) and the basis for the results of that process.

In addition, the failure prevention visualization module 142 performs the failure prediction visualization in the process of visualizing the process and results of processing the failure-causing factors by backtracking the contents recorded in the record data of the source code for the second failure data. Model-agnostic (Model-Agnostic) to extract evidence for the process of outputting the second failure data and the results of that process by backtracking undefined second failure data that is determined to be a failure-causing element, such as a module. ), and the Layer-wise Relevance Propagation (LRP) method can be used to visualize the process of outputting the second failure data and the basis for the results of the process.

FIG. 8 is a conceptual diagram of the machine learning unit 150 of FIG. 1. Referring to FIG. 8, the machine learning unit 150 uses the cognitive model learning unit 151 and analysis to generate or update the cognitive model, analysis model, failure prediction model, and failure response model according to a reinforcement learning algorithm. It includes a model learning unit (152), a disability prediction model learning unit (153), and a disability response model learning unit (154).

The cognitive model learning unit 151 creates or updates the cognitive model according to a reinforcement learning algorithm. The cognitive model learning unit 151 creates a cognitive model according to a reinforcement learning algorithm using an anomaly detection algorithm. can be created or updated, and the cognitive model defines outliers as things that do not appear in general clusters, clusters, groups, etc. based on past raw data, and analyzes the raw data transmitted from the data filtering unit 112 to identify abnormalities. Extract the value as a failure factor.

The analysis model learning unit 152 generates or updates the analysis model according to a reinforcement learning algorithm. The analysis model learning unit 152 follows a reinforcement learning algorithm using an association rule learning algorithm. An analysis model can be created or updated, and the analysis model determines whether the labeled source code transmitted from the microagent 110 is likely to cause a failure when specific services or functions are called based on past failure history data. This outputs the first failure data. At this time, the Apriori algorithm can be used to extract frequent item groups while reducing the amount of calculation using an association rule learning algorithm.

The failure prediction model learning unit 153 generates or updates the failure prediction model according to a reinforcement learning algorithm. The failure prediction model learning unit 153 uses an anomaly detection algorithm and association rule learning. rule learning) algorithm and CRNN (Classified Recurrent Neural Network) algorithm, select at least two reinforcement learning algorithms and generate or update at least two failure prediction models according to each reinforcement learning algorithm.

The failure response model learning unit 154 generates or updates the failure response model according to a reinforcement learning algorithm. The failure response model learning unit 154 can generate or update an analysis model according to a reinforcement learning algorithm. The above failure response model removes failure-causing factors before an IT infrastructure failure occurs depending on the type of specific service or function, based on past failure handling data and administrator approval.

In order to confirm the performance of the IT infrastructure failure pre-processing system 100 using the machine learning algorithm of the present invention, tests were performed as in Examples 1 to 8. At this time, the hardware specifications of the client used as IT infrastructure 10 and the server on which the system of the present invention is installed are shown in Table 1.

server operating system CentOS7 hardware CPU Intel zeon E5 10Core 20Thread 2.2Ghz RAM 32GB HDD 1TB Client operating system Windows10 hardware CPU Intel core i7 3.2Ghz RAM 16GB HDD 500GB

1. Using the HTTP load generator, 1,000 users start viewing posts on the web page, and the number of concurrent users is increased by 1,000 every 5 minutes until the number of concurrent users reaches 30,000.

2. Next, when the number of concurrent users reaches 30,000, the number of users viewing posts with errors is reduced by 1,000 every 5 minutes for 1 hour.

3. Next, increase the number of users viewing posts with errors by 1,000 every 5 minutes until the number of concurrent users reaches 40,000.

4. Next, when the number of concurrent users reaches 40,000, the number of users viewing posts with errors is reduced by 1,000 every 5 minutes for 1 hour.

5. Next, increase the number of users viewing posts with errors by 1,000 every 5 minutes until the number of concurrent users reaches 50,000.

6. Next, when the number of concurrent users reaches 50,000, load generation is stopped, and while procedures 2 to 5 are in progress, the log of the IT infrastructure failure pre-processing system by the machine learning algorithm of the present invention is analyzed to determine the second Calculate the response time until the failure data is output.

7. After repeating steps 1 to 6 10 times, calculate the average of the response times.

1. Prepare 6000 HTTP request/response information datasets and 600 system resource (CPU, memory, disk) test datasets to trigger IT infrastructure failure situations.

2. Transmit 6000 HTTP request/response information datasets for 1 hour and 600 system resource (CPU, memory, disk) test datasets for 10 minutes to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention. do.

3. The IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention measures the time and detection rate required to predict IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the response time and detection rate.

1. Prepare 80 DB Lock case test data sets to trigger IT infrastructure failure situations.

2. For 1 hour, 80 DB Lock case test data sets are transmitted to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention.

3. The IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention measures the time and detection rate required to predict IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the response time and detection rate.

1. Prepare 100 Connection Leak case test datasets to trigger IT infrastructure failure situations.

2. For 1 hour, 100 Connection Leak case test datasets are transmitted to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention.

3. The IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention measures the time and detection rate required to predict IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the response time and detection rate.

1. Prepare 100 test datasets of increased CPU usage to trigger IT infrastructure failure situations.

2. 100 CPU usage increase test datasets for 1 hour are transmitted to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention.

3. The IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention measures the time and detection rate required to predict IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the response time and detection rate.

1. Prepare 100 memory usage increase test datasets to trigger IT infrastructure failure situations.

2. 100 memory usage increase test datasets for 1 hour are transmitted to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention.

3. The IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention measures the time and detection rate required to predict IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the response time and detection rate.

1. Prepare 100 test datasets of increased disk usage to trigger IT infrastructure failure situations.

2. 100 disk usage increase test datasets for 1 hour are transmitted to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention.

3. The IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention measures the time and detection rate required to predict IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the response time and detection rate.

In addition, to test whether IT infrastructure failures caused by non-normal raw data can be predicted in advance, the test was conducted in the following order.

1. Prepare 100 irregular user case test datasets to trigger IT infrastructure failure situations.

2. For 1 hour, 100 test datasets of irregular user cases are transmitted to the IT infrastructure failure pre-processing system using the machine learning algorithm of the present invention.

3. The IT infrastructure failure pre-processing system based on the machine learning algorithm of the present invention measures the detection rate for predicting IT infrastructure failure.

4. After repeating procedures 1 to 3 10 times, calculate the average of the detection rate.

Next, in order to check the performance of the IT infrastructure failure pre-processing system 100 using the machine learning algorithm of the present invention, “maxigent”, an IT infrastructure failure pre-detection solution from Samsung SDS (hereinafter referred to as “conventional solution”). In the same manner as Examples 1 to 8, the time required to recognize IT infrastructure failures and the detection rate were measured.

The response times and detection rates of measured in Examples 1 to 8 using the IT infrastructure failure pre-processing system 100 using the machine learning algorithm of the present invention and conventional solutions are shown in Table 2 below.

System of the present invention conventional solution response speed Detection rate Response speed Detection rate Example 1 2.07 99.15 5.96 95.65 Example 2 2.11 99.55 4.63 93.63 Example 3 2.49 99.09 5.93 91.23 Example 4 2.77 99.93 5.66 95.21 Example 5 2.44 99.27 3.19 91.89 Example 6 2.22 99.72 3.36 95.36 Example 7 2.01 99.54 4.41 94.86 Real example 8 2.86 99.20 3.45 93.74

As shown in Table 2, the IT infrastructure failure pre-processing system 100 based on the machine learning algorithm of the present invention recognizes various types of failure-causing factors that cause IT infrastructure failure within 3 seconds based on the source code. was able to do so, and recorded a detection rate of over 99% of failure-causing factors. However, the conventional solution took between 3 and 6 seconds to recognize IT infrastructure failures in advance, and the detection rate was also recorded between 91% and 96%. was able to confirm.

Hereinafter, a method for predicting and processing IT infrastructure failures by a machine learning algorithm in advance according to an embodiment of the present invention will be described in detail with reference to FIGS. 9 to 13.

According to the method of predicting and processing IT infrastructure failures in advance using the machine learning algorithm, not only can failure-causing factors for the IT infrastructure 10 be automatically recognized and processed in advance based on the source code, but also the machine Based on a learning algorithm, it is possible to provide grounds for predicting IT infrastructure failures and processing results. Figure 9 is a flowchart of a method for predicting and processing IT infrastructure failures by a machine learning algorithm in advance according to an embodiment of the invention. .

FIG. 10 is a flow chart illustrating the first step (S100) of FIG. 9 in detail, FIG. 11 is a flow chart illustrating the first and second steps (S120) of FIG. 10 in detail, and FIG. 12 is a flow chart illustrating the third step (S120) of FIG. 9. This is a flowchart showing the step (S300) in detail, and FIG. 13 is a flowchart showing the fourth step (S400) of FIG. 9 in detail.

Referring to FIGS. 9 to 13, the method of predicting and processing IT infrastructure failures using a machine learning algorithm is largely performed including four steps from the first step (S100) to the fourth step (S400).

In the first step (S100), the microagent 110 extracts raw data for each IT infrastructure 10 and source code development system 20, recognizes failure-causing factors using a cognitive model, and The corresponding source code is extracted and labeled in units of extracted source code.

At this time, the first step (S100) is performed including six steps from the 1-1 step (S110) to the 1-6 step (S160). The 1-1 step (S110) is a data collection module ( 111) extracts and collects the raw data from the IT infrastructure 10 and the source code development system 20, and in the first and second steps (S120), the data filtering unit 112 extracts and collects the raw data included in the raw data. Identify and process information.

The 1-2 step (S120) is again performed including three steps from the 1-21st step to the 1-23rd step. In the 1-21st step, the sensitive information identification module 112a The sensitive information included is identified, and in steps 1 to 22, the sensitive information determination module 112b recognizes the failure-causing element in the recognition module 113 when the sensitive information identified in steps 1 to 21 is identified. It is determined whether it is necessary essential sensitive information, and in steps 1-23, the sensitive information processing module 112c automatically replaces the sensitive information determined as essential sensitive information with general information, and the sensitive information that is not essential sensitive information is raw data. Delete from

Next, in step 1-3 (S130), the recognition module 113 receives the raw data from the data filtering unit, inputs it into the recognition model, and recognizes the failure-causing element by the recognition model.

Next, in steps 1-4 (S140), the source code collection module 114 collects source codes related to the failure-causing elements from the source code development system 20, and in steps 1-5 (S150) After the labeling module 115 automatically labels the source code according to the labeling algorithm, the data transmission module 116 is labeled in steps 1-5 (S150) in steps 1-6 (S160). The source code is transmitted to the data collection unit 120.

Next, in the second step (S200), the data collection unit 120 receives the labeled source code from the microagent 110 and outputs first failure data, which is source code that is likely to cause a failure, through an analysis model. do.

Next, in the third step (S300), the failure prediction unit 130 inputs the first failure data into the failure prediction model and outputs second failure data that divides the first failure data into non-failure elements and failure elements. Predict failures in IT infrastructure (10).

At this time, the third step (S300) is again performed including the 3-1 step (S310) and the 3-2 step (S320). The 3-1 step (S310) is the preliminary obstacle screening unit 131. ) receives the first failure data from the data collection unit 120 and inputs it into each of at least two failure prediction models, and each of the failure prediction models divides the first failure data into a spare failure element and a spare failure element. The second preliminary failure data is output, and in step 3-2 (S320), the failure element selection unit 132 compares the second preliminary failure data output by each of the two failure prediction models to predict the at least two failures. The second preliminary failure data output as a preliminary failure element by at least one failure prediction model among the models is output as second failure data, which is source code that is likely to cause a failure to the IT infrastructure 10.

Next, in the fourth step (S400), when the failure prevention unit 170 receives the second failure data containing the failure elements, it processes the failure-causing elements before an IT infrastructure failure occurs according to a failure response model.

At this time, the fourth step (S400) is again performed including three steps from the 4-1 step (S410) to the 4-3 step (S430), and the 4-1 step (S410) is the first judgment. After the module 172 receives the second failure data and determines whether it belongs to the first failure type, in step 4-2 (S420) the failure processing module 176 receives the first failure data from the first judgment module 172. The second failure data determined to be a failure type is input into the failure response model, the failure-causing factors are processed according to the failure response model, and the processing results are output. Finally, in step 4-3 (S430) The result transmission module 177 transmits the processing result output by the error processing module 176 in step 4-2 (S420) to the machine learning unit 150.

In addition, as shown in Figure 13, the fourth step (S400) is a step 4-1a (S410a) and a 4-1b step between the 4-1 step (S410) and the 4-2 step (S420). It may be performed further including (S410b), wherein the first judgment module 172 is performed by the second judgment module 174 after the fourth-1a step (S410) is performed. ) to determine whether the second failure data belonging to the first failure type is the 1-1 failure type or the 1-2 failure type, and the 4-1b step (S410b) is the 4-1a step ( After performing S410a), if the approval module 175 determines that the second failure data is the 1-1 failure type in step 4-1 (S410), it reports the 1-1 failure type to the administrator terminal. Then, approval for fault handling is requested and approval is received from the manager terminal.

On the other hand, when step 4-1a (S410a) and step 4-1b (S410b) are further included between step 4-1 (S410) and step 4-2 (S420), step 4-2 In step S420, the failure processing module 176 stores the second failure data for the 1-1 failure type and the 1-2 failure type approved for failure handling by the administrator terminal into the failure response model. Input, the failure-causing factors are processed using the failure response model, and the processing results are output.

According to the IT infrastructure failure pre-processing system 100 using the machine learning algorithm of the present invention and the method for predicting and processing IT infrastructure failure in advance using the machine learning algorithm, the cognitive model, analysis model, and failure are designed in a hierarchical structure. By analyzing raw data through a distributed artificial intelligence model using a predictive model to recognize and process failure-causing factors, not only is it possible to quickly recognize and process failure-causing factors from raw data, but it is also possible to predict and prevent IT infrastructure failures. Time is shortened, and the accuracy of IT infrastructure failure prediction is improved.

In addition, based on the source code, it is possible to automatically prevent factors that cause failures in IT infrastructure (10) in advance, and provide a basis for predicting factors that cause failures in IT infrastructure (10) based on machine learning algorithms. ), the leakage of sensitive information can be prevented by deleting the sensitive information included in the raw data or automatically replacing it with general information.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

100: IT infrastructure failure pre-processing system using machine learning algorithm
110: Microagent
120: Data collection department
130: Disability prediction department
140: Visualization unit
150: Machine Learning Department
160: Records Management Department
170: Disability Prevention Department

Claims (7)

A microagent that extracts raw data for each IT infrastructure and source code development system, recognizes failure-causing elements using a cognitive model, extracts source code corresponding to the raw data and failure-causing elements, and labels the extracted source code in units;
a data collection unit that receives labeled source code from the microagent and outputs first failure data, which is source code likely to cause a failure, through an analysis model;
a failure prediction unit that inputs the first failure data into a failure prediction model and outputs second failure data that divides the first failure data into non-failure elements and failure elements to predict failures in the IT infrastructure;
A failure prevention unit that, upon receiving the second failure data comprised of the failure elements, processes the failure-causing elements before an IT infrastructure failure occurs according to a failure response model;
A visualization unit that visualizes and provides the basis for the process of outputting the first failure data and the second failure data and the results of the process, and the results of processing failure-causing factors in the failure prevention unit; and,
It includes a machine learning unit that generates or updates the cognitive model, analysis model, failure prediction model, and failure response model according to a reinforcement learning algorithm,
The Department of Disability Prevention,
A failure information management module that creates a database and manages IT infrastructure failure types that can be prevented before IT infrastructure failures occur (hereinafter referred to as “first failure type”) based on past failure information, which is information on failures in actual IT infrastructure that occurred in the past. ;
A first judgment module that receives the second failure data and determines whether it belongs to the first failure type;
a failure processing module that inputs the second failure data determined as the first failure type by the first judgment module into the failure response model, processes the failure-causing factors according to the failure response model, and outputs the processing result; and,
An IT infrastructure failure pre-processing system using a machine learning algorithm, including a result transmission module that transmits the processing results output by the failure processing module to the machine learning unit. According to paragraph 1,
The microagent is,
a data collection module that extracts and collects the raw data from the IT infrastructure and source code development system;
a data filtering unit that identifies and processes sensitive information included in the raw data;
a recognition module that receives the raw data from the data filtering unit, inputs it into the recognition model, and recognizes the failure-causing factors by the recognition model;
a source code collection module that collects source code related to the failure-causing element from the source code development system;
A labeling module that automatically labels the source code collected in the source code collection module according to a labeling algorithm; and,
A data transmission module that transmits the source code labeled in the labeling module to the data collection unit. An IT infrastructure failure pre-processing system using a machine learning algorithm including a. According to paragraph 2,
The data filtering unit,
a sensitive information identification module that identifies sensitive information included in the raw data;
a sensitive information determination module that determines whether the sensitive information identified in the sensitive information identification module is essential sensitive information necessary for the recognition module to recognize the failure-causing element; and,
A sensitive information processing module that automatically replaces the sensitive information determined as essential sensitive information with general information and deletes the sensitive information that is not essential sensitive information from raw data; an IT infrastructure failure pre-processing system using a machine learning algorithm that includes a . According to paragraph 1,
The Department of Disability Prevention,
Disability prevention that divides the above 1st failure type into 1-1 failure type, which requires administrator's approval for failure handling, and 1-2 failure type, which does not require administrator's approval for failure handling, and then manages them in a database. Type management module;
a second judgment module that determines whether the second fault data belonging to the first fault type is a 1-1 fault type or a 1-2 fault type by the first judgment module;
When the second failure data is determined to be the first failure type, an approval module reports the first failure type to the manager terminal, requests approval for failure processing, and receives approval from the manager terminal. further includes ;,
The failure processing module inputs the second failure data for the 1-1 failure type and the 1-2 failure type approved for failure handling from the administrator terminal into the failure response model, and An IT infrastructure failure pre-processing system using a machine learning algorithm that processes the above failure-causing factors and outputs the processing results. According to paragraph 1,
The machine learning department,
A cognitive model learning unit that generates or updates the cognitive model according to a reinforcement learning algorithm;
An analysis model learning unit that generates or updates the analysis model according to a reinforcement learning algorithm;
A failure prediction model learning unit that generates or updates the failure prediction model according to a reinforcement learning algorithm; and,
An IT infrastructure failure pre-processing system using a machine learning algorithm, including a failure response model learning unit that generates or updates the failure response model according to a reinforcement learning algorithm. According to paragraph 1,
Record data divided into labeled source code units is generated in the microagent and converted into a database, and the process in which source code labeled in the microagent is processed by the data collection unit, failure prediction unit, and failure prevention unit is recorded. It further includes a record management unit that records and manages data,
The visualization unit,
a failure prediction visualization module that visualizes the cause of the failure and the failure prediction process by backtracking the contents recorded in the record data of the source code for the second failure data; and,
A failure prevention visualization module that visualizes the process and results of processing the failure-causing factors by backtracking the contents recorded in the record data of the source code for the second failure data; IT infrastructure failure dictionary by a machine learning algorithm including a processing system. According to paragraph 1,
The failure prediction unit,
The first failure data is transmitted from the data collection unit and input into each of at least two failure prediction models, and each of the failure prediction models is second preliminary failure data that divides the first failure data into a spare failure element and a spare failure element. A preliminary obstacle selection unit that outputs; and,
By comparing the second preliminary failure data output from each of the two failure prediction models, the second preliminary failure data output as a preliminary failure element by at least one of the at least two failure prediction models is stored in the IT infrastructure. An IT infrastructure failure advance prediction system using a machine learning algorithm that includes a failure element selection unit that outputs second failure data, which is source code that is likely to cause a failure.