KR20190138037A - An information retrieval system using knowledge base of cyber security and the method thereof - Google Patents

Abstract

The present invention relates to an information search system using cyber a security specialized-knowledge base, and to a method thereof. The information search system comprises: a collection unit collecting web documents from cyber security-related websites shared on the surface web and dark web; a preprocessing unit extracting triples which are knowledge information related to cyber security from the collected web documents; a knowledge base generation unit generating a knowledge base using the set of triples extracted by the preprocessor; a knowledge base inference unit inferring the relationship and knowledge of the triples, based on the ontology stored in the knowledge base and representing the relationship between the triples; and a cyber security knowledge base constructing unit merging the knowledge bases inferred by a knowledge base constructing unit to construct a final cyber security knowledge base.

Description

Information retrieval system and its method using cyber security specialized knowledge base {AN INFORMATION RETRIEVAL SYSTEM USING KNOWLEDGE BASE OF CYBER SECURITY AND THE METHOD THEREOF}

The present invention relates to an information retrieval system and method using a cyber security-specific knowledge base, and more particularly, to an information retrieval system for implementing a cyber security search engine function, visualization function, trend grasping function, and query / response function. It is about a method.

Web search engines, a representative form of information retrieval, are based on keyword-based search techniques, which collect documents on the web, index words that appear frequently in the documents, and include many words that match the keywords entered by the user. Find the location of the document you are doing.

Korean Patent Laid-Open Publication No. 10-2014-0047952, which is an existing prior art, relates to a technique for generating multiple unit knowledge bases including only some basic knowledge, inferring them independently, and then merging them to build a final knowledge base.

However, the existing prior art discloses a method of building a general knowledge base, and there is a limit to the method of building, extending, and maintaining a knowledge base related to a special field of cyber security.

Korean Patent Publication No. 10-2014-0047952 (published Apr. 23, 2014), "Knowledge base construction device and its knowledge base construction method"

The present invention aims to propose an information retrieval system specialized for cyber security by utilizing a cyber security knowledge base.

An object of the present invention is to provide more accurate and relevant information about a user's query related to cyber security than a general information retrieval system.

An information retrieval system using a cyber security-specific knowledge base according to an embodiment of the present invention includes a collection unit that collects web documents from a web site related to cyber security shared on a surface web and a dark web, A preprocessor extracting triples which are knowledge information related to cyber security from the collected web documents, a knowledge base generator generating a knowledge base using the set of triples extracted by the preprocessor, and storing the knowledge base Based on the ontology representing the relationship between the triples, merging the knowledge bases inferred by the knowledge base inference unit and the knowledge base inference unit that infers the relationship and knowledge of the triples to build a final cyber security knowledge base. It includes a cyber security knowledge base construction unit.

The information retrieval system is characterized by providing information specific to cyber security for a user's query using the cyber security knowledge base.

In a method of operating an information retrieval system using a cyber security-specific knowledge base according to an embodiment of the present invention, web documents from a web site related to cyber security shared on a surface web and a dark web Collecting, extracting triples which are knowledge information related to cyber security from the collected web documents, generating a knowledge base using the extracted set of triples, stored in the knowledge base, Based on the ontology representing the relationship, inferring the association and knowledge of the triples and merging the inferred knowledge bases to build a final cyber security knowledge base.

According to an embodiment of the present invention, the cyber security knowledge base may be used to provide more accurate and highly relevant information about a user's query related to cyber security than a general information retrieval system.

1 is a block diagram illustrating a detailed configuration of a cyber security specialized information retrieval system according to an embodiment of the present invention.
2 is a flowchart illustrating a cyber security specialized information retrieval method according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Also, like reference numerals in the drawings denote like elements.

In addition, terms used in the present specification (terminology) are terms used to properly express preferred embodiments of the present invention, which may vary depending on the intention of the viewer, the operator, or customs in the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification.

The present invention provides an information retrieval system specialized for cyber security by utilizing a cyber security knowledge base.

The cyber security-specific information retrieval system proposed in the present invention can provide a total of four functions, and each function is provided through a web interface to be used through a web browser. The web interface proposed by the present invention can use each of the following functions in separate sections, and each section can be divided into individual tabs.

The first is the cybersecurity search engine.

The cyber security search engine function searches web documents that are most suitable for the user's query, not only on the surface web but also on the web documents accessible on the dark web. In particular, the cyber security search engine function can search for information that is closely related to cyber security unlike a general search engine. For example, if an attack or a query for malware is entered, the cyber security search engine function may: i) be attacked by the attack / malware or cause symptoms on an infected information system, ii) attack / A solution that can be used to remove an attack or infection caused by malware, iii) how the attack / malware spreads over the Internet, and iv) to search for detailed information about the victim / range, and to retrieve relevant web documents. You can search.

For this reason, the information retrieval system and its method using the cyber security-specific knowledge base according to an embodiment of the present invention establishes a separate section for checking the result retrieved by the cyber security search engine function on a detailed subject. In addition, the user can search for a desired detailed subject and related information by selecting a tab for each compartment.

The second function is a visualization function that can comprehensively understand the characteristics of a cyber attack or malware when a user is given a query.

The visualization function expresses entities related to each attack or malware given by user input as a node, and connects the relationships between the concepts by edges. In addition, it is possible to express the objects related to the query given by the user's input and related relations in the form of pictures. At this time, if a relationship exists between any two of the plurality of entities, this is called a fact or fact information, and the relationship between the two entities is triple in the form of <subject, predicate, object>. Can be expressed as

Furthermore, the source of the fact information expressed in triples can be displayed at the same time, in which case the above-described search technique can be used.

In other words, the visualization function provides an easy to see relationship between the information related to the user's query, and at the same time provides the source of the information, thereby helping the user to understand the query. For example, suppose that the sentence 'Conficker' is a type of malware 'is described in Web Document A. It connects two points, meaning Conficker and Malware, and the edge connects the' is-A 'relationship. To mean. At this point, a triple called <Conficker, is-A, Malware> is created, and the information represented by this triple appears in Web document A. You can check the source of the information.

Therefore, when a user inputs an arbitrary query word, the user can easily grasp the associations of a plurality of related entities, and the information can be trusted by checking the source thereof.

The third function is to receive the user's interests related to cyber security, collect information related to the received interests on the Internet and the dark web, and identify and summarize the latest trends of the user's interests.

For example, suppose you own a personal desktop computer with a variety of components, and you install and use a specific operating system as well as many applications on that computer. Information about hardware and software that are being used can be registered as an interest in an information retrieval system using a cyber security-specific knowledge base proposed by the present invention.

Then, the information retrieval system and the method using the cyber security-specific knowledge base according to an embodiment of the present invention can collect the important events in terms of cyber security based on the registered interest. For example, if a vulnerability has been found for an operating system installed on a user's computer, and there is a cyber security news article or an official announcement by an operating system manufacturer indicating that a patch has been developed and released, an embodiment of the present invention. The information retrieval system and its method using the cyber security-specific knowledge base according to the present invention can collect and aggregate the web documents including the information through the function of identifying and summarizing the latest trends and notify the user.

In addition, when a user defines 'information about a denial of service attack' as an interest and registers it in the information retrieval system, the information retrieval system and the method using the cyber security-specific knowledge base according to the embodiment of the present invention are The attack time, attack target, expected damage size, and subject of the attack can be provided to the user in order from attack to past attack cases. At this time, the information retrieval system and the method using a cyber security-specific knowledge base according to an embodiment of the present invention by providing a link to a web document that is a source to the user through the above-described visualization function, It can improve the reliability.

The fourth function is a query / response function specialized for cyber security.

Unlike the information search function which finds the web documents related to the user's query word and lists them in the order of relevance, the query / response function extracts the key information by analyzing the related documents one more time and provides the 'best answer' for the user's query word. 'Can be provided. A query of an information retrieval system is generally composed of a plurality of words, whereas a query of a query / response function has a sentence-like sentence form. Accordingly, the query / response function may understand a query expressed as a question, generate a correct answer that is closest to the query, and provide the correct answer to the user.

An information retrieval system using a cyber security-specific knowledge base according to an embodiment of the present invention and the cyber security-specific query / response function proposed by the method may provide an answer to a question that a user wants to obtain an answer in the cyber security domain. Can be.

Hereinafter, referring to FIGS. 1 and 2, an information retrieval system using a knowledge base of cyber security specialization and a method thereof will be described in detail.

1 is a block diagram showing a detailed configuration of a cyber security-specific information retrieval system according to an embodiment of the present invention, Figure 2 is a flowchart of a cyber security-specific information retrieval method according to an embodiment of the present invention .

1 and 2, the cyber security specialized information retrieval system according to an embodiment of the present invention implements a cyber security search engine function, a visualization function, a trend grasping function, and a question / answer function.

To this end, the cyber security-specific information retrieval system 100 according to an embodiment of the present invention is a collection unit 110, pre-processing unit 120, knowledge base generation unit 130, knowledge base inference unit 140 and cyber Security knowledge base construction unit 150 is included. In addition, each of the steps (steps 210 to 250) of FIG. 2 includes components of the cyber security-specific information retrieval system 100 according to the embodiment of the present invention, that is, the collecting unit 110 and the preprocessor. 120, the knowledge base generating unit 130, the knowledge base inference unit 140, and the cyber security knowledge base building unit 150 may be performed.

In operation 210, the collector 110 collects web documents from a web site related to cyber security shared on a surface web and a dark web.

The collection unit 110 may collect web documents related to cyber security from generally accessible web sites called the Internet or the surface web. In addition, the collector 110 may collect cyber security-related web documents that are shared on a dark web that can be accessed only by using a method different from the general surface web access method.

Agents planning cyber attacks exchange information related to cyber attacks on inaccessible dark web. Thus, the collection unit 110 may collect more in depth information related to cyber security by collecting information shared on the dark web.

In order to collect and collect web documents related to cyber security in common on the surface web and the dark web, the information retrieval system 100 according to an embodiment of the present invention is a vocabulary set related to cyber security selected by cyber security experts. build and apply vocabulary The vocabulary set includes hardware and software security vulnerabilities discovered and reported, patch information for vulnerabilities, names of published cyber attacks, names of cyber attack tools, names and versions of software, names of countermeasures against cyber attacks, software And a name of a hardware manufacturing company, names of organizations corresponding to cyber security threats, and names of companies.

When the collection unit 110 visits the web sites of the surface web and the dark web and finds web documents indicated by the vocabularies included in the vocabulary set, the information retrieval system 100 according to the embodiment of the present invention may include a storage unit. The copy may be stored at 160. This is called web crawling.

The information retrieval system 100 according to the embodiment of the present invention may utilize a configuration list defined as a representative web site containing specialized information on cyber security. The configuration list may be stored in the storage 160.

The collector 110 may visit the web sites included in the configuration list and collect posts (web documents) posted on the web site. At this time, the collection unit 110 is included in the vocabulary set related to cyber security among the vocabulary included in the post, if the vocabulary having a link to the address of the external web site is found, through the storage unit 160 You can add the address of the linked external website to your list of future websites.

However, unlike the surface web, special techniques are required to visit the web site of the dark web. Dark web needs to be accessed using Tor protocol to ensure anonymity, so web crawling technology that supports Tor protocol is required to collect web documents on dark web.

The information retrieval system 100 according to the embodiment of the present invention builds a web crawling technology that supports the Tor protocol, and the collection unit 110 utilizes a web crawling technology to publish cyber security-related web documents on the dark web. Can be collected.

However, since there is no known web site for handling web documents related to cyber security on the dark web, the information retrieval system 100 according to the embodiment of the present invention uses a cyber engine on the dark web through a search engine for the dark web. A web site related to security may be searched, a cyber security related web site list may be generated on the dark web based on the search result, and the web document may be collected through the generation unit 110 as described above.

In operation 220, the preprocessor 120 extracts triples, which are knowledge information related to cyber security, from the collected web documents.

Web documents collected by the collection unit 110 should be represented in a form that can be utilized by the information retrieval system 100 according to an embodiment of the present invention. Since the web document is written in a text language, the information retrieval system 100 according to an exemplary embodiment of the present invention uses a natural language processing (NLP) based preprocessor 120 to extract necessary information from the web document.

The preprocessing unit 120 may structurally extract necessary text information from the collected web document and remove unnecessary information included in the web document. In general, Web documents include Cascading Style Sheets (CSS) code to help you present your Web document in a nicer form, various script code to add dynamic functionality, and Hyper Text Markup Language (HTML) code. Since they do not greatly help the information retrieval process, the preprocessor 120 may remove them.

At this time, since some HTML code acts as metadata for text information, the preprocessor 120 may extract structural text information (or knowledge information, triples) using the HTML code. For example, some text may have link information added to other web documents, and some text may have information added to indicate a title. Since most of them are written in HTML code and added to the text information, the preprocessing unit 120 grasps the structure and distinguishes the text written as the title from the text pointing to another external web document, and the same text information can be used more importantly. Selecting text can extract structural text information.

Subsequently, the preprocessor 120 may classify the extracted text into respective sentences.

The basic unit for understanding the meaning of text data is a sentence. By dividing the text into sentences, it is possible to identify which event or event each sentence intends to represent, and to recognize the subject and object of the event or event. In particular, for events and events related to cyber security, it is important to clearly identify the subject and target of the attack. Therefore, the subject (ie, subject) and the target (ie, object) in the sentence are important. ) Is important. Accordingly, grasping the sentence should be preceded, followed by the identification of the subject and the object, and the verb indicating the relationship between the subject and the object.

To this end, the preprocessor 120 may extract a sentence and a vocabulary related to cyber security from the collected web document. In this case, the triple is mainly used as a subject and object in the sentence extracted above, and this may be referred to as a cyber-security named entity.

According to an embodiment, the information retrieval system 100 according to an embodiment of the present invention may use a vocabulary included in a vocabulary set to recognize a vocabulary related to cyber security. You can build an Entity Recognizer.

Specifically, the information retrieval system 100 according to an embodiment of the present invention applies a bidirectional Long Short Term Memory (LSTM) based cyclic neural network algorithm that represents an effective language modeling result at a word level. It is possible to determine whether or not the vocabulary included in the entity names related to cyber security. According to an embodiment, the entity name related to cyber security may include at least one or more of a vulnerability, an attack, a software, and a hardware name.

In cyber security, in particular, it is important to accurately recognize the version name of the hardware and software where the vulnerability is found, but it is difficult to identify it accurately with the entity name recognizer. Accordingly, the information retrieval system 100 according to an embodiment of the present invention constructs a hardware and software version name recognizer used as a regular expression by cyber security experts, and includes the same in the cyber security entity name recognizer. Can be.

In order to learn the cyber security entity name recognizer by a machine learning method of deep learning technology, cyber security-related vocabulary tagging (tagging or annotation) is required. In order to facilitate tagging, the information retrieval system 100 according to an embodiment of the present invention may build a cyber security-related vocabulary tagging assistance system in web documents based on a web browser interface.

Tagging related vocabularies for object name recognition requires a lot of time and effort, so it is important to make this task easy and simple. Therefore, the information retrieval system 100 according to an embodiment of the present invention helps cyber security experts to easily tag related vocabularies for a given document, and the cyber security entity name recognizer can recognize and learn the related vocabulary. The tagging results can be stored in data form.

In addition, the preprocessing unit 120 may extract a triple composed of each sentence <subject, verb, object> based on the extracted sentence and the cyber security related entity name. The triples represent an event or event with minimal information, allowing the machine to easily grasp the meaning of the event or event.

According to an embodiment, a nested triple may occur when the object of the triple is a triple other than a single vocabulary, that is, when a sentence representing an event or event is a complex sentence. For example, in the words "Microsoft has acknowledged that its operating system has vulnerabilities", when the Triple <Microsoft Operating System, Has, Vulnerability> is called Triple 1, Microsoft acknowledges, Triple such as Triple 1> can be generated. The triple generated through this process can be used as an index to the collected web documents.

That is, for a web document, the set of triples extracted from the text information of the web document can be regarded as a kind of summary of the entire information contained in the web document, and the index is according to an embodiment of the present invention. The information retrieval system 100 can be of great help in finding web documents most relevant to the user's query.

In operation 230, the knowledge base generator 130 generates a knowledge base using a set of triples extracted by the preprocessor 120.

The knowledge base refers to high-level information represented by the relationship between objects, and may represent 'knowledge' possessed by experts in a specific field, unlike simple data or information. In the knowledge base, 'knowledge' is usually expressed in triples. Thus, the knowledge base generation unit 130 of the information retrieval system 100 according to an embodiment of the present invention may build a knowledge base using the set of triples generated by the preprocessor 120 described above.

The knowledge base of the information retrieval system 100 according to an embodiment of the present invention is a key role that assists the four main functions (for example, cyber security search engine function, visualization function, trend grasping function, and query / response function). Can be performed.

More specifically, since the triples included in the knowledge base are extracted from each web document and used as an index for each web document, the triples included in the knowledge base can be used to search related web documents for user queries. It can also be used for query expansion for user queries, so that it can be used to retrieve more relevant web documents using queries that are related to or similar to the user's query, but are not entered directly by the user.

In addition, the Knowledge Base provides a graphical representation of the graphs in triple and edge directions for i) characterization of attacks / malware and ii) summary of trend information related to cybersecurity user concerns, and iii) cybersecurity-related query / response capabilities. It can be visualized in the form, provides a summary of triples over time, and analyzes the subjects, verbs, and objects that appear in question-type user queries, and extracts and presents the most relevant and high reliability triples.

In operation 240, the knowledge base inference unit 140 infers the association and knowledge of the triples based on the ontology representing the relationship between the triples stored in the knowledge base.

The information retrieval system 100 according to an embodiment of the present invention proposes an ontology and cyber security knowledge base management system specialized in the cyber security field for systematic management of a cyber security related knowledge base.

In general, knowledge bases include ontologies for which knowledge representation systems are defined. Ontology defines the relationship between triples stored in the knowledge base. Accordingly, based on the relationship between the triples defined in the ontology, it is possible to grasp the relationship of the triples.

In particular, ontology defines the relationship between class and instance as well as the constraints that exist between objects.These relationships apply to triples stored in the knowledge base. This allows us to infer new relationships and knowledge that we have not discovered before.

There are two important things to consider regarding ontologies: one is to continuously expand the ontology, and the other is to work with the proposed general purpose ontology. Cyber security is one of the fastest changing areas. Every day new concepts emerge, new means of attack and defense are developed.

Thus, the ontology already built may not contain new concepts. To do this, the ontology must be continuously expanded so that new concepts related to cyber security can be included in the existing ontology.

The knowledge base inference unit 140 of the information retrieval system 100 according to an embodiment of the present invention collects instances that do not correspond to the existing ontology, creates a new ontology class, and extends the ontology by a method of updating the existing ontology. can do.

In addition, the ontology of cyber security proposed by the information retrieval system 100 according to an embodiment of the present invention has an ontology such as YAGO and Freebase that have been developed since the field of cyber security is closely related to real life. In order to interoperate with, it can be constructed to be compatible with general ontology, including vocabulary used in the existing ontology.

In operation 250, the cyber security knowledge base construction unit 150 merges the knowledge bases inferred by the knowledge base construction unit 140 to construct a final cyber security knowledge base.

For example, the cyber security knowledge base construction unit 150 may be based on a URI in which all knowledge objects including a class, an instance, and a property have a uniform resource identifier (URI). The knowledge base can be mechanically merged to build the final cyber security knowledge base.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

Method according to the embodiment is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (3)

In the information retrieval system using the cyber security-specific knowledge base,
A collection unit that collects web documents from a web site related to cyber security shared on a surface web and a dark web;
A preprocessor extracting triples which are knowledge information related to cyber security from the collected web documents;
A knowledge base generation unit generating a knowledge base using the set of triples extracted by the preprocessor;
A knowledge base inference unit for inferring associations and knowledge of the triples based on an ontology stored in the knowledge base representing the relationships between the triples; And
Cyber security knowledge base construction unit for constructing the final cyber security knowledge base by merging the knowledge bases inferred by the knowledge base construction unit
Cyber security specialized information retrieval system comprising a. The method of claim 1,
The information retrieval system
The cyber security specialized information retrieval system, characterized in that for providing information specific to cyber security for the user's query using the cyber security knowledge base. In the operation method of the information retrieval system using a cyber security specialized knowledge base,
Collecting web documents from a web site related to cyber security shared on a surface web and a dark web;
Extracting triples which are knowledge information related to cyber security from the collected web documents;
Generating a knowledge base using the extracted set of triples;
Inferring associations and knowledge of the triples based on an ontology stored in the knowledge base representing the relationships between the triples; And
Merging the inferred knowledge bases to build a final cyber security knowledge base
Cyber security-specific information retrieval method comprising a.

Images