CN117786088A - Threat language model analysis method, threat language model analysis device, threat language model analysis medium and electronic equipment - Google Patents
Threat language model analysis method, threat language model analysis device, threat language model analysis medium and electronic equipment Download PDFInfo
- Publication number
- CN117786088A CN117786088A CN202410051928.6A CN202410051928A CN117786088A CN 117786088 A CN117786088 A CN 117786088A CN 202410051928 A CN202410051928 A CN 202410051928A CN 117786088 A CN117786088 A CN 117786088A
- Authority
- CN
- China
- Prior art keywords
- information
- threat
- text
- language model
- extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 71
- 238000000605 extraction Methods 0.000 claims abstract description 138
- 238000000034 method Methods 0.000 claims abstract description 41
- 239000012634 fragment Substances 0.000 claims description 47
- 238000012545 processing Methods 0.000 claims description 19
- 239000000284 extract Substances 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 abstract description 7
- 206010000117 Abnormal behaviour Diseases 0.000 abstract description 4
- 230000010365 information processing Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 9
- 230000008520 organization Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000010223 real-time analysis Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 208000015181 infectious disease Diseases 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a language model analysis method of threat, comprising the following steps: acquiring threat information and judging the type of the threat information; selecting an information extraction mode for carrying out language model analysis on threat information according to the type of the threat information; extracting the information of the threat information by using the selected information extraction mode; and obtaining the information extraction result so as to obtain a threat knowledge graph corresponding to the threat information. By the method, information can be efficiently extracted and integrated from a large amount of unstructured network threat information, the information processing efficiency is improved, and the recognition and analysis capability of APT attacks are enhanced; abnormal behaviors in the network can be identified quickly, responses can be made timely, and risks of network attacks are reduced effectively; the interpretability of the large language model is increased by fusing the knowledge graph, and the understandability and reliability of the output result of the model are improved.
Description
Technical Field
The invention relates to the technical field of network security and event extraction, in particular to a threat language model analysis method, a threat language model analysis device, a threat language model analysis medium and an threat language model analysis electronic device.
Background
With the rapid development of the internet and information technology, the attack patterns of advanced persistent threats (APT, advanced Persistent Threats) are more and more complex and difficult to prevent. This type of attack typically involves multiple attack stages, including initial infection, lateral movement, and data theft, and often uses highly customized attack means. Thus, traditional security measures, such as firewalls and intrusion detection systems, have been difficult to meet the requirements of modern network security. At present, the network space is highly resistant, the practical requirement of resisting network threat is based on, the combined application of artificial intelligence technology, threat information and knowledge patterns is explored, and the method has important significance for the research and application of constructing large-scale APT threat knowledge patterns in a more efficient and intelligent mode.
The rise of artificial intelligence becomes a core driving force for assisting intelligent network attack and defense, and the knowledge graph serving as an important branch of the artificial intelligence solves the problem of threat information structural association to a certain extent.
Prior to the LLMs era, named Entity Recognition (NER) and relationship extraction (Relation Extraction) techniques relied primarily on rule-based methods, machine learning algorithms, and some deep learning models. Into the era of large language models (Large Language Model, LLMs), large models such as GPT-4, chatGLM, etc., have demonstrated excellent performance in a variety of tasks and scenarios. Unlike traditional models, LLMs can automatically extract meaningful features and patterns from large amounts of network data, thereby helping security analysts to more effectively identify and prevent potential threats. By means of the real-time analysis capability of LLMs, a security team can rapidly identify abnormal behaviors in a network and respond timely, so that risks brought by network attacks are greatly reduced.
While large language models are favored for their excellent performance in various natural language processing tasks, they are also of interest due to the generation of false information and lack of interpretability. To overcome these problems, researchers have proposed using knowledge-graph to augment large language models. There are two main approaches proposed: one is to integrate knowledge graph into model in pre-training stage of LLMs, so that LLMs can draw knowledge in knowledge graph; and the other is to integrate the knowledge graph into LLMs in the reasoning stage, and search the domain-specific knowledge from the knowledge graph, thereby obviously improving the performance of the model.
However, while large language model enhanced knowledge graphs (LLM-authorized KGs) are theoretically an effective solution, many problems still remain in practice: efficient fusion of structured information of knowledge-graphs with unstructured data of large language models is a challenge, requiring accurate algorithms and model designs to ensure efficient integration of both, while maintaining performance and reliability of the model, which presents great difficulty in terms of implementation. While LLMs such as GPT-4 exhibit excellent performance in multitasking and scenarios, applications in specific areas still face the challenge of generalization capability, and the model requires further training and tuning for specific areas. Furthermore, while LLMs offer new possibilities in terms of real-time monitoring and response of APT protection, it remains a challenge to ensure accuracy and timeliness of real-time analysis, and response time and processing power of the model will become limiting factors when dealing with large amounts of data and complex network environments.
Therefore, there is a need to provide a threat analysis method that can respond timely, accurately and reliably in the face of current complex and diverse network attacks.
Disclosure of Invention
The invention aims to provide a language model analysis method, device, medium and electronic equipment for threat, which are used for solving the problems of slow threat information analysis response and poor reliability.
In a first aspect, the present invention provides a method for language model analysis of threats, comprising: acquiring threat information and judging the type of the threat information; selecting an information extraction mode for carrying out language model analysis on the threat information according to the type of the threat information; extracting information from the threat information by using the selected information extraction mode; and obtaining an information extraction result so as to obtain a threat knowledge graph corresponding to the threat information.
The threat language model analysis method provided by the invention has the beneficial effects that: by combining a large language model and a knowledge graph, information can be efficiently extracted and integrated from a large amount of unstructured network threat information, so that the information processing efficiency is improved, and the recognition and analysis capability of APT attacks is enhanced. Further, by utilizing the real-time analysis capability of the large language model, abnormal behaviors in the network can be rapidly identified, and responses can be timely made, so that enhanced real-time monitoring and response are realized, and the risk of network attack is effectively reduced. The interpretability of the large language model is improved by fusing the knowledge graph, so that a security analyst can better understand and trust the output of the model.
In a possible embodiment, selecting an information extraction manner for performing language model analysis on the threat intelligence according to the type of the threat intelligence includes: when the threat information is of a short text threat information type, adopting a large model enhancement extraction mode based on a prompt to extract information of language model analysis of the threat information; when the threat information is of the type of M comprehensive threat information, adopting a large model enhancement extraction mode based on a search engine to extract information of language model analysis of the threat information, wherein M is more than 1; when the threat information is of a long text threat information type, extracting information of language model analysis of the threat information by adopting a large model enhancement mode oriented to the long threat information text.
In another possible embodiment, when the type of the threat information is short text threat information, the information extracting method is applied to extract information of the threat information, including: determining an information extraction target; designing a prompt according to the extraction target; and guiding the large language model to identify and extract information according to the prompt so as to obtain an information extraction result.
In other possible embodiments, when the type of the threat information is M threat information, the information extraction method is selected to extract information from the threat information, where M is greater than 1, and includes: extracting a target construction problem according to the information; extracting keywords in the problem by using a keyword extraction algorithm; searching by a search engine according to the keywords to obtain a feedback text; constructing an enhanced query text according to the feedback text and the questions; and carrying out large-scale language model query according to the enhanced query text to obtain an information extraction result.
When the threat information is of a long text threat information type, the information extraction method is applied to extract the threat information, and the method comprises the following steps: dividing the long text threat information to obtain N text fragments, wherein N is more than 1; the N text fragments are stored after vectorization processing; retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments; constructing a question-answering environment according to the related text fragments; and carrying out question-answering processing corresponding to the question-answering environment by using a large language model to obtain an information extraction result.
In a second aspect, the present invention also provides a language model analysis device for threats, including:
the judging unit is used for acquiring threat information and judging the type of the threat information;
the selection unit is used for selecting an information extraction mode for carrying out language model analysis on the threat information according to the type of the threat information;
the extraction unit is used for extracting information from the threat information by applying the selected information extraction mode;
and the acquisition unit is used for acquiring the information extraction result so as to obtain a threat knowledge graph corresponding to the threat information.
The selecting unit selects an information extraction mode for carrying out language model analysis on the threat information according to the type of the threat information, and the information extraction mode comprises the following steps: when the threat information is of a short text threat information type, adopting a large model enhancement extraction mode based on a prompt to extract information of language model analysis of the threat information; when the threat information is of the type of M comprehensive threat information, adopting a large model enhancement extraction mode based on a search engine to extract information of language model analysis of the threat information, wherein M is more than 1; when the threat information is of a long text threat information type, extracting information of language model analysis of the threat information by adopting a large model enhancement mode oriented to the long threat information text.
When the threat information is of a short text threat information type, the extracting unit performs information extraction on the threat information by using the selected information extracting mode, and the method comprises the following steps: determining an information extraction target; designing a prompt according to the extraction target; and guiding the large language model to identify and extract information according to the prompt so as to obtain an information extraction result.
When the threat information is of the type of M comprehensive threat information, the extraction unit performs information extraction on the threat information by using the selected information extraction mode, wherein M is more than 1, and the method comprises the following steps: extracting a target construction problem according to the information; extracting keywords in the problem by using a keyword extraction algorithm; searching by a search engine according to the keywords to obtain a feedback text; constructing an enhanced query text according to the feedback text and the questions; and carrying out large-scale language model query according to the enhanced query text to obtain an information extraction result.
When the threat information is of a long text threat information type, the extracting unit performs information extraction on the threat information by using the selected information extracting mode, and the method comprises the following steps: dividing the long text threat information to obtain N text fragments, wherein N is more than 1; the N text fragments are stored after vectorization processing; retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments; constructing a question-answering environment according to the related text fragments; and carrying out question-answering processing corresponding to the question-answering environment by using a large language model to obtain an information extraction result.
In a third aspect, the present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the language model analysis method of the threat described above.
In a fourth aspect, the present invention also provides an electronic device, including: a processor and a memory; the memory is used for storing a computer program; the processor is configured to execute the computer program stored in the memory, so that the electronic device performs the language model analysis method of the threat.
The advantageous effects concerning the above second to fourth aspects can be seen from the description of the above first aspect.
Drawings
FIG. 1 is a schematic flow chart of a method for analyzing a language model of a threat according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a specific implementation method of a threat language model analysis method according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a prompt example of a large model enhancement extraction mode based on a prompt provided in an embodiment of the present invention;
FIG. 4 is a diagram illustrating an exemplary analysis result of a large-model enhancement extraction method based on a prompt provided in an embodiment of the present invention;
FIG. 5 is a diagram illustrating an exemplary method for providing a large model enhancement extraction method based on a search engine according to an embodiment of the present invention;
FIG. 6 is a diagram of an example query result for a search engine-based large model enhancement extraction without and with search engine query enhancement according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a specific flow chart of a large model enhancement mode for long threat information text provided in an embodiment of the present invention;
FIG. 8 is a diagram of an example of a result of extracting threat information using a large model enhancement mode for long threat information text according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a threat language model analysis apparatus provided by an embodiment of the invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. Unless otherwise defined, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. As used herein, the word "comprising" and the like means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof without precluding other elements or items.
Aiming at the problems existing in the prior art, the embodiment of the invention provides a threat language model analysis method, a threat language model analysis device, a threat language model analysis medium and an electronic device.
The embodiment provides a language model analysis method for threats. Referring to fig. 1 of the specification, the method comprises:
s101: threat information is acquired, and the type of the threat information is judged.
In S101, in one possible embodiment, the method of the present invention obtains the threat information of the network to perform the language model analysis of the threat, the analyzed threat information has multiple types, and after obtaining the threat information, the type of the obtained threat information needs to be determined. Illustratively, the threat intelligence that is obtained to be used for one time language model analysis includes short text threat intelligence, long text threat intelligence, and comprehensive M threat intelligence, where M > 1.
S102: and selecting an information extraction mode for carrying out language model analysis on the threat information according to the type of the threat information.
In S102, in one possible embodiment, the present invention uses different methods to extract threat information by using language model analysis according to the characteristics of different types of threat information, so that the information extraction result can better enable the user to understand and trust.
Referring to fig. 2 of the specification, an information extraction method for performing language model analysis on the threat intelligence is selected according to the type of the threat intelligence, and includes: when the threat information is of a short text threat information type, adopting a large model enhancement extraction mode based on a prompt to extract information of language model analysis of the threat information; when the threat information is of the type of M comprehensive threat information, adopting a large model enhancement extraction mode based on a search engine to extract information of language model analysis of the threat information; when the threat information is of a long text threat information type, the threat information is extracted by adopting a large model enhancement mode oriented to the long threat information text to carry out language model analysis on the threat information.
S103: and extracting the information of the threat information by using the selected information extraction mode.
In S103, in one possible embodiment, when the type of threat information is short text threat information, the information extraction method selected is applied to extract the threat information, including: determining an information extraction target; designing a prompt according to the extraction target; and guiding the large language model to identify and extract information according to the prompt so as to obtain an information extraction result. When the threat information is of the type of M comprehensive threat information, the information extraction method is applied to extract the threat information, and the method comprises the following steps: extracting a target construction problem according to the information; extracting keywords in the problem by using a keyword extraction algorithm; searching by a search engine according to the keywords to obtain a feedback text; constructing an enhanced query text according to the feedback text and the problems; and carrying out large-scale language model query according to the enhanced query text to obtain an information extraction result. When the threat information is of a long text threat information type, the information extraction method selected by the application is used for extracting the threat information, and the method comprises the following steps: dividing the long text threat information to obtain N text fragments, wherein N is more than 1; the N text fragments are stored after vectorization treatment; retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments; constructing a question-answer environment according to the related text fragments; and carrying out question-answering processing corresponding to the question-answering environment by using the large language model to obtain an information extraction result. In addition, the method of the invention can also extract the threat information in PDF file format, comprising: storing the PDF document related to threat information, then decomposing the PDF document to extract text content, and dividing the text content to obtain N text fragments; the N text fragments are stored after vectorization treatment; retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments; constructing a question-answer environment according to the related text fragments; and carrying out question-answering processing corresponding to the question-answering environment by using the large language model to obtain an information extraction result.
In one possible embodiment, the information extraction targets for extracting the threat information include entity relationship types, entity attribute information and entity identification, entity relationships and attributes.
For example, when entity relation extraction is performed on threat information by adopting a large model enhancement extraction mode based on prompt, the specific process comprises the following steps: and (3) determining which type of entity relationship is required to be extracted from the threat information, then designing one or a series of prompt languages based on the type of the entity relationship to be extracted, guiding the large-scale language model to identify and extract information through the prompt languages, and further obtaining the extracted entity relationship information. Wherein, the design of suggestion is: firstly, an extraction target is formulated according to APT expert knowledge, then an output format is designated in a sample form, and finally an interaction mode of a large-scale language model in the extraction process is designated. The design flow of the prompt can be preset according to the type of the entity relationship, and the design of the prompt can be completed according to the design flow of the set prompt after the type of the entity relationship to be extracted is clarified.
The prompter is shown in fig. 3 of the specification, taking as an example a report on Bahamut APT organization issued by ESET2022, month 11. The report reveals that Bahamut is active against an active malware propagation activity of Android users since month 1 of 2022. These malware are distributed through a fake secure VPN web site, which is a tampered version of a legitimate VPN application (such as SoftVPN or OpenVPN), with Bahamut's spyware code embedded. These software are intended to extract sensitive user data and monitor the instant messaging application of the victim. The entity relation extraction is carried out on threat information by utilizing the large model enhancement extraction mode based on the prompt language designed by the invention, and the obtained analysis result is shown in figure 4. The chatgpt3.5 well extracts entity relation information related to the APT attack. Therefore, the invention can well guide the ChatGPT3.5 to perform entity relationship joint extraction according to specific requirements aiming at unstructured texts in the threat intelligence field.
When entity attribute extraction is carried out on threat information by adopting a large model enhancement extraction mode based on a search engine, the concrete process comprises the following steps: inputting a question about entity attribute information to be marked, extracting keywords in the question by using a ChatGPT or other keyword extraction algorithm, inputting the keywords into a search engine to search the search engine to obtain abstract text fed back by the search engine, expanding the abstract text fed back by the search engine and the question together to construct an enhanced query text, providing the enhanced query text as input to the ChatGPT to perform line large language model query, and obtaining an answer generated by the ChatGPT, namely the extracted entity attribute information.
LLMs and web search engines each have distinct advantages, LLMs can provide more humanized, contextually relevant answers, and search engines are generally more suited for rapid retrieval of information. The combination of the two allows the user to obtain a more comprehensive solution on one platform. First, by the natural language processing capabilities of LLMs, the accuracy and relevance of search engines may be significantly improved, especially for complex or ambiguous queries. Second, LLMs can guide users through conversational patterns to more accurately describe their needs so that search engines can more efficiently provide relevant information. Therefore, the method of the invention combines the search engine and the ChatGPT to extract the entity attribute. Referring to fig. 5 of the specification, a specific example of a method for improving ChatGPT generated answers using the results of a search engine is given.
The assumption is that attribute information of the malware CROSSWALK is marked according to a large model enhancement extraction mode based on a search engine. The example shown in fig. 6 illustrates an answer example of chatgpt3.5 using the method proposed by the present invention without search engine query enhancement a) and with search engine query enhancement b). The difference between the two answers in terms of alias, version, operating system, description and official address can be clearly seen from figure 6. In the alias aspect, the answer with search engine query enhancement b) is easily retrieved by the search engine as an alias to the cross-bar. In version, using malware as an APT attack is not disclosed with an exact version number, so that the answer with search engine query enhancement b) is unqualified, and the answer without search engine query enhancement a) is v1.0, creating a "illusion" phenomenon (large model answer is not in agreement with the fact). Similarly, the absence of a search engine query enhancing answer to a) creates a "illusion" phenomenon that is inconsistent with the fact that it is not an answer to three aspects of operating system, description and office address.
When entity identification, entity relation and attribute extraction are carried out on threat information in PDF file format by adopting a large model enhancement mode oriented to long threat information text, the flow is shown in fig. 7, and the process specifically comprises the following steps: first, PDF documents related to network threat intelligence (CTI) are stored. Then, the PDF document is decomposed by using a LangChain technology through a PyPDF2 library, text content is extracted, and segmentation is performed by using a CharacterTextSpliter, so that N independent text fragments are generated. The text fragments are vectorized and then stored in a vector space database designed specifically for this purpose. For entity identification, entity relation and attribute information to be extracted, the system retrieves text fragments most relevant to the entity identification, entity relation and attribute information content to be extracted from a vector space database, constructs a context environment of a problem according to the most relevant text fragments to obtain a question-answer environment, and carries out question-answer processing on the question-answer environment by means of a large language model to realize accurate question-answer on the entity identification, entity relation and attribute information to be extracted, so as to obtain the entity identification, entity relation and attribute information of the corresponding threat information. The PDF document is decomposed through a PyPDF2 library, and the text content extraction comprises the following steps: and reading the PDF file of the CTI text by using the PdfReader object of the PyPDF2, and extracting the text content of each page. Segmentation of the text by a CharacterTextSplitter enables the text to form smaller, easily handled blocks. Vectorizing the text segment and then storing the text segment in a vector space database specially designed for the text segment comprises: text embedding is performed using the openaiededings module of Langchain, while the FAISS library is incorporated, to create and maintain a text retrieval system capable of storing and processing the vectorized representation of each text block. The method for obtaining the question-answering environment according to the context environment of the most relevant text fragments for constructing the questions comprises the following steps: and integrating the text blocks through the QA Chain function in Langchain to form a coherent question-answering environment. And the text blocks are subjected to question-answering processing by using large language models such as GPT and the like, so that entity identification, entity relation and attribute extraction can be efficiently executed.
A major limitation of large language models such as GPT-3.5 in processing long text and PDF content is that they have limitations on text length, which may affect understanding of the overall context of long text. Furthermore, the model cannot directly parse PDF files, requiring PDF content to be converted to plain text, which may result in loss of format and layout information. The large model enhancement mode for the long threat information text designed in the method can well solve the problems.
Fig. 8 shows an example of acquiring key entities, relationships, and entity attribute information by question-answering, taking the threat intelligence pdf text of a 28-page publication of the aps organization darkhote by kaspersky at 11 in 2014 as an example. The answer to the first question "What does this threat intelligence say; for the second question "What kinds of malware does Darkhotel use: "Information Stealer", thereby obtaining a relationship of < Darkhotel, use, information Stealer >; aiming at the third problem, the 'large model enhancement facing long threat information text' extracts the related attribute of APT organization DARCHOTEL according to defined rules:
motiation (Motivation): espionage (spy event)
Sophisitization (threat level): advanced (advanced)
resource_level: organization of the organization level
Category: spy (Spy).
S104: and obtaining the information extraction result so as to obtain a threat knowledge graph corresponding to the threat information.
In S104, in a possible embodiment, the information extraction method designed in the method of the present invention is applied to extract the threat information to obtain an information extraction result, so that the fragmented threat information can be constructed into a structured information containing multiple entity types and multiple logic relationship types, such as APT attack organization information, threat information entities, and campaign weapon sets, so as to implement deep analysis and feature extraction of multi-modal data (such as network traffic, operating system logs, security reports of human experts, etc.), and help construct a comprehensive and highly dynamic threat knowledge map.
The language model analysis method of the threat provided by the invention can efficiently extract and integrate information from a large amount of unstructured network threat information by combining LLMs and knowledge patterns, thereby not only improving the efficiency of information processing, but also enhancing the recognition and analysis capability of APT attack. Further, by utilizing the real-time analysis capability of LLMs, the invention can quickly identify abnormal behaviors in the network and respond in time, thereby realizing enhanced real-time monitoring and response and effectively reducing the risk of network attack. The interpretability of the large language model is increased by fusing the knowledge graph, and the understandability and reliability of the output result of the model are improved, so that a security analyst can better understand and trust the output of the model. In addition, the invention adopts special technology to deal with the challenges of processing long text and PDF content, and improves the accuracy and reliability of the model in processing such data.
Referring to fig. 9 of the specification, the present embodiment further provides a language model analysis device for threat, where the device is used to implement the above method embodiment. The device comprises:
a judging unit 201, configured to obtain threat information and judge a type of the threat information.
A selection unit 202, configured to select an information extraction mode for performing language model analysis on threat information according to the type of threat information.
The extracting unit 203 is configured to apply the selected information extracting method to extract information from threat information.
And the obtaining unit 204 is configured to obtain the information extraction result, thereby obtaining a threat knowledge graph corresponding to the threat information.
Wherein the selecting unit 202 selects an information extraction mode for performing language model analysis on threat information according to the type of the threat information, including: when the threat information is of a short text threat information type, adopting a large model enhancement extraction mode based on a prompt to extract information of language model analysis of the threat information; when the threat information is of the type of M comprehensive threat information, adopting a large model enhancement extraction mode based on a search engine to carry out information extraction of language model analysis on the threat information, wherein M is more than 1; when the threat information is of a long text threat information type, the threat information is extracted by adopting a large model enhancement mode oriented to the long threat information text to carry out language model analysis on the threat information.
When the threat information is of a short text threat information type, the extracting unit 203 performs information extraction on the threat information by applying the selected information extraction method, including: determining an information extraction target; designing a prompt according to the extraction target; and guiding the large language model to identify and extract information according to the prompt so as to obtain an information extraction result.
When the threat information is of the type M threat information, the extracting unit 203 performs information extraction on the threat information by applying the selected information extraction method, where M is greater than 1, and includes: extracting a target construction problem according to the information; extracting keywords in the problem by using a keyword extraction algorithm; searching by a search engine according to the keywords to obtain a feedback text; constructing an enhanced query text according to the feedback text and the problems; and carrying out large-scale language model query according to the enhanced query text to obtain an information extraction result.
When the threat information is of the type of long text threat information, the extracting unit 203 performs information extraction on the threat information by applying the selected information extraction method, including: dividing the long text threat information to obtain N text fragments, wherein N is more than 1; the N text fragments are stored after vectorization treatment; retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments; constructing a question-answer environment according to the related text fragments; and carrying out question-answering processing corresponding to the question-answering environment by using the large language model to obtain an information extraction result.
All relevant contents of each step related to the above method embodiment may be cited to the functional descriptions of the corresponding functional modules, which are not described herein.
In other embodiments of the present application, embodiments of the present application disclose an electronic device, as shown in fig. 10, the electronic device 300 may include: one or more processors 301; a memory 302; a display 303; one or more applications (not shown); and one or more computer programs 304, which may be connected via one or more communication buses 305. Wherein the one or more computer programs 304 are stored in the memory and configured to be executed by the one or more processors 301, the one or more computer programs 304 comprise instructions that may be used to perform the various steps as in fig. 1, 9 and corresponding embodiments.
From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above. The specific working processes of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which are not described herein.
The functional units in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: flash memory, removable hard disk, read-only memory, random access memory, magnetic or optical disk, and the like.
The foregoing is merely a specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the embodiments of the present application should be covered by the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. A method of language model analysis of a threat, comprising:
acquiring threat information and judging the type of the threat information;
selecting an information extraction mode for carrying out language model analysis on the threat information according to the type of the threat information;
extracting information from the threat information by using the selected information extraction mode;
and obtaining an information extraction result so as to obtain a threat knowledge graph corresponding to the threat information.
2. The method of claim 1, wherein selecting an information extraction manner for performing language model analysis on the threat intelligence according to the type of the threat intelligence comprises:
when the threat information is of a short text threat information type, adopting a large model enhancement extraction mode based on a prompt to extract information of language model analysis of the threat information;
when the threat information is of the type of M comprehensive threat information, adopting a large model enhancement extraction mode based on a search engine to extract information of language model analysis of the threat information, wherein M is more than 1;
when the threat information is of a long text threat information type, extracting information of language model analysis of the threat information by adopting a large model enhancement mode oriented to the long threat information text.
3. The method of claim 1, wherein when the threat intelligence is of a short text threat intelligence type, applying the selected information extraction method to extract information from the threat intelligence comprises:
determining an information extraction target;
designing a prompt according to the extraction target;
and guiding the large language model to identify and extract information according to the prompt so as to obtain an information extraction result.
4. The method of claim 1, wherein when the threat intelligence type is M threat intelligence pieces, performing information extraction on the threat intelligence by using a selected information extraction method, wherein M > 1, includes:
extracting a target construction problem according to the information;
extracting keywords in the problem by using a keyword extraction algorithm;
searching by a search engine according to the keywords to obtain a feedback text;
constructing an enhanced query text according to the feedback text and the questions;
and carrying out large-scale language model query according to the enhanced query text to obtain an information extraction result.
5. The method of claim 1, wherein when the threat intelligence is of the type long text threat intelligence, applying the selected information extraction method to extract information from the threat intelligence comprises:
dividing the long text threat information to obtain N text fragments, wherein N is more than 1;
the N text fragments are stored after vectorization processing;
retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments;
constructing a question-answering environment according to the related text fragments;
and carrying out question-answering processing corresponding to the question-answering environment by using a large language model to obtain an information extraction result.
6. A language model analysis device for threats, the device comprising:
the judging unit is used for acquiring threat information and judging the type of the threat information;
the selection unit is used for selecting an information extraction mode for carrying out language model analysis on the threat information according to the type of the threat information;
the extraction unit is used for extracting information from the threat information by applying the selected information extraction mode;
and the acquisition unit is used for acquiring the information extraction result so as to obtain a threat knowledge graph corresponding to the threat information.
7. The apparatus of claim 6, wherein the selecting unit selects an information extraction manner for performing language model analysis on the threat intelligence according to a type of the threat intelligence, comprising:
when the threat information is of a short text threat information type, adopting a large model enhancement extraction mode based on a prompt to extract information of language model analysis of the threat information;
when the threat information is of the type of M comprehensive threat information, adopting a large model enhancement extraction mode based on a search engine to extract information of language model analysis of the threat information, wherein M is more than 1;
when the threat information is of a long text threat information type, extracting information of language model analysis of the threat information by adopting a large model enhancement mode oriented to the long threat information text.
8. The apparatus of claim 6, wherein when the type of threat intelligence is a short text threat intelligence, the extracting unit performs information extraction on the threat intelligence using the selected information extraction method, including:
determining an information extraction target;
designing a prompt according to the extraction target;
and guiding the large language model to identify and extract information according to the prompt so as to obtain an information extraction result.
9. The apparatus of claim 6, wherein when the type of threat intelligence is a comprehensive M threat intelligence, the extracting unit performs information extraction on the threat intelligence by using the selected information extraction method, where M > 1, and includes:
extracting a target construction problem according to the information;
extracting keywords in the problem by using a keyword extraction algorithm;
searching by a search engine according to the keywords to obtain a feedback text;
constructing an enhanced query text according to the feedback text and the questions;
and carrying out large-scale language model query according to the enhanced query text to obtain an information extraction result.
10. The apparatus of claim 6, wherein when the type of threat intelligence is a long text threat intelligence, the extracting unit performs information extraction on the threat intelligence using the selected information extraction method, including:
dividing the long text threat information to obtain N text fragments, wherein N is more than 1;
the N text fragments are stored after vectorization processing;
retrieving text fragments most relevant to the information extraction target content from the N text fragments to obtain relevant text fragments;
constructing a question-answering environment according to the related text fragments;
and carrying out question-answering processing corresponding to the question-answering environment by using a large language model to obtain an information extraction result.
11. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the language model analysis method of a threat of any of claims 1 to 5.
12. An electronic device, comprising: a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to execute the computer program stored in the memory to cause the electronic device to perform the method of language model analysis of a threat as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410051928.6A CN117786088A (en) | 2024-01-15 | 2024-01-15 | Threat language model analysis method, threat language model analysis device, threat language model analysis medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410051928.6A CN117786088A (en) | 2024-01-15 | 2024-01-15 | Threat language model analysis method, threat language model analysis device, threat language model analysis medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117786088A true CN117786088A (en) | 2024-03-29 |
Family
ID=90401730
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410051928.6A Pending CN117786088A (en) | 2024-01-15 | 2024-01-15 | Threat language model analysis method, threat language model analysis device, threat language model analysis medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117786088A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118013046A (en) * | 2024-04-02 | 2024-05-10 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Unstructured network threat information extraction method, system and medium |
-
2024
- 2024-01-15 CN CN202410051928.6A patent/CN117786088A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118013046A (en) * | 2024-04-02 | 2024-05-10 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Unstructured network threat information extraction method, system and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lin et al. | Retrieval of relevant historical data triage operations in security operation centers | |
| Kaloroumakis et al. | Toward a knowledge graph of cybersecurity countermeasures | |
| CN110717049A (en) | Text data-oriented threat information knowledge graph construction method | |
| Fawei et al. | A semi-automated ontology construction for legal question answering | |
| Shim et al. | A link2vec-based fake news detection model using web search results | |
| Huang et al. | JSContana: Malicious JavaScript detection using adaptable context analysis and key feature extraction | |
| US20130159346A1 (en) | Combinatorial document matching | |
| CN117786088A (en) | Threat language model analysis method, threat language model analysis device, threat language model analysis medium and electronic equipment | |
| CN116668057A (en) | Knowledge graph-based network security situation understanding and analyzing method | |
| Zhang et al. | EX‐Action: Automatically Extracting Threat Actions from Cyber Threat Intelligence Report Based on Multimodal Learning | |
| CN112580331A (en) | Method and system for establishing knowledge graph of policy text | |
| KR20190138037A (en) | An information retrieval system using knowledge base of cyber security and the method thereof | |
| Hu et al. | Cross-site scripting detection with two-channel feature fusion embedded in self-attention mechanism | |
| Amato et al. | An application of semantic techniques for forensic analysis | |
| Abaimov et al. | A survey on the application of deep learning for code injection detection | |
| Zhu et al. | Ontology-based approach for the measurement of privacy disclosure | |
| Rahmani et al. | Improving code example recommendations on informal documentation using bert and query-aware lsh: A comparative study | |
| Déneckère et al. | A framework for comparative analysis of intention mining approaches | |
| Hadi et al. | Introduction to ChatGPT: A new revolution of artificial intelligence with machine learning algorithms and cybersecurity | |
| Aktayeva et al. | Cognitive computing cybersecurity: social network analysis | |
| Yadav et al. | Generalized rough and fuzzy rough automata for semantic computing | |
| Chen et al. | An Efficient ROS Package Searching Approach Powered By Knowledge Graph. | |
| Lin et al. | Novel JavaScript malware detection based on fuzzy Petri nets | |
| Quang et al. | Recent research on phishing detection through machine learning algorithm | |
| Liu et al. | Toward a better alignment between the research and practice of code search engines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |