KR20210097408A - Device updating harmful website information and method thereof - Google Patents

Abstract

Disclosed are an apparatus for updating information on a harmful website and method according to the same, capable of automatically collecting an addresses of a harmful website. The apparatus for updating information on a harmful website includes: a crawling unit; a web content analyzing unit; and an update unit. The crawling unit searches for a website by a search engine, based on search data related to feature data according to the feature analysis of the harmful site previously stored in a harmful site database unit, and periodically and automatically acquires object information related to the content of the website together with the address of the website from the searched website. The web content analyzing unit determines whether calculated harmfulness exceeds a harmfulness threshold value by analyzing feature data extracted from the object information, determines that the website is a harmful site, when the calculated harmfulness exceeds the harmfulness threshold value, and generates harmful analysis information at least including the address of the relevant website and the feature data. The update unit automatically updates the harmful site database unit based on the harmful analysis information.

Description

Device and method for updating harmful website information {DEVICE UPDATING HARMFUL WEBSITE INFORMATION AND METHOD THEREOF}

The present disclosure relates to an apparatus and method for updating harmful website information, and more specifically, to automatically analyze topics, keywords, images, etc. on harmful websites without human intervention to determine whether harmful websites are harmful, A device for updating harmful website information and a method therefor for automatically collecting the address of the harmful site to provide useful information to an external service that determines and blocks the site.

Recently, with the development of media and information and communication technology, the tendency to produce harmful media information at a rate that cannot be processed by humans is increasing. In the flood of harmful media information, various methods for information consumers or sellers to discover and block harmful media in the information are being proposed. Typically, there is a method of blocking a malicious site address by a human being searched for by a firewall or Internet backbone.

In the past, a method was mainly used in which a person visited the site, judged a harmful site, and collected the site address. In order to improve this, it is efficient to automatically preselect and find harmful site addresses through a program and automatically update the information, but there is no method for this.

The technical problem of the present disclosure is to provide useful information to an external service that automatically analyzes topics, keywords, images, etc. without human intervention to determine whether a harmful website is a harmful website, and determines and blocks harmful websites. It is to provide a device for updating harmful website information and a method therefor for automatically collecting the address of the harmful website.

Another technical problem of the present disclosure is a device for updating harmful website information to periodically determine whether a harmful website is a harmful website and update the information on the harmful website to the latest even if the situation changes, such as when the address of the harmful website is changed or closed and to provide a method thereby.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be able

According to an aspect of the present disclosure, an apparatus for updating harmful website information may be provided. The device for updating harmful website information searches for a website by a search engine based on search data related to characteristic data according to characteristic analysis of a harmful site previously stored in the harmful website database unit, and searches the website from the searched website. A crawling unit that periodically and automatically acquires object information related to the content of the website together with the address, and analyzes the feature data extracted from the object information to determine whether or not the calculated degree of harm exceeds a harmful threshold, and exceeds a web content analysis unit that determines the website as a harmful site and generates harmful analysis information including at least the address and characteristic data of the website, and automatically updates the harmful site database unit with the harmful analysis information Includes update.

According to another embodiment of the present disclosure, the content is composed of at least one of text, video data, and image data, and the search data is at least one of keywords, images, and videos constituting the previously stored characteristic data of harmful sites. can be

In addition, in the case of the text, the characteristic data is a word extracted with reference to the harmful word stored in the harmful site database unit as well as the keyword, and in the case of the video data or the image data, a characteristic element estimated as a harmful shape can

In addition, in the case of the text, the web content analysis unit uses a word2vec, doc2vec, or LDA (Latent Dirichlet Allocation) topic analysis algorithm based on the extracted word and the keyword constituting the feature data extracted from the object information. hazard can be calculated.

In addition, in the case of the video data and the image data, the web content analysis unit may calculate the harmfulness by using machine learning or deep learning on the feature element of the feature data extracted from the object information.

According to another embodiment of the present disclosure, the crawling unit may further include obtaining object collection information including metadata related to the object information and classifying the object collection information for each website.

In addition, when the content is text, the harmful analysis information further includes at least one of the object collection information, the topic, and a context result value between words of text, and the content is video data or image data In the case of , the harmful analysis information may further include one or more of the object collection information, the topic, and data constituting the object information.

In addition, keywords, words, and feature elements estimated as harmful shapes extracted in relation to the feature data may be managed in at least one of the web content analysis unit and the harmful site database unit.

According to another embodiment of the present disclosure, the web content analysis unit is connected to a harmful media external detector configured as an external module through a harmfulness detection connection unit, and the harmful media external detector analyzes and calculates characteristic data extracted from the object information It may be determined whether the determined harmfulness exceeds a harmful threshold, and a result of the determination may be notified to the web content analysis unit.

According to another embodiment of the present disclosure, harmful analysis information including the address of the website determined as the harmful site and the search data is expressed as data using an ontology and stored in the harmful site database unit. can

According to another embodiment of the present disclosure, the update unit updates the harmful analysis information when the object information and the harmful analysis information of the website determined as the harmful site are different from information previously stored in the harmful site database unit. can do.

According to another aspect of the present disclosure, a method of updating harmful website information may be provided. The method of updating harmful website information includes searching a website by a search engine based on search data related to characteristic data according to characteristic analysis of a harmful site pre-stored in a harmful site database unit, and searching for a website from the searched website. A crawling step of periodically automatically acquiring object information related to the content of the website together with an address, and determining whether the degree of harm calculated by analyzing the feature data extracted from the object information exceeds a harmful threshold, exceeding a web content analysis step of determining the website as a harmful site and generating harmful analysis information including at least an address and characteristic data of the website, and automatically updating the harmful site database unit with the harmful analysis information Includes update steps.

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows, and do not limit the scope of the present disclosure.

According to the present disclosure, it is possible to automatically collect and update the address of a harmful website without human intervention so as to provide useful information to an external service that performs judgment and blocking of harmful websites, and collects object information that is the media of harmful websites A device and method for updating harmful website information may be provided.

According to the present disclosure, an apparatus for updating harmful website information and a method therefor that periodically determine whether a harmful website is a harmful website and update the information on the harmful website to the latest even if the situation changes, such as when the address of the harmful website is changed or closed can provide

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 is a block diagram illustrating an apparatus for updating harmful website information according to an embodiment of the present invention.
2 is a flowchart illustrating an operation flow of a crawling unit.
3 is a flowchart illustrating an operation flow of a web content analysis unit.
4 is a flowchart illustrating an operation flow of an update unit.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein.

In describing an embodiment of the present disclosure, if it is determined that a detailed description of a well-known configuration or function may obscure the gist of the present disclosure, a detailed description thereof will be omitted. And, in the drawings, parts not related to the description of the present disclosure are omitted, and similar reference numerals are attached to similar parts.

In the present disclosure, when a component is "connected", "coupled" or "connected" with another component, it is not only a direct connection relationship, but also an indirect connection relationship in which another component exists in the middle. may also include. In addition, when a component is said to "include" or "have" another component, it means that another component may be further included without excluding other components unless otherwise stated. .

In the present disclosure, terms such as first, second, etc. are used only for the purpose of distinguishing one component from another, and do not limit the order or importance between the components unless otherwise specified. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment is referred to as a first component in another embodiment. can also be called

In the present disclosure, components that are distinguished from each other are for clearly explaining each characteristic, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated to form one hardware or software unit, or one component may be distributed to form a plurality of hardware or software units. Accordingly, even if not specifically mentioned, such integrated or distributed embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, an embodiment composed of a subset of components described in an embodiment is also included in the scope of the present disclosure. In addition, embodiments including other components in addition to components described in various embodiments are also included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

An apparatus for updating harmful website information according to an embodiment of the present invention will be described with reference to FIG. 1 .

1 is a block diagram illustrating an apparatus for updating harmful website information according to an embodiment of the present invention.

The harmful website information update device 100 can automatically collect and update addresses of harmful websites, and collects object information that is media of harmful websites.

Specifically, the apparatus 100 for updating harmful website information includes a crawling unit 102 , a web content analysis unit 104 , a harmfulness detection connection unit 106 , a harmful media external detector 202 , an update unit 108 and a harmful site It may include a database unit 112 . The harmful media external detector 202 and the harmful media blocking external service module 204 function as separate members from the device 100 .

The crawling unit 102 searches for a website by a search engine based on search data related to characteristic data according to characteristic analysis of a harmful site pre-stored in the harmful site database unit 112, and provides a website address from the searched website. In addition, object information related to the content of the website is automatically acquired periodically. Also, the crawler 102 may obtain object collection information including metadata related to object information and classify the object collection information for each website.

A page provided by a website may simply provide only one type of content, such as video data such as text, video or video clip, or image data such as a photo, but is usually composed of a combination of various types of content. can be Accordingly, the content may be composed of at least one type of text, video data, and image data.

Even if the content that the user accesses through the web page is video data or image data, the search data may be a keyword composed of words, or may be an image or video clip itself. Alternatively, even if the content to be accessed is text, the search data may be not only keywords, but also images or videos from which text can be inferred. Accordingly, the search data may be at least one of keywords, images, and videos constituting previously stored characteristic data of harmful sites. The website search performed by the crawling unit 102 may be a search engine that is provided in the device 100 itself based on the search data or is already commercially available.

The object information extracts at least a portion of each data from one or more of text, video data, and image data posted on the website, and such extraction may be performed by various known techniques. In this case, the metadata of the object collection information may include the title, author, source, creation time, and other descriptions of the content.

The web content analysis unit 104 analyzes the feature data extracted from the object information to determine whether the calculated harmfulness exceeds a harmful threshold, and if it exceeds, determines the website as a harmful site, and the corresponding website and generates harmful analysis information including at least the characteristic data extracted from the object information of the searched website. In this embodiment, the web content analysis unit 104 extracts the characteristic data from the object information, The harmful analysis information is generated according to the example, and the harmful media external detector 202 connected to the web content analysis unit 104 determines the harmfulness of the website. Specifically, the harmful media external detector 202 composed of an external module is connected to the web content analysis unit 104 through the harmfulness detection connection unit 106, and whether the harmfulness calculated by analyzing the characteristic data exceeds the harmful threshold , and notifies the determination result to the web content analysis unit 104 . The present embodiment is for the reduction of data processing and acceptance of various harmfulness determination logic by distributed processing through the device 100 and external modules, but in another embodiment, extraction of feature data, determination of harmfulness, and analysis of harmfulness information The generation may be entirely implemented in the web content analysis unit 104 .

The feature data may be configured as follows when extracted from object information extracted from the searched website as well as the existing one. When the content is text, the characteristic data may be words extracted by referring to harmful words stored in the harmful site database unit 112 together with keywords that are search data. When the content is video data or image data, the characteristic data may be a characteristic element estimated to be a harmful shape, and the estimated characteristic element is extracted from harmful image and harmful video data stored in the harmful site database unit 112 and is already stored. You can refer to the feature element. Although it is possible to extract the same characteristic element as the previously stored characteristic element from the object information, it is common that the previously stored harmful object information and the newly extracted object information are different. It is performed within a certain degree of similarity, such as the Euclidean distance from .

Accordingly, the characteristic data may be the same as the characteristic data of the harmful site pre-stored in the harmful site database unit 110 , or may be newly extracted from the harmful site searched with search data related to the existing characteristic data. Not only the existing but also the newly extracted feature data is inferred from the existing feature data, and as described above, it is managed in at least one of the web content analysis unit 104 and the harmful site database unit 110, and the new feature data is then It can be used as search data in the search of a website. The harmfulness determination and harmfulness analysis information will be described later in the method of updating a harmful website using the device 100 as this embodiment.

The update unit 108 automatically updates the harmful site database unit 112 with harmful analysis information of a website as a harmful site related to the characteristic data determined to be harmful by the web content analysis unit 104 . This update expresses harmful analysis information including at least the address of the website determined to be a harmful site and characteristic data extracted from object information to the harmful site as data using an ontology as a form of knowledge information to express the harmful site database. It can be stored in the unit 112 .

The harmful site database unit 112 stores harmful analysis information extracted from the website determined to be harmful, and transmits it when there is an external request, and an external module that performs whether or not to block the website accessed by the user It is also possible to transmit harmful analysis information to the harmful media blocking external service module 204 as a .

Harmful analysis information includes not only the web address of harmful sites and search data used to search for them, but also harmful words that can be referred to for extracting characteristic data, and characteristic elements extracted from harmful images and harmful video data, as described above. can do.

Hereinafter, a method of updating harmful website information according to another embodiment of the present invention will be described with reference to FIGS. 1 to 4 .

2 is a flowchart illustrating an operation flow of a crawling unit. 3 is a flowchart illustrating an operation flow of a web content analysis unit. 4 is a flowchart illustrating an operation flow of an update unit.

First, referring to FIG. 2 , when the automatic execution cycle of the harmful website information update device 100 arrives (Y in S205), the crawling unit 102 performs a characteristic analysis of harmful sites pre-stored in the harmful site database unit. A website is searched by the search engine based on the search data related to the feature data (S210).

The content may be composed of at least one type of text, video data, and image data.

Even if the content that the user accesses through the web page is video data or image data, the search data may be a keyword composed of words, or may be an image or video clip itself. Alternatively, even if the content to be accessed is text, the search data may be not only keywords, but also images or videos from which text can be inferred. That is, the search data may be at least one of keywords, images, and videos constituting previously stored characteristic data of harmful sites.

Here, the pre-stored characteristic data of the harmful site is the existing characteristic data extracted from the object information of the website that has already been determined to be a harmful site by the characteristic analysis of the web content analysis unit 104, and the web content analysis unit 104 and the harmful site It may be managed by at least one of the site database unit 110 . The crawling unit 102 receives the managed feature data and processes it into search data. In the present embodiment, a website predicted to be highly harmful can be easily found by automatically searching for a website by using existing characteristic data included in the harmful site as search data.

The web site search performed by the crawling unit 102 may be a search engine provided in the device 100 itself based on the search data or a service that is already commercialized.

Accordingly, not only the harmful sites related to the characteristic data already stored in the harmful site database unit 112 but also the changed addresses of the existing harmful sites and the newly opened provisional harmful sites are searched using the search data processed from the characteristic data. can do.

Next, the crawling unit 102 acquires the address of the searched website and content-related object information (S215).

The object information extracts at least a portion of each data from one or more of text, video data, and image data posted on the website, and such extraction may be performed by various known techniques.

Next, the crawler 102 acquires object collection information from the acquired object information (S220).

The object collection information includes metadata related to the object information, and the metadata may include the title, author, source, creation time, and other descriptions of the content. Object collection information is classified for each website, and then is composed of harmful analysis information and stored and managed in the form of knowledge information when updated.

Next, referring to FIG. 3 , when object information is generated (Y in S305), the web content analysis unit 104 analyzes the object information of the searched website, and features related to any one of text, image data, and video data. Data is extracted (S310).

When the content is text, the characteristic data may be words extracted by referring to harmful words stored in the harmful site database unit 112 together with keywords that are search data. When the content is video data or image data, the feature data may be a feature element estimated as a harmful shape, and the estimated feature element is a feature extracted from harmful image and harmful video data stored in the harmful site database unit 112 and already stored. element can be referenced. In the case of video data, the feature data may be, for example, a predetermined feature element extracted based on the MPEG-7 standard. Although it is possible to extract the same characteristic element as the previously stored characteristic element from the object information, it is common that the previously stored harmful object information and the newly extracted object information are different. It is performed within a certain degree of similarity, such as the Euclidean distance from .

Next, the harmful media external detector 202 connected through the web content analysis unit 104 and the harmfulness detection connection unit 106 determines that the harmfulness level based on at least one of characteristic data related to text, image data, and video data is harmful. It is determined whether the threshold value is exceeded (S315).

In the case of text, the harmful media external detector 202 may calculate the degree of harmfulness using a word2vec, doc2vec, or LDA (Latent Dirichlet Allocation) topic analysis algorithm based on the extracted words and keywords constituting the feature data. .

Based on a context correlation inferred from a combination of words and a harmful dictionary of harmful words, phrases, sentences, etc. related to sexual language and violent situations, the harmful media external detector 202 is a keyword extracted from the searched website. , analyzes the harmfulness of the website by performing emotional evaluation of words and the entire text composed of them. For example, the harmfulness analysis calculates the ratio of the words of the feature data in the whole text words, or analyzes whether the topic of the feature data is a sexual or violent topic in terms of the whole context even if the words of the feature data appear with high frequency.

In the case of video data and image data, the harmful media external detector 202 calculates the degree of harmfulness by using machine learning or deep learning on feature elements.

Specifically, a large amount of feature data of video and image data of a harmful site already stored in the harmful site database unit 112 is learned, and it is determined whether the newly inputted characteristic data is a harmful image or video, so that the degree of harmfulness is analyzed do.

As a result of the harmfulness analysis, if the harmfulness exceeds the harmfulness threshold (Y in S315), the harmful media external detector 202 transmits the result of analyzing the harmfulness of the characteristic data to the web content analysis unit 104 to the web content analysis unit ( 104), the web content analysis unit 104 determines that the website related to the notified result is a harmful site, and generates harmful analysis information related thereto (S320).

When the content type is text, the harmful analysis information includes the address of the website determined to be harmful, characteristic data of the searched website, search data, object collection information, topic of text calculated according to the harmfulness analysis, characteristic data, and At least one of a related word and a context result value between words in the text may be further included.

When the content type is video data or image data, the harmful analysis information includes the corresponding website address and characteristic data, as well as search data, object collection information, topic, characteristics of video data and/or image data calculated from harmful analysis. It may further include one or more from at least a portion of data constituting a characteristic element and object information related to data and estimated as a harmful shape.

At least one of the web content analysis unit 104 and the harmful site database unit 110 manages keywords, words, and characteristic elements estimated as harmful shapes extracted in relation to the characteristic data. The managed feature data is provided to the crawling unit 102 to be used as search data during a periodic search of a website.

Data constituting the object information may be at least one of video data and image data extracted from the website by the crawling unit 102 . The data of object information is stored and learned in the harmful site database unit 112 , so that it can be used to determine characteristic data and harmfulness level when analyzing video and image data of a website that is input later.

The harmful analysis information is not limited to the address and characteristic data of the harmful site, and by further including at least any one of the above-mentioned other data, it is possible to extract the object information and characteristic data of the newly input website and to analyze the harmfulness level. It can be used to derive more accurate results.

On the other hand, if the harmfulness analysis result does not exceed the harmfulness threshold (N in S315), it is determined that the website is not a harmful site.

Next, referring to FIG. 4 , when harmful analysis information is generated (Y in S405), the update unit 108 reads object information determined to be a harmful site and harmful analysis information (S410).

Next, the updater 108 converts the object information and the harmful analysis information into an ontology form (S415).

In order to easily share and link data collected through the web, when the data stored in the harmful site database unit 112 is stored and managed in the form of knowledge information, the update unit 108 is configured to match the form of knowledge information; Converts related information into ontology form. Accordingly, it is possible to easily provide an external service through the web in a standardized method for related information on harmful sites stored in the harmful site database unit 112 .

In detail about the ontology, it may be data using a semantic web technology. The Semantic Web is an ontology in which information about resources (web documents, various files, services, etc.) and relation-semantic information between resources in a distributed environment such as the current Internet can be processed by machines, that is, computers. It is a framework and technology that expresses it in a form and allows an automated machine to process it.

An ontology is a model that abstracts and shares what people think about things, and refers to a technology in which the types of concepts and constraints on usage are explicitly defined. In the field of computer science, an ontology is a data model that expresses a specific domain, and is defined as a set of formal vocabulary describing concepts belonging to a specific domain and relationships between concepts. In particular, Ontology is a tool that can implement the Semantic Web, and is used as a tool that can semantically connect knowledge concepts. give.

The semantic web may be expressed as a semantic markup language based on XML (Extensible Markup Language). In this semantic web, concepts are expressed in triple form of subject, predicate, and object, and each subject, predicate, and object can be expressed as XML Uniform Resource Identifier can RDF (Resource Description Framew), RDF-Schema, OWL (web ontology language), SW RL (Semantic Web Rule Language) proposed by W3C (World Wide Web Consortium), and TopicMaps proposed by ISO as a standard language for describing current semantic web ontology. etc.

Next, when the object information and harmful analysis information of the website are different from the information previously stored in the harmful site database unit 112 (N of S420), the update unit 108 may include keywords, image data, and /or the harmful site database unit 112 is updated with at least a part of the above-mentioned data of harmful analysis information, together with the video data and the address of the harmful website (S425).

Contrary to this, when the object information and harmful analysis information of the website are the same as the information previously stored in the harmful site database unit 112 (Y in S420), the update unit 108 performs the pre-stored information in order to prevent unnecessary processes. is maintained

According to this embodiment, it is possible to automatically collect and update the address of a harmful website without human intervention so as to provide useful information to an external service that determines and blocks harmful sites, for example, the harmful media blocking external service module 204 . and may collect object information, which is the media of harmful websites.

In addition, according to the present embodiment, even if the address of the harmful website is changed or the situation is changed, such as being closed, it is possible to periodically determine whether the harmful website is a harmful website and update the information on the harmful website to the latest.

Example methods of the present disclosure are expressed as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, other steps may be included in addition to the illustrated steps, steps may be excluded from some steps, and/or other steps may be included except for some steps.

Various embodiments of the present disclosure do not list all possible combinations, but are intended to describe representative aspects of the present disclosure, and the details described in various embodiments may be applied independently or in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure includes software or machine-executable instructions (eg, operating system, application, firmware, program, etc.) that cause an operation according to the method of various embodiments to be executed on a device or computer, and such software or and non-transitory computer-readable media in which instructions and the like are stored and executed on a device or computer.

100: Device for updating harmful website information
102: crawling unit 104: web content analysis unit
106: hazard detection connection 108: update unit
110: harmful site database unit

Claims (20)

A website is searched by a search engine based on search data related to characteristic data according to characteristic analysis of a harmful site previously stored in the harmful site database unit, and the contents of the website and the website address are retrieved from the searched website. a crawling unit that periodically and automatically acquires related object information;
By analyzing the characteristic data extracted from the object information, it is determined whether the calculated degree of harm exceeds a harmful threshold, and if it exceeds, the website is determined as a harmful site, and the address and characteristic data of the corresponding website are stored at least a web content analysis unit for generating harmful analysis information including; and
and an update unit for automatically updating the harmful site database unit with the harmful analysis information.
The method of claim 1,
The content is composed of at least one of text, video data, and image data, and the search data is at least one of keywords, images, and videos constituting the pre-stored characteristic data of the harmful website.
3. The method of claim 2,
In the case of the text, the characteristic data is a word extracted by referring to the harmful word stored in the harmful site database unit as well as the keyword in the case of the text. Update device of site information.
4. The method of claim 3,
In the case of the text, the web content analysis unit uses a word2vec, doc2vec, or LDA (Latent Dirichlet Allocation) topic analysis algorithm based on the extracted word and the keyword constituting the feature data extracted from the object information to determine the degree of harm An update device of harmful website information that produces
4. The method of claim 3,
In the case of the video data and the image data, the web content analysis unit updates harmful website information for calculating the degree of harm by using machine learning or deep learning on the feature element of the feature data extracted from the object information Device.
The method of claim 1,
and wherein the crawling unit obtains object collection information including metadata related to the object information and classifies the object collection information for each website.
7. The method of claim 6,
When the content is text, the harmful analysis information further includes at least one of the object collection information, topic, and context result value between words of text, and when the content is video data or image data For example, the harmful analysis information may further include one or more of the object collection information, topics, and data constituting the object information.
4. The method of claim 3,
An apparatus for updating harmful website information in which keywords, words, and characteristic elements estimated as harmful shapes extracted in relation to the characteristic data are managed by at least one of the web content analysis unit and the harmful site database unit.
The method of claim 1,
The web content analysis unit is connected to a harmful media external detector configured as an external module through a harmfulness detection connection unit, and whether the harmful media external detector analyzes the characteristic data extracted from the object information and the degree of harmfulness calculated by the harmfulness threshold exceeds the harmfulness threshold A device for updating harmful website information for determining and notifying the determination result to the web content analysis unit.
The method of claim 1,
The update unit expresses harmful analysis information including the address of the website determined as the harmful site and the search data as data using an ontology and stores the harmful website information in the harmful site database unit. .
The method of claim 1,
The update unit updates the harmful website information to the harmful analysis information when the object information and the harmful analysis information of the website determined to be the harmful site are different from the information previously stored in the harmful site database unit.
A website is searched by a search engine based on search data related to characteristic data according to characteristic analysis of a harmful site previously stored in the harmful site database unit, and the contents of the website and the website address are retrieved from the searched website. a crawling step of periodically automatically acquiring related object information;
By analyzing the characteristic data extracted from the object information, it is determined whether the calculated degree of harm exceeds a harmful threshold, and if it exceeds, the website is determined as a harmful site, and the address and characteristic data of the corresponding website are stored at least Web content analysis step of generating harmful analysis information including; and
and an update step of automatically updating the harmful site database unit with the harmful analysis information.
13. The method of claim 12,
The content is composed of at least one of text, video data, and image data, and the search data is at least one of keywords, images, and videos constituting the previously stored characteristic data of the harmful site.
14. The method of claim 13,
In the case of the text, the characteristic data is a word extracted by referring to the harmful word stored in the harmful site database unit as well as the keyword in the case of the text. How to update site information.
15. The method of claim 14,
In the case of the text, the web content analysis step uses a word2vec, doc2vec, or LDA (Latent Dirichlet Allocation) topic analysis algorithm based on the extracted word and the keyword constituting the feature data extracted from the object information in the case of the text. Update of harmful website information that calculates a degree and calculates the degree of harm using machine learning or deep learning for the feature element of the feature data extracted from the object information in the case of the video data and the image data method.
13. The method of claim 12,
The crawling step further includes obtaining object collection information including metadata related to the object information and classifying the object collection information for each website.
17. The method of claim 16,
When the content type is text, the harmful analysis information further includes at least one of the object collection information, a topic, and a context result value between words of text, wherein the content type is video data or image data , the harmful analysis information further includes one or more of the object collection information, the topic, and data constituting the object information.
13. The method of claim 12,
In the web content analysis step, the harmful website information is updated by analyzing the characteristic data extracted from the object information by a harmful media external detector composed of an external module and determining whether the calculated degree of harm exceeds a harmful threshold. method.
13. The method of claim 12,
In the updating step, harmful website information including the address of the website determined as the harmful site and the search data is expressed as data using an ontology and stored in the harmful website database unit. method.
13. The method of claim 12,
In the updating step, when the object information and the harmful analysis information of the website determined as the harmful site are different from the information previously stored in the harmful site database unit, the harmful website information is updated with the harmful website information updating method.

Images