KR20220093637A - Method for obtaining automatically user identification object in web page - Google Patents

Abstract

Disclosed is a method for identifying an object of a webpage. The method for identifying the object of the webpage according to one embodiment of the present specification selects a specific DOM object among a DOM tree of the document object model (DOM) of a specific webpage, extracts a feature value according to predetermined criteria from the selected DOM object, and may prepare the learning data to be applied to a learning target classifier model. Therefore, the present invention enables to more efficiently recognize an object desired by a user in an arbitrary webpage whose structure is unknown through the learned classifier.

Description

How to automatically obtain a user identification object from a web page {METHOD FOR OBTAINING AUTOMATICALLY USER IDENTIFICATION OBJECT IN WEB PAGE}

This specification relates to a method for automatically obtaining a user identification object in a web page.

Although web pages are generally constructed according to the HTML standard, content and design layouts, etc. posted on the actual web pages may be very diverse. Therefore, it is common that content having a specific subject among various contents provided on a web page is also posted in various forms, such as location and shape, rather than being posted according to a specific standard.

Therefore, in order to automatically identify a specific item within a plurality of web pages having various structures, there is a need for a unified standard for automatic identification. is difficult in reality. Also, it is very inefficient to create and manage explicit rules for efficient identification of specific items. Accordingly, there is a need for a method for more efficiently obtaining an object desired by a user in a web page.

An object of the present specification is to solve the above problems, and an object of the present specification is to provide a method for identifying an object of a web page based on structural and morphological common characteristics of specific content or objects.

In addition, the present specification provides an object identification method of a web page that can more efficiently identify a user who posted content in a web page by objectifying each element of the HTML document of the web page and automatically controlling the HMTL tag aim to do

In addition, the present specification aims to train a machine learning-based classifier by acquiring training data from a document object model (DOM) to dynamically control HTML tags.

The technical problems to be achieved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned are clear to those of ordinary skill in the art to which the present invention belongs from the detailed description of the invention below. can be understood clearly.

A method for identifying an object of a web page according to an embodiment of the present specification includes parsing HTML of the web page; generating a DOM Tree of a document object model (DOM) corresponding to the web page; selecting at least one DOM object corresponding to the lowest node in the dome tree; extracting a feature value from the selected dome object according to a predetermined criterion; and learning a classifier by setting the extracted feature values and labels as input values to a learning target model, wherein the classifier is learned to automatically identify a specific object included in the web page .

The specific object may include a user identification object.

The selecting of the at least one dome object may include selecting an object including text from the dome tree.

The predetermined criterion includes at least one of a text feature included in the selected dome object, an attribute of an HTML tag, or similarity of a hierarchical structure of a plurality of dome objects in the dome tree, and the step of extracting the feature value includes: , generating a feature vector according to the at least one predetermined criterion.

The text characteristic includes at least one of a length of text, whether special characters are included, or whether a space is used, and determines whether the dome object is used for displaying the specific object according to the characteristic of the text. can be determined.

The HTML tag includes at least one of a class, href, and user-defined tag, and displays the specific object according to whether an attribute of the HTML tag includes user identification information. The feature vector can be determined by determining whether it is used.

The similarity of the hierarchical structure of the plurality of dome objects is determined by the similarity of the hierarchical structure when the length of the path to the lowest common ancestor in the dome tree and the sequence of tags located in the path are the same. , and the feature vector may be determined according to the similarity of the hierarchical structure.

The classifier may include a machine learning-based binary classifier.

The method for identifying the object of the web page further includes: storing the learned classifier model in a memory; collecting a first web page; extracting the feature value with respect to the first web page according to the predetermined criterion; and setting the feature value as an input of the learned classifier model, and automatically identifying the specific object according to an output result of the learned classifier model.

A computer program according to another embodiment of the present specification includes a plurality of instructions executed by one or more processors, the computer program comprising: instructions to parse HTML of a web page; a command for generating a DOM tree of a document object model (DOM) corresponding to the web page and selecting at least one DOM object corresponding to the lowest node among the dome tree; a command to generate a feature vector of the selected dome object based on at least one of a text feature included in the selected dome object, an attribute of an HTML tag, or a similarity of a hierarchical structure of a plurality of dome objects in the dome tree; and a command to train a classifier by setting the extracted feature vector and label as input values to a learning target model, wherein the classifier learns to automatically identify a specific object included in the web page do.

An apparatus according to another embodiment of the present specification includes: a data learning unit for controlling data learning by acquiring learning data for automatically identifying a user identification object on a web page; and a memory for storing the learned model, wherein the data learning unit parses the HTML of the web page, generates a DOM Tree of a document object model (DOM) corresponding to the web page, and the a learning data acquisition unit that selects at least one DOM object corresponding to the lowest node in the dome tree and extracts a feature vector of the selected dome object according to a predetermined criterion; and a model learning unit for learning a classifier by setting the extracted feature vector and label as input values to a learning target model, wherein the classifier automatically selects the user identification object included in the web page. learn to identify.

The learning data acquisition unit may extract the feature vector based on at least one of a text feature included in the selected dome object, an attribute of an HTML tag, or a similarity of a hierarchical structure of a plurality of dome objects in the dome tree.

In the present specification, a method for identifying an object of a web page according to an embodiment may more efficiently identify an object desired by a user within a web page based on structural and morphological common characteristics of specific content or objects.

In addition, the object identification method of a web page according to an embodiment of the present specification objectifies each element of the HTML document of the web page and automatically controls the HMTL tag, so that the user who has posted the content in the web page can be identified more efficiently can be identified.

In addition, according to an embodiment of the present specification, by acquiring learning data from a document object model (DOM) to dynamically control HTML tags, and learning a machine learning-based classifier, users who posted content on a web page can be identified more efficiently.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. .

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to help the understanding of the present specification, provide embodiments of the present specification, and together with the detailed description, explain the technical features of the present specification.
1 is a block diagram of an apparatus according to an embodiment of the present specification.
2 is a diagram for explaining in more detail a learning step and a data processing step for implementing the user identification object method in a web page according to an embodiment of the present specification.
3 is a flowchart of a method for identifying an object of a web page according to an embodiment of the present specification.
4 is an example of a HTML parsing result of a web page to obtain learning data according to an embodiment of the present specification.
5 is a diagram for exemplarily explaining a DOM tree generated for acquiring learning data according to an embodiment of the present specification.
6 is a diagram for explaining a process of learning a classifier based on acquired learning data according to an embodiment of the present specification.
7 is a flowchart for describing S330 of FIG. 3 in more detail.
8 to 9 are diagrams for exemplarily explaining a process of analyzing a DOM object to obtain learning data according to an embodiment of the present specification.
FIG. 10 is a diagram for exemplarily explaining a process of acquiring a feature vector of a DOM object required for classifier learning according to an embodiment of the present specification.
BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as a part of the detailed description to facilitate the understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical features of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

1 is a block diagram of an apparatus according to an embodiment of the present specification.

The method for identifying an object of a web page disclosed herein may be implemented in a computer device. The computer device may include an AI processor 100 and a memory 150 . The computer device may include an electronic device including the AI processor 100 capable of performing AI processing, or a server including the AI processor 100 .

The AI processor 100 is a data learning unit 100, a memory for storing the learned model in order to learn a learning model that enables each method of starting the actual content to automatically identify a specific object from a plurality of web pages. At 150 , the data processing unit 140 may include a data processing unit 140 that processes data through a learning model with respect to a newly acquired web page after the model learning is finished.

The computer device is a device capable of learning a neural network, and may be implemented as various electronic devices such as a server, a desktop PC, a notebook PC, and a tablet PC.

The AI processor 100 may learn the neural network through the model learning unit 130 using a program stored in the memory 150 . In particular, the AI processor 100 may learn a neural network for recognizing a user who has posted content on a web page.

Here, a neural network for recognizing a user who has posted content on a web page may be designed to simulate a human brain structure on a computer, and a plurality of network nodes having weights that simulate neurons of a human neural network may include The plurality of network modes may transmit and receive data according to a connection relationship, respectively, so as to simulate a synaptic activity of a neuron in which a neuron sends and receives a signal through a synapse. Here, the neural network may include a deep learning model developed from a neural network model. In a deep learning model, a plurality of network nodes can exchange data according to a convolutional connection relationship while being located in different layers. Examples of neural network models include deep neural networks (DNN), convolutional deep neural networks (CNN), Recurrent Boltzmann Machine (RNN), Restricted Boltzmann Machine (RBM), deep trust It includes various deep learning techniques such as neural networks (DBN, deep belief networks) and deep Q-networks, and can be applied to fields such as computer vision, speech recognition, natural language processing, and voice/signal processing.

Meanwhile, the processor performing the above-described function may be a general-purpose processor (eg, CPU), but may be an AI-only processor (eg, GPU) for artificial intelligence learning.

The memory 150 may store various programs and data necessary for the operation of the AI processor 100 . The memory 150 may be implemented as a non-volatile memory, a volatile memory, a flash-memory, a hard disk drive (HDD), or a solid state drive (SDD). The memory 150 is accessed by the AI processor 100 , and reading/writing/modification/deletion/update of data by the AI processor 100 may be performed. Also, the memory 150 may store a neural network model (eg, an artificial neural network model, a classification model, etc.) generated through a learning algorithm for data classification/recognition according to an embodiment of the present specification.

Meanwhile, the AI processor 100 may include a data learning unit 120 that learns a neural network for data classification/recognition.

The data learning unit 110 may learn a criterion regarding which training data to use to determine data classification/recognition and how to classify and recognize data using the training data. The data learning unit 110 may learn the deep learning model by acquiring learning data to be used for learning and applying the acquired learning data to the deep learning model.

The data learning unit 110 may be manufactured in the form of at least one hardware chip and mounted on the computer device and/or the AI processor 100 . For example, the data learning unit 110 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics-only processor (GPU) for a computer device and/or AI It may be mounted on the processor 100 .

Also, the data learning unit 110 may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a computer-readable non-transitory computer readable medium. In this case, the at least one software module may be provided by an operating system (OS) or may be provided by an application.

The data learning unit 110 may include a training data acquiring unit 120 and a model learning unit 130 .

The training data acquisition unit 120 may acquire training data required for a neural network model for classifying and recognizing data. For example, the training data acquisition unit 120 may acquire, as training data, a feature vector and/or labeling data (label) of a DOM object to be input to the neural network model.

The model learning unit 130 may use the acquired training data to learn the neural network model to have a criterion for determining how to classify predetermined data. In this case, the model learning unit 130 may train the neural network model through supervised learning using at least a portion of the training data as a criterion for determination. Alternatively, the model learning unit 130 may learn the neural network model through unsupervised learning for discovering a judgment criterion by self-learning using learning data without guidance. Also, the model learning unit 24 may train the neural network model through reinforcement learning using feedback on whether the result of the situation determination according to the learning is correct. In addition, the model learning unit 130 may A neural network model can be trained using a learning algorithm including error back-propagation or gradient decent.

When the neural network model is learned, the model learning unit 130 may store the learned neural network model in the memory 150 . The model learning unit 130 may store the learned neural network model in the memory of the server connected to the AI device through a wired or wireless network.

The data learning unit 110 further includes a training data preprocessor (not shown) and a training data selection unit (not shown) to improve the analysis result of the recognition model or to save resources or time required for generating the recognition model You may.

The learning data preprocessor may preprocess the acquired data so that the acquired data can be used for learning for situation determination. For example, the training data preprocessor parses the HTML of the web page for the model learning unit 130 to learn to recognize the user identification object in the web page, defines the contents in the HTML document (model configuration), and each element You can see the hierarchical structure of the Document Object Model (DOM) that specifies how to access the .

In addition, the training data selection unit may select data necessary for learning from among the training data acquired by the training data acquisition unit 120 or the training data preprocessed by the preprocessing unit. The selected training data is to be provided to the model learning unit 130 . can For example, the learning data selector may select only data for a desired object as the learning data by selecting a specific dome object from the DOM tree. A process of selecting a specific dome object from among a plurality of dome objects constituting the dome tree will be described in more detail with reference to FIG. 5 .

The operation of the learning data preprocessor and/or the learning data selection unit may be processed separately from the learning data acquisition unit 120 illustrated in FIG. 1 , but may be implemented by being integrated into the training data acquisition unit 120 .

In addition, the data learning unit 110 may further include a model evaluation unit (not shown) in order to improve the analysis result of the neural network model.

The model evaluator may input evaluation data to the neural network model and, when an analysis result output from the evaluation data does not satisfy a predetermined criterion, cause the model learning unit 130 to learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. As an example, the model evaluation unit may evaluate as not satisfying a predetermined criterion when, among the analysis results of the learned recognition model for the evaluation data, the number or ratio of evaluation data for which the analysis result is not accurate exceeds a preset threshold. have. According to an embodiment of the present specification, a learning model is defined according to a result of comparing a label corresponding to the DOM object with a model output value as a result of defining a feature vector extracted from a DOM object as learning data and inputting it to the learning target classifier model. can be evaluated

The data processing unit 140 applies the input data to the learning model (classifier) stored in the memory to automatically recognize the user identification object included in the specific web page.

Meanwhile, although not shown, the computer device shown in FIG. 1 may further include a communication unit (not shown). The communication unit may transmit the AI processing result by the AI processor 100 to an external electronic device (eg, a network server, etc.).

Meanwhile, the above-described classifier learning process may be implemented in a server distinct from the client terminal. For example, only a plurality of web page information to be learned may be collected through the client terminal, and the web page information may be transmitted to the server. The server may generate a learning model through the above-described learning process based on the web page information. Also, even after the building of the learning model is completed, after new web page information collected from the client terminal is transmitted to the server, only the results applied to the classifier model previously learned by the server may be transmitted to the client terminal.

On the other hand, the client terminal performs only at least some operations for building learning data, such as a preprocessing process for building learning data, for example, HTML parsing of a web page, and creating a dome tree, and performing model learning in the server. can In some cases, at least some operations necessary for the learning may be executed together through an edge computing environment between the client terminal and the server for resource distribution according to the construction of the learning model.

On the other hand, although the computing device shown in FIG. 1 has been functionally described as an AI processor 100, a memory 150, a communication unit (not shown), etc., the above-described components are integrated into one module and are referred to as an AI module. It should be pointed out that it may be

2 is a diagram for explaining in more detail a learning step and a data processing step for implementing the user identification object method in a web page according to an embodiment of the present specification.

Referring to FIG. 2 , the method for identifying an object of a web page according to an embodiment of the present specification may be largely divided into (a) a learning step and a prediction step after learning is completed.

In the learning step (a), a label 201 and input data 205 may be provided for supervised learning. The input data 205 may be web page information. More specifically, the AI processor (refer to FIG. 1 , hereinafter referred to as a processor) may acquire learning data through the learning data acquisition unit 120 of FIG. 1 . More specifically, the learning data acquisition unit 120 may include an HTML parser 121 , a dome object extraction unit 123 , and a feature extraction unit 125 .

The HTML parser 121 reads the URL address of a specific web page, and parses the content area of the response into HTML through the HTML parser. The dome object extraction unit 123 may store the parsed HTML as a dome object. Python BeautifulSoup may be applied to the HTML parser, but the present specification is not limited thereto.

The Document Object Model (hereinafter DOM) is a programming interface for HTML and XML documents. The DOM provides a structured representation of the document and provides a way for programming languages to access the DOM structure, allowing them to change the document structure, style, content, etc. DOM represents documents as objects with structured modes, properties, and methods. The DOM is responsible for linking web pages so that they can be used in scripts or programming languages.

A web page is a kind of document. The document is interpreted through a web browser and displayed on the web browser screen, or it appears as an HTML source itself. It may appear in different forms using the same document. The DOM provides a way to represent, store, and manipulate the same document. The DOM is an object-oriented representation of a web page, and the DOM can be modified using a scripting language such as JavaScript.

The W3C DOM and WHATWG DOM standards are the standards for implementing DOM in most browsers. Since many browsers provide additional functions in addition to the functions provided by the standard protocol, user-written documents can operate in various browser environments with different DOMs applied.

Meanwhile, the dome object extraction unit 123 may create the dome objects in a tree structure according to the inclusion relationship of the HTML tag. Each branch of the dome tree ends with a node, and each node has an object.

According to an embodiment of the present specification, at least one object including text among the dome objects included in the created dome tree may be selected as the learning data. At least one of the selected at least one object may be at least one dome object including user identification information. In addition, the feature extractors 125 and 210 may extract a feature value to be defined as an input value of the learning model from the selected dome object. The feature value may be determined according to whether a specific area of the web page includes information for identifying the user, such as a user name.

The data learning unit 110 may express the extracted feature value as a feature vector 220 .

The model learning unit 130 may learn through a machine learning algorithm by inputting the feature vector 220 and the pre-prepared label 201 to the learning target classification model 230 .

The learned classification model 260 may be stored in the memory, and when a new web page 240 is designated as input data, the feature extractor 210 performs a feature vector 220 from each dome object in the new web page. is extracted and set as an input to the learned classification model 260 . According to the output result of the learned classification model 260, it may be determined whether a user identification object is included in each dome object in the new web page. According to the embodiment of the present specification, the output of the learned classification model 260 may output any one result 270 of True and False.

Hereinafter, a pre-processing process for the training data will be described in more detail with reference to FIGS. 3 to 5 .

3 is a flowchart of a method for identifying an object of a web page according to an embodiment of the present specification. The method illustrated in FIG. 3 may be implemented through the computer device illustrated in FIGS. 1 and/or 2 .

Referring to FIG. 3 , the processor may parse the HTML of the web page to be used for training the classifier model ( 300 ). The processor constructs a document object model (DOM) of HTML and checks a hierarchical structure between DOM objects constituting the dome tree ( 310 ).

Referring to FIG. 4 , the web page to be used for learning may be a web page in the form of an online forum where different users can upload content or post an article. For example, in this specification, when the same user creates a forum on different web pages, when a post is posted, each user tracks a post, or when cross-analysis is required, a user identification object is created in an arbitrary web page whose structure is unknown. It has the advantage of being automatically recognized. 4 discloses an interface configured so that a plurality of users having different IDs can post different articles or create different meetings. The processor 300 parses the HTML of the web page and constructs a dome tree of the parsed HTML.

5, all nodes of the dome tree have a mutual hierarchical relationship, for example, a top-level root node, a parent node, an ancestor node, a child node, Hierarchical relationships can be defined with descendant nodes and sibling nodes, and nodes in the dome tree are document mode, element node, attribute node, and text. It can be composed of four types of nodes (text nodes).

The root node existing at the top of the dome tree is a document node, meaning the entire web page (HTML). Since it is a top-level node, parent and ancestor nodes do not exist. And, all HTML elements correspond to element nodes (eg, head, title, body, main, h1) in the dome tree. And an element node can have a unique attribute node (eg, class) in the dome tree. Also, the attribute specified in the HTML Element corresponds to an attribute node in the dome tree. The attribute may correspond to a part of the corresponding element node, not the concept of a child node of the specified element node. And, all texts in the HTML Element An element corresponds to a text node in the dome tree, and it is common to exist at the bottom of the dome tree.

Referring back to FIG. 3 , the processor may select a specific dome object in order to obtain training data applied to the learning object classifier within the dome tree ( S320 ).

According to an embodiment of the present specification, it may be determined whether user identification information is included based on the properties of the text included in the dome tree and the properties of the tags included in the dome object. As described above, the specific dome object is a dome object that can be estimated to include user identification information, and may correspond to a text node or an attribute node. The user identification information may include a tag for identifying the user. For example, tags such as user, author, poster, member, name, id, and profile may be used to indicate a user. In addition, text such as a user name, nickname, or ID directly indicating a user may be included along with the tag.

Accordingly, the processor may select the lowest dome objects in the dome tree. For example, the processor may select a plurality of dome objects 510 , 521 , 522 , 523 , and 524 corresponding to the lowest node in the dome tree of FIG. 5 .

The processor may extract a feature value from the selected dome object (S330), and train a classifier based on the extracted feature value and label (S340).

6 is a view for explaining a process of learning a classifier based on the acquired learning data according to an embodiment of the present specification.

Referring to FIG. 6 , the processor extracts features for each of a plurality of dome objects selected from the dome tree. The feature of the dome object may be expressed as a feature vector. The processor can train the classifier by setting the extracted characteristics of each dome object as input values to the machine learning-based binary classifier model together with the label (True or False) corresponding to each dome object. The above-described binary classifier is an example, and the present specification is not limited thereto, and any classification method requiring specific object recognition may be applied to a web page.

7 is a flowchart for describing S330 of FIG. 3 in more detail. 8 to 9 are diagrams for exemplarily explaining a process of analyzing a DOM object to obtain learning data according to an embodiment of the present specification. Hereinafter, it will be described with reference to FIGS. 7 to 9 together.

As described above, an embodiment of the present specification may be based on text characteristics and attributes of HTML tags in the DOM tree in the process of selecting learning data.

Referring to FIG. 7 , the processor analyzes the dome object ( S700 ) and configures a feature vector of each dome object ( 710 ). Thereafter, the feature vector and the label may be input to the learner to learn ( S720 ).

The processor may analyze the characteristics of the text included in the dome object (S701). For example, the characteristics of text existing in the dome object are characteristics of user identification information. The length of the string does not exceed a certain number of characters, the point that special characters are not used in the ID, etc. In the case of ID, spaces are not used. Considering that it is not generally applied, the processor may check the string length of the text in the dome object, whether special characters are included, whether there is a space, and the like.

In addition, the processor may analyze the attribute of the HTML tag (S703). The HTML tag includes at least one of a class, href, and user-defined tag, and displays the specific object according to whether an attribute of the HTML tag includes user identification information. You can decide which one to use.

In addition, the processor may analyze the user-defined attribute (S703). It may include text information corresponding to the attribute id tag defined by the user.

Also, the processor may analyze the hierarchical structure of the dome object (S707). When the hierarchical structure of the dome object has the same length of a path to the lowest common ancestor in the dome tree and the same sequence of tags located in the path, it may be determined that the hierarchical similarity exists.

Referring to FIG. 8 , a dome object 810 corresponding to a user ID OB1 in HTML corresponding to a specific region on a web page screen is analyzed. The processor may determine that the text node "member" 820 among the text nodes of href corresponding to the hyperlink attribute of the HTML tag can recognize the user, and extract a feature vector. Referring to FIG. 10 , the feature vector extracted through the analysis of the dome object may be configured in the form of True(0) or False(1) to correspond to the result of the binary classifier. The vector extracted as a result of analyzing the href tag among HTML tags has a value of "0".

And, as the class attribute of the HTML tag "username" 830 is found, the processor may also extract "0" as a feature vector according to the class attribute of the dome object.

In addition, when a user-identifiable keyword such as “user, id” is included in the user-defined attribute “data-user-id” 840, the processor similarly sets “0” as a feature vector according to the user-defined attribute among the dome objects. can be extracted.

In addition, when the character included in the text node located at the lowest node of the dome tree is "member" 850, and a user identifiable keyword is included, the processor can extract a feature vector corresponding to the text node as "0". have. In addition, the processor extracts "0" as a feature vector when the length of the string of "Pofecker" (841) among the text nodes does not exceed the predetermined number of characters, consists of numbers or characters without special characters, and there is no space. can

Meanwhile, when it is determined that the dome objects selected as training data in the dome tree correspond to sibling dome objects in the dome tree, the processor may extract a feature vector of each dome object as “0”.

Referring to FIG. 9 , in the dome tree, a dome object 1 900a, a dome object 2 900b, and a dome object 3 900c have primary upper nodes 910a, 910b, and 910c, respectively. In addition, each of the primary upper nodes has secondary upper nodes 920a , 920b , and 920c , and each of the secondary upper nodes has the same tertiary upper node 930 . In this case, the processor may recognize the dome objects 1, 2, and 3 as a sibling relationship. The sibling DOM object relationship generally has the same type of HTML tag and attribute value, and further, the path length to the lowest common ancestor and the tag sequence on the path may be the same.

According to the DOM object analysis process illustrated in FIGS. 8 to 9 , the processor as shown in FIG. 10 , the text length, whether special characters are included, whether spaces are used, HTML attributes (id, href, class), existence of sibling nodes The final feature vector can be extracted by composing whether or not it is a feature element of the dome object, and extracting and connecting the feature values of each feature element.

In the present specification, the feature vector shown in FIG. 10 is exemplary, and it goes without saying that the order of the feature elements constituting the feature vector and the type of feature elements may be variously modified.

Meanwhile, according to an embodiment of the present specification, as shown in FIG. 2 , in the process of learning a classifier based on a machine learning algorithm, a feature vector corresponding to at least one feature element extracted from the dome object is extracted from the learning data. The results may be used as training data, but the present specification is not limited thereto. For example, after selecting a dome object corresponding to a text node or an attribute node in the dome tree, a result of encoding text included in the selected dome object may be set as an input of the learning target classifier. However, in this case, the amount of training data increases compared to the above-described examples, and the number of labels increases accordingly.

On the other hand, when the result of first extracting the feature vector from the dome object is set as the training data before inputting the training data into the learning target classifier, the accuracy of the learning model may be improved. Furthermore, since the training data itself is primarily data that has undergone a preprocessing process due to feature extraction, there is an advantage in that the amount of training data and the number of labels required for training the final classifier model can be reduced.

According to the exemplary embodiments of the present specification, it is possible to more efficiently track a post written by a specific user in a web page in which a plurality of users register a post. In addition, through cross-analysis, it is easier to check whether postings are registered on different web pages by the same user.

The present invention described above can be implemented as computer-readable codes on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that is implemented in the form of. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (12)

parsing the HTML of the web page;
generating a DOM Tree of a document object model (DOM) corresponding to the web page;
selecting at least one DOM object corresponding to the lowest node in the dome tree;
extracting a feature value from the selected dome object according to a predetermined criterion; and
Including; setting the extracted feature values and labels as input values to the learning target model to learn a classifier;
The classifier is an object identification method of a web page, characterized in that it is learned to automatically identify a specific object included in the web page.
The method of claim 1,
The specific object is an object identification method of a web page, characterized in that it includes a user identification object.
The method of claim 1,
Selecting the at least one dome object comprises:
The object identification method of a web page, characterized in that selecting an object including text from the dome tree.
The method of claim 1,
The predetermined criterion is
including at least one of a text feature included in the selected dome object, an attribute of an HTML tag, or similarity of a hierarchical structure of a plurality of dome objects in the dome tree,
The extracting of the feature value comprises generating a feature vector according to the at least one predetermined criterion.
5. The method of claim 4,
The text feature is
At least one of the length of the text, whether special characters are included, or whether spaces are included,
The object identification method of a web page, characterized in that the feature vector is determined by determining whether the dome object is used for displaying the specific object according to the text feature.
5. The method of claim 4,
The HTML tag is
Contains at least one of a class (class), href, user-defined (User-defined) tag,
The method for identifying an object of a web page, wherein the feature vector is determined by determining whether the attribute of the HTML tag is used for displaying the specific object according to whether the user identification information is included.
5. The method of claim 4,
The similarity of the hierarchical structure of the plurality of dome objects is,
If the length of the path to the lowest common ancestor in the dome tree and the sequence of tags located in the path are the same, it is determined that the hierarchical similarity exists,
The method of identifying an object of a web page, characterized in that the feature vector is determined according to the similarity of the hierarchical structure.
The method of claim 1,
The classifier is
An object identification method of a web page, characterized in that it is a machine learning-based binary classifier.
The method of claim 1,
Storing the learned classifier model in a memory; further comprising,
collecting a first web page;
extracting the feature value with respect to the first web page according to the predetermined criterion; and
setting the feature value as an input of the learned classifier model and automatically identifying the specific object according to an output result of the learned classifier model;
Object identification method of a web page, characterized in that it further comprises.
A computer program stored on a computer-readable medium comprising a plurality of instructions executed by one or more processors, the computer program comprising:
The computer program is
a command to parse the HTML of a web page;
a command for generating a DOM tree of a document object model (DOM) corresponding to the web page and selecting at least one DOM object corresponding to the lowest node among the dome tree;
a command to generate a feature vector of the selected dome object based on at least one of a text feature included in the selected dome object, an attribute of an HTML tag, or a similarity of a hierarchical structure of a plurality of dome objects in the dome tree;
Including; a command to learn a classifier by setting the extracted feature vector and label as input values to the learning target model;
and the classifier is learned to automatically identify a specific object included in the web page.
a data learning unit for controlling data learning by acquiring learning data for automatically identifying a user identification object on a web page; and
A memory for storing the trained model; includes,
The data learning unit,
Parsing the HTML of the web page, generating a DOM tree of a document object model (DOM) corresponding to the web page, and generating at least one DOM object corresponding to the lowest node among the dome trees. a learning data acquisition unit for selecting and extracting a feature vector of the selected dome object according to a predetermined criterion; and
a model learning unit configured to learn a classifier by setting the extracted feature vector and label as input values to a learning target model; and
The classifier is characterized in that it is learned to automatically identify the user identification object included in the web page.
12. The method of claim 11,
The learning data acquisition unit,
and extracting the feature vector based on at least one of a text feature included in the selected dome object, an attribute of an HTML tag, or similarity of a hierarchical structure of a plurality of dome objects in the dome tree.

Images