KR20210051175A - Method, system, and computer program for dedecting multimodal abusing pattern to select document - Google Patents

Abstract

A method and a system for detecting a multimodal abusing pattern for document screening are disclosed. The abusing pattern detecting method comprises the steps of: detecting a burstiness pattern representing a repeating pattern of a document based on a multimodal feature between documents; and screening an original document by using the burstiness pattern.

Description

Method, system, and computer program for detecting multimodal abusing patterns for document selection {METHOD, SYSTEM, AND COMPUTER PROGRAM FOR DEDECTING MULTIMODAL ABUSING PATTERN TO SELECT DOCUMENT}

The following description relates to a technique for detecting an abusing pattern document.

Due to the commercialization of the Internet, many users can search for information on various topics using the Internet. However, it takes time and effort to find the optimal information desired by the user in the vast amount of information.

Recently, search engines have introduced various search methods so that users can find the optimal information they want. One of them is a document intention-based ranking algorithm that allows documents with rich experiences or opinions or documents with high information to be exposed to the top of search results.

On the other hand, there is an increasing problem of abusing in which similar documents are intensively produced to occupy the ranking of documents for advertisements or malicious purposes.

A lot of research is being conducted on technologies for solving such abusing problems. For example, Korean Patent Publication No. 10-2012-0005797 (published on January 17, 2012) prevents the registration or exposure of documents determined to be filtered. The technology is disclosed.

Provides a method and system for selecting original documents based on the detected patterns by detecting multimodal abusing patterns for documents produced for a certain period of time.

A method and system capable of selecting an abusing document intentionally produced through these patterns by detecting the abusing characteristics of a document pattern and network information between document authors are provided.

A method for detecting an abusing pattern executed in a computer system, wherein the computer system includes at least one processor configured to execute computer-readable instructions contained in a memory, and the method for detecting an abusing pattern is performed by the at least one processor. And detecting an explosive pattern representing a repetition pattern of the document based on multimodal features between documents; And selecting an original document using the explosive pattern by the at least one processor.

According to an aspect, the detecting may include detecting the explosive pattern appearing in the document set for a document set within a time window.

According to another aspect, in the detecting, the explosive pattern appearing in the document set is determined using at least one of a word, a link, a title, a body, and an image included in each document for a document set in a time window. It may include the step of sensing.

According to another aspect, the detecting may include detecting the explosive pattern by using a weight of a word or expression selected on the basis of the frequency of occurrence of words in the document set by using a weight in each document.

According to another aspect, the detecting may include measuring a plurality of features representing the repetition pattern using content included in the document; And calculating an explosive score by summing the plurality of features.

According to another aspect, the detecting may include selecting a document set using the plurality of features; And learning the plurality of features based on the document set or adjusting weights for each feature.

According to another aspect, the detecting may include converting content included in the document into a numeric vector and then assigning a pattern ID corresponding to the numeric vector; And calculating an explosive score by scoring the number of pattern IDs in the time window by applying a time window to the pattern ID over time.

According to another aspect, the detecting may further include measuring a sharpness of the explosive pattern by adjusting the time window when the explosive score is greater than or equal to a predetermined level.

According to another aspect, the method of detecting an abusing pattern may further include generating, by the at least one processor, network information between a document in which the explosive pattern is detected and an author who has created the document.

According to another aspect, the detecting step includes converting content included in the document into a numeric vector and then assigning a pattern ID corresponding to the numeric vector, and the generating step comprises: the pattern ID And generating the network information by connecting a link between the author who wrote the document and the document.

According to another aspect, the selecting may include selecting at least one of an abusing document and an abuser using the network information together with the explosive pattern.

There is provided a computer program stored in a non-transitory computer-readable recording medium for executing the method of detecting an abusing pattern on the computer system.

It provides a non-transitory computer-readable recording medium in which a program for executing the method for detecting an abusing pattern on a computer is recorded.

A computer system comprising at least one processor configured to execute computer readable instructions contained in a memory, wherein the at least one processor generates an explosive pattern representing a repeating pattern of the document based on multimodal features between documents. A pattern detection unit to detect; And a document sorting unit for selecting original documents using the explosive pattern.

A computer system comprising at least one processor configured to execute computer readable instructions contained in a memory, wherein the at least one processor generates an explosive pattern representing a repeating pattern of the document based on multimodal features between documents. A pattern detection unit to detect; A network generator for generating network information between the document in which the explosive pattern is detected and the author who created the document; And a document sorting unit for selecting original documents using the explosive pattern and the network information.

According to embodiments of the present invention, a document having an abusing pattern and a document with originality may be selected based on the detected pattern by detecting a multimodal abusing pattern for a document produced for a certain period.

According to embodiments of the present invention, it is possible to improve document selection performance and document search quality by selecting abusing documents intentionally produced by using an abusing characteristic of a document pattern and network information between document authors.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention.
3 is a diagram illustrating an example of components that may be included in a processor of a server according to an embodiment of the present invention.
4 is a flowchart illustrating an example of a method that can be performed by a server according to an embodiment of the present invention.
5 is an exemplary diagram for explaining collective characteristics that may appear in a document according to an embodiment of the present invention.
6 to 7 are exemplary diagrams for explaining a process of expressing a document as a numeric vector in an embodiment of the present invention.
8 is an exemplary diagram for explaining a process of learning an abusing pattern using detailed features in an embodiment of the present invention.
9 is an exemplary diagram for explaining network information between a document and an author in an embodiment of the present invention.
10 to 13 are exemplary diagrams for explaining a process of selecting an abusing document and a normal document according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to a technique for selecting an abusing pattern document.

In the embodiments including those specifically disclosed in the present specification, it is possible to search for a repetitive abusing pattern without ingenuity, and to select an original document on the contrary, thereby improving document selection performance and document search quality.

1 is a diagram showing an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 shows an example including a plurality of electronic devices 110, 120, 130, and 140, a plurality of servers 150 and 160, and a network 170. 1 is an example for explaining the invention, and the number of electronic devices or servers is not limited as in FIG. 1.

The plurality of electronic devices 110, 120, 130, and 140 may be a fixed terminal implemented as a computer system or a mobile terminal. Examples of the plurality of electronic devices 110, 120, 130, and 140 include smart phones, mobile phones, navigation systems, computers, notebook computers, digital broadcasting terminals, personal digital assistants (PDAs), and portable multimedia players (PMPs). ), tablet PCs, game consoles, wearable devices, internet of things (IoT) devices, virtual reality (VR) devices, and augmented reality (AR) devices. As an example, in FIG. 1, the shape of a smartphone is shown as an example of the electronic device 110, but in the embodiments of the present invention, the electronic device 110 is substantially different through the network 170 using a wireless or wired communication method. It may mean one of various physical computer systems capable of communicating with the electronic devices 120, 130, 140 and/or the servers 150 and 160.

The communication method is not limited, and not only a communication method using a communication network (for example, a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network 170 may include, but also short-range wireless communication between devices is included. I can. For example, the network 170 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). And any one or more of networks such as the Internet. In addition, the network 170 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

Each of the servers 150 and 160 is a computer device or a plurality of computers that communicates with a plurality of electronic devices 110, 120, 130, and 140 through a network 170 to provide commands, codes, files, contents, services, etc. It can be implemented with devices. For example, the server 150 may be a system that provides a first service to a plurality of electronic devices 110, 120, 130, and 140 accessed through the network 170, and the server 160 is also a network ( It may be a system that provides a second service to a plurality of electronic devices 110, 120, 130, and 140 accessed through 170). As a more specific example, the server 150 provides a service (for example, a search service, etc.) targeted by the application through an application as a computer program installed and driven in a plurality of electronic devices 110, 120, 130, and 140. As a first service, it may be provided to a plurality of electronic devices 110, 120, 130, and 140. As another example, the server 160 may provide a service for distributing the above-described application installation and running file to the plurality of electronic devices 110, 120, 130, and 140 as the second service.

2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention. In FIG. 2, internal configurations of the electronic device 110 and the server 150 are described as examples of the electronic device. In addition, other electronic devices 120, 130, 140 or server 160 may also have the same or similar internal configuration as the electronic device 110 or server 150 described above.

The electronic device 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input/output interfaces 214 and 224. The memories 211 and 221 are non-transitory computer-readable recording media, such as random access memory (RAM), read only memory (ROM), disk drive, solid state drive (SSD), flash memory, and the like. It may include a permanent mass storage device. Here, a non-destructive mass storage device such as a ROM, SSD, flash memory, disk drive, etc. may be included in the electronic device 110 or the server 150 as a separate permanent storage device separated from the memories 211 and 221. In addition, the memories 211 and 221 include an operating system and at least one program code (for example, a browser installed and driven in the electronic device 110 or a code for an application installed in the electronic device 110 to provide a specific service). Can be saved. These software components may be loaded from a computer-readable recording medium separate from the memories 211 and 221. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memories 211 and 221 through communication modules 213 and 223 other than a computer-readable recording medium. For example, at least one program is a computer program installed by files provided by a file distribution system (for example, the server 160 described above) that distributes the installation files of developers or applications through the network 170 It may be loaded into the memories 211 and 221 based on (for example, the above-described application).

The processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to the processors 212 and 222 by the memories 211 and 221 or the communication modules 213 and 223. For example, the processors 212 and 222 may be configured to execute a command received according to a program code stored in a recording device such as the memories 211 and 221.

The communication modules 213 and 223 may provide a function for the electronic device 110 and the server 150 to communicate with each other through the network 170, and the electronic device 110 and/or the server 150 A function for communicating with an electronic device (for example, the electronic device 120) or another server (for example, the server 160) may be provided. As an example, a request generated by the processor 212 of the electronic device 110 according to a program code stored in a recording device such as a memory 211 is sent to the server 150 through the network 170 under the control of the communication module 213. ) Can be delivered. Conversely, control signals, commands, contents, files, etc. provided under the control of the processor 222 of the server 150 are transmitted through the communication module 223 and the network 170 to the communication module 213 of the electronic device 110. ) Through the electronic device 110. For example, a control signal, command, content, file, etc. of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and the content or file may be transmitted to an electronic device ( 110) may be stored as a storage medium (permanent storage device described above) that may further include.

The input/output interface 214 may be a means for an interface with the input/output device 215. For example, the input device may include a device such as a keyboard, a mouse, a microphone, and a camera, and the output device may include a device such as a display, a speaker, and a haptic feedback device. As another example, the input/output interface 214 may be a means for interfacing with a device in which input and output functions are integrated into one, such as a touch screen. The input/output device 215 may be configured with the electronic device 110 and one device. In addition, the input/output interface 224 of the server 150 may be a means for interface with a device (not shown) for input or output that is connected to the server 150 or that the server 150 may include. As a more specific example, when the processor 212 of the electronic device 110 processes a command of a computer program loaded in the memory 211, a service configured by using data provided by the server 150 or the electronic device 120 A screen or content may be displayed on the display through the input/output interface 214.

In addition, in other embodiments, the electronic device 110 and the server 150 may include more components than those of FIG. 2. However, there is no need to clearly show most of the prior art components. For example, the electronic device 110 may be implemented to include at least some of the input/output devices 215 described above, or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, and a database. May include more. As a more specific example, when the electronic device 110 is a smartphone, an acceleration sensor or a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input/output port, and a vibrator for vibration generally included in the smartphone Various components such as may be implemented to be further included in the electronic device 110.

Hereinafter, a specific embodiment of a method and system for detecting a multimodal abusing pattern for document selection will be described.

In this specification, a document may mean an information unit that is a search target on the Internet.

Documents produced for advertisements or malicious purposes have the characteristics of being intensively written at similar times with similar contents, mainly by similar groups of authors, without originality due to repetitive patterns of similar contents.

In the present embodiment, the burstiness pattern is for measuring the quality of a document, and refers to an abuzz pattern that appears as a repeating pattern as documents are written with similar content by a similar group of authors at a similar time. The explosive pattern of a document may mean a multimodal feature between documents produced within a specific period rather than a single document.

The present embodiments seek to find documents having a repetitive pattern without originality, and through this, to select original documents that search service users prefer.

3 is a block diagram showing an example of components that may be included in a processor of a server according to an embodiment of the present invention, and FIG. 4 is an example of a method that can be performed by a server according to an embodiment of the present invention. Is a flow chart showing.

The server 150 according to the present embodiment serves as a platform for providing a search service. In particular, the server 150 may detect an explosive pattern, which is a repeating pattern, targeting a document to be searched, and select a document with originality through this pattern.

The processor 222 of the server 150 is a component for performing the method of detecting an abusing pattern according to FIG. 4, and as shown in FIG. 3, a pattern detection unit 310, a network generation unit 320, and a document are selected. It may include a unit 330. Depending on the embodiment, components of the processor 222 may be selectively included or excluded from the processor 222. In addition, according to an embodiment, components of the processor 222 may be separated or merged to express the function of the processor 222.

Components of the processor 222 and the processor 222 may control the server 150 to perform the steps (S410 to S430) included in the method of detecting an abusing pattern of FIG. 4. For example, the processor 222 and the components of the processor 222 may be implemented to execute an instruction according to the code of the operating system included in the memory 221 and the code of at least one program.

Here, the components of the processor 222 may be expressions of different functions performed by the processor 222 according to an instruction provided by the program code stored in the server 150. For example, the pattern detection unit 310 may be used as a functional expression of the processor 222 that controls the server 150 according to the above-described command so that the server 150 detects an explosive pattern representing a repetition pattern of a document. have.

The processor 222 may read necessary commands from the memory 221 loaded with commands related to control of the server 150. In this case, the read command may include a command for controlling the processor 222 to execute steps S410 to S430 to be described later. The steps S410 to S430 to be described later may be performed in an order different from the order shown in FIG. 4, and some of the steps S410 to S430 may be omitted or an additional process may be further included.

In step S410, the pattern detection unit 310 may detect an explosive pattern representing a repetition pattern of a document targeting a document generated in a predetermined period. Since the documents produced by the abuzzer have a certain commonality and tend to appear intensively in a specific period, it is possible to select an abusing document by finding an explosive pattern corresponding to this repetition pattern. As an example, the pattern detection unit 310 includes various features representing a repeating pattern using words, images, links, the title or body of the document, whether or not it has a price (hereinafter, referred to as'detail features'). ) Can detect explosive patterns. At this time, the pattern detection unit 310 periodically (eg, 4 hours) measures detailed features for a set of documents produced or modified in a corresponding period, and then sequentially accumulates detailed features for each period for a specific period (eg, 8 hours). Explosive patterns can be found from detailed features accumulated over time, 1 day, etc.). The pattern detection unit 310 may detect an explosive pattern through an increase or decrease in a specific pattern occurring in the time window while adjusting the time window.

In step S420, the network generator 330 may generate network information between authors who have created the corresponding document targeting the document in which the explosive pattern is detected. The network generator 330 may generate a network between the document and the author and between the author and the author based on the explosive pattern of the document. The network generator 330 may generate network information by connecting links between authors who have created a document having a similar repetition pattern, and connecting a link between a document having a similar repetition pattern and an author who created the document.

In step S430, the document sorting unit 330 may select a document based on the explosive pattern detected in step S410 and the network information generated in step S420. The document sorting unit 330 may filter out documents including a repetitive pattern and authors who write a document with a repetition pattern by using the explosive pattern of the document and network information between authors, and select normal documents having a unique pattern.

The process of detecting the explosive pattern of a document will be described in detail as follows.

5, in this embodiment, an explosive pattern is a collective feature that can appear between documents produced within a specific period (time window), and includes a document explosive pattern 501, a personal explosive pattern 502, and a group explosive pattern ( 503). The document explosive pattern 501 refers to a pattern in which documents including similar words, similar titles, and similar texts are concentrated for a certain period, and the personal explosive pattern 502 refers to documents classified as similar documents by authors for a certain period. The group explosive pattern 503 may refer to a pattern that is intensively created, and the group explosive pattern 503 may refer to a pattern in which similar documents are intensively created in a similar period by creators of a specific group.

As described above, the explosive pattern of the document may be made as a combination of detailed features related to at least one of the document explosive pattern 501, the personal explosive pattern 502, and the group explosive pattern 503.

Detailed features can be calculated in units of a document set of a certain period, and most of them can be normalized using rank statistics in order to easily scale between features.

Examples of detailed features used in calculating the explosive score are as follows.

(1) word burstiness score

The word explosiveness corresponds to a feature that measures how many more words used than usual appear in a document based on the time window. At this time, the word is a space separation criterion, and only words containing nouns are counted. In addition, whether or not it is used more than usual can be measured by the ratio of TF-IDF within the time window of each word compared to TF-IDF (Term Frequency-Inverse Document Frequency).

For example, the word explosiveness (WBScore) can be scored through [Equation 1].

[Equation 1]

WBScore = sum{ log(TF + 1) × IDF / (NORM_TF × IDF)} (if DF >= 5 else 0)

(2) link burstiness score

Link explosiveness is a feature that measures how often links contained in a document appear within other documents in the time window. In other words, link explosiveness is a measure of how many different authors wrote links in a document.

For example, link explosiveness (LBScore) can be scored through [Equation 2].

[Equation 2]

LBScore = 1 (if WF == 1)

LBScore = (DF + WF) × 0.8 (if WF == 2)

LBScore = DF + WF (if WF> 2)

Here, WF means Writer Frequency.

(3) title burstiness score

Title explosive is a feature that measures how often a document's title appears in other documents in the time window. To measure the same number of titles, the titles of the document can be vector-quantized. At this time, the word vector is a 128-dimensional vector, and a title vector is created by adding all the word vectors appearing in the title. When quantizing the title vector, it can be converted to a numeric ID by changing the positive (+) value to 1 and the negative (-) value to 0 for each of the 128 dimensions. After measuring DF (Document Frequency) and WF (Writer Frequency) based on the numeric ID, it can be scored in the same way as link explosiveness.

For example, the title explosiveness (TBScore) can be scored through [Equation 3].

[Equation 3]

TBScore = 1 (if WF == 1)

TBScore = (DF + WF) × 0.8 (if WF == 2)

TBScore = DF + WF (if WF> 2)

(4) body burstiness score

The text explosive score can be calculated by creating a numeric ID in a similar way to the title explosive. For example, it is possible to create multiple numeric IDs by cutting the body of a document by a certain number of words (eg, 100 words).

For example, the text explosiveness (BBScore) can be scored through [Equation 4].

[Equation 4]

BBScore = 1 (if WF == 1)

BBScore = (DF + WF) × 0.8 (if WF == 2)

BBScore = DF + WF (if WF> 2)

(5) High cost score

The cost score is a measure of whether a document was written at a price, and refers to a feature that is classified as 1 if the author directly specified it in the document, and 0 otherwise. For example, it is possible to classify whether or not the end of the document (eg, 100 bytes) is learned through probabilistic learning (for example, Naive Bayes). Rather than being used alone, the cost score can be used to create a learning set of other features.

(6) Pay word burstiness score

The weight of words with high cost is a feature that calculates the word explosiveness in documents classified as high cost, selects the top n words for each topic, and measures how much weight the selected word exists in the document.

For example, the weight of the cost words (PWBScore) can be scored through [Equation 5].

[Equation 5]

PWBScore = sum{ log(TF + 1) × IDF / (NORM_TF × IDF)}

(7) Expression burstiness score

The explosive high-level document presentation weight score can be calculated in a similar way to the high cost word weight score. However, the summation units use'two-word expression' that includes verbs or adjectives, not nouns. The proportion of high-level explosive document expressions is a feature for detecting expressions that frequently appear in high-level explosive documents. In this case, the high-level explosive documents refer to documents having a summed explosive score equal to or higher than a predetermined level. The summed explosiveness score will be described again below.

For example, the high explosive document expression weight (EBScore) can be scored through [Equation 6].

[Equation 6]

EBScore = sum{ log(TF + 1) × IDF / (NORM_TF × IDF)}

(8) expression rate burstiness score

The explosive high-level document expression ratio score is a value obtained by dividing the explosive high-level document expression ratio score by the number of words in the entire body, and may be defined as [Equation 7].

[Equation 7]

ERBScore = EBScore / (word count)

(9) Explosive score, total score for each author (by author)

The summed score for each creator of the explosive score is a value obtained by summing the summed explosive score for each creator and converting it into a ranking statistical score.

(10) image burstiness score

Image explosiveness corresponds to a feature that measures how repeatedly similar images are used in multiple documents based on a time window. Image similarity is expressed by one vector ID, and image explosiveness can be measured in a similar manner to text-based explosiveness (word explosiveness, title explosiveness, text explosiveness) using the vector ID of the image.

The processor 222 may convert text or an image included in the document into a numeric vector according to detailed features, and then convert the converted text or image into a numeric vector to be ID.

In the case of text, as shown in FIG. 6, a numeric vector for the text may be generated using a deep learning technology such as word2vce. Text is a numeric ID (e.g., simhash ID) for estimating the similarity between two sets by using a vector for 1 word, a vector for 2 words, a vector for titles, and a vector for content such as links. I can express it.

Referring to FIG. 7, in the case of an image 600, ID can be achieved by clustering based on similarity between images using an image search deep feature. For example, after clustering an image deep learning vector (dimension 4096) with a k nearest neighbors (kNN) algorithm, an image can be IDed by listing cluster IDs in the nearest order. For example, if the distance of the cluster is close in the order of 1, 3, 5, and 29, the image ID can be expressed as [1.3.5.29]. At this time, the image deep learning vector may be clustered using the image signature obtained by the SHA-256 hash algorithm.

The detailed features described above can be changed to a ranking statistical score to fit the scale between features. Ranking statistics scores are listed in order of score size, and then normalized to 0.99 for the top 1%, 0.5 for the top 50%, and 0.01 for the bottom 1%. In this case, if the score size is the same, the ranking statistics score is made the same.

Additionally, a summary metric can be applied to simultaneously measure how large the feature with the highest value is while all the detail features have high values. For example, the summary index can be defined as [Equation 8].

[Equation 8]

Summary indicator = 0.7 × score of 1st feature + 0.3 × (average score of other features)

The above summary indicator can be used for the purpose of detecting when the feature with the highest score very clearly exhibits explosiveness even if the score of other detailed features is low.

The processor 222 summing the scores of the detailed features, selects a document set having a higher summation score of at least a predetermined level, and then re-learns detailed features based on the selected document set, adjusts the combination weight between the detailed features, or performs a new detailed feature. Can be further measured. The processor 222 may repeat the above-described process and adjust the detailed features using the correct answer data set evaluated by the tester so that the detailed features are not learned in the wrong direction.

8 shows an example of a result of learning an advertisement document as a decision tree as an example of an abusing document.

The processor 222 may firstly select a document having a high probability of being an advertisement by applying a predetermined filtering condition during the learning process. For example, referring to FIG. 8, the processor 222 classifies a document having an explosive high-level document expression weight score (expression density) exceeding 0.888 as an advertisement document, and among documents having an explosive high-level document expression weight score of 0.888 or less, the text explosiveness Documents with a score exceeding 0.472 can be classified as advertising documents, documents with a link explosive score exceeding 0.487 among documents with an explosive score of 0.472 or less are classified as advertising documents, and other documents can be classified as original and normal documents. have. As another example, a document having a word explosive score exceeding 0.824 and a document having a high explosive document expression ratio score exceeding 0.426 may be additionally selected as advertisement documents, and then the remaining documents may be classified as normal documents.

In FIG. 8, the score used as the document classification criterion corresponds to an exemplary numerical value, but is not limited thereto.

The processor 222 may create a final explosive score with a linear model for a document classified as an advertisement document through the above filtering conditions. For example, the final explosive score is a sum explosive score obtained by summing all scores for each detailed feature. It can be defined as Equation 9].

[Equation 9]

dq_score = 0.5639 × total + 0.012 × w + 0.189 × l + 0.210 × t + 0.245 × b + 0.024 × pt + 0.005 × p + 0.127 × max(er, e)

dq_score =-min(dq_score, 0.99)

(Where, total is a summary index, w is a word explosive score, l is a link explosive score, t is a title explosive score, b is a text explosive score, pt is a false price, p is a false word weight score, and er is an explosive top document expression. Percentage score, r is the explosive top document expression weight score)

The weights included in Equation 9 correspond to exemplary values, and are not limited thereto.

The image explosive score is not included in the summed explosive score, but this is only an example, and the method of adding the explosive score may be differently applied depending on the importance of the detailed features. For example, the sum explosive score may be calculated in a manner in which the highest weight is applied to the word explosive score and the image explosive score.

The processor 222 may finally use only the explosive score in a predetermined range (eg, -0.5 to -1.0), and post-process the explosive score to increase the accuracy of the score and expand the application range.

As an example of post-processing, the processor 222 may include a model for correcting the explosive score when a document having an explosive score is written several times by the same author in a short time (eg, a wallpaper advertisement, etc.). If several documents with similar patterns are written by several people within a short period of time, it is highly likely to be an abusing pattern. For example, to calibrate the explosive score, three time windows can be used: 1 hour, 10 minutes, and 1 minute. When measuring the explosive score, the basic time window of 8 to 1 day is first applied, and then the time window is reduced to 1 hour by post-processing, so that the sharpness of the explosive pattern can be additionally measured.

The explosive score is based on a method of finding an abusing pattern through the increase or decrease of a specific pattern occurring within the period while moving the time window due to the characteristics of the explosive pattern. This way, in order to find all documents with a repeating pattern in the entire document collection, the entire period must be iterated over. As another example of post-processing, the processor 222 may shorten the calculation time for the explosive score by using network information between document authors. In detail, the processor 222 may vectorize all documents and assign a pattern ID for each detailed feature through quantization to have similarity between documents. The pattern ID-author ID-pattern ID network can be created by linking the link between the remaining pattern ID and the author who wrote the document after leaving only the pattern ID with a score above a certain level in the pattern ID of each document. The calculation time for explosive scores can be shortened by finding clusters with a certain distribution or more in the pattern ID-author ID-pattern ID network and propagating the detailed feature scores of the pattern ID.

Referring to FIG. 9, the processor 222 connects the link between the document and the creator who created the document to create the creator-document-author network 900, so that the same pattern ID may be assigned to the creator who created the document with a repetition pattern. have. In other words, the fact that clusters are clustered in a star-like shape in the author-document-author network 900 corresponds to an abusing condition as that several people wrote the document of the pattern ID at the same time. When the processor 222 averages and applies the explosive scores of documents belonging to the cluster corresponding to the abusing condition, the processor 222 can estimate the explosive score for all documents in the entire collection, and the author (i.e., the abuser) who created the document with a repetitive pattern. You can find additional.

The process of detecting explosive patterns in documents is summarized as follows.

Referring to FIG. 10, the processor 222 expresses the text included in the document 1010 as a word vector 1011 for the recently created document 1010 and the image included in the document 1010 is an image vector 1012. ) To represent both text and images included in the document 1010 with numeric IDs. In this case, the same pattern ID is assigned to the similar text and the similar image based on the similarity. In addition, the processor 222 may measure and score the explosive pattern of the document 1010 through the pattern ID assigned to the document 1010, and may select the abusing document 1001 of the repetitive pattern based on this.

As illustrated in FIG. 11, when the same pattern ID of the document is accumulated over time, the processor 222 may apply a time window to calculate an explosive score within the corresponding period. At this time, the explosiveness score is a score obtained by calculating the quantity of the number of patterns in the time window. First, the processor 222 applies a basic time window (e.g., 7 days), and if the explosive score is higher than a certain level, the time window is gradually reduced (e.g., 1 day, 1 hour, 10 minutes, 1 minute) and the explosive score Can be calculated.

Referring to FIG. 12, the processor 222 may assign a pattern ID for each detailed feature. For example, a document set 1201 to which a pattern ID of a similar word is assigned, a document set to which a pattern ID of a similar sentence is assigned ( 1202), documents belonging to two or more sets at the same time in the document set 1203 to which the pattern ID of the same author is assigned, and the document set 1204 to which the pattern ID of similar images is assigned, that is, explosive patterns are commonly found in several detailed features. The document set 1210 that appears may be selected as a candidate document suspected of being an abusing document.

The processor 222 may relearn detailed features based on the document set 1210 selected as a candidate document, adjust a combination weight between detailed features, or add a new detailed feature. Referring to FIG. 13, the processor 222 may classify a document set 1210 selected as a candidate document into an abusing document with a repetition pattern and a normal document with originality while repeating the reinforcement type machine learning. As an example, the processor 222 samples a high-ranking document with a high explosive score from the candidate document set 1210 or a document that occupies a high ranking in the search result, and determines an abusing pattern according to the tester's abusing tagging or pattern selection for the sampled document You can relearn. For example, the processor 222 may exclude a document containing pre-tagged or labeled images from the candidate document set 1210, or retrain a document that is incorrectly classified as an abusing document through weight adjustment between detailed features. You may.

The processor 222 may apply a document selection result based on a multimodal explosive pattern between documents to the document ranking. For example, a document finally selected as an abusing document of a repetitive pattern occupies a higher ranking than a normal document in the search result. Penalties can be applied not to do so.

As described above, according to embodiments of the present invention, a document having an abusing pattern and a document with originality may be selected based on the detected pattern by detecting a multimodal abusing pattern for a document produced for a certain period. Further, according to embodiments of the present invention, it is possible to improve document selection performance and document retrieval quality by selecting intentionally produced abusing documents by using network information between document authors together with a collective characteristic of a document pattern.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), and a programmable gate array (PLU). It may be implemented using one or more general purpose or special purpose computers, such as a logic unit), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be embodyed in any type of machine, component, physical device, computer storage medium or device to be interpreted by the processing device or to provide instructions or data to the processing device. have. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. In this case, the medium may be one that continuously stores a program executable by a computer, or temporarily stores a program for execution or download. Further, the medium may be a variety of recording means or storage means in a form in which a single or several pieces of hardware are combined, but is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And there may be ones configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media include an app store that distributes applications, a site that supplies or distributes various software, and a recording medium or a storage medium managed by a server.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description to those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and those equivalent to the claims also fall within the scope of the claims to be described later.

Claims (15)

In the method for detecting an abusing pattern executed in a computer system,
The computer system includes at least one processor configured to execute computer readable instructions contained in a memory,
The abusing pattern detection method,
Detecting, by the at least one processor, an explosive pattern representing a repetition pattern of the document based on multimodal features between documents; And
Selecting an original document using the explosive pattern by the at least one processor
Abusing pattern detection method comprising a. The method of claim 1,
The detecting step,
Detecting the explosive pattern appearing in the document set for a document set in a time window
Abusing pattern detection method comprising a. The method of claim 1,
The detecting step,
Detecting the explosive pattern appearing in the document set by using at least one of a word, link, title, body, and image included in each document for a document set in a time window
Abusing pattern detection method comprising a. The method of claim 3,
The detecting step,
Detecting the explosive pattern by using the weight of a word or expression selected on the basis of the frequency of occurrence of words in the document set by using the weight in each document
Abusing pattern detection method comprising a. The method of claim 1,
The detecting step,
Measuring a plurality of features representing the repeating pattern by using content included in the document; And
Calculating an explosive score by summing the plurality of features
Abusing pattern detection method comprising a. The method of claim 5,
The detecting step,
Selecting a document set using the plurality of features; And
Learning the plurality of features based on the document set or adjusting weights for each feature
Abusing pattern detection method further comprising a. The method of claim 1,
The detecting step,
Converting content included in the document into a numeric vector and then assigning a pattern ID corresponding to the numeric vector; And
Calculating an explosive score by applying a time window to the pattern ID over time to score the number of pattern IDs in the time window
Abusing pattern detection method comprising a. The method of claim 7,
The detecting step,
Measuring sharpness of the explosive pattern by adjusting the time window when the explosive score is higher than a certain level
Abusing pattern detection method further comprising a. The method of claim 1,
The abusing pattern detection method,
Generating, by the at least one processor, network information between the document in which the explosive pattern is detected and the author who wrote the document
Abusing pattern detection method further comprising a. The method of claim 9,
The detecting step,
Converting the content included in the document into a numeric vector and then assigning a pattern ID corresponding to the numeric vector
Including,
The generating step,
Generating the network information by connecting the link between the pattern ID and the author who created the document
Abusing pattern detection method comprising a. The method of claim 9,
The step of selecting,
Selecting at least one of an abusing document and an abuser using the network information together with the explosive pattern
Abusing pattern detection method comprising a. A computer program stored in a non-transitory computer-readable recording medium for executing the method of detecting an abusing pattern according to any one of claims 1 to 11 on the computer system. A non-transitory computer-readable recording medium in which a program for executing the method of detecting an abusing pattern of claim 1 on a computer is recorded. In a computer system,
At least one processor configured to execute computer readable instructions contained in memory
Including,
The at least one processor,
A pattern detector configured to detect an explosive pattern representing a repeating pattern of the document based on multimodal features between documents; And
Document sorting unit that selects original documents using the explosive pattern
Computer system comprising a. In a computer system,
At least one processor configured to execute computer readable instructions contained in memory
Including,
The at least one processor,
A pattern detector configured to detect an explosive pattern representing a repeating pattern of the document based on multimodal features between documents;
A network generator for generating network information between the document in which the explosive pattern is detected and the author who created the document; And
Document sorting unit for selecting original documents using the explosive pattern and the network information
Computer system comprising a.

Images